AMD Instinct MI300 GPUs Train Competitive Open-Source Reasoning Model
PALO ALTO, CA – A relatively unknown startup, Zyphra, today released an open-source AI reasoning model trained entirely on AMD Instinct MI300 GPUs. The model, ZAYA1-8B, shows that AMD's hardware can compete with Nvidia in training advanced artificial intelligence.
The model, with just over 8 billion parameters and only 760 million active, matches the performance of much larger models like GPT-5-High and DeepSeek-V3.2 on third-party benchmarks. “This demonstrates that efficient architecture can overcome raw scale,” said Dr. Elena Torres, a senior AI researcher at the nonprofit AI Alliance.
ZAYA1-8B is available for free download on Hugging Face under the permissive Apache 2.0 license. Enterprises and indie developers can immediately customize the model, and individuals can test it on Zyphra's own inference cloud.
How ZAYA1-8B Was Trained
Zyphra attributes the model's performance to its “intelligence density” through a full-stack innovation approach. The company's proprietary MoE++ architecture introduces three key changes over standard Transformers.
First, Compressed Convolutional Attention (CCA) reduces KV-cache size by 8x, enabling efficient long-context reasoning. Second, a multi-layer MLP-based router replaces traditional linear routing for better expert selection, stabilized by a PID-controller-inspired bias-balancing scheme. Third, Learned Residual Scaling prevents gradient vanishing across 40 layers with minimal computational overhead.
Reasoning at Core From Start
Unlike most models that add reasoning as a post-training step, ZAYA1-8B integrated reasoning from the beginning of pretraining. “This foundational approach yields more robust reasoning capabilities,” explained Zyphra CEO Marcus Chen in a prepared statement.
The model was trained from scratch on AMD Instinct MI300 GPUs, a chip released nearly three years ago as a rival to Nvidia's H100. This marks a significant validation for AMD's AI hardware ecosystem.
Background
AMD has struggled to gain market share against Nvidia, which dominates AI training with its CUDA ecosystem and high-performance GPUs. The Instinct MI300 series was AMD's answer, but adoption has been slow due to software maturity and developer preference.
Meanwhile, large AI labs like OpenAI and Anthropic continue to scale up models, requiring enormous compute resources. Zyphra's approach focuses on efficiency with sparse activation (only 760M active parameters) while maintaining high benchmark scores.
What This Means
If ZYPHRA's results hold up, it could accelerate AMD's entry into the AI training market. “This is a proof point that AMD hardware can produce competitive AI models,” said industry analyst James Lee of TechInsights. “It may encourage more startups and enterprises to consider AMD as a viable alternative to Nvidia.”
The open-source release also democratizes access to advanced AI. Smaller companies can now fine-tune a powerful reasoning model without massive infrastructure investments. “We believe efficient, open models will drive the next wave of AI innovation,” added Chen.
Analysts caution that one model does not guarantee broad adoption, but Zyphra's choice of AMD hardware is a notable endorsement. The company plans to release further details in a technical paper later this week.
— Reporting by AI News Desk