NVIDIA just released a 120B model that activates only 12B parameters during inference—while training the entire thing in 4-bit precision. That's not generosity. That's infrastructure positioning.
The model, Nemotron-3-Super-120B-A12B-NVFP4, ships with a million-token context window, multilingual support, a LatentMoE architecture, and the full training dataset. Open weights, the tech report, pre-training data—all released. But the proprietary piece, the NVFP4 training recipe that makes this architecture actually work at scale, stays locked to NVIDIA's ecosystem.
Here's why this matters technically: NVFP4 quantization during pre-training is genuinely novel for open models. Standard quantization happens post-training—compressing a trained model into lower precision. NVIDIA went further, training in 4-bit from the start. This required custom CUDA kernels, intimate knowledge of Hopper GPU tensor core behavior, and training infrastructure most organizations simply don't have. The result is a 120B model that behaves like it was trained on significantly more compute than it actually was.
The LatentMoE architecture enables the 120B total / 12B active split. Rather than activating all 120B parameters for every token, the model routes through a compressed latent space, activating only 12B at inference time. This is the sparseMixture-of-Experts trick that DeepSeek and others have popularized—but NVFP4 pre-training is what makes it train efficiently at this scale.
For developers, the practical reality cuts both ways. Yes, you can download 120B weights and run them on your own hardware. But NVFP4 training creates a performance gap: models trained this way achieve benchmarks that feel out of proportion with the raw compute involved. Replicating that training process requires NVIDIA's stack. Running the model efficiently requires understanding how the quantization interacts with your inference hardware.
The strategy becomes clear when you zoom out. As the open-weight ecosystem expands—Meta's Llama series, DeepSeek's R1, Mistral's releases—developers face a growing choice: which inference infrastructure powers their applications? NVIDIA's answer is to make their hardware the obvious answer. Release the weights. Publish the data. Standardize the tooling on CUDA. When every efficient open model traces back to NVIDIA-optimized training, the inference layer becomes non-negotiable.
This doesn't make the release illegitimate. Nemotron-3-120B will be genuinely useful—1M context, strong multilingual performance, MoE efficiency. Practitioners should absolutely use it. But the release signals that NVIDIA sees open weights not as competition to their cloud API business, but as a rising tide that lifts their infrastructure. The company that trains the best open models on the most efficient stack wins regardless of whether those weights are open or closed.
For the broader open-source ecosystem, the lesson is uncomfortable: "open weights" and "open training" are different things. NVIDIA released one and kept the other. As quantization-aware training becomes standard, expect more of these asymmetric releases—impressive models with proprietary recipes that only run optimally on one vendor's hardware.