# NVIDIA Releases Nemotron 3.5 Lightning, a Fast Open Model for Agents

NVIDIA released [Nemotron 3.5 Lightning](https://blogs.nvidia.com/blog/nemotron-lightning-switchyard-rtx-dgx/), a fast, customizable open model built to power agents on a single GPU. It is a hybrid Mixture-of-Experts (MoE) model with 30B total parameters and just 3B active per forward pass, light enough to run on a single GPU in a laptop or desktop.

## What's new in Nemotron 3.5 Lightning

- **Hybrid MoE architecture** — 30B total parameters, 3B active per forward pass, keeping compute low while retaining a large parameter pool to draw from
- **Up to 4x the throughput** of comparable models, around **~410 tokens/sec**, and **30% faster task completion**
- **Distilled from Nemotron 3 Ultra**, NVIDIA's larger flagship model, to retain much of its capability in a far smaller footprint
- **Up to 1M tokens of context**, handled through **DFlash speculative decoding**
- **Direct integration with popular agent harnesses**, so it drops into existing agent stacks rather than requiring a new one
- **Fully open-trained on open datasets**, letting enterprises post-train the model on their own tools, workflows, and policies, and deploy it with complete ownership across edge, on-premises, or cloud infrastructure

## Jensen Huang on the release

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Announcing Nemotron 3.5 Lightning — fast, open, and built for agents.</p>&mdash; Jensen Huang (@jensenhuang) <a href="https://x.com/jensenhuang/status/2087184542050496763?s=46">August 12, 2026</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

## How it compares

Nemotron 3.5 Lightning's primary competitors are lightweight, high-throughput models built for the same job — low-latency tool execution and sub-agent task routing — rather than raw benchmark leadership: Qwen 3.6 35B, Gemma 4 31B, and Gemini 3.5 Flash-Lite. [Thinking Machines Lab's Inkling](https://vedcraft.com/tech-news/thinking-machines-lab-launches-inkling-and-inkling-small/) is a direct architectural and deployment competitor too — another open-weight, MoE model built for the same high-throughput agent space.

| Model | Params | Quality focus | Throughput | Pricing | Available on |
|---|---|---|---|---|---|
| **NVIDIA Nemotron 3.5 Lightning** | 30B total / 3B active (MoE) | Agentic tool use, distilled from Nemotron 3 Ultra | ~410 tok/s, up to 4x comparable models | Open weights — free to self-host; usage-based on hosted endpoints | OpenRouter, Tinker, AWS SageMaker, Hugging Face |
| Qwen 3.6 35B | 35B dense | Strong general + tool-calling accuracy | High, tuned for low-latency execution | Open weights — free to self-host; usage-based on hosted endpoints | OpenRouter, Hugging Face |
| Gemma 4 31B | 31B dense | Balanced quality for sub-agent routing | High, single-GPU friendly | Open weights — free to self-host | OpenRouter, Hugging Face |
| Gemini 3.5 Flash-Lite | Undisclosed (hosted-only) | Fast, cost-optimized routing/orchestration | Very high, low latency | Per-token, hosted API pricing | Google AI Studio / Vertex AI |
| Thinking Machines Inkling-Small | 276B total / 12B active (MoE) | Reasoning + agentic coding, distilled from Inkling | Optimized for agentic workloads at a quarter of Inkling's size | Open weights — 50% launch discount on hosted usage | Tinker |

*Figures for Nemotron 3.5 Lightning are as reported in NVIDIA's announcement; other rows are approximate and drawn from each vendor's own release notes, included for relative context rather than a like-for-like benchmark.*

## Why it matters

Nemotron 3.5 Lightning is NVIDIA's pitch for the next phase of agentic AI: a model small enough to run locally on a single GPU, fast enough to feel responsive in an interactive agent loop, and open enough that enterprises can post-train it on their own data, tools, and policies without giving up ownership of the deployment. Distilling it from Nemotron 3 Ultra and pairing it with DFlash speculative decoding for long-context, million-token workloads makes it a notable option for architects evaluating open-weight models for edge and on-premises agent deployments — competing directly with Qwen 3.6 35B, Gemma 4 31B, and Gemini 3.5 Flash-Lite on low-latency tool execution and sub-agent task routing, and with Thinking Machines Lab's Inkling on architecture and deployment in the broader open-weight, high-throughput agent space.

Sources:

- [NVIDIA — Nemotron Lightning Switchyard for RTX & DGX](https://blogs.nvidia.com/blog/nemotron-lightning-switchyard-rtx-dgx/)
- [Jensen Huang on X](https://x.com/jensenhuang/status/2087184542050496763?s=46)

Disclaimer:

_All data and information provided on this blog are for informational purposes only. All the image sources used are for reference only. The author makes no representations as to the accuracy, completeness, correctness, suitability, or validity of any information on this blog and will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. This is a personal view and the opinions expressed here represent my own and not those of my employer or any other organization._