If you've ever tried building AI agents that navigate a computer screen or pull balance sheets out of scanned PDFs, you already know their Achilles' heel. First-generation Vision-Language Models (VLMs) are great at spotting a golden retriever on a lawn or a coffee mug on a desk, but hand them a dense 80-page financial filing packed with multi-level tables and handwritten margin notes and the wheels fall off: rows drift, columns scramble, and when the agent tries to click an on-screen button, it misses by a couple of inches.
The reason for that nearsightedness is painfully simple: most VLMs were bolted together by gluing a generic image encoder (like CLIP) onto a text LLM through a flimsy linear adapter. Fine print and thin grid lines got smeared into mush during the initial image compression.
NVIDIA just unveiled a radical architectural fix for this problem—the open model Nemotron 3.5 Super VL.
Fusing Mamba-2 and Transformers: Why Older VLMs Stumbled on Documents
Nemotron 3.5 Super VL is a genuine feat of hybrid engineering. The model totals 120 billion parameters, yet its Mixture-of-Experts routing dynamically activates just 12 billion weights per token.
The real magic sits inside its compute blocks:
- The internal layers fuse classic transformer attention with Mamba-2 (State Space Models, SSM). Mamba gives the network linear memory scaling across ultra-long sequences, letting it hold a 1,000,000-token context window without breaking a sweat.
- On the visual front, NVIDIA plugged in a specialized optical encoder called C-RADIO. Trained on a massive corpus of high-resolution scans, it cleanly segments hairline table borders, 6-point footnotes, and tiny GUI radio buttons.
DocVQA Records and Pixel-Accurate Screen Grounding
Independent benchmarks show this 12B-active specialist easily outmuscles far heavier monolithic models:
- On scanned document comprehension (DocVQA) and financial chart analysis (ChartQA), Nemotron 3.5 Super VL topped 91% accuracy.
- On GUI Screen Grounding—where the model gets an app screenshot and must return exact pixel coordinates for an instruction like "click Export to CSV in the bottom right corner"—the average localization error was under 3 pixels. That means reliable clicks without accidentally hitting neighboring controls.
4-Bit NVFP4 and NIM Microservices: Packing Vision into an Agentic Pipeline
For production workloads, the model is tuned for NVIDIA's new 4-bit floating-point format, NVFP4. On Blackwell hardware, NVFP4 is supported directly in silicon, delivering a 4x savings in memory bandwidth with zero drop in recognition accuracy.
You can spin up the model as a ready-to-roll NVIDIA NIM (Inference Microservice) with full multi-threaded parallelism:
# Launch the NVIDIA NIM container with Nemotron 3.5 Super VL
docker run -d --gpus all \
-e NGC_API_KEY=$NGC_API_KEY \
-p 8000:8000 \
nvcr.io/nim/nvidia/nemotron-3.5-super-vl:latest
Or, for quick local inference in Python using vLLM, a few lines are all it takes:
from vllm import LLM, SamplingParams
from PIL import Image
# Initialize Nemotron 3.5 Super VL via vLLM
vlm = LLM(
model="nvidia/nemotron-3.5-super-vl",
trust_remote_code=True,
tensor_parallel_size=2,
max_model_len=131072,
quantization="fp8"
)
# Load a UI screenshot and prepare the prompt
dashboard_image = Image.open("finance_dashboard.png")
prompt = "<image>\nИзвлеки все показатели таблицы EBITDA в формате JSON и проверь наличие предупреждающих бейджей."
outputs = vlm.generate({
"prompt": prompt,
"multi_modal_data": {"image": dashboard_image}
}, SamplingParams(temperature=0.1, max_tokens=2048))
for output in outputs:
print(output.text)
Plugged into agentic frameworks, Nemotron acts as a razor-sharp visual sensor—freeing text models from having to guess their way through mangled OCR dumps.

