MiniCPM-V 4.7: Running a 35B Multimodal Model in Just 3 GB of RAMBrief · digest for October 18
Brief2 sections≈ 2 min

MiniCPM-V 4.7: Running a 35B Multimodal Model in Just 3 GB of RAM

If you follow local AI, you are used to a blunt rule of thumb: if you want to run a serious computer-vision model, bring a beefy GPU with 16 to 24 GB of fast VRAM—or learn to love glacial speeds. We have already seen clever attempts to tame large models, from quantizing Saluki 27B down to 8 GB for offline tools to hybrid Attention and Mamba-2 in Nvidia Nemotron 3 Nano.

What OpenBMB, Tsinghua University's THUNLP lab, and ModelBest just pulled off with MiniCPM-V 4.7, though, looks like a genuine engineering magic trick: they got a 35-billion-parameter multimodal model running straight on smartphones while sipping only about 3 GB of RAM.

Architectural Magic: Look-Ahead Expert Scheduling (LES)

How do you physically cram 35 billion weights into three gigabytes of memory?

The trick comes down to smart division of labor and Look-Ahead Expert Scheduling (LES):

  1. Sparse 35B-A3B architecture: Out of the 35 billion total Mixture-of-Experts parameters, only 3 billion activate for any given token.
  2. Preemptive flash prefetching: The phone doesn't need to hold every weight in RAM at once. The full model sits in flash storage. While the current neural layer computes, LES reads the activations, predicts exactly which experts the next layer will need, and preloads them into RAM ahead of time. That runs 5x faster than standard OS page swapping with zero accuracy loss.
  3. DSpark speculative sampling: On chips like the Apple A18 Pro (iPhone 17 Pro Max), generation tops 13 tokens per second, keeping interactions smooth and responsive.

The model handles photos, multi-page PDFs (OCR), and even video streams without stumbling. To compress visual inputs efficiently, it uses adaptive tokenization—16x compression for broad scenes and 4x compression for fine UI elements or small text.

RAM Footprint vs. Storage Space

Don't confuse active memory with permanent storage: 3 GB is the dynamic RAM footprint while running. The full quantized weight file on your phone's storage still takes 10 to 20 GB depending on the format (GGUF, AWQ, or INT4).

Why This Matters for the Industry

The release ships under a permissive Apache 2.0 license, with both the flagship 35B-A3B MoE model and an ultra-light 1B variant. Out of the box, it supports Apple, Qualcomm Snapdragon, MediaTek, Intel, AMD, and ARM silicon, along with popular local runners like Ollama and llama.cpp.

Being able to parse personal documents, receipts, and video clips right on your phone—without shipping sensitive data off to someone else's servers—is exactly what on-device AI was built for.

Want a deep diveVotes help us choose what to break down in detail. This material has 0 votes.
Sources · 3