MiniCPM-V 4.7: A 35B Multimodal Model Running in 3 GB of Smartphone RAMBrief · digest for October 19
Brief≈ 2 min

MiniCPM-V 4.7: A 35B Multimodal Model Running in 3 GB of Smartphone RAM

Running a 35-billion-parameter multimodal model right on an everyday smartphone—and squeezing it into just 3 GB of RAM? It sounds like sci-fi, yet that's the exact engineering feat OpenBMB pulled off in MiniCPM-V 4.7.

The trick lies in its 35B-A3B Mixture of Experts (MoE) architecture: out of the full parameter pool, only 3 billion parameters activate per token. The team added Predictive Expert Paging—dormant expert weights stay in fast flash storage (UFS 4.0 or NVMe) and stream into RAM in microseconds only when needed. Paired with INT4 quantization, the model runs comfortably inside a consumer phone's memory budget.

Here is a basic Python inference example:

import torch
from PIL import Image
from transformers import AutoModel, AutoTokenizer

model_path = "openbmb/MiniCPM-V-4_7-int4"
model = AutoModel.from_pretrained(
    model_path,
    trust_remote_code=True,
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True
).eval()

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
image = Image.open("document_sample.jpg").convert("RGB")
msgs = [{"role": "user", "content": [image, "Распознай рукописную таблицу в Markdown."]}]

res = model.chat(image=None, msgs=msgs, tokenizer=tokenizer, sampling=True, temperature=0.3)
print("Результат:\n", res)

And here is the command to launch the optimized mobile profile via the MLC-LLM CLI runtime:

# Compile and run MiniCPM-V-4.7 with a 3GB RAM cap
mlc_llm run MiniCPM-V-4_7-int4-mobile \
  --device "metal" \
  --max-memory-mb 3072 \
  --enable-expert-paging true \
  --flash-storage-cache "/tmp/moe_cache"

As Julian Goldie SEO notes, on chips like Apple A18 and Snapdragon 8 Gen 3/4 the model hits 15–20 tokens per second, reliably parsing tables and handwritten notes without sending a single byte to the cloud.

Want a deep diveVotes help us choose what to break down in detail. This material has 0 votes.
Sources · 2