A 2.8-point average is a leaderboard argument. It is not a reason to unplug the 70B.
Shattered.io’s September 27 recap of MiniCPM5-2B is the cleanest version of a noisy weekend. OpenBMB’s 2.52-billion-parameter model posted a 53.9 average across 34 benchmarks against 51.1 for the largest rival in that write-up. MarkTechPost’s machine learning desk, and the tracker LMMarketCap, logged a pile of other small releases in the same September 25–27 window. The recap’s frame is right: several labs bet against “bigger is always better” in the same 48 hours. The frame is also how people forget to ask what the number was measured on.
We already spent last week on open weights taking tokens while closed models kept the bill. This is the other half. Small models are cheap to run. They are still specific tools. A speech model is not a coding agent. A draft model is not a base model. An abstention paper is not a release.
The cluster is five products, not one trend
Shattered.io’s table is the piece worth stealing.
MiniCPM5-2B, OpenBMB, 2.52 billion parameters, 53.9 average on 34 benches versus 51.1, window September 26–27, described as general-purpose long-context reasoning. That is the headline model because “beats bigger rivals by 2.8 points” fits a title. 2.8 points on an average can be one task family moving a lot and another not moving. The article does not replace a model card. If you care, open the 34.
Ornith-1.5-9B, Ornith AI, 9 billion, 47.0 on Terminal-Bench 2.1 and 70.6 on SWE-Bench Verified, September 25–26. Hermes AI, in that recap, said it performed close to 35-billion-parameter models on the coding-agent benches they cared about. That is a different sport from MiniCPM’s 34-bench average. Terminal-Bench and SWE-Bench are “can it do work in a repo,” not “can it take a quiz.”
LFM2.5-VL-3B-DSpark, Liquid AI, 279.5 million parameters, a draft model for speculative decoding of LFM2.5-VL-3B. Not a chatbot. A second network whose job is to guess tokens so the bigger vision-language model can verify them faster.
Julia 1, Supersonic Labs, 144.3 million, built on mmBERT-small, September 25. Beat the reference on three of four pilots, trailed on Banking77. That is a classifier-scale decision model. Comparing it to MiniCPM is a category error.
Saaras V4, Sarvam AI, September 26, speech-to-text for all 22 official Indian languages plus English. Parameter count not disclosed in the table. If your job is Hindi audio, this is the only row that matters. If your job is SWE-Bench, it is trivia.
A “small-model week” is a calendar coincidence. Ship the row that matches the job.
Speculative decoding does not shrink the image
Liquid’s own September 24 blog on LFM2.5-VL-DSpark is more useful than the cluster table, because it says where the speedup is allowed to exist.
VLM inference has a fat prefill. The image goes through a vision encoder. Then the language backbone eats hundreds of visual tokens plus the text prompt. Prefill is largely compute-bound. Cost grows, they write, (sub)quadratically with prompt length. Speculative decoding speeds the decode phase. Vision encoding and prefill do not move. If those stages already dominate wall time, a large decode speedup becomes a modest end-to-end win. They name Amdahl’s law instead of pretending otherwise.
That paragraph should be taped to every “edge VLM” slide. A 279.5-million-parameter draft model is a decode trick. It is available on Hugging Face in Safetensors and GGUF. That is a real artifact. It is not MiniCPM. It will not make your screenshot-to-text pipeline feel like a 2.52B general model, and MiniCPM will not make your image encoder cheaper.
If you are choosing hardware this week, ask what fraction of your latency is decode. Chat with short prompts: decode matters, speculation can help. Photo plus a paragraph: you bought an encoder bill. Liquid is telling you the second case out loud. Believe them.
The rest of the small-model pile should get the same question. Julia 1’s four pilots include a miss on Banking77. That is a product telling you the domain shifted. Saaras V4 is a language-coverage claim. Ornith is an agent-bench claim. MiniCPM is an average. Averages hide the task you actually pay for.
Uncertainty is not the same feature as being right
Tech Xplore’s September 24 write-up of a sparse autoencoder paper landed in the same news window and does not belong in the parameter table.
Het Patel et al., “Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoders,” arXiv:2604.19974. In two language models, confidence and correctness were linked to partly distinct internal features. Expressed uncertainty did not reliably indicate error. Suppressing features linked to both uncertainty and error improved accuracy by up to 1.1 percent without retraining. Using three such features to flag risky answers raised accuracy on answered questions from 62 percent to 81 percent, with 53 percent of questions answered.
Read the last sentence twice. They did not make the model omniscient. They made it quieter. 81 percent on the questions it still answered, at 53 percent coverage. That is an abstention policy with a microscope on residual streams. It is also a reminder that a 53.9 benchmark average does not tell you when the 2.52B model should shut up.
If you are shipping MiniCPM, or anything in its weight class, the SAE paper is the adult in the room. Small models will be confidently wrong in a cheaper way. A 1.1 percent bump from suppressing features is a research result, not a product switch. The 62-to-81 move with a 47 percent skip rate is the product shape: fewer answers, better remaining answers. Support tools can live there. Autocomplete that must always speak cannot.
Do not cite Tech Xplore’s auto-generated gist as if it were the paper. The numbers above are the gist of the gist. If this is load-bearing for a paper you are writing, open arXiv:2604.19974. If this is load-bearing for a bot you are shipping, measure your own skip rate. Their 53 percent coverage is not your 53 percent.
What a 2.52B average is worth in a week of 70B bills
Last week’s token-economics piece still holds. Closed APIs keep the invoice. Open weights keep the GPUs you already own. MiniCPM5-2B is an argument for the GPU you already own, if the 34-bench mix looks like your mix.
It is not an argument for replacing a coding agent with a 2.52B general model because Ornith posted 70.6 on SWE-Bench Verified at 9B. Different checkpoint. Different bench. Different failure mode. People will mash them together in Slack because they shipped in the same 48 hours. That mash is how you get a speech model in a PR review.
Mercury 2.5 was a speed claim you had to test. MiniCPM5-2B is an average you have to unpack. Same discipline. Download. Run the three tasks you actually pay for. Keep the 70B if those three tasks fall over. Keep the 2.52B if they do not.
Long context is in the MiniCPM one-line description. We already walked through what actually changed in context windows. A small model that “does long context” can still degrade in the middle, still blow RAM when you stuff a repo in, still look fine on a synthetic needle. If long context is why you clicked, test the middle of a long file, not the leaderboard average.
On-device is the other sales word this week. Liquid’s draft model is the honest on-device story: GGUF and Safetensors, decode acceleration, prefill still hurts. MiniCPM at 2.52B is a different memory envelope than 279.5 million. Do not say “edge” about both without a RAM number.
Julia 1 and Saaras are the rows people will skip
Supersonic Labs putting a 144.3-million-parameter decision model on mmBERT-small is not a mini-MiniCPM. Classification at that size is a different stack: short inputs, a label space, a pilot set. Three of four pilots beat the reference. Banking77 did not. Banking77 is a well-known intent set. If you cannot win it, you say so and you keep the model in the domains where the three pilots worked. That is more honest than a 34-bench average.
Sarvam’s Saaras V4 is the other skip-risk. Twenty-two official Indian languages plus English is a coverage claim, not a parameter claim. The table does not even print a size. If your product is Hindi or Tamil audio, you evaluate this checkpoint against the ASR you already pay for, on your audio, with your noise. If your product is English coding agents, you do not owe this row a paragraph in the launch blog.
The reason to keep both rows in the week-in-review is anti-mash. MarkTechPost logged them next to Ornith and MiniCPM. Logging is not taxonomy. A speech model, a classifier, a draft decoder, a 9B agent, and a 2.52B generalist can share a Friday and still not share a GPU budget.
Hermes AI’s line that Ornith-1.5-9B sat close to 35B models on the agent benches they watched is the kind of sentence that will get copied without the bench names. Keep Terminal-Bench 2.1 at 47.0 and SWE-Bench Verified at 70.6 next to it. “Close to 35B” without those two numbers is marketing. With them, it is a starting point for a reproduction.
What to do with the table on Monday
If you run evals: add MiniCPM5-2B to the cheap row. Keep the 34-bench average out of the slide unless you also show two tasks you care about. Put Ornith on the agent row only. Put Saaras on the ASR row only. Put Julia 1 on the classification row, with Banking77 as the miss.
If you run a VLM on a box with a camera: read Liquid’s Amdahl paragraph, then measure prefill. If prefill is 70 percent of wall time, DSpark is a rounding error and you should be looking at the encoder. If decode is 70 percent, try the draft model from the Hub and keep the logs.
If you run a support bot: the SAE paper is the design prompt. Build a “not sure” path that is allowed to be 47 percent of traffic. Your business will hate that sentence. Your accuracy will not.
If you write commentary: stop calling this a coordinated bet as if the labs shared a calendar. MarkTechPost logged a dense window. Dense windows happen when everyone has the same GPU budget and the same counter-narrative. The counter-narrative is useful. It is not a merger.
The 2.8 points are real in the recap that printed them. So is the 14-hour battery on a cheap laptop, and so is the 16-hour battery on a dear one, and they are still two machines. MiniCPM5-2B and a 9B coding model and a 279.5M draft are three machines. The week made them look like a movement. They are a shopping list. Shop it by job.