Apple’s machine learning team says a cluster of four Mac Studios generates more than 36 tokens per second on Muse Spark Open, “well above average human reading speed,” at an 8,192-token context. The same post says native BF16 weights for that model would need more than 1.6TB for parameters alone, and that 4-bit NVFP4 brings the footprint to about 493GB, which still does not fit one 512GB Studio once you count intermediates. That is a lab configuration. It is not a change ticket.
Computerworld wrapped the same week as Apple Intelligence plans for China plus Meta’s open weights on a couple of Macs. TechTimes, dated October 15, is the two-machine version: layers split, activations over RDMA, one mlx.launch and a hostfile. Three write-ups, one stack. Mix them and you will order Thunderbolt cables because a blog said “supercomputers.”
We already filed open weights taking tokens while closed models kept the bill and MiniCPM5-2B not being a server. This week the small cluster is made of Macs.
36 is four boxes, not one
Apple’s figure is explicit. Four Mac Studios. Muse Spark Open. Over 36 tok/s. Context 8,192. Human reading speed is the marketing comparison, not a benchmark suite. If your eval is GSM8K, this number does not speak. If your eval is whether the cursor keeps up with a person reading, 36 is the claim they wanted.
Single-box reality is in the same post. Qwen and Mistral, in the sizes they discuss, fit one Mac Studio with 512GB unified memory. Muse Spark Open does not, at native precision. More than 1.6TB for parameters in BF16. NVFP4 at about 493GB plus working memory. You are already in multi-box land before you generate token one.
The 4.6× they print is startup, time-to-first-token, against a comparison the extract cuts off. Do not turn 4.6× into decode speed. Do not turn it into a four-box claim. TTFT is the wait before the first token. Decode is the 36. Keep the rows apart.
M5 Ultra, Neural Accelerators in the M5 GPU family, Thunderbolt 5 RDMA, MoE implementations in MLX: Apple lists those as the year’s work that makes the combo possible. Each item is a driver or a kernel path. Together they are why the post exists. Separately, an M4 Mac Mini on someone’s desk is not this paragraph.
Computerworld repeats the 36 tok/s sentence and frames it as a white paper from the ML team. Fine. It also says almost anyone in AI research already works on a Mac. That is columnist color. It is not a utilization metric. Do not put it in a procurement deck.
If you have one Studio, you can run the models Apple says fit. If you have two, you are in the TechTimes pipeline. If you have four, you can quote 36. If you have a laptop and a hope, you have a laptop.
Two machines is a pipeline, not NCCL-on-your-behalf in spirit only
TechTimes describes the two-box setup as pipeline parallelism. Transformer layers split across nodes. RDMA carries activation tensors from machine one to machine two. CPU is not in that hop. From the developer’s side: mlx.launch pointed at a hostfile of two addresses. No MPI config, no driver wrangling, no handwritten parallel code, in that account. The MLX adapter for Mac Studio is “available now” there.
JACCL is the collectives library underneath, open source, named for Jack Beasley (Thunderbolt 5 RDMA) and Angelos Katharopoulos (MLX), a deliberate nod to NVIDIA’s NCCL. AllReduce, AllGather, and related primitives. MLX calls it without you wiring it. Introduced at WWDC 2026. That is a year of runway before this inference post. If your mental model is “Apple just invented clusters,” the WWDC timestamp says otherwise.
Meta’s contribution, in the same piece, is GPU kernel work for M5 Ultra. The launch blog numbers TechTimes cites are 1.03× to 3.95× on key compute kernels. A 1.03× is a rounding error you would not blog. A 3.95× is the slide. Both are kernel microbenchmarks, not end-to-end tok/s. Do not average them into 2.5× and put it next to 36.
The stack is not Muse-only. TechTimes is clear: any open-weight frontier model whose weights fit the combined unified memory of a Thunderbolt-linked Studio cluster can, in principle, use the same path. That sentence is a category claim. It is also a memory-budget claim. Add up the RAM. If the quantized model plus KV plus activations blow the sum, the hostfile will not save you.
Pipeline across two boxes is not tensor parallel across an eight-GPU node. Failure modes differ. If machine two hangs, you do not have a replica, you have a severed pipe. Apple’s post is a how-to for people who already own the hardware. It is not a PaaS.
We keep a similar distinction on geometry-vs-RAG scoring: the number only means the setup that produced it. Four Studios at 36 tok/s is that setup.
A desk is not a ticket queue
Computerworld’s headline says supercomputers. The body is more useful when it lists what Apple actually shipped: distributed inference tooling, a China Apple Intelligence plan that is a different story, and on-prem Muse Spark Open with Meta on a couple of Studios. Bundle those in a week-in-review if you write for IT. Unbundle them if you run ML.
On-prem here means the Macs you can touch. It does not mean SOC2, a job scheduler, or a GPU cloud invoice. If your company requires a ticket to stand up inference, four Mac Studios on a lab bench will fail the ticket. They will also fail the fire code if you stack them without thinking about power. Apple did not print a wattage table in the extracts we have. Do not invent one. Do print the 512GB figure, because that is the reason the 493GB model still needs friends.
RDMA over Thunderbolt 5 is a cable and a driver, not a fabric you already have in the rack. If your cluster is InfiniBand, this post is a different store. If your cluster is two Macs on a desk, this post is the manual.
Muse Spark Open at “more than 800B” in Apple’s wording and “nearly a trillion” in the same post is a size class, not a Hugging Face filename you should guess. Use the name they used. Use the precision they used. BF16 vs NVFP4 is the whole reason one box fails.
Open weights on a desk also do not cancel the license. Meta’s access limits showed up in other coverage this week. Apple’s post is about fitting and speed. It is not your counsel’s memo. If you cannot download the weights, 36 tok/s is a screenshot.
For people who already live in MLX, the actionable piece is the launch path and the memory math. For people who live in vLLM on NVIDIA, the actionable piece is maybe JACCL as a curiosity and then back to the queue. We have been through speed-of-light claims on that stack before. This one is Apple’s number on Apple’s metal.
Unified memory is the whole trick
Apple’s post keeps returning to unified memory because that is the reason a Studio looks like a dense node instead of a GPU plus a host. 512GB is the ceiling they keep citing for one box. Qwen and Mistral, in the sizes they discuss, live under that ceiling. Muse Spark Open at BF16 does not. NVFP4 gets you to about 493GB for weights, which is a cruel neighbor to 512GB once activations and the KV cache show up. The two-box pipeline is not a flex. It is arithmetic.
MoE support in MLX is listed as a year of work alongside Neural Accelerators and RDMA. Mixture-of-experts models are why a “nearly a trillion” class even fits a desk cluster at 4-bit: you are not activating every parameter on every token. Apple did not print an experts-active count in the extracts we have. Do not invent 8-of-128. Do print that they bothered to implement MoE at all, because a dense 800B at 4-bit is a different memory story.
Thunderbolt 5 RDMA is the interconnect they chose instead of asking you to buy a NIC. Low latency across several Macs, in their wording. Several is the four-box 36 tok/s row and the two-box layer split. It is not a Clos fabric. Cable length, hop count, and whether someone unplugged a display are now your cluster management. JACCL sitting under MLX is how AllReduce happens without a sysadmin who has used nccl-tests. It is still a collective. Collectives still hang.
Computerworld’s China Apple Intelligence note belongs in a product roundup, not in a decode-speed paragraph. Same week, different stack, different compliance. If you paste it into an ML infra doc you will spend a meeting explaining why. Unbundle.
The 8,192 context on Figure 1 is also a limit, not a footnote. If your production prompt is a 32k dump of logs, you do not have Apple’s 36. You have a different memory tax and a different RDMA payload. Measure at the context you actually use. Pipeline parallelism across two machines will feel that tax first, because activations move.
None of this replaces a job queue. mlx.launch plus a hostfile is a developer command. It is not Slurm. If two people on the same desk both launch, you will find out who owns the Studios. Write that down before you call it on-prem inference for a team of twelve.
What to try, and what to skip
If you have a 512GB M5 Ultra Studio and a model Apple says fits, try it. That is the one-box row. Measure your own TTFT. Do not paste 4.6×.
If you have two, write the hostfile, run mlx.launch, and watch whether decode holds when the prompt is not 8,192 of synthetic context. Pipeline bubbles are real. Apple’s two-box section exists because 493GB plus overhead is rude to a single 512GB machine.
If you have four, you may reproduce something in the neighborhood of 36 tok/s on Muse Spark Open at that context. Neighborhood, not identity. Cooling, cable quality, and background load are not in Figure 1.
Skip buying a fourth Studio because a headline said supercomputer. Skip quoting NCCL as if JACCL were a drop-in on Linux. Skip assuming MoE support in MLX means your favorite MoE checkpoint is tuned. Skip mixing China product news into a kernel post.
If you cannot name the quantization, you are not ready to name the token rate. NVFP4 at 493GB is the load-bearing sentence. Everything else is a cable.