Unite.AI on October 20: IBM Research says a retail agent with learned guidelines sits first on τ²-bench at 67.5 percent Pass^4, ahead of RAFT-30B-A3B at 61.4 percent, a model IBM identifies as fine-tuned on synthetic τ²-bench retail data, by 6.1 points. The snapshot date in that recap is October 12, 2026. That is a leaderboard number. It is not a weight file.
We already filed four Mac Studios at 36 tok/s that were not a cluster ticket and Nature cutting geometry while RAG cut a masonry score. This week the named metric is Pass^4. The named claim is “no weight updates, human annotation, or test-set leakage.” Read the split before you copy the rank.
67.5 percent is base split, train plus test
Unite.AI is explicit: leaderboard submissions are scored on the full base split, train and test tasks combined, reported at Pass^4, so those figures differ from the held-out numbers later in the same piece. The retail filing is a τ²-bench pull request dated September 14, 2026. Pass^1 81.14 percent through Pass^4 67.54 percent across 114 base-split tasks at four trials. Average cost $0.1084 per conversation on tau2-bench v1.0.1.
The recap also says the guidelines were learned from a retail train split that the base evaluation split includes. That sentence is the one to keep on the whiteboard. If train is inside the board you are first on, you are first on a board that includes the data you learned from. IBM discloses modified prompts and says it expects the submissions to be listed as Unverified. Both PRs remain open in the recap we have.
RAFT-30B-A3B at 61.4 percent is the comparison IBM wants, because that one is described as fine-tuned on synthetic retail data. IBM’s pitch is: we beat a fine-tune without touching weights. The pitch is only as strong as the split. Pass^4 on a mixed base split is not Pass^4 on a frozen held-out set. Do not quote 67.5 percent next to a held-out table from another paper.
Cost is the other number that survives. Eleven cents a conversation is a budget. It is not a proof that the method is cheap in your stack, because your stack is not tau2-bench v1.0.1.
Pass^1 at 81.14 percent versus Pass^4 at 67.54 percent is the reliability drop you should expect when the same task is tried four times. Leaderboards that only print Pass^1 are selling the lucky run. IBM printed both. Quote both. If your own agent eval is a single trial, you are publishing Pass^1 and calling it production.
Airline 82 percent is 50 tasks and a different model
The airline leaderboard submission is dated September 21, 2026. Pass^1 88.50 percent through Pass^4 82.00 percent across 50 tasks, average $0.1690 per conversation. IBM reports the gpt-5.5 agent ranks first at 82.0 percent, above frontier entries it lists at 66 to 72 percent. A gpt-4.1 “sup3” variant placed eighth at 44 percent, ahead of GPT-5.2 running without reasoning, and nearly matching Claude Sonnet 4.5 in that telling.
Fifty tasks is not 114 tasks. Airline and retail are different domains on this bench. Collapsing them into “IBM is first on τ²-bench” is how a 44 percent eighth place and an 82 percent first place become the same tweet.
The recap’s frontier band of 66 to 72 percent is IBM’s list, not an independent table we extracted. Treat it as IBM’s list. If you need the live board, open the PR. Unverified means the board has not blessed the run. A blog post is not the blessing.
$0.1690 per airline conversation is higher than the retail $0.1084. That is the direction you expect if airline policies are longer. It is still a bench cost, not your production trace.
gpt-4.1 sup3 at 44 percent Pass^4, eighth, is the reminder that “learned guidelines” is not a free upgrade you paste onto any model. The recap puts that variant ahead of GPT-5.2 without reasoning and near Claude Sonnet 4.5 on that slice. Near is not 82 percent. If your agent is a 4.1-class model, do not paste the gpt-5.5 airline headline on your quarterly slide.
Held-out airline is where the 15-point Pass^4 lives
IBM also reran airline with gpt-5.5 as both the agent and the guideline-extraction model, re-mining guidelines from that agent’s own runs. No extracted guideline reached support of at least three, so the high-support core was “sup2” with 103 guidelines, next to a 146-guideline lossless set.
On the same held-out airline tasks, the gpt-5.5 baseline scored 72.5 percent accuracy and 55.0 percent Pass^4. sup2 reached 80.0 percent and 70.0 percent. Lossless reached 82.5 percent and 70.0 percent. IBM states those as gains of 10.0 accuracy points and 15.0 Pass^4 points. The lossless result reproduced exactly across reruns, according to the recap.
This is the cleaner comparison than 67.5 percent on the mixed base split. Baseline versus sup2 versus lossless, same held-out tasks, same model family. Support less than three got dropped from the “high-support” set. That is a hyperparameter. If you reimplement this, the support threshold is the thing that will move your count of guidelines from 146 to 103 and your Pass^4 from a lossless tie at 70 to… still 70, in this table. Accuracy moved 80.0 to 82.5. Pass^4 did not. Do not sell lossless as a Pass^4 win over sup2 on this rerun. It was not.
We already lived through Bolmo reading bytes while the tokenizer stayed in Stage 1. Method cards love a named component. The component here is a guideline list with a support cutoff. Name the cutoff.
No weight update is a claim about the agent, not about the board
IBM states the approach improves stock agents without weight updates, human annotation, or test-set leakage. The leakage sentence sits next to the earlier sentence about learning from a train split that the base evaluation split includes. Those two sentences can both be in the article. They cannot both be flattened into “no leakage, trust the 67.5.”
What “no weight update” actually buys you: you can keep the vendor model pinned and still inject a learned policy document. That is ops-friendly. It is also how you get a prompt-injection surface the size of the guideline file. If the guidelines are learned from traces, the traces are now part of your policy. Treat the file like you treat a system prompt that an intern can edit.
Human annotation skipped is a cost save. It is also a miss for the cases τ²-bench does not score. A retail bench will not tell you your refund guideline just authorized a $2,000 gesture. Pass^4 is four trials. Four is not your incident review.
If you are going to try this at work, log three things the IBM recap actually gives you: the support threshold, the Pass^k, and whether your eval split contains the train split. If you cannot answer the third, you are quoting 67.5 percent in the dark.
MemoraX’s 10 NeurIPS papers are a company page
Markets Insider on October 22 says MemoraX AI researchers and collaborators have 10 papers accepted to NeurIPS 2026. Jianye Hao, founder and CEO: memory should be a learnable capability and an independent infrastructure layer, rather than an extension of a foundation model. The product names in that page are Memory API (write, search, update, delete) and MemoraX Code, which is supposed to keep repository structure, architectural decisions, coding preferences, and debugging experience across sessions.
That is a vendor thesis in the same week as IBM’s guideline extractor. It is not a τ²-bench number. Do not stack “10 NeurIPS papers” next to 67.5 percent as if they were the same result. One is an acceptance count on a company wire. The other is a Pass^4 on a public bench with an Unverified flag.
The useful overlap is the sentence both camps are selling: memory as something you train, not a context window you hope is long enough. IBM’s version, in the Unite.AI recap, is guidelines mined from runs. MemoraX’s version is an API plus a coding-agent memory tool. The API verbs on that page are write, search, update, and delete. Those are CRUD. CRUD is not Pass^4. If you need a paper, wait for the NeurIPS camera-ready. If you need a bench, open the τ²-bench PR. A Markets Insider recap of a CEO quote is neither.
MemoraX Code’s claimed contents are repository structure, architectural decisions, coding preferences, and debugging experience across sessions and agents. That list is a product brochure. It does not include a τ²-bench domain, a Pass^k, or a cost per conversation. Put it in the same week as IBM only as a reminder that “learned memory” is about to be a category label every vendor will wear. The label is not 67.54 percent.
GPTZero 4o is 1 in 10,402. It is not Pass^4.
GPTZero’s own post introduces GPTZero 4o, “O standing for Omni,” aimed at mixed and modified AI text. False positive rate of 1 in 10,402 on diverse student writing corpora, per that post. A recall table: GPT-5.6, 10,898 AI docs, 99.89 percent; Gemini 3.6, 8,452, 99.87 percent; Grok 4.5, 2,871, 99.37 percent; Claude 5, 11,707, 99.04 percent. The post says the updated model has been the default since 20 September 2026. The search hit we used is dated October 18. If you turned GPTZero on in summer, you may already be on 4o without a banner. Check the model name in the dashboard before you cite last year’s FPR in a faculty email.
If your actual job this week is agents, pin the IBM split language and the $0.1084. If your actual job is classroom integrity, pin the 1 in 10,402 and ignore τ²-bench. Mixing them is how you get a slide that cannot be audited.
A detector FPR is a student-writing number. τ²-bench Pass^4 is an agent-policy number. The only reason this URL is in the same research pile is that it landed in the same week’s “LLM” net. Keep it in a footnote in your head: people will paste IBM’s 67.5 percent into a detector conversation and call it evaluation. It is not.
If your actual job this week is agents, pin the IBM split language and the $0.1084. If your actual job is classroom integrity, pin the 1 in 10,402 and ignore τ²-bench. Mixing them is how you get a slide that cannot be audited.
The rank that should survive from October 20 is not “IBM won memory.” It is: 67.54 percent Pass^4 on 114 retail base-split tasks, Unverified, train inside the split, $0.1084 a conversation, no weight update claimed. Quote that paragraph or quote nothing.