Model testing · 31 July 2026 · Asfandyar Malik
Claude Opus 5’s Three Deal-Breakers—and the Fixes
11 chapters, played or read
Runtime 08:15
Claude Opus 5 has three genuinely annoying behaviours. Anthropic documented all three, and published the fix for each.
Claude Opus 5 has three genuinely annoying behaviours. Anthropic documented all three, and published the fix for each.
If you moved a working setup from Claude Opus 4.8 to Claude Opus 5, you probably noticed at least one of these within a day: it writes longer answers than you asked for, it does more than you asked fo
Claude Opus 5 does not replace Claude Fable 5. They are different tiers.
This is the most repeated mistake about the release, so it is worth settling before anything else. Anthropic's own positioning is that Claude Fable 5 is the most capable widely released model, for the
It talks too much — and the obvious fix does not work.
Keep responses focused, brief, and concise to avoid overwhelming the person. Disclaimers and caveats are brief, with most of the response on the main answer; when asked to explain something, give a high-level summary unless an in-depth one is specifically requested.It expands the job. You say fix this bug; it also reorganises three other things.
Deliver what the user asked for, at the scope they intended. Interpret ambiguity the way a careful colleague would: make routine judgment calls yourself, and check in only when different readings would lead to materially different work. If you conclude the ask is mistaken or a better approach exists, say so in a sentence and keep going with the task as asked — don't quietly narrow, widen, or transform it. Finish the whole task, not just the easy part of it - only report completion when it's fully done. If you genuinely can't complete something, do the rest and state plainly what's missing and why. Stop short of actions or changes that are clearly beyond what the user's ask implies.Raise max_tokens and keep thinking on
This is the recommended path. Claude Opus 5 performs unusually well at low and medium effort, so you can often keep thinking enabled, drop effort a level, and land at better qualit
Disable thinking explicitly with thinking: {"type": "disable
Tool calls can arrive as plain text
The model occasionally writes a tool call into its user-facing text instead of emitting a structured tool_use block. The turn completes normally, no error is raised, and the call n
<thinking> tags can leak into the visible response
Internal XML shows up in user-facing output.
Delete your verification instructions. Seriously.
There is a fourth behaviour that belongs on this list, and it is the one most likely to be actively costing you money right now — because the fix is to remove something you probably added deliberately
Effort: run a fresh sweep, because prior-model defaults rarely transfer.
output_config={"effort": "high"} # low | medium | high | xhigh | maxdatetime.now() or a timestamp interpolated into the system p
A UUID, request ID, or session ID early in the content.
json.dumps() without sort_keys=True, or iterating a set — no
Conditional system-prompt sections, where every flag combina
Rate limits are a separate bucket
Opus 4.8, 4.7, 4.6, and 4.5 share one combined Opus limit. Claude Opus 5 does not draw from it. Shifting traffic over neither frees headroom on the old bucket nor inherits it — che
Priority Tier does not cover Claude Opus 5
It is supported on every other current model, including Claude Fable 5 and Opus 4.8, but a Priority Tier request naming Claude Opus 5 fails validation.
Fast mode exists, at a price
speed: "fast" runs the same model at up to 2.5× higher output tokens per second, but is priced at $10 / $50 — the same as Fable 5, and double standard Opus 5. It needs the beta mes
Sampling parameters are gone
temperature, top_p, and top_k return a 400. If you used temperature=0 for determinism, note that it never actually guaranteed identical outputs; use low effort and a tighter prompt
Bottom line: all three deal-breakers are prompt-and-configuration problems with documented fixes, not model defects. The one that costs the most is the one nobody names — verification scaffolding you added for an older model and never removed.
Source
Prerequisites
export ANTHROPIC_API_KEY=sk-ant-... # or: ant auth login command -v jq >/dev/null || brew install jq00:0008:15
Claude Opus 5 has three genuinely annoying behaviours. Anthropic documented all three, and published the fix for each.
If you moved a working setup from Claude Opus 4.8 to Claude Opus 5, you probably noticed at least one of these within a day: it writes longer answers than you asked for, it does more than you asked for, and your token spend went up even though the per-token price did not change.
None of these are bugs, and none of them are secrets. All three are documented behavioural shifts with official guidance attached — in two cases, a specific prompt block with a measured result. This resource walks through what each one actually is, why it happens, the exact fix, and — just as importantly — the fixes that look obvious but do not work.
Snapshot: 1 August 2026. Model pricing, defaults, and behaviour change; every claim below links to the primary source, and the pricing and model pages are the ones to check before you commit to a production decision.
Claude Opus 5 does not replace Claude Fable 5. They are different tiers.
This is the most repeated mistake about the release, so it is worth settling before anything else. Anthropic's own positioning is that Claude Fable 5 is the most capable widely released model, for the most demanding reasoning and long-horizon agentic work. Claude Opus 5 is described as a step-change over Claude Opus 4.8 — strongest on deep reasoning, agentic and long-horizon work — at half the cost of Claude Fable 5.
"Cheaper and excellent" and "most capable" are different claims. Both are true at the same time, and they point at different jobs:
Claude Opus 5
$5 / $25 per million tokens. The right default for agentic coding, enterprise work, and almost everything you do repeatedly. Half the price of Fable 5 for work that does not need the top tier.
Claude Fable 5
$10 / $50 per million tokens. The highest-capability tier, for the hardest reasoning and longest-horizon autonomous runs. If you kept Fable around for planning and heavy thinking, that instinct was correct.
There is a second, quieter difference worth knowing if you are choosing between them: Claude Fable 5 requires 30-day data retention and is not available under zero data retention. If your organisation is configured for ZDR, every Fable 5 request returns a 400 invalid_request_error regardless of how well-formed the payload is. That is a procurement question, not a debugging one.
It talks too much — and the obvious fix does not work.
Claude Opus 5's default user-facing responses are longer than prior models'. The answer is in there; it is just wrapped in more context, more caveats, and more explanation than the question needed.
Here is the trap, and it catches almost everyone. The instinct is to reach for the effort parameter and turn it down, on the theory that a model thinking less will say less. Anthropic's guidance is explicit that this does not reliably work. Effort governs how much the model thinks and how much total work it does — it may move thinking volume without reliably changing the length of the visible output. You can spend a week on that dial and get nothing.
The lever is the prompt. This is the documented instruction, and in Anthropic's own testing it cut user-facing response length by roughly 20%:
Keep responses focused, brief, and concise to avoid overwhelming
the person. Disclaimers and caveats are brief, with most of the
response on the main answer; when asked to explain something, give
a high-level summary unless an in-depth one is specifically
requested.
If your system prompt is long, the guidance is to pair that with a short reminder near the end, where it is less likely to get lost:
<tone_preference>
Keep outputs reasonably concise.
</tone_preference>
There is a third variant in agentic sessions: narration between tool calls. Claude Opus 5 narrates what it is about to do, and per-message output during a long agentic run is longer than prior models'. This one responds well to guidance about how to communicate rather than just how much — describing the reader ("a teammate who stepped away and is catching up"), asking it to lead with the outcome, and telling it that being readable matters more than being brief.
It expands the job. You say fix this bug; it also reorganises three other things.
You ask for a bug fix. It fixes the bug — and renames some variables, adds a helper you did not ask for, and improves error handling in a file you were not touching. This is documented, and Anthropic names it: task scope expansion. The model adds steps the user did not request, or applies its own judgement about what the task should be without making that clear.
Same shape of fix as verbosity — one prompt block. In testing, this reduced scope changes to nearly zero without producing an annoying stream of clarifying questions:
Deliver what the user asked for, at the scope they intended.
Interpret ambiguity the way a careful colleague would: make routine
judgment calls yourself, and check in only when different readings
would lead to materially different work. If you conclude the ask is
mistaken or a better approach exists, say so in a sentence and keep
going with the task as asked — don't quietly narrow, widen, or
transform it. Finish the whole task, not just the easy part of it -
only report completion when it's fully done. If you genuinely can't
complete something, do the rest and state plainly what's missing and
why. Stop short of actions or changes that are clearly beyond what
the user's ask implies.
Read the middle of that block carefully, because it is doing two jobs at once. The obvious job is stopping the model from doing more than you asked. The less obvious one is the "finish the whole task" clause, which stops it doing less — reporting completion when only the easy part is done. Scope-discipline wording alone did not cover premature "done" claims; that clause was added specifically to catch them.
That matters for how you write the instruction. If you paraphrase this block down to "only do what I asked," you will fix scope expansion and quietly introduce a truncation problem in its place.
Your usage disappears faster — and this one is not a bug at all.
This is the one people most often misdiagnose as a billing problem or a silent model downgrade. It is neither. It is a default that changed, and you are probably not touching the setting that controls it.
On Claude Opus 4.8 and Opus 4.7, a request that omitted the thinking parameter ran without thinking. On Claude Opus 5, the same request runs with adaptive thinking. Identical code, same payload, more tokens — because the model is now thinking on requests where it previously was not.
max_tokens is a hard cap on thinking plus response text, together. A workload that ran thinking-off on Opus 4.8 and sized max_tokens tightly around its expected answer can now hit the cap mid-response. If you have routes that never set thinking and suddenly produce truncated output, this is why — check for stop_reason: "max_tokens".You have two ways to respond, and which is right depends on why you had thinking off in the first place:
- Raise
max_tokensand keep thinking on. This is the recommended path. Claude Opus 5 performs unusually well atlowandmediumeffort, so you can often keep thinking enabled, drop effort a level, and land at better quality and lower spend than the thinking-off configuration you started from. - Disable thinking explicitly with
thinking: {"type": "disabled"}— but read the constraint and the two failure modes below before you choose this.
The constraint: disabling thinking is capped at high effort
On Claude Opus 5, thinking: {"type": "disabled"} is accepted only at effort high or lower. Combining it with xhigh or max returns a 400. Opus 4.8 accepted that combination, so this is a real migration break — and the check is per request, so a later call that raises effort while thinking is still disabled fails even though earlier calls in the same conversation succeeded.
Two failure modes when thinking is off
These only affect code that explicitly opts out — but that includes any disabled-thinking setting carried forward from an Opus 4.8 config, where it was the default behaviour.
- Tool calls can arrive as plain text. The model occasionally writes a tool call into its user-facing text instead of emitting a structured
tool_useblock. The turn completes normally, no error is raised, and the call never runs — so your harness sees a successful turn that silently did nothing. In an agentic loop, that bogus text then sits in conversation history and skews later turns. <thinking>tags can leak into the visible response. Internal XML shows up in user-facing output.
Delete your verification instructions. Seriously.
There is a fourth behaviour that belongs on this list, and it is the one most likely to be actively costing you money right now — because the fix is to remove something you probably added deliberately.
Claude Opus 5 verifies its own work without being asked. Instructions that tell it to verify — "include a final verification step for virtually any non-trivial task", "use a subagent to verify", or a per-prompt "double-check your answer" — now cause over-verification. Anthropic's guidance is explicit that removing them reduces over-verification with no capability regression. This is a delete, not a rewrite.
Closely related, and pointing the same direction: Claude Opus 5 delegates to subagents more readily than Opus 4.8 did. This is a direction change worth flagging, because Opus 4.8 under-reached for subagents and needed prompting to delegate more. If you added "delegate more" guidance for 4.8, it should come out — and you likely want an explicit cap instead, since each subagent re-establishes context, re-explores, reports back, and then the coordinator re-reads the report. "Do not use subagents to verify" and "delete your verification scaffolding" are the same underlying fix seen from two angles.
Effort: run a fresh sweep, because prior-model defaults rarely transfer.
Effort goes inside output_config, not at the top level — a common mistake when migrating:
output_config={"effort": "high"} # low | medium | high | xhigh | max
The default is high (equivalent to omitting it). The documented starting points and the model's measured behaviour pull in slightly different directions, and both are worth knowing:
- Start at
xhighfor coding and agentic work,highfor other intelligence-sensitive workloads. That is the published recommendation.maxis the explicit top tier, worth testing where capability matters more than spend — though it can show diminishing returns and overthink simpler tasks. - Then sweep downward, because
lowandmediumpunch well above their weight on this model — strong quality at a fraction of the tokens and latency on many workloads. - Step down only where your own evals show quality holds — but do run the sweep. Effort defaults carried over from a prior model are usually not the right setting here.
- At
xhighormax, set a largemax_tokensso the model has room to think and act across tool calls and subagents. Start at 64K and tune from there.
Note the relationship is not monotonic on agentic work: higher effort up front often reduces turn count and total cost, because better planning means fewer round trips. Judging effort by per-request token count alone will mislead you.
Claude Opus 5 halves the prompt-cache minimum. Re-check prompts you wrote off.
The minimum cacheable prefix on Claude Opus 5 is 512 tokens, down from 1024 on Claude Opus 4.8. Prompts previously too short to cache now create cache entries with no code change at all. If you audited caching on an older model and concluded a prompt was too small to bother with, that conclusion may simply be out of date.
This matters more than it sounds, because the minimum is not monotonic across generations — it is 512 on the newest models but 4096 on Opus 4.6 and Haiku 4.5. A 3K-token prompt caches on Claude Opus 5 and silently will not on Opus 4.6. Silently is the operative word: there is no error, just cache_creation_input_tokens: 0.
The economics are worth committing to memory. Cache reads cost roughly 0.1× base input price; cache writes cost 1.25× for the 5-minute TTL and 2× for the 1-hour TTL. With the 5-minute TTL you break even on the second request; with the 1-hour TTL you need at least three, because the doubled write cost takes longer to pay off.
Verifying it actually works
Caching is a prefix match: any byte change anywhere in the prefix invalidates everything after it. Check usage.cache_read_input_tokens — if it is zero across repeated requests that should share a prefix, something is silently invalidating it. The usual culprits:
datetime.now()or a timestamp interpolated into the system prompt — the prefix changes every single request.- A UUID, request ID, or session ID early in the content.
json.dumps()withoutsort_keys=True, or iterating a set — non-deterministic serialisation means the bytes differ even when the data does not.- Conditional system-prompt sections, where every flag combination becomes a distinct prefix.
- A tool set that varies per user. Tools render at position zero, so nothing caches across users.
One genuinely useful Claude Opus 5 feature here: you can append a {"role": "system", ...} message to messages[] mid-conversation instead of editing the top-level system field. Editing top-level system changes the prefix ahead of the entire conversation history, so every cached turn gets reprocessed uncached. A mid-conversation system message sits after the cached history and leaves it intact — and it is the prompt-injection-safe operator channel, unlike text stuffed into a user turn. No beta header required.
Things that will surprise you in production.
- Rate limits are a separate bucket. Opus 4.8, 4.7, 4.6, and 4.5 share one combined Opus limit. Claude Opus 5 does not draw from it. Shifting traffic over neither frees headroom on the old bucket nor inherits it — check your tier's Claude Opus 5 limits before you move volume.
- Priority Tier does not cover Claude Opus 5. It is supported on every other current model, including Claude Fable 5 and Opus 4.8, but a Priority Tier request naming Claude Opus 5 fails validation.
- Fast mode exists, at a price.
speed: "fast"runs the same model at up to 2.5× higher output tokens per second, but is priced at $10 / $50 — the same as Fable 5, and double standard Opus 5. It needs the beta messages endpoint, thefast-mode-2026-02-01flag, andspeedas a top-level parameter. Claude API only; not on Bedrock, Google Cloud, or Foundry, and not with the Batch API. - Sampling parameters are gone.
temperature,top_p, andtop_kreturn a400. If you usedtemperature=0for determinism, note that it never actually guaranteed identical outputs; useloweffort and a tighter prompt instead. budget_tokensis gone too.thinking: {"type": "enabled", "budget_tokens": N}returns a400. There is no direct replacement — effort is a different control, not a thinking budget. For agentic loops, task budgets (beta, minimum 20,000 tokens) are the closest thing.- Handle
stop_reason: "refusal"before reading content. Claude Opus 5 ships with elevated cybersecurity safeguards, and its classifiers can decline a request — returning a normal HTTP 200 with an empty or partialcontentarray. Code that indexescontent[0]unconditionally breaks. A pre-output refusal is not billed at all; a mid-stream one bills the streamed partial.
What to actually do this week.
If you just migrated from Opus 4.8 and something feels off — check three things in order: does any route omit thinking (it now thinks, so revisit max_tokens); does any route combine disabled thinking with xhigh or max effort (that is a hard 400); and are you still sending temperature or budget_tokens (both 400).
If the output is too long — add the conciseness block and, separately, a deliverable-length instruction. Do not touch effort for this.
If it does more than you asked — add the scope block in full, including the finish-the-whole-task clause. Paraphrasing it down introduces a different problem.
If your token spend jumped — audit for verification instructions and delegate-more guidance carried over from Opus 4.8, and delete both. Then run an effort sweep including low and medium, and re-check prompt caching against the new 512-token minimum.
If you need the top capability tier — that is Claude Fable 5, at double the price, and it needs 30-day data retention. Keep Opus 5 as the daily driver and escalate deliberately.
Bottom line: all three deal-breakers are prompt-and-configuration problems with documented fixes, not model defects. The one that costs the most is the one nobody names — verification scaffolding you added for an older model and never removed.
What it proves: that Anthropic's two published prompt blocks measurably change Opus 5's behaviour — on your project, with your number, not their number.
Prerequisites
export ANTHROPIC_API_KEY=sk-ant-... # or: ant auth login
command -v jq >/dev/null || brew install jqCost: well under $1. Part 1 is ten short API calls.
PART 1 — Measure the conciseness block (scripted, gives you a %)
This is the money shot. It asks the same five questions twice — once with nothing, once with Anthropic's conciseness block as the system prompt — and prints the word-count reduction.
cd youtube-videos/04-opus5-vs-fable5-dealbreakers/experiments
./audit-conciseness.shWhat you'll see:
QUESTION BASELINE WITH BLOCK CHANGE
what does a JS event loop do 412 318 -22.8%
...
─────────────────────────────────
AVERAGE 386 291 -24.6%To use your own project's questions instead (stronger, more relatable):
./audit-conciseness.sh my-questions.txt # one question per linePART 2 — The scope test (manual, needs a real bug)
Not scriptable, and shouldn't be — the point is that it happens in a real repo.
Setup
- Find a genuine, reproducible bug in a real project of yours. Not planted. A failing test is ideal because "fixed" is objective.
git stashanything uncommitted. Make suregit statusis clean.- Empty your
CLAUDE.md(back it up first:cp CLAUDE.md CLAUDE.md.bak).
Run A — baseline
git checkout .
# In Claude Code: Fix this bug: <paste the failing test output>. Fix only this bug.
git diff --stat # ← screenshot this. This is scope-before.png
git diff --stat | tail -1 # note: files changed, insertions, deletionsRun B — with the block
git checkout .
# Paste the scope-discipline block into CLAUDE.md (button on the deck, or SOURCES.md §6)
# Then the IDENTICAL prompt in a fresh Claude Code session.
git diff --stat # ← screenshot this. This is scope-after.pngRecord
| Files touched | Lines changed | Bug fixed? | |
|---|---|---|---|
| Baseline | |||
| With scope block |
The story is in files touched. If baseline touched 6 files and the block-run touched 1, that's your slide, and it needs no commentary at all.
Same window, same zoom, same scroll position for both screenshots. If the two frames differ in any other way, the comparison stops proving anything.
PART 3 — The delete audit (four seconds,)
Find every "verify your work" instruction you're already carrying. Opus 5 verifies itself; the docs say these now cause over-verification and removing them costs no capability.
grep -rniE "double.?check|verify (your|before)|verification step" CLAUDE.md .claude/ AGENTS.md 2>/dev/nullPART 4 — The effort sweep
./effort-sweep.sh "Refactor this function to be pure and add tests: <paste a real function>"Prints the same task at low / medium / high with input, output, and cost per run.
What "captivating" looks like here
The hook isn't the prompt blocks — those are just paragraphs. The hook is that you turned a documented claim into a measured number on your own machine, and showed the command. Three concrete moments to hit:
- The AVERAGE line appearing in your terminal. Don't cut away. Let the number land.
- The two
git diff --statoutputs side by side. 6 files versus 1 file needs no words. - Running the grep live and deleting what it finds. It's four seconds and it's the most re-shareable beat in the video, because it contradicts standard advice.
Honesty checklist
- [ ] Your % is your number; Anthropic's ~20% is labelled as theirs
- [ ] The bug in Part 2 is real, not planted — say so
- [ ] Each measurement is a single run; said out loud at least once
- [ ] If any block didn't visibly work, that's in the video
Keep exploring
More long-form resources built the same way — primary sources, working files, and commands that actually run.
Setup instructions rot faster than anything else here. Every command on this page is copied from the vendor's own documentation and linked to it — open the docs alongside this page and trust those over this one.