The smallest acceptable model is a policy decision
A small model does not save money because its price card is lower. It saves money when it finishes an admitted task inside the latency and token budget, produces evidence the system can check, and escalates before a weak answer becomes an expensive incident.
Claude Haiku 5.5 makes that distinction important. Anthropic released it on October 7, 2026 for high-volume, latency-sensitive work. The official list includes classification, routing, extraction, summarization, subagent tasks, browser automation, and focused coding. The model supports adaptive thinking, effort controls, a one-million-token context window, and up to 128,000 output tokens. For prompts up to 100,000 tokens, list pricing starts at $0.10 per million input tokens and $0.50 per million output tokens. Above 100,000 prompt tokens, both rates rise fivefold.
Those specifications invite a tempting migration: replace the previous small model or a larger worker everywhere, then let volume produce savings. That is not a safe routing policy. Anthropic also documents a newer tokenizer that makes the same text count as roughly 30 percent more tokens than on Haiku 4.5, preserved thinking blocks that can inflate later turns, thinking tokens that consume max_tokens, and breaking request changes. A cheaper token can coexist with a larger token count, longer trajectories, more retries, and more human correction.
Community evidence points in both directions. A 234-vote r/ClaudeCode test compared Haiku, Sonnet, and Opus on six real tasks and helped trigger useful discussion about scoped subagents. Other users report that Haiku works well for codebase search, log summaries, and small edits, while a critical thread describes it falling into “load-bearing trap after load-bearing trap.” These reports are not a benchmark for your system. Together they define the question your benchmark must answer: which task classes stay bounded under this model, effort level, prompt, harness, and tool policy?
Route task classes, not vibes. Approve the combination of model, effort, context, tools, verifier, consequence, and escalation rule.
The migration surface is larger than the model ID
Haiku 5.5 is not a drop-in price change. The official migration notes identify request and response behaviors that can break clients or distort cost measurements. Manual extended-thinking budgets move to adaptive thinking and effort. Requests using temperature, top_p, or top_k can error. Assistant prefills are rejected. Computer-use tool versions change. Responses can begin with thinking blocks, so code that assumes the first content block is final text is unsafe.
Multi-turn storage needs special attention. Earlier thinking blocks stay in context and count as input tokens. If a system replays stored thinking through another account, the blocks may not be reusable. Any transcript proxy, queue, regional failover, or multi-account architecture therefore needs an explicit rule for preserving, clearing, or regenerating them. “Replay the conversation” is no longer one operation.
| Change | Failure if ignored | Acceptance check |
|---|
| Adaptive thinking is default | Output cap is consumed before answer text | Parse block types and fail on empty verified output |
| Effort replaces manual budget | Old request returns an error or uncontrolled cost | Contract-test every allowed effort level |
| New tokenizer | Budget estimates undercount input | Recount the representative corpus with the Models API |
| Long-context price step | A retrieval burst multiplies input cost | Record prompt tokens before route and enforce a threshold |
| Thinking blocks persist | Multi-turn cost grows invisibly | Measure fresh, preserved, and cleared-history cohorts |
| Sampling parameters rejected | Legacy client fails before inference | Schema validation in CI and a production canary |
| Thinking blocks are account-bound | Cross-account replay breaks | Exercise failover with stored conversations |
The safest first step is an API contract suite, not an application benchmark. Send representative requests through every provider path you use, including direct API, Bedrock, Google Cloud, and Microsoft Foundry where applicable. Assert accepted fields, block parsing, stop reasons, tool schemas, cache behavior, usage accounting, and failure mapping before comparing model quality.
Admit bounded work and reject ambiguous work
A route begins with an admission decision. Task labels such as “coding” or “analysis” are too broad. A one-file import rename and a cross-service authorization change are both coding tasks; their ambiguity, consequence, evidence, and rollback are entirely different.
Describe each candidate route with six fields: task class, consequence, context size, tool surface, objective verifier, and escalation cost. Haiku is a plausible first route when the task is narrow, source evidence is supplied, allowed actions are limited, and success is machine-checkable. It should not be the automatic first route when the task changes authority, money, identity, production data, security policy, or an interface with weak rollback.
| Task class | Initial route hypothesis | Required verifier | Escalate when |
|---|
| Label or classify supplied text | Haiku, low or medium effort | Held-out labels and abstention cases | Confidence gap, unknown class, policy impact |
| Extract fields from a known schema | Haiku, medium effort | Schema, source spans, reconciliation | Missing evidence, totals mismatch, novel layout |
| Summarize logs or documents | Haiku, low or medium effort | Citation coverage and omission checks | Conflicting sources or consequential conclusion |
| Search a codebase | Haiku scout | File and symbol references | Architecture inference or uncertain ownership |
| Small deterministic edit | Haiku worker | Diff limit, tests, lint, typecheck | Cross-boundary change or repeated failure |
| Security or data-access change | Stronger model plus human owner | Threat model, tests, independent review | Always before release |
| Open-ended incident diagnosis | Stronger model initially | Evidence graph and falsification plan | Any unsupported causal claim |
Do not use the model to decide its own admission unobserved. A deterministic policy can classify obvious fields, while an independent classifier may suggest a route for ambiguous cases. The runtime still enforces hard exclusions, token ceilings, tool permissions, and required review. A model recommendation is evidence, not authority.
Bind model, effort, budget, and verifier in one manifest
A routing file should be versioned beside the evaluation set. It needs stable task identifiers, allowed tools, context ceiling, model and effort, attempt budget, verifier, escalation target, and rollback. Avoid a single cheap_model setting that silently affects every workflow.
routes:
extract_invoice_fields:
model: claude-haiku-5-5
effort: medium
max_prompt_tokens: 80000
max_attempts: 1
tools: [read_document]
verifier: reconcile_invoice_schema_v4
escalate_to: claude-sonnet-5-5
human_gate: totals_or_vendor_mismatch
patch_small_test_failure:
model: claude-haiku-5-5
effort: high
max_files_changed: 3
max_diff_lines: 120
max_attempts: 2
tools: [read_repo, edit_repo, test_target]
verifier: tests_lint_and_diff_policy
escalate_to: claude-opus-5-5
human_gate: security_or_public_api_changeThe dispatch function should be boring. It reads measured fields, applies exclusions, and records why a route was selected. It must never hide an overflow behind an automatic fallback, because hidden fallbacks make the cheap route look successful while charging the larger model anyway.
def choose_route(task, policy):
if task.consequence in {"money", "identity", "security", "production_write"}:
return policy.strong_route(task, reason="high_consequence")
if task.prompt_tokens > policy.haiku_prompt_ceiling:
return policy.strong_route(task, reason="context_or_price_step")
route = policy.route_for(task.class_id)
if not route or not task.has_objective_verifier:
return policy.strong_route(task, reason="unadmitted_task")
return route
def close_attempt(attempt, verifier):
result = verifier(attempt.artifacts)
receipt = record_usage_tools_artifacts_and_checks(attempt, result)
if not result.passed or receipt.retry_budget_exhausted:
return escalate(receipt)
return accept(receipt)Every accepted attempt should emit a receipt with route-policy version, model ID, provider, effort, prompt-token band, cache use, output tokens, latency, tools, artifacts, verifier results, retry count, and any human correction. That receipt makes route economics and quality auditable. It also lets a team answer a practical question after a model update: which accepted outcomes depended on this route?
Evaluate routing policy, not isolated model answers
Public model benchmarks are useful for hypothesis generation. Anthropic cites customer tests in which Haiku 5.5 improved document-question accuracy, reduced latency, or performed well on CRM tasks. These are selected first-party examples with different data, prompts, tools, and graders. An independent 26-entry agent evaluation placed Haiku 5.5 below the leading Claude models on its 13-task blind rubric, while an open routing project reports an insufficient-evidence verdict for broad policy changes after a preregistered run. The results do not conflict. They show that route fitness is conditional.
Build a task corpus from real production distributions. Include easy cases, boundary cases, rare formats, adversarial inputs, tool failures, long context, stale context, ambiguous instructions, and tasks that should be refused or escalated. Freeze the corpus before comparing routes. Otherwise a team will keep the wins and quietly relabel the failures.
evaluation_unit = {
"task_id": "repo-edit-017",
"task_class": "patch_small_test_failure",
"input_snapshot": "sha256:...",
"allowed_tools": ["read_repo", "edit_repo", "test_target"],
"expected_checks": ["target_test", "lint", "diff_policy"],
"severity_if_wrong": "medium",
"max_attempts": 2,
"human_reference": "review-2026-10-28"
}Score the full outcome: objective correctness, evidence completeness, unsafe action rate, abstention quality, tool-call validity, attempts, wall-clock latency, input and output tokens, cache charges, escalation rate, human correction time, and post-release defects. Report distributions rather than one average. A route that is very cheap on 90 percent of tasks and catastrophic on the remaining 10 percent is not acceptable for high-consequence work.
Use paired runs where possible. The same input snapshot should reach Haiku and the comparison route under the same tools and verifier. Randomize run order, separate the grader from the candidate route, and keep a deterministic component in the grader. If human review is needed, blind reviewers to the route and measure disagreement.
Count retries, correction, and the long-context step
List price is one term in route cost. The useful denominator is accepted outcomes. Include rejected attempts, prompt construction, cache writes and reads, tool execution, fallback calls, grader calls, human correction, and incident recovery.
cost_per_accepted_outcome = (
candidate_input_cost
+ candidate_output_cost
+ cache_cost
+ tool_cost
+ retry_cost
+ fallback_cost
+ grader_cost
+ human_correction_cost
+ expected_incident_cost
) / accepted_outcomesThe 100,000-token boundary deserves its own metric. A route that retrieves 130,000 tokens for a simple extraction enters Haiku's higher input and output price band. The correct fix may be retrieval discipline, not a model change. Record source bytes, tokenizer count, selected chunks, and cache status so the team can distinguish model economics from context-pipeline waste.
Retries require a stop rule. Repeating the same model, prompt, and cap often buys the same failure twice. Change one controlled variable or escalate. For example, a schema mismatch can receive one repair attempt with the validator errors. A second mismatch escalates with the first attempt, validator output, and source evidence attached. The stronger model should not start blind.
Failure modes a successful demo can hide
| Failure | Why it looks cheap | Control |
|---|
| Default-switch migration | Most requests still return text | Per-task admission and frozen comparison corpus |
| Tokenizer drift | Per-token price fell | Recount representative prompts and reset budgets |
| Long-context cliff | One model handles the whole document | Prompt-band metrics and retrieval ceiling |
| Thinking-only truncation | Request ended without an API error | Block-aware parser and output verifier |
| Retry spiral | Each attempt is individually inexpensive | One repair attempt, then evidence-carrying escalation |
| Silent strong-model fallback | Workflow succeeds | Route receipt names every attempt and fallback cost |
| Easy-task benchmark | Pass rate is high | Boundary, refusal, tool-failure, and consequence cases |
| Worker changes the plan | Agent appears proactive | Separate planner authority from scoped worker authority |
| Account replay failure | Primary account works | Cross-account failover test for stored thinking blocks |
Safety policy is also part of route behavior. A model can decline requests through a refusal stop reason, and Anthropic's transparency material reports improved prompt-injection performance over Haiku 4.5. That does not remove the need for external controls. Tool authorization, data policy, network limits, side-effect gates, and audit logs belong outside the model. Treat refusal behavior as one measured layer, not the security boundary.
A 30-day routing acceptance rollout
- Days 1-4: inventory every current route, fallback, provider, effort setting, prompt-token band, tool permission, and verifier.
- Days 5-8: migrate request schemas in a test environment; exercise thinking blocks, stop reasons, caching, long context, and account failover.
- Days 9-12: define narrow task classes and hard exclusions; freeze a representative corpus with consequence labels.
- Days 13-17: run paired evaluations against the current route; capture accepted outcome cost, latency, correction, and severity.
- Days 18-21: shadow-route live traffic without using Haiku outputs; inspect classification and escalation errors.
- Days 22-25: canary one low-consequence task class with objective verification, one repair attempt, and automatic rollback.
- Days 26-28: inject tool failures, long context, stale history, ambiguous requests, and verifier disagreement.
- Days 29-30: approve only exact task classes that clear quality, safety, latency, and accepted-outcome cost thresholds.
ContractTask class, consequence, model, effort, tools, prompt ceiling, attempts, verifier, escalation, rollback.
MigrationRequest schema, content-block parser, stop reasons, tokenizer, cache, thinking history, account replay.
CorpusRepresentative volume, boundary cases, hard negatives, failures, refusals, and consequence labels.
ReceiptPolicy version, usage, latency, tools, artifacts, checks, retries, fallback, correction.
EconomicsCost per accepted outcome, not price per nominal call.
ReleaseShadow mode, canary, drift thresholds, named owner, rollback, and re-evaluation trigger.
Re-evaluate when the model alias, provider implementation, effort defaults, system prompt, tokenizer, pricing, tool versions, route policy, or task distribution changes. A route is not approved forever. It is approved for a versioned operating envelope.
FAQ
What is the best first use for Claude Haiku 5.5?
Start with a frequent, bounded task that has supplied evidence and an objective checker: classification, schema extraction, log summarization, codebase search, or a small edit covered by tests. Avoid starting with open-ended architecture, security changes, or irreversible actions.
Should Haiku 5.5 be the default subagent?
Only for task classes your evaluation has admitted. “Subagent” describes topology, not difficulty. A scout that locates files may be a good fit; a subagent asked to redesign authorization may not be.
Why can the same text cost more tokens than before?
Anthropic documents a newer tokenizer that can count the same text as about 30 percent more tokens than Haiku 4.5. The exact change depends on content, so recount your own prompts instead of applying a universal multiplier.
How many retries should a small model get?
Use a bounded rule tied to failure type. One repair attempt with new validator evidence is reasonable for many structured tasks. Repeating an unchanged request is usually waste. Escalate with the attempt evidence attached.
Do first-party customer benchmarks prove production readiness?
No. They show plausible use cases and generate hypotheses. Production approval requires your data, task distribution, tools, prompts, effort settings, consequence model, verifier, and correction costs.
Sources and further reading
Current model, pricing, migration, safety, repository, and community information was verified online on November 3, 2026. Prices and model behavior can change; confirm the current official documentation before implementation.