Quick Reference

Condensed cheat sheets for the five exam domains, crawled from the Quick Reference sheets of the source guide. Review these after finishing the lessons in a domain — they are summaries, not substitutes for the full content.

Domain 1 — Agentic Architecture & Orchestration

The Agentic Loop

The core execution cycle: Send request → Inspect stop_reason → Execute tools or terminate.

• stop_reason: "tool_use" — Claude wants to call tools. Continue the loop.
• stop_reason: "end_turn" — Claude is finished. Terminate the loop.
• Tool results must be appended to conversation history before the next iteration. Without this, Claude cannot reason about what the tool returned.
• The stop_reason field is the reliable termination signal. It is deterministic and unambiguous, unlike parsing the response text.

Current state (checked 14 August 2026): the exam keys on end_turn and tool_use. The live Messages API also returns pause_turn, max_tokens, stop_sequence, refusal and model_context_window_exceeded, so a production loop handles more than the two exam values.

Three anti-patterns to recognise on sight:

Anti-Pattern	Why It Fails
Parsing natural language (“I’m done”)	Ambiguous — Claude may say “finished step 1” while intending to continue
Iteration caps as primary control	Either cuts off useful work or runs unnecessary iterations
Checking content[0].type == "text"	Claude can return text alongside tool_use blocks in the same response

Iteration caps are acceptable only as a safety net (maximum bound to prevent runaway agents), never as the primary stopping mechanism.

Orchestration Patterns

Pattern	When to Use	Key Characteristic
Sequential	Steps depend on previous output	A → B → C, each step gets prior result
Parallel	Independent subtasks, latency matters	Fan-out, fan-in; tasks share no state
Pipeline	Stages with different specialisations	Assembly line; output of one is input to next
Dynamic Adaptive	Task complexity unknown upfront	Model decides decomposition at runtime
Hub-and-Spoke	Coordinator + specialist pattern	Central agent delegates to focused subagents

Decision rule: Use parallel when subtasks are independent. Use sequential when each step needs the previous result. Use dynamic adaptive when you cannot predict the decomposition at design time.

Guardrails Hierarchy

Prompt instructions are probabilistic. Hooks are deterministic. This distinction is the single most tested concept in this domain.

Mechanism	Type	Enforcement	Use For
System prompt rules	Probabilistic	Model may not comply	Style guidance, soft preferences
PreToolUse hooks	Deterministic	Code-level, pre-execution	Block dangerous tool calls, validate parameters
PostToolUse hooks	Deterministic	Code-level, post-execution	Validate outputs, sanitise results, audit logging

Hook execution order: PreToolUse fires before the tool runs. PostToolUse fires after. Both are programmatic — they cannot be bypassed by prompt injection.

When the exam says “must always” or “guaranteed” — the answer is hooks, not prompts.

Claude Agent SDK

• AgentDefinition: Declares an agent’s description, system prompt, and its tools field — the list that scopes which tools the subagent can access. Scope it to a small, role-focused set; the guide’s illustration contrasts 4–5 tools with 18, where selection reliability degrades.
• Task tool: Used by a coordinator to delegate work to a subagent; the coordinator’s allowedTools must include "Task" (or "Agent", its current name in Claude Code) or it cannot spawn subagents at all. The subagent runs in its own context.

Key constraint: Subagents do not share memory. All context must be passed explicitly through task definitions or handoff payloads.

Multi-Agent Systems

Coordinator + Specialist pattern:

• Coordinator handles task decomposition, sequencing, and result aggregation.
• Specialists are scoped to a single domain, with a small tool set each.
• Context passing must be explicit — no shared memory, no implicit state.

Why specialists over one large agent:

• Focused tool sets reduce selection errors.
• Smaller context windows per agent improve accuracy.
• Independent scaling and testing of each specialist.

Task decomposition: The coordinator decides at runtime how to split work. Fixed decomposition suits predictable tasks; dynamic decomposition suits open-ended exploration.

Human-in-the-Loop

Structured handoff format (the exam tests this specific pattern):

1. Customer ID — Who is affected
2. Summary — What happened (factual, not interpretive)
3. Root cause — Why it happened
4. Recommended action — What the agent suggests

When to escalate: Policy exceptions, destructive operations, and genuine ambiguity that cannot be resolved with available context. Note the distinction the exam draws: a classifier score below a per-category threshold is a valid trigger, but an agent’s self-reported confidence is not — the guide rejects it as poorly calibrated.

Key rule: Never silently fail. If the agent cannot complete a task, it must produce a structured handoff, not a generic error message.

Error Recovery & Resilience

Strategy	When to Use
fork_session	Divergent exploration — try multiple approaches without polluting main context
Fresh start + summary injection	Context has become stale or polluted; start new conversation with extracted facts
Retry with error feedback	Transient failure; send original + failed output + specific error back to model
Graceful degradation	Partial results are better than no results; return what you have with metadata

Stale context signals: Model repeats itself, contradicts earlier statements, or ignores recent tool results. The fix is not “more context” — it’s a fresh start with a curated summary.

Decision Rules for the Exam

If the question says…	The answer is likely…
“guaranteed”, “must always”, “enforce”	Hooks (deterministic), not prompts
“flexibility”, “adapt”, “unexpected”	Model-driven decision-making
“independent subtasks”	Parallel orchestration
“each step needs previous output”	Sequential orchestration
“premature termination”	Check stop_reason, not iteration caps
“runaway agent”	Iteration cap as safety net
“share context between agents”	Explicit passing (subagents have no shared memory)
“complex task, unknown structure”	Dynamic adaptive orchestration
“compliance”, “regulatory”, “audit”	Programmatic enforcement (hooks), not model judgment

Common Exam Traps

Trap	Correct Answer
“Increase iteration cap to fix premature termination”	Wrong — fix the stop_reason check
“Subagents can read the coordinator’s context”	Wrong — all context must be passed explicitly
“System prompt rules guarantee compliance”	Wrong — prompts are probabilistic; hooks guarantee
“Use one agent with many tools for simplicity”	Wrong — scope each agent to a small, role-focused tool set
“Iteration caps are the primary loop control”	Wrong — stop_reason is primary; caps are safety nets
“Text content in response means agent is done”	Wrong — text can appear alongside tool_use blocks

Domain 2 — Tool Design & MCP Integration

Tool Description Design

Tool descriptions are the primary mechanism Claude uses to select which tool to call. They matter more than tool names.

What to include in a tool description:

• What the tool does (one sentence)
• Expected input formats and constraints
• What it returns (shape of the response)
• Boundary conditions (what it does NOT do)
• Example queries that would trigger this tool

Count the tools before you pick the remedy. Misselection has two different causes and they take opposite fixes:

The tools are…	The fix
Few enough to reason about, but two read alike	Sharpen the descriptions (TS 2.1)
Different jobs (query, transform, export)	Split by role into focused agents (TS 2.3)
Variations on one job, sharing a shape	Consolidate into one parameterised tool (TS 2.3)

Descriptions are the first fix only when the toolkit is already a workable size. Past roughly 4–5 tools per agent, selection degrades on decision complexity alone, and rewriting 22 descriptions leaves that untouched. Few-shot examples are not a sanctioned step for misselection — the bank keys them wrong.

Schema Design Rules

• Keep each agent’s tool set small and role-focused. The guide contrasts 4–5 tools with 18 to show how selection reliability degrades as the set grows.
• Use descriptive parameter names — customer_email not email, order_date_range not range.
• Mark parameters as required only when truly mandatory. Optional parameters with defaults reduce friction.
• Use enum types for constrained choices — they guide the model better than freeform strings.
• Diagnose before you fix. A small toolkit with two lookalike tools is a description problem. An oversized toolkit is a distribution problem, and no amount of description quality rescues it.

tool_choice Modes

Mode	Behaviour	Use When
auto	Model decides whether to call a tool	Default for most agentic loops
any	Model must call at least one tool (chooses which)	Guaranteed structured output when the input could match one of several schemas
tool (forced)	Model must call a specific named tool	Guaranteed schema compliance for one known structure

Key exam point: Use tool_choice: { type: "tool", name: "extract_data" } when you need one specific structure every time. When the document type is unknown and any of several extraction schemas could apply, tool_choice: "any" still guarantees structured output while letting the model pick the right tool.

auto is the correct default for agentic loops — the model needs freedom to decide when to call tools and when to respond with text.

MCP Architecture

Three-layer model: Host ⊃ Client ↔ Server (the host application contains the client; the client connects to servers)

Layer	Role	Example
Host	The application that manages client lifecycle	Claude Desktop, an IDE extension
Client	The connector inside the host, which connects to one server and routes its tool calls	The client instance the desktop app creates per server
Server	Exposes tools, resources, prompts	A database connector, file system server

Protocol: JSON-RPC 2.0 over stdio or streamable HTTP.

Configuration files:

• .mcp.json in project root — project-level MCP servers (shared with team)
• ~/.claude.json — personal/global MCP servers (not committed)

Key rule: Use community MCP servers first. Only build custom servers when no community server meets your requirements.

Tool Error Handling

Structured error metadata (the exam-tested pattern). This is an application-level convention carried inside the result content, not part of the MCP envelope — the protocol’s CallToolResult defines only content, structuredContent and isError:

{
  "errorCategory": "transient" | "validation" | "business" | "permission",
  "isRetryable": true | false,
  "description": "Rate limit hit; retry after 5 seconds"
}

The four categories: transient (timeouts, service unavailability — isRetryable: true, resend as-is), validation (invalid input — false, correct the input and send a new call), business (policy violations — false, take an alternative path), permission (missing access — false, escalate to a principal with access). Only transient is retryable: the flag asks whether resending this call can work, and everything else needs something to change first. “Not found” is deliberately absent: a query that finds nothing is a valid empty result, not an error.

Critical distinction:

• Access failure (auth error, network timeout) → Retry or escalate. Something went wrong.
• Valid empty result (search returned 0 results) → Accept. The absence of data IS the answer.

Never treat a valid empty result as an error. Never silently swallow an access failure.

Tool Selection in Claude Code

Tool	Purpose	Use When
Grep	Search file contents by pattern	Looking for code patterns, string occurrences
Glob	Find files by name/path pattern	Looking for files by extension or naming convention
Read	Read a specific file	You know the exact file path
Write	Write a complete file	Creating a file, or replacing one wholesale
Edit	Modify file contents	Making targeted changes to existing files
Bash	Run shell commands	Build, test, git operations, anything not covered above

Selection principle: Use the most specific tool. Grep for content search, Glob for file discovery, Read for known files. Avoid Bash for tasks that specialised tools handle better.

When Edit fails on a non-unique match: the exam guide names Read + Write as the tested fallback — load the full file, then write the complete modified version.

Current state (checked 14 August 2026): the live tools reference describes Claude first supplying a longer anchor string with enough surrounding context to pin down one occurrence, or setting replace_all: true. Answer Read + Write on the exam; recognise the widening and replace_all behaviour in current Claude Code.

Decision Rules for the Exam

If the question says…	The answer is likely…
“Claude keeps picking the wrong tool” (small toolkit)	Improve tool descriptions
“Claude keeps picking the wrong tool” (20+ tools)	Split by role or consolidate — not descriptions
“guaranteed structured output”	Forced tool_choice for one known schema; tool_choice: any across multiple schemas
“model should decide which tool”	tool_choice: auto
“must call a tool but can choose which”	tool_choice: any
“search returned no results”	Valid empty result — accept it
“API returned 401/timeout”	Access failure — retry or escalate
“too many tools, selection errors”	Split into role-scoped agents, or consolidate variants of one job
“need a custom MCP server”	Check community servers first
“project-wide MCP config”	.mcp.json in project root
“personal MCP config”	~/.claude.json

Common Exam Traps

Trap	Correct Answer
“Improve all 22 tool descriptions to fix misselection”	Wrong — past ~4–5 tools the problem is decision complexity, not wording
“Add few-shot examples to fix misselection”	Wrong — not a sanctioned remedy; fix the descriptions or the distribution
“tool_choice: any guarantees a specific tool”	Wrong — any forces a tool call, not a specific one
“MCP servers connect directly to each other”	Wrong — all communication goes through the client/host
“Empty search results mean the tool failed”	Wrong — absence of data is a valid result
“Return generic error string from tools”	Wrong — return structured metadata (category, retryable, suggestion)
“Build a custom MCP server for common integrations”	Wrong — check community servers first
“Tool name is the primary selection signal”	Wrong — tool description is the primary signal

Domain 3 — Claude Code Configuration & Workflows

Configuration Hierarchy

CLAUDE.md files are concatenated into context, not overridden. Every applicable file loads together, in a documented order from broadest scope to most specific:

Load order	Location	Scope	Committed to Git?
1 (broadest)	~/.claude/CLAUDE.md	User-global, all projects	No
2	.claude/CLAUDE.md (project root)	Project-wide	Yes
3	CLAUDE.md (any directory)	Directory and below	Yes

Where .claude/rules/ sits: not at the end of that chain. Rules without paths frontmatter load at launch at the same priority as .claude/CLAUDE.md — they are a way to split a monolithic file, not a more-specific layer that wins. Rules with paths load conditionally, when Claude reads a file matching the pattern. Separately, user-level rules in ~/.claude/rules/ load before project rules, which gives project rules the higher priority.

Key rule: This is a load order, not a precedence chain — no file replaces another. If two rules contradict each other, Claude may pick one arbitrarily, so fix contradictions at the source. Distractors claiming “more specific scope wins” or “user-level overrides project-level” are wrong.

.claude/rules/ — Conditional Rules

Files in .claude/rules/ use YAML frontmatter with a paths field for conditional loading:

---
paths:
  - "src/api/**"
  - "src/middleware/**"
---
Always validate authentication tokens before processing API requests.
Use structured error responses with proper HTTP status codes.

Rules are loaded only when Claude Code operates on files matching the glob patterns. This prevents irrelevant rules from consuming context window space.

Skills System

Property	Project Skills	Personal Skills
Location	.claude/skills/	~/.claude/skills/
Entry point	SKILL.md in skill directory	SKILL.md in skill directory
Shared with team	Yes (committed)	No (personal)

Skill properties:

• allowed-tools — The exam guide describes this as restricting the skill’s tool access, and that is the expected exam answer. In current Claude Code (July 2026) it pre-approves the listed tools so they run without a permission prompt; disallowed-tools and permission deny rules do the restricting
• context: fork — Runs in a forked context so skill execution does not pollute the main conversation

Skills are reusable capability modules. They encapsulate a workflow (e.g., “run tests”, “deploy to staging”) with tool access configured in frontmatter.

Commands

Type	Location	Scope
Project commands	.claude/commands/	Shared with team
Personal commands	~/.claude/commands/	Personal only

Commands are invoked with / prefix. They are templates — predefined prompts that can include $ARGUMENTS placeholders for user input.

Difference from skills: Commands and skills have been merged into one system — both create /commands. A command is a single flat .md file (a prompt template, $ARGUMENTS supported); a skill is a directory with a SKILL.md entrypoint that adds supporting files, frontmatter configuration, and automatic invocation when relevant.

Hooks — Deterministic Enforcement

Hook	Fires When	Use For
PreToolUse	Before a tool executes	Block dangerous calls, validate parameters, require confirmation
PostToolUse	After a tool returns	Validate output, sanitise results, audit logging, trigger side effects

Critical property: Hooks are deterministic — they run as code, not as model instructions. They cannot be bypassed by prompt injection or model reasoning.

Contrast with prompt instructions:

• Prompt: “Never delete production files” → Probabilistic, may be violated
• PreToolUse hook blocking rm on /prod/ paths → Deterministic, cannot be violated

Working Modes

Mode	When to Use
Plan mode	Complex tasks, multiple possible approaches, need to think before acting
Direct execution	Clear scope, well-defined task, no ambiguity about approach
-p flag (non-interactive)	CI/CD pipelines, automated workflows, no human present

Plan mode signals: The task is complex, has multiple valid approaches, or the consequences of a wrong approach are high. Plan mode forces Claude to outline its approach before executing.

-p flag: Runs Claude Code in non-interactive mode. Essential for CI/CD integration. No confirmation prompts, no interactive input — the command must be self-contained.

CLI Flags (Headless & Non-Interactive)

-p is the most-tested flag, but know the rest of the headless toolkit too.

Flag	What it does
-p / --print	Non-interactive (print) mode for CI/CD and piping
-c / --continue	Resume the most recent conversation in this directory
-r / --resume <id|name>	Resume a specific session
--bare	Minimal mode: skips hooks, skills, plugins, MCP, memory, and CLAUDE.md for faster scripted runs
--output-format text|json|stream-json	Output shape for -p (json and stream-json are parseable)
--json-schema '<schema>'	Schema-validated output for -p (lands in the JSON envelope’s structured_output field)
--max-turns <n>	Cap agentic turns in print mode
--permission-mode <mode>	default, acceptEdits, plan, auto, dontAsk, bypassPermissions, manual (alias for default)
--allowedTools / --disallowedTools	Allow or deny tool calls without prompting
--add-dir <path>	Add a readable/editable working directory
--model <alias|name>	Set the session model

System prompt — append vs replace (exam favourite):

Flag	Effect
--append-system-prompt "<text>"	Adds to the default prompt; keeps tool guidance and safety instructions
--system-prompt "<text>"	Replaces the whole default prompt; you own everything it needs

The *-file variants (--append-system-prompt-file, --system-prompt-file) load the same text from a file.

Feedback Techniques

• Concrete examples beat prose descriptions. Show the code you want, not a paragraph describing it.
• Batch interacting fixes in a single message so the model sees all the constraints at once. Fix independent issues sequentially, one focused change at a time.
• Independent review sessions: Never review code in the same session that wrote it. The model retains reasoning bias from the writing session. Start a fresh session for review.
• Severity calibration: Use examples to show what constitutes a critical issue vs. a minor style nit. Without calibration, the model treats all issues as equally important.

Permissions & Security

• Permissions are controlled at the tool level — you grant or deny access to specific tools.
• A subagent’s tool set is scoped with the tools field on its AgentDefinition; skills use allowed-tools frontmatter (see Skills System above).
• Principle of least privilege: Each agent/skill should have access only to the tools it needs.
• Project-level settings (.claude/) are committed and shared. Personal settings (~/.claude/) are not.

Decision Rules for the Exam

If the question says…	The answer is likely…
“guaranteed enforcement”, “cannot be bypassed”	Hooks (PreToolUse/PostToolUse)
“style guidance”, “preferred approach”	Prompt instructions in CLAUDE.md
“applies only to specific file paths”	.claude/rules/ with paths frontmatter
“reusable workflow with restricted tools”	Skills (.claude/skills/)
“prompt template with arguments”	Commands (.claude/commands/)
“CI/CD pipeline”, “automated”, “non-interactive”	-p flag
“extra rules but keep default coding behaviour”	--append-system-prompt (not --system-prompt)
“fast scripted run, skip project config/hooks/skills”	--bare
“machine-parseable / schema-validated CI output”	--output-format json + --json-schema
“review quality of generated code”	Independent session (not the writing session)
“multiple approaches, complex task”	Plan mode first
“personal preference, not shared”	~/.claude/ (user-level config)

Common Exam Traps

Trap	Correct Answer
“Put all rules in the project CLAUDE.md”	Wrong — use .claude/rules/ for path-specific rules
“Hooks are prompt-based guardrails”	Wrong — hooks are deterministic code, not prompt instructions
“Review code in the same session that wrote it”	Wrong — model retains reasoning bias; use independent session
“A flat .md file directly inside .claude/skills/ creates a command”	Wrong — skills are directories with a SKILL.md entrypoint; flat files belong in .claude/commands/
“User CLAUDE.md overrides project CLAUDE.md”	Wrong — CLAUDE.md files are concatenated, none overrides another; contradictions may be resolved arbitrarily
“-p flag enables plan mode”	Wrong — -p enables non-interactive (piped) mode for CI/CD
“--append-system-prompt replaces the system prompt”	Wrong — it appends and keeps the defaults; --system-prompt replaces

Domain 4 — Prompt Engineering & Structured Output

System Prompts

Concrete code examples beat prose descriptions. Instead of writing “use clear variable names”, show a before/after code snippet.

Severity calibration: Without examples of what constitutes “critical” vs. “minor”, the model treats all issues as equally important. Provide 2–3 calibration examples showing severity levels with specific code samples.

Structure a system prompt as:

1. Role and context (who the model is, what it is doing)
2. Rules and constraints (what it must/must not do)
3. Output format specification
4. Calibration examples (severity, tone, detail level)

Key rule: System prompts are the most cost-effective way to control behaviour, because one prompt shapes every request rather than being repeated per call.

Structured Output — tool_use vs Text

Method	Guarantee	Use When
Forced tool_choice ({ type: "tool", name: ... })	Tool call guaranteed; the tool’s JSON schema constrains the shape	You need one specific structure every time
tool_choice: any	A tool call is guaranteed, model picks which tool	Multiple extraction schemas, document type unknown
tool_choice: auto	Model may or may not use the tool	Agentic loops where text responses are also valid
Prompt-based JSON	No schema enforcement	Simple cases, prototyping only

The exam’s preferred pattern: Define a tool whose sole purpose is to structure output (e.g., extract_entities), then force it with tool_choice: { type: "tool", name: "extract_entities" }. This guarantees the response matches your schema.

Why tool_use over prompt-based JSON:

• Schema is validated by the API, not by post-processing
• Eliminates syntax errors: malformed JSON, missing fields, extra fields
• Works with streaming (structured chunks)

The caveat the exam tests most: a strict JSON schema eliminates syntax errors but does not prevent semantic ones — line items that do not sum to the total, a value extracted into the wrong field, a plausible-looking date that is not the one in the document. Options claiming tool_use prevents all extraction errors are wrong.

Schema Design for Structured Output

• Make fields optional or union-typed with null when data may be missing. If a field is required but the data is absent, the model will fabricate a value to satisfy the schema.
• Use enum for constrained choices to prevent freeform values.
• Keep schemas flat where possible — deeply nested schemas increase extraction errors.
• Include description fields on each property — these guide the model on what to extract.

Fabrication prevention rule: If a field might not have a real value, give it a union type — {"type": ["string", "null"]} — and leave it out of the parent object’s required array. That gives the model permission to return null instead of inventing data.

Note the syntax: nullable: true is OpenAPI, not JSON Schema, and required is an array on the parent object rather than a per-property boolean. An input_schema using either form is rejected.

Few-Shot Examples

When to use few-shot: When instructions alone produce inconsistent output. Few-shot examples demonstrate the exact format and reasoning you expect.

Best practices:

• 2–4 examples is the sweet spot. More adds context cost; fewer may not cover edge cases.
• Include reasoning in examples, not just input → output. Show why the answer is what it is.
• Cover edge cases — at least one example should demonstrate boundary behaviour.
• Place examples in the system prompt for caching benefits, or in the user message for per-request variation.

Diminishing returns: Going from 0 to 2 examples has the largest impact. Going from 4 to 8 rarely improves quality and costs more context.

Prompt Chaining

Scope note: the guide places prompt chaining under Task Statement 1.6 (fixed sequential pipelines versus dynamic adaptive decomposition), not Domain 4. It is repeated here because validation between steps is a natural companion to TS 4.4 retry loops, but expect it to be keyed as a Domain 1 construct.

Multi-step pipelines where each step is a focused prompt with a single responsibility.

Pattern: Step 1 (extract) → Step 2 (validate) → Step 3 (format) → Step 4 (synthesise)

Advantages:

• Each step has a smaller, focused prompt — higher accuracy per step
• Intermediate results can be validated before proceeding
• Individual steps can be retried without rerunning the entire chain
• Different models can be used for different steps (cost optimisation)

When to chain vs. single prompt: Chain when the task has distinct phases with different requirements. Use a single prompt when the task is cohesive and the output format is straightforward.

Retry Pattern — retry-with-error-feedback

When output fails validation, send back: original prompt + failed output + specific validation error.

Original: "Extract all dates from this contract"
Failed output: { dates: ["2024-01-15", "next Tuesday"] }
Error: "dates[1] is relative ('next Tuesday'), not absolute. All dates must be ISO 8601 format."

The model sees what it produced, what was wrong, and can correct specifically. Never just say “try again” — always include the specific error.

When to retry vs. escalate:

• Retry: Validation error, format mismatch, missing field — the model can fix it
• Escalate: Repeated failures (>2 retries), confidence below threshold, fundamentally wrong interpretation

Pydantic’s role (Python): parsing enforces the schema; custom validators enforce semantic rules a JSON schema cannot express (sums that must match, ordered dates). Its ValidationError yields per-field messages you feed straight into the retry prompt.

Batch API

Property	Value
Cost saving	50% cheaper than synchronous
Latency	Up to 24 hours (not guaranteed faster)
Use case	Latency-tolerant bulk workloads
Not for	Real-time, interactive, or user-facing requests

Key exam point: Batch API is for throughput and cost, not speed. If the question mentions “real-time” or “user-facing”, Batch API is wrong.

Self-Review Limitation

A model cannot effectively review its own output in the same session. It retains the reasoning that produced the original output and is biased toward confirming it.

Fix: Use a separate model instance (new conversation, no shared history) for review. The reviewing instance sees only the output, not the reasoning that produced it.

For large inputs: Use per-file passes (analyse each file independently) plus a cross-file integration pass (synthesise findings). This is the “attention dilution” mitigation — processing everything in one pass causes the model to miss details.

Decision Rules for the Exam

If the question says…	The answer is likely…
“guaranteed schema compliance”	Forced tool_choice with specific tool; tool_choice: any when the document type is unknown
“output sometimes has wrong format”	Add few-shot examples (2–4)
“model fabricates missing data”	Make schema fields optional/nullable
“validation failed, need to fix”	retry-with-error-feedback (original + failed + error)
“50% cost reduction”, “bulk processing”	Batch API
“real-time”, “user-facing”	NOT Batch API — use synchronous
“inconsistent output quality”	Few-shot examples or prompt chaining
“review its own output”	Separate instance (not same session)
“large document, missing details”	Per-file passes + cross-file integration
“instructions alone aren’t working”	Add few-shot examples

Common Exam Traps

Trap	Correct Answer
“Use prompt-based JSON for production”	Wrong — use forced tool_choice for guaranteed schema
“Just say ‘try again’ on validation failure”	Wrong — include original + failed output + specific error
“Batch API for faster responses”	Wrong — Batch API trades latency for cost savings
“Review output in the same conversation”	Wrong — same-session review is biased; use separate instance
“Add 10+ few-shot examples for best results”	Wrong — 2–4 is the sweet spot; diminishing returns after that
“Required fields prevent fabrication”	Wrong — required fields CAUSE fabrication when data is missing
“One big prompt handles everything”	Wrong — chain prompts when task has distinct phases

Domain 5 — Context Management & Reliability

Progressive Summarisation Trap

The most common mistake: Relying on the model to progressively summarise earlier context as the conversation grows. Each summarisation pass loses specific details — names, numbers, exact quotes, precise timestamps.

The fix: Extract concrete facts into persistent structured blocks that are carried forward verbatim. Don’t ask the model to “summarise what we’ve discussed” — maintain a structured facts section instead:

## Persistent Facts
- Customer: Jane Smith (ID: 12847)
- Order: #ORD-2024-8821, placed 2024-01-15
- Issue: Delivery address incorrect (postcode SW1A 1AA, should be SW1A 2AA)
- Status: Refund approved, reshipment pending

This block is copied forward exactly, not summarised. The model can’t lose these details.

Scratchpad Files

Use scratchpad files to persist findings across context boundaries during multi-step exploration.

When to use: Long-running tasks that may exceed the context window, or tasks where intermediate findings need to survive a context reset.

Pattern:

1. Write findings to a scratchpad file as you discover them
2. At context boundaries, read the scratchpad to restore state
3. The scratchpad is a file on disk — it survives any context reset

Key distinction: Scratchpads are external persistence (files). Conversation history is internal persistence (context window). When the context window fills up, conversation history is lost — scratchpad files are not.

Two TS 5.1 Facts the Guide Names Explicitly

• Lost in the middle. Models reliably process information at the beginning and end of long inputs, but may omit findings from middle sections. Order matters: put what must not be missed at the edges, not buried mid-context.
• The API is stateless. Nothing is remembered between calls. Every request must carry the complete history it needs, which is exactly why tool results accumulate and why the window fills.

Error Propagation

Structured error context must include:

Field	Purpose	Example
Failure type	What category of failure	"network_timeout", "validation_error"
What was attempted	The query, parameters and target system	"Searched academic DB for 'renewable energy policy', 2022-2024"
Partial results	What was successfully retrieved before the failure	{ "orders": [...], "payments": null }
Potential alternatives	Approaches the coordinator could try next	"Try the mirror index, or broaden the date range"

Note the last two carefully: the guide asks for potential alternative approaches the coordinator can choose from, not a log of recovery already attempted. “Retried twice” belongs under what was attempted.

Never propagate generic error messages. “Something went wrong” gives the next agent (or human) nothing to work with. Structured error context enables intelligent recovery.

Valid Empty Results vs Access Failures

Situation	Type	Correct Action
Search returns 0 results	Valid empty	Accept — absence of data is the answer
API returns 401 Unauthorised	Access failure	Retry with correct credentials or escalate
Database query returns empty set	Valid empty	Accept — the record does not exist
Network timeout	Access failure	Retry or use fallback
Filter matches no items	Valid empty	Accept — refine the filter or report no matches
Rate limit (429)	Access failure	Wait and retry (respect Retry-After)

The exam tests this distinction repeatedly. If the tool executed successfully and returned nothing, that’s a valid result. If the tool failed to execute, that’s an error requiring recovery.

Confidence Calibration & Stratified Validation

Aggregate metrics mask per-type issues. An overall accuracy of 95% can hide the fact that one category has 60% accuracy.

Stratified validation pattern:

1. Group extractions by type/category
2. Calculate per-type accuracy
3. Identify categories where the model underperforms
4. Apply targeted fixes (better examples, tighter schema) to weak categories

Stratified random sampling: Select validation samples from each category proportionally. This detects novel error patterns that random sampling across all categories would miss.

Key exam point: When the question mentions “high overall accuracy but users report errors”, the answer is stratified validation — break down accuracy by category to find the weak spots.

Source Attribution

Structured claim-source mappings survive synthesis. Inline links do not.

Approach	Survives Synthesis?	Use When
Structured mapping: { claim: "...", source: "...", url: "...", date: "..." }	Yes	Production systems, multi-step pipelines
Inline links in text	No — lost during summarisation	Quick prototyping only

Conflicting sources: Annotate both with provenance and dates. Never average conflicting claims or silently pick one. Present both with their sources and let the consumer decide.

Claim: "Sonnet's context window is 1M tokens"
  Source A: Anthropic docs (2026-08-14) — 1M tokens
  Source B: Blog post (2024-03-15) — 200K tokens
  Annotation: temporal change, not a conflict — the window grew
  between the two publication dates. Both kept, dated, for the
  consumer to interpret.

Escalation & Ambiguity

The three valid escalation triggers:

• The customer explicitly asks for a human — honour it immediately, without attempting investigation first
• Policy gap or exception — the policy is silent or ambiguous on the request (e.g. competitor price matching when policy only covers own-site adjustments)
• The agent cannot make meaningful progress

Unreliable triggers (exam distractors): sentiment-based escalation and self-reported confidence scores. Neither tracks actual case complexity. A frustrated customer with a resolvable issue gets an acknowledgement and an offer to resolve; escalate only if they reiterate their preference.

Multiple matches: when a tool returns several customer records, ask for additional identifiers. Never select by heuristic.

Policy gap vs violation: a gap (policy silent) means escalate; a violation (policy forbids) means refuse with the policy explanation.

Context Window Strategies

Strategy	When to Use
Persistent fact blocks	Critical details that must survive summarisation
Scratchpad files	Multi-step exploration across context boundaries
Per-file passes + integration	Large codebases — avoid attention dilution
Fresh start + summary injection	Context has become stale or contradictory
Prompt caching	Repeated system prompts across requests (cost + latency savings)

Context window pressure signals: Model starts repeating itself, contradicts earlier statements, or ignores recent information. These indicate the window is too full or the context is stale.

Decision Rules for the Exam

If the question says…	The answer is likely…
“details lost over long conversation”	Persistent fact blocks, not progressive summarisation
“findings need to survive context reset”	Scratchpad files (external persistence)
“tool returned nothing”	Distinguish: valid empty (accept) vs access failure (retry)
“high overall accuracy, users report errors”	Stratified validation — check per-type accuracy
“sources disagree”	Annotate both with provenance and dates
“generic error message”	Replace with structured error context
“model contradicts itself”	Fresh start + summary injection
“inline citations lost after processing”	Switch to structured claim-source mappings
“monitoring shows stable metrics but quality complaints”	Per-type error rates instead of aggregate
“long task exceeds context window”	Scratchpad files + context boundary management

Common Exam Traps

Trap	Correct Answer
“Summarise the conversation to save context”	Wrong — summarisation loses details; use persistent fact blocks
“Overall 95% accuracy means the system is reliable”	Wrong — check per-type accuracy; aggregate masks category issues
“Pick the more recent source when they conflict”	Wrong — annotate both with provenance; let consumer decide
“Return ‘No results found — please try again’”	Wrong if search legitimately found nothing — accept the empty result
“Inline citations in the text are sufficient”	Wrong for production — they are lost during synthesis; use structured mappings
“Monitor overall error rate for quality”	Wrong — monitor per-type error rates to catch category-specific degradation
“Keep all context in the conversation history”	Wrong — use scratchpad files for external persistence across boundaries