4.6 Multi-Instance and Multi-Pass Review

What You Need to Know

When Claude reviews its own output, it starts at a disadvantage: it still carries the reasoning it used to generate that output. The model remembers why it made each decision and is less likely to question it. That’s not a bug. It’s just how self-review works inside a single session. The job is to design around it.

The Self-Review Limitation

A model reviewing its own output in the same conversation session retains its original reasoning chain. It already “knows” why it chose each approach, classified each finding at a particular severity, or selected certain values. When asked to review, it tends to confirm rather than challenge those decisions.

An independent instance — a separate Claude invocation without the prior reasoning context — approaches the output fresh. It judges the code, findings, or extraction on what it sees alone, without the bias of “I chose this because…” That’s what makes independent review so much better at catching subtle issues.

The exam tests this directly. When presented with options for improving review quality, the correct answer involves using a separate model instance, not adding “please review carefully” instructions to the same session or relying on extended thinking within the generating session.

// Anti-pattern: self-review in the same session
const generation = await client.messages.create({
  messages: [
    { role: "user", content: "Write a function to process orders" },
    { role: "assistant", content: generatedCode },
    { role: "user", content: "Now review your code for bugs" }
    // Model retains its reasoning — less likely to find its own mistakes
  ]
});

// Correct: independent review instance
const review = await client.messages.create({
  messages: [
    {
      role: "user",
      content: `Review this code for bugs, security issues, and edge cases:\n\n${generatedCode}`
    }
    // Fresh instance — no prior reasoning context
  ]
});

Multi-Pass Review Architecture

Large reviews (multi-file PRs, complex extraction pipelines, broad code audits) suffer from attention dilution when processed in a single pass. The symptoms are specific and recognisable:

Detailed feedback on some files, superficial comments on others
Obvious bugs missed in the middle of the review
Contradictory findings — flagging a pattern as problematic in one file while approving identical code elsewhere

The fix is to split the review into focused passes:

Pass 1: Per-file local analysis. Analyse each file individually with a focused review prompt. This ensures consistent depth across all files. Each invocation examines only one file, so the model gives it full attention.

Pass 2: Cross-file integration. After all per-file analyses are complete, run a separate pass that receives all per-file findings and checks for cross-file issues: data flow between modules, consistent API usage across services, dependency conflicts, and contradictions in the per-file findings themselves.

// Pass 1: Per-file analysis
const perFileFindings = await Promise.all(
  files.map(file =>
    client.messages.create({
      messages: [{
        role: "user",
        content: `Review this file for local issues (bugs, security, logic errors):\n\n${file.content}`
      }]
    })
  )
);

// Pass 2: Cross-file integration
const integrationReview = await client.messages.create({
  messages: [{
    role: "user",
    content: `Given these per-file findings, identify cross-file issues:\n` +
      `- Data flow inconsistencies between modules\n` +
      `- Contradictory patterns flagged in different files\n` +
      `- API contract violations across service boundaries\n\n` +
      `Findings:\n${JSON.stringify(perFileFindings)}`
  }]
});

This architecture directly addresses the three symptoms of attention dilution. Per-file passes ensure consistent depth. The integration pass catches cross-file issues that no single-file review would identify. And the separation prevents contradictory findings from appearing in the same output.

Why Larger Context Windows Do Not Fix This

The exam includes a specific distractor: “switch to a higher-tier model with a larger context window.” This sounds reasonable — if the model can’t handle 14 files at once, give it more capacity. But the problem isn’t context size. It’s attention quality. A bigger context window won’t stop the model from spreading its attention unevenly across files. Only focused, per-file passes ensure consistent depth.

Confidence-Based Routing

For findings that are uncertain, the model can self-report confidence alongside each finding. This enables a routing strategy:

High confidence findings: Report directly to developers
Low confidence findings: Route to human review for validation
Threshold calibration: Use labelled validation sets to calibrate what confidence score correlates with actual accuracy

{
  "finding": "Potential race condition in order processing",
  "severity": "major",
  "confidence": 0.65,
  "reasoning": "The lock acquisition pattern appears correct but the unlock timing depends on an async callback whose ordering I cannot fully verify.",
  "route": "human_review"
}

The confidence score isn’t self-reported accuracy. It’s the model’s read on its own certainty. Calibrate it by running labelled examples (where you already know the answer) through the system and measuring how reported confidence tracks actual accuracy. Then adjust routing thresholds from that data.

The exam distinguishes between raw confidence scores (uncalibrated, unreliable for automated decisions) and calibrated confidence thresholds (validated against labelled sets, suitable for routing). Using uncalibrated confidence for automated decisions is an anti-pattern.

Putting It All Together

A production review architecture combines all three concepts:

Generation: First instance generates code, extraction, or analysis
Per-file review: Independent instances review each output unit individually
Integration review: Separate instance checks cross-unit consistency
Confidence routing: Low-confidence findings go to human review
Calibration loop: Labelled validation sets continuously calibrate confidence thresholds

This architecture is more expensive than single-pass review. The trade-off is worth it when review quality directly affects production reliability — CI/CD pipelines, financial extraction, compliance analysis, and any system where missed issues have downstream consequences.

Exam Traps

Practice Scenario

A pull request modifying 14 files receives inconsistent review: detailed feedback on some files, superficial comments on others, obvious bugs missed, and contradictory findings — the same pattern is flagged as problematic in one file but approved in another. How should you restructure the review?

A. Switch to a higher-tier model with a much larger context window so that all 14 files receive adequate attention within a single review pass
B. Split into per-file local analysis passes for consistent depth, then run a separate cross-file integration pass for data flow issues
C. Run three independent review passes over the full PR and only flag those issues that at least two of the three separate runs agree on
D. Require developers to split large pull requests into smaller submissions of 3-4 files before the automated review runs

Answer & explanation

Correct: B

A — Larger context windows do not solve attention quality issues. The model can hold more text but still gives uneven attention across files. This is an attention dilution problem, not a context size problem.
B — Per-file analysis ensures every file gets consistent, focused attention. The separate integration pass catches cross-file issues that no single-file review would identify. This directly addresses all three symptoms: inconsistent depth, missed bugs, and contradictory findings.
C — This suppresses real bug detection by requiring consensus on issues that may only be caught intermittently. It trades sensitivity for false consistency.
D — This shifts burden to developers and changes the team workflow without improving the review system itself. The system should handle large PRs through better architecture.

Sources

Claude Certified Architect Foundations Exam Guide — Task Statement 4.6 — Anthropic
Prompt Engineering Overview — Anthropic
Building with Claude API (Skilljar) — Anthropic

Exam Simulator

Six exam-style multiple-choice questions on Multi-Instance and Multi-Pass Review. Pick an answer, then open the explanation.

Question 1

A pull request modifying 14 files receives inconsistent review: detailed feedback on some files, superficial comments on others, obvious bugs missed, and contradictory findings (the same pattern flagged as problematic in one file but approved in another). How should you restructure the review?

A. Switch to a higher-tier model with a larger context window to handle all 14 files in one pass
B. Per-file local analysis passes for consistent depth, then a separate cross-file pass for data flow issues
C. Run three independent review passes on the full PR and only flag issues found in at least two of three runs
D. Require developers to split large PRs into smaller submissions of 3-4 files before automated review

Answer & explanation

Correct: B

A is wrong because larger context windows do not solve attention quality issues. The model can hold more text but still gives uneven attention across files.
B is correct because per-file analysis ensures every file gets consistent, focused attention. The integration pass catches cross-file issues and contradictions.
C is wrong because requiring consensus on issues found in at least 2/3 passes suppresses real bug detection. Some bugs are caught intermittently.
D is wrong because this shifts burden to developers without improving the review system itself.

Question 2

Why is self-review (asking the model to review its own output in the same conversation session) less effective than independent review?

A. The model retains its reasoning context and is less likely to question its own decisions
B. Self-review uses more tokens than independent review, reducing output quality
C. The model forgets its earlier output and cannot compare it accurately
D. Self-review is slower because the model must re-process the entire conversation history

Answer & explanation

Correct: A

A is correct because the model remembers why it made each decision and tends to confirm rather than challenge those decisions. An independent instance evaluates the output fresh.
B is wrong because token usage is not the issue. The reasoning bias from retained context is.
C is wrong because the model does not forget — it retains the full conversation context, which is the actual problem.
D is wrong because speed is not the issue. The quality of review is lower because of reasoning context retention.

Question 3

Your multi-pass code review runs per-file analysis on each of 10 files. What should the cross-file integration pass check for?

A. Grammar and spelling consistency across file comments
B. Whether each file passes compilation independently
C. Data flow issues, contradictions, and API violations
D. Total line count and code complexity metrics across the PR

Answer & explanation

Correct: C

A is wrong because grammar checking is not the purpose of cross-file integration.
B is wrong because compilation checking is a build system concern, not a review integration concern.
C is correct because the integration pass catches systemic issues no single-file review identifies: data flow problems, contradictions in per-file findings, and API contract violations across services.
D is wrong because metrics collection is separate from review quality. The integration pass is about finding cross-file logical issues.

Question 4

Your review system adds confidence scores to each finding. A finding has confidence 0.92 but independent verification reveals it is incorrect. What does this indicate?

A. The model is defective and should be replaced
B. The independent verifier made an error; high-confidence findings are always correct
C. Confidence scores above 0.90 should be automatically accepted without review
D. Raw self-reported confidence is uncalibrated

Answer & explanation

Correct: D

A is wrong because poorly calibrated confidence is a known characteristic, not a defect.
B is wrong because the independent instance approaches the finding fresh and may be more accurate than the self-assessed confidence.
C is wrong because this question directly demonstrates why automatic acceptance of high-confidence findings is risky without calibration.
D is correct because raw self-reported confidence does not reliably correlate with accuracy. Calibration using labelled validation sets reveals the actual relationship and allows you to set reliable routing thresholds.

Question 5

How should you calibrate confidence thresholds for routing findings to human review?

A. Set the threshold at 0.5 since that represents the midpoint between confident and uncertain
B. Run labelled validation sets through it and compare reported confidence against accuracy
C. Ask the model to self-calibrate by reporting its own accuracy statistics
D. Gradually lower the threshold until no findings reach human review, optimising for efficiency

Answer & explanation

Correct: B

A is wrong because an arbitrary midpoint has no empirical basis. The relationship between confidence scores and accuracy varies by task.
B is correct because labelled validation sets provide ground truth. Measuring confidence vs actual accuracy reveals which threshold produces reliable routing.
C is wrong because the model’s self-assessment of its accuracy has the same calibration problem as its confidence scores.
D is wrong because eliminating human review removes the safety net for uncertain findings, reducing overall quality.

Question 6

A production review architecture combines generation, per-file review, integration review, confidence routing, and calibration. This is more expensive than single-pass review. When is the cost justified?

A. Always — quality should never be compromised for cost
B. Only when using the batch API to offset the multi-pass cost with 50% savings
C. Only for codebases with more than 100 files
D. When review quality directly affects production reliability downstream

Answer & explanation

Correct: D

A is wrong because cost-quality trade-offs are necessary. Not every review needs multi-pass architecture.
B is wrong because the batch API addresses cost timing, not review quality. The decision to use multi-pass depends on quality requirements, not API choice.
C is wrong because codebase size is not the determining factor. A 5-file PR in a financial system may warrant multi-pass review.
D is correct because the multi-pass cost is justified when missed issues have real downstream consequences — production systems, financial data, compliance requirements.