5.5 Human Review & Confidence Calibration

What You Need to Know

Human review is the safety net for automated extraction and classification systems. The exam tests your understanding of when and how to deploy human reviewers effectively. The core challenge is not whether to use human review, but how to allocate limited reviewer capacity to maximise accuracy while minimising cost. This requires understanding confidence calibration, the trap of aggregate metrics, and stratified sampling strategies.

The Aggregate Metrics Trap

This is the most dangerous misconception in production extraction systems. A system reports 97% overall accuracy. The team celebrates. Management approves full automation for all high-confidence extractions.

The problem: that 97% hides catastrophic failure rates on specific document types. The system extracts dates from standard invoices at 99.5% accuracy. But handwritten receipts? 60%. Scanned PDFs with poor OCR? 72%. International documents with non-standard formatting? 45%.

The aggregate masks the segments where the system fails most. And those segments are often the ones where errors have the highest business impact — handwritten receipts from field staff, international invoices from new suppliers, scanned historical documents for compliance audits.

The rule: always validate accuracy by document type AND field segment before automating. Never make automation decisions based on aggregate metrics alone.

Document Type	Date Accuracy	Amount Accuracy	Name Accuracy
Standard invoices	99.5%	98.2%	97.8%
Handwritten receipts	60.1%	55.3%	71.2%
Scanned PDFs	72.4%	69.8%	80.1%
International formats	45.2%	52.1%	63.4%
Aggregate	97.0%	96.1%	95.8%

The aggregate looks excellent because standard invoices dominate the volume. But three document types have unacceptable accuracy, hidden by the volume-weighted average.

Stratified Random Sampling

Even after validating by document type and field, you need ongoing verification. Stratified random sampling means selecting a representative sample from each stratum (document type, confidence band, field type) and having humans verify it.

The critical insight is that you must sample high-confidence extractions, not just low-confidence ones. Low-confidence items are already routed to human review. High-confidence items are automated. If the model develops a novel error pattern that affects high-confidence extractions, only stratified sampling will catch it.

Stratified sampling serves two purposes:

Ongoing accuracy measurement — confirm that each segment maintains its validated accuracy rate.
Novel error pattern detection — discover new failure modes that did not exist in the original validation set.

Without stratified sampling, you’re flying blind on your automated extractions. The system could develop a systematic error on a new document format and you wouldn’t know until downstream business processes fail.

Field-Level Confidence Calibration

The model can output confidence scores per field. For an invoice extraction, it might report:

{
  "vendorName": {"value": "Acme Corp", "confidence": 0.98},
  "invoiceDate": {"value": "2024-03-15", "confidence": 0.95},
  "totalAmount": {"value": "$1,247.83", "confidence": 0.72},
  "lineItems": {"value": [...], "confidence": 0.61}
}

But raw model confidence scores are not calibrated. A model that reports 0.95 confidence might actually be correct 88% of the time on certain field types. Or 99% of the time on others. The confidence score is relative, not absolute.

Calibration requires labelled validation sets (ground truth data). You take a set of documents with known correct extractions, run the model, compare its confidence scores to actual accuracy, and build a calibration curve. This tells you: “When the model reports 0.90 confidence on date fields, it’s actually correct 94% of the time. When it reports 0.90 on amount fields, it’s actually correct 82% of the time.”

Calibrated thresholds then drive routing:

Fields above the calibrated threshold → automated (with stratified sampling)
Fields below the calibrated threshold → human review
Fields in the ambiguous zone → prioritised human review

Reviewer Capacity Prioritisation

Human reviewers are expensive and limited. The exam tests whether you understand how to allocate their capacity effectively.

Route the highest-uncertainty items to reviewers first. This means:

Low model confidence fields
Extractions from ambiguous or contradictory source documents
Document types with historically poor accuracy
Fields where the model expresses uncertainty (e.g., multiple possible interpretations)

Do NOT spread reviewer capacity evenly across all extractions. An even distribution wastes time reviewing high-confidence items that the model handles well while leaving insufficient capacity for the uncertain items that actually need human judgement.

The prioritisation should be dynamic, not static. As the system processes documents, the queue of items awaiting human review should be ordered by uncertainty. When a reviewer finishes one item, the next item in their queue should be the highest-uncertainty item remaining, not simply the next in chronological order.

Validation Before Automation

The sequence matters:

Measure accuracy by document type and field segment — not aggregate.
Calibrate confidence scores using labelled validation sets.
Set calibrated thresholds for automation versus human review.
Implement stratified random sampling for ongoing verification of automated extractions.
Only then reduce human review on segments that demonstrate consistent, validated accuracy.

Skipping to step 5 based on aggregate metrics is the trap. Every step in this sequence exists to prevent a specific failure mode.

Exam Traps

Practice Scenario

A structured data extraction system achieves 97% overall accuracy across all document types. The team proposes automating all extractions where model confidence exceeds 95% to reduce human review costs. What is the critical risk in this approach?

A. The 95% confidence threshold is too low for automation and should be raised to 99% before any extractions bypass human review
B. Aggregate accuracy may mask poor performance on specific document types or fields, and confidence scores need calibration against labelled validation sets before use
C. The model will become overconfident as it processes more documents over time, so the system will require regular retraining to stay calibrated
D. Automated extractions should always receive human review regardless of the confidence score, which makes the proposal fundamentally flawed no matter where the threshold is set

Answer & explanation

Correct: B

A — The threshold value is not the core issue — the problem is that aggregate metrics hide per-type performance disparities regardless of where the threshold is set.
B — 97% overall can hide 40% error rates on specific document types. Without stratified validation and confidence calibration, automation will silently fail on certain inputs.
C — LLMs do not train during inference. The issue is existing calibration gaps in different document types, not model drift.
D — Automation is valid when properly validated by document type and field segment. The issue is making the decision based on uncalibrated aggregate metrics, not the concept of automation itself.

Sources

Claude Certified Architect Foundations Exam Guide — Domain 5, Task Statement 5.5 — Anthropic
Anthropic Structured Data Extraction Guide — Anthropic
Anthropic Human-in-the-Loop Patterns — Anthropic

Exam Simulator

Five exam-style multiple-choice questions on Human Review & Confidence Calibration. Pick an answer, then open the explanation.

Question 1

A. The 95% confidence threshold is too low and should be raised to 99% for automation
B. The model will become overconfident over time as it processes more documents, requiring regular retraining
C. Aggregate accuracy can mask poor performance on specific document types and on specific fields alike
D. Automated extractions should always have human review regardless of confidence, making the proposal fundamentally flawed

Answer & explanation

Correct: C

A is wrong: The threshold value is not the core issue. Aggregate metrics hide per-type disparities regardless of where the threshold is set.
B is wrong: LLMs do not train during inference. The issue is existing calibration gaps across document types, not model drift.
C is correct: 97% overall can hide 40% error rates on specific document types. Without stratified validation by document type and field, and confidence calibration using labelled validation sets, automation will silently fail on certain inputs.
D is wrong: Automation is valid when properly validated per document type and field. The issue is making the decision based on uncalibrated aggregate metrics.

Question 2

A team implements a review system that routes all extractions with model confidence below 0.80 to human reviewers. Extractions above 0.80 are fully automated. What is missing from this approach?

A. The 0.80 threshold is too low and should be raised to 0.95
B. Human review of all extractions regardless of confidence for the first month
C. A second model to verify the first model’s extractions before automation
D. Stratified random sampling of high-confidence extractions for novel errors

Answer & explanation

Correct: D

A is wrong: Raising the threshold does not address the blind spot. Without sampling automated extractions, any novel error pattern in high-confidence items goes undetected.
B is wrong: A one-month review period does not provide ongoing detection. Novel error patterns can emerge at any time.
C is wrong: A second model adds cost and complexity without addressing the fundamental need for ongoing human verification of automated outputs.
D is correct: High-confidence items are automated and not reviewed. If the model develops a systematic error affecting high-confidence extractions on certain document types, only stratified sampling catches it. Low-confidence items already get review.

Question 3

A model reports 0.92 confidence on a date extraction from a standard invoice and 0.92 confidence on a date extraction from a handwritten receipt. Should these be treated the same way?

A. No, raw confidence is uncalibrated and varies a lot by document type
B. Yes, the same confidence score indicates the same reliability regardless of document type
C. Yes, but only if both document types were included in the training data
D. No, handwritten receipts should always go to human review regardless of confidence

Answer & explanation

Correct: A

A is correct: Confidence scores must be calibrated using labelled validation sets. The calibration curve will likely show that 0.92 on standard invoices is far more reliable than 0.92 on handwritten receipts.
B is wrong: Raw model confidence is not calibrated. The same score means different things for different document-field combinations.
C is wrong: Training data inclusion does not guarantee calibrated confidence. Calibration against ground truth is always required.
D is wrong: While handwritten receipts may generally need more review, the correct approach is calibrated thresholds, not blanket rules.

Question 4

A review team has 5 reviewers handling 500 extractions per day. Currently, items are assigned in chronological order. How should the queue be restructured?

A. Assign items randomly to ensure fair distribution across document types
B. Order the queue by uncertainty so each reviewer always gets the highest-uncertainty item remaining
C. Assign items by document type so each reviewer specialises in one type
D. Divide items equally among reviewers to balance workload

Answer & explanation

Correct: B

A is wrong: Random assignment wastes reviewer time on high-confidence items that do not need human judgement.
B is correct: Prioritising by uncertainty ensures limited reviewer capacity is used where it adds the most value. When a reviewer finishes one item, the next should be the highest-uncertainty item remaining.
C is wrong: Specialisation may help accuracy but does not address the priority problem of which items need review most urgently.
D is wrong: Equal division is chronological with extra steps. It still wastes capacity on high-confidence items.

Question 5

What is the correct sequence for reducing human review on automated extractions?

A. Measure aggregate accuracy -> Set a confidence threshold -> Automate above threshold -> Sample occasionally
B. Start with full automation -> Add human review only for fields that show errors in production
C. Run the system for 30 days -> Calculate error rate -> Automate if error rate is acceptable
D. Measure by type and field, calibrate, set thresholds, sample, then reduce the review

Answer & explanation

Correct: D

A is wrong: Uses aggregate accuracy (the trap) and uncalibrated thresholds. Occasional sampling is insufficient.
B is wrong: Starting with full automation before validation guarantees undetected errors. Review should precede automation, not follow it.
C is wrong: A 30-day period with aggregate metrics does not reveal per-type problems. Novel error patterns can emerge after 30 days.
D is correct: Each step prevents a specific failure: per-type measurement prevents the aggregate trap, calibration prevents unreliable thresholds, stratified sampling provides ongoing verification, and only validated segments get reduced review.