4.5 Batch Processing Strategies

What You Need to Know

The Message Batches API is a cost optimisation tool with hard constraints that the exam tests directly. Understanding when to use it — and when not to — is the core of this task statement.

Message Batches API: The Facts

The constraints are fixed, and you have to design around them:

50% cost savings compared to synchronous API calls
Up to 24-hour processing window — results may arrive in minutes or take up to 24 hours
No guaranteed latency SLA — you cannot rely on results arriving within any specific timeframe
No multi-turn tool calling within a single batch request — the model cannot execute tools mid-request and use the results to continue processing
custom_id fields for correlating request/response pairs — each request in a batch gets a unique identifier used to match it with its response

The Matching Rule

This is the single most tested concept from this task statement:

Synchronous API: For blocking workflows where someone or something is waiting for the result. Pre-merge checks in CI/CD, real-time code review feedback, any workflow where developers are blocked pending completion.

Batch API: For latency-tolerant workflows where results are consumed later. Overnight technical debt reports, weekly code audit summaries, nightly test generation runs, batch document extraction.

The exam specifically presents a scenario (Question 11 in the sample questions) where a manager proposes switching everything to batch processing for the cost savings. The correct answer keeps blocking workflows synchronous and only moves latency-tolerant workflows to batch.

// Synchronous — developer is waiting for this
const preMergeReview = await client.messages.create({
  model: "claude-sonnet-5",
  max_tokens: 4096,
  messages: [{ role: "user", content: prDiffContent }]
});

// Batch — results consumed tomorrow morning
const batchRequest = await client.messages.batches.create({
  requests: technicalDebtDocuments.map((doc, i) => ({
    custom_id: `debt-report-${i}`,
    params: {
      model: "claude-sonnet-5",
      max_tokens: 4096,
      messages: [{ role: "user", content: doc }]
    }
  }))
});

SLA Calculation

When designing batch processing schedules, you must account for the 24-hour maximum processing window. If your organisation requires a 30-hour SLA for a report:

The 24 hours is a maximum window, not a delivery guarantee. A batch that does not finish inside it comes back expired, so size the schedule against that worst case and treat an expired batch as a resubmission
30 hours total SLA minus the 24-hour worst case = 6 hours of buffer for collecting requests, validating inputs, or absorbing operational delays
Submit batches every 4 hours within that buffer window so a fresh batch is always in flight. A 6-hour cadence leaves no margin at all

The exam may present a scheduling question where you need to work backwards from the SLA to determine submission frequency.

Batch Failure Handling

Not all documents in a batch succeed. The correct failure handling pattern has three steps:

1. Identify failures by custom_id. Each request has a unique identifier. Parse the batch results to find which custom_id values failed.

2. Resubmit only failures with modifications. Do not resubmit the entire batch. Common modifications include:

Chunking oversized documents that exceeded context limits
Simplifying extraction prompts for documents with unusual structures
Adding format-specific few-shot examples for documents that failed due to structural variety

3. Refine prompts on a sample set BEFORE batch processing. This is the proactive step that maximises first-pass success and reduces resubmission costs. Test your prompts against a representative sample (5-10 documents covering the range of formats and edge cases) before processing the full batch.

// Poll until the batch has finished before reading results
let batch = await client.messages.batches.retrieve(batchId);
while (batch.processing_status !== "ended") {
  await new Promise(r => setTimeout(r, 60_000));
  batch = await client.messages.batches.retrieve(batchId);
}

// results() returns a JSONL async iterable, not an array — accumulate it.
// Treat `expired` as a failure too: that is what an overrun batch returns.
const failedIds: string[] = [];
for await (const result of client.messages.batches.results(batchId)) {
  if (result.result.type === "errored" || result.result.type === "expired") {
    failedIds.push(result.custom_id);
  }
}

// Resubmit only failures with modifications
const retryRequests = failedIds.map(id => {
  const originalDoc = documentsById[id];
  return {
    custom_id: `${id}-retry-1`,
    params: {
      model: "claude-sonnet-5",
      max_tokens: 8192,  // increased for oversized docs
      messages: [{
        role: "user",
        content: chunkIfNeeded(originalDoc)
      }]
    }
  };
});

Multi-Turn Tool Calling Limitation

The batch API doesn’t support multi-turn tool calling within a single request. This means you cannot:

Define tools and have the model call them mid-request
Process tool results and continue the conversation within the same batch item
Run agentic loops within a single batch request

If your workflow requires tool execution mid-processing, you must use the synchronous API. This limitation is a direct exam test point — if a scenario describes a batch workflow that needs to call external tools during processing, the correct answer is to use the synchronous API for that step.

Exam guide v1.0 states the limitation exactly as above, and that is the keyed answer. Anthropic’s batch processing docs (checked September 2026) now split it by tool type. Server tools (web search, web fetch, code execution, MCP connectors, tool search) run the same server-side agentic loop inside a batch request as they do synchronously, and a batch result can come back with stop_reason: "pause_turn" for you to continue in a follow-up request. Client tools, the ones your own code executes, still cannot complete a loop inside a batch item: the request ends at tool_use, you run the tool after retrieving the result, then submit a follow-up request. When a question names a tool your own code runs (an internal lookup, a database query), the guide’s rule and the current docs give the same answer. On the exam, answer per the guide: no multi-turn tool calling in a batch request, so a step that needs tool execution mid-processing runs on the synchronous API.

Prompt Optimisation Before Batch Submission

The most cost-effective batch processing strategy is to invest time in prompt refinement before submitting large volumes:

Sample set testing: Take 5-10 representative documents covering the range of formats, edge cases, and document types in your batch
Iterate on the sample: Refine your extraction prompts, add few-shot examples, adjust schema design until the sample set achieves high accuracy
Submit the full batch: With refined prompts, your first-pass success rate will be significantly higher
Handle failures: Resubmit only the failed documents with targeted modifications

This workflow slashes total cost. A 90% first-pass success rate on 1,000 documents means only 100 retries. A 60% first-pass rate means 400 retries, four times the resubmission cost, plus the batch processing cost for those retries.

Exam Traps

Practice Scenario

Your team wants to reduce API costs for automated analysis. You have two workflows: (1) a blocking pre-merge check that must complete before developers merge, and (2) a technical debt report generated overnight for review the next morning. Your manager proposes switching both to the Message Batches API for 50% cost savings. How should you evaluate this proposal?

A. Switch both to batch processing with status polling to check for completion
B. Use batch processing for the technical debt reports only; keep real-time calls for pre-merge checks
C. Keep real-time calls for both workflows to avoid batch result ordering issues
D. Switch both workflows to batch processing, with a timeout fallback to real-time if the batch takes too long

Answer & explanation

Correct: B

A — Status polling does not change the fundamental constraint: the batch API has no guaranteed latency SLA. Pre-merge checks cannot depend on a 24-hour processing window regardless of polling strategy.
B — Pre-merge checks are blocking workflows — developers wait for results. The 24-hour batch processing window is unacceptable. Technical debt reports are overnight and latency-tolerant, making them ideal for batch processing at 50% savings.
C — Batch results are correlated using custom_id fields, so ordering is not an issue. The real concern is latency requirements, which this answer misidentifies.
D — This adds unnecessary complexity. The simpler and correct approach is to match each workflow to the appropriate API based on its latency requirements.

Sources

Claude Certified Architect Foundations Exam Guide — Task Statement 4.5 — Anthropic
Message Batches API — Anthropic
Building with Claude API (Skilljar) — Anthropic

Exam Simulator

Six exam-style multiple-choice questions on Batch Processing Strategies. Pick an answer, then open the explanation.

Question 1

Your manager proposes switching both your blocking pre-merge code review and your overnight technical debt report to the Message Batches API for 50% cost savings. How should you evaluate this proposal?

A. Switch both to batch processing with status polling to check for completion
B. Keep real-time calls for both workflows to avoid batch result ordering issues
C. Use batch processing for the technical debt reports only, keeping real-time calls for pre-merge checks
D. Switch both to batch processing with a timeout fallback to real-time if the batch takes too long

Answer & explanation

Correct: C

A is wrong because status polling does not change the fundamental constraint: the batch API has no guaranteed latency SLA. Pre-merge checks cannot depend on a 24-hour processing window.
B is wrong because batch results are correlated using custom_id fields, so ordering is not an issue. The real concern is latency requirements.
C is correct because pre-merge checks are blocking workflows where developers wait. The 24-hour window is unacceptable. Technical debt reports are overnight and latency-tolerant — ideal for batch at 50% savings.
D is wrong because this adds unnecessary complexity. Match each workflow to the appropriate API based on latency requirements.

Question 2

Your organisation requires a weekly code audit report delivered by Monday 09:00. You want to use the Message Batches API. When is the latest you can submit the batch?

A. Sunday 09:00 (24 hours before the deadline)
B. Friday 09:00 (72 hours before, to be safe)
C. Monday 06:00 (3 hours before the deadline, since batches usually complete quickly)
D. Sunday 03:00, which is 30 hours before the deadline and inside the batch window

Answer & explanation

Correct: D

A is wrong because submitting exactly 24 hours before leaves zero buffer. If the batch takes the full 24 hours, you hit the deadline with no margin.
B is wrong because 72 hours is unnecessarily conservative. A 6-hour buffer beyond the 24-hour maximum is adequate.
C is wrong because “usually completes quickly” is not a guarantee. The batch API has no latency SLA. Designing around best-case timing is unreliable.
D is correct because you must account for the worst case: full 24-hour processing. 30 hours before the deadline provides a 6-hour buffer for any issues.

Question 3

A batch of 500 documents completes with 450 successes and 50 failures. What is the correct failure handling approach?

A. Identify the 50 failures by custom_id, apply targeted modifications, and resubmit only those 50
B. Resubmit the entire batch of 500 documents to ensure consistency
C. Discard the failures and report only the 450 successful extractions
D. Reduce the batch size to 50 documents and reprocess everything in smaller batches

Answer & explanation

Correct: A

A is correct because custom_id fields identify which documents failed. Targeted modifications (chunking, simplified prompts, format-specific examples) address the specific failure causes.
B is wrong because resubmitting all 500 wastes cost on the 450 already-successful documents and doubles processing expense.
C is wrong because discarding 10% of results reduces data completeness. Many failures are fixable with targeted retry.
D is wrong because batch size is not the issue. Targeted retry of failures is more efficient than reprocessing everything.

Question 4

Your extraction workflow requires Claude to call a tool mid-request, use the tool results, and continue processing within the same request. Which API should you use?

A. Message Batches API with tool definitions included in the batch request
B. Synchronous Messages API, because it supports multi-turn tool calling
C. Message Batches API with a webhook to handle tool calls asynchronously
D. Either API, since tool calling works the same in both

Answer & explanation

Correct: B

A is wrong because the batch API does not support multi-turn tool calling within a single request, even with tool definitions.
B is correct because the batch API cannot execute tools mid-request and use results to continue. Multi-turn tool calling requires the synchronous API.
C is wrong because webhooks cannot inject tool results back into an in-progress batch request.
D is wrong because tool calling behaviour differs between the APIs. The batch API lacks multi-turn tool calling.

Question 5

Before submitting a batch of 1,000 documents, your colleague suggests testing prompts on a 5-document sample first. Why is this the correct approach?

A. To refine prompts and maximise first-pass success rate, reducing total cost from resubmissions
B. To verify the API connection and authentication before a large submission
C. To estimate processing time so you can plan the batch submission schedule
D. To check that the model supports the document format before committing to full processing

Answer & explanation

Correct: A

A is correct because prompt refinement on a representative sample maximises first-pass success. A 90% success rate means 100 retries. A 60% rate means 400 retries — four times the resubmission cost.
B is wrong because API connection testing is a basic operational check, not the primary purpose of sample testing.
C is wrong because batch processing has no guaranteed latency SLA, so you cannot estimate processing time from a sample.
D is wrong because format support is a secondary concern. The primary benefit is prompt quality improvement before large-scale submission.

Question 6

Which of the following is NOT a valid use case for the Message Batches API?

A. Nightly generation of test cases from code documentation
B. Weekly audit summary of code review findings
C. Pre-merge check a developer waits on before merging
D. Overnight extraction of financial data from 2,000 invoices

Answer & explanation

Correct: C

A is wrong because nightly test generation is latency-tolerant — results consumed next business day. Valid batch use case.
B is wrong because weekly audit summaries have no real-time dependency. Valid batch use case.
C is correct because pre-merge checks are blocking workflows. Developers wait for the result. The 24-hour batch processing window is unacceptable for blocking workflows.
D is wrong because overnight extraction is consumed the following morning. Valid batch use case.