16 Context Management & Reliability Practice Questions & Answers

1. 1. A support agent is on turn 280 of a shipping-dispute conversation. To fit the context window, the system progressively summarizes older turns. The agent has now 'forgotten' the order number and the refund amount it promised on turn 40. What is the most reliable fix?

   • A.Ask the customer to re-confirm key details whenever they appear to be missing
   • B.Increase the maximum summary length so fewer details are dropped
   • C.Summarize more frequently so each summary covers fewer turns
   • D.Maintain a persistent, structured case-facts block (order ID, amounts promised, commitments) carried forward verbatim and updated, summarizing only the conversational narrative✓ Answer

   Progressive summarization is inherently lossy: each pass can drop or distort details, and losses compound across passes. Extracting durable, high-stakes facts into a structured block that is carried forward verbatim keeps them outside the lossy compression path, so no number of summarization cycles can erase them. Summarizing more often or at greater length still leaves critical facts exposed to compression, and re-asking the customer degrades the experience and admits data loss.

   Source: docs.claude.com Context Windows — context management via summarization plus structured note-taking outside the summarized narrative; CCA-F Exam Guide Domain 5 (context management)Report a problem with this question

2. 2. You build a prompt containing a 60-page warranty policy document plus a customer's question. Which arrangement best mitigates the 'lost in the middle' effect?

   • A.Place the long document at the top and the question plus instructions at the very end of the prompt✓ Answer
   • B.Insert the question in the middle of the document, next to the clauses it concerns
   • C.Ordering does not matter because attention is uniform across the context window
   • D.Split the question into fragments and interleave them every few pages of the document

   Models recall content at the beginning and end of a long context more reliably than content buried in the middle. The recommended long-context pattern is to put bulky reference material first and the query/instructions last, so the model's most recent tokens are the task itself. Placing or fragmenting the question mid-document pushes it into exactly the low-recall region, and attention is empirically not uniform across position.

   Source: docs.claude.com — long context prompting tips: place longform documents near the top and the query at the end (lost-in-the-middle positioning); CCA-F Exam Guide Domain 5Report a problem with this question

3. 3. An agent's order-lookup tool returns a 150 KB JSON payload per call, but only five fields are ever used in the workflow. Over a long support conversation the context fills up quickly. What is the best practice?

   • A.Encode the payload in base64 to reduce the tokens it consumes
   • B.Raise the model's context window setting so the payloads always fit
   • C.Trim the tool output to the relevant fields (or a compact structured summary) before it enters the context✓ Answer
   • D.Keep the full payload in context every time so no information is ever lost

   Verbose tool outputs are one of the fastest ways to exhaust a context window, and irrelevant tokens also dilute the model's attention over the tokens that matter. Filtering or projecting tool results down to the fields the workflow actually uses preserves all decision-relevant information at a fraction of the cost. Base64 encoding increases token count rather than reducing it, and a larger window only delays the problem while raising latency and cost.

   Source: docs.claude.com Context Windows — manage context by trimming/filtering verbose tool results before they enter the window; CCA-F Exam Guide Domain 5Report a problem with this question

4. 4. A team is defining when their support agent must escalate to a human. Which escalation rule is the most reliable?

   • A.Escalate when the customer explicitly asks for a human, or the request falls outside the documented policy coverage✓ Answer
   • B.Escalate when sentiment analysis detects the customer is becoming frustrated
   • C.Escalate when the model reports its own confidence is below 70%
   • D.Escalate whenever the conversation exceeds 15 turns

   Reliable escalation triggers are objective and verifiable: an explicit request for a human is unambiguous, and a policy gap is checkable against the documented policy set. Sentiment scores and a model's self-reported confidence are poorly calibrated proxies — models routinely state high confidence when wrong and low confidence when right — and a fixed turn count is arbitrary, escalating easy long conversations while missing hard short ones.

   Source: CCA-F Exam Guide Domain 5 — escalation design: explicit human requests and policy-coverage gaps as triggers; sentiment and self-reported confidence identified as unreliable proxiesReport a problem with this question

5. 5. In a multi-agent research system, a retrieval subagent returns an empty list. The orchestrator reports 'no matching records exist' — but the subagent had actually received a 403 permission error. What design prevents this failure mode?

   • A.Treat every empty result as 'no data found' to keep the protocol simple
   • B.Have subagents describe any problems in free-form prose for the orchestrator to interpret
   • C.Have the orchestrator retry the subagent until it returns a non-empty result
   • D.Have subagents return a structured result envelope with an explicit status field distinguishing success-with-empty-result from access or tool errors, and make the orchestrator branch on it✓ Answer

   An empty payload is semantically ambiguous: it can mean 'the data genuinely does not exist' or 'I could not look.' Only an explicit, machine-readable status channel lets the orchestrator distinguish the two and respond correctly — retrying or escalating on access errors while confidently reporting true empty results. Blind retries waste budget when the answer really is empty, collapsing both cases into 'no data' converts infrastructure failures into false factual claims, and free-form prose is unreliable to parse programmatically.

   Source: CCA-F Exam Guide Domain 5 — structured error propagation in multi-agent systems: distinguishing access/tool failures from valid empty resultsReport a problem with this question

6. 6. An agent must map how authentication works across a 500,000-line codebase, far exceeding its context window. Which approach best manages context for this exploration?

   • A.Read every potentially relevant file sequentially into the lead agent's context
   • B.Rely on the model's pretraining knowledge of similar codebases instead of reading files
   • C.Delegate exploration to subagents that each investigate one area, write condensed findings to scratchpad files, and return short summaries, keeping the lead agent's context lean✓ Answer
   • D.Paste the repository's full directory listing and all file contents at the start of the conversation

   Subagent delegation works because each subagent spends its own context window on raw exploration and returns only distilled conclusions, while scratchpad files persist intermediate findings outside any context window so they survive and can be re-read on demand. The lead agent thus consumes summaries, not source. Reading everything into one context exceeds the window and buries key facts; pretraining knowledge cannot describe a specific private repository; and pasting the whole repo fails for the same capacity reason.

   Source: docs.claude.com Claude Code — common workflows: use subagents and scratchpad/notes files to explore large codebases without exhausting the lead context; CCA-F Exam Guide Domain 5Report a problem with this question

7. 7. An extraction pipeline processes 10,000 invoices per month and produces a confidence score per extracted field. The team can human-review only 5% of output. How should they decide what gets reviewed?

   • A.Route documents to review based on page length, since longer documents contain more errors
   • B.Validate calibration with stratified samples across confidence bands and document types, then route individual low-confidence fields to human review✓ Answer
   • C.Review a flat random 1% of documents and extrapolate the error rate to everything else
   • D.Ask the model to state an overall confidence per document and review the ones it calls 'unsure'

   Confidence scores are only useful for routing if they are calibrated — a stated 90% should be right about 90% of the time — and calibration can only be verified by sampling within each confidence band and each document type, since errors often concentrate in rare strata that a flat random sample barely touches. Field-level routing then sends humans exactly the uncertain values instead of whole documents. Verbally self-reported confidence is a known unreliable proxy, and page length has no principled link to extraction accuracy.

   Source: CCA-F Exam Guide Domain 5 — extraction QA: stratified sampling to validate confidence calibration and field-level thresholds for human-review routingReport a problem with this question

8. 8. A lead agent synthesizes findings from four research subagents into a report. Two subagents cite sources giving contradictory figures for the same market size. What is the correct synthesis behavior?

   • A.Adopt the figure supported by the majority of subagents and discard the minority source
   • B.Omit the market-size claim entirely, since conflicting data cannot be reported responsibly
   • C.Always use the figure from the most recently published source and drop the older one
   • D.Attach source attribution to every claim, and where sources conflict, present the disagreement explicitly with both figures and their citations✓ Answer

   Per-claim provenance is what makes a synthesized report auditable: a reader can trace any statement back to the source that supports it. When sources genuinely conflict, silently picking one — by majority, recency, or any other automatic heuristic — hides real uncertainty and can encode a wrong choice as fact; majority among subagents may just mean several agents read the same flawed source, and newer is not automatically more accurate. Surfacing the conflict with both citations preserves the information the reader needs to judge, while omitting the claim discards useful evidence.

   Source: CCA-F Exam Guide Domain 5 — synthesis reliability: per-claim source provenance and explicit surfacing of conflicting sourcesReport a problem with this question

9. 9. An orchestrator fans a research task out to four subagents. Three complete successfully; one fails with a network timeout after retries are exhausted. How should the orchestrator respond to the user?

   • A.Discard all results and report that the task failed, since the output is incomplete
   • B.Keep silently re-running the failed subagent until it succeeds, however long that takes
   • C.Return the three completed results explicitly labeled as partial, with a structured note identifying which sub-task failed and the error type✓ Answer
   • D.Present the three completed results as the full answer to avoid confusing the user

   Graceful degradation preserves the value of completed work while keeping the user's mental model accurate: they know exactly what they have, what is missing, and why, so they can decide whether to retry, proceed, or seek the missing piece elsewhere. Discarding three good results throws away paid-for work; presenting partial output as complete is a silent correctness failure that can drive wrong decisions; and unbounded silent retries block the response indefinitely on an error that already survived its retry budget.

   Source: CCA-F Exam Guide Domain 5 — multi-agent reliability: propagate partial results with explicit labeling of failed sub-tasks instead of silent completion or total failureReport a problem with this question

10. 10. An AI support agent handles sessions in which customers routinely raise three or four unrelated issues (billing, shipping, a product defect, and an account change) in a single conversation. As the transcript grows, the agent begins conflating details between issues. Which architectural change most directly addresses this failure mode?

    • A.Append a reminder at the end of every turn telling the model to be careful not to mix up the issues
    • B.Increase the model's temperature so it explores more interpretations of ambiguous references
    • C.Restart the conversation for each new issue and ask the customer to re-explain everything from the beginning
    • D.Extract each issue's key facts (issue type, status, relevant IDs, resolution steps) into a structured issue-tracking layer maintained separately from the conversational transcript✓ Answer

    A separate structured layer gives each issue a compact, authoritative record the agent can consult regardless of transcript length, so every detail stays attributed to the correct issue; relying on the raw transcript alone degrades as context grows. Restarting conversations or appending reminders does not mechanically prevent cross-issue contamination.

    Source: CCAR-F Exam Guide Domain 5 — structured state externalization for multi-issue sessions; Anthropic context-management guidance (structured note-taking outside the transcript)Report a problem with this question

11. 11. A research subagent produces 20-page reports that an orchestrator agent must synthesize alongside outputs from five other subagents. The orchestrator frequently misses the reports' most important conclusions. Per context-management best practice, how should the report format be changed?

    • A.Place a key-findings summary at the top of each report, followed by explicitly headed sections (e.g., 'Key Findings', 'Evidence', 'Methodology') for the supporting detail✓ Answer
    • B.Bold every sentence containing a conclusion throughout the document
    • C.Compress each report to a single paragraph, discarding the evidence and methodology entirely
    • D.Move all conclusions to the final paragraph so they benefit from recency effects

    Models attend most reliably to material at the start of a document, and explicit section headers let the orchestrator locate supporting detail on demand; conclusions buried mid-document suffer 'lost in the middle' degradation, and collapsing everything to one paragraph destroys the evidence needed for synthesis and verification. Bolding changes styling, not position in the context.

    Source: CCAR-F Exam Guide Domain 5 — report structuring for downstream consumption; Anthropic long-context guidance (key findings first, explicit section headers)Report a problem with this question

12. 12. An orchestrator merges structured findings from several research subagents. During synthesis it cannot tell whether two conflicting revenue figures come from different fiscal years, nor which underlying documents produced them. What requirement on subagent outputs prevents this class of failure?

    • A.The orchestrator should re-run the cheaper subagent whenever two figures conflict
    • B.Subagents should return the full raw source documents so the orchestrator can re-derive provenance itself
    • C.Subagents should return only findings that agree with each other, discarding conflicts before handing off
    • D.Every finding in a subagent's structured output must carry metadata fields such as the data's effective or publication date and the source location (document ID, URL, or page)✓ Answer

    Metadata travels with each finding, so the orchestrator can resolve apparent conflicts (for example, figures from different dates) and audit provenance without re-fetching sources. Discarding conflicts hides real signal, re-running a subagent does not explain the discrepancy, and returning full raw documents defeats the context savings that motivate the subagent pattern.

    Source: CCAR-F Exam Guide Domain 5 — subagent output contracts requiring metadata (dates, source locations) in structured outputsReport a problem with this question

13. 13. A pipeline's upstream analysis agents each return several thousand tokens of narrative reasoning. The downstream decision agent, which must also hold a large policy manual in context, is hitting its context limit and truncating inputs. Which change best fits context-management guidance?

    • A.Ask the upstream agents to write shorter sentences while keeping the full reasoning narrative
    • B.Modify the upstream agents to return structured data (fields, values, confidence, citations) and omit the verbose reasoning narrative from the handoff✓ Answer
    • C.Remove the policy manual from the downstream agent's context so more narrative reasoning fits
    • D.Have the downstream agent read the upstream outputs twice to compensate for the truncation

    When a downstream consumer's context is limited, the correct fix is at the producer: the decision agent needs conclusions and data, not the token-heavy reasoning path, so structured handoffs preserve all decision-relevant information at a fraction of the tokens. Removing the policy manual deletes information the decision actually depends on, and shorter sentences or double reading do not materially reduce or recover truncated content.

    Source: CCAR-F Exam Guide Domain 5 — adapting upstream agents to emit structured data instead of verbose reasoning when downstream context is constrainedReport a problem with this question

14. 14. Ten minutes into troubleshooting a router problem, the customer writes: 'Stop — I want to speak to a human agent right now.' The AI agent believes it is one step away from resolving the issue. What should it do?

    • A.Transfer to a human immediately, passing along a summary of the troubleshooting steps already attempted✓ Answer
    • B.Explain that human agents have long wait times and ask the customer to reconsider
    • C.Complete the final troubleshooting step first, then offer the transfer only if it fails
    • D.Offer a discount to keep the customer in the automated flow

    An explicit request for a human is a hard escalation trigger that must be honored immediately, regardless of the agent's confidence that resolution is imminent; squeezing in one more step or dissuading the customer overrides their stated intent and erodes trust. Passing a summary of attempted steps preserves context so the human agent does not force the customer to start over.

    Source: CCAR-F Exam Guide Domain 5 — escalation handling: explicit customer requests for a human are honored immediately, not deferred pending resolution attemptsReport a problem with this question

15. 15. A customer asks an AI retail-support agent to match a competitor's lower advertised price. The agent's policy documentation covers refunds, exchanges, and the store's own price-adjustment window, but says nothing about competitor price matching. What is the best action?

    • A.Escalate to a human agent, noting that the request falls outside documented policy✓ Answer
    • B.Ask the customer to upload proof of the competitor's price, then approve it if it looks legitimate
    • C.Decline the request, since anything not documented must be unsupported
    • D.Approve the match by analogy to the store's own price-adjustment policy

    A policy gap means the agent has authority neither to grant nor to deny the request: improvising an approval by analogy risks making unauthorized financial commitments, while auto-declining may contradict the business's actual (undocumented) practice. Escalation routes the decision to someone with the authority to set or interpret policy, which is the required behavior for requests the documentation does not cover.

    Source: CCAR-F Exam Guide Domain 5 — escalation handling: requests falling in policy gaps (e.g., competitor price matching not covered) route to humans rather than improvised decisionsReport a problem with this question

16. 16. While verifying identity for a warranty claim, an AI agent searches by the caller's full name and city and gets two distinct customer records. What should it do next?

    • A.Merge the information from both records to be safe
    • B.Read both records' details to the caller and ask which one is theirs
    • C.Ask the caller for an additional identifier — such as the email on file, an order number, or a phone number — before proceeding✓ Answer
    • D.Proceed with the record showing the most recent purchase, since that customer is the likelier caller

    Acting on an ambiguous match risks disclosing or modifying the wrong customer's account, and reading out both records' details would itself leak another customer's personal data to the caller. Requesting an additional identifier disambiguates the match without exposing information from either record, which is why multiple-match situations require gathering more identifiers before proceeding.

    Source: CCAR-F Exam Guide Domain 5 — identity resolution: on multiple customer matches, request additional identifiers rather than guessing, merging, or disclosing record detailsReport a problem with this question

Practice questions based on the official Claude Certified Architect – Foundations (CCAR-F) exam guide and Anthropic's public documentation. This is an independent study tool, not affiliated with or endorsed by Anthropic, and does not grant certification. The real exam is 60 questions, 120 minutes, passing at a scaled 720/1000, delivered via Pearson VUE ($125). Official certification page →