Information Provenance and Uncertainty in Multi-Source Synthesis

Information provenance and uncertainty in multi-source synthesis is the closing task statement of the Claude Certified Architect — Foundations (CCA-F) exam, covering task 5.6: "Preserve information provenance and handle uncertainty in multi-source synthesis." Domain 5 (Context Management and Reliability) carries a 15 % exam weight, and task 5.6 anchors the domain's most architecturally demanding pattern — a multi-agent research system that must track which source produced which claim, recognize when sources disagree, and surface uncertainty rather than paper over it. Community pass reports place information provenance among the highest-difficulty task statements on Domain 5 precisely because Claude does not automatically track sources or mark uncertainty without explicit design: every provenance guarantee you want in the final output must be engineered into prompts, schemas, and orchestration logic.

This study note walks through every outline bullet a CCA-F candidate is expected to architect at the system level: the provenance problem itself, structured claim-source mappings, the identification fields that make a citation actionable, multi-source conflict resolution, uncertainty representation in output schemas, the distinction between model confidence and source reliability, hedged language patterns, source-quality weighting, cross-source triangulation, provenance audit trails, and uncertainty-driven escalation. Information provenance and uncertainty in multi-source synthesis is most heavily exercised in the multi-agent-research-system scenario and the structured-data-extraction scenario; worked examples from both anchor the patterns. Five exam traps and a seven-question FAQ close the note.

The Provenance Problem — Losing Track of Which Source Produced Which Claim

Information provenance is the property of a synthesized output that every claim in the final answer can be traced back to the specific source document, chunk, and retrieval step that produced it. The provenance problem is the default failure mode of a naive multi-source synthesis pipeline: Claude reads five documents, writes a confident paragraph, and you have no mechanical way to tell which sentence came from which PDF. Without information provenance, you cannot audit hallucinations, you cannot respect source-level access controls, you cannot cite correctly in regulated domains, and you cannot repair a downstream dispute without re-running the entire synthesis.

Why Naive Synthesis Loses Provenance

A typical naive pattern looks like this: retrieve five passages from a vector index, concatenate them into a prompt under a heading "Sources:", ask Claude to "summarize the relevant information." Claude will produce a clean summary — but the summary is a blend. The attribution is lost the moment Claude starts a new paragraph. This is the default shape of information provenance failure, and every CCA-F scenario that mentions "research system", "synthesis", or "multi-source" is testing whether you know how to engineer around it.

Provenance as a First-Class Design Goal

Information provenance is not a feature you add after the fact. It is a constraint that reshapes every stage of the pipeline: retrieval stores a source ID with each chunk, prompts ask Claude to produce claims not prose, the output schema enforces a citation field on every claim, and downstream validation rejects outputs where any claim lacks a source ID. CCA-F questions reward candidates who treat information provenance as a pipeline-level property, not a post-hoc annotation.

Structured Claim-Source Mappings — Attaching Source Citations to Each Extracted Claim

The mechanical core of information provenance is the structured claim-source mapping: a record that pairs every atomic claim in the synthesized output with the source identifiers that support it. This pattern replaces "prose with footnotes" (unverifiable) with "array of claims, each with an explicit source array" (verifiable and machine-checkable).

Claim as the Atomic Unit

An information provenance design decomposes the synthesized output into atomic claims — single, checkable assertions — rather than paragraphs. "Acme Corp reported $4.2 B revenue in Q2 2026" is a claim. "Acme Corp is performing well this year with strong revenue and improving margins" is a paragraph hiding three claims. Claims are the unit on which information provenance can be enforced; paragraphs are not.

The Output Schema Enforces Information Provenance

Information provenance is enforced through the output schema passed to strict tool use. A typical schema looks like:

{
  "claims": [
    {
      "claim_id": "c-001",
      "text": "Acme Corp reported $4.2 B revenue in Q2 2026.",
      "sources": [
        {"source_id": "doc-acme-10Q-2026Q2", "chunk_index": 14}
      ],
      "uncertainty": "low"
    }
  ]
}

With strict: true on the tool definition, the Claude API refuses to return an output that omits the sources array on any claim. The information provenance guarantee is mechanical, not polite.

Why Structured Mappings Beat Inline Citations

Inline citations in prose ("Acme reported $4.2 B [1]") look reasonable but are brittle: Claude may forget the marker, duplicate it, or attach the wrong number. A structured claim-source mapping in a JSON schema is mechanically verifiable — a validator can programmatically assert every claim has ≥ 1 source, every source ID exists in the retrieval log, and no hallucinated IDs slipped in. Information provenance at this level of rigor is what separates research-grade pipelines from toy demos.

Source Identification Fields — URL, Document ID, Chunk Index, Retrieval Timestamp

A source ID that is just "Acme 10-Q" is not enough to satisfy information provenance. A production-grade source reference carries enough fields to uniquely locate the exact span of text that supports the claim and to prove the evidence was available at the time of synthesis.

Minimum Viable Source Reference

For CCA-F purposes, the minimum fields on every information provenance source reference are:

• URL or document URI — canonical locator for the source.
• Document ID — stable, internal identifier (hash, UUID, or external primary key).
• Chunk index or page number — sub-document span identifier.
• Retrieval timestamp — the moment the content was pulled from the source.
• Source kind — enumerated type (web, internal-wiki, CRM, knowledge-base, uploaded-pdf, etc.).

Why Every Field Matters

The URL makes the source resolvable by a human auditor. The document ID lets you de-duplicate and join against retrieval logs. The chunk index narrows the evidence to a specific span. The retrieval timestamp proves the evidence existed in the source at the time of synthesis — critical when sources mutate (wikis, web pages, ticket systems) and a later reader needs to know which version was cited. Information provenance without a retrieval timestamp is weaker because the source can silently change between synthesis and audit.

Source Reference in a Multi-Agent Research System

In the multi-agent-research-system CCA-F scenario, several subagents may independently retrieve overlapping content. The information provenance schema must carry enough identifiers to detect when two subagents cite the same chunk (for cross-source triangulation) versus when they cite different chunks that happen to come from the same document. Chunk index is the field that makes that distinction possible.

Multi-Source Conflict Resolution — When Sources Disagree on the Same Fact

Multi-source synthesis routinely pulls in sources that disagree. The oldest CRM note says the customer is on the Basic plan; the billing API says Pro; the support-case transcript from last week says "I just upgraded." Information provenance lets you see the conflict; conflict resolution policy decides what the synthesized output says.

Four Conflict-Resolution Policies

1. Most-recent-wins — Break ties by retrieved_at or source publication date. Appropriate for mutable facts (plan tier, address, status).
2. Authoritative-source-wins — Break ties by source kind (billing API > CRM note > free-text transcript). Appropriate when one source is the system of record.
3. Report-both — Present both values with their sources. Appropriate when the fact genuinely cannot be resolved and a downstream human must adjudicate.
4. Escalate — Refuse to synthesize; return the conflict to a human reviewer. Appropriate for high-stakes facts (legal, medical, financial).

Encoding Policy in Prompts

Conflict-resolution policy must be explicit in the system prompt. "If billing API disagrees with CRM notes, trust billing API and record the CRM value in the conflicting_sources field of the claim." Without this instruction, Claude will pick a plausible value silently — undermining information provenance by hiding the conflict.

The conflicting_sources Field

A mature information provenance schema extends each claim with a conflicting_sources array so that resolved conflicts are still visible in the output. The resolved value lives in sources, the losing value in conflicting_sources. Downstream auditors can inspect the conflicting_sources list to verify the policy was applied correctly.

Uncertainty Representation — Explicit Uncertainty Flags in Structured Output

Information provenance tells you where a claim came from; uncertainty representation tells you how much to trust it. A CCA-F-grade synthesis pipeline attaches an explicit uncertainty flag to every claim in the output, rather than burying uncertainty in prose hedges like "likely" or "reportedly."

Uncertainty as an Enumerated Field

The cleanest representation is an enumerated uncertainty field on each claim with a small, well-defined set of values:

• high — low source coverage, conflicting sources, or the claim required inference from multiple chunks.
• medium — single authoritative source; no corroboration.
• low — multiple corroborating authoritative sources; no conflicts.
• unverified — no source provides direct evidence; claim is Claude's inference or extrapolation.

Why Enumerated Beats Free-Form

An enumerated uncertainty field is mechanically filterable — "show me only high and unverified claims" is a downstream query — while a free-form uncertainty string is not. CCA-F scenarios consistently reward structured enumerations over prose annotations because structure enables validation, routing, and automated escalation.

Claude Confidence Is Not the Same as Source Reliability

A subtle but exam-critical distinction: Claude's internal confidence that it extracted the right value from a source is NOT the same as the reliability of the source itself. A claim can be extracted with high confidence from a source that is itself unreliable. Information provenance schemas must separate these two axes, typically through a second enumerated field like source_quality: authoritative | reputable | user-generated | unverified.

Confidence vs Uncertainty — Distinguishing Model Confidence from Source Reliability

The confidence / uncertainty distinction deserves its own H2 because it is the single most-missed concept in Domain 5 community pass reports. A CCA-F candidate who treats these as synonyms will lose at least one scored question.

Model Confidence

Model confidence is Claude's internal signal about extraction quality: "I am 95 % sure the revenue figure in this 10-Q is $4.2 B." It is a statement about the synthesis step itself. Model confidence tells you nothing about whether the 10-Q is correct; it only tells you whether Claude read the 10-Q correctly.

Source Reliability

Source reliability is a property of the source, not the model: "the 10-Q is a regulated financial filing authored by a public company under SEC oversight." Source reliability is high for regulated filings, medium for curated wikis, low for unverified user-generated content. The same extraction task can hit high model confidence on a low-reliability source (Claude clearly read the Reddit comment; the Reddit comment may be wrong).

Output Uncertainty Is the Join

Uncertainty in the final output is what you report to the downstream consumer. It is the join of model confidence and source reliability, further adjusted by cross-source corroboration. A claim reaches low uncertainty only when all three conditions are satisfied: high model confidence, authoritative source, at least one corroborating source.

The Exam Trap in This Distinction

A classic distractor reads: "Claude's log probabilities are high, so the synthesized claim is reliable." Wrong. High log probs mean Claude is confident about the extraction, not about the source. Another distractor: "The source is the SEC, so the claim is reliable regardless of whether Claude summarized it correctly." Also wrong — source reliability is necessary but not sufficient. Correct answers handle both axes.

Hedged Language Patterns — Linguistic Markers of Uncertainty in Generated Text

Even when your output schema enforces structured uncertainty fields, the text field of each claim still contains natural language, and the hedging that Claude chooses in that prose is meaningful. A well-designed prompt instructs Claude to align hedged language with the structured uncertainty value.

Canonical Hedge Markers

• "according to [source]" — neutral attribution, no hedge.
• "reportedly" / "reported to" — medium hedge; implies the claim exists in a source but is not independently verified.
• "may" / "might" / "could" — high hedge; implies substantial uncertainty or inference.
• "appears to" / "is likely" — high hedge; Claude is drawing inference.
• "inferred from" — explicit marker of extrapolation; should always pair with unverified or high uncertainty.

Aligning Prose Hedges with Structured Uncertainty

A common failure mode is prose that hedges aggressively while the structured uncertainty field says low, or vice versa. This mismatch confuses downstream humans who read the prose and automated systems that read the structure. The system prompt should explicitly require alignment: "Use hedged language in the text field that matches the uncertainty level; low claims use direct statements, medium and above must use an explicit hedge marker."

Hedging in the Customer-Support Scenario

In a customer-support synthesis, "the customer's plan is Pro" (no hedge, low uncertainty) is very different from "the customer's plan appears to be Pro based on the most recent ticket" (hedge, medium uncertainty). The difference determines whether the agent can take autonomous billing action or must escalate. Information provenance is what makes this distinction mechanically visible.

Source Quality Weighting — Treating Authoritative Sources as Higher Confidence

Not all sources deserve equal weight. A production information provenance pipeline weights sources by a configured authority hierarchy before deciding the final uncertainty of each claim.

A Tiered Source Hierarchy

Typical tiers for an enterprise multi-agent research system:

1. Tier 1 — Systems of record (billing API, HRIS, source-controlled repository). Authoritative for the facts they own.
2. Tier 2 — Curated internal knowledge (engineering wiki, approved runbooks). Reliable but occasionally stale.
3. Tier 3 — Transactional logs (support-ticket history, chat transcripts). Accurate for what happened but not always for current state.
4. Tier 4 — External public sources (vendor docs, news, web search results). Use as background; verify against higher tiers when possible.
5. Tier 5 — User-generated (forum posts, community answers). Lowest weight; require corroboration before asserting.

Weighting in Practice

Weighting is encoded in the system prompt and enforced by the output schema. "When a Tier 1 source disagrees with a Tier 3 source, trust Tier 1 and record the Tier 3 value in conflicting_sources. A claim supported only by a Tier 4 or Tier 5 source must have uncertainty: medium or higher unless at least one Tier 1-3 source corroborates it."

Why Weighting Matters for Information Provenance

Without explicit weighting, Claude applies an implicit, unstable ordering that the pipeline owner cannot audit. With explicit weighting, information provenance is deterministic — the same inputs always produce the same trust decisions, and a human auditor can verify the policy was applied.

Cross-Source Triangulation — Requiring Corroboration Before Asserting High Confidence

Triangulation is the information provenance technique of requiring multiple independent sources to agree before a claim is promoted to low uncertainty. It is the strongest defense against both hallucination and the "one Reddit comment cited as fact" failure mode.

The Triangulation Rule

A typical triangulation policy reads: "A claim may be emitted with uncertainty: low only if at least two sources from different documents (and ideally from different source tiers) corroborate the claim. A claim supported by only one source, regardless of that source's authority, defaults to uncertainty: medium."

Why Different Documents, Not Different Chunks

Triangulation requires different documents (or ideally different source types), not different chunks of the same document. Two chunks from the same 10-Q are not independent evidence — they share a single author and a single editorial process. Information provenance that counts intra-document chunks as triangulation is a common naive bug; CCA-F scenario questions test whether you distinguish intra-document agreement (not triangulation) from cross-document agreement (triangulation).

Triangulation in the Multi-Agent Research System

In the multi-agent-research-system scenario, the coordinator can deliberately dispatch multiple subagents to independent source pools specifically to enable triangulation. Subagent A searches the company wiki, subagent B searches the ticket system, subagent C searches external web results. When all three return the same fact, the coordinator promotes that fact to low uncertainty. This is a direct application of information provenance architecture that frequently appears in CCA-F practice questions.

Provenance Audit Trails — Logging Retrieval and Synthesis Steps for Traceability

Information provenance in the output is valuable; information provenance in the pipeline logs is what makes the output auditable weeks or months later. An audit trail captures the complete retrieval-and-synthesis history so that any final claim can be reproduced or disputed.

What a Provenance Audit Trail Records

A minimal information provenance audit trail logs, for every synthesis run:

• Run ID — a unique identifier for the synthesis invocation.
• Retrieval events — query text, vector index version, returned chunks with IDs and retrieval timestamps.
• Tool calls — every tool_use block emitted by Claude, every tool_result returned.
• Subagent dispatches — which subagent handled which sub-task, with its own run sub-ID.
• Conflict resolutions — every multi-source conflict detected and the policy that resolved it.
• Final synthesis output — the complete structured claims array, including uncertainty flags and conflicting_sources.
• Session ID — the Agent SDK session identifier that binds the synthesis to a conversation.

Why Audit Trails Exceed the Context Window

The audit trail must live outside the Claude context window — typically in an external log store (object storage, durable queue, or SIEM). The context window can only hold a small fraction of what a long-running synthesis accumulates, and pruned context is unrecoverable. Information provenance requires that retrieval and synthesis events be durable, not ephemeral.

The Link Between Session IDs and Audit Trails

Agent SDK session IDs are the linkage primitive between the synthesis run and the audit trail. Fork a session, and the audit trail must record the fork point; resume a session, and the audit trail must link the resumed log to the parent. This linkage is what allows information provenance to survive across session boundaries — a common pattern in long-running research workflows.

Uncertainty Escalation — Routing Low-Provenance Assertions to Human Verification

Information provenance pays off at escalation time. A synthesis pipeline that tags every claim with explicit uncertainty can route high-uncertainty claims to human verification automatically, without forcing humans to read the entire output.

The Escalation Routing Rule

A typical information provenance escalation policy reads: "Any claim with uncertainty: high or unverified, OR any claim with conflicting_sources non-empty under an unresolved policy, OR any claim whose sources are all Tier 4 / Tier 5, is routed to a human reviewer before the final output is released to downstream consumers."

Escalation in the Customer-Support Scenario

In the customer-support-resolution-agent scenario, an information provenance pipeline ingests the customer ticket, retrieves from CRM + billing + knowledge base, and synthesizes a proposed resolution with per-claim sources. Claims about the customer's plan tier (Tier 1, billing API) are low uncertainty and the agent can act autonomously. Claims about the customer's sentiment inferred from prior tickets (Tier 3, transactional) are medium or high uncertainty and must be surfaced to a human before sending a tone-sensitive reply. Information provenance is what makes this routing mechanical.

Escalation vs Refusal

Escalation is not the same as refusing to synthesize. A well-architected pipeline produces the complete synthesis with per-claim uncertainty, then escalates only the high-uncertainty subset. Refusing the entire synthesis because one claim is uncertain wastes the work already done on reliable claims.

Plain-English Explanation

Abstract information provenance mechanics become intuitive when anchored to physical systems candidates already know. Three very different analogies cover the full sweep of provenance and uncertainty.

Analogy 1: The Library Card Catalogue

Picture the card catalogue of a traditional research library. Every card carries a specific book's call number, shelf location, author, publication date, and edition. When a librarian answers a patron question, they do not say "somewhere in the building there is a book that mentions this" — they cite the call number. The patron can walk to the shelf, pull the exact edition, and read the paragraph on the cited page. That is information provenance. Now imagine a librarian who summarizes three books into a paragraph and refuses to say which fact came from which book. The patron cannot verify anything. That is multi-source synthesis without provenance. A CCA-F-grade pipeline gives Claude the librarian's card catalogue discipline: every claim in the synthesized output carries the source document, the chunk index, and the retrieval timestamp — the equivalent of call number, page, and edition date. When two books disagree, the librarian does not silently pick one; she notes the disagreement on the card so the patron can adjudicate. That is multi-source conflict resolution with conflicting_sources. Information provenance, in one mental model, is just the library card catalogue applied to Claude.

Analogy 2: The Insurance Adjuster's Claim File

An insurance adjuster investigating a loss does not accept claims at face value. They collect evidence — police reports, witness statements, photos, repair estimates — and stamp each piece with date, source, and provenance notes. When the adjuster writes the final determination, every factual assertion points to an evidence item in the file. If two witnesses disagree, the adjuster records both statements and explains which was weighted more heavily and why. Uncertain items — "claimant alleges but no corroborating evidence" — are flagged explicitly rather than blended into prose. The claim file is the audit trail; the final determination letter is the synthesized output; the evidence stamps are the source reference fields. An adjuster who writes a fluent summary without supporting evidence is unemployable. Information provenance in a Claude multi-source pipeline imposes the same discipline: every claim points to evidence, every disagreement is documented, and every uncertain item is flagged. The enumerated uncertainty field is the adjuster's "alleged but uncorroborated" stamp. Triangulation is the adjuster's rule that a single witness is not enough for a contested fact.

Analogy 3: The Newsroom with Two-Source Confirmation

Traditional newsrooms operate under an explicit two-source rule: before publishing a contested fact, reporters must obtain independent corroboration from a second source. A single anonymous tip is not enough; a single leaked document is not enough. Independence matters — two reporters from the same outlet citing the same press release is not two sources, it is one source read twice. When the two-source rule cannot be met, the reporter uses hedged language ("sources say", "reportedly", "is alleged to") that mirrors the structured uncertainty of the information. When sources disagree, the news story documents both positions and attributes each. The newsroom's standards desk reviews the fact-check notes before publication — the equivalent of uncertainty escalation routing high-uncertainty claims to human review. A CCA-F information provenance architecture is a newsroom implemented in prompts and schemas: triangulation is the two-source rule, source-quality weighting is the newsroom's tier hierarchy (wire services > press release > unnamed tip), hedged language is the reporter's prose, and the fact-check file is the audit trail.

Which Analogy Fits Which Exam Question

• Questions about source identification fields and audit trails → library card catalogue analogy.
• Questions about uncertainty flagging and conflict resolution → insurance adjuster analogy.
• Questions about triangulation and source-quality weighting → newsroom two-source analogy.

Common Exam Traps

CCA-F task 5.6 consistently exploits five recurring information provenance trap patterns. All five are documented in community pass reports and appear as plausible distractors.

Trap 1: Assuming Claude Cites Sources Naturally

The single most-cited task 5.6 mistake is assuming that if you include source documents in the prompt, Claude will automatically cite them in the output. It will not. Information provenance in a multi-source Claude pipeline requires an explicit output schema with a mandatory sources array per claim, enforced via strict tool use. Answers that propose "ask Claude to cite sources in its prose" without structural enforcement are wrong.

Trap 2: Conflating Model Confidence with Source Reliability

A claim can be extracted with high model confidence from a low-reliability source. Distractors frame "Claude is confident" as equivalent to "the claim is reliable." The correct mental model separates model confidence (extraction quality) from source reliability (source authority), and only considers a claim low-uncertainty when both axes are satisfied and cross-source corroboration is present.

Trap 3: Storing Audit Trails in the Context Window

A naive approach is to accumulate retrieval and synthesis logs inside the conversation history so "Claude can refer back to them." The context window will be trimmed, compacted, or capped before the audit trail is useful. Information provenance audit trails must live in durable external storage keyed by session ID; answers that propose in-context audit trails are architecturally unsound.

Trap 4: Counting Intra-Document Chunks as Triangulation

Two chunks from the same document are not independent evidence. Distractors offer "multiple chunks from the 10-Q agree, so the claim is triangulated." Wrong. Triangulation requires different documents (and ideally different source tiers) because independence of editorial process is what makes corroboration meaningful. CCA-F scenario questions specifically test this distinction.

Trap 5: Escalating the Entire Synthesis Instead of the Flagged Subset

When uncertainty is detected on some claims, the correct response is partial release: emit the full structured output, flag the uncertain subset, and route only the flagged subset for human review. Distractors propose "if any claim is uncertain, refuse to return anything" or "resynthesize from scratch until all claims are confident." Both waste the reliable work already done and defeat the purpose of structured uncertainty representation.

Practice Anchors

Information provenance and uncertainty appear most heavily in two of the six CCA-F scenarios. Treat the following as the architecture spine for scenario-cluster questions on task 5.6.

Multi-Agent-Research-System Scenario

In this scenario, a coordinator agent dispatches multiple subagents to independent source pools — company wiki, ticket system, external web, internal CRM — each tasked with answering the same research question from its own evidence base. The subagents return structured claims arrays, and the coordinator synthesizes a final answer that carries full information provenance: every output claim cites the subagent that produced it, the source document and chunk, and the uncertainty level assigned. Expect questions that test cross-source triangulation (when can the coordinator promote a claim to low uncertainty?), conflict resolution policy (who decides which subagent wins when they disagree?), audit trail externalization (where do retrieval logs live?), and uncertainty escalation (which claims route to a human?).

Structured-Data-Extraction Scenario

In this scenario, Claude is asked to extract a structured record (contract terms, invoice fields, compliance attributes) from one or more source documents. Information provenance here is less about multi-agent orchestration and more about per-field source attribution: every extracted field carries the document URI, page or chunk, and an uncertainty flag. Expect questions that test strict tool use enforcement of a per-field source schema, handling of missing evidence (what uncertainty value do you emit when the source does not contain the requested field?), hedged language alignment between structured fields and prose summaries, and validation that source IDs in the output actually exist in the retrieval log.

FAQ — Information Provenance and Uncertainty Top 7 Questions

What exactly is information provenance in a Claude multi-source pipeline?

Information provenance is the property that every atomic claim in Claude's synthesized output can be traced back to a specific source document, chunk, retrieval timestamp, and (in multi-agent systems) the subagent that produced it. Claude does NOT automatically produce information provenance — it must be engineered through a structured output schema with mandatory source fields per claim, strict tool use to reject schema violations, and post-synthesis validation that the cited source IDs actually exist in the retrieval log. Information provenance is the central competency tested by CCA-F task 5.6, and community pass reports flag "assumed Claude cites naturally" as one of the highest-frequency Domain 5 mistakes.

How do I enforce that every claim has at least one source?

Define a structured output schema (via tool use) where each claim object has a required, non-empty sources array, set strict: true on the tool definition, and add post-synthesis validation that rejects outputs where any claim's sources array is empty or contains IDs that are not present in the retrieval log. Strict tool use gives you schema-level enforcement; post-validation gives you semantic enforcement (no hallucinated source IDs). Both are required for production information provenance — schema enforcement alone cannot catch a well-formed-but-fabricated source ID.

What is the difference between model confidence and source reliability?

Model confidence is Claude's internal signal about extraction quality — how sure Claude is that it read the source correctly. Source reliability is a property of the source itself — how trustworthy the authoring party and editorial process are. They are orthogonal: Claude can extract with high confidence from a low-reliability source (correctly reading an unreliable Reddit comment) or extract with low confidence from a high-reliability source (partial tooling access to an SEC filing). The final output uncertainty should be a join of both axes plus cross-source corroboration, NOT just Claude's confidence. Conflating them is a top-five CCA-F Domain 5 trap.

How do I handle conflicting sources?

Specify a conflict-resolution policy in the system prompt — most-recent-wins, authoritative-source-wins, report-both, or escalate — and extend each claim in the output schema with a conflicting_sources array so that losing values remain auditable. The resolved value lives in sources; the losing values live in conflicting_sources with their own IDs and retrieval timestamps. Never let Claude silently pick a value without a documented policy. CCA-F questions consistently penalize answers that propose "let Claude decide" without an explicit conflict-resolution rule.

Why doesn't intra-document corroboration count as triangulation?

Triangulation is about independence of editorial process, not repetition of text. Two chunks from the same 10-Q filing share a single author, a single legal-review cycle, and a single publishing event. If the filing is wrong about a fact, every chunk is wrong about that fact. Cross-source triangulation requires agreement across different documents and ideally different source tiers (a billing API record plus a CRM note plus a ticket transcript) because that agreement survives a single point of editorial failure. CCA-F scenario questions specifically offer "multiple chunks from the same source" as a distractor and expect candidates to reject it as insufficient for a low uncertainty assertion.

Where should the provenance audit trail live?

In durable external storage — object storage (S3 / R2), a dedicated audit database, or a SIEM — keyed by the Agent SDK session ID (and run ID for each synthesis invocation). The Claude context window is not a viable location because it will be trimmed, summarized, or compacted long before the audit matters. The audit trail must capture retrieval events (query, index version, returned chunks), every tool_use and tool_result, every subagent dispatch, every conflict resolution, and the final structured output. Information provenance is a durability property, and durability means out-of-band storage linked by session identifiers.

When should uncertainty trigger human escalation versus just a hedge?

Escalate when the claim sits above a configurable uncertainty threshold (typically high or unverified), when conflicting_sources is non-empty under an unresolved policy, or when the supporting sources are all in the lowest tiers (user-generated, unverified external). Hedge (prose markers like "reportedly" or "appears to", aligned with medium structured uncertainty) when the claim is supported by a single authoritative source with no contradicting evidence. The decision rule is: escalate when a downstream autonomous action would be unsafe given the uncertainty; hedge when a human reader should be alerted but the system can still proceed. Always escalate only the flagged subset of claims, not the whole synthesis — partial release preserves the reliable work already done.

Further Reading

• Session management — Agent SDK: https://docs.anthropic.com/en/docs/claude-code/sdk/sdk-sessions
• Long context prompting tips: https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/long-context-tips
• Context windows — Claude API Docs: https://docs.anthropic.com/en/docs/build-with-claude/context-windows
• Increase output consistency — structured outputs: https://docs.anthropic.com/en/docs/test-and-evaluate/strengthen-guardrails/increase-consistency
• Strict tool use — schema-guaranteed output: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/strict-tool-use
• Building effective agents — escalation and human-in-the-loop: https://docs.anthropic.com/en/docs/build-with-claude/agentic-loop

Related ExamHub topics: Conversation Context Across Long Interactions, Human Review Workflows and Confidence Calibration, Multi-Agent Orchestration with Coordinator-Subagent Patterns, Escalation and Ambiguity Resolution.