Your AI cited a company document. Does it actually support the answer?
Contents
Does the source support it?
Check the claim. Then the citation.
The answer says a customer can keep last year’s terms. It includes a link to your renewal policy. You open it: the policy says nothing about that customer’s exception. The document exists. The conclusion still lacks support. Before you change the prompt or add more documents, isolate the exact claim that somebody is about to act on.
- Check whether a source supports the particular claim, not merely whether the link opens.
- Match the account, date, conditions and authority before treating a policy as an answer.
- Separate a false statement from an unresolved claim; both need attention, but they are different findings.
- Correct the affected decision and preserve a small, permitted example for checking the repair.
Start with the sentence that changes a decision
“The summary is wrong” gives the next person too much to investigate. Copy the decision-relevant sentence into a restricted review record: what is being asserted, about whom, and for which period? Keep the original question and only the necessary source references. Do not turn the review into a new broadly shared store of customer records.
A useful distinction is supported, contradicted or unresolved. Supported means the permitted evidence establishes the claim within its stated conditions. Contradicted means relevant evidence conflicts with it. Unresolved means the material you can inspect is insufficient. Absence of evidence does not prove the claim false, and a plausible sentence does not make it true.
NIST’s Generative AI Profile, section 2.2, describes confabulation as confidently presented erroneous content. For this review, an unsupported claim deserves a hold even before you can establish whether it is false. This is an editorial decision rule, not NIST certification.
Use a claim-check card
This original worksheet is for the reviewer who is authorised to inspect the relevant material. A teammate without that access can flag a claim and request review; they should not receive broader permissions just to complete the sheet.
Check the words around the cited passage. A heading may identify the right subject while the paragraph limits the rule to another plan, customer type or effective date. A genuine citation can support only part of a sentence.
| Check | Record or ask |
|---|---|
| Claim | Exact assertion, subject and period; split combined claims. |
| Evidence actually supplied | Source reference, relevant passage and whether it reached the assistant. |
| Scope | Does the passage apply to this account, role, plan or situation? |
| Time and authority | Is it effective now? Is it policy, a draft, a discussion or an exception? |
| Finding | Supported, contradicted or unresolved; state the reason. |
| Action and owner | Who checks or corrects the affected decision, and what evidence is still needed? |
A fictional renewal answer with a real-looking citation
Consider this invented example. A general policy says standard renewals use the current catalogue. A signed amendment for Account Alder preserves an earlier rate through November. An AI answer says all Alder seats keep that rate next year and cites only the general policy. These names and terms are fictional; this is not a HeyBrain customer incident or a tested product workflow.
Split the answer into two claims: which seats are covered, and for how long? The general policy establishes neither exception. The amendment might establish a bounded exception, but the reviewer still needs its exact scope and dates. A safe conclusion is not “the AI was close enough.” It is “the cited material does not establish the claimed scope or next-year period.”
If the amendment is unavailable to the requester, route the question to its authorised owner. Do not infer its contents from a file title, ask an assistant to guess, or copy restricted clauses into a shared incident note. The right next step may be a narrow confirmation from the owner rather than another model run.
Repair the layer that failed
A missing source, a stale record and an unsupported inference need different responses. Ask what the assistant actually received. If the relevant amendment was never supplied, investigate source selection or retrieval coverage. If a superseded policy was supplied, resolve ownership and freshness. If the correct passage was present but the conclusion stretched it, inspect the task instructions and how the answer was evaluated.
More stored information is not automatically more useful context. Context is what the current task receives; persistent memory stores information for later; retrieval selects stored material; source write-back edits the original system. A read connection does not imply write permission. Repairing an answer should not quietly change an authoritative record.
NIST recommends checking output sources and citations during evaluation and ongoing monitoring (MS-2.5-003). The practical lesson is to inspect the relationship between each important claim and its evidence, rather than count links. For organising the supplied material, use the context-engineering worksheet.
A retry can help diagnose a problem, but a better-looking second answer is not proof of a repair. Keep a permitted example with the expected bounded conclusion. Check it alongside a case with no supporting source and a case where a dated record has been replaced. These are suggested checks, not a benchmark or a claimed success rate.
Close the loop on the work, not just the answer
Find out whether someone already used the statement in a briefing, quote or customer commitment. The relevant owner should correct the affected work through the normal authorised process. If you do not know where the answer travelled, record that uncertainty instead of calling the issue resolved.
There is a cost to checking every sentence. Start with claims that change commitments, permissions or an operational decision, and let the responsible owner define the review standard. For casual brainstorming, a lighter check may be appropriate. For a consequential answer, confident prose should not substitute for inspectable support.
Keep the review record small: the claim, permitted evidence references, finding, correction owner and a check for recurrence. If the problem is several assistants disagreeing, use the answer-comparison worksheet. If you are deciding whether the whole pilot is worth expanding, use the executive pilot decision sheet.
Prepared by HeyBrain Editorial Team with AI assistance, primary-source research and independent editorial review. The claim-check card is an editorial tool; the renewal example is fictional. No customer incident, integration or product outcome was tested for this article.
