Building AI that answers from documents

AI is increasingly used to make long, dense documents easier to understand: annual reports, prospectuses, regulatory filings, investor presentations. The promise is plain-English answers to complex questions. The risk is answers that read convincingly but are not supported by the document.

For any system built on source documents, one principle should govern the design: the reader must be able to verify the answer for themselves.

The sections below set out the common failure modes and the practices that address them.

01Do not let the AI do everything in one step

A common first design asks the AI to read a document, extract figures, perform calculations, apply judgement and write the answer, all at once. It often works well on the document used during development.

The problems appear on the next document. New errors surface, fixes are applied, and those fixes quietly break results on the first document. Development turns into a loop with no clear progress, because when an answer is wrong there is no way to tell which step failed.

In practice:

• Break the pipeline into distinct stages: extraction, validation, calculation, rule application and explanation.
• Make each stage’s output inspectable on its own.
• When an answer is wrong, trace it to the stage that failed rather than adjusting the prompt as a whole.

02Get the numbers right before interpreting them

The most effective shift is to stop asking “is the answer right?” and first ask “are the figures right?”

Convert each document into a structured ledger of key figures (revenue, profit, cash flows, borrowings, related-party transactions and so on), with every figure tied to its source page. Then check the ledger with code before anything else uses it.

Useful checks include:

• Do segment figures add up to the reported total?
• Do opening cash, reported cash movements and closing cash reconcile, allowing for disclosed exchange-rate and other adjustments?
• Is a figure labelled as a given year actually from that year?
• Are units (thousands, millions, lakhs, crores) applied consistently?

Answer numerical questions only from figures that have passed the relevant checks. Other questions may use independently supported narrative evidence; unresolved figures should not enter the answer.

03Test on documents the system has never seen

A system tuned on a handful of documents will eventually perform well on them. That proves very little.

In practice:

• Keep a set of unseen documents, ideally from different sectors and formats, for evaluation only.
• Stop refining against development documents once they reach a stable state.
• Include documents with known later problems (restatements, regulatory action) to test whether the system would have flagged them, without revealing which ones in advance to whoever is tuning the system.
• Later events must not become the answer key: assess whether the disclosures available at the time warranted attention, rather than whether the system predicted the eventual outcome.

04Decide clearly what the AI is responsible for

Business questions rarely fit predefined rules. “What drove growth?” may require connecting several years of commentary, acquisitions and segment results. “Why is working capital increasing?” may require examining inventories, receivables, payment terms and management’s assumptions across different sections.

A system can retrieve accurate numbers and still miss the explanation. Giving the AI more freedom helps it find and connect evidence, but it also increases the risk that it fills gaps with plausible explanations the document never provided.

A workable division of responsibility:

• AI interprets the question, searches the document and connects relevant evidence.
• Deterministic code validates structured data, performs calculations and enforces explicit rules.
• AI drafts an explanation that distinguishes disclosed facts, management statements, calculations and inference.
• Further checks, and human review where warranted, test whether the conclusions are supported.

And a workable order for answering:

• Use checked figures and relevant source passages as the evidence base.
• Use code for reproducible operations and AI for evidence-based explanation; both may be needed in the same answer.
• If a document has not been processed and checked, say so instead of improvising.

05Treat citations as necessary, not sufficient

A citation shows where something came from. It does not prove the answer is correct.

Common citation failures:

• The cited page contains the number, but for a different period or entity.
• The page supports part of the answer but not its conclusion.
• The page exists, but the relevant text is on a different page.

Equally important is the difference between identifying a disclosure and interpreting its significance. Negative operating cash flow does not, by itself, establish a failing business. A legal proceeding does not establish wrongdoing. An acquisition followed by revenue growth does not prove the acquisition caused it.

In practice:

• Check that each cited page contains the specific figure or statement, for the stated period.
• Check that the conclusion follows from the cited evidence, not just that evidence exists nearby.
• Distinguish what the document states from what the system calculates or infers. Label analytical inference clearly and cite the evidence supporting it.

06Define “stay within the document” precisely

“Do not go beyond the source document” is an essential rule, but a vague version can make a system either unsafe or useless.

A precise version:

• Allowed: arithmetic on disclosed figures (growth rates, ratios, rounding, unit conversion), provided any assumption is stated.
• Not allowed: conclusions, forecasts or characterisations the document does not support.
• Missing information is reported as missing, never silently assumed or filled from general knowledge.

07Handle advice-seeking questions deliberately

Users will ask “Should I invest?” or “Is this a good company?” A bare refusal is unhelpful and frustrating.

A better pattern:

• Detect advice-seeking questions by intent, not exact wording, since users phrase them in many ways.
• Decline to give a recommendation clearly and consistently.
• Pair the refusal with the relevant disclosed facts and their citations, so the reader can form their own view.

The reader makes the decision; the system shows the evidence.

08Watch for design choices that undermine trust

Some well-intentioned features backfire.

• Verification labels. Avoid blanket “verified” badges that imply more assurance than the checks provide. State material limitations precisely, for example, “The document does not disclose acquisition-level profitability.” If essential evidence cannot be validated, narrow or withhold the affected claim.
• Ratings and traffic-light scores. These help prioritise, but they can read as verdicts. If a label risks being taken as a judgement on the company, remove the label and keep the issue, cited to the source.
• Confident tone. Fluent writing is not evidence. The style of an answer should never imply more certainty than the evidence supports.

09Validate the code as well as the AI

Deterministic code catches many AI errors, including:

• whether a source and cited page exist;
• whether figures use consistent periods, units and definitions;
• whether calculations reconcile;
• whether a finding follows the stated rule; and
• whether missing information is treated as missing.

But code has its own failure modes. It can apply an incorrect rule consistently, or calculate correctly from incorrectly extracted data. Rules need review, and checks need testing, just like any other component.

Some things remain hard for code. Confirming that a page exists is easy; confirming that it fully supports a nuanced conclusion is not. That is where structured AI checks and human review still matter.

10Acknowledge limits openly

When the evidence is insufficient, the answer should say so. When a matter needs attention, it should be presented without turning it into an unsupported allegation or a recommendation.