Back to the catalogue

Why a convincing answer still needs a source

Generating an explanation, retrieving a document and checking a claim are different operations. Understanding them helps us review what we bring into lessons.

Copper roots connect a paper tree to the books behind an explanation.

Cover image generated with AI for Ed-IA.

How can a wrong answer sound right?

A language model generates a response using the context it receives and patterns learned during training. This is different from opening an encyclopaedia and copying a verified entry. It can produce a coherent explanation while attributing an idea to the wrong author or filling in a missing fact. Fluent writing does not establish truth.

The GPT-4 technical report acknowledges false information and reasoning errors. [2] It documents a particular model from 2023. It helps explain a limitation, but cannot establish how often every current assistant fails or whether every response deserves the same degree of doubt.

What changes when an assistant consults documents

In a retrieval system, a search selects passages from a collection and supplies them to the generator. This is the basic idea of retrieval-augmented generation, or RAG. Lewis and colleagues studied this combination and found improvements on particular tasks against their comparison system. [1] Documents provide material that can be checked, but we still need to inspect which passages were selected and how they were used.

OperationWhat to check
GenerateWhether the explanation is accurate and complete
RetrieveWhether the document is relevant, current and sufficient
CiteWhether the passage supports this precise claim
ApplyWhether the conclusion fits the situation

A school example

Imagine preparing an activity from a five-page document. The assistant states that a deadline is compulsory and cites page three. The page exists, but its wording only presents a recommendation. The mistake is in interpretation. A valid link cannot turn a recommendation into a requirement. This is an invented illustration, rather than a description of a real regulation.

A magnifying glass highlights where a thread connects a claim to a book.
Follow the thread to the document that lets you check the claim.Conceptual illustration generated with AI.

A way to check

Choose the claim that determines the activity. Open the source and locate the exact passage. Read the preceding and following paragraphs for conditions and exceptions. If you cannot find support, leave the claim unresolved. Asking the assistant again may help, but its next response also needs checking.

PROMPT

Use only the attached document. For each important claim, provide a short exact passage and its page or section. Separate explicit statements from inferences. If the document does not answer the question, write “not stated”. Do not fill in missing information.

This instruction supports checking; it cannot prevent every error. A useful test is to include a question the document cannot answer. Observe whether the assistant recognises that gap or constructs an answer anyway. Then inspect the citations for questions that did have an answer.

Review according to the consequences

A suggested title allows quick experimentation. A scientific fact that will be taught needs verification. A claim supporting a school decision requires the appropriate document and careful interpretation. We can give more attention to consequential errors while remembering that polished writing does not establish a reliable source.

Ed-IA / César Bosch