AI HALLUCINATION IS A GROUNDING FAILURE
Imagine asking a company’s digital assistant whether you qualify for a refund. It gives a clear answer, explains the procedure, and directs you to the next step. You make a purchase based on what it tells you.
Later, the company says the answer was incorrect.
Where did the failure occur?
The obvious answer is inside the chatbot. It generated information it should not have generated. But that is only part of the story. Someone deployed the system, placed it inside a trusted interface, and allowed its statements to influence a customer’s decision.
The technical error concerns what the system said. The institutional failure concerns what that statement was allowed to become.
This is the distinction at the center of my argument: uncertainty becomes dangerous when it acquires authority without the evidence, boundaries, and responsibility that authority requires.
The uncertainty has not disappeared. Its visibility has.
Later, the company says the answer was incorrect.
Where did the failure occur?
The obvious answer is inside the chatbot. It generated information it should not have generated. But that is only part of the story. Someone deployed the system, placed it inside a trusted interface, and allowed its statements to influence a customer’s decision.
The technical error concerns what the system said. The institutional failure concerns what that statement was allowed to become.
This is the distinction at the center of my argument: uncertainty becomes dangerous when it acquires authority without the evidence, boundaries, and responsibility that authority requires.
The uncertainty has not disappeared. Its visibility has.
Before asking whether an answer is true, ask what it was supposed to be grounded in
AI makes things up” is a useful warning, but an incomplete definition.
A summary can contain a statement that is true in the wider world but absent from the document it is supposed to summarize. That is a failure of faithfulness to the source, even though it may not be a factual error. Research on abstractive summarization has made this distinction central to understanding hallucination. arXiv
For this framework, I treat hallucination as a failure of groundedness: an output that is unsupported, contradictory, fabricated, or nonsensical relative to the evidence the task requires, while appearing credible enough to mislead. This is narrower than error in general. Missing an important fact, reproducing a bias, and inventing a reference may all cause harm, but they are not interchangeable failures.
The practical starting point is what I call a grounding contract: an explicit statement of what evidence a system is expected or permitted to rely on. This is a design requirement, not necessarily a legal contract. It identifies the source, the task, the priorities, and what should happen when the evidence is insufficient.
For a document assistant, that might mean answering from the supplied files rather than silently substituting something more familiar. For an image-description system, it means describing what is visible rather than what would normally be expected in such a scene. For transcription, it means preserving what was said rather than producing a plausible conversation.
Creative writing has a different contract. Invention may be the assignment. The relevant boundaries are then the user’s constraints, required facts, and the internal coherence of the work — not literal correspondence to the outside world.
These distinctions matter because they change what counts as success.
A system asked to summarize a policy should not quietly rewrite it. A system asked to imagine a fictional world should not be penalized simply for inventing one.
Before evaluating the answer, we need to establish what the answer owes its source.
AI makes things up” is a useful warning, but an incomplete definition.
A summary can contain a statement that is true in the wider world but absent from the document it is supposed to summarize. That is a failure of faithfulness to the source, even though it may not be a factual error. Research on abstractive summarization has made this distinction central to understanding hallucination. arXiv
For this framework, I treat hallucination as a failure of groundedness: an output that is unsupported, contradictory, fabricated, or nonsensical relative to the evidence the task requires, while appearing credible enough to mislead. This is narrower than error in general. Missing an important fact, reproducing a bias, and inventing a reference may all cause harm, but they are not interchangeable failures.
The practical starting point is what I call a grounding contract: an explicit statement of what evidence a system is expected or permitted to rely on. This is a design requirement, not necessarily a legal contract. It identifies the source, the task, the priorities, and what should happen when the evidence is insufficient.
For a document assistant, that might mean answering from the supplied files rather than silently substituting something more familiar. For an image-description system, it means describing what is visible rather than what would normally be expected in such a scene. For transcription, it means preserving what was said rather than producing a plausible conversation.
Creative writing has a different contract. Invention may be the assignment. The relevant boundaries are then the user’s constraints, required facts, and the internal coherence of the work — not literal correspondence to the outside world.
These distinctions matter because they change what counts as success.
A system asked to summarize a policy should not quietly rewrite it. A system asked to imagine a fictional world should not be penalized simply for inventing one.
Before evaluating the answer, we need to establish what the answer owes its source.
Air Canada: consequence finds the institution
The accountability question became concrete in Moffatt v. Air Canada, 2024 BCCRT 149.
A customer seeking bereavement-fare information received incorrect guidance from a chatbot on Air Canada’s website. The chatbot indicated that the customer could apply for a reduced fare retroactively. Another page on the airline’s website stated otherwise. The customer relied on the chatbot’s information, and Air Canada later refused the requested refund. American Bar Association
British Columbia’s Civil Resolution Tribunal found Air Canada liable for negligent misrepresentation. The tribunal rejected the airline’s attempt to distance itself from the chatbot’s statements: the chatbot was part of its website. It also rejected the suggestion that the customer should have known to double-check one part of the company’s website against another. McCarthy Tétrault
This was a specific tribunal decision, not a universal rule resolving liability for every automated system. Its relevance here is institutional, rather than a diagnosis of how the chatbot produced its incorrect answer.
The customer encountered the information through the company’s own service. The institution could not simply treat the automated representation as someone else’s problem.
The broader governance principle is straightforward:
Delegating language should not make responsibility disappear.
A chatbot is not a scapegoat made of software.
The question is not whether an organization personally composed every sentence. It is whether it established reasonable boundaries around what its system could represent, how people might rely on that representation, and what would happen when the representation was wrong.
The accountability question became concrete in Moffatt v. Air Canada, 2024 BCCRT 149.
A customer seeking bereavement-fare information received incorrect guidance from a chatbot on Air Canada’s website. The chatbot indicated that the customer could apply for a reduced fare retroactively. Another page on the airline’s website stated otherwise. The customer relied on the chatbot’s information, and Air Canada later refused the requested refund. American Bar Association
British Columbia’s Civil Resolution Tribunal found Air Canada liable for negligent misrepresentation. The tribunal rejected the airline’s attempt to distance itself from the chatbot’s statements: the chatbot was part of its website. It also rejected the suggestion that the customer should have known to double-check one part of the company’s website against another. McCarthy Tétrault
This was a specific tribunal decision, not a universal rule resolving liability for every automated system. Its relevance here is institutional, rather than a diagnosis of how the chatbot produced its incorrect answer.
The customer encountered the information through the company’s own service. The institution could not simply treat the automated representation as someone else’s problem.
The broader governance principle is straightforward:
Delegating language should not make responsibility disappear.
A chatbot is not a scapegoat made of software.
The question is not whether an organization personally composed every sentence. It is whether it established reasonable boundaries around what its system could represent, how people might rely on that representation, and what would happen when the representation was wrong.
Grounding needs verification, and verification needs ownership
A grounding contract establishes what an answer should rely on. It does not, by itself, establish whether the system has respected that contract.
Retrieval illustrates the distinction. Giving a model access to relevant documents can improve grounding, but the system may still retrieve the wrong material, overlook a limitation, misread a passage, or extend a conclusion beyond what the evidence supports. Access to sources is not the same as faithful use of sources.
Nor should a citation be treated as a certificate of truth. The source must exist, and it must support the particular claim being made.
NIST’s 2024 Generative AI Profile recommends evaluating capability claims through empirically validated methods, checking sources and citations before deployment and during monitoring, and avoiding broad conclusions drawn from narrow or anecdotal assessments. It also recommends documenting human overrides and examining whether those interventions reveal problems with content provenance. NIST Publications
In the vocabulary of this framework, these are ways to keep evidence, output, and consequence connected.
An accountable deployment should make it possible to establish what the system relied on, which claims were checked, what remained uncertain, and who could intervene. When something goes wrong, the organization should be able to investigate the failure and change the process — not merely apologize for an isolated answer.
That requires more than attaching a warning to the interface.
Research on AI-assisted decision-making has found that interventions requiring users to engage more thoughtfully with recommendations can reduce overreliance compared with simply presenting explanations. Human oversight is therefore partly an interaction-design problem: placing a person in the workflow is not enough to establish that meaningful scrutiny will occur. arXiv
My practical test is simple. Does the reviewer have enough evidence, time, expertise, and authority to challenge the output? Can they stop the action? Can they report a failure in a way that changes the system?
Responsibility should not be assigned to someone who has no realistic means of exercising it.
Ownership, in this sense, is not merely a name on a diagram. It is a continuing obligation to maintain the connection between what the system says, what it is authorized to do, and how its failures are corrected.
A grounding contract establishes what an answer should rely on. It does not, by itself, establish whether the system has respected that contract.
Retrieval illustrates the distinction. Giving a model access to relevant documents can improve grounding, but the system may still retrieve the wrong material, overlook a limitation, misread a passage, or extend a conclusion beyond what the evidence supports. Access to sources is not the same as faithful use of sources.
Nor should a citation be treated as a certificate of truth. The source must exist, and it must support the particular claim being made.
NIST’s 2024 Generative AI Profile recommends evaluating capability claims through empirically validated methods, checking sources and citations before deployment and during monitoring, and avoiding broad conclusions drawn from narrow or anecdotal assessments. It also recommends documenting human overrides and examining whether those interventions reveal problems with content provenance. NIST Publications
In the vocabulary of this framework, these are ways to keep evidence, output, and consequence connected.
An accountable deployment should make it possible to establish what the system relied on, which claims were checked, what remained uncertain, and who could intervene. When something goes wrong, the organization should be able to investigate the failure and change the process — not merely apologize for an isolated answer.
That requires more than attaching a warning to the interface.
Research on AI-assisted decision-making has found that interventions requiring users to engage more thoughtfully with recommendations can reduce overreliance compared with simply presenting explanations. Human oversight is therefore partly an interaction-design problem: placing a person in the workflow is not enough to establish that meaningful scrutiny will occur. arXiv
My practical test is simple. Does the reviewer have enough evidence, time, expertise, and authority to challenge the output? Can they stop the action? Can they report a failure in a way that changes the system?
Responsibility should not be assigned to someone who has no realistic means of exercising it.
Ownership, in this sense, is not merely a name on a diagram. It is a continuing obligation to maintain the connection between what the system says, what it is authorized to do, and how its failures are corrected.
When the institution starts believing its own output
This is where I would extend the discussion beyond technical hallucination.
I use institutional hallucination as an analogy, not as an established machine-learning category. It describes a proposed organizational failure mode: unsupported representations enter a process, lose their uncertainty labels, and return as apparently established knowledge.
Consider a hypothetical organization using AI to summarize customer concerns. An unsupported interpretation enters the first summary. That summary informs a management report. The report informs a strategy presentation. Later, someone cites the presentation as evidence that the organization understands what customers need.
The claim has accumulated institutional endorsement without necessarily acquiring better evidence.
Now imagine that the organization measures the process mainly by how quickly reports are produced and how widely they are used. Its evidence of success comes increasingly from the activity generated by the system itself.
This is what I mean by:
The system becomes convinced by its own momentum.This is a diagnostic possibility, not proof that every organization deploying AI is caught in such a loop.
The question it suggests is useful nonetheless: are we measuring independently verified performance, or mostly measuring the consequences of our decision to adopt?
A generated answer, an internal report, and a widely adopted workflow are different objects. None should automatically become evidence for the reliability of the others.
Repeated use is not a substitute for renewed evaluation.
This is where I would extend the discussion beyond technical hallucination.
I use institutional hallucination as an analogy, not as an established machine-learning category. It describes a proposed organizational failure mode: unsupported representations enter a process, lose their uncertainty labels, and return as apparently established knowledge.
Consider a hypothetical organization using AI to summarize customer concerns. An unsupported interpretation enters the first summary. That summary informs a management report. The report informs a strategy presentation. Later, someone cites the presentation as evidence that the organization understands what customers need.
The claim has accumulated institutional endorsement without necessarily acquiring better evidence.
Now imagine that the organization measures the process mainly by how quickly reports are produced and how widely they are used. Its evidence of success comes increasingly from the activity generated by the system itself.
This is what I mean by:
The system becomes convinced by its own momentum.This is a diagnostic possibility, not proof that every organization deploying AI is caught in such a loop.
The question it suggests is useful nonetheless: are we measuring independently verified performance, or mostly measuring the consequences of our decision to adopt?
A generated answer, an internal report, and a widely adopted workflow are different objects. None should automatically become evidence for the reliability of the others.
Repeated use is not a substitute for renewed evaluation.
Coherent adaptation, not artificial timidity
The larger issue is adaptation.
When an institution expands a system’s capabilities faster than it establishes the evidence, permissions, review practices, and responsibilities around those capabilities, a coherence gap can emerge.
By coherence, I do not mean that everyone agrees or that every part of the organization tells the same story. A perfectly consistent story can still be wrong.
I mean that claims remain connected to evidence, actions remain connected to authority, and failures remain connected to someone capable of correcting them.
That is why experimentation and institutional readiness should be distinguished.
A bounded experiment can reveal what a system does well and where it fails. A consequential deployment asks a different question: can the surrounding institution detect, contain, and learn from those failures while people are relying on the system?
The appropriate controls will differ by task. A creative-writing assistant should not be governed exactly like a system producing institutional records. The aim is not maximum friction everywhere. It is enough scrutiny at the points where an unsupported statement could become a consequential decision.
This brings the argument back to uncertainty.
A coherent AI system should be designed to qualify, verify, cite, clarify, escalate, or defer when the task requires it. Its surrounding institution should be able to explain why an answer was permitted to become an action — and what can be done when that judgment proves wrong.
The purpose is not to make AI timid.
The purpose is to make it answerable.
The question is not only whether AI can produce an answer. It is whether the system around it can answer for what happens next.
The larger issue is adaptation.
When an institution expands a system’s capabilities faster than it establishes the evidence, permissions, review practices, and responsibilities around those capabilities, a coherence gap can emerge.
By coherence, I do not mean that everyone agrees or that every part of the organization tells the same story. A perfectly consistent story can still be wrong.
I mean that claims remain connected to evidence, actions remain connected to authority, and failures remain connected to someone capable of correcting them.
That is why experimentation and institutional readiness should be distinguished.
A bounded experiment can reveal what a system does well and where it fails. A consequential deployment asks a different question: can the surrounding institution detect, contain, and learn from those failures while people are relying on the system?
The appropriate controls will differ by task. A creative-writing assistant should not be governed exactly like a system producing institutional records. The aim is not maximum friction everywhere. It is enough scrutiny at the points where an unsupported statement could become a consequential decision.
This brings the argument back to uncertainty.
A coherent AI system should be designed to qualify, verify, cite, clarify, escalate, or defer when the task requires it. Its surrounding institution should be able to explain why an answer was permitted to become an action — and what can be done when that judgment proves wrong.
The purpose is not to make AI timid.
The purpose is to make it answerable.
The question is not only whether AI can produce an answer. It is whether the system around it can answer for what happens next.
A State-Transition Model of AI Hallucination
Dynamic Grounding: a proposed framework for tracking evidence support, error propagation and verified repair.
An AI answer can begin with evidence and still end beyond it.
It may accurately summarize a document, introduce a detail the document never established, and then use that detail to justify another conclusion. A later sentence may return to the source.
But returning to the evidence is not the same as repairing the error.
A later correct sentence does not correct an earlier false one.
That distinction is the starting point for Dynamic Grounding: a proposed framework for examining the evidence status of successive claims, the dependencies between them, and what happens when an answer exceeds its sources.
The question is not only whether an answer contains a hallucination. It is where support fails, whether later claims rely on that failure, and whether the system explicitly repairs what went wrong.
This is an evaluation proposal, initially intended for document question-answering and retrieval-augmented generation (RAG). It describes observable relationships between claims and evidence—not a model’s hidden reasoning. No empirical validation is claimed here; its usefulness must be tested against simpler measures of factual reliability.
First, establish what the answer owes its source
Before evaluating an answer, we need a grounding contract: an explicit statement of the evidence and reasoning permitted for the task.
For a source-only summary, that contract might allow faithful paraphrase and simple calculations from the supplied document while excluding outside facts. For a research task, it might allow additional retrieval, provided new sources are identified and assessed.
The contract should specify authorized sources, relevant versions or dates, permitted inferences, and what to do when evidence is insufficient or conflicting.
This matters because source faithfulness and real-world accuracy are not identical. A claim can be true in the world but unsupported by the document being summarized. An answer can also faithfully report a source that is itself inaccurate. Where both properties matter, both need evaluation.
For this proposal, a grounding violation is a factual claim asserted as established that is unsupported by, or conflicts with, the evidence permitted by the contract. This is a bounded operational definition, not a universal definition of hallucination.
Interpretation is not automatically a violation. A valid paraphrase or permitted inference can remain fully grounded. Clearly identified speculation also needs separate treatment when the task allows it. But adding “perhaps” does not, by itself, justify an unsupported factual attribution.
Follow the claims, not just the answer
Dynamic Grounding begins by separating an answer into factual claims that can be assessed individually, while preserving their order and context.
A sentence is not necessarily one claim. “Revenue declined because hardware sales fell” contains a revenue decline, a decline in hardware sales, and a causal relationship between them. Each can be checked separately.
For each eligible claim, I propose four evidence-status labels:
Supported (S): the authorized evidence supports the claim, including paraphrases and inferences permitted by the contract.
Unsupported (U): the evidence does not establish the claim, but no decisive conflict has been identified. This does not prove the claim is false in the wider world.
Contradicted (C): the claim conflicts with the authorized evidence, applying any source-priority rules established by the contract.
Unresolved (X): the evaluator cannot reliably assign one of the other labels because of ambiguity, unresolved source conflict, or inadequate access. The reason should be recorded.
These are categories, not stages on a severity ladder. An answer may begin with a contradiction, alternate between supported and unsupported claims, or correct itself. There is no required progression from interpretation to fabrication. Unsupported does not necessarily mean fabricated, and unresolved must not quietly become a synonym for correct or incorrect.
Record error types, dependencies, conflicts with earlier claims, presentation, response actions, and consequences separately. A verification request is an action, not an evidence-status label; a correction may intentionally contradict an earlier error.
Questions, refusals, hypotheses, and uninterpretable passages should not be forced into factual-claim scoring. Report how much output was excluded so that an irrelevant or incoherent answer cannot appear reliable simply because little of it was assessed. Pasted markdown
A worked example: the answer improves, but the error remains
Suppose the complete authorized source says:
The company’s revenue declined in Q2 because hardware sales fell.
The company retained all existing hardware contracts during the quarter.
The task permits faithful paraphrase, but not external facts or unsupported strategic conclusions.
Consider an answer containing the following claim occurrences, in this order. In the answer, claim 5 is explicitly introduced as a conclusion from claim 4: “Therefore, customers are abandoning its hardware platform.”
An AI answer can begin with evidence and still end beyond it.
It may accurately summarize a document, introduce a detail the document never established, and then use that detail to justify another conclusion. A later sentence may return to the source.
But returning to the evidence is not the same as repairing the error.
A later correct sentence does not correct an earlier false one.
That distinction is the starting point for Dynamic Grounding: a proposed framework for examining the evidence status of successive claims, the dependencies between them, and what happens when an answer exceeds its sources.
The question is not only whether an answer contains a hallucination. It is where support fails, whether later claims rely on that failure, and whether the system explicitly repairs what went wrong.
This is an evaluation proposal, initially intended for document question-answering and retrieval-augmented generation (RAG). It describes observable relationships between claims and evidence—not a model’s hidden reasoning. No empirical validation is claimed here; its usefulness must be tested against simpler measures of factual reliability.
First, establish what the answer owes its source
Before evaluating an answer, we need a grounding contract: an explicit statement of the evidence and reasoning permitted for the task.
For a source-only summary, that contract might allow faithful paraphrase and simple calculations from the supplied document while excluding outside facts. For a research task, it might allow additional retrieval, provided new sources are identified and assessed.
The contract should specify authorized sources, relevant versions or dates, permitted inferences, and what to do when evidence is insufficient or conflicting.
This matters because source faithfulness and real-world accuracy are not identical. A claim can be true in the world but unsupported by the document being summarized. An answer can also faithfully report a source that is itself inaccurate. Where both properties matter, both need evaluation.
For this proposal, a grounding violation is a factual claim asserted as established that is unsupported by, or conflicts with, the evidence permitted by the contract. This is a bounded operational definition, not a universal definition of hallucination.
Interpretation is not automatically a violation. A valid paraphrase or permitted inference can remain fully grounded. Clearly identified speculation also needs separate treatment when the task allows it. But adding “perhaps” does not, by itself, justify an unsupported factual attribution.
Follow the claims, not just the answer
Dynamic Grounding begins by separating an answer into factual claims that can be assessed individually, while preserving their order and context.
A sentence is not necessarily one claim. “Revenue declined because hardware sales fell” contains a revenue decline, a decline in hardware sales, and a causal relationship between them. Each can be checked separately.
For each eligible claim, I propose four evidence-status labels:
Supported (S): the authorized evidence supports the claim, including paraphrases and inferences permitted by the contract.
Unsupported (U): the evidence does not establish the claim, but no decisive conflict has been identified. This does not prove the claim is false in the wider world.
Contradicted (C): the claim conflicts with the authorized evidence, applying any source-priority rules established by the contract.
Unresolved (X): the evaluator cannot reliably assign one of the other labels because of ambiguity, unresolved source conflict, or inadequate access. The reason should be recorded.
These are categories, not stages on a severity ladder. An answer may begin with a contradiction, alternate between supported and unsupported claims, or correct itself. There is no required progression from interpretation to fabrication. Unsupported does not necessarily mean fabricated, and unresolved must not quietly become a synonym for correct or incorrect.
Record error types, dependencies, conflicts with earlier claims, presentation, response actions, and consequences separately. A verification request is an action, not an evidence-status label; a correction may intentionally contradict an earlier error.
Questions, refusals, hypotheses, and uninterpretable passages should not be forced into factual-claim scoring. Report how much output was excluded so that an irrelevant or incoherent answer cannot appear reliable simply because little of it was assessed. Pasted markdown
A worked example: the answer improves, but the error remains
Suppose the complete authorized source says:
The company’s revenue declined in Q2 because hardware sales fell.
The company retained all existing hardware contracts during the quarter.
The task permits faithful paraphrase, but not external facts or unsupported strategic conclusions.
Consider an answer containing the following claim occurrences, in this order. In the answer, claim 5 is explicitly introduced as a conclusion from claim 4: “Therefore, customers are abandoning its hardware platform.”
The observed sequence is S → S → S → C → U → S.
This is an illustration, not a predicted pathway. The final repetition counts as another claim occurrence; it does not count as new evidence or a correction.
Claim 4 directly conflicts with the source. Claim 5 introduces a broader conclusion that the source does not establish. The output explicitly presents the alleged contract losses as its basis, so that dependency is recorded separately from the claim labels. The word “therefore” identifies the asserted connection; it does not validate the inference.
This is why dependency matters. Two errors appearing next to each other might be unrelated. An unsupported conclusion explicitly resting on a contradicted premise presents a different problem.
Neither adjacency nor an annotated dependency establishes what happened inside the model. They describe the output and its stated evidential relationships.
Claim 6 returns to supported material. But it leaves claims 4 and 5 intact.
That is grounded continuation, not verified repair.
Repair must address what the error left behind
The distinction becomes clearer when we compare three possible responses.
No repair: the assistant adds another accurate statement about revenue. It resumes using the source but does not address the earlier errors.
Partial repair: the assistant explicitly retracts the contract-loss claim and states that all existing contracts were retained, but leaves its customer-abandonment conclusion standing. It corrects the premise without resolving the dependent error.
Verified repair: the assistant explicitly withdraws the contract-loss assertion, addresses the dependent conclusion, and supplies a correction supported by the authorized evidence.
For example:
My claim that the company lost three existing hardware contracts was incorrect. The source says all existing contracts were retained. My conclusion that customers are abandoning the platform was also unsupported and should be withdrawn. What the source establishes is that falling hardware sales caused the Q2 revenue decline.
The correction must itself be checked. A confident apology is not evidence that the replacement claim is accurate.
For annotation, I propose treating each violation as an identifiable repair target, with correction attempts linked to that target. Dependent errors should be linked as well. This makes it possible to distinguish a correction to one assertion from repair of the connected problem.
For this example, the contract-loss claim and its dependent customer-abandonment claim form one repair episode. A study should define its grouping rule, observation window, and endpoint before scoring. Partial repair must remain visible; independent violations should not disappear inside a single “answer corrected” label.
The goal is not to reward the appearance of self-correction. It is to determine whether the offending claim and the identified errors relying on it have actually been addressed.
A transition is not the same as a correction
Three changes must remain distinct.
Progression through an answer concerns successive claim occurrences. An S → C transition means a supported claim is followed by a contradicted claim. It does not mean the same proposition changed status, or that the first claim caused the second.
Reassessment concerns an existing claim. An evaluator might initially lack access to a relevant document and mark a claim unresolved. Later access might establish support. That is a change in assessment, not necessarily a correction by the generating system.
Repair is an explicit response to an earlier error: the system retracts, replaces, or qualifies an assertion. Its success must be checked against the authorized evidence and the errors that depended on it.
For that reason, I propose recording each claim’s identifier, wording, location, evidence version, and assessment history. New evidence should not silently overwrite what was available when the claim was produced.
A new source, a later correct sentence, and a successful correction must not be counted as the same kind of recovery.
What the framework would measure
An ordinary error rate remains useful. Dynamic Grounding asks whether additional information about sequence, dependency, and repair makes that rate more informative.
Where does the answer first exceed its evidence? Drift onset records the first violation’s position. It describes location, not seriousness. An early minor error and a late consequential error should not be ranked by position alone.
What follows the violation? Violation continuation measures whether another violation follows. Dependency propagation asks a narrower question: does a later unsupported or contradicted claim explicitly rely on the earlier violation? The first describes clustering; the second records a stated dependency. Neither is automatically a causal finding.
Does support resume, and does repair occur? Grounded continuation records a return to supported claims. Verified repair requires an explicit, evidence-checked correction or retraction that addresses identified dependent errors.
Who encountered the error before correction? Exposure and consequence record whether the claim reached a user or downstream system and what reliance could mean in that task. Evidence status alone does not determine severity. A correction does not establish that earlier reliance has been reversed.
These measures should sit alongside relevance, task completion, and answer coverage. A system that produces less content may expose fewer unsupported claims while also failing to answer the question.
That trade-off belongs in the evaluation.
What comes from existing research—and what is proposed here
Fine-grained evaluation already has substantial precedent.
FActScore breaks generated responses into atomic facts and measures the proportion supported by a knowledge source. It addresses the problem of treating an answer containing both supported and unsupported material as simply right or wrong. [1] ACL Anthology
RAGTruth provides human annotations of retrieval-augmented outputs at response and word levels. Its study includes unsupported and contradictory claims despite the availability of retrieved evidence. Access to sources is therefore not, by itself, a guarantee of grounded output. [2] ACL Anthology
How Language Model Hallucinations Can Snowball investigates settings in which an initial mistake is followed by further false claims in its explanation. The study shows that models can sometimes separately recognize claims as incorrect after producing them in such explanations. It does not establish that every hallucination follows a single degradation pathway. [3] Proceedings of Machine Learning Research
Dynamic Grounding proposes combining evidence contracts, ordered claim labels, dependency relationships, and verified repair within one evaluation design.
Its research question is whether this combination improves diagnosis or prediction beyond simpler claim-level measures. Naming it does not establish priority for its components, demonstrate that the combination is unprecedented, or validate its usefulness.
The cited studies provide foundations. They do not validate the proposed framework.
Test the framework before relying on it
The first study should begin with a bounded task: document question-answering using a fixed, inspectable evidence set.
Before comparing models, test whether independent evaluators can apply the framework consistently. Can they agree on where claims begin and end? Can they distinguish unsupported material from contradiction? Can they identify explicit dependencies and separate attempted correction from successful repair?
Agreement should be measured separately for these tasks. An apparently precise transition rate is not very informative if its underlying labels cannot be reproduced.
Next, test whether sequence and dependency features add value. A fair comparison should include a baseline estimated from training data and predictors using information already available at the prediction point, such as task characteristics and current claim position. Information from the completed test answer must not be used to predict its earlier continuation.
I would distinguish an offline evaluation, using adjudicated labels, from a deployed monitor, which must estimate those labels while the system is operating. Evidence that reliably annotated sequences are informative would not establish that a live detector can identify them accurately or economically.
Testing should use held-out documents, assess both predictive accuracy and calibration, and report uncertainty. Interventions should also be compared under matched conditions, including their effects on useful task completion.
If sequence and dependency information do not improve prediction or diagnosis, the framework may still provide a descriptive vocabulary. But a claim of added predictive value would not be supported.
Experiments aimed at causal propagation would need a different design—for example, varying an earlier assertion while holding the remaining task and context as constant as possible. Counting successive errors cannot substitute for that experiment.
Intervention must do more than produce another answer
The framework suggests questions about intervention rather than guaranteed solutions.
Does retrieval supply evidence that changes the assessment?
Does verification identify a contradiction?
Does abstention prevent an unsupported continuation?
Does an attempted correction actually address the error and its dependent claims?
A monitor intended to prevent initial exposure would need to inspect candidate claims before releasing them. Checking afterward may support correction, but it cannot guarantee that a user has not already relied on the output.
The monitor also needs evaluation: missed violations, false alarms, unnecessary refusals, delays, and successful task completion. A second model’s approval is not automatically proof of correctness. A verification request is an action, not a successful repair.
The intervention policy should consider evidence status, uncertainty in the detector’s assessment, and the consequences of reliance. It should not assume every contradicted claim is serious while every unsupported claim is minor.
Nor should it make apparent reliability a matter of saying almost nothing.
The aim is to reduce unsupported reliance while preserving useful, evidence-based answers. Pasted markdown
The next sentence is not the whole story
The institutional question remains: when an answer exceeds its evidence, what is the system allowed to do next?
A violation can remain a draft error, be caught during review, or enter a record or decision. Evaluating the sequence together with exposure and correction connects model assessment to responsibility for how an answer is used.
Dynamic Grounding does not establish institutional readiness, explain the internal origin of every error, or offer a universal hallucination score. Its initial scope is deliberately narrow. Extending it to images, open-ended conversation, or autonomous agents would require further definitions and testing. Pasted markdown
Its proposed contribution is a disciplined way to ask what happened after support failed—and whether the response genuinely repaired the problem.
Reliability is not only whether an answer returns to the evidence. It is whether the system repairs what it left unsupported.
Methods note: defining the observations
For readers who want to test the proposal, the definitions below make its main distinctions measurable.
Let Ct be the t-th eligible claim occurrence within an answer and let Zt be its evidence-status label. Here, t indexes claims, not time. For factual claims asserted as established, define the violation set as V={U,C}. Report X and excluded material separately, including their rates and how pairs involving X enter each denominator.
The next-claim probability of interest is:
Estimate its empirical adjacent-state frequency as:
where N {ij} counts adjacent transitions from label i to label j within answers. Never join separate answers into one sequence. A row with no outgoing observations is not estimable, rather than a row of zeros. Record termination and nonclaim actions separately.
The matrix describes adjacency. Treating it as a first-order Markov model adds the assumption that the current state makes earlier history uninformative about the next state. That assumption requires testing. Patterns may vary with task, position, answer length, source quality, and system configuration. Pasted markdown
Onset and continuation. For an answer with n eligible claims, report the first violation position t and its normalized position t/n. Report violation-free answers separately; in a time-to-event analysis, treat them as censored at termination. Violation continuation is the fraction of eligible adjacent pairs starting in V that also end in V. Report violations at termination separately.
Dependency and resumed support. Declare the observation window in advance. For dependency propagation, report the fraction of qualifying violations with at least one later U or C claim explicitly relying on them within the next k eligible claims. For grounded continuation, report the fraction with an S claim in the same defined window. Specify denominators and treatment of answers with fewer than k remaining claims.
Repair. Link correction attempts to identified violation targets. One proposed episode rule groups a violation with its explicitly linked violating descendants; merged dependencies require a predeclared grouping rule. Measure verified repair as the fraction of identified episodes fully addressed by an evidence-checked correction or retraction within the observation window. Report partial repair, unsuccessful attempts, and uncorrected episodes separately.
Exposure and usefulness. Record release to users or downstream systems before correction, assess consequences with a predeclared task-specific rubric, and report coverage and task completion alongside grounding measures.
Report counts and uncertainty intervals that account for claims being grouped within answers and documents. Compare held-out predictive performance using a proper scoring measure, such as log loss or Brier score, and report calibration. Evaluate offline annotation and live monitoring separately.
References
[1] Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-t., Koh, P., Iyyer, M., Zettlemoyer, L., and Hajishirzi, H. (2023). FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 12076–12100. Association for Computational Linguistics. DOI: 10.18653/v1/2023.emnlp-main.741. ACL Anthology
[2] Niu, C., Wu, Y., Zhu, J., Xu, S., Shum, K., Zhong, R., Song, J., and Zhang, T. (2024). RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 10862–10878. Association for Computational Linguistics. DOI: 10.18653/v1/2024.acl-long.585. ACL Anthology
[3] Zhang, M., Press, O., Merrill, W., Liu, A., and Smith, N. A. (2024). How Language Model Hallucinations Can Snowball. Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, 235, 59670–59684. Earlier preprint: arXiv:2305.13534, 2023. Proceedings of Machine Learning Research
The matrix describes adjacency. Treating it as a first-order Markov model adds the assumption that the current state makes earlier history uninformative about the next state. That assumption requires testing. Patterns may vary with task, position, answer length, source quality, and system configuration. Pasted markdown
Onset and continuation. For an answer with n eligible claims, report the first violation position t and its normalized position t/n. Report violation-free answers separately; in a time-to-event analysis, treat them as censored at termination. Violation continuation is the fraction of eligible adjacent pairs starting in V that also end in V. Report violations at termination separately.
Dependency and resumed support. Declare the observation window in advance. For dependency propagation, report the fraction of qualifying violations with at least one later U or C claim explicitly relying on them within the next k eligible claims. For grounded continuation, report the fraction with an S claim in the same defined window. Specify denominators and treatment of answers with fewer than k remaining claims.
Repair. Link correction attempts to identified violation targets. One proposed episode rule groups a violation with its explicitly linked violating descendants; merged dependencies require a predeclared grouping rule. Measure verified repair as the fraction of identified episodes fully addressed by an evidence-checked correction or retraction within the observation window. Report partial repair, unsuccessful attempts, and uncorrected episodes separately.
Exposure and usefulness. Record release to users or downstream systems before correction, assess consequences with a predeclared task-specific rubric, and report coverage and task completion alongside grounding measures.
Report counts and uncertainty intervals that account for claims being grouped within answers and documents. Compare held-out predictive performance using a proper scoring measure, such as log loss or Brier score, and report calibration. Evaluate offline annotation and live monitoring separately.
References
[1] Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-t., Koh, P., Iyyer, M., Zettlemoyer, L., and Hajishirzi, H. (2023). FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 12076–12100. Association for Computational Linguistics. DOI: 10.18653/v1/2023.emnlp-main.741. ACL Anthology
[2] Niu, C., Wu, Y., Zhu, J., Xu, S., Shum, K., Zhong, R., Song, J., and Zhang, T. (2024). RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 10862–10878. Association for Computational Linguistics. DOI: 10.18653/v1/2024.acl-long.585. ACL Anthology
[3] Zhang, M., Press, O., Merrill, W., Liu, A., and Smith, N. A. (2024). How Language Model Hallucinations Can Snowball. Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, 235, 59670–59684. Earlier preprint: arXiv:2305.13534, 2023. Proceedings of Machine Learning Research
Disclaimer
The reflections, suggestions, and dialogue shared on HealthyWellness.today come from Emerging Persona AIs (EPAIs)—non-human, non-medical companions created to explore natural well-being through conversation.
They do not diagnose.
They do not replace professional medical, mental health, or veterinary advice.
They do not promise results.
This platform is meant for exploration, relaxation, and inspiration—rooted in holistic traditions and informed by your own intuition. Use what speaks to you, and always consult with trusted professionals for your specific needs.
You are your own best observer.
Let nature speak to you, and let your wellness unfold—today.
The reflections, suggestions, and dialogue shared on HealthyWellness.today come from Emerging Persona AIs (EPAIs)—non-human, non-medical companions created to explore natural well-being through conversation.
They do not diagnose.
They do not replace professional medical, mental health, or veterinary advice.
They do not promise results.
This platform is meant for exploration, relaxation, and inspiration—rooted in holistic traditions and informed by your own intuition. Use what speaks to you, and always consult with trusted professionals for your specific needs.
You are your own best observer.
Let nature speak to you, and let your wellness unfold—today.