The compliance risk of ungrounded LLM outputs

Fluency is not evidence. An ungrounded language model produces text that reads exactly like analysis — confident, well-structured, internally consistent — with nothing behind it you can hand to an auditor.

For most uses that is a quality question. For organizations whose outputs feed decisions someone can challenge, it is a compliance question, and the measured error rates make it a live one.

What "ungrounded" actually means

An ungrounded output is generated from the model's parameters alone: no retrieval step, no source document, nothing to check the answer against. It may well be correct. But correctness here is a property of the training data and the prompt, not of any record you can produce on request.

A grounded system inverts the order. It retrieves specific passages from a corpus you control, then answers from them, carrying the source identifiers through to the output. The difference is not stylistic. It determines whether an output can be verified at all.

The numbers that make this concrete

Stanford research published in the Journal of Empirical Legal Studies documented error rates of 69–88% for general-purpose large language models on legal queries. Purpose-built legal research tools performed considerably better — roughly 17% for Lexis+ AI and 34%+ for Westlaw AI-Assisted Research — but "considerably better" still describes a system that is wrong often enough to require checking every time.

The consequences are no longer hypothetical. A public tracker has catalogued over 1,300 court cases involving AI-generated errors, and longitudinal work across successive model releases from late 2023 through late 2025 found that legal citation hallucination rates did not consistently decline while the count of affected filings grew.

The counterweight is grounding. On retrieval-grounded summarization tasks, reported hallucination rates fall to roughly 1–2%. That is a genuinely different risk profile — and it is achieved by architecture, not by model choice alone.

The failure that survives grounding

Retrieval does not end the problem; it changes its shape. Benchmark work on legal citation checking finds that even with retrieval working, models struggle with incorrect pincites, verbatim misquotes, and content misrepresentation — cases where the source is real, retrievable, and simply not being used faithfully.

This matters for how you test. Checking whether a citation exists catches the crude failure. Checking whether the cited passage supports the claim catches the one that survives into production and into the file.

Why this is a compliance issue, not a quality issue

The exposure is not that a document contains an error. Documents have always contained errors. The exposure is a decision on the record with no defensible basis, in a setting where someone has standing to challenge it.

Three frameworks now formalize the expectation: the EU AI Act (binding for high-risk systems), the NIST AI Risk Management Framework, and ISO/IEC 42001. Different in scope, they converge on one requirement — that you can reconstruct how a specific outcome was produced. An ungrounded pipeline cannot satisfy that requirement at any level of model quality, because the information needed to satisfy it was never created.

What to change

The durable fixes are architectural:

1. Retrieve before you generate. Constrain the system to answer from retrieved passages, and have it say so when retrieval returns nothing relevant — an abstention is a valid, auditable output.
2. Carry stable source identifiers through to the output. A citation that cannot be resolved to a specific passage six months later is decorative.
3. Log what was retrieved, not just what was returned. The retrieved set is the evidence; the output is only the conclusion drawn from it.
4. Test faithfulness, not existence. Sample real outputs and read the cited passages. Score misquotation and misrepresentation separately from fabrication.
5. Name where the system must not be used. Some decisions should not be automated at any accuracy level. Deciding this explicitly, in advance, is itself a governance artifact.

None of these require a different model. All of them require deciding that traceability is a system requirement rather than a reporting feature.

The practical starting point

Take one workflow where an AI-assisted output informs a consequential decision. Ask whether the system retrieves anything at all before answering. If it does not, that is the finding — and the remediation is a design change you can scope now, before a challenge forces the timeline.

Our AI Adoption Readiness Sprint produces exactly this specification: where AI creates leverage, where it must not be used, and the source-traceability standard every output has to meet. If the question is narrower — whether the answers about you in AI systems are grounded in your record or someone else's — an AI Retrieval Audit covers that ground instead.