How to reduce LLM hallucinations in production

Every team that ships an LLM feature eventually gets the same bug report: the assistant stated something with total confidence, and it was wrong. A refund policy that does not exist. An API parameter that was never part of the product. A contract clause summarised into its opposite. The model was not malfunctioning. It did exactly what language models do: produce the most plausible continuation of the text it was given.

That framing matters, because it tells you where the fix lives. You will not find a magic sentence for the system prompt that makes a model honest. What works is engineering: give the model the facts, force it to show where each claim came from, constrain what it can output, make "I don't know" an acceptable answer and check the result before anyone sees it. Each layer catches failures the previous one missed.

Understand which hallucinations you actually have

"Hallucination" covers several different failures, and each has a different remedy. Before changing anything, pull a sample of bad outputs from your logs and sort them:

• Missing knowledge: the answer was never in the context, so the model filled the gap from its training data or invented something. Fix with retrieval and abstention.
• Ignored context: the right document was retrieved, but the model answered from memory instead. Fix with instructions, citations and verification.
• Distortion: the model used the source but changed a number, merged two items or flipped a condition. Fix with quoting, structured extraction and checks.
• Fabricated structure: invented fields, IDs, URLs or function names. Fix with constrained outputs and validation against real data.

This sort is also the beginning of your eval set. Every misclassified example you find becomes a test case that proves a later fix actually worked.

Ground every answer in sources the model can see

The single biggest lever is to stop asking the model to remember and start asking it to read. For anything factual about your business, retrieve the relevant documents and put them in the prompt with stable identifiers. Then tell the model plainly that it must answer only from those sources.

Instructions alone are not enough, so make grounding checkable. Require the model to attach a source ID to every claim, and ideally a short verbatim quote. A quote is valuable because you can verify it mechanically: if the quoted text does not appear in the cited document, the claim is suspect. This turns a vague quality problem into a string comparison.

Constrain the output and make abstention a valid answer

Free-form prose is the hardest thing to verify. Wherever you can, ask for structured output with a schema: an answer, a list of citations and an explicit status field. Most model APIs now support JSON schemas or tool definitions that enforce the shape. Once the shape is guaranteed, you can validate the content.

The status field is where abstention lives. Models hallucinate partly because the prompt implies that an answer is always expected. Give them a sanctioned way out, such as "status": "insufficient_context", and treat that outcome as a success in your product: route to a human, show a search result or ask a clarifying question. An honest "I can't find that in our documentation" is far cheaper than a confident fabrication.

import { z } from "zod";

const GroundedAnswer = z.object({
  status: z.enum(["answered", "insufficient_context"]),
  answer: z.string(),
  citations: z.array(
    z.object({
      sourceId: z.string(),
      quote: z.string().min(10),
    }),
  ),
});

type Source = { id: string; text: string };

export function checkGrounding(raw: unknown, sources: Source[]) {
  const parsed = GroundedAnswer.safeParse(raw);
  if (!parsed.success) return { ok: false as const, reason: "schema" };

  const result = parsed.data;
  if (result.status === "insufficient_context") return { ok: true as const, result };
  if (result.citations.length === 0) return { ok: false as const, reason: "no_citations" };

  const byId = new Map(sources.map((s) => [s.id, normalize(s.text)]));
  for (const c of result.citations) {
    const text = byId.get(c.sourceId);
    if (!text) return { ok: false as const, reason: `unknown source ${c.sourceId}` };
    if (!text.includes(normalize(c.quote))) {
      return { ok: false as const, reason: `quote not found in ${c.sourceId}` };
    }
  }
  return { ok: true as const, result };
}

const normalize = (s: string) => s.replace(/\s+/g, " ").trim().toLowerCase();

This check is deliberately simple. It will not catch every subtle distortion, but it catches invented sources, invented quotes and answers with no grounding at all, at zero model cost. When it fails, you can retry once with the failure reason, or fall back to abstention.

Add a verification pass for high-stakes outputs

For answers where a mistake is expensive (pricing, legal terms, medical or financial guidance, anything that triggers an action), add a second model call whose only job is to check the first. Give the verifier the sources and the draft answer, and ask it to list each claim and mark it as supported, unsupported or contradicted. A focused checking task is easier for a model than open-ended generation, and a smaller, cheaper model often does it well.

Keep the verifier's output structured, and decide in code what happens next: send the answer, strip unsupported sentences, regenerate or escalate. Do not let the verifier rewrite the answer freely, or you have just moved the hallucination risk to a new place.

Check facts against systems of record

Some of the most damaging hallucinations are not about documents at all. They are order numbers, prices, dates, stock levels and account details. Never let the model produce these from memory. Fetch them with a tool call, and when the model includes them in an answer, compare them with the value you fetched. If a support agent says an order shipped on the 4th, your code can confirm that the order record says the same before the message goes out.

The same applies to generated code and configuration: validate field names against the real schema, run the query against a read-only replica, lint the output. Anything that can be checked by a computer should be.

Measure it, or you are guessing

Hallucination work without measurement turns into prompt superstition. Build a small eval set from real questions, including ones your sources cannot answer, and track a few numbers on every change: how often answers are fully supported, how often the model correctly abstains when it should, and how often it abstains when it should not. That last number matters, because a system that refuses everything has no hallucinations and no value either.

In production, log the grounding check result and the verifier's verdict for every response. A rising rate of failed checks after a document update or model change is an early warning you would otherwise only get from an angry customer.

Key takeaways

• Classify your failures first: missing knowledge, ignored context, distortion and fabricated structure need different fixes.
• Ground answers in retrieved sources with stable IDs, and require citations with verbatim quotes you can check in code.
• Use structured outputs with an explicit abstention status, and treat "insufficient context" as a good outcome.
• Add a verifier pass for high-stakes answers, and check facts against systems of record rather than trusting the model.
• Track supported-answer and abstention rates on an eval set so improvements are measured, not assumed.

If you are putting an assistant or agent in front of customers and want it to be reliable from day one, see how we approach agentic AI or get in touch.