Stopping AI Hallucinations in Business Answers: Grounding, Citations and Refusals
Reduce AI hallucinations by grounding answers in your own documents, requiring a citation for every claim and letting the assistant say it doesn't know.
7 min read

Contents
To reduce hallucinations when AI answers business questions, give the model the right source documents for each question, tell it to answer only from them, require a citation for every claim, and allow it to say it doesn't know. Then check answers against the sources before users see them, and measure the error rate on a fixed test set.
Picture an operations team that adds an AI assistant to its internal portal. In the first week it answers a question about the refund policy by quoting a 14-day window. The policy says 30. Nobody had given the assistant the policy document, so it filled the gap with something plausible. The fix was not a better model. It was feeding the assistant the right document, making it show where each answer came from, and letting it say "I can't find that" when the documents were silent.
What is an AI hallucination, and why does it happen in business answers?
An AI hallucination is an answer that sounds confident but is not supported by any source, such as an invented policy, a wrong figure or a made-up product feature. It happens because language models generate likely text, and when the facts are missing they fill the gap with likely-sounding ones.
In business settings, hallucinations usually come from three places:
Missing context: the document that holds the answer was never retrieved or never indexed, so the model answers from general knowledge.
Wrong context: retrieval returns an outdated version, a similar document for another region, or a fragment cut off mid-table.
Pressure to answer: the prompt implies an answer is always expected, so the model produces one even when the sources don't contain it.
Each cause has its own fix. Better prompts help with the third, but the first two are data and retrieval problems.
What does grounding mean, and how do you set it up?
Grounding is connecting a model's answer to specific, verifiable sources, such as your policy documents, product database or a web search, and instructing it to answer only from them. Google's grounding overview defines it as "the ability to connect model output to verifiable sources of information."
In practice, grounding means retrieval plus instructions. A retrieval step finds the passages relevant to the question and places them in the prompt; the instructions tell the model to stay inside them. Anthropic's guide to reducing hallucinations recommends you "explicitly instruct Claude to only use information from provided documents and not its general knowledge."
The major vendors each offer a grounded path:
Anthropic Claude: Approach: you supply the documents in the request. Grounding aids: the Citations feature, plus the techniques in the hallucination guide.
Google Gemini: Approach: Grounding with Google Search "connects the Gemini model to real-time web content," and Google Cloud's grounding overview lists options for grounding on your own data, including its RAG Engine. Grounding aids: responses include inline citations to the sources used.
OpenAI: Approach: the file search tool in the Responses API retrieves from files you upload to a knowledge base. Grounding aids: the answer comes back with file citations.
For questions over company documents, the retrieval quality decides most of the outcome. Our post on when RAG works for company documents covers stale files, scanned PDFs and permissions, which are the usual reasons the right passage never reaches the model.
How do citations make AI answers checkable?
Citations link each claim in an answer to the exact passage, page or document it came from, so a person or a program can check it. An answer with citations can be audited; an answer without them can only be trusted or doubted.

Anthropic's guide suggests making the model "cite quotes and sources for each of its claims," and adds: "If it can't find a quote, it must retract the claim." That rule turns citations from decoration into a check. For long documents, over about 20,000 tokens, the same guide recommends asking the model to "extract word-for-word quotes first" before it answers.
Some vendors return citations as structured data rather than text in the answer. Claude's Citations feature cites plain text by character position, PDFs by page number and custom content by block, which lets your interface link each sentence to its source. Two limits are worth knowing: citing images inside PDFs is not supported, and citations can't be combined with structured outputs in the same request.
What to show users
The source name and date next to the answer, so users can see if a policy is out of date.
A link to the exact passage, not just the document.
A clear marker when an answer draws on several sources, so conflicting ones are visible.
When should an AI assistant refuse to answer?
An assistant should say it doesn't know when the retrieved sources don't contain the answer, when the sources conflict, or when the question is outside the topics it is meant to cover. A clear "I can't find that in our documents" is cheaper than a confident wrong answer.

Models don't refuse on their own as often as you'd like, so you have to give permission explicitly. Anthropic's guide puts it simply: "Explicitly give Claude permission to admit uncertainty." In the system prompt, say what to do when the documents are silent: name the missing information, suggest who to ask, and stop.
Refusal rules worth writing down
No source, no answer: if no retrieved passage supports the claim, the assistant says so.
Conflicting sources: it shows both and names the newer one, rather than choosing quietly.
Out of scope: legal, medical, HR or pricing questions it isn't meant to answer go to a named person or team.
High-stakes actions: anything that changes a record or a commitment needs a human, not a best guess.
Track refusals as a metric. A rising refusal rate on one topic usually means a document is missing from the index, which is a fix in the content, not the model.
Can you check AI answers automatically before users see them?
Yes. Grounding checks compare the answer with the source passages and flag or block claims the sources don't support. They catch some hallucinations that slip past the prompt, but they add cost and delay, and none of them is complete.
Two cloud services offer this as a feature:
Amazon Bedrock Guardrails, contextual grounding check: Scores: a grounding score (is the response supported by the source) and a relevance score (does it answer the query). Thresholds: set between 0 and 0.99; AWS's example blocks every response scoring below 0.7 on either. Action: block, or detect only. Limits: up to 100,000 characters of source, 1,000 of query and 5,000 of response.
Azure AI Content Safety, groundedness detection: Purpose: checks that responses are based on the source material you provide, with an optional correction feature, in preview, that rewrites ungrounded text. Status: English only, and Microsoft says it retires on 30 April 2027, so plan around a replacement.
You can also build the check yourself: a second model call that reads the answer and the cited passages and confirms each claim is supported. Whichever you use, log what was flagged, because those cases make the best test examples.
How do you measure whether hallucinations are going down?
Build a fixed test set of real questions with known correct answers, including questions the documents can't answer, and score every change against it. Track the share of correct answers, the share of unsupported claims and the share of correct refusals.
Collect 50 to 100 real questions from support tickets, chat logs or the team that will use the assistant.
Write the correct answer and its source for each, and mark the ones where the right answer is "not in our documents."
Run the assistant and have a person, or a checked grading prompt, mark each answer as correct, wrong, unsupported or correctly refused.
Re-run on every change to the model, prompt, retrieval settings or document set.
Add new failures from production to the set each week.
Our checklist for testing an AI agent before it goes live covers evaluation sets and shadow runs in more depth. If you are still deciding whether you need an agent or a simpler assistant, start with what an AI agent is.
Anthropic's guide is candid that these techniques "don't eliminate them entirely." The goal is a known, low error rate and a design where the remaining errors are visible and cheap to catch.
How we approach it at Vizio AI
We treat hallucination as a system problem, not a model problem. We start with the questions people actually ask and the documents that should answer them, fix gaps and stale versions in the content, and build the evaluation set before we tune prompts.
The assistants we build answer from retrieved sources only, cite the passage behind each claim, refuse when the sources are silent, and log every flagged or corrected answer so the test set grows with real use. For actions that change records, a person approves. This is part of the AI and operations work of our DX Studio.
Sources
Claude Developer Platform, Reduce hallucinations: https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinations
Claude Developer Platform, Citations: https://platform.claude.com/docs/en/build-with-claude/citations
Google AI Studio documentation, Grounding with Google Search: https://aistudio.google.com/docs/google-search
Google Cloud documentation, Grounding overview: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/grounding/overview
OpenAI API documentation, File search: https://developers.openai.com/api/docs/guides/tools-file-search
Amazon Bedrock User Guide, Use contextual grounding check to filter hallucinations in responses: https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-contextual-grounding-check.html
Microsoft Learn, Groundedness detection in Azure AI Content Safety: https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/groundedness
Vizio AI, Chat With Your Company Documents: When RAG Works and When It Doesn't: https://www.vizio.ai/resources/blog/rag-company-documents
Vizio AI, Before You Put an AI Agent in Front of a Client: The Checks We Run: https://www.vizio.ai/resources/blog/ai-agent-testing-production
Vizio AI, What Is an AI Agent, Really? Agents vs Chatbots vs Workflows: https://www.vizio.ai/resources/blog/what-is-an-ai-agent
Vizio AI, DX Studio: https://www.vizio.ai/studios/dx-studio
FAQs.
Read More

Chat With Your Company Documents: When RAG Works and When It Doesn't

An AI Helpdesk in Slack or Teams: Answering Internal Questions Your Team Will Trust

The AI Visibility Audit: How to Check What ChatGPT, Gemini and Perplexity Say About Your Company

How to Get Your Company Cited by ChatGPT, Perplexity and Google AI Mode

Reading Invoices, Contracts and Forms With AI: Document Processing Without Retyping

Claude, GPT or Gemini? Choosing a Model for Business Workflows in Late 2026

Claude Code for Operations Teams: Automating Internal Tools Without a Dev Team

What Does an AI Agent Cost to Run? Tokens, Tools and the Hidden Line Items



