How to Prepare Your Business Knowledge Base Before Launching an AI Chatbot

Most business owners who plan an AI chatbot spend their time choosing the model or the chat widget. The part that decides whether the bot gives correct answers gets far less attention: the documents you feed it. A modern chatbot for customer support usually answers from your own material, such as price lists, policies, FAQs and service pages. If that material is outdated, contradictory or scattered across WhatsApp threads and old PDFs, the bot will repeat those problems to your customers, politely and at scale.

The technical name for this setup is retrieval-augmented generation, often shortened to RAG. When a customer asks a question, the system searches your documents for the most relevant passages and hands them to the AI model, which writes an answer based on them. Anthropic, the company behind Claude, explains in its engineering write-up on contextual retrieval that traditional RAG splits documents into small chunks, and a chunk stripped of its surroundings can lose key details. Their example is a line saying revenue grew 3 percent, with no mention of which company or which quarter. A chunk like that is hard to find and easy to misuse.

Anthropic's own tests in that write-up showed how much retrieval quality can change with better preparation. Adding short, chunk-specific context to each piece before indexing cut failed retrievals by 49 percent, and adding a reranking step brought the reduction to 67 percent. These are results on Anthropic's internal benchmarks, and InfoQ's independent coverage reported the same figures, so your own gains will vary. The lesson for a business owner is simple: how your knowledge is written and organised has a direct effect on how often the bot finds the right answer.

The same write-up makes a point that saves many small businesses money. If your knowledge base is under about 200,000 tokens, which Anthropic puts at roughly 500 pages, you may not need a retrieval system at all. You can place the whole knowledge base in the model's prompt and use prompt caching to keep it fast and affordable. For a clinic, a real estate office or a small online store, the useful material often fits within that size. A simpler architecture means fewer moving parts and fewer things to break.

Whichever architecture you choose, preparation follows the same steps. First, collect every source a customer-facing staff member actually uses: the website, price sheets, refund and cancellation policies, delivery terms, opening hours, doctor or agent schedules, and the answers your team types most often on WhatsApp and email. Second, remove duplicates and old versions. If two PDFs show different consultation fees, decide which one is correct and delete the other. The bot cannot know which document your team stopped trusting last year.

Third, rewrite key facts so each one stands on its own. Instead of a table cell that just says 1,500, write that the general consultation fee at the Saket clinic is 1,500 rupees. Use clear headings, one topic per section, and plain sentences. Fourth, add simple labels to each document, such as the branch, the service, the date it was last checked and who owns it. These labels help the system pick the right passage, and they tell your team what needs reviewing when prices change.

Fifth, write down what the bot must not answer. Anthropic's documentation on reducing hallucinations recommends explicitly giving the model permission to say it does not know, and instructing it to use only the information in the documents provided rather than its general knowledge. In practice this means your bot should hand over to a person for medical advice, legal questions, custom quotes or complaints, and it should say clearly when something is not in its sources. Finally, collect 30 to 50 real customer questions from your inbox and test the bot against them before launch, and again every time you update your documents.

Consider a few realistic examples. A dental clinic in Delhi or Dubai can prepare one clean document per treatment with fees, duration, preparation steps and aftercare, plus a separate page on appointment rules. The bot can then answer routine questions and book visits, while anything about symptoms goes to the front desk. A real estate agency in London or Gurugram can keep one structured page per listing with price, area, possession date and the agent responsible, so the bot does not mix up two similar projects. A Shopify store in the USA or India can turn its shipping, returns and size guides into short, dated sections, and connect live order status through the store's systems instead of static text.

There are honest limits to keep in mind. Good preparation reduces wrong answers, but it does not remove them. Anthropic's documentation states that these techniques significantly reduce hallucinations without eliminating them, and that critical information should always be validated. Knowledge bases also go stale. If your fees change in April and nobody updates the bot, it will quote old prices with full confidence. You also need to think about privacy: do not put patient records, customer phone numbers or internal salary data into a knowledge base meant for public answers. And someone on your team has to own the content, or quality will slowly drift.

This is where Legacy Services can help. We start with a short audit of your existing material, then clean, deduplicate and restructure it into a knowledge base written for AI retrieval, with clear sections and labels for branch, service and review date. Based on the size of your content, we recommend either a simple full-context setup with prompt caching or a retrieval system with contextual chunking. We then build the chatbot for your website or WhatsApp, set clear rules for what it must hand to a human, and test it against real questions from your own inbox before it goes live.

We also make sure the knowledge base stays current. Using n8n workflow automation, we can set up a process where an update to your price sheet in Google Sheets or a change on your website automatically refreshes the bot's content and notifies the content owner. The same knowledge base can power an AI calling agent, so your virtual receptionist on the phone and your website chatbot give the same answers. Unanswered questions are logged, so you can see exactly which gaps to fill next.

If you are planning an AI chatbot or voice agent and are unsure whether your documents are ready, we are happy to look at them with you. Book a free 30-minute strategy call with Legacy Services at legacyservices.in/contact, and we will walk through your current material, suggest the right setup for your size, and outline a practical plan to launch a bot your customers can trust.