How to Train an AI on Your Documentation
Published on by Antoni
The Single Biggest Mistake Teams Make
Most teams spend hours configuring AI settings, tweaking prompts, and adjusting tone — and then wonder why their AI gives wrong answers.
The answer is almost never in the settings. It's in the docs.
The AI is only as good as what you put into it. Bad documentation produces bad AI answers. Good documentation, properly structured and kept up to date, produces an AI that customers trust. This guide is about getting the docs right.
Step 1: Audit What You Have
Before you import anything, do a 30-minute content audit. Go through your existing documentation and categorize everything:
Keep (high confidence):
- Articles written in the last 6 months
- Pages that accurately describe how your product works today
- Policy pages (refund, shipping, cancellation) that are current and authoritative
Update before importing:
- Articles that reference old features, old pricing, or old UI
- Pages that say "coming soon" for things that have since launched
- Anything with placeholder text or incomplete sections
Archive or delete:
- Documentation for features you've removed
- Duplicate articles covering the same topic differently
- Pages with contradictory information
Rule of thumb: If a new employee reading this doc would be confused or misled, the AI will be too.
Step 2: Structure Your Content for AI Retrieval
Modern AI retrieval works by breaking your content into chunks, embedding them as vectors, and finding the most relevant chunks for each question. The way you structure your content directly affects how well this works.
Use clear, descriptive headings. An article titled "Billing" is harder to retrieve than one titled "How to Update Your Billing Method" or "Understanding Your Invoice." The heading is a major signal for what the content is about.
Write one topic per article. If a single article covers setup, billing, and integrations, the AI may retrieve only the billing section when someone asks a setup question. Keep articles focused.
Put the answer first. Start each section with the direct answer, then add context. "Yes, you can export your data at any time. Here's how..." is better than three paragraphs of background before you get to the point.
Use explicit Q&A format for high-stakes answers. For questions where precision matters — refund eligibility, cancellation policies, pricing — write them as explicit questions with explicit answers:
Q: Can I get a refund if I cancel mid-month?
A: Yes. We prorate refunds for unused days in the billing period.
Refunds are processed within 5–7 business days.
Avoid jargon without explanation. Your AI will repeat the language it learns. If your docs are full of internal terms customers don't know, the AI will use those terms too.
Step 3: Choose Your Knowledge Sources
Different documentation types are suited for different content import methods:
Public website content
Best for: Help center articles, product pages, public FAQs, pricing pages
Method: Enter your website URL in FutureBase. The auto-crawler fetches and indexes all public pages. You can exclude specific URL patterns (e.g., /blog/*, /legal/*) if you don't want them in the AI's knowledge.
Update frequency: Set a sync schedule (daily or weekly) so the AI always has current information.
Notion
Best for: Internal wikis, SOPs, onboarding guides, product specs
Method: Connect Notion via Integrations → Notion in FutureBase. Select which pages and databases to sync. FutureBase indexes them with incremental sync — only changed pages are re-processed on each run.
Tip: Keep a dedicated Notion database for "AI Knowledge Base" content. This makes it easy to control exactly what the AI learns without worrying about accidental inclusion of sensitive internal docs.
Uploaded files
Best for: PDFs, Word documents, markdown files, spreadsheets with structured data
Method: Upload directly in FutureBase. Supported formats include PDF, DOCX, MD, and TXT. Images inside PDFs are not indexed — only text.
Tip: For PDFs, make sure they're text-based (not scanned images). Run OCR on scanned documents before uploading.
FAQ entries (manual)
Best for: Questions where you need exact, controlled answers
Method: In FutureBase, go to Content → FAQs and create entries manually. These bypass the retrieval system — if the customer's question matches your FAQ entry, the AI responds with your exact answer, not a generated one.
Use FAQs for: pricing details, legal/compliance answers, anything where "close enough" isn't acceptable.
Step 4: Prioritize Depth Over Breadth
A common mistake is trying to import every piece of documentation at once. Resist this urge.
Start narrow and deep. Pick your 20 most common support questions (pull from your ticket history or ask your support team). Make sure your documentation answers each one thoroughly. Import those docs first, test thoroughly, then expand.
An AI with 20 topics covered deeply is more useful than one with 200 topics covered shallowly.
Signs of shallow coverage:
- The AI answers the first part of a question but misses the follow-up
- Customers keep asking the same questions despite the AI being live
- The AI escalates more than 40% of conversations to humans
Signs of deep coverage:
- The AI gives complete, correct answers on the first response
- Customers say "that was helpful" and close the chat
- Escalation rate drops below 20–30% within 2 weeks
Step 5: Handle Sensitive Content
Not everything in your docs should go into the AI. Some content should stay private:
Keep out of the AI:
- Internal pricing spreadsheets or discount structures
- Customer-specific pricing or custom contracts
- HR documents, financial reports, investor materials
- Personal data (customer PII, employee information)
- Pre-release product information
How to control it in FutureBase:
- When connecting Notion, only select public-facing pages — not your entire workspace
- Use URL exclusion rules to block specific paths from the website crawler
- Use separate "AI Knowledge Base" databases/folders for content you explicitly want the AI to learn from
Step 6: Write for the AI, Not Just for Humans
Your documentation already serves human readers. To also serve the AI well, make small adjustments:
Add synonyms and alternate phrasings. If customers ask "How do I unsubscribe?" but your doc says "How to cancel your account," the AI may miss the connection. Add a note: "Also referred to as unsubscribing, downgrading, or account closure."
Spell out acronyms on first use. "SLA (Service Level Agreement)" helps the AI connect the abbreviated form to the concept.
Include negative information. Docs often say what you can do; they rarely say what you can't. "We don't offer phone support" is just as important as "You can reach us via chat." If you leave out the "no," the AI may invent a "yes."
Keep formatting consistent. The crawler and chunker handle clean markdown and HTML well. Heavily nested tables, multi-column layouts, or complex CSS formatting can cause content to be misread. When in doubt, simpler is better.
Step 7: Set Up Incremental Sync
Your docs change. Your AI's knowledge should too.
For website content: Configure FutureBase to re-crawl your site daily or weekly. New and updated pages are automatically re-indexed.
For Notion: FutureBase tracks page modification timestamps. On each sync, only updated pages are re-embedded — so even large Notion workspaces sync efficiently.
For uploaded files: When you update a document, re-upload it to FutureBase. The old version is replaced and the new one is indexed.
For FAQs: Update them directly in FutureBase whenever your policies change. These take effect immediately — no sync needed.
Step 8: Test, Identify Gaps, Fill Them
After your initial import, spend 30 minutes asking your AI the questions your customers actually ask. Keep a running list of any answer that's wrong, incomplete, or missing.
For each gap:
- Identify which doc (or missing doc) caused the issue
- Update or create that doc
- Re-sync or re-import it
- Re-test
Do this weekly for the first month. By the end of month one, you should have a solid knowledge base that handles 60–80% of questions without human intervention.
Common Mistakes
| Mistake | Why it hurts | Fix |
|---|---|---|
| Importing everything at once | Low-quality content dilutes good content | Curate before importing |
| Never updating the AI after launch | Stale docs → wrong answers | Set up incremental sync |
| Relying only on the auto-crawler | Misses Notion, internal docs | Connect multiple sources |
| No FAQ entries for policy questions | AI paraphrases policies incorrectly | Add explicit FAQ entries |
| Including confidential docs | AI may expose internal information | Audit what you import |
The Continuous Improvement Loop
Training your AI isn't a one-time task. The best-performing AI support agents have a feedback loop:
- Monitor conversations daily for the first two weeks
- Flag wrong or incomplete answers
- Trace each issue back to the source doc
- Update the doc
- Re-sync
- Verify the fix
After the initial ramp-up, most teams settle into monthly doc audits with immediate fixes for anything critical that comes up. The AI gets better over time — not because the model improves, but because the knowledge base does.
Getting Started Today
You don't need perfect documentation to get started. You need good enough documentation on your 10 most common questions. Here's a concrete starting point:
- Pull your last 30 support tickets
- Find the 10 most common question topics
- Review the docs that cover those topics — update anything stale
- Add 5 FAQ entries for the questions that need exact answers
- Connect FutureBase to your website and any Notion workspace you use
- Test with those 10 questions
- Iterate
The rest can come later. Start small, verify the quality, then expand.
— Antoni
