RAG explained: a chatbot that uses your documents

Why a general chatbot doesn't know your business

A pre-trained model does not automatically have your contracts, procedures and prices. Even when he knows a subject, his information may be old or too general. The RAG adds a research step in your content before requesting a response. This separation allows documents to be updated without retraining the model for each modification. It does not remove errors: the system can find the wrong passage, ignore an exception or interpret a sentence incorrectly. The architecture must therefore make the sources visible and allow abstention when information is missing.

Prepare documents before any research

The first difficulty is often documentary. Identify official versions, owners, expiration dates and access rights. Properly extract text from PDFs, including useful tables. Remove duplicates and keep the title structure. A sentence about reimbursements loses its meaning if separated from the relevant contract category. Add metadata such as service, language, document type, and effective date. They will be used to filter the results and explain why a passage was presented.

Cut without destroying the meaning

Long documents are usually divided into passages, often called chunks. Too fine a division loses conditions and exceptions; too broad a breakdown fills the context with unnecessary information. Start by sticking to paragraphs and sections, then experiment with the size according to your documents. A slight overlap can maintain continuity, but increases indexed volume and repetitions. For a table, repeat the headings necessary for understanding it. Assess cutting quality with real questions rather than a randomly chosen size value.

Understanding embeddings and hybrid search

An embedding represents text in the form of a digital vector. Texts similar in meaning can thus be found even if they do not use the same words. This semantic search remains imperfect for exact references, invoice numbers or very similar product names. A hybrid search therefore often combines keywords and vector similarity. Filters apply language, date or organization constraints. The choice of embedding model and index must be tested on your business vocabulary, particularly if the queries and documents mix several languages.

Classify passages and build context

Retrieving a lot of results does not guarantee a good answer. A reranking step can compare the passages to the question more precisely and then select the best ones. The context sent to the model must remain limited and explicit: original question, extracts, source identifiers and response instructions. Avoid filling the entire available window with related but unnecessary documents. For a question with multiple subproblems, multiple focused searches may be necessary. However, keep a budget for time and calls to prevent costly exploration without measurable improvement.

Example: answering a question about leave

An employee asks how many days he can postpone. The search should take into account the country, contract, year and policy version. The system finds the applicable passage, presents the rule and cites the specific section. If the employee's information does not allow him to choose between two policies, he asks an additional question. If the documents contradict each other, he reports this contradiction. This behavior is more useful than a made-up universal response. The link to the source should allow the user to reread the exceptions in context.

Manage permissions and attacks per document

Rights must be applied before snippets reach the template. A restriction displayed in the interface does not protect a search performed on all files. In a multi-enterprise system, isolation concerns indexes, metadata, cache and logs. Another risk is the injection of instructions into a document: a file may contain an instruction asking to ignore the rules or reveal other data. Treat this content as unreliable information. It should never be able to modify permissions or authorize a tool call in place of the application.

Measure research and response separately

Construct a set of questions with the documents that actually contain the answer. First check if the correct passages are found, then if the model uses them correctly. An answer may seem good despite poor research, because the model completes it with its general knowledge; this hides a weakness in the system. Measure relevance of sources, coverage of issues, accuracy and ability to tell when information is missing. Include cases where no documents respond. An excellent answer to an easy question does not compensate for the disclosure of a single prohibited document.

Maintain the index and monitor versions

Predict how document creations, modifications, and deletions propagate. An old procedure should not remain accessible indefinitely in the index or in a cache. Use stable identifiers and preserve the provenance of passages to be able to rebuild the index. Detect extraction failures and documents that have become unreadable. Monitor the freshness of the data with the business manager. The RAG is a living documentary service: its maintenance matters as much as the initial choice of a vector base or a model supplier.

Define a realistic first delivery

Choose a mastered corpus and a well-identified family of questions, such as the support procedures. Deliver research, sourced response, user feedback and tracking table before connecting the entire company. Document retention limits and rules. Tuplify Studio can develop this journey in a portal, a CRM or a business application, with authentication and access control. The objective is not to have the chatbot respond to everything, but to allow users to find reliable information more easily than with their current process.

Workshop: Diagnosing an Incorrect RAG Response

Choose a question for which you know the exact source. Record the passages you find before looking at the answer. If the correct information does not appear, look at extraction, clipping, filters and search. If it appears but is not used, review snippet order, instructions, and ambiguities. Add an outdated version of the same document to check date processing, then ask with a permissionless account. Finally, remove the reference document: the system should recognize the absence rather than continuing to cite an old cache. This method separates the causes and avoids changing the model for each error when the problem often comes from documentary preparation or access rules.

Frequently asked questions: vector base, quotes and languages

Is a vector base mandatory? A solution can use textual or hybrid search depending on the corpus. The need is to find relevant passages, not to adopt a component because it is associated with the word RAG.

Does a quote prove the answer? No: the passage cited may be incomplete, old or misinterpreted. Check the link between the statement and its source.

Can you ask an Arabic question on a French document? Some search models and methods allow this, but quality must be measured on actual languages ​​and terms. Plan for product names, abbreviations and mixed wording; a successful multilingual demonstration does not guarantee all cases in the corpus.