RAG explained: how AI answers from company knowledge with sources

What RAG means in plain terms

RAG (Retrieval-Augmented Generation) is a setup in which a language model first finds the relevant passages in a knowledge base and only then writes an answer based on them. Instead of recalling facts from memory, the model behaves like an employee who opens the policy document before answering and quotes the right clause.

The term comes from a research paper by Patrick Lewis and co-authors: "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" was posted on arXiv in May 2020 and published in the NeurIPS 2020 proceedings. The authors described two kinds of memory:

• parametric — knowledge stored in the model's weights during training;
• non-parametric — an external document index the model queries through a retriever.

The paper states that pre-trained models handle their own knowledge imprecisely, and that providing provenance and updating knowledge remain open problems. RAG was proposed as an answer to both. The authors report that RAG models produce more specific and factual language than a comparable model without retrieval.

Why a generic chatbot makes things up

A large language model predicts the next word from the statistics of its training data, so it can confidently state something that isn't true. This is not a bug in one product; it is a property of the technology.

The US National Institute of Standards and Technology calls this "confabulation" in its Generative AI Profile, NIST AI 600-1 (July 2024): confidently stated but erroneous or false content, colloquially known as "hallucinations". NIST makes three points that matter for a business:

• confabulations are a natural result of how generative models are designed, not a rare glitch;
• the risk is higher for long-form answers and specialised domains that need context;
• a model can invent not only a fact but also the reasoning or citation that seems to support it.

For a company the implication is simple. A general-purpose chatbot does not know your policies, prices, contracts or meeting decisions. Ask it about your return window for wholesale clients and you will get a plausible answer — just not one based on your rules. AWS lists the same issues in its RAG overview: models can present false information when they lack an answer, give out-of-date or generic information, and draw on non-authoritative sources.

How RAG works: four steps

A RAG system has three parts: a knowledge base, search over it, and a model that writes the answer from what search returns. AWS describes the process like this:

1. Prepare the data. Documents, database exports and API data are split into chunks and converted into numeric vectors (embeddings) stored in a vector database.
2. Retrieve. The user's question is also converted into a vector, and the system finds the closest passages by meaning.
3. Augment the prompt. The retrieved passages are added to the question, and the model answers from them.
4. Keep it current. Documents and their vectors are refreshed on a schedule or in real time so answers don't go stale.

The key difference from fine-tuning: to teach the system a new policy you don't retrain the model — you add the document to the base. AWS notes this is cheaper than retraining a foundation model.

Why source links reduce the risk of error

A citation turns an AI answer from a claim you have to trust into a claim you can check in seconds. AWS names this as a core RAG benefit: responses can cite sources so users can look up the originals.

But, as NIST warns, a model can invent a citation too. What matters is not the footnote itself but where it points. In a well-built RAG system the link is not text the model generated; it is a pointer to the passage that search actually returned. If the user clicks and sees the clause, the answer is confirmed. If not, the error is visible immediately.

Company Brain builds this into the product: every statement carries a numbered link to the original source, and every query and source is logged. When confidence falls below a threshold, it answers "I don't know" and hands the question to a person. That is more honest — and cheaper — than a plausible mistake someone acts on.

Access rights: why the filter must come before generation

In RAG, everything search returns goes into the model's context, so permissions must be checked at retrieval time, not in the finished answer. If a payroll sheet or a key client contract lands in the context, the model may summarise it for someone who should never see it.

Microsoft's Azure AI Search documentation calls this document-level access control. Each indexed document carries permission metadata, and a query returns only what the specific user is allowed to read. Microsoft also notes that permission changes in the source system take effect only after the metadata is synchronised to the index — worth planning for.

Company Brain describes the same principle: permissions are applied before generation, so documents outside a role's access never physically enter the answer's context. For banks, government and regulated data there is a private installation on the client's own servers, and data does not leave the perimeter.

Where RAG can still go wrong

RAG lowers the risk of invented answers but does not remove it: answer quality is capped by the quality of search and of the documents themselves. If search misses the right passage, the model answers from incomplete data; if the base holds two versions of a policy, it may cite the outdated one.

Common causes and fixes:

• Stale documents. Re-index regularly and define which version of a document is current.
• Poor chunking. A table or contract clause cut in half loses meaning, so chunking is tuned to the document type.
• Questions beyond the knowledge base. If the documents don't hold the answer, the system should say so rather than fill gaps with the model's general knowledge.
• Permissions out of sync. When someone moves to another department, their access in the index must update too.

That is why a RAG assistant is measured on real employee questions rather than a demo, with the metric agreed before the start.

How Company Brain applies this approach

Company Brain connects documents, chats, meetings, CRM and 1C, indexes them together with access rights, and answers with sources in Russian, Uzbek or English. The product page lists four steps: collect, structure, answer, act. The first three match the classic RAG pattern; the fourth is skills that prepare reports, document drafts and tasks.

What the site states:

• connectors: Telegram, Google Drive, Gmail, Bitrix24, amoCRM, 1C;
• channels: Telegram bot and web interface;
• prototype on your data in 2 weeks, pilot of 2–4 weeks, production launch in 6–10 weeks;
• pilot acceptance: 50 real employee questions and a threshold of 85% acceptable answers — correct in substance, citing a source, and containing nothing from documents outside the asker's permissions; below the threshold, W2W keeps improving at its own cost;
• pricing: Business in the W2W cloud — implementation from $5,000 plus $500–1,500 per month; Enterprise in your own environment — implementation from $15,000 plus $1,500–5,000 per month.

Why it matters: a 2012 McKinsey Global Institute study found that interaction workers spend nearly 20% of the workweek looking for internal information or tracking down colleagues who can help. A RAG assistant shrinks that search to one question — as long as the answer can be verified. See how implementation runs on the process page.

Where to start

Start with 20–50 questions your employees actually ask each other every week, plus a list of the systems where the answers live. That is enough to scope the knowledge base, the integrations and the access model.

A practical sequence:

1. Collect questions from chats, email and requests to managers.
2. Note where the answers live: policies, 1C, CRM, Google Drive, Telegram.
3. Define who may see which data — this becomes your permission model.
4. Decide where the data should sit: cloud or your own environment.
5. Agree on the acceptance metric before the pilot starts.

Get a preliminary budget and timeline from the implementation calculator, or discuss your case on a 30-minute call — book a meeting.