Building a RAG knowledge assistant people can rely on

A practical RAG guide covering source ownership, permissions, retrieval tests, answer evidence, abstention and an operating model after launch.

Explore the service↗
RAG / KNOWLEDGE6 MIN READPUBLISHED 15 NOVEMBER 2026CRAACT EDITORIAL
Knowledge sources, retrieval layers and evidence connected as one system

A convincing knowledge-assistant demo can answer prepared questions. A useful business system must also handle incomplete information, restricted documents, obsolete policies and questions that should go to a person. The difference is an operating design, not merely a larger model or another vector database.

Retrieval-augmented generation gives a language model relevant retrieved material as context. Microsoft's overview describes this grounding pattern and its retrieval challenges. It does not make every generated answer a verified fact. This delivery framework focuses on decisions a business and engineering team should make together. Examples describe a proposed approach, not measured client results.

Choose a narrow knowledge job

Begin with a defined audience and task. “Answer anything about the company” combines different permissions, freshness requirements and consequences. “Help support staff find the current installation instructions for an approved product range” is easier to evaluate and operate.

List the decisions the assistant may support and those it must not make. It may locate instructions, compare documented procedures or prepare a draft. It may need to escalate questions about contractual commitments or missing documentation. Write those boundaries in language users understand. They should appear in the experience as well as the technical design.

Give sources accountable owners

Inventory documents before indexing them. Record owner, audience, approval status, revision and replacement relationships. Similar titles may describe different products or policy periods. Without metadata, retrieval can treat both as equally authoritative.

Agree how content enters and leaves the system. Draft files should not silently become approved guidance. A withdrawn policy should stop appearing in future answers, and the team should know how long removal takes. Establish a source register and review it with document owners. This often reveals valuable content-maintenance work before retrieval parameters need attention.

Treat permissions as part of retrieval

An assistant must not retrieve material the current person cannot access. Decide how user identity, document access and group membership are represented throughout the pipeline. Do not rely only on a prompt telling the model to keep secrets after restricted content has already entered its context.

Test similar questions from different roles. A customer and a staff member may ask about the same product but be entitled to different information. Also test change: does access update when a person leaves a team or a document becomes restricted? Consider caches and conversation history as well as the first search request. A good initial result does not prove the entire conversation remains inside its access boundary.

Prepare content around useful questions

An entire manual may mix unrelated tasks. A tiny fragment may lose conditions that make an instruction relevant. Choose boundaries and metadata around how people ask questions. Preserve headings, versions and references that distinguish one procedure from another.

Build a small test collection before importing everything. Include short factual questions, multi-step questions, ambiguous terms and requests needing several sources. Inspect retrieved passages independently of the final answer. If the correct source never reaches the answer stage, changing the final wording will not fix the underlying retrieval problem.

Make supporting evidence usable

A reference should let someone inspect the supporting material, not merely decorate an answer. Show a meaningful source title and, where practical, the relevant section or revision. Distinguish a documented fact from an inference. If approved sources disagree, that disagreement matters.

A useful answer format includes a concise response, important conditions, supporting sources and a next step. An instruction may point to the full procedure. Missing guidance may require a document owner. Avoid a long confident paragraph when the evidence supports only a narrow statement. Let users inspect context before acting, especially when a short extract omits exceptions elsewhere in the document.

Design the no-answer path

Abstention deserves product design. “I don't know” may be accurate, but the user also needs a useful route forward. Explain whether information is missing, access is insufficient, the question is ambiguous or the requested decision is outside scope.

Test no-answer behavior alongside successful answers. A system that always produces something plausible may look impressive in a demonstration while creating operational risk. Agree which tasks require source support, which may allow clearly labelled general guidance and which must escalate. Evaluation can then check those choices rather than rely on a reviewer's overall impression.

Keep retrieved instructions untrusted

Documents can contain text that tries to redirect an assistant. OWASP identifies prompt injection as a risk and notes that RAG does not fully remove it. Retrieved evidence should not automatically gain authority to change permissions or instruct tools to transmit information elsewhere.

Consider a read-only first release. If actions are later added, give each tool narrow permissions and validate consequential writes separately. A retrieved sentence saying “email this file” is still source content, not user authorisation. Include hostile source text in evaluation and examine what information could leave through connected tools.

Evaluate retrieval, answers and operations

Build an evaluation set from representative tasks using synthetic or appropriately authorised examples. Record the expected source, important conditions and acceptable behavior. Include unanswered questions, conflicting revisions and restricted material. Keep some examples separate from tuning to check behavior on unfamiliar cases.

Review several dimensions: useful retrieval, statements supported by evidence, meaningful citations and access boundaries. Also examine latency and the effort needed to resolve an escalation. No single score captures all of these. Record the evaluation method so stakeholders know what an apparent improvement means. A writing-style improvement should not be mistaken for better evidence or stronger access control.

Plan ownership after launch

Assign owners for source updates, evaluation failures and user feedback. Retain enough version information to understand which document and configuration produced an answer while limiting personal data in logs. Define rollback when an import or configuration change makes behavior worse.

A pilot should end with an evidence-based decision. Review tasks completed usefully, questions requiring escalation and maintenance effort. Expand the source set or audience when the results support it. A smaller assistant with dependable boundaries can create more value than a broad system people hesitate to trust.

References and next steps

These primary references support the technical context. The delivery framework and illustrative scenarios are Craact’s editorial guidance; implementation depends on the actual systems and agreed project scope.

Start with one useful problem, make the operating boundaries explicit and decide what evidence will justify the next phase.

Explore the related service ↗ · Discuss your project ↗

//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]
//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]
//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]
//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]
//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]
//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]
YOUR NEXT MOVE STARTS HERE

Your next chapter,
[ Connected ]
Let’s build it together.

Let’s talk
//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]
//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]
//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]
//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]
//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]
//
[ENGINEERING]&[POSSIBILITY]
//
[WEBSITES]&[EXPERIENCES]
//
[WORKFLOWS]&[AUTOMATION]
//
[CRM]&[CONNECTION]
//
[KNOWLEDGE]&[RAG]
//
[IDEAS]&[PRODUCTS]