Do You Need RAG? Use This Workflow Test Before You Build

You need RAG when an AI workflow must repeatedly answer from a large, private or changing body of approved knowledge that users should not paste into every request. You do not need RAG merely because the task uses documents or because the word “knowledge base” sounds advanced.

Most failed RAG projects begin with architecture before diagnosis. Teams buy ingestion, embeddings, vector search and a chat interface before proving that missing knowledge is the actual reason the workflow fails.

RAG solves one problem, not every AI problem

Retrieval-augmented generation gives a model selected information from an external source at answer time. The retrieval layer finds relevant material; the model uses that material to produce a response.

That can help when the answer depends on internal policies, product documentation, changing service rules, contracts, technical manuals or a large operational knowledge base.

RAG does not repair unclear tasks, contradictory documents, weak ownership, bad permissions or an absent review process. It can retrieve confusion faster.

Use the lowest-complexity solution that passes

Before building retrieval, test the workflow against this ladder.

1. Prompt only: the task needs general reasoning and a clear brief, not private knowledge.
2. Prompt plus document: the user can attach the relevant file for a one-off or bounded task.
3. Fixed assistant or project knowledge: a stable set of instructions and references supports recurring work.
4. Automation without retrieval: the bottleneck is moving information between systems, not finding knowledge.
5. Governed RAG: the workflow repeatedly needs searchable, changing, approved knowledge across a larger source set.

The right answer is the first level that produces reliable work. Complexity is justified by a proven requirement, not by ambition.

The RAG Readiness Test

Score each statement from 0 to 2: 0 means no, 1 means partly, 2 means clearly yes.

• The answer depends on private or frequently changing knowledge.
• The knowledge is too large or repetitive to attach manually every time.
• The same class of question occurs often enough to justify a maintained system.
• There is one accountable owner for each source and version.
• Approved and excluded sources can be defined.
• Wrong answers have a meaningful operational cost.
• A test set of real questions and approved answers can be created.
• The system can show sources or enough traceability for review.
• Permissions can restrict who may retrieve which information.
• There is a maintenance and rollback process.

A high score does not authorize a build by itself. It shows that a retrieval pilot may be justified. A low score is a signal to fix the prompt, documents or workflow first.

What a minimum viable RAG pilot looks like

The first project should answer one narrow class of question from one controlled source family.

Good pilot: support staff can ask about returns, cancellations and warranty rules using only the current approved policy set.

Bad pilot: an assistant that “knows everything about the company.”

1. Choose one user and one decision.
2. Define the allowed source set.
3. Remove duplicates, obsolete files and drafts.
4. Create 25–40 realistic test questions, including questions with no answer.
5. Define the refusal rule when approved information is missing.
6. Measure retrieval quality separately from answer quality.
7. Assign a source owner and an answer-quality owner.
8. Review failures weekly before expanding scope.

Separate retrieval failure from generation failure

When an answer is wrong, ask two different questions.

• Did the system retrieve the correct source or section?
• Did the model use the retrieved material accurately and obey the answer rules?

A wrong source is a retrieval problem. A correct source with a wrong answer is a generation or instruction problem. Blending the two produces vague debugging and expensive redesign.

The source-governance problem

A retrieval system inherits the quality of its source library. Duplicate policies, unclear filenames, missing effective dates and conflicting instructions become automated uncertainty.

Every included source should have an owner, status, version, effective date, audience and retirement rule. Sensitive sources need permission boundaries. Drafts and expired documents should not be discoverable by default.

A worked decision

A consultancy wants AI to draft proposals from discovery-call notes, service descriptions and pricing rules. At first, users can attach the notes and approved service sheet. A structured prompt produces a draft that a consultant reviews.

RAG becomes justified only when the approved service library grows, pricing and exclusions change, several teams need the workflow, and users repeatedly attach the same documents. The retrieval system then solves a proven friction: selecting the current approved material automatically.

The stop rule

Do not build RAG until you can complete this sentence with evidence: “This workflow fails because the model cannot access a specific approved knowledge source at the moment it answers.”

When the honest diagnosis is vague instructions, messy documents, no owner or no acceptance test, repair that first. Retrieval is an operating layer, not a substitute for operating discipline.

Before you bolt on another tool, it is worth knowing whether your business runs on systems or on you. I put together a free 2-minute assessment that gives you a straight read on exactly that, and the first thing to fix. Take the free assessment.