Classic RAG does one thing: turn a question into a search, stuff the results into a prompt, and hope the answer is in there. For a straightforward lookup that works beautifully.
Now try this one. Find our Q3 sales figures, then retrieve the commission structure that applied during Q3, then calculate what we owe. One search cannot answer that, because you do not know the second query until you have the first answer.
What Actually Changed
Agentic RAG puts an agent in the middle of the retrieval loop. Instead of retrieving once, the system plans what to look for, retrieves, evaluates whether the result is sufficient, and goes back for more if it is not.
The academic surveys now classify these architectures by agent cardinality and control structure, which is a formal way of saying the interesting question is no longer how do we search but who decides what to search next.
The Four Capabilities That Define It
- Query planning. Breaking a question into the sub-questions that actually answer it.
- Multi-hop retrieval. Using the result of one search to construct the next.
- Source selection. Choosing between the wiki, the database and the ticket system rather than searching everything.
- Self-checking. Noticing that the retrieved context does not support an answer, and saying so.
When It Is Worth the Complexity
| Your situation | What to build |
|---|---|
| One knowledge base, factual lookups | Classic RAG. Do not overthink it. |
| Questions that need two or more hops | Agentic RAG |
| Several sources with different shapes | Agentic RAG with source routing |
| Answers must be defensible | Agentic RAG with citation checking |
| Latency budget under two seconds | Classic RAG, and cache aggressively |
The honest trade is latency and cost. Every extra hop is another model call. If your users expect an instant answer, a planning loop will feel broken no matter how correct it is.
Where Teams Get It Wrong
Letting the Loop Run Free
An agent that can keep retrieving will keep retrieving. Cap the hops. Three is usually plenty, and an agent on hop seven is not being thorough, it is lost.
Ignoring Retrieval Quality
Agentic retrieval on a badly chunked corpus is just an expensive way to find the same bad passages repeatedly. Fix chunking and metadata first. That unglamorous work beats the architecture upgrade more often than anyone admits.
No Answer of Last Resort
Give the system a way to say it does not know. Without one, the loop terminates in a confident guess, which is the worst possible ending.
Conclusion
Agentic RAG is not a replacement for retrieval, it is retrieval that knows when it is not finished. Use it when questions genuinely need several hops or several sources, cap the loop, and fix your chunking before you blame your architecture. Most teams who think they need agentic RAG actually need better metadata and one extra week on their index.
Frequently Asked Questions
Does agentic RAG remove hallucinations?
No, but it reduces one common cause: answering from insufficient context. Self-checking catches some of it, and citations catch more.
How much slower is it?
Roughly proportional to hops. A three-hop answer costs about three times a single retrieval in both latency and tokens, so cache the sub-answers you will reuse.
Do I need a graph database?
Only if your questions are genuinely relational. Many teams reach for graphs when the real problem is that their documents have no useful metadata.