A language model only knows two things: patterns learned during training, and whatever text is in its current context window. Everything else, a company's live database, this morning's news, a private document, has to be supplied at request time. RAG and MCP are two different mechanisms for supplying it, built for two different kinds of gaps.
RAG supplies missing knowledge: it retrieves relevant text from an external source and inserts it into the context before the model generates an answer. MCP supplies missing capability: it defines a standard interface so a model can invoke functions in external systems, not just read about them.
Learning Objectives
- Define RAG and MCP precisely, and explain the specific gap each one fills.
- Read the architecture and terminology behind both, step by step.
- Apply a decision framework to determine which one a given system design actually requires.
Key Terms
| Term | Definition |
|---|---|
| Embedding | A piece of text converted into a list of numbers (a vector) that represents its meaning, so similarity can be compared mathematically |
| Vector database | A database built specifically to store embeddings and quickly find the ones closest in meaning to a new query |
| Retriever | The component that searches a vector database and returns the most relevant matches for a given question |
| MCP host | The AI application the user actually interacts with (a chat app, an IDE assistant) |
| MCP server | A program that exposes a specific set of tools or data sources to any MCP host that connects to it |
| Tool | One specific callable action an MCP server exposes, such as send_email or run_query |
RAG: Retrieval-Augmented Generation
RAG was introduced in a 2020 Meta AI research paper as a way to combine a language model's generative ability with a separate, updatable knowledge source. Structurally, it's a search step placed before a generation step: retrieve relevant passages first, then generate an answer conditioned on them.
This matters because a model's training data has a fixed cutoff and is fixed once training ends. A support document updated an hour ago doesn't exist in the model's weights at all, no matter how it's prompted. RAG sidesteps that entirely by not depending on the model to already know the answer, it only needs the model to summarize text it's handed directly.
The RAG Architecture
Query
Embedding Model
text → vector
Vector Database
Retriever
top-K matches
Language Model
generates answer
An embedding model converts an incoming query into a vector. That vector is compared against a vector database holding embeddings of pre-processed source documents, using a similarity measure (commonly cosine similarity). The retriever returns the top-scoring matches, and those get inserted into the prompt sent to the language model.
A Minimal RAG Pipeline
function answerWithRAG(query, vectorDB, llm) {
queryVector = embed(query)
matches = vectorDB.search(queryVector, topK: 3)
context = matches.map(m => m.text).join("\n---\n")
return llm.generate({
system: "Answer only using the provided context.",
context: context,
question: query
})
}The system instruction matters as much as the retrieval step, without it, nothing stops the model from ignoring the retrieved context and answering from memory anyway if the context seems insufficient.
RAG: Advantages and Disadvantages
- + Answers reflect current source data, not a fixed training cutoff
- + Reduces hallucination by grounding generation in retrieved text
- + Updating knowledge means editing a document, not retraining a model
- + Works with private, domain-specific data the model was never trained on
- − Adds retrieval latency to every request
- − Answer quality is bounded by retrieval quality, not just model quality
- − Requires infrastructure: an embedding model, a vector database, a chunking strategy
- − Poorly chunked or irrelevant source documents degrade output silently
MCP: Model Context Protocol
MCP is an open protocol, published by Anthropic in late 2024, that standardizes how an AI application discovers and calls external tools. Before a shared protocol existed, every AI application that wanted to integrate with, say, GitHub, had to write its own custom GitHub-specific integration. MCP inverts that: a GitHub MCP server is written once, and any MCP-compatible application can use it without custom integration work.
The protocol defines three primitives an MCP server can expose: tools (callable functions with side effects), resources (readable data, like files), and prompts (reusable prompt templates). Most practical usage centers on tools.
The MCP Architecture
AI Client
the host app
MCP Client
MCP Server
exposes tools
Tool
e.g. Gmail, DB
Result Returned
The host application (where the user types their request) contains an MCP client. That client holds a connection to one or more MCP servers, each exposing its own set of tools. When the model decides a tool call is needed, the client sends a structured call to the relevant server, the server executes it against the real system, and the result is returned to the model as part of its context.
A Minimal MCP Tool Call
// Defined once, inside an MCP server
registerTool("get_order_status", async (params) => {
const order = await db.findOrder(params.orderId)
return { status: order.status, eta: order.estimatedDelivery }
})
// Called from any connected MCP client, no custom integration needed
result = mcpClient.callTool("get_order_status", { orderId: "A1042" })
// result: { status: "in_transit", eta: "2026-09-14" }Notice registerTool and callTool are both generic, nothing about this shape changes whether the tool touches a database, a filesystem, or a calendar. That uniformity is what a shared protocol buys.
MCP: Advantages and Disadvantages
- + One protocol replaces N custom integrations
- + A tool built once works across every MCP-compatible host
- + Tool calls are structured and typed, not free-form text parsing
- + Separates "what the model decides to do" from "how the action is actually executed"
- − Capability is entirely bounded by which servers and tools are connected
- − Executing real actions introduces real risk: a malformed or malicious call can mutate real data
- − Requires deliberate authorization and permission design per tool, not just per server
A Decision Framework
| System requirement | Use |
|---|---|
| Answer questions from a large, changing body of documents | RAG |
| Perform a side-effecting operation (write, update, delete, send) | MCP |
| Ground answers in private data the model wasn't trained on | RAG |
| Read live, structured data from one specific known source | MCP tool (read-only) |
| Verify a condition against a document, then act on the result | RAG + MCP together |
The dividing question: is the missing piece information, or capability? Information gaps point to RAG. Capability gaps point to MCP. Some workflows have both gaps in the same request.
Worked Example: An IT Helpdesk Assistant
A request comes in: "My laptop won't connect to the VPN, and I think my access needs to be reset." This has one information gap and one capability gap, handled in sequence.
- RAG retrieves the relevant section of the internal IT troubleshooting guide covering VPN connection failures, and the model checks it against what the user already described.
- If the guide's steps don't resolve it and an access reset is genuinely warranted, MCP calls a tool against the identity management system to actually reset the user's VPN credentials.
- The assistant reports back what it found in the guide and confirms the reset actually happened, not just that it should happen.
User Request
RAG: Check Guide
information gap
MCP: Reset Access
capability gap
Confirm Resolved
Notice the ordering isn't arbitrary: acting (MCP) before checking whether the guide even calls for that action (RAG) would risk resetting credentials unnecessarily.
Best Practices
- Keep RAG's retrieved chunks small and specific rather than passing entire documents, precision in retrieval matters more than volume.
- Treat every MCP tool that modifies state as requiring explicit authorization, never invoke it automatically without a permission check.
- Don't reach for an MCP tool when the need is read-only lookup of static knowledge, RAG is the better fit and carries less risk.
- Fix an embedding model choice deliberately before scaling a RAG system, switching later usually means re-embedding the entire corpus.
Common Beginner Mistakes
- Treating RAG and MCP as interchangeable rather than as solutions to two different classes of problem.
- Wrapping a read-only lookup in an MCP tool call when a retrieval step would do the same job with less operational risk.
- Expecting RAG to let a model take an action, retrieval only ever returns text; it never executes anything on its own.
- Skipping validation of retrieved chunks, a retriever returning low-relevance matches produces confidently wrong answers just as easily as no retrieval at all.
FAQ
Does using RAG or MCP require a different or specially trained model?
No. Both work with an off-the-shelf language model. RAG changes what's placed in the prompt before generation; MCP changes what the model is able to invoke during generation. Neither requires retraining.
Can an MCP server return large amounts of retrieved text instead of using RAG?
Technically yes, a tool could return an entire document, but that skips the similarity search that makes retrieval scale. RAG exists specifically to search across a large, unstructured corpus and return only the relevant fraction of it.
Is RAG a replacement for fine-tuning?
They solve different problems. Fine-tuning adjusts a model's weights to change its behavior or style, a slow, resource-intensive process. RAG changes nothing about the model itself, it supplies fresh context at request time. Many systems use fine-tuning for behavior and RAG for knowledge.
Interview Questions
In one sentence, what gap does RAG close, and what gap does MCP close?
RAG closes an information gap by retrieving relevant external text before generation. MCP closes a capability gap by giving a model a standard way to invoke real actions in external systems.
Why is a system prompt instructing the model to "answer only from the provided context" important in a RAG pipeline?
Without it, the model can silently fall back to its training data when retrieved context seems thin or irrelevant, defeating the purpose of retrieval and reintroducing the staleness/hallucination problem RAG exists to fix.
What are the three primitives an MCP server can expose, and which one is used most in practice?
Tools (callable functions with side effects), resources (readable data), and prompts (reusable templates). Tools see the most practical usage, since they're what let a model actually perform actions.
Design a system for an assistant that answers HR policy questions and can also submit a leave request. Where does RAG fit, and where does MCP fit?
RAG retrieves the relevant leave-policy text to answer questions and to check whether a given request is valid (enough accrued days, correct notice period). MCP calls a tool against the HR system to actually create the leave request record once validity is confirmed.
Why can RAG only reduce hallucination, not eliminate it?
The model still generates its final answer probabilistically from the retrieved text and its own parameters, it can still misread, over-generalize, or blend retrieved content with prior training data incorrectly. Retrieval reduces the model's need to guess, but generation is still a generative process, not a lookup.
What's a security failure mode specific to MCP that doesn't apply to RAG?
An MCP tool call executes a real, potentially irreversible action, so a model invoking the wrong tool, or invoking the right tool with wrong parameters, can cause real damage (deleting a record, sending an unintended message). A RAG system only ever returns and summarizes text; a bad retrieval produces a wrong answer, not a side effect.
Summary
RAG and MCP address two structurally different limitations of a language model: RAG supplies missing information by retrieving it before generation, MCP supplies missing capability by standardizing how the model invokes real actions. Identifying which kind of gap a system has, information or capability, determines which one to reach for, and many real systems need both.
What's Next?
The next lessons in this course cover the pieces RAG and MCP are built from in more depth: what a language model actually is, tokens and context windows, embeddings, and vector databases.