Setting	Value	Meaning
chunk_size	256 (try 1500)	Characters per chunk
chunk_overlap	32	Characters repeated between chunks
Embedding model	all-MiniLM-L6-v2	Outputs 384 numbers per text
Retriever k	4	Return the 4 closest chunks
LLM	gpt-4o-mini, temp 0	Most repeatable answers
BM25	k1=1.5, b=0.75	Frequency saturation, length penalty
RRF	1/(k+rank), k=60	Fuse ranked lists
Hybrid weights	[0.5, 0.5]	Semantic, keyword (sum to 1)
Call limits	model 5, tool 2	Guardrails per question

If the question is about...	Reach for
Meaning, "how does X work?"	Semantic (embeddings)
Exact names, errors, constants	BM25
Both kinds of question	Hybrid + RRF
Who calls or depends on what	GraphRAG
Varied, multi-step questions	Agentic RAG
Short, vague questions	HyDE
Well-organised docs, explainability	PageIndex (vectorless)

a free learning module ✍️

Learn the skills behind AI engineering roles.

A learning module for anyone curious about AI Full-Stack Engineer and Forward Deployed Engineer (FDE) roles. You will learn RAG and agentic AI, the core of that job, in 9 short lessons. Every lesson starts with the idea, then the picture, then (optionally) the code. No experience needed.

What is this role? Skip to the basics

AI Full-Stack + FDEthe roles this prepares you for

9 lessonsideas first, code folded

Tap dashed wordsinstant meanings

Try it live3 playgrounds inside

drag the notes around!

who this is for

What do these roles actually do?

Companies now hire engineers to put AI to work inside real products and real businesses. Two job titles you will see again and again are below. This module teaches the technical core they share.

🧱

AI Full-Stack Engineer

Builds the whole product around an AI model: the web app people click on, the API behind it, the database, and the AI part (search over documents, agents that take actions). Think "from the button on the screen to the model and back".

🤝

Forward Deployed Engineer (FDE)

Works directly with a customer to make AI solve their real problem. They learn the customer's messy data and workflow, then build and ship a working solution there, rather than only building a general product.

🔎

Shared skill: RAG

Make an AI answer from a company's own documents instead of guessing. Lessons 2 to 8.

🤖

Shared skill: agents

Give an AI tools and let it decide the next step, safely. Lessons 2, 5 and 9.

📏

Shared skill: proving it works

Tracing and measurement so you can trust and improve the system. Lessons 4 and 8.

Not sure you want this career? That is fine. Read the first lesson or two and see if it clicks. The whole module is free and runs in your browser.

start here

The big picture in 60 seconds

An LLM like ChatGPT knows a lot, but it has never seen your files. Ask it about your project and it will often make something up (that's called a hallucination). It also can only read a limited amount of text at once (its context window).

RAG (Retrieval-Augmented Generation) fixes this in three moves: find the pieces of your data that matter, paste them into the question, and let the LLM answer from them.

📚 Think of an open-book exam. The LLM is a smart student. RAG is the librarian who hands over the right 4 pages before the student writes the answer.

🔎

Retrieval

Search your data and pull out the few pieces that match the question.

➕

Augmentation

Add those pieces to the prompt, so the model reads real facts.

💬

Generation

The LLM writes an answer grounded in what you gave it.

Click each step to see what happens ↓

Part A: prepare your data (once)

Part B: answer a question (every time)

Pick a stepEach bubble above is one small job. Together they are the whole RAG system.

What is an agent?

A plain RAG pipeline always runs the same steps. An agent is an LLM that decides for itself what to do next. You give it tools (like "search the codebase") and a system prompt (its job description). It then loops: think → pick a tool → read the result → answer, or use another tool.

📝

System prompt

Job description: "You are a senior engineer. Always search first."

🔧

Tools

Functions it may call, like search_codebase.

🔄

Reasoning loop

Think, call a tool, read the result, repeat if needed.

✅

Final answer

Once it has enough facts, it replies in plain language.

The story of this course

Each lesson exists because the one before it fails at something. That is how real search systems get built: try, break, fix. You will see a red "Problem" box and a green "Fix" box in every lesson.

Your progress: 0 of 9 lessons done

1Project setup
Tools every project needs 2Basic semantic RAG
Chunks, embeddings, agent 3AST chunking
Split code the smart way 4LangSmith tracing
See inside your agent 5Limiting calls
Guardrails against runaway bills 6BM25 keyword search
Exact-word matching 7Hybrid search + RRF
Best of both 8Measuring quality
Recall, precision, nDCG 9Advanced patterns
Graph, Agentic, HyDE, PageIndex

before any AI

Lesson 1Project setup

Four everyday developer skills. Nothing here is AI, but every AI project depends on them. Each one below says what it is, why you need it, and what you actually type.

📦

Virtual environment

What: a private box of Python packages that belongs to one project only.

Why: project A may need version 1 of a library and project B version 2. Separate boxes stop them fighting, and keep your computer's main Python clean.

Type: create it once, then "activate" it every time you open the project.

📋

pip and requirements.txt

What: pip installs Python libraries. requirements.txt is a shopping list of every library the project needs.

Why: anyone (or any server) can rebuild your exact setup with one command.

Type: pip install -r requirements.txt

🔑

.env file and API keys

What: an API key is a password that lets your code use a paid service (like OpenAI). A .env file is a private text file that stores it.

Why: never type keys directly in code. If you upload that code to GitHub, strangers can spend your money.

Type: put OPENAI_API_KEY=... in .env, and never commit that file.

🌿

Git and GitHub

What: Git saves snapshots ("commits") of your project so you can undo mistakes. GitHub is the website where those snapshots are stored and shown to others.

Why: it is your backup, your history, and your public portfolio.

Type: git init, add, commit, push

🧳 A virtual environment is a separate suitcase per trip. requirements.txt is the packing list. .env is the pocket where the passport stays hidden.

Show the commands Mac and Windows

# 1. make a virtual environment
# Mac / Linux
python3 -m venv .venv
source .venv/bin/activate
# Windows
python -m venv venv
venv\Scripts\activate

# 2. requirements.txt (one library per line)
python-dotenv
langgraph
langchain
langchain-core
langchain-openai
langchain-huggingface
langchain-text-splitters
langchain-chroma
chromadb
sentence-transformers

# install everything on the list
pip install -r requirements.txt

# 3. .env  (never commit this file)
OPENAI_API_KEY=sk-your-key-here

# 4. save your work with Git
git init
git add .
git commit -m "first working version"
git push

Tip: add a file called .gitignore containing .env and .venv so they never get uploaded.

Try it Make a folder called rag, create the virtual environment, install the requirements, and add a .env with your key. Your prompt will show (.venv) when it is active.

Optional: a free key from Groq

The course lists Groq as another place to get a model key. All the code samples use OpenAI's gpt-4o-mini, so Groq is a bonus.

How the code loads your key

Two lines: load_dotenv() reads .env, and LangChain finds OPENAI_API_KEY on its own.

Next With a clean project ready, we can build the first RAG system.

the core lesson

Lesson 2Basic semantic RAG

You will build a bot that answers questions about a folder of Python code. File: 1_semanticrag.py. Five ideas make it work: chunking, embeddings, a vector database, a retriever, and an agent.

ProblemYou can't paste a whole codebase into an LLM. It's too big, too expensive, and the model gets lost.

FixCut the code into small pieces, store them in a searchable form, and fetch only the best 4 for each question.

Idea 1: Chunking (cutting into pieces)

A chunk is one small piece of a file. We cut files up because search works better on focused pieces, and because the LLM can only read so much. Chunk size is a trade-off:

Small chunks (256 characters)

Good: precise, cheap, the match is exactly the right lines.
Bad: a function may be cut in half, so the meaning gets lost.

Big chunks (1500 characters)

Good: whole functions stay together, more context.
Bad: a chunk mixes several topics, search is fuzzier, and each answer costs more.

Characters vs tokens

The course sets chunk_size=256 in characters. An LLM counts in tokens (about 4 characters each in English). Same idea, different ruler.

Overlap (32)

The last 32 characters of one chunk repeat at the start of the next. That way a sentence cut at the border is still understood.

RecursiveCharacterTextSplitter

A splitter that tries the biggest natural break first (class), then smaller ones, until each chunk fits. See the ladder below.

The splitter's ladder: it goes down only if the piece is still too big

split at "class"→then "def"→then blank lines→then new lines→then characters

It knows Python-looking text patterns, but it does not truly understand code. Lesson 3 fixes that.

🧪 Playground: watch code get chunked

Chunk size150

Idea 2: Embeddings (meaning as numbers)

An embedding turns text into a list of numbers that captures its meaning. This course uses the model all-MiniLM-L6-v2, which always outputs 384 numbers per text (a 384-dimensional vector). Texts with similar meaning get similar numbers.

🗺️ Imagine a map where every sentence has GPS coordinates. "Create invoice" and "generate bill" sit right next to each other. "Refund a payment" is a few streets away. "Bake a cake" is in another country.

1. Tokenize

The model breaks the text into small word-pieces.

2. Look up

Each piece has learned numbers. The model blends them, aware of the surrounding words.

3. Merge

Everything is squeezed into one fixed-length list: 384 numbers, whatever the input length.

Why it beats keywords: it finds "generate bill" when your code says "create invoice". Similarity is measured with cosine similarity (do two arrows point the same way?).

Idea 3: Vector database (Chroma)

A vector database stores every chunk with its embedding, and can quickly find the ones closest to a new question. Chroma is a beginner-friendly one that runs in your project. For a bigger production app, the course later switches to PostgreSQL with pgvector. Pinecone and Weaviate are similar tools.

k=4 means top-k: return the 4 closest chunks, no more.

Idea 4 and 5: Retriever tool + Agent

A retriever is "question in, best chunks out". We wrap it with create_retriever_tool so the agent can use it as a tool named search_codebase. The agent's system prompt says: always search first, name the file and function, and say "I could not find that in the codebase" if it isn't there. The model is gpt-4o-mini at temperature=0 (least random, most repeatable).

Show the code 1_semanticrag.py, the core functions

from dotenv import load_dotenv
load_dotenv()
from langchain_core.documents import Document
from langchain_text_splitters import Language, RecursiveCharacterTextSplitter
from langchain_chroma import Chroma
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_openai import ChatOpenAI
from langchain_core.tools.retriever import create_retriever_tool
from langchain.agents import create_agent

CHUNK_SIZE = 256

# STEP 1: LOAD  read every .py file into a Document
def load_codebase(repo_path):
    docs = []
    for path in Path(repo_path).rglob("*.py"):
        text = path.read_text(encoding="utf-8", errors="ignore")
        docs.append(Document(page_content=text, metadata={"source": str(path)}))
    return docs

# STEP 2: CHUNK  cut files into small pieces
def chunk_code(docs):
    splitter = RecursiveCharacterTextSplitter.from_language(
        language=Language.PYTHON, chunk_size=CHUNK_SIZE, chunk_overlap=32)
    return splitter.split_documents(docs)

# STEP 3: EMBED + STORE  numbers for each chunk, saved in Chroma
def build_vector_store(chunks):
    embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
    return Chroma.from_documents(chunks, embedding=embeddings)

# STEP 4: RETRIEVER TOOL + AGENT
def build_agent(vector_store):
    retriever_tool = create_retriever_tool(
        vector_store.as_retriever(search_kwargs={"k": 4}),
        name="search_codebase",
        description="Search the codebase for relevant functions, classes, or logic.")
    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
    return create_agent(llm, tools=[retriever_tool], system_prompt=(
        "You are a senior engineer. Always use search_codebase before answering. "
        "Reference specific file and function names. "
        "If not found say 'I could not find that in the codebase'."))

# STEP 5 and 6: THE QUESTION LOOP
docs = load_codebase(repo_path)
chunks = chunk_code(docs)
agent = build_agent(build_vector_store(chunks))
while True:
    question = input("\nYou: ").strip()
    if not question or question.lower() in ("exit", "quit"): break
    for step in agent.stream({"messages": [{"role": "user", "content": question}]},
                             stream_mode="values"):
        last = step["messages"][-1]
        if not getattr(last, "tool_calls", None):
            print(f"Agent: {last.content}")

Reading it: steps 1 to 3 run once when the program starts (Part A in the diagram). The loop at the bottom runs for every question (Part B).

Try it Run python 1_semanticrag.py and ask "What is in auth.py?". Then change CHUNK_SIZE to 1500 and ask again. Which questions now work, and which got worse?

What breaks A character splitter can cut a function in half. Then the bot sees half a function and answers badly. We need to cut where the code naturally ends.

smarter chunks

Lesson 3AST-based chunking

Code is not just text. It has structure: classes contain functions, functions contain lines. An AST (Abstract Syntax Tree) is the tree Python builds internally to understand your code. We can use that tree to cut exactly along the natural seams.

ProblemCharacter splitting treats code like a novel. It doesn't know where a function begins or ends.

FixParse the code into a tree first, then make one chunk per class or function. Understand first, chunk second.

🧩 A character splitter cuts a pizza with your eyes closed. An AST splitter sees the slices already marked and cuts on them.

What a tree looks like

For x = 1 + 2 Python sees:
Module → Assign → (target x) and (BinOp: 1, Add, 2). Every piece of code becomes a labelled node.

Where else ASTs are used

Compilers and interpreters (to run code), linters (to find mistakes), and editors (autocomplete, refactoring, "go to definition").

The five tools from Python's `ast` module

Function	Plain-English job
ast.parse()	Turn source code text into a tree.
ast.dump()	Print the tree so a human can read it. Great for learning.
ast.walk()	Visit every node in the tree, one by one.
ast.iter_child_nodes()	List a node's direct children. We use it to tell each node who its parent is.
ast.get_source_segment()	Give back the original text for one node. This is the chunk we store.

The AST chunking flow

Source code→AST tree→Find classes and functions→Extract their text→Embed and store

Rule used: one chunk per top-level class (methods stay inside it) and one per top-level function. Scroll up to the playground and switch to "AST splitter" to see it.

Where AST chunking still fails (great to understand)

Questions that work well

"What is AuthService for?"
"What functions are inside AuthService?"
"What protects against privilege escalation?"
"Trace the call chain when create_task is called."

Questions that struggle

"Where is BillingError raised?"
"Which file imports both Task and Status?"
"Where is _user_plans defined?"

Why? Our rule keeps only top-level classes and functions. Imports and module-level variables like _user_plans are thrown away, so they are never searchable. The playground shows them as striped "dropped" boxes.

Show the code 2_ast_demo.py and 3_semanticrag_ast.py

import ast

# See a tree for yourself
tree = ast.parse("x = 1 + 2")
print(ast.dump(tree, indent=2))

# The AST chunker: one chunk per top-level class / function
def chunk_code(docs):
    chunks = []
    for doc in docs:
        source = doc.metadata.get("source", "unknown")
        code = doc.page_content
        try:
            tree = ast.parse(code)
        except SyntaxError:
            chunks.append(doc)      # can't parse? keep the whole file
            continue

        # ast.walk doesn't tell us parents, so add them ourselves
        for node in ast.walk(tree):
            for child in ast.iter_child_nodes(node):
                child.parent = node

        for node in ast.walk(tree):
            parent = getattr(node, "parent", None)
            # top-level class -> whole class including its methods
            if isinstance(node, ast.ClassDef) and isinstance(parent, ast.Module):
                text = ast.get_source_segment(code, node)
                if text:
                    chunks.append(Document(page_content=text,
                        metadata={"source": source, "type": "class", "name": node.name}))
            # top-level function (not a method)
            elif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and isinstance(parent, ast.Module):
                text = ast.get_source_segment(code, node)
                if text:
                    chunks.append(Document(page_content=text,
                        metadata={"source": source, "type": "function", "name": node.name}))
    return chunks

Bonus: every chunk now carries metadata (type and name), which helps the agent cite exact names.

Try it Run python 2_ast_demo.py and read the three trees. Then run 3_semanticrag_ast.py, set k to 1, and ask "What are the functions inside AuthService?"

What's next Our bot works, but it's a black box. How do we see what it searched, how long it took, and what it cost? Tracing.

see inside

Lesson 4LangSmith tracing

When your bot gives a wrong answer, you need to know why. Did it search? What did it find? Which step was slow? Tracing records every step of one request so you can replay it.

ProblemThe agent is a black box. Wrong answers, slow answers and big bills are all invisible.

FixTurn on LangSmith. Every LLM call, tool call and retrieval shows up on a dashboard.

Logging vs tracing

Logging is a diary of messages you chose to print. Tracing is a full recording of one request, step by step, with inputs, outputs, time and cost attached to each step.

Observability

The ability to understand what a system is doing from the outside, just by looking at its data. Tracing is how you get it for AI apps.

⏱️

Latency

Which step took the longest?

💸

Cost

Tokens used per query, and which questions are expensive.

🔍

Retrieval quality

Did the search return the chunks you expected, or miss the answer?

Setup is tiny. Create a free LangSmith account, copy your key, and add two lines to .env. No code changes.

Show the setup .env

LANGSMITH_TRACING="true"
LANGSMITH_API_KEY=your-langsmith-key

Docs: docs.langchain.com/langsmith

Try it Ask a question, open the LangSmith dashboard, and click the newest trace. Find (1) what the agent searched for, (2) which chunks came back, (3) how many tokens it used.

What's next Now that we can see the agent, we notice it can loop and spend money. Time for guardrails.

safety rails

Lesson 5Limiting model and tool calls

An agent decides how many times to call the LLM and its tools. Left alone, a confused agent can loop again and again, and every loop costs money.

ProblemAn agent can spiral: search, search again, search again. That means slow answers and runaway API bills.

FixAdd middleware that counts calls and stops the agent politely at a limit.

🚦 Middleware is a checkpoint on the road. Every request passes through it, and it can count, block or reshape the request before it moves on.

ModelCallLimitMiddleware

Caps how many times the LLM may be called in one run. run_limit=5 means at most 5 model calls per question.

ToolCallLimitMiddleware

Caps how many times a specific tool may be used. tool_name="search_codebase", run_limit=2 means at most 2 searches per question.

exit_behavior="end" means when the limit is hit, the agent ends the run gracefully instead of crashing. Good apps also show the user a clear message rather than failing silently.

Show the code

from langchain.agents.middleware import ModelCallLimitMiddleware, ToolCallLimitMiddleware

return create_agent(
    llm, tools=[retriever_tool],
    system_prompt="...",
    middleware=[
        ModelCallLimitMiddleware(run_limit=5, exit_behavior="end"),
        ToolCallLimitMiddleware(tool_name="search_codebase",
                                run_limit=2, exit_behavior="end"),
    ])

Watch the name! tool_name must match the tool's real name exactly. The course's hybrid example writes "retriever_tool" (the Python variable) while the tool is actually named "search_codebase", so that limit would never trigger. Always copy the string given in name=.

Try it Set ToolCallLimitMiddleware(... run_limit=1) and ask a question that needs two searches. Watch how the answer changes.

What's next Semantic search misses exact names sometimes. Let's learn the older, simpler alternative: keyword search.

the classic

Lesson 6BM25: keyword search

BM25 finds documents by matching the actual words. There are no embeddings and no AI, but it is what powers Elasticsearch and Solr. It is superb for exact things: function names, class names, error messages.

ProblemAsk for BillingError and semantic search may return "something about billing" instead of the exact class.

FixAdd a keyword retriever that rewards exact word matches.

Step 1: the inverted index

A normal index says "document → words". An inverted index flips it: "word → documents that contain it". It is the index at the back of a textbook. To find "invoice" you don't read every page, you look it up.

Step 2: three scoring factors

🔁

Term frequency (TF)

The more times your word appears in a document, the higher the score, with diminishing returns (the 5th mention adds less than the 1st).

💎

Rarity (IDF)

A word in every document is useless for ranking. A word in only one document is a strong clue, so it earns a big weight.

📏

Length penalty

Long documents contain more words by accident. BM25 slightly discounts them. (Tuned by k1=1.5 and b=0.75.)

Small honesty note: the course shows this as "TF + rarity − length penalty" to keep it simple. In the real formula, each word's rarity multiplies a length-adjusted term frequency. The idea is the same.

🧪 Playground: see BM25 succeed, then fail

Five tiny documents. Pick a query and see how BM25 ranks them.

The lexical gap

Query "generate bill" but the code says "create invoice". Different words, same meaning, so BM25 sees no link at all. That gap is exactly why semantic search exists.

When to pick which

BM25: exact identifiers, error text, constants, names.
Semantic: "how does X work?", vague or paraphrased questions.
Neither wins everywhere.

Show the code BM25 retriever with rank_bm25

# add to requirements.txt:  rank_bm25
from rank_bm25 import BM25Okapi
from langchain_core.retrievers import BaseRetriever
from pydantic import ConfigDict

class BM25Retriever(BaseRetriever):
    """Thin LangChain-compatible retriever backed by rank_bm25."""
    docs: list
    bm25: object
    k: int = 4
    model_config = ConfigDict(arbitrary_types_allowed=True)

    def _get_relevant_documents(self, query, *, run_manager):
        tokens = query.lower().split()               # split the question into words
        scores = self.bm25.get_scores(tokens)        # one score per chunk
        top = sorted(range(len(scores)), key=lambda i: scores[i], reverse=True)[: self.k]
        return [self.docs[i] for i in top]

def build_bm25_retriever(chunks):
    tokenized = [doc.page_content.lower().split() for doc in chunks]
    return BM25Retriever(docs=chunks, bm25=BM25Okapi(tokenized))

Everything else (agent, prompt) stays the same. Only the retriever is swapped.

Try it Run the BM25 bot and ask "Where is BillingError defined?" then "How does the app handle payments?". Notice which question BM25 is better at.

What's next We now have two retrievers with opposite strengths. Combine them.

best of both

Lesson 7Hybrid search and RRF

Hybrid search is not one technique. It is a design: run several retrievers, then merge their results into one list. The most common pair is keyword (BM25) plus semantic (embeddings). Others: semantic + graph, semantic + SQL, or two vector models together.

ProblemSemantic misses exact names. BM25 misses meaning. Each has blind spots. But their scores are on different scales, so you can't just add them.

FixMerge by rank position, not by score, using Reciprocal Rank Fusion (RRF).

How RRF works

🗳️ Two judges each rank their top 4 chunks. Points go to every chunk based on its rank, and a chunk named by both judges gets points twice.

The formula

score = Σ 1 / (k + rank)
Add this up over every retriever that returned the chunk. k is usually 60.

Why ranks, not scores

BM25 scores and cosine scores use different scales and can't be compared. Ranks always mean the same thing: 1st, 2nd, 3rd.

Why k = 60

Without it, rank 1 = 1.0 and rank 2 = 0.5, a huge gap. With 60: 0.0164 vs 0.0161. Now no single #1 can dominate, and agreement between retrievers matters most.

🧪 Worked example: "How does the app enforce task limits per user?"

k60

Weights in LangChain's EnsembleRetriever

EnsembleRetriever does the RRF merge for you. Its weights=[0.5, 0.5] give semantic and keyword equal say. Raise the first for conceptual questions ("how does X work?"). Raise the second for exact identifiers or error strings. The demo lets you pass --weights 0.7 0.3 from the command line.

✅

Simple and fast

No training needed.

🔌

Works with anything

Fuse any retrievers without normalising scores.

🤝

Rewards consensus

Chunks found by several retrievers rise to the top.

Show the code hybrid retriever + a plain RRF function

from langchain_classic.retrievers.ensemble import EnsembleRetriever

def build_hybrid_retriever(chunks):
    vector_retriever = build_vector_retriever(chunks)   # semantic (Lesson 2)
    bm25_retriever   = build_bm25_retriever(chunks)     # keyword  (Lesson 6)
    return EnsembleRetriever(
        retrievers=[vector_retriever, bm25_retriever],
        weights=[0.5, 0.5])                             # must sum to 1.0

# RRF from scratch, so you can see there's no magic
RRF_K = 60
scores = {}
def add_scores(results):
    for rank, chunk in enumerate(results, 1):
        scores[chunk] = scores.get(chunk, 0.0) + 1 / (RRF_K + rank)

add_scores(SEMANTIC_RESULTS)
add_scores(LEXICAL_RESULTS)
ranked = sorted(scores.items(), key=lambda x: x[1], reverse=True)

Try it Run the hybrid bot with --weights 0.8 0.2 and then --weights 0.2 0.8. Ask both a conceptual and an exact-name question each time.

What's next "Feels better" is not proof. How do we measure whether retrieval improved?

prove it works

Lesson 8Measuring retrieval quality

To improve a search system you must measure it. The recipe: write questions where you already know which documents are correct, run your retriever, and score it.

Golden dataset

A hand-made answer key: each question paired with the IDs of the documents that truly answer it. "Truly answers" means the content answers it, not just the same topic.

The retrieved list

What your system returned for that question, in order, cut off at K results (K = 1, 3 or 5).

🧪 Worked example

Question: "What is Acme's refund policy?"
Correct docs: doc_1, doc_2
System returned (K=3): doc_3, doc_1, doc_9

Metric	Question it answers	Here
Recall@K	Of all the correct docs, how many did we find?	1 of 2 = 0.50
Precision@K	Of what we returned, how much was correct?	1 of 3 = 0.33
MRR	How high was the first correct doc? (1 ÷ its rank)	rank 2 → 0.50
Hit rate	Did we find at least one correct doc?	1.00 (yes)
nDCG@K	Are the correct docs near the top? (higher = better order)	≈ 0.39

Why check several K values

K=1 tests "is the very top result right?". K=5 tests "did the answer make it anywhere in the list?". A drop between them tells you ranking is the issue, not coverage.

Failure analysis

List the questions where recall < 1.0 and see which correct docs were missed. That list is your to-do for fixing chunking, weights or embeddings.

✅ 0.7 or more

Healthy.

⚠️ 0.5 to 0.7

Needs work.

❌ below 0.5

Something is broken.

Show the code the five metric functions

import math

def num_hits_at_k(retrieved_ids, relevant_ids, k):
    relevant = set(relevant_ids)
    return sum(1 for d in retrieved_ids[:k] if d in relevant)

def recall_at_k(retrieved_ids, relevant_ids, k):
    return num_hits_at_k(retrieved_ids, relevant_ids, k) / len(relevant_ids) if relevant_ids else 0.0

def precision_at_k(retrieved_ids, relevant_ids, k):
    return num_hits_at_k(retrieved_ids, relevant_ids, k) / k if k > 0 else 0.0

def mrr(retrieved_ids, relevant_ids):
    relevant = set(relevant_ids)
    for rank, d in enumerate(retrieved_ids, start=1):
        if d in relevant: return 1.0 / rank
    return 0.0

def hit_rate_at_k(retrieved_ids, relevant_ids, k):
    return 1.0 if set(relevant_ids) & set(retrieved_ids[:k]) else 0.0

def ndcg_at_k(retrieved_ids, relevant_ids, k):
    relevant = set(relevant_ids)
    dcg = sum(1.0 / math.log2(r + 1) for r, d in enumerate(retrieved_ids[:k], 1) if d in relevant)
    idcg = sum(1.0 / math.log2(r + 1) for r in range(1, min(len(relevant), k) + 1))
    return dcg / idcg if idcg > 0 else 0.0

The course's script builds a 10-document refund/shipping/password knowledge base (with one distractor about company history) and six golden questions, then loops K over 1, 3, 5.

Try it Add two questions of your own to the golden set. Predict the metrics on paper first, then run the script.

What's next Some questions need more than "find similar text". Lesson 9 covers four advanced ideas.

level up

Lesson 9Advanced RAG patterns

You do not need these on day one. Each one solves a specific limit of everything so far. Read them as "when would I reach for this?".

🕸️

GraphRAG: follow the connections

Problem: "Which classes depend on PaymentGateway?" is about relationships, not similar text.

Idea: an LLM reads each file and writes facts as (subject, relationship, object), like OrderService -[CALLS]-> PaymentGateway. Those facts form a knowledge graph. At question time: find the entities in the question, then walk out depth hops (default 2) to collect nearby facts.

Two versions: NetworkX keeps the graph in memory (easy for learning). Neo4j stores it in a real graph database (persists, scales).

Use when: questions are about dependencies, call chains, "what touches what".

🧭

Agentic RAG: let the agent choose

Problem: one fixed search method can't suit every question.

Idea: give the agent several tools and let it decide. Example 1: a router agent that picks semantic or lexical search. Example 2: one agent with four tools: semantic search, keyword search, symbol_search (find where a function or class is defined via AST) and dependency_search (what a function calls). It can combine tools and break a complex question into smaller ones.

Use when: questions vary a lot, or need several steps. Cost: more LLM calls, so keep your call limits.

🪄

HyDE: search with a fake answer

Problem: a short question ("how are limits enforced?") looks nothing like real code.

Idea: ask the LLM to write a hypothetical answer first. It may be wrong in details, but it looks like the real thing. Embed that and search with it. Same-shaped text finds same-shaped text.

Use when: questions are short or vague and documents are long and technical.

🌲

Vectorless RAG (PageIndex): reason, don't compare

Problem: similar is not the same as relevant. "Payment failures" may not look like "Refund Processing Timeline", yet that is the answer.

Idea: no embeddings at all. Build a tree: folders → files → chunks, each with an LLM-written summary. The agent reads only the summaries (get_structure), decides which branches matter, then fetches only those chunks (get_chunk_content).

Use when: documents are well organised and you want explainable, reasoning-based retrieval. Cost: an LLM call for every file when building.

Compare every approach

Approach	How it finds things	Best at	Weak at
Semantic (vectors)	Meaning similarity	Concepts, paraphrases	Exact names, IDs
BM25 (keyword)	Word matching	Identifiers, errors	Synonyms, meaning
Hybrid + RRF	Merge ranked lists	General purpose	Two indexes to maintain
GraphRAG	Walk relationships	Dependencies, call chains	Costly to build; depends on LLM extraction
Agentic RAG	Agent picks tools	Varied, multi-step questions	More LLM calls, less predictable
HyDE	Search with a fake answer	Short or vague questions	Extra LLM call; a bad guess misleads
PageIndex (vectorless)	LLM reasons over a summary tree	Structured docs, explainability	Build-time cost; large trees

And how chunking methods compare

Method	Understands code?	Strength	Weakness
Fixed characters	No	Trivial to build	Cuts anywhere
RecursiveCharacterTextSplitter	Text patterns only	Prefers class and def breaks	Doesn't know real structure
AST-based	Yes	Whole functions and classes	Drops imports and constants; Python-specific

Show key snippets one idea from each pattern

# ---- GraphRAG: facts as edges, retrieval by walking the graph (NetworkX) ----
class CodeRelationship(BaseModel):
    subject: str      # e.g. "OrderService"
    predicate: str    # e.g. "CALLS"
    obj: str          # e.g. "PaymentGateway"

graph = nx.DiGraph()
for r in relationships:
    graph.add_edge(r.subject.strip(), r.obj.strip(), relation=r.predicate)

neighbourhood = nx.ego_graph(graph, node, radius=depth, undirected=True)

# ---- Agentic RAG: tools the agent may choose between ----
@tool
def symbol_search(symbol: str) -> str:
    """Find the exact definition of a function or class by name."""
    ...
agent = create_agent(model=llm,
    tools=[semantic_search, lexical_search, symbol_search, dependency_search],
    system_prompt="Investigate before answering. Use multiple tools when needed.")

# ---- HyDE: embed a hypothetical answer, not the question ----
hypothetical_doc = hyde_agent.invoke({"messages": [{"role": "user", "content": query}]})
docs = vector_store.similarity_search(hypothetical_doc_text, k=4)

# ---- PageIndex: the agent reads summaries, then fetches chunks ----
@tool
def get_structure() -> str:
    """Full codebase tree with summaries. Call this first."""
@tool
def get_chunk_content(node_id: str) -> str:
    """Source code of one chunk."""

These are condensed.

Papers to explore later

HyDE

Query expansion by hypothetical documents. arXiv 2212.10496

RAPTOR

Tree-based, hierarchical RAG. arXiv 2401.18059

GraphRAG (Microsoft)

Graph-based retrieval and reasoning. arXiv 2404.16130

Self-RAG

Self-reflective, validating RAG. arXiv 2310.11511

PageIndex

Vectorless, structure-aware retrieval. GitHub

what's next

Turn it into a real product

Once the ideas click, the next step is a portfolio project: DocBot, a full-stack AI app. You will rebuild this RAG system as a web service that other people can use.

FastAPI backendEndpoints like /upload and /ask, with streaming answers.

PostgreSQL + pgvectorA production database for embeddings instead of Chroma.

Streamlit frontendA chat window and file upload in a few lines of Python.

Docker + deployPackage the app, host it (for example on Render), share a live link.

Finish with a strong README (architecture diagram, setup steps, example questions). Then tailor your resume and GitHub around what you built.

stuck on a word?

Learn the skills behind AI engineering roles.

What do these roles actually do?

AI Full-Stack Engineer

Forward Deployed Engineer (FDE)

Shared skill: RAG

Shared skill: agents

Shared skill: proving it works

The big picture in 60 seconds

Retrieval

Augmentation

Generation

What is an agent?

System prompt

Tools

Reasoning loop

Final answer

The story of this course

Lesson 1Project setup

Virtual environment

pip and requirements.txt

.env file and API keys

Git and GitHub

Optional: a free key from Groq

How the code loads your key

Lesson 2Basic semantic RAG

Idea 1: Chunking (cutting into pieces)

Small chunks (256 characters)

Big chunks (1500 characters)

Characters vs tokens

Overlap (32)

RecursiveCharacterTextSplitter

Idea 2: Embeddings (meaning as numbers)

1. Tokenize

2. Look up

3. Merge

Idea 3: Vector database (Chroma)

Idea 4 and 5: Retriever tool + Agent

Lesson 3AST-based chunking

What a tree looks like

Where else ASTs are used

The five tools from Python's ast module

Where AST chunking still fails (great to understand)

Questions that work well

Questions that struggle

Lesson 4LangSmith tracing

Logging vs tracing

Observability

Latency

Cost

Retrieval quality

Lesson 5Limiting model and tool calls

ModelCallLimitMiddleware

ToolCallLimitMiddleware

Lesson 6BM25: keyword search

Step 1: the inverted index

Step 2: three scoring factors

Term frequency (TF)

Rarity (IDF)

Length penalty

The lexical gap

When to pick which

Lesson 7Hybrid search and RRF

How RRF works

The formula

Why ranks, not scores

Why k = 60

Weights in LangChain's EnsembleRetriever

Simple and fast

Works with anything

Rewards consensus

Lesson 8Measuring retrieval quality

Golden dataset

The retrieved list

Why check several K values

Failure analysis

✅ 0.7 or more

⚠️ 0.5 to 0.7

❌ below 0.5

Lesson 9Advanced RAG patterns

GraphRAG: follow the connections

The five tools from Python's `ast` module