skills · Retrieval-augmented generation
RAG Fundamentals
Retrieval on your own documents, measured.
By the end, you'll be able to:
- A test set written before the system
- A baseline that each later change is compared with
- Keyword, embedding, hybrid and reranked search compared on your set
- Answers that cite their chunks, and a fixed reply when the answer is not found
- A planted-instruction test
- A ship note with numbers, failures and cost
From $9/month7.5 hours5 chapters20 lessons
“The model does not know your documents.”
About this course
Who it's for
- You write basic Python and want to build a working tool around a language model, not only call one
- You have documents (course notes, a project's docs, a handbook) you want to ask questions of, and want answers you can check
- You have seen a chat assistant answer confidently and wrongly, and want to know how much retrieval fixes
What you'll need
- Basic Python and a terminal
- No machine learning background, no GPU
Skills you'll practice
- Retrieval-augmented generation
- BM25
- Embeddings
- Hybrid search
- Reranking
- Prompt assembly
- Retrieval evaluation
- Prompt injection testing
How long it takes
About 7.5 hours in total
- Chapter 1 · The model does not know your documents1.5 hours
- Chapter 2 · Getting documents in1.5 hours
- Chapter 3 · Search by meaning, and when it misses1.5 hours
- Chapter 4 · Answers you can check1.5 hours
- Chapter 5 · Evaluate, decide, ship1.5 hours
What's included
- 20 lessons
- 86 readings
- 20 worked examples
- 40 exercises
- 40 quick checks
- 20 common myths
- 20 quizzes
- 5 end-of-chapter checks
Curriculum
5 chapters, 20 lessons
The full course: every chapter and lesson. Tap a chapter to expand it. Lessons open when you start.
Career
Where this course can take you
The roles this course maps to, and the skills you can list on your resume and LinkedIn once you finish it.
Skills you will build
Ready to paste into your resume or LinkedIn profile once you have finished the course.
- Scoping a retrieval-augmented generation problemExplain what retrieval-augmented generation does and does not fix
- Writing a retrieval test setBuild evaluation datasets for search and retrieval
- Building a keyword-search retrieval pipeline in PythonImplement document chunking and BM25 search in Python
- Measuring retrieval with hit rate and MRREvaluate retrieval quality with hit rate at k and mean reciprocal rank
- Inspecting and cleaning extracted document textBuild document loaders for HTML, PDF and plain text
- Choosing chunk size and overlap by measurementTune document chunking parameters against an evaluation set
- Structure-aware chunking with context headersSplit documents by section and add contextual headers to chunks
- Using document metadata for versioning and dated sourcesExtract and use document metadata for filtering in retrieval
- Implementing embedding-based searchImplement semantic search with sentence embeddings and cosine similarity
- Comparing retrieval methods on a test setCompare keyword and dense retrieval question by question on an evaluation set
- Hybrid search with reciprocal rank fusionCombine keyword and dense retrieval with reciprocal rank fusion
- Reranking with a cross-encoder and measuring latencyAdd a cross-encoder reranking stage to a retrieval pipeline
- Prompting for cited answers and a fixed not-found replyDesign prompts that require source citations for each statement
- Choosing context size from measured answersTune the number of retrieved passages against answer quality
- Testing a retrieval pipeline for indirect prompt injectionTreat retrieved content as untrusted input
- Checking generated answers against their sourcesEvaluate answer faithfulness claim by claim against retrieved context
- Diagnosing failures in a retrieval-augmented generation pipelinePerform error analysis on RAG system outputs
- Measuring the cost and latency of a model-backed pipelineMeasure token usage, cost per request and latency by stage
- Choosing between retrieval, long context, structured queries and promptingAssess whether retrieval-augmented generation fits a use case
- Reporting evaluation results and limitsDocument system performance, failure modes, cost and limitations
Questions
Before you commit
Start today