Skip to main content

skills · Retrieval-augmented generation

RAG Fundamentals

Retrieval on your own documents, measured.

By the end, you'll be able to:

  • A test set written before the system
  • A baseline that each later change is compared with
  • Keyword, embedding, hybrid and reranked search compared on your set
  • Answers that cite their chunks, and a fixed reply when the answer is not found
  • A planted-instruction test
  • A ship note with numbers, failures and cost

From $9/month7.5 hours5 chapters20 lessons

“The model does not know your documents.”

About this course

Who it's for

  • You write basic Python and want to build a working tool around a language model, not only call one
  • You have documents (course notes, a project's docs, a handbook) you want to ask questions of, and want answers you can check
  • You have seen a chat assistant answer confidently and wrongly, and want to know how much retrieval fixes

What you'll need

  • Basic Python and a terminal
  • No machine learning background, no GPU

Skills you'll practice

  • Retrieval-augmented generation
  • BM25
  • Embeddings
  • Hybrid search
  • Reranking
  • Prompt assembly
  • Retrieval evaluation
  • Prompt injection testing

How long it takes

About 7.5 hours in total

  1. Chapter 1 · The model does not know your documents1.5 hours
  2. Chapter 2 · Getting documents in1.5 hours
  3. Chapter 3 · Search by meaning, and when it misses1.5 hours
  4. Chapter 4 · Answers you can check1.5 hours
  5. Chapter 5 · Evaluate, decide, ship1.5 hours

What's included

  • 20 lessons
  • 86 readings
  • 20 worked examples
  • 40 exercises
  • 40 quick checks
  • 20 common myths
  • 20 quizzes
  • 5 end-of-chapter checks

Curriculum

5 chapters, 20 lessons

The full course: every chapter and lesson. Tap a chapter to expand it. Lessons open when you start.

    • Read What the Loader Produced
    • Chunk Size Is a Measured Choice
    • A Chunk Forgets Where It Came From
    • Two Versions of the Truth
    • Text to Vectors
    • Where Meaning-Based Search Fails
    • Hybrid Search With Rank Fusion
    • Rerank the Top Twenty
    • Make It Cite, and Let It Say Not Found
    • More Context Is Not Better Context
    • Retrieved Text Is Untrusted Input
    • Check the Answer Against Its Sources
    • Find Which Stage Failed
    • What One Answer Costs
    • When RAG Is the Wrong Tool
    • The Ship Note

Career

Where this course can take you

The roles this course maps to, and the skills you can list on your resume and LinkedIn once you finish it.

Skills you will build

Ready to paste into your resume or LinkedIn profile once you have finished the course.

  • Scoping a retrieval-augmented generation problemExplain what retrieval-augmented generation does and does not fix
  • Writing a retrieval test setBuild evaluation datasets for search and retrieval
  • Building a keyword-search retrieval pipeline in PythonImplement document chunking and BM25 search in Python
  • Measuring retrieval with hit rate and MRREvaluate retrieval quality with hit rate at k and mean reciprocal rank
  • Inspecting and cleaning extracted document textBuild document loaders for HTML, PDF and plain text
  • Choosing chunk size and overlap by measurementTune document chunking parameters against an evaluation set
  • Structure-aware chunking with context headersSplit documents by section and add contextual headers to chunks
  • Using document metadata for versioning and dated sourcesExtract and use document metadata for filtering in retrieval
  • Implementing embedding-based searchImplement semantic search with sentence embeddings and cosine similarity
  • Comparing retrieval methods on a test setCompare keyword and dense retrieval question by question on an evaluation set
  • Hybrid search with reciprocal rank fusionCombine keyword and dense retrieval with reciprocal rank fusion
  • Reranking with a cross-encoder and measuring latencyAdd a cross-encoder reranking stage to a retrieval pipeline
  • Prompting for cited answers and a fixed not-found replyDesign prompts that require source citations for each statement
  • Choosing context size from measured answersTune the number of retrieved passages against answer quality
  • Testing a retrieval pipeline for indirect prompt injectionTreat retrieved content as untrusted input
  • Checking generated answers against their sourcesEvaluate answer faithfulness claim by claim against retrieved context
  • Diagnosing failures in a retrieval-augmented generation pipelinePerform error analysis on RAG system outputs
  • Measuring the cost and latency of a model-backed pipelineMeasure token usage, cost per request and latency by stage
  • Choosing between retrieval, long context, structured queries and promptingAssess whether retrieval-augmented generation fits a use case
  • Reporting evaluation results and limitsDocument system performance, failure modes, cost and limitations

Questions

Before you commit

You are studying or starting out in computing or engineering, you write basic Python, and you want to build a question-answering tool over your own documents that you can measure and check.

You do not need a GPU. The models used for search and reranking run on a laptop CPU. An API key for a hosted model is optional, because each pipeline step can print the assembled prompt for you to paste into an assistant you already use.

It reduces one cause, the model not having your text. It does not make answers true, so in the course you ask for a source after each sentence and check the answers against those sources.

No. The code uses open libraries for search and one hosted model for answers. Each pipeline step also has a path that prints the prompt, so you can paste it into a chat assistant you already use.

Your own documents make the course worth more, and each lesson has one required step on them. If you have none you may use, take the invented sample set, which has a path through each lesson.

Start today