skills · Retrieval quality engineering
Advanced Retrieval: Hybrid Search and Reranking
Hybrid search and reranking, measured on real labels.
Al terminar, podrás:
- A labeled query set from your own users
- A scorecard with nDCG, recall, latency and p-values per change
- Tuned BM25 and a chosen embedding model
- A fusion weight tuned on train and checked on held-out queries
- A reranker and depth chosen under a latency budget
- LLM labels checked against yours
- A regression gate and a one-page retrieval spec
Desde $9/mes8 horas5 capítulos20 lecciones
Por ahora, los cursos están en inglés.
“Our reranker made results worse. The per-query numbers showed why.”
Sobre este curso
Para quién es
- You have search or RAG retrieval working and need to make it measurably better
- You are deciding between BM25, embeddings, hybrid search and a reranker, and want evidence for your data
- You need a regression check before retrieval changes reach users
Lo que necesitas
- You write Python and use a virtual environment
- You have built or run a retriever (course 055 covers the basics)
Habilidades que practicarás
- Retrieval evaluation
- Significance testing
- BM25 tuning
- Embedding models
- HNSW
- Hybrid search
- Cross-encoder reranking
- HyDE
Cuánto tiempo lleva
Unas 8 horas en total
- Capítulo 1 · Measure before you tune1,5 horas
- Capítulo 2 · Tune each retriever1,5 horas
- Capítulo 3 · Fusion, measured1,5 horas
- Capítulo 4 · Reranking1,5 horas
- Capítulo 5 · Queries, labels, and keeping it good2 horas
Qué incluye
- 20 lecciones
- 87 lecturas
- 20 ejemplos resueltos
- 40 ejercicios
- 40 comprobaciones rápidas
- 20 mitos comunes
- 20 cuestionarios
- 5 controles de fin de capítulo
Plan de estudios
5 capítulos, 20 lecciones
El curso completo: cada capítulo y lección. Toca un capítulo para expandirlo. Las lecciones se abren al empezar.
Carrera
Adónde puede llevarte este curso
Los roles con los que se relaciona este curso y las habilidades que podrás incluir en tu currículum y en LinkedIn cuando lo termines.
Habilidades que desarrollarás
Listas para pegar en tu currículum o en tu perfil de LinkedIn cuando termines el curso.
- Building a labeled retrieval test setBuild a relevance-labeled evaluation set from real user queries
- Choosing retrieval metrics by pipeline stageEvaluate retrieval with nDCG, recall and MRR
- Significance testing for retrieval changesCompare retrieval systems per query with a paired significance test
- Tuning BM25 on a held-out splitTune BM25 analyzers and parameters for a search corpus
- Selecting an embedding model by measurementEvaluate embedding models on domain-specific labeled queries
- Evaluating approximate nearest neighbor searchMeasure HNSW recall against exact search and its effect on ranking quality
- Per-query loss analysis for retrieversAnalyze retrieval failures per query and categorize them by type
- Tuning weighted score fusion on a held-out splitCombine keyword and vector search scores with normalization and a tuned weight
- Evaluating reciprocal rank fusionImplement reciprocal rank fusion for hybrid search and test its rank constant
- Setting candidate depth from first-stage recallMeasure first-stage recall at several depths to size a reranking stage
- Configuring hybrid search in a search engineConfigure hybrid search in Elasticsearch, OpenSearch or Postgres with pgvector
- Evaluating a cross-encoder reranker against its first stageAdd and evaluate a cross-encoder reranking stage in a retrieval pipeline
- Choosing rerank depth under a latency budgetMeasure p50 and p95 latency of a reranking stage at several candidate depths
- Comparing rerankers on domain labelsCompare open and hosted reranking models on a labeled domain query set
- Testing an LLM as a listwise rerankerEvaluate LLM-based listwise reranking with cost and latency per query
- Evaluating LLM query rewriting for retrievalEvaluate query rewriting and HyDE for search and RAG
- Validating LLM relevance labels against human judgmentsUse LLM judges for search relevance labeling and measure their agreement with human labels
- Regression testing and monitoring for search qualityBuild offline regression checks for retrieval quality with significance testing
- Documenting a retrieval system for handoverDocument search and retrieval architecture decisions with their measurements and trade-offs
Preguntas
Antes de comprometerte
Empieza hoy