Saltar al contenido principal

Curso completo próximamente

Building LLM Evaluation Systems for Production AI

7 capítulos · 32 lecciones

  1. 1. Defining Evaluation Scope and Use Cases

    4 lecciones

    Map product requirements to testable behaviors and identify what good looks like for your specific LLM application.

  2. 2. Building Representative Test Sets

    4 lecciones

    Create test datasets that capture real-world diversity, edge cases, and adversarial inputs for your domain.

  3. 3. Writing Scoring Rubrics and Metrics

    5 lecciones

    Define explicit, measurable criteria for correctness, safety, faithfulness, and task-specific quality.

  4. 4. Implementing LLM-as-Judge Pipelines

    5 lecciones

    Build scalable automated evaluation systems using LLMs as judges, with calibration and quality controls.

  5. 5. Evaluating RAG Systems End to End

    5 lecciones

    Separate retrieval quality from generation quality and measure faithfulness, citation, and refusal behavior.

  6. 6. Running Error Analysis and Regression Testing

    4 lecciones

    Turn evaluation results into insights and build safety nets that catch regressions when systems change.

  7. 7. Evaluation in Deployment and Production

    5 lecciones

    Extend evaluation to live environments, monitor drift, and adapt to device, latency, and user context constraints.

Recibe un correo cuando abra este curso