Senior Engineer, QE at AlphaSense

AlphaSense

SeniorTogether, AlphaSenseai

The acquisition of Tegus by AlphaSense in 2024 advances our shared mission to empower professionals to make smarter decisions through AI-driven market intelligence. Together, AlphaSense and Tegus will accelerate growth, innovation, and content expansion, with complementary product and content capabilities that enable users to unearth even more comprehensive insights from thousands of content sets. Our platform is trusted by over 6,000 enterprise customers, including a majority of the S&P 500. Fo

Apply on employer site

Skills in this role

AI fundamentals

Courses relevant to this posting

Relevance is based on skills and concepts — not a guarantee that you qualify.

Designing AI Evaluation Systems for Production Decisions
Covered skills:AI fundamentals
Skills to close:None listed

Build the skills for this job

Coming-soon courses matched to this posting — outline, waitlist, and skill path.

Course DNA for this role

Designing AI Evaluation Systems for Production Decisions

7 chapters · 29 lessons

1. Evaluation Design Principles
4 lessons
Understand what makes evaluations useful and how to align benchmarks with real-world tasks.
2. Building Custom Test Sets
4 lessons
Create representative, unbiased test data that reflects production conditions.
3. Metrics for Multi-Dimensional Performance
5 lessons
Choose and combine metrics that reveal tradeoffs between accuracy, speed, cost, and safety.
4. Evaluating Retrieval and RAG Systems
4 lessons
Test retrieval quality, grounding, and answer faithfulness in knowledge-augmented architectures.
5. Error Analysis and Failure Pattern Detection
4 lessons
Move beyond aggregate scores to understand where and why models fail.
6. Building Evaluation Pipelines
4 lessons
Automate evaluation workflows for continuous testing and monitoring.
7. Communicating Evaluation Results for Decisions
4 lessons
Present findings that support vendor selection, deployment, and risk management.

Want the full course when it launches? Join the waitlist and we will notify you.

Similar jobs

Other verified postings in the same role family or with overlapping skills.

Staff Software Engineer, Resiliency (Federal) | Okta
Okta
If reasonable accommodation is needed to complete any part of the job application, interview process, or onboarding please use this Form to…
Staff
- AI fundamentals
- Machine learning
- Closed-Loop AI Systems: Validation, Feedback, and Workflow Design
Greenhouse
Multi Skilled Engineer in Derby, England, United Kingdom | Supply Chain Management & Operations at HelloFresh
HelloFresh
At HelloFresh, is simple: to change the way people eat, forever. As a global digital-native FMCG company and the world's leading mealkit pr…
HelloFresh, Green, GB
- Closed-Loop AI Systems: Validation, Feedback, and Workflow Design
Greenhouse
Staff Software Engineer, Dev Environments - Greenhouse
Deliveroo
Apply for this job indicates a required field Accepted file types: pdf, doc, docx, txt, rtf Accepted file types: pdf, doc, docx, txt, rtf U…
RemoteStaff
- AI fundamentals
- Closed-Loop AI Systems: Validation, Feedback, and Workflow Design
Greenhouse
Full Stack Software Engineering - Team Lead - Lever
Mitek Systems
Mitek Systems logo Full Stack Software Engineering - Team Lead Mitek (NASDAQ: MITK) is a global leader in digital & biometric identity auth…
San Diego, California
- AI fundamentals
- Machine learning
- Cloud
- Closed-Loop AI Systems: Validation, Feedback, and Workflow Design
Lever

Skills in this role

Courses relevant to this posting

Build the skills for this job

Designing AI Evaluation Systems for Production Decisions

1. Evaluation Design Principles

2. Building Custom Test Sets

3. Metrics for Multi-Dimensional Performance

4. Evaluating Retrieval and RAG Systems

5. Error Analysis and Failure Pattern Detection

6. Building Evaluation Pipelines

7. Communicating Evaluation Results for Decisions

Similar jobs

Staff Software Engineer, Resiliency (Federal) | Okta

Multi Skilled Engineer in Derby, England, United Kingdom | Supply Chain Management & Operations at HelloFresh

Staff Software Engineer, Dev Environments - Greenhouse

Full Stack Software Engineering - Team Lead - Lever