ROADMAP / BLOCK 4 / CH. 10

Evaluation & Testing for AI Systems

INTERVIEW QUESTIONS THIS CHAPTER ANSWERS

Q1

How do you evaluate an LLM pipeline?

Q2

What is LLM-as-judge and what are its limitations?

Q3

What is RAGAS and what does it measure?

Q4

How do you regression test a prompt change?

Q5

How do you run A/B tests on LLM outputs?

← CH. 09 LLM Observability & MonitoringCH. 11 Shipping It →