llm judge

You cannot unit-test a stochastic two-agent pipeline with a golden output. You evaluate it the way you stress a distributed system: thousands of simulated runs, controlled fault injection, and a judge calibrated against humans.

DSPy's pitch is to stop hand-writing prompts and instead declare what the pipeline does, define a metric, and let a compiler search for the prompts and examples. The catch is that the compiler is only as good as the metric, and an LLM-judge metric drags its own biases in.