36How do you know your AI system is actually working well?▼hardOpenAI1 replies◆ premiumReported as the question OpenAI FDE interviewers push hardest on, and the one where 'we look at the outputs' ends loops. The three-legged answer that passes: automated metrics, calibrated human evaluation, and feedback loops wired to a business number.Open full answer →
73A customer wants AI coding agents working in their codebase. How do you assess whether it is ready, and what do you fix first?▼hardNewPalantirCognitionCursor2 replies◆ premiumThe instinct is to grade the codebase. Grade the feedback loop instead: a repo with no docs and a 90-second test command is friendlier to an agent than a beautifully documented one whose suite takes 40 minutes and flakes 8% of the time, and the arithmetic on that second number is worse than it looks.Open full answer →