Calibration and Uncertainty

A model is calibrated when its confidence matches its accuracy: of the predictions it calls 80% likely, about 80% should be right. Modern neural nets and LLMs are usually overconfident, so a raw probability or a self-reported 'I'm 95% sure' is not trustworthy on its own. You fix it with temperature scaling or isotonic regression, get distribution-free coverage with conformal prediction, and then use the calibrated confidence to abstain, route, or escalate to a human.