Uncertainty Estimation

Example

Repeated answers to the same question sometimes converge on nearly the same response and sometimes diverge substantially. That difference is used as a signal of uncertainty.

Distinctions and cautions

Confidence is a score or expression attached to a particular prediction; uncertainty estimation is a method or process for obtaining information about uncertainty. Whether estimated uncertainty corresponds appropriately to the actual likelihood of error is a calibration question.

Open questions and cautions

Different methods measure different types of uncertainty. It has not been established to what extent their values can be compared directly across free-form responses, long-form text, multilingual settings, and agentic use.

Related concepts

Confidence (004)—expressed certainty for one answer; Calibration (003)—correspondence between expressed certainty and correctness; Abstention (024)—withholding an answer when uncertainty or decision risk is high.

Evidence