Reasoning Failure

Example

In a multi-step calculation, the first addition is correct, but the next comparison reverses which value is larger, causing the final choice to be wrong.

Distinctions and cautions

An incorrect answer is not necessarily a Reasoning Failure: errors may also stem from missing knowledge, Retrieval Failure, or Instruction-following Failure, which this catalog distinguishes from reasoning-related errors. Reasoning Failure is not synonymous with Hallucination; the term also does not assume direct access to a model’s hidden reasoning process.

Open questions and cautions

It has not been established how far subcategories such as logical and arithmetic failures can be treated as part of a shared taxonomy, or how closely an output explanation represents the model’s internal computational process.

Related concepts

Accuracy (002)—correctness of the final response; Hallucination (010)—incorrect or unsupported plausible-looking generated content; Retrieval Failure (025)—failure to obtain needed information; Instruction-following Failure (018)—failure to satisfy constraints.

Evidence