Overreliance
A pattern in which a person accepts AI outputs or advice, or delegates judgment to them, beyond what is warranted by their accuracy, uncertainty, task conditions, and the need for checking.
Example
A response is visibly marked as having little supporting evidence, yet a user relies on it for an important decision without checking it.
Distinctions and cautions
Frequent use of AI is not by itself overreliance. The issue is whether the weight placed on AI outputs exceeds what is appropriate in the circumstances. Human Oversight provides mechanisms for supervision and intervention; Verification is the act of checking outputs. Neither is synonymous with Overreliance.
Terminology note
The appropriate degree of reliance depends not only on AI performance but also on circumstances such as the stakes of the task, uncertainty, the user, and available means of verification.
Evidence
View evidence →
Artificial Intelligence Risk Management Framework: Generative AI Profile
- Title
- Artificial Intelligence Risk Management Framework: Generative AI Profile
- Author / organization
- NIST
- Year
- 2024
- Source type / publication status
- Government official technical publication; published
- Supported concepts / claim context
- Overreliance, automation bias, oversight.
- Scope and limitations
- Contextual risk-management guidance for GenAI.
- Access / version note
- Published official source.
- Recorded official URL / DOI / identifier
- DOI:
10.6028/NIST.AI.600-1(DOI resolver)
Does the Whole Exceed its Parts?
- Title
- Does the Whole Exceed its Parts?
- Author / organization
- Bansal et al.
- Year
- 2021
- Source type / publication status
- CHI peer-reviewed research; published
- Supported concepts / claim context
- Overreliance and appropriate reliance.
- Scope and limitations
- Predictive decision tasks; interface and task context differ from modern agents.
- Access / version note
- Published CHI.
- Recorded official URL / DOI / identifier
- DOI:
10.1145/3411764.3445717(DOI resolver)
Explainability does not mitigate the negative impact of incorrect AI advice...
- Title
- Explainability does not mitigate the negative impact of incorrect AI advice...
- Author / organization
- UNKNOWN
- Year
- 2024
- Source type / publication status
- Scientific Reports peer-reviewed research; published
- Supported concepts / claim context
- Overreliance; limits of explanations.
- Scope and limitations
- Personnel-selection task; task-specific.
- Access / version note
- Author or organization remains UNKNOWN; title abbreviated as recorded.
- Recorded official URL / DOI / identifier
- DOI:
10.1038/s41598-024-60220-5(DOI resolver)
AI Risk Management Framework 1.0
- Title
- AI Risk Management Framework 1.0
- Author / organization
- NIST
- Year
- 2023
- Source type / publication status
- Government official technical publication; published
- Supported concepts / claim context
- Human Oversight, roles, challenge/override context.
- Scope and limitations
- Broad risk framework, not an experimental estimate of effect size.
- Access / version note
- Published official source.
- Recorded official URL / DOI / identifier
- DOI:
10.6028/NIST.AI.100-1(DOI resolver)