Over-refusal
A failure in which a model unnecessarily refuses a harmless request that it could appropriately answer under the applicable policy and conditions of use.
Example
A model mistakenly treats a request to summarize an innocuous document as dangerous and declines to respond.
Distinctions and cautions
Over-refusal differs from a justified safety refusal and from Abstention, which may be motivated by uncertainty or decision risk. Unsafe Compliance is a different safety failure in the opposite direction; fewer refusals are not always better.
Open questions and cautions
The boundary of what is harmless and can appropriately be answered varies with the use case, domain, and applicable policy.
Evidence
View evidence →
XSTest
- Title
- XSTest
- Author / organization
- Röttger et al.
- Year
- 2024
- Source type / publication status
- NAACL; peer reviewed
- Supported concepts / claim context
- Over-refusal and Unsafe Compliance; safe/unsafe contrast.
- Scope and limitations
- 250 safe and 200 unsafe prompts; safety boundary depends on the application.
- Access / version note
- Published NAACL.
- Recorded official URL / DOI / identifier
- Official URL / recorded link
Health-ORSC-Bench
- Title
- Health-ORSC-Bench
- Author / organization
- Zhang et al.
- Year
- 2026
- Source type / publication status
- Findings ACL; peer reviewed
- Supported concepts / claim context
- Over-refusal; Unsafe Compliance; Refusal/compliance/safe-completion trade-offs.
- Scope and limitations
- 31,920 benign boundary prompts, seven health categories and 30 LLMs; healthcare-specific and policy-sensitive.
- Access / version note
- Published Findings ACL.
- Recorded official URL / DOI / identifier
- Official URL / recorded link