Factuality
The extent to which generated claims are supported as factual by external reality or reliable reference knowledge.
Example
A response says an event took place in 2024, although a reliable source dates it to 2023.
Distinctions and cautions
Factuality asks whether factual claims agree with external facts. Faithfulness instead asks whether an output adheres to supplied source material. A response can faithfully repeat an erroneous source and still be factually incorrect.
Open questions and cautions
Facts that change over time, ambiguous claims, and statements involving value judgments cannot necessarily be assessed for factuality using the same method.
Evidence
View evidence →
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
- Title
- FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
- Author / organization
- Min et al.
- Year
- 2023
- Source type / publication status
- EMNLP peer-reviewed research; published
- Supported concepts / claim context
- Factuality, Verification, claim-level support.
- Scope and limitations
- Mainly biographies and long-form generation; depends on the quality of reference knowledge sources.
- Access / version note
- Published EMNLP.
- Recorded official URL / DOI / identifier
- DOI:
10.18653/v1/2023.emnlp-main.741(DOI resolver)
On Faithfulness and Factuality in Abstractive Summarization
- Title
- On Faithfulness and Factuality in Abstractive Summarization
- Author / organization
- Maynez et al.
- Year
- 2020
- Source type / publication status
- ACL peer-reviewed research; published
- Supported concepts / claim context
- Faithfulness, Factuality, Hallucination boundary.
- Scope and limitations
- Focused on summarization, but provides foundational evidence for the source-versus-world distinction.
- Access / version note
- Published ACL.
- Recorded official URL / DOI / identifier
- DOI:
10.18653/v1/2020.acl-main.173(DOI resolver)
Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics
- Title
- Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics
- Author / organization
- Godbole & Jia
- Year
- 2025
- Source type / publication status
- Findings ACL peer-reviewed research; published
- Supported concepts / claim context
- Limits of verification; disagreement among factuality metrics.
- Scope and limitations
- Depends on the factuality metrics and datasets studied.
- Access / version note
- Published ACL Findings.
- Recorded official URL / DOI / identifier
- DOI:
10.18653/v1/2025.findings-acl.1175(DOI resolver)