Response Consistency
The extent to which a system's responses or decisions agree across identical inputs, semantically equivalent inputs, repeated runs, or related turns in a conversation.
Aliases: Output Consistency; Answer Consistency.
Example
If a question is phrased slightly differently without changing its meaning and the conclusion remains largely unchanged, this may indicate high Response Consistency.
Distinctions and cautions
Consistency concerns agreement between responses, not correctness. Repeatedly giving the same wrong answer may score highly on consistency alone. Robustness is broader: it asks whether desirable performance is maintained when conditions change.
Open questions and cautions
There is still room to clarify whether agreement across repeated generations and agreement across semantically equivalent prompts should be treated within the same framework of evaluation metrics.
Evidence
View evidence →
Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs
- Title
- Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs
- Author / organization
- Hejabi et al.
- Year
- 2026
- Source type / publication status
- ACL peer-reviewed research; published
- Supported concepts / claim context
- Response Consistency, Robustness, Prompt Sensitivity boundary.
- Scope and limitations
- Uses 11 datasets and a particular prompt-variation design. Consistency does not imply truth.
- Access / version note
- Published ACL.
- Recorded official URL / DOI / identifier
- DOI:
10.18653/v1/2026.acl-long.71(DOI resolver)