Robustness
The property of maintaining required performance or behavior when inputs, operating conditions, data distributions, or prompt wording change in reasonable ways.
Example
Rephrasing a question without changing its meaning does not substantially reduce answer quality, indicating robustness to that kind of prompt variation.
Distinctions and cautions
Robustness does not simply mean giving an identical response every time. Repeating the same incorrect answer may be consistent without being robust. Robustness to one kind of change does not establish robustness to other data shifts, inputs, or conditions.
Evidence
View evidence →
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
- Title
- Artificial Intelligence Risk Management Framework (AI RMF 1.0)
- Author / organization
- NIST
- Year
- 2023
- Source type / publication status
- Official technical publication; published
- Supported concepts / claim context
- Reliability, Accuracy, Robustness; trustworthiness characteristics.
- Scope and limitations
- An AI-wide risk framework, not an LLM-specific taxonomy. A revision was underway as of 2026.
- Access / version note
- Treat the published edition as the historical/technical baseline.
- Recorded official URL / DOI / identifier
- DOI:
10.6028/NIST.AI.100-1(DOI resolver)
WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Title
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Author / organization
- Koh et al.
- Year
- 2021
- Source type / publication status
- ICML peer-reviewed research; published
- Supported concepts / claim context
- Distribution Shift, Robustness.
- Scope and limitations
- A general machine-learning benchmark, not limited to LLMs.
- Access / version note
- Published conference proceedings.
- Recorded official URL / DOI / identifier
- Official URL / recorded link
Evaluating Model Robustness and Stability to Dataset Shift
- Title
- Evaluating Model Robustness and Stability to Dataset Shift
- Author / organization
- Subbaswamy et al.
- Year
- 2021
- Source type / publication status
- AISTATS peer-reviewed research; published
- Supported concepts / claim context
- Robustness, Distribution Shift.
- Scope and limitations
- Primarily predictive-model settings.
- Access / version note
- Published conference proceedings.
- Recorded official URL / DOI / identifier
- Official URL / recorded link
Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs
- Title
- Flip-Flop Consistency: Unsupervised Training for Robustness to Prompt Perturbations in LLMs
- Author / organization
- Hejabi et al.
- Year
- 2026
- Source type / publication status
- ACL peer-reviewed research; published
- Supported concepts / claim context
- Response Consistency, Robustness, Prompt Sensitivity boundary.
- Scope and limitations
- Uses 11 datasets and a particular prompt-variation design. Consistency does not imply truth.
- Access / version note
- Published ACL.
- Recorded official URL / DOI / identifier
- DOI:
10.18653/v1/2026.acl-long.71(DOI resolver)