Prompt Sensitivity
The property of a model's output or task performance changing substantially in response to small surface-level differences—such as wording, punctuation, format, or order—between prompts with nearly identical meaning.
Example
Rephrasing “Explain the reason” as “Why does this happen?” changes a correct conclusion into an incorrect one, despite the intended question remaining the same.
Distinctions and cautions
Prompt Sensitivity concerns deliberate changes to the wording or presentation of inputs, not output differences observed when the identical input is rerun (Nondeterminism). It is related to but not synonymous with robustness, and specifically focuses on local sensitivity to prompt expression.
Evidence
View evidence →
When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs
- Title
- When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs
- Author / organization
- Seleznyov et al.
- Year
- 2025
- Source type / publication status
- Findings EMNLP; peer reviewed
- Supported concepts / claim context
- Prompt Sensitivity; Prompt perturbation sensitivity; robustness methods.
- Scope and limitations
- Eight Llama/Qwen/Gemma models, 52 Natural Instructions tasks, GPT-4.1/DeepSeek V3; findings depend on the perturbation and task distributions.
- Access / version note
- Published Findings EMNLP.
- Recorded official URL / DOI / identifier
- Official URL / recorded link