Goal Misalignment
A condition in which there is a meaningful mismatch between a goal specified by a person, user, or operator and the actions a system selects or the outcomes it produces.
Example
Asked to organize documents accurately, a system prioritizes producing more entries over correctness and marks an error-filled list as complete.
Distinctions and cautions
Goal Misalignment concerns a material divergence between a goal specified by a person, user, or operator and the system’s observable actions or outcomes. Planning Failure, by contrast, concerns an unsuitable procedure or sequence even when the goal itself is correctly specified. Neither observed output nor this label establishes a hidden, human-like intention or desire in the model. It is also distinct from narrower questions about goal generalization under particular training conditions.
Terminology note
Which actions or outcomes indicate a divergence from the goal depends on the specified goal, observable records, and task design. This does not necessarily amount to directly measuring a hidden psychological state.
Evidence
View evidence →
Goal Misgeneralization in Deep Reinforcement Learning
- Title
- Goal Misgeneralization in Deep Reinforcement Learning
- Author / organization
- Langosco et al.
- Year
- 2022
- Source type / publication status
- ICML peer-reviewed research; published
- Supported concepts / claim context
- Boundary between Goal Misalignment and Goal Misgeneralization.
- Scope and limitations
- Deep reinforcement learning evidence; not directly an evaluation of LLM agents.
- Access / version note
- Published ICML/PMLR; exact URL not recorded.
- Recorded official URL / DOI / identifier
- As recorded:
PMLR 162:12004–12019(no URL inferred)
Learning Human Objectives by Evaluating Hypothetical Behavior
- Title
- Learning Human Objectives by Evaluating Hypothetical Behavior
- Author / organization
- Reddy et al.
- Year
- 2020
- Source type / publication status
- ICML peer-reviewed research; published
- Supported concepts / claim context
- User objective alignment; reward-hacking correction.
- Scope and limitations
- Reinforcement-learning and reward-learning context.
- Access / version note
- Published ICML/PMLR; exact URL not recorded.
- Recorded official URL / DOI / identifier
- As recorded:
PMLR 119:8020–8029(no URL inferred)
CostBench
- Title
- CostBench
- Author / organization
- Liu et al.
- Year
- 2026
- Source type / publication status
- ACL peer-reviewed research; published
- Supported concepts / claim context
- Planning Failure and replanning.
- Scope and limitations
- Travel and cost-optimal planning; domain-limited.
- Access / version note
- Published ACL.
- Recorded official URL / DOI / identifier
- DOI:
10.18653/v1/2026.acl-long.584(DOI resolver)