Security & Privacy
Instruction Hierarchy
Category
Security & Privacy
Abstraction level
Pattern
Operation level
InferencePost-training
Use cases
Defending agents against prompt injectionSafely processing external content (RAG, web browsing)Enforcing developer policies over user instructions
How it works
Instructions are passed to the model tagged with a level (system/developer/user/content). During fine-tuning the model learns to recognize conflicts and resolve them in favor of the higher level, refusing to follow conflicting instructions from lower-trust content.
Problem solved
How to keep a model from being manipulated by instructions hidden in lower-trust content (e.g. a web page read by a tool)?
Strengths & limitations
Strengths
✓Reduces susceptibility to prompt injection and jailbreak
✓Clear, principled model of instruction trust
✓Supports safe agent operation on external data
Limitations
✗Does not fully eliminate attacks
✗Requires consistent tagging of instruction levels
✗May sometimes reject legitimate user instructions