Model providers have shipped an explicit fix for the trust-boundary problem: an instruction hierarchy, where system-level messages are trained to outrank developer messages, which in turn outrank user messages, so that a lower-privilege message attempting to override a higher one should be refused. This is a genuine improvement over having no hierarchy at all, and it measurably raises the bar an attacker has to clear. The mistake a team makes is treating "the model has an instruction hierarchy" as equivalent to "the model enforces an instruction hierarchy the way an operating system enforces kernel privilege" — the first is true, the second is not, and the gap between them is where a specific, well-documented class of attacks lives.
The hierarchy is learned behavior, trained through examples that teach the model to weight system-level instructions more heavily than user-level ones when they conflict. Learned behavior generalizes, and generalization has edges — a conflict phrased in a way training did not anticipate can land outside the region where the learned preference reliably holds. This is not a flaw unique to any one vendor's implementation; it is a structural consequence of enforcing priority through training rather than through an architectural mechanism outside the model's own weights.
Understanding exactly where and why the hierarchy leaks is what lets a team calibrate how much weight to put on it — using it as one real layer of defense, never as the layer that makes action-level guardrails optional.
Analogy🏏Cricket
🏏 Think of it like cricket: The chain of command on a cricket field has a formal hierarchy — the captain's word outranks a senior player's, a senior player's outranks a debutant's, and everyone is trained from junior cricket onward to respect that order under pressure. But the hierarchy is enforced by training and culture, not by anything that physically prevents a lower-ranked player from acting on his own judgment in a split second. When a young fielder, throws to the wrong end against a shouted instruction because his own read felt more urgent, the formal hierarchy did not stop him — it just made that kind of override less likely, not impossible. Just as a formal chain of command reduces but does not physically prevent a junior player acting outside it under enough pressure, a trained instruction hierarchy reduces but does not architecturally prevent a lower-privilege instruction winning under the right adversarial pressure. Just as a well-drilled team studies exactly the situations where young players are most likely to freelance and builds specific communication protocols for those moments, a well-built guardrail system studies exactly the situations where the instruction hierarchy is most likely to leak and adds specific checks for those moments. The insight is that a trained hierarchy shifts probabilities; it does not install a hard rule that cannot be crossed.
🏏 Showing the Cricket analogy — a Cricket version isn’t available for this concept yet.