From boss is always right to the model is always right
AI models may reinforce what people are most likely to accept, strengthening existing beliefs and collective overconfidence.
By Vishnu Rajkumar · Marginalia
Originally published on linkedin

note · LinkedIn
Resharing something I wrote in 2016 about the concept of “boss is always right.” Back then, the issue was authority. However, the environment has changed; it’s now AI models. AI models and systems are designed to predict what you’re most likely to accept. They reinforce patterns and smooth things into coherence. Over time, this tends to strengthen existing belief structures rather than challenge them. Humans naturally lean towards agreement and social harmony, and systems trained on past patterns only accelerate this tendency.
Therefore, disagreement becomes more valuable, not less. The ability to question the prevailing view, to introduce friction into a room that has reached comfortable agreement, is a distinctly human strength. Systems optimize for probability, and organizations need people willing to test the edges of that probability.
For leaders, this creates a subtle responsibility. Dissent and deterrence often appear similar on the surface. Both slow things down, question direction, and introduce friction into what seems like momentum. However, they are not the same. Deterrence protects comfort, while dissent protects truth; or at least the chance that the majority might be wrong.
Organizations that learn to recognize and value this difference gain something rare; disagreement as an asset rather than a disruption.
“Boss is always right” produced compliance. “The model is always right” risks producing collective overconfidence. [Link in comments]