Teams got smaller. Budgets got tighter. And the person who used to lean into your doorway and say “this doesn’t feel right” stopped showing up. (Maybe they got laid off, maybe they retired. Sometimes, it’s a bit of both!)
I’ve watched this play out with managers who lost their teams through RIFs and reorganizations, then started leaning on large language models to fill the space. On the surface, it can work—for a little while.
But those team members provided more than output. They provided a feedback loop — someone with enough context to push back, to say the uncomfortable thing before it became an expensive thing.
So now, many of those managers hope to get the same kind of guidance from LLM chatbots.
This week, in a series of posts, I’m exploring why that’s not a great idea, and what the best managers I know are really doing to leverage AI in their businesses.
The bias built into the foundation
One of the deep biases in a large language model is that its output, by definition, is trying to please whoever’s asking. The model predicts the response most likely to satisfy you. Agreement is baked into the architecture the way a load-bearing element is baked into a floor plan. You can stick a big plant in front of that pillar. You can dress it up with artwork, or put velvet ropes around it. You can’t remove it.
Compare that to a good team member. Early in my own career, I didn’t understand how to place the right value on a solid “red team” contributor. It turns out that someone who disagrees with you carries more value than one who always agrees, because disagreement is where quality control actually lives.
An LLM can’t give you that kind of productive friction. It will refine your bad idea beautifully. It will never lean into your doorway to say, “hey, I think I need to tell you something.”
The technology does exactly what it was designed to do. The trouble starts when we hand it a role it was never designed to fill.
A consultant learns. A model just keeps sounding confident.
There’s a trap here that catches even careful leaders, and it comes from how human these tools feel in conversation. When you externalize your decision-making to an AI, you lose something a human advisor carries by default: accountability that compounds over time.
A consultant who gives you bad advice can be corrected. They feel the weight of the miss and carry the lesson into the next engagement. I’ve built a career on exactly that loop — understanding where I got it wrong, and getting sharper because of it. It stings every time. It works anyway. (After three decades of this business, I’m happy to say I don’t get things wrong quite as often.)
On the other hand, an AI that lacks a lived context has no loop to close.
And the reliability problem grows as your work gets more specific. An LLM gives you its best guess based on the thousands of subject matter experts it was trained on. For broad questions, that material runs deep. The more niche your work becomes, the thinner it gets. If you’re asking without valid data of your own to ground the answer, you’re getting a guess — a well-dressed, thoroughly convincing guess.
Notice, too, that the model sounds exactly as sure about your niche operational question as it does about state capitals. Your confidence should change with the question. The model’s never does.
Why this weighs heavier for mission-driven organizations
In a commercial setting, a decision made on unchallenged AI advice costs money. In a mission-driven setting, the cost lands on a person — a family that couldn’t find an intake form, or a caregiver who gave up seeking help after getting only three screens into a confusing process.
Mission-driven organizations already run lean. Replacing a departed staff role with an AI subscription makes budget sense on paper. What rarely appeared on paper was that staff member’s most valuable contribution: the moment in a meeting when they said, “I’ve talked to the people we serve, and this plan doesn’t match what they need.”
No model can say that with authority. It has read about communities like yours. That’s a different kind of knowledge, the way reading a blueprint differs from standing inside the building.
Friction is a feature of a healthy organization
To be clear: I use these tools daily, for many of the very specific things they’re actually good at. They’ve made real, mission-central work affordable for organizations that couldn’t touch it a few years ago. The value in analysis, in clearing busywork, in freeing up attention for judgment calls is real and worth building around.
The risk lives in one specific substitution: letting the model occupy the seat where your honest disagreement used to live.
Remove productive friction and everything feels lighter for a while. Then the settling starts, slow and quiet, until “we should revisit this” has become “we need to undo this” — and undoing costs far more than challenging would have.
A practical test: think back over your last month of AI-assisted decisions and count how many times the tool told you your premise was flawed. If the answer is zero, what you’ve been getting is agreement with extra steps.
Protect the people who push back
Before you adopt another tool, take inventory of your feedback loops. Write down the names of the people in your organization who have both the context and the standing to tell you when something is wrong. Then look honestly at whether recent cuts, restructures, or automation decisions have thinned that list.
If it’s thinner than it was a year ago, rebuild it before you automate anything else. Give your remaining people explicit permission — and protected time — to challenge what your AI tools produce. That disagreement is load-bearing. Treat it accordingly.
And if you want an outside feedback loop for your digital experience — one grounded in what your audience actually does rather than what a model guesses they do — start with a Website Reality Check. For $27, you get an honest, human read on where your website serves your community and where it quietly fails them.
The machine will keep telling you you’re doing great. Make sure someone in the room is paid to tell you otherwise.
