AI coding is a trust problem

In an earlier article, I wrote about the habit that changed my AI workflow: I stopped accepting "I've fixed the bug" as evidence. Tests and verification had to settle that question.

Since then, I've kept seeing the same gap outside the coding tool. Teams and leaders also confuse confidence with evidence, especially when the feedback loops around them are weak.

The confidence gap

AI coding tools tend to deliver every answer in the same confident tone. "I've fixed the bug." "Here's the solution." "This should work now."

Sometimes the bug has only moved. Sometimes the solution breaks another path. The wording tells you very little about whether the code works.

I've seen a similar gap in junior developers. Someone says a task is done because the code compiles, without checking the behavior or running the wider test suite. They explain a solution confidently because they haven't yet found the part they don't understand.

I don't see that as a character flaw. Experience teaches people to separate "I think this works" from "I've verified this works." Being burned by a change, reviewing failures, and seeing consequences all sharpen that judgment.

An AI tool can use feedback in the current task, but it doesn't get woken at 3am because its fix took down production. I have to judge its work by the checks I can run, however reassuring the response sounds.

Trust needs a feedback loop

For AI-generated code, I use automatic tests, explicit verification, and review that looks for regressions. Those checks let the result earn trust.

The same idea applies to a team. Code review checks one person's judgment against another's. Retrospectives compare the team's sense of how work is going with what actually happened. One-on-ones give a manager a chance to compare their assumptions with an individual's experience.

None of those rituals guarantees a good outcome. They can become theater: rubber-stamp reviews, retrospectives where nobody says anything real, and one-on-ones that are status reports. The meeting exists, but the feedback loop doesn't.

Without useful feedback, a team can keep saying the project is on track long after it has drifted. Everyone may believe it, especially if nobody has a reason to look beyond the status report.

Assumed trust feels fine until pressure arrives

I once wrote about assumed trust versus tested trust in organizations. "We have a great culture" can mean "we haven't been stressed yet." The trust feels real, but it hasn't been tested.

"The AI said it fixed the bug" has the same shape. The confident response feels like progress, but you don't know until something checks the result.

The organizational version can take months to show up. Leadership says the strategy is clear, but teams interpret it differently and move in different directions. By the time someone calls for a realignment, they've already done a lot of work on conflicting assumptions.

The useful question is how leadership knows the strategy is understood. Ask teams to explain the choices they're making because of it. Look at what gets prioritized when two goals compete. Those signals reveal more than another presentation declaring alignment.

Working with AI has made me more likely to ask how something was checked. I want that to be an ordinary part of saying a task is done, or a strategy is understood. Honest, capable people can be confidently mistaken, and I'd like us to find out while there's still time to do something about it.