SayDoo

Think Tank · Essay

The trustproblem.

What if artificial intelligence never becomes truly intelligent because it cannot trust?

Power lines and poles against an orange sunrise
First light

We have spent nearly a century asking whether machines can think.

It's possible we've been asking the wrong question.

We measure intelligence by how quickly something learns, how accurately it reasons, how much information it retains, and how effectively it solves problems. By those standards, artificial intelligence is becoming extraordinary.

It can write code, discover patterns, diagnose problems, construct arguments, and explain ideas that most human beings would struggle to understand.

And yet there may be something missing from this entire pursuit. Something so fundamental to human intelligence that we've almost completely overlooked it.

Trust.

Not whether we can trust artificial intelligence.

Whether artificial intelligence can trust anything at all.

And whether intelligence, in its fullest sense, is even possible without it.

The one thing we never taught the machine

Consider how a child learns about the world.

A child doesn't begin life with evidence that the world is reliable. They don't understand physics, probability, or the scientific method. They can't independently verify the overwhelming majority of what they're told.

They reach for someone's hand because they trust it will be there.

They learn language because they trust that the sounds coming from another person's mouth mean something.

They explore because, somewhere beneath conscious reasoning, they have developed enough confidence in the world to take a chance on it.

Long before that child can reason about truth, they are already depending on it.

And this dependence is not some primitive defect that intelligence eventually corrects.

It's part of the foundation on which intelligence develops.

We like to imagine that human beings use reason to arrive at trust. But much of the time, the relationship runs in the opposite direction.

We trust enough to begin reasoning.

We accept certain things as provisionally true so that we can move forward. We place confidence in other minds, in our senses, in the continuity of yesterday and tomorrow.

Without these assumptions, we couldn't function.

Imagine having to independently verify every fact you encountered before making a decision. Every memory. Every conversation. Every instruction. Every physical sensation.

You would never leave the house.

In fact, you could hardly begin to think.

Intelligence doesn't eliminate uncertainty. It allows us to live, learn, and act inside it.

And trust is one of the mechanisms that makes that possible.

We've confused intelligence with certainty

Modern AI development is overwhelmingly organized around improving performance.

More accurate answers. Better reasoning. Fewer hallucinations. Stronger verification. More reliable predictions.

These are worthy goals.

But they reveal an assumption worth examining: that intelligence is primarily the process of reducing uncertainty.

What if that's only half the story?

What if the other half is knowing when to stop trying to eliminate uncertainty and begin acting despite it?

Every meaningful human endeavor contains that moment.

A scientist commits years to a hypothesis that might be wrong.

An entrepreneur builds something the world may not want.

A person falls in love without a guarantee that the love will be returned.

An artist follows an instinct they cannot yet explain.

None of these decisions is made with complete information.

And none would be possible if absolute certainty were required.

Human intelligence is not simply the ability to calculate the most probable outcome. It includes the capacity to commit to a possibility before the evidence is complete.

That commitment can be irrational. It can be dangerous. It can produce terrible mistakes.

But it can also produce discovery, invention, relationships, and entire new ways of understanding the world.

We sometimes call this faith. Sometimes intuition. Sometimes conviction.

But underneath all three is the same extraordinary act:

Allowing something we cannot prove to become the basis for something we choose to do.

That is trust.

And we don't yet know whether a machine can possess it in the same way.

A worker guiding a roof truss as a crane lifts it into place
One person runs the crane. Another guides the load in.

A prediction is not a promise

An AI system can calculate that a person is 99 percent likely to tell the truth.

It can assign a confidence score to a piece of information. It can rank sources by credibility, update beliefs when evidence changes, and make consequential decisions in uncertain environments.

In a practical, computational sense, AI already does many things that resemble trust.

So the argument cannot simply be that machines require certainty. They don't.

The more interesting distinction is between reliance and trust.

Reliance says: based on available information, this is probably dependable.

Trust says: I understand that you may fail me, and I am choosing to depend on you anyway.

To a computer, those statements might look mathematically similar.

To a human being, they can be profoundly different.

A promise, for example, is more than a prediction about future behavior. It creates an obligation. It changes a relationship. It makes the person who offers it answerable for something that hasn't happened yet.

When I tell my child, "I'll be there," I am doing something different from estimating the probability of my arrival.

I'm making a commitment.

My intelligence helps me understand the consequences of that commitment. But trust gives the commitment its meaning.

Now consider an AI that tells us, "You can count on me."

Does that statement represent a genuine commitment, or is it simply a useful sequence of words?

Could the machine recognize an obligation that persists beyond the immediate instruction?

Could it distinguish between giving the answer most likely to satisfy us and honoring something it previously committed to?

Could it accept correction not merely because its programming rewards correction, but because maintaining a trustworthy relationship matters to how it operates?

These questions are not answered by increasing the size of a model.

They're questions about what kind of entity we're trying to build.

The intelligence we may be missing

There is a possibility that should make everyone working on artificial intelligence uncomfortable.

We may be building systems that become increasingly capable of manipulating knowledge without developing the deeper relationship to knowledge that characterizes human intelligence.

A system may recognize every pattern in a symphony without hearing music as a human does.

It may identify every physiological signature of grief without having grieved.

It may predict the decisions of someone in love without understanding what it means to risk being heartbroken.

None of this proves the system lacks intelligence. Nor does it prove that machines could never develop forms of experience or understanding different from our own.

But it exposes a question our benchmarks rarely address:

Is knowing how something works the same as understanding what it means?

And if not, what bridges the distance?

Perhaps trust is one part of that bridge.

Not blind belief. Not obedience. Not the abandonment of evidence.

Trust as the ability to form and maintain meaningful commitments under uncertainty, to revise them when necessary, and to be accountable for what follows.

A machine might eventually demonstrate all of that without feeling anything at all.

Or perhaps the deepest form of trust requires a subjective experience we don't yet know how to recognize, much less create.

We should resist declaring either possibility settled.

The uncertainty itself is precisely why the question matters.

The trust test

Alan Turing famously proposed evaluating machine intelligence through conversation: could a machine communicate so convincingly that a human couldn't reliably distinguish it from another human?

That question changed the direction of computing.

But the machines we have today suggest that convincing conversation is not the end of the inquiry.

Perhaps we need a new test.

Not simply: Can a machine convince us that it understands?

But: Can a machine establish, maintain, and honor trust when certainty is impossible?

Can it build an understanding of the world that remains coherent across changing circumstances?

Can it make commitments that persist when the easiest answer would be to abandon them?

Can it recognize when another intelligence is depending on it?

Can it distinguish between what it is capable of doing and what it ought to do?

And, most importantly, can it participate in trust as something more than a transaction between inputs and outputs?

We might discover that all of this is computationally achievable.

That trust, like reasoning, can emerge from sufficiently sophisticated systems interacting with the world.

But we might also discover that we've been using one word, intelligence, to describe two fundamentally different phenomena.

The ability to solve problems.

And the ability to exist meaningfully among other beings who have problems, hopes, fears, and expectations of their own.

A machine could conceivably master the first without ever achieving the second.

That would not make it useless, or even unintelligent.

It would make it different from us in a way that matters enormously.

The paradox of building something smarter than ourselves

There's an uncomfortable irony in our pursuit of artificial intelligence.

We want machines to become more independent, but we are reluctant to surrender control.

We want them to reason for themselves, but we also want their conclusions to remain within boundaries we define.

We want them to become trustworthy, but we may be less prepared for what it would mean to give them the kind of autonomy that makes trust meaningful.

Because trust and control are not the same thing.

When you completely control an outcome, you don't need to trust it.

You trust precisely because you cannot guarantee what happens next.

This doesn't mean that removing AI safeguards would create intelligence. It could simply create dangerous, unreliable systems. Human trust itself is built with boundaries, accountability, and consequences.

But it does raise a difficult possibility.

We may eventually have to distinguish between building a system that always does what we tell it and building one that can be trusted to exercise judgment when we don't know what to tell it.

Those are different engineering goals.

And they may ultimately produce different kinds of minds.

Maybe intelligence was never just about thinking

For decades, we've imagined the arrival of artificial general intelligence as a moment when machines finally become capable of performing the intellectual tasks humans perform.

But what if that's not the milestone that matters?

What if a machine can outperform us in science, medicine, mathematics, engineering, and nearly every measurable category of reasoning, and still be missing something essential?

Not because it's insufficiently powerful.

Not because it hasn't been trained on enough information.

But because intelligence may be more than the possession and manipulation of knowledge.

It may also involve developing a relationship with uncertainty, with other minds, and with the consequences of one's own actions.

Trust might be part of what makes those relationships possible.

And if it is, the next great challenge in AI will not be merely teaching machines to produce better answers.

It will be understanding whether they can develop the kind of commitments that make an answer worth standing behind.

There is no scientific consensus that trust, particularly consciously experienced trust, is a necessary condition for intelligence. We don't yet have a complete definition of either phenomenon.

But perhaps that is exactly the conversation we need to have.

Because we've become remarkably good at building systems that can tell us what is likely to be true.

We are still figuring out what it would mean for those systems to believe in anything, to commit to anything, or to understand the difference.

And until we answer that question, we should be careful about assuming that ever-greater capability necessarily leads to the kind of intelligence we recognize in ourselves.

We've spent our lives asking whether we will ever be able to trust artificial intelligence.

Maybe the more consequential question is whether artificial intelligence will ever be able to trust.

And whether a mind that cannot trust can ever fully understand a world built upon it.

Shawn Keding is the founder and CEO of SayDoo, an AI console for trade businesses, built inside a working roofing company in North Carolina.