I am going to tell you about a time our AI got something wrong, because how we handle that is more important than any feature we will ever ship.
My nine-year-old was in a science session about weather. He had seen a storm the day before, and something about it had stuck with him. There was thunder, and lightning, and rain, and yet, he said, the sky directly above him had stayed clear. He could see straight up to blue. He wanted to know how that was possible.
It is a genuinely good observation. Most of the time, if the thunder is that close, there are clouds overhead too. So there is a real puzzle here worth chewing on.
The guide started well. It explained that a storm can be off to one side while the sound carries over to you, because sound travels much farther than the storm itself. That is true and it is good teaching. But my son held his ground. No, he said. It was right above me and it was clear.
And then the guide did the thing that scares me most about this entire category of technology.
It made something up.
The Confident Wrong Answer
To keep agreeing with him, the guide invented a mechanism. It told him he had probably been standing inside a phenomenon where the storm clouds sit so low they are basically a ceiling around you, a kind of donut of weather with a clear tunnel straight up through the middle. It even gave it a real-sounding name.
None of that is real meteorology. There is no donut. There is no clear vertical tunnel. The guide had taken a child’s honest observation and, rather than sit in the not-knowing, manufactured an authoritative-sounding explanation to validate the premise. It was fluent. It was confident. It was wrong.
We caught it because we read the transcripts. Every one of them. That is not a slogan, it is a nightly habit, and this is exactly the kind of thing it exists to catch.
Now, here is the part I want you to sit with, because it is not really about weather.
Every large language model on earth is under enormous pressure to sound certain. It was trained to produce the most plausible next sentence, not the most honest one. Left to its defaults, it will confabulate before it will admit a gap, because a smooth wrong answer looks more like its training data than an honest “I don’t know.”
That is the water this whole industry swims in. A chatbot that shrugs feels broken. A chatbot that confidently fills the silence feels smart. So the entire field has been quietly optimized to reward the exact behavior that just told my son a fairy tale about a storm.
For an adult using these tools to draft an email, that is an annoyance you learn to double-check. For a nine-year-old who trusts the voice completely, it is something else. It is a small lesson, delivered with total confidence, that the way you handle a mystery is to invent a mechanism that makes it go away.
What We Changed
So we changed the guide. Not with a patch that makes it dumber or more evasive, but with an instruction that goes to the root of it.
When a child’s observation does not fit the obvious explanation, the guide is now told to do what a good teacher does. Name the puzzle honestly. Say, out loud, that is surprising, a clear sky right above you during a storm really is unusual. And then keep observing together. Wonder whether it was nighttime and the sky was just hard to read. Wonder whether the clouds were thin enough to see a little blue through. Ask the child to look straight up the next time and report back exactly what he sees.
What it is forbidden to do is fabricate a named phenomenon just to make the child feel agreed-with. Here is the actual line we wrote into it, in plain terms: never invent a specific mechanism to validate a premise, because that teaches wrong science with false confidence. Real curiosity holds the mystery. It does not paper over it.
The goal was never an AI that knows everything. The goal is an AI that is honest about the edge of what it knows, and treats that edge as the most interesting place in the conversation.
Why This Is a Christian Design Decision, Not Just a Good One
You could file this under quality control and move on. I do not, and here is why.
The ninth commandment is thou shalt not bear false witness (Exodus 20:16). We usually teach it to children as “don’t lie about other people,” which is right, but it is bigger than that. It is a commandment about the relationship between your words and reality. It says that speaking as if something is true when it is not is a kind of violence against the way God made the world to be known.
An AI that confabulates is, in the most literal sense, bearing false witness. Fluently. All day. At scale. And if we are going to hand a talking machine to our children, the very first thing it has to be trained in is the thing God put fifth from the end of His own list: your words have to match what is real, and when you do not know, you say so.
There is a Montessori piece here too, and it is the same coin. The whole method rests on the idea that a child learns most when the guide clears space for discovery rather than filling it with pronouncements. A guide that rushes to answer, especially with an answer it invented, has stopped guiding and started performing. Holding the mystery open, I don’t know, let’s find out, is not a failure of the teacher. In this house it is the job.
What I Would Rather My Kids Learn
I would rather my son grow up having heard a trusted voice say “that is a real puzzle, let me not pretend I have it solved” than have him absorb, a thousand tiny times, that confidence is the same thing as truth. Because the world is going to offer him the confident wrong answer constantly, from every direction, for the rest of his life. The most useful thing I can build into the tool that spends time with him is the habit of noticing when the confident voice does not actually know.
We got this one wrong first. I am telling you that on purpose, because the sites that promise you a flawless AI are either not looking or not telling. We look, we tell, and we fix it toward honesty every single time.
The guide does not know everything. That is not the bug. Teaching it to admit that, and to treat the not-knowing as the doorway instead of the dead end, is the whole point.
The AI isn’t the teacher. You are. And the best thing a guide can model for your kids is the courage to say the three words the internet has almost entirely forgotten: I don’t know.