What the AI Actually Knows (and What It Doesn't)
10 min read
You don't need to understand engines to drive a truck. But you should probably know it runs on diesel and not optimism — otherwise you'll spend a lot of time angry at the truck for problems that were never the truck's fault.
Consider this lesson the founder's look under the hood. No math, no jargon beyond two terms worth learning, and three ideas that each explain a frustration you've probably already had.
Idea one: it predicts words
Claude, ChatGPT, and every tool like them were built the same basic way: feed a model an enormous amount of text — books, websites, forums, manuals — and train it to guess the next word. Repeat billions of times. Eventually it gets remarkably good at continuing any piece of text in the most plausible direction.
Plausible is the key word. The model learned how quotes, apologies, job postings, and business plans usually sound. It absorbed the patterns of the world's writing. What it never absorbed: your prices, your customers, your margins, or anything else that wasn't in its training text.
Maya owns a 14-person landscaping company in Calgary. The model has read thousands of pages about landscaping. It has read exactly nothing about Maya's company. It can talk fluently about spring cleanups in general — and can only guess about hers.
Idea two: its knowledge has a cutoff date
Training happens once, on text collected up to a certain date, and then the model is frozen. Anything after that moment — this season's fuel prices, a new city bylaw, the competitor who opened across the street in May — simply isn't in there. Some tools bolt on web search to patch the gap, but the model itself is a snapshot, not a live feed.
The practical rule: never assume it knows anything recent or anything local. If a fact matters and it's newer than a year or two, hand it over yourself.
Idea three: the desk
Now the mental model that will change how you work. Every chat has a working space called the context window. Picture it as a desk. The model can use anything sitting on the desk — your messages, whatever you pasted, its own earlier replies — and absolutely nothing that isn't.
Your price list is not on the desk unless you put it there. Neither is the conversation you had yesterday; close a chat and the desk gets swept clean. Each new conversation starts with an empty surface.
This is exactly why the briefing habit from Brief It Like a New Hire works. A briefing isn't a magic incantation. It's you loading the desk with the facts the model needs to do the job.
Watch what the desk does for Maya:
Without context:
"Draft a booking confirmation text for a spring cleanup."
→ Generic. Invented services, no price, sounds like a call
center in nowhere-in-particular.
With context on the desk:
Context about my business (use only this):
I run a 14-person landscaping company in Calgary. Spring cleanup
is our rush season — about 300 residential jobs in six weeks. A
standard cleanup is $340: dethatching, first mow, bed edging,
and debris haul-away. A two-person crew takes about 90 minutes.
We confirm bookings by text message.
Task: Draft the booking confirmation text. Include what the crew
will do, the $340 price, and a reminder to leave the side gate
open. Under 60 words, friendly but efficient.Same model, same afternoon. The first version guesses at every fact. The second works with Maya's actual numbers, because for the length of that chat, they're on the desk.
The confident guess
One more thing, and it matters enough to get its own chapter later. When the model doesn't know something, it says "I don't know" far less often than it should. Instead it predicts the most plausible-sounding answer and delivers it in the same assured tone as everything else. The industry term is hallucination: invented facts, made-up statistics, the occasional citation to a study that doesn't exist. The tone never wavers, which is what makes it dangerous — treat confidence as a writing style, not as evidence. Chapter four covers this, including how to catch a confident guess before your customers do.
Three ideas, three habits
It predicts patterns, so feed it your specifics or expect the average. Its knowledge is frozen at a cutoff, so supply anything recent or local yourself. And it can only use what's on the desk — so put the important stuff there, every chat, every time.
None of this makes the tool less impressive. It makes it predictable, which for a business owner is better.
Try it now
Write one paragraph about your business: what you sell, roughly what it costs, who buys it, and one thing that makes you different. Save it in your notes app — you'll reuse it constantly. Then ask an AI a question about your business twice: once cold, once with your paragraph pasted first. The gap between those two answers is the desk, made visible.