Most of what gets written about AI is either breathless enthusiasm or existential dread. Neither is particularly useful if you’re trying to figure out when to trust the thing and when not to. Here’s a more practical take: where AI consistently falls down, and why knowing that in advance saves you a lot of wasted time.
It Confuses Confidence With Accuracy
This is the one that catches people most often. ChatGPT, Claude, Gemini — all of them produce text that reads with the same tone of certainty whether they’re right or completely fabricating something. There’s no internal signal that says “this is a guess” versus “this is well-established.” The output just comes out fluent and authoritative either way.
Ask Claude who won a regional theater award in 2019 and you might get a confident, plausible-sounding name that doesn’t exist. Ask it about a court case and it might cite a real case number but get the ruling wrong. The dangerous part isn’t that it makes stuff up — it’s that it doesn’t sound like it’s making stuff up.
The practical fix is boring but it works: never use AI as your primary source for anything where accuracy actually matters. Use it to understand a topic, generate questions to go research yourself, or draft something you’ll verify. The moment you start treating its output as fact you haven’t checked, you’re in trouble.
Counting, Arithmetic, and Anything That Requires Precision
Ask ChatGPT to count the number of times the letter “r” appears in a paragraph and there’s a decent chance it gets it wrong. Ask it to do multi-step arithmetic without a code interpreter and errors creep in more often than you’d expect from something that seems so capable elsewhere. This is a known quirk of how large language models work — they’re predicting tokens, not running calculations.
The newer models have gotten better at this, especially when they can run Python in the background. But on a basic setup without code execution, don’t trust it with a spreadsheet formula you haven’t verified, don’t ask it to total numbers in a document, and don’t assume it correctly parsed a date range you gave it. Small errors in those cases compound.
For anything numerically precise, run it through actual software. AI is good at explaining what formula to use in Excel. It’s less reliable at being your calculator.
Tasks That Depend on Real-Time Information
Most AI models have a training cutoff — often six months to a year or more before you’re actually talking to them. Anything about current events, recent product releases, live prices, or what happened in the news last week is either unavailable or potentially stale.
Some tools now have web search baked in, which helps. But even then, the model still has to interpret what it found and summarize it, and that introduces its own failure points. If you need to know whether a specific policy changed, whether a software library updated its API, or what the current interest rate is — just go look it up. Don’t route it through an AI and assume the answer is current.
Understanding Your Specific Context Without You Spelling It Out
AI has no idea who you are, what your situation is, or what actually matters to you — unless you tell it. Ask for advice on handling a difficult conversation at work and you’ll get reasonable, broadly applicable suggestions. Ask again but explain that you’re a contractor, the person is a client you’ve had for three years, and the issue involves a missed deadline that was partly your fault — suddenly the advice is actually useful.
The model isn’t lazy. It just doesn’t have access to the context that would make its response meaningful. Every bit of relevant detail you withhold is a bit of quality you’re leaving on the table. This is why prompt quality matters — not because there’s some magic formula, but because the model is working only with what you give it.
I’ve started treating every AI prompt like I’m briefing someone sharp who just started the job today. They can do the work, but they need the context I’d otherwise assume someone already knows.
Anything Requiring Genuine Taste or Judgment
AI can write. It writes a lot, pretty fluently. But there’s a difference between writing that’s technically competent and writing that’s actually good — writing that has a specific voice, makes a surprising choice, cuts something that doesn’t need to be there. AI tends toward the expected. It pattern-matches against an enormous amount of text and produces something that sits near the average of what that text looked like.
That’s fine for first drafts, outlines, and routine content. It’s a problem if you want something distinctive. The same goes for design decisions, editorial judgment, creative direction — anything where the value is in making a specific non-obvious call. AI can generate options. It’s not great at knowing which option is right for a specific audience, moment, or brand.
If you use it knowing that, you’ll edit its output differently — looking for places where it defaulted to the safe, predictable version of something, and making the more interesting call yourself.
Long Tasks It Can’t Check Against Itself
Give AI a long document to write from scratch and by the time it’s halfway through, it may have quietly contradicted something from the first section. It doesn’t re-read itself the way a human writer would. Context windows have gotten large enough that this is less of a problem than it used to be, but it still happens — especially with complex, structured content where internal consistency matters.
If you’re using AI to produce something long, break the task into pieces, review each piece before feeding in the next, and do a pass at the end looking specifically for contradictions or drift. Not because AI is sloppy, but because it doesn’t have the same self-correcting instinct a person gets from re-reading their own work.
Where This Leaves You
The people who get burned by AI are usually the ones who handed over something that required precision, real-time knowledge, or genuine judgment — and assumed fluent output meant reliable output. Knowing where it fails is most of the battle.
One thing worth keeping in mind: even these failure modes are moving targets. Arithmetic with code execution is much better than arithmetic without it. Search-enabled models are more current than ones without. The specific version of the tool you’re using matters more than people give it credit for — the gap between a base model and one with tool use enabled is not small.
