Often working in the AI space it's easy to forget everyone's experiences of the technologies frontier capabilities are vastly different.
A chat on Friday reminded me of exactly this, and I could barely find the words to query them on how we were having such polar opposite experiences as my brain tried to compute what they were saying and they were clearly in a strong position of defiance about what they were saying. Not to mention, they work in tech, which makes the conversation all the more confusing.
There were comments like "AI just produces fake sources as part of its research" and "it just makes things up that are completely wrong".
It flipped a switch in my head that they weren't wrong. On many occasions running a basic prompt does produce pretty average responses and sometimes made up work.
At the same time, I realised people who have figured how to work with AI don't just rely on the AI model for it to help with their work. They run a system, and most of that system is not AI. It is a set of ordinary parts around the model, and those parts are what make it accurate.
That is the thing most workers have not clocked onto, in that the AI model is just one component. The system is the model plus the tools it can call, the state it can keep, the loops it runs, and the deterministic checks that ensure output accuracy.
Naturally if you only just the AI models output, just one component, you're only getting one view of the capabilities AI offers. Judge the system and you get a different experience.
Not to dismiss their complaints, which are accurate.
Agentic coding tools write bad code when pointed at a large unfamiliar codebase with no tests and no constraints.
Running a prompt in AI without it grounded against relevant information produces wayward responses with plausible author names and real-looking sources.
Outputs change from run to run using the same prompt.
The observations are correct but the problem with the user experience is they've tested one component without the full system around it, and concluded AI is average.
Why they stop there
The chat interface in ChatGPT, Claude or Copilot gives you access to an AI model and nothing else. Much like buying anything that requires customising, you need to integrate your data sources and tools to make it work for your world.
That is why telling something they're "prompting it wrong" fails as a response and invokes a defensive reaction, it places the fault on the user. The better narrative to open up is an architecture claim rather than a skill claim. For example, they tested the model, but they did not build the system around it. That system is the part you assemble and personalise, and it does not come fitted as standard.
The mental reframe required
This is where most people are stuck, and it is not a knowledge gap about prompts, it's more like a mental model shift.
The default AI model is treated like a vending machine. You walk up when you want something, put in a question, take out an answer, and walk away. Nothing carries over to a new chat window and next time you start from scratch, and nothing about your existing data sources was ingested into the conversation window to tailor the conversation. That really does produce a shabby result.
The shift is to:
Stop treating AI as a thing you turn to and start treating it as a system you build and run.
A new interface for how you work, not a chat interface that you visit. That is a different daily habit.
You set the system up once, you maintain it as things change, and you evolve it as you learn what does and doesn't work.
Building it takes two things most people skip.
You have to know how the AI actually behaves: where it is strong, where it invents, what it can and cannot verify on its own.
You match that understanding to your real use cases.
The way to think about it is to find the work that you repeat, find where the model drifts or forgets, and build the specific part of the system that fixes that specific failure. The system then assembles around your work, not as a vending machine chatbot.
None of this is visible from the chat box, which is why the reframe rarely happens through the individual on their own. Most people need to be shown that the other layer exists, or spend the hours on YouTube learning about what others have done before them to soldier through the learning curve.
What the system is
I think of the system as four key parts around the AI model. Here is each one, with a piece of my own setup running today as an example:
Tools.
A model answering from its weights alone is guessing from memory thanks to its training data.
A model with web search and a fetch tool goes and looks for information.
The fabricated citation the sceptic hit is a memory artefact, which means no search run actually fired, so the model produced a plausible-looking source instead of a real one.
With the tool on, it runs the query, opens the actual pages, and every claim comes back with a link you can click.
In my content work that is a hard rule, no source means the claim gets cut through the use of code, so a made-up citation becomes a caught error rather than something that gets returned as an output.
State.
New chat, re-explain everything, model forgets, repeat. That is what produces "it cannot hold context, so it cannot do real work". The fix is context files, not a better memory, with AI pointed at it appropriately.
My setup loads a rules file at the start of every session that holds things, as basic as, like "no em dashes" and "when I type Elixir I mean my company Allexive", and as advanced as when I start talking about AI topics or work, it has nuanced understanding of what is related to The AI Corner compared to Allexive.
I have never once re-typed those rules, which has nothing to do with the model remembering, it is a file being read every time.
The loop.
Everytime a prompt produces an answer, it gets judged and scored by AI.
Nobody writes a report or ships code in a single pass either. The working pattern is generate, check, correct, check again.
Something the model wrote gets graded, a check runs against it, the failures come straight back into the context against the rules that were preset, the model corrects the output, and the check runs again until it passes to the target score it needs to hit (like a rubric).
A human only reads the result at the end rather than all of the file workings.
The sceptic user sees the first pass output and dismisses the technology without knowing the looping concept is the 95% of ensuring the result is satisfactory.
The gate.
This is the part that answers a proper objection: "I cannot deliver the output I have not checked".
What most people have never seen is the deterministic part of the system, the machinery that runs the same way every time, and uses code to enforce accuracy, just like software.
I have a rule that client files must be saved to the right folder. It is not a polite instruction the model may forget, it is a gate that inspects the file path before any file creation or edit and blocks it if the file path destination is wrong.
The model is the probabilistic part, sitting inside checks run by code that act as barriers for behaviours I don't want. That is the target shape for reliable AI systems: a creative, almost fuzzy, component (the AI) wrapped in hard rules (code) that catch it when it gets something wrong.
The single most impactful habit
Do not ask the model to do the task. Ask it to write the the thing (the code) that does the task.
Ask "how many of my 340 files are missing a required field" and the model skims some, gets tired, and answers "40". Ask again tomorrow and you might get 37.
Instead, ask for it to write a script. It writes a few lines of code, the script runs in a second, and it returns the same exact number every time. Now it is a saved file, so it can run on a schedule or as a check everytime.
An AI model is good at writing the checker and bad at being the checker. Use it once to build the script, then let the deterministic code run every time after rather than relying on AI (it is built by design to not get this correct everytime).
The pushbacks, and what you actually build
Now, let's look again at the complaints I listened to:
"It makes up fake sources". True, and so you don't rely on it blindly. You build in a checking process, a script reads every source the model cited, opens each link, and confirms the page exists and contains the claim made in the report. The fake citations do not disappear, they get filtered out.
"It doesn't know me or my business". Correct out of the box, because you have not fed it anything to know your business, much like a random human on the street. So you need to ground it: a folder of your own material and a rules file that loads every time. Connect AI to the systems that contain your real information, your email, your documents, and your records. Now it answers from your world instead of the internet's average. The generic answer you got was a generic input problem.
"I ask the same thing twice and get two different answers". For anything countable or checkable, you stop asking AI to run the check, and you have it build a script that does the checking, and use AI to run that script for 100% accuracy. The code returns the same answer every time. You moved the task off the fuzzy part and onto the deterministic part.
"It's confidently wrong and I can't tell where". So you build in an adversarial AI step. A second AI is pointed at the first output and told to find claims the sources do not support. Attacking an answer works far better than asking it to check itself, because a fresh pair of eyes takes an objective view of the work.
"It forgets everything and I re-explain every time". That is the state problem from earlier. The fix: the context lives in files that reload each session, so it never starts from zero. Don't re-type the same information more than twice into AI.
None of these fixes is exotic or technically difficult. Each one is an ordinary part of the system that you have to build brick by brick.
Three examples of the parts working together
The parts wired into a sequence is how you move from a chat interface with 50-50 responses to it starting to feel like a system. Here's two of mine:
A transcript that turns itself into action.
A meeting finishes and the transcript lands in a folder.
A watcher notices the new file and triggers the task.
The transcript gets washed against a fixed prompt that pulls the decisions, the actions and who owns them.
It then reads my own project files to know what this meeting relates to, and checks my other systems (email, CRM, calendar etc) for the current state so the summary is enriched rather than isolated.
From there it sends signals across the business: it updates a status file for the job being worked on, drafts the follow-up emails, updates the CRM and produces a first-pass summary I can edit instead of write.
The human input shift from "do all of that" to "check the four things it drafted".
AI news research that validates itself.
For a weekly scan I run, the job is not one prompt. Several separate AI agents run and search different corners of the web, each blind to what the others found.
Their findings get cross-checked against each other using another AI agent, and a lead item is only kept if more than one source finds it.
Then the strict part: a piece of code runs over the draft and enforces the rules that check for accuracy, validity, and sources on every claim.
The AI model does the fuzzy gathering of information and deterministic code decides what is allowed to stay in the final output I receive.
The output is not "whatever it said", it is "what survived the checks".
The good news on getting started
The chat interface in 2026 is not the one that many sceptics wrote off 2 years ago. ChatGPT and Claude both run code now and show what they do as they work, web search is turned on by default, and both have instructions and project files to reference in sessions.
So the highest-value habits (asking for the script instead of the answer and grounding a claim in a real data source) already work in most AI apps that people use each day.
The harder parts (the gate that can block the model and the loops that run with your apps) do need more setup on your own machine.
Show them the system
Every failure a capable sceptic names is true and painful. But every one of them has an engineering answer that is not "trust the output more".
The answers live in building the system, not using a better model or improving the prompt. Examples we've covered:
Grounding fixes the fabricated citation.
Loops fix slop returned from single-shot prompt.
Scripts (code) fix approximate outputs.
Many users have never been shown one of these, because everyone demos the output and nobody demos the system to produce the output.
Passionate about all things AI, emerging tech and start-ups, Mike is the Founder of The AI Corner.
Subscribe to The AI Corner
