Consistently the first thing I hear from leaders is that we've trained our people on AI, but nothing in the business has changed.

AI training is 20% of what it takes to change how a business works. But most organisations do the 20% and wonder why nothing changes.

The team gets trained and the CEO mandate gets delivered for the organisation to change. What happens in practice is six months later, the best people are still doing the same work the same way using Claude or Copilot, and occasionally using a scheduled task to automate their morning daily plan.

The organisation has hit a ceiling, as the business and the work haven't changed.

That is not a training problem that's caused it. AI training is a fundamental requirement for every person in the workforce.

But it is far from the end goal.

The problem is more structural about why a business isn't seeing material value from AI. And until leaders understand the difference, they will keep buying more training and getting the same disappointing results.

Why training produces a false ceiling

Training teaches people to use a new tool. It does not teach them to redesign how they work, and those are completely different things, even though most organisations treat them as the same.

McKinsey research found that nearly 90% of organisations are experimenting with AI, while fewer than 10% have changed their business thanks to the technology.

That gap is not down to any lack of ambition or budget. It is structural problem, and it is not going to close by sending another cohort through the same training workshop.

The Dunning-Kruger effect runs hard through most AI programmes.

  • Someone attends training, gets comfortable with prompting, and starts producing first drafts faster.

  • They feel like they have levelled up, and in one narrow sense they have.

  • But prompting is the first rung on a much longer ladder, and most people mistake it for the strategy.

Worth noting: none of what comes next (context setup, agents, system integration, model selection) is covered in a standard training programme. It comes from months of hands-on trial and error, the kind most people only go through if they are building something for themselves.

Where it breaks in practice

Recognising AI training hasn't got the business far, most organisations then decide to 'do AI properly' by nominating someone internally.

Often it is an enthusiastic person who has spent time picking up AI on their own: hours of YouTube and Twitter, testing n8n and every new tool under the sun.

Good intentions, wicked enthusiasm, but there are five gaps that consistently show up once that person tries to make something real happen:

  1. They do not know how to redesign work. Using AI to do the same job faster is not transformation. Knowing which workflows to rebuild, in which order, and how to get a team to actually change how they operate is a different skill entirely.

  2. **They do not have AI engineering depth.**Knowing how to prompt is not the same as knowing when to use a model versus a script, how to structure context, how agents manage state, or when retrieval-augmented generation is the right architecture versus a fine-tuned model.

  3. They assume personal builds translate to business contexts. Something that works brilliantly for one person breaks when it needs to be reliable, auditable, and maintained by someone else. The gap between a personal n8n workflow and a production system is not a small one.

  4. **They do not understand enterprise deployment.**Data residency, hosting requirements, model access policies, integration with existing systems: none of this features in self-directed AI learning, and all of it becomes a blocker the moment something goes near a real business process.

  5. They are not resourced to drive change. Most people nominated for these roles still have a full-time job. They are not measured on whether workflows change. The organisation has trained someone and called it a programme, without building in the mandate, the time, or the accountability to actually deliver anything.

I've talked about work redesign at length in the past (including here), so let's pick on another point to bring it to life: the third gap is where it gets most visible in practice.

Most self-taught AI builders learn to prompt, but very few learn to engineer (or have a software background. I've learnt this myself the hardway!). They do not know when a model is the wrong tool entirely, how to structure context so outputs stay reliable at scale, or when the right answer is a script rather than a conversation. Here is what that looks like when an AI engineer approaches these problems in real business context.

  1. A manufacturing business worked with technical documents: electrical diagrams, architectural drawings, and supplier quotes in PDF format. Someone proposed using Copilot to read these automatically. But if a single measurement, wiring spec, or dollar figure is misread, that is not a minor glitch. That is a safety issue or a costly procurement error. AI-based extraction is not reliable enough here, and deterministic methods are what is needed: scripts and structured processes that pull exact data every time with built-in verification and rules to check against.

  2. A financial services client wanted to pull customer records from an internal system. The first problem was compliance, not accuracy: if data is being sent to a model hosted offshore, the residency implications alone should be enough to stop the project. The second was scale: relying on prompts to retrieve data gets less reliable as volumes grow, because context window limits create a hard ceiling on what can be processed at once. A basic script would have done the same job more accurately, for a fraction of the cost.

  3. Scale is what makes the third case problematic.Two separate clients (a financial services organisation and a tertiary institution) both wanted insights pulled from thousands of documents collected over several years. The obvious shortcut was to connect Claude directly to the transcript system and ask questions. It does not work because no context window can hold thousands of transcripts at once. What works is building a knowledge base first: extracting themes, validating accuracy, creating a pipeline so new transcripts get added correctly, and then running ongoing consistency checks to make sure the answers stay grounded in what is in the data. That is a piece of engineering work, not something that can be solved with a prompt.

The cost when it goes wrong

When an inexperienced enthusiast builds something the wrong way, it usually works in testing. Then it goes into full production use and the cracks appear: wrong records get pulled, numbers do not match, and people notice the mistakes and stop trusting the tool altogether.

The damage is not just the wasted budget, it is the loss of momentum and confidence across the whole business in AI as a capability, and that is much harder to win back than it was to lose.

What actually fixes this

There are two paths forward, and the right one depends on how much time the organisation has.

The first is to wait it out, letting the team build capability organically around their existing workload, and hoping a technical person (ideally with a software and/or data background) figures out what good AI engineering looks like. This can work, but it is slow, and it relies on people finding time they likely do not have.

The second is to bring in external capability, such as myself and my time at Allexive. Not a switched-on AI enthusiast, but a swat team who combine three things:

  1. **Technical depth:**built and deployed AI solutions across different environments, able to judge when to script, when to use a model, and when to use both.

  2. **Commercial and change management sense:**able to sequence change so it sticks, not just build for building's sake.

  3. **Cross-functional fluency:**able to translate between a technically-minded engineer and a business function leader, and bridge what is technically possible with what the business needs to adopt it.

This combination is rare, and the two obvious internal candidates each fall short.

  1. Not the average engineer (most engineers are rightly focused on the technology and the data, not on how an executive perceives the output or how it rolls up into a board report).

  2. Not the average business strategist either, because they typically do not understand the practical pitfalls of, say, building in Microsoft Copilot Studio versus Foundry versus a custom-built solution elsewhere, or the trade-offs between building with Claude's Agents SDK vs another ADK/SDK.

This is not a job for someone six months into their AI journey, no matter how enthusiastic they are.

The real takeaway

Training a team on AI tools is necessary, but it is the easy 20%. The other 80% takes real experience: understanding when to use AI and when not to, redesigning how work flows, and building the kind of trust that survives contact with production.

If AI results have stalled despite the training, this is probably why. Not a people problem, but a capability and structural problem, and it has a fixable solution.

Passionate about all things AI, emerging tech and start-ups, Mike is the Founder of The AI Corner.

Subscribe to The AI Corner

The fastest way to keep up with AI in New Zealand, in just 5 minutes a week. Join thousands of readers who rely on us every Monday for the latest AI news.