Klarna can point to millions of customer conversations handled by AI. Air Canada had to answer for a chatbot giving a customer the wrong advice. If you’re signing off AI investment, which story is your organisation preparing for?
If you’re a CEO accountable for the outcome, a COO trying to turn efficiency promises into reliable delivery, an HR Director watching how new tools affect confidence and trust, or a Transformation Director trying to close the gap between strategy and reality, this matters to you.
The temptation is obvious.
A chatbot answers customers at any hour. It can handle routine enquiries, help employees find information and take repetitive work off people’s desks. Klarna’s much-publicised AI assistant offered a compelling example of that potential, handling a large volume of customer service conversations shortly after launch.
For a leadership team under pressure to improve service and control costs, it’s an attractive proposition.
But then there’s Air Canada.
Its chatbot gave a customer incorrect information about bereavement fares. The airline was held responsible for that information by the British Columbia Civil Resolution Tribunal.
The important detail? The customer didn’t have a relationship with a language model. They had a relationship with Air Canada.
The organisation remained accountable.
That distinction sounds obvious when you say it aloud. Yet organisations are introducing AI into customer-facing work, internal decision-making and regulated processes at a pace that can leave the underlying responsibilities unclear.
I’ve seen how quickly a good idea can become an expensive problem when nobody has properly worked through what happens after the decision gets approved.
The technology demonstration goes well. The business case promises efficiency. The project plan turns green. Then somebody asks a question nobody prepared for.
Who’s checking the answer before it reaches the customer?
Who owns the decision if the chatbot gets it wrong?
And what happens when the employee who spots the problem feels that challenging the new system will make them look resistant to change?
That last question tends to get less attention than the software.
It shouldn’t.
Organisations need to understand the legal, ethical, and operational risks of AI, rather than treating ease of access as evidence of safety.
For regulated businesses, which includes confidentiality, privacy obligations, intellectual property, the quality of the information used by AI systems and the controls surrounding their use.
These are practical business questions, not paperwork to complete after implementation.
Yet the human side can be just as difficult to get right.
Imagine a manager who’s spent fifteen years building expertise in a particular area. Suddenly, a new chatbot can produce a plausible first answer in seconds. The manager is told to encourage adoption, improve productivity and help the team adapt.
What happens if they aren’t confident in the tool?
They might check every answer themselves. They might quietly tell their team to stick with the existing process for anything important. Or they might publicly support the rollout while privately warning colleagues not to trust it.
The adoption dashboard still looks healthy.
The behaviour tells you something else.
It’s tempting to call this resistance. Sometimes it is. But sometimes people are protecting their professional reputation because nobody has explained what good judgement looks like when AI enters the process.
AI adoption doesn’t stall simply because the technology is unfamiliar; it stalls when people are expected to trust a new system without clarity about their responsibilities, their boundaries, or the consequences of getting it wrong.
That’s the distinction I want leaders to take seriously.
My military service taught me that accountability matters most when circumstances change and pressure rises. Running a business taught me that the cost of a poor decision rarely ends with the decision itself. Someone must fix the work, explain the mistake, or rebuild the customer’s confidence.
AI brings those lessons into everyday management.
A chatbot can generate a convincing answer without that answer being accurate. It can produce a citation that doesn’t exist or offer advice that conflicts with an approved policy. Its fluency can make a weak answer look dependable, especially to someone who hasn’t been trained to recognise the limitations.
And a warning in a training slide doesn’t automatically change behaviour.
The BCG Henderson Institute’s experiment with consultants illustrates why task selection matters. Generative AI helped participants on certain creative tasks but hindered performance on a business problem-solving task outside the technology’s capabilities. The lesson isn’t that AI should be avoided. It’s that the value depends on what you’re asking it to do and how people judge the answer.
This is where the conversation needs to move beyond adoption targets.
Have you decided which tasks AI can support and which require a qualified human decisionmaker? Do employees know when to verify an answer against an authoritative source? Is confidential information being entered into tools that have been approved for that purpose? Can someone stop an unsafe process without being accused of holding up progress?
Technical controls matter. So do people.
A properly designed knowledge base, retrieval-augmented generation and suitable connections to approved systems can improve the relevance of chatbot responses. They don’t make errors impossible. Human review still needs a clear purpose, a named owner, and enough time to do the job properly.
The change becomes visible when you fix those conditions.
An employee receives an answer that doesn’t match the approved guidance. Instead of copying it into a customer email, they flag it. Their manager knows who reviews the issue. The team corrects the process, and the next employee has clearer guidance.
No grand announcement. Just better decisions.
That’s the work at Elvorta: helping leaders understand what happens to people when technology changes their roles, expectations, and sense of control, then addressing those pressures so the organisation can change in practice.
Because the real measure of AI adoption isn’t how many people have opened the chatbot or completed the training.
It’s what happens when the answer looks convincing, the deadline is close, and somebody must decide whether to trust it.