Who is responsible for an AI mistake? The roles of the employee, the team and the company

Why a wrong AI answer can look credible

A team is preparing a proposal, and a language model supplies three detailed sources. They are correctly formatted, with authors and dates. Only the client notices that one of the publications does not exist. The problem is not just the model’s “hallucination”. What also matters is that the confident-sounding sentences and familiar document layout made it easier to accept the answer without checking it. It is easy to judge how credible information is by its form alone, even though form does not confirm facts.

In a review of research, Goddard and colleagues described a tendency to over-rely on suggestions from automated systems. Much of the analysis concerned medicine and older decision-support systems, so the findings should not be mechanically applied to all generative models. The mechanism remains useful, though: even a competent specialist may stop looking for errors when a suggestion is presented convincingly and time is short. That is why simply saying “but we checked it” is not enough without specifying what was actually checked.

Why responsibility does not rest solely with the person who signs off

A document may pass through several pairs of hands. The first person chooses the tool and the data, the second edits the text, the third checks the figures and the fourth signs off the finished proposal. If the company has not defined these roles, everyone will point at their neighbour after a mistake. “I thought the analytics team was checking that” will collide with “I was only looking at the language”. Legal liability has to be assessed separately, but the organisational causes of the error can be established straight away.

It helps to separate four questions: who was allowed to use the tool, who was responsible for the accuracy of the data, who had the authority to stop the document going out and who was supposed to provide the conditions for checking it. The employee carrying out the final step may have a duty of care, but the company may at the same time be responsible for a flawed procedure. It is unjustified to automatically shift all the blame onto the person who had the least say over how the work was done. It would be equally wrong to claim that because “the AI got it wrong”, no one needs to draw any lessons.

Two examples show the difference between using a tool and checking its output

In the well-known US case of Mata v. Avianca, lawyers filed a document containing non-existent court rulings generated with the help of AI. The court imposed sanctions. This is an example of failing to verify the sources cited, not proof that every model error automatically gives rise to the same liability. In a Polish company, a more everyday problem might be sending a client a proposal with the wrong payment terms because no one compared the text with the approved table.

Imagine two teams. In the first, checking means three minutes of skim-reading a generated report. In the second, the reviewer has a list of fields to check, access to the original data and the right to send the document back for correction. In both teams a person checks the output, but only in the second do they have a real chance to challenge it. The conclusion is not to create a mountain of bureaucracy for every message, but to match the level of checking to the weight of the decision and the risk of error.

What AI regulation says about oversight – and what it does not settle

The EU AI Act sets out, among other things, requirements for human oversight of systems classified as high-risk and obligations for the organisations that use them. The rules distinguish between categories of use and the dates from which individual requirements apply. So it is wrong to claim that every company conversation with a chatbot is subject to an identical regime, or that the AI Act alone determines who is personally liable for every instance of damage.

In practice, a manager should check which system the company uses, what it is intended for and what rules follow from the relevant regulations, contracts and procedures. For uses involving health, employment or fundamental human rights, the stakes and obligations are different from those for editing a neutral note. What matters most for the team is that the decision to use a tool has an owner, and that the person checking has the competence, the time and the right to say “I’m not approving this output”.

How to set up checks before material leaves the company

It is worth preparing a short responsibility matrix. For each process, record: the approved use of AI, the input data, the elements that must be checked, the person who signs off and how to report a problem. For a numerical report, the key step will be comparing the values with the source system. For a legal text, it will be checking the provisions against an up-to-date source. For medical information, it will be proper expert review by an authorised person. An AI tool is not a substitute for that check.

The purpose of such a matrix is not to identify a future culprit. It is meant to make it easier to take decisions and stop errors. If an employee discovers that the tool has made up a source, they need to know whom to report it to, without fear of being penalised for halting the work. A manager should also check whether, under deadline pressure, they are asking the team for checks that physically cannot be carried out. Accountability without the right conditions is nothing more than a declaration.

How to talk after a mistake so the next one is reported sooner

When a mistake comes to light, the first question should be: “How do we limit its effects?” Only then is it worth reconstructing the sequence of events. Instead of opening with “Who let this through?”, a manager can say: “Some incorrect figures ended up in the document. Let’s check where they came from, what checks we carried out and where the process stopped working.” Talking this way does not mean accepting negligence. It makes it possible to distinguish between an unclear task, inadequate checking and a deliberate failure to follow what was agreed.

The analysis must lead to a decision: we update the instructions, add a check, restrict the use or train the team. Also agree who will inform the recipients about the correction, and by when. If an employee had warned about the risk earlier, the manager should take that into account. Teams quickly notice whether owning up to a mistake leads to learning or merely to a hunt for a scapegoat. That reaction determines whether the next error is brought to light early.

Behind the consequences of an AI error lie both the users’ decisions and the way the process is organised. You need to define who verifies the information, what exactly they check and whether they have the conditions to check effectively. After a mistake, first repair the damage, then establish accountability and change the procedure that allowed the error to slip through.

How Empatyzer supports a difficult conversation after an AI-related mistake

After an incorrect proposal has been sent, a manager can describe the facts to Em: what was produced with AI, who was supposed to check the document, what had been agreed and what the client said. Em will help prepare a meeting at which people won’t have to defend themselves against a presumption of guilt, but won’t be able to avoid a conversation about their responsibilities either. With Em, the manager can put together questions that separate facts from judgements, rehearse a response to “but everyone was doing it this way” and prepare clear feedback. If the participants are from the same organisation, the available relationship tips help match the language to people’s different ways of reacting to criticism and uncertainty; they do not, however, give access to other people’s private diagnoses. In the team context, Em can help plan the incident review and how to present the new rules for working together. As a result, the manager goes into the meeting not just with a list of questions about who is to blame, but with a plan for putting the errors right, a clear division of responsibilities and a way of encouraging people to report problems earlier.