règle : θ = n × 137,508° · r = c√n · graine : responsibility-gap

Concept entry · Moral philosophy | Ethics of technology | Law | Applied cognitive science

Responsibility gap

A situation in which it becomes impossible or unjust to attribute full moral responsibility to an identifiable human agent for the harms caused by an autonomous, learning-capable system, because neither the designer, nor the operator, nor the machine itself meets the classical conditions of responsibility.

Origin: The responsibility gap: Ascribing responsibility for the actions of learning automata (2004), Ethics and Information Technology, 6(3), 175–183, 2004

The responsibility gap designates a structural lacuna in the moral and legal attribution of responsibility that emerges when autonomous systems — in particular those endowed with learning capabilities — produce decisions or actions that cause harm. Andreas Matthias's original formulation (2004) poses the problem in terms of ‘learning automata’: once a system can modify its own behaviour through experience, the designer can no longer foresee its future actions, which erodes the classical epistemic conditions of responsibility (knowledge and control). This erosion creates a dilemma: either one continues to deploy autonomous learning systems and gives up on having a human agent fully responsible for their behaviours, or one preserves the chain of responsibility and gives up on the learning autonomy of the systems.

Robert Sparrow (2007) formulated a particularly influential version of the problem as a trilemma applied to autonomous weapons systems: who can be held responsible when such a system commits what would be a war crime? Sparrow examines three candidates — the designer-programmer, the commander-operator, and the machine itself — and shows that none satisfies the conditions necessary for just accountability. The programmer did not design a specifically wrongful behaviour; the commander has lost control over the concrete actions; the machine possesses neither moral conscience nor vulnerability to punishment. This trilemma illustrates a situation in which ‘the amount of moral responsibility that ought to be attributed exceeds the amount that can be’.

Since then, the literature has considerably broadened the concept beyond autonomous weapons: predictive-justice algorithms (COMPAS), automated administrative decision systems (Robodebt), diagnostic medical AI, autonomous vehicles, and mental-health chatbots. Santoni de Sio and Mecacci (2021) identified four distinct gaps — of culpability, of public moral responsibility, of active responsibility, and of legal accountability — each with its own specific sources and remedies. In parallel, John Danaher distinguished the ‘retribution gap’ (linked to the absence of a punishable subject) and advanced the counter-intuitive argument that certain responsibility gaps might be morally desirable when they make it possible to delegate ‘tragic choices’ at a low psychological cost for human agents.

What this concept is not

It is not a legal vacuum. It is not the absence of applicable law, but the difficulty of attaching a fault to an identifiable agent when the decision is delegated to an autonomous system.
It is not a technical inevitability. The gap is a tendency, not a necessity: it often results from organisational choices — non-oversight, opacity — as much as from the technology. Naming it must not exonerate those who dug it.
It is not an entirely new problem. It extends older questions — the problem of many hands, organisational fault, oversight responsibility — that moral philosophy and organisational law already addressed before AI.

Examples

The Robodebt affair (Australia, 2016–2019)

The Australian automated welfare-fraud detection system issued 2 billion AUD in fictitious debts to 700,000 beneficiaries by calculating incomes averaged from annual tax data rather than actual fortnightly earnings. The faulty algorithm caused severe psychological distress and several suicides. The question of responsibility dissolved among Centrelink (the agency), the federal government, the IT designers and the politicians. The Commonwealth ultimately agreed to repay 751 million AUD, but no individual was criminally convicted. An emblematic case of the responsibility gap in algorithmic governance.

COMPAS — Predictive recidivism scoring (United States, 2016)

The COMPAS algorithm (Correctional Offender Management Profiling for Alternative Sanctions), used in several US states to assess recidivism risk in release decisions, was analysed by ProPublica (2016), which showed a systematic racial bias: Black defendants were almost twice as often wrongly classified as ‘high risk’ than white defendants. As the model was proprietary (Northpointe, now Equivant), neither defendants nor judges could access the decision logic. No individual agent — developer, judge-user, deploying State — could be held responsible for the systemic bias. An illustration of the explanatory gap and the culpability gap.

The death of Elaine Herzberg — autonomous Uber vehicle (Tempe, Arizona, 2018)

On 18 March 2018, an Uber vehicle in autonomous mode fatally struck Elaine Herzberg while the human safety driver was distracted. The emergency systems had been disabled by Uber to reduce ‘erratic’ behaviour. The NTSB established that the system had detected Herzberg six seconds before impact but had failed to classify her correctly. Arizona declined to prosecute Uber criminally; only the human driver was charged with criminal negligence. The company Uber (neither its executives nor its engineers) was not personally implicated. A textbook case of Danaher's retribution gap.

The ‘deferential radiologist’ — diagnostic medical AI

An illustrative case theorised by Lang, Nyholm and Blumenthal-Barby (2024): a radiologist trusts a diagnostic AI system against their initial clinical judgement; the patient dies of an undetected cancer. Who is responsible? The developing company did not design a specific defect; the radiologist followed the recommendation of a validated tool; the hospital correctly deployed the system. The authors propose ‘shared responsibilization’ as a palliative.

The Tessa chatbot — eating disorders (National Eating Disorders Association, 2023)

The Tessa chatbot, deployed by the (American) National Eating Disorders Association (NEDA) as a support tool, provided harmful advice on dietary restriction to users suffering from anorexia. The association had to withdraw the chatbot. No commercial provider was subject to the codes of ethics of mental-health professionals. An illustration of the responsibility gap in automated psychological-support systems and of the absence of regulatory mechanisms comparable to those governing therapists.

The Therac-25 radiation system (Canada/United States, 1985–1987)

A precursor case — before the philosophical formalisation of the concept — in which a computer-controlled radiotherapy system caused fatal overdoses following software bugs. Responsibility was distributed among the engineers (design defects), the operators (inadequate training), and the company AECL (lack of testing). No single agent bore full responsibility. Regularly cited in the literature on the ‘many hands problem’ as a prefiguration of the algorithmic responsibility gap.

Tesla Autopilot — fatal accidents

Several fatal accidents involving Tesla's Autopilot system (notably the Joshua Brown accident in 2016 and the Walter Huang case in 2018) brought to light the grey zone between ‘driving assistance’ and ‘autonomous driving’. Tesla argues that drivers remain responsible; victims and regulators argue that the system's marketing induces excessive delegation. The NTSB investigated without leading to criminal proceedings against the company. An illustration of the gap between nominal responsibility (the driver) and actual responsibility (the system's designer).

Other perspectives

Tigard (2020): there is no structurally new techno-responsibility gap — our practices of holding responsible are dynamic and can adapt. The argument rests on too rigid and retributive a conception of responsibility.
Königs (cited in the SEP): we do not assume for every event that there is someone responsible; the real question is the distribution of risk, not the attribution of blame.
Oimann (2025): a novelty critique — ‘old wine in new bottles’. The problem of many hands, organisational fault and oversight responsibility existed well before AI; the concept merely renames classical problems.
Instrumental critique (Henman, 2025 on Robodebt): the responsibility gap in cases such as Robodebt is not a technical inevitability but the result of deliberate organisational choices of non-oversight — ‘venality, incompetence and cowardice’, according to the commission of inquiry. Naming a ‘gap’ risks exonerating agents who actively avoided responsibility.
Will machines ever be able to be legitimate moral agents, thereby closing the gap through direct attribution of responsibility to the machine? (Floridi & Sanders vs. Sparrow)
Is Lang et al.'s ‘shared responsibilization’ ethically acceptable if it amounts to requiring supererogation (beyond duty) from stakeholders?
Does Danaher's argument on the ‘virtue’ of responsibility gaps for tragic choices justify delegating difficult clinical decisions to AI, or does it set a dangerous precedent for the moral atrophy of health professionals?
Does the European AI Act (2024/1689), with its obligations of transparency, traceability and human oversight for high-risk systems, constitute a sufficient regulatory response to the responsibility gap, or does it create new gaps between the responsibility of providers and deployers?

The concept in detail

Broken epistemic condition

Classical moral responsibility requires that the agent be in a position to foresee and control the consequences of their acts. Learning systems (neural networks, genetic algorithms, agent architectures) modify their behaviour through experience in ways that are unpredictable for their designer, invalidating this condition. This is the heart of Matthias's argument (2004).

Sparrow's trilemma

Faced with harm caused by an autonomous system, the three natural candidates for responsibility — (1) the designer/programmer, (2) the operator/commander, (3) the system itself — each fail to satisfy the conditions of responsibility: the designer did not will the specific action; the operator has ceded control; the machine cannot be punished in any morally relevant way. The result is a structural void of responsibility.

The problem of many hands

A technological reformulation of the ‘problem of many hands’ identified by Thompson (1980) in an organisational context: individually harmless or morally neutral causal contributions from multiple actors (engineers, managers, users, policymakers) combine to produce harm without any one of them bearing responsibility for the whole. The complexity of sociotechnical systems distributes causality to the point of making it evaporate.

The active responsibility gap

The prospective dimension of the gap: beyond the retrospective imputation of blame, agents also lack the cognitive resources, the motivation or the authority needed to discharge their positive obligations of oversight, correction and anticipation of the system's behaviours. A concept developed by Santoni de Sio and Mecacci (2020, 2021).

Algorithmic opacity

‘Black box’ systems (deep learning, LLMs) make it impossible for users, clinicians or judges to understand the causal chain leading to a specific decision. This opacity deprives human agents of the capacity to explain, contest or correct decisions, creating an explanatory gap within the broader responsibility gap.

Automation bias

A cognitive phenomenon whereby human operators tend to over-grant their trust to automated recommendations, reducing their exercise of independent judgement. This bias erodes the ‘meaningful control’ without which responsibility cannot be attributed to the human ‘in the loop’. The operator becomes nominally responsible for a decision that they have, in practice, neither assessed nor controlled.