Get In Touch

Developer Liability and Product Safety Regimes: The Application of Product Liability, Strict Liability, and Regulatory Certification to Lethal AI Systems

Developer Liability and Product Safety Regimes: The Application of Product Liability, Strict Liability, and Regulatory Certification to Lethal AI Systems

Introduction

Introduction of AI in military technologies has revolutionized warfare today. Most notable of the developments is the emergence of Lethal Autonomous Weapon Systems, otherwise known as “killer robots.” Such technology allows for identification, selection, and engagement of targets in different levels of human intervention. Unlike traditional weapons, autonomous weapons rely on sophisticated computer algorithms and machine-learning capabilities. Even though these innovations bring about significant benefits for military operations, including increased efficiency, precision, and low risk of human casualties, at the same time, these developments entail serious legal and ethical considerations regarding responsibility for the damage resulting from autonomous decision-making.

In connection with liability of the designers and manufacturers of autonomous weapons, there appears the question of adaptation of existing principles of product safety and product liability to lethal AI systems. The legal rules concerning safety and liability for injuries have been established long ago and were applicable only to ordinary products. As far as lethal AI is concerned, its autonomous nature and adaptability to changing circumstances complicates identification of the person who bears responsibility in case when autonomous weapon causes unlawful harm. Therefore, product liability, strict liability, and regulatory certification of lethal AI may need to be reconsidered.

In particular, this topic is especially significant because accountability is one of the essential principles of domestic legislation as well as international humanitarian law. In case autonomous weapons can make independent decisions about who should be targeted, it is extremely important to prevent the emergence of the so-called “accountability gap” and avoid the disappearance of legal accountability. Thus, a sufficient legal framework of developer liability and product regulation is required.

Product Liability and Lethal AI Systems

Product liability is a legal discipline which places the responsibility for the damages that occur due to the use of faulty products on their manufacturer and seller. In the majority of jurisdictions, there are three main grounds for placing the responsibility for the injury or death of a person – design defect, manufacturing defect, and lack of adequate instructions and warnings concerning the risk associated with the product. The idea of product liability lies in the fact that developers always have more information about the risks of their products.

The application of this principle to lethal AI systems seems quite obvious at first glance. If an AI system regularly misclassifies civilians as combatants due to biased data or erroneous programming, those problems might be considered design flaws.

The use of established rules of product liability in connection with deadly AI systems raises a number of difficulties, however. Unlike regular goods, autonomous systems can adapt to the environment and change their behavior based on new information. In the case of machine learning systems, the consequences might be something the developer did not predict or program for. Thus, it is very challenging to prove that a certain injury is a consequence of a “defect.”

The collaborative aspect of AI development is another difficulty that needs to be taken into account. Modern autonomous weapons systems are hardly ever developed by a single entity. They require software developers, data scientists, hardware manufacturers, military contractors, and government agencies. In such a situation, identifying the party responsible for the harm becomes problematic. A software developer will claim the damage is a result of wrong deployment by the military, and vice versa.

Additionally, product liability typically presumes the intended civilian use of the product in question and the accidental nature of damage. In the case of lethal autonomous weapons, the products in question were explicitly developed to cause physical damage or death. Hence, conventional product liability doctrines will have to undergo drastic adjustments in order to regulate military AI systems.

Product liability, however, still represents an essential tool of regulation. Imposing liability on the producers of faulty AI products serves as an incentive for the development of safer algorithms, their rigorous testing, and constant monitoring of their efficiency. Moreover, holding the producers of an AI product liable guarantees that no victims of wrongful harm will suffer due to the existence of autonomous machines.

Strict Liability

Due to the weaknesses inherent in conventional product liability principles, many researchers support the application of the doctrine of strict liability to lethal AI systems. Strict liability means that an entity is held accountable for harm without establishing fault or negligence. The doctrine of strict liability is usually applied in cases of activities regarded as being inherently dangerous.

Lethal autonomous weapon systems belong to this category.

Independent autonomous systems able to inflict deadly harm pose dangers which differ significantly from those posed by regular products. Malfunctions, hacks, manipulations or unexpected behavior in a complex environment may lead to dire outcomes, such as unintended civilian deaths and breaches of international humanitarian law.

This means that the strict liability principle would guarantee the existence of responsibility even in the absence of negligence. This is especially important since the complexity of AI-based autonomous systems sometimes prevents the identification of a human mistake at all. If victims must prove negligence on the part of the designers and producers of such weapons, it may become impossible to obtain any compensation owing to the opaque and very complex nature of machine-learning algorithms.

The strict liability regime encourages the behavior of deterrence. In order to minimize the risk of liability, manufacturers and developers will be encouraged to implement appropriate safety protocols, thorough testing, and other measures aimed at preventing harm. They will also see no alternative to developing explainable AI and protecting cybersecurity.

Nevertheless, the implementation of the strict liability principle does face some criticism. Unlimited liability imposed on developers may strongly disincentivize the development of beneficial AI technologies.

Autonomous weapons may be used to decrease casualties on the battlegrounds, enhance precision targeting, and shield the military forces from injury. Legal risks will thus act as an obstacle to the development of valuable technologies.

Moreover, there might be the situation where autonomous weapons will demonstrate unpredictable behavior even after a thorough testing process and meeting all the safety requirements. It would be considered to be very unjust to make the developers responsible for every unexpected event, especially when the injury was caused by the misuse of the technology by military forces.

Thus, some balance must be made. Strict liability does not have to impose endless responsibility but rather should distribute the risk among all parties associated with the development and use of autonomous weapons. The liability can be distributed between the developers, manufacturers, operators, and states according to their contribution to the undesirable result.

Regulatory Certification and Product Safety Systems

Apart from civil liability systems, another important element of a regulatory system for lethal AI technologies is regulatory certification. All industries dealing with the development of high-risk technologies are heavily regulated before the products can go to market.

Testing and certification processes are essential in ensuring that the products used in the aviation, pharmaceutical, and nuclear industries meet the safety standards set out. Similarly, certification processes are also being suggested in the case of autonomous weapons. These processes will involve proving by the developers that the systems comply with predetermined safety and legal standards prior to their deployment. The standards may include reliability, predictability, transparency, resilience to cybersecurity threats, and conformity with international humanitarian law.

Certification processes would also involve proving that autonomous systems are able to distinguish between combatants and non-combatants with a high level of accuracy and that they operate reliably in varying battlefield conditions. Systems would be tested under realistic circumstances to prove that they comply with certain criteria of safety and legality.

Furthermore, certification programs should not only cover testing done prior to deployment. AI technology is dynamic, and there might be a need for modifications after deployment. It is important, therefore, to ensure ongoing testing and certification.

Requirements for meaningful human control can also be imposed on regulatory certification. Several countries and international bodies have stressed the necessity to keep decisions regarding the application of lethal force under the control of humans. As such, certification requirements can include the introduction of human override functions, review procedures, and other means of emergency termination of operations.

In addition, regulatory certification should not serve as a full defence against developer liability. In case developers violate safety rules even after certification of the weapon, they still need to be held accountable regardless of certification status.

International Humanitarian Law and State Responsibility

Developer liability cannot be considered without referring to the principles of international humanitarian law (IHL). According to IHL, states have the duty to ensure that weapons used in armed conflicts meet certain legal obligations including distinction, proportionality, and precaution principles.

Under Article 36 of Additional Protocol I to  Geneva Conventions, states have the duty to examine every new weapon and establish if its usage will be forbidden by international laws. Autonomous weapons bring the issue into play since unpredictability of AI systems might hinder compliance with international rules.

Nevertheless, the question of state responsibility must not overshadow the question of individual liability. With the participation of private defense firms and technology companies in the production of these weapons, there exists a need to acknowledge the liability of individuals in the development of autonomous weapons. The failure to hold individuals liable for the production of such weaponry will only contribute to accountability gaps.

Thus, a combination of both state and individual responsibility must be established. States have to be held accountable for violations in the use of the said weapon systems while individuals such as developers and manufacturers of the weapons should be responsible for defects and lack of compliance with safety requirements. This will help to distribute responsibility among all parties involved in the creation of such weapons.

Conclusion

The emergence of lethal autonomous weapon systems poses a great challenge to the current legal regime on liability for defects in products. Product liability rules offer an excellent starting point but are ineffective in addressing the issues surrounding autonomous weapons due to their autonomous and adaptive nature. The sophistication of the machine learning systems makes it hard to establish defects and assign responsibility for harmful actions. Strict liability is an equally attractive concept, as it enables accountability regardless of the inability to prove negligence. At the same time, innovations and issues of fairness demand a moderate approach to liability, in order not to undermine both of them. Moreover, regulatory certification systems can also serve as a tool of accountability by requiring ex ante safety compliance with technical and legal standards.

Therefore, the system of governance of lethal autonomous weapons should combine the features of product liability and strict liability with regulatory certification. In doing so, it is possible to minimize the likelihood of the accountability gap and ensure legal liability of developers of such devices for the consequences of their use. As AI technologies are gradually changing the face of modern warfare, a coherent system of liability and safety measures becomes a necessity for ensuring the preservation of human dignity and protection of civilians.

 

 

 

Bibliography

Treaties and International Instruments

  • Additional Protocol I to the Geneva Conventions of 12 August 1949, and Relating to the Protection of Victims of International Armed Conflicts, June 8, 1977, 1125 U.N.T.S. 3.
  • Convention on Certain Conventional Weapons, Oct. 10, 1980, 1342 U.N.T.S. 137.
  • Protocol on Prohibitions or Restrictions on the Use of Mines, Booby-Traps and Other Devices (Protocol II), May 3, 1996, 2048 U.N.T.S. 93.
  • United Nations Charter arts. 2(4), 51, June 26, 1945, 1 U.N.T.S. XVI.

International Reports and Official Documents

  • Int’l Comm. of the Red Cross, Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons (2016).
  • Int’l Comm. of the Red Cross, Ethics and Autonomous Weapon Systems: An Ethical Basis for Human Control? (2018).
  • N. Inst. for Disarmament Research, The Weaponization of Increasingly Autonomous Technologies: Considering Ethics and Social Values (2015).
  • N. Inst. for Disarmament Research, The Human Element in Decisions About the Use of Force (2019).
  • N. Secretary-General, Agenda for Disarmament: Securing Our Common Future, U.N. Doc. A/73/460 (2018).
  • Group of Governmental Experts on Emerging Technologies in the Area of Lethal Autonomous Weapons Systems, Report of the 2019 Session, U.N. Doc. CCW/GGE.1/2019/3 (2019).
  • Group of Governmental Experts on Emerging Technologies in the Area of Lethal Autonomous Weapons Systems, Report of the 2021 Session, U.N. Doc. CCW/GGE.1/2021/3 (2021).

 

Books

  • Gary E. Marchant et al., The Growing Gap Between Emerging Technologies and Legal-Ethical Oversight (2011).
  • Page Keeton et al., Prosser and Keeton on the Law of Torts 692–771 (5th ed. 1984).
  • Richard A. Epstein, Cases and Materials on Torts (11th ed. 2016).
  • Mark A. Lemley & Bryan Casey, Remedies for Robots (2020).
  • Yoram Dinstein, The Conduct of Hostilities Under the Law of International Armed Conflict (4th ed. 2022).
  • Marco Sassòli, International Humanitarian Law: Rules, Controversies, and Solutions to Problems Arising in Warfare (2d ed. 2019).
  • Michael N. Schmitt, Essays on Law and War at the Fault Lines (2012).

Journal Articles

  • Ryan Calo, Robotics and the Lessons of Cyberlaw, 103 Calif. L. Rev. 513 (2015).
  • Gabriel Hallevy, Liability for Crimes Involving Artificial Intelligence Systems, 21 Buff. J. Sci. & Tech. L. 1 (2015).
  • Micaela McMurrough, Killing by Algorithm: The Rise of Autonomous Weapons and the Implications for International Humanitarian Law, 27 J.L. Info. & Sci. 115 (2018).
  • Rebecca Crootof, The Killer Robots Are Here: Legal and Policy Implications, 36 Cardozo L. Rev. 1837 (2015).
  • Matthew U. Scherer, Regulating Artificial Intelligence Systems: Risks, Challenges, Competencies, and Strategies, 29 Harv. J.L. & Tech. 353 (2016).
  • Thomas Burri, International Law and Artificial Intelligence, 60 Ger. Y.B. Int’l L. 91 (2017).
  • Andreas Matthias, The Responsibility Gap: Ascribing Responsibility for the Actions of Learning Automata, 6 Ethics & Info. Tech. 175 (2004).
  • N. Schmitt & Jeffrey S. Thurnher, “Out of the Loop”: Autonomous Weapon Systems and the Law of Armed Conflict, 4 Harv. Nat’l Sec. J. 231 (2013).
  • Noah Goodall, Machine Ethics and Automated Vehicles, in Road Vehicle Automation 93 (2014).

Reports and Policy Papers

  • Human Rights Watch & Harvard Law School Int’l Human Rights Clinic, Losing Humanity: The Case Against Killer Robots (2012).
  • Human Rights Watch & Harvard Law School Int’l Human Rights Clinic, Heed the Call: A Moral and Legal Imperative to Ban Killer Robots (2018).
  • Stockholm Int’l Peace Research Inst., Artificial Intelligence, Strategic Stability and Nuclear Risk (2019).
  • Nicolas J. Davies, Artificial Intelligence and Accountability in Armed Conflict (Chatham House Research Paper, 2021).
  • European Parliament, Artificial Intelligence in Weapon Systems (2020).

Online Sources

 

 

 

 

Mansi Mishra
Author: Mansi Mishra

Worked with Little Akshar Pre-Primary School Undergraduate law student at GGSIPU, Delhi (BBA LLB)