Follow Facebook MP-IDSA
Follow Page
Follow Linkdin MP-IDSA
Follow Page
Follow X MP-IDSA
Follow Page
Subscribe Youtube MP-IDSA
subscribe

Lethal Autonomous Weapon Systems and the Legal Regime

Abhimanyu Singh
Lieutenant Commander Abhimanyu Singh is working as Deputy Judge Advocate (Naval Operations) at Integrated Headquarters Ministry of Defence (Navy), Delhi. read more
Volume:15
Issue:2
Focus

Present-day world order marks a new dawn in the field of international law. The unusual pace and nature of technological advancements has resulted in the creation of a world where problem solving is leading to the creation of more complex problems. Development and deployment of lethal autonomous weapon systems (LAWS), on land, air, sea and space, generally gains momentum as a force multiplier. Military advantages are huge, but there are calls for ban on such systems from international humanitarian law (IHL) activists, which would be wishful thinking and an effort to snub the destined eventuality of future’s technologically advanced weapon systems. The wiser option is to regulate this contingency before it snowballs, with disastrous outcome. The experts have just scratched the surface of the grave challenges posed by the operation of LAWS in the field of IHL. The issue is similar for LAWS in the maritime domain from a naval operational viewpoint. The general area of operations for warships is usually beyond territorial waters and within international domain. Complexities in application of maritime laws are further aggravated by operation of LAWS in maritime domain. Thus, seeking accountability in case of commission of unlawful acts alongside or in addition to the conventional weapons with LAWS is a vexed question in maritime or any other domain.

INTRODUCTION

The operation of lethal autonomous weapon systems (LAWS) in present-day world order has brought in complexities in the application of existing set of rules concerning international humanitarian laws (IHLs), such that the state of rules is like that which existed in pre- to mid-1800s, namely, non-existent. Robots and unmanned systems have been incredibly valuable on the battlefield during recent wars, including in Afghanistan and Iraq, and are likely to play a larger and more sophisticated role for militaries in the future. From 2000 onwards, the use of unmanned aerial vehicles (UAVs) has proliferated, with similar astounding increase among land and sea-based unmanned systems. In future, all branches of military are poised to rely more heavily on unmanned systems,1 also known as LAWS. The LAWS are autonomous military robots capable of searching and engaging targets in air, water or land, as per specifically programmed military needs. The potential deployment of LAWS raises significant legal and ethical concerns. These profound concerns in maritime domain, including whether such systems would even conform with the law of armed conflict (LOAC) and United Nations Convention on the Law of the Sea (UNCLOS), have not yet been definitively resolved. The technology continues to race forward regardless; however, a need is felt for operational commanders to examine the command-and-control implications, in consonance with the legal implications, of using such LAWS. This would encourage the future development and doctrine of unmanned systems, as also steer it on to the correct path.2

Naval application of LAWS in the maritime domain is further complicated. Historically, in the maritime domain, the practice of freedom of navigation was followed by mariners universally as per the applicable maritime laws and traditions. However, there was an absence of a legal doctrine concerning the status of sea, or a legal code about the rights of men. As regards naval operations, for most of the nineteenth century, sailors and diplomats in distant waters had no promising means of consulting with the sovereign and were practising and shaping international law with a handicap. At times, commanders had to combine naval force with diplomacy in dealing with the various issues. Later, with the advent of worldwide communication and technology, a naval officer’s wide latitude to determine foreign policy declined, but not necessarily his ability to affect war and peace in crisis situations at sea. This partnership has continued to this day, evident in the variety of conferences and conventions that followed World War II to regulate the legal regime concerning merchant ships, as also the warships during peacetime. The most prominent of these are: Convention on the Territorial Sea and the Contiguous Zone (1958); Convention on the High Seas (1958); Convention on Fishing and Conservation of the Living Resources of High Seas (1958); Convention on the Continental Shelf (1958); and UNCLOS (1982).3 

The present-day technological advances, however, again pose a conundrum for a naval commander at sea. The likes of these issues only existed in the days when maritime practices were at a nascent stage and were being developed based on maritime traditions. The legal status of LAWS in the present day can be best described as unsettled. Although it does not pose an insurmountable problem in peacetime, it could prove to be a major impediment during war considering the LOAC.4 The LAWS would qualify as a means of warfare and though they may not be unlawful per se, the use will have to be in compliance with LOAC.

Further, maritime coastlines are vast and resources are limited, thus posing significant challenges in the protection of marine resources. To monitor and respond to marine peacetime activities, nations need maritime capability. In this regard, unmanned maritime systems (UMSs) are useful. From a naval standpoint, being an autonomous system, the UMSs installed with weapons do form part of the LAWS as they have the capability to operate autonomously. These UMSs enhance the capabilities of defence and coast guard authorities, and their assets, to execute multiple tasks by providing low-cost resources, capable of operating over large areas for extended periods.

The incident of the seizure of a United States (US) government-operated (USNS Bowditch) unmanned underwater vehicle (UUV), on 15 December 2016, by Chinese forces has highlighted the legal ambiguity surrounding the operations of such UMSs. The incident took place at about 50 nautical miles (nm) from the coast of the Philippines in the South China Sea (SCS) region.5 The legal basis for its actions were not clarified by the Chinese government, although statements referred to the ambiguity of the law surrounding the use and seizure of ‘drones’ as well as to repeated US ‘reconnaissance’ in Chinese waters as a basis for staking claim. Responding to this, the US government demanded the return of the device, which it stated had been ‘conducting routine operations in accordance with international law’ and which, it claimed, was a ‘sovereign immune vessel of the United States’.6 As regards the disputes arising out of the SCS, varied interpretations under international law have been applied by the People’s Republic of China (PRC) and the US. Accordingly, the seizure of a US Navy UUV by the PRC was also no exception. There is simply no plausible international legal basis for China’s actions and according to the US Department of Defense, the USNS Bowditch was conducting routine operations.7 

Although the positioning of the UUV was within the Philippines exclusive economic zone (EEZ), China could have argued that the UUV operated by USNS Bowditch was within the EEZ limit as claimed by it across Scarborough Shoal. Therefore, China could have invoked the violation of its Surveying and Mapping Law concerning marine scientific research in Chinese EEZ. There are two issues which merit attention with regard to this incident. First, the disputed Chinese jurisdictional claim over the area of recovery of UUV. This should be seen in the light of multinational claims over Scarborough Shoal and the fact that 2016 SCS Arbitral Tribunal award declared Scarborough Shoal as rock, entitling China only to a territorial water and not an EEZ. Second issue is with regard the status of UUV. Till date, UUVs have not been defined under any bilateral, multilateral or international treaty or convention, thus making the status of UUVs ambiguous. However, the US has always accorded it a ‘sovereign immune craft’ status, as mentioned in the US Naval Commanders Handbook (2017 edition). Be that as it may, jurisdictional point concerning the area of recovery aside, the status of vessel (UUV), due to lack of any recognised definition, has to be considered as per the national laws of the state operating it. Thereby, as the US presently claims it, in its own interpretation the UUV is to be considered as ‘sovereign immune craft’. Therefore, the Chinese action is further illegal as under Article 32 of UNCLOS, ‘“warships and other government ships operated for non-commercial purposes” retain their immunities under customary international law. This absolute sovereign immunity from seizure would be applicable even within China’s territorial waters.’8 Although the incident was resolved peacefully, the resolution is still surrounded by the legal issues relating to navigational rights and obligations of UMSs under the umbrella of the International Maritime Law. The possibility of use of UMSs has come of age and drawn widespread attention in the legal community with focus on its combat role.9 Use of UMSs for surveillance, intelligence and reconnaissance purposes is a preferred option of armed forces of many states.

The growing use of UMSs needs wider attention in the legal spectrum,10 although there is low-pitch use of such systems at present in the maritime operational domain in comparison to the air and ground assets. However, future security operations in the maritime/naval warfare domain are likely to increase the use of UMSs at a fast pace. The likelihood of these systems expanding peacetime monitoring capability of naval forces is an accepted reality. In the peacetime operations context, UMSs could be deployed for counter-piracy, counter-drug and refugee operations, marine biological survey missions, as also for countering proliferation of weapons of mass destruction. During wartime, operational deployments are particularly promising with respect to improving maritime battlespace awareness/transparency and enhancing anti-access/area denial capabilities. Also, UMSs are likely to prove themselves vital in maintaining the security of fragile sea lanes of communication, during both peacetime and periods of armed conflict, upon which global economic prosperity depends.11 What necessitates immediate deliberation is the present-day laws vis-à-vis UMSs activities during international conflicts and their rights and obligations.12  

UNCLOS AND LAWS

The LAWS have been named differently in their different areas of operation. They, as perceived in maritime domain, broadly engulf UMSs and take on a larger role. The terminology UMSs includes inter alia ships, warships and weapons systems installed onboard such ships/warships. The issue of the legal status of UMSs has been debated in various academic circles. What needs to be seen is the status of such UMS as a ship and in case the definition could be extended to include it as a warship. Both are complex and, as mentioned earlier, somewhat unsettled issues. The US position in this context is interesting. Although it is a non-party to the UNCLOS, it is of the view that many of the provisions of UNCLOS reflect customary international law.13 The 2017 Commander’s Handbook on the Law of Naval Operations mentions UMSs as:

Unmanned Surface Vehicles (USVs) are water craft that are either autonomous or remotely navigated and may be launched from the surface, subsurface, air or land. The anticipated stealth, mobility, flexibility of employment and network capabilities of USVs make them extremely valuable as force multipliers, particularly in the littoral environment. Missions envisioned for USVs include laying undersea sensor grids, antisubmarine warfare (ASW) prosecution, barrier operations, sustainment of carrier operating areas, mine countermeasures, intelligence, surveillance and reconnaissance, bottom mapping and survey, and special operations support.14

The UNCLOS, in Article 29, defines warship as:

For the purposes of this Convention, ‘warship’ means a ship belonging to the armed forces of a State bearing the external marks distinguishing such ships of its nationality, under the command of an officer duly commissioned by the government of the State and whose name appears in the appropriate service list or its equivalent and manned by a crew which is under regular armed forces discipline.15

This definition replicates the legal requirements set forth by 1907 Hague Convention (VII). Thus, to include UMSs under the ambit of Article 29 would be stretching the definition too far to interpret it as a warship.

If UMSs are qualified as warships, they could become a part of the defence forces and marked appropriately. However, the notion of command by a commissioned officer would have to be stretched to include operations undertaken remotely. Moreover, as these systems are unmanned (or remotely manned), the requirement of it to be manned by a crew which is under regular military discipline would not let it qualify as a warship. In time, these principles may get diluted. However, in the present-day scenario, even if UMSs qualify as ships, it may not be appropriate to declare them as warship. Warship status under UNCLOS affords certain rights designed to safeguard the maritime domain.16 Ships authorised and on-government service, as clearly marked and identifiable, are also granted few of these rights, thereby placing them on a similar pedestal as a warship.

No additional conditions are imposed under the UNCLOS, so there is no reason why a UMS cannot be duly authorised to exercise peacetime rights by a government as afforded to warships.17 Sovereign immunity of warships is recognised under UNCLOS regime. Similar rights and obligations are bestowed upon other vessels on government non-commercial service, thereby putting an embargo over the enforcement jurisdiction of other states over such ships.

As regards the LOAC, the same has evolved throughout history to respond to changes in the means and methods of warfare.18 War has evolved dramatically and the existing laws are ill-equipped to deal with it.19 International laws must evolve once more to cover these new challenges, so as to protect mankind in times of conflict. There, thus, exists a need to highlight the deficiencies in the current law and propose reforms to bring humanitarian law in line with the realities of modern warfare.20

During wartime, attack upon or by UMSs cannot be ruled out as they, in every sense of the meaning, meet the criteria for being a military target and thereby would be devoid of any sovereign immunity. The status of being a warship during an international armed conflict would entitle UMSs to exercise belligerent rights and, in that sense, they would be authorised to use force against enemy warships and take part in imposing effective naval blockade. Qualification as a warship authorises UMSs to take part in many of the wartime activities, despite the legal issues surrounding their status.

MODERN WARFARE AND ACCOUNTABILITY

The LAWS in other domains, like UMSs in maritime domain, are majorly devoid of human element. However, for the purpose of accountability, a human being is required in the chain of responsibility to enable enforcement of IHL. The vexed question is to seek accountability in case of an unlawful act due to replacement of conventional weapons with autonomous weapons. Thus, in this context, the accountability for operation of autonomous weapons may lie with the following.

Weapons Operators

In future weapons platforms, the human role envisaged is that of ‘human-in-the-loop’. Under human-in-the-loop mode, the system has the ability to autonomously suggest to the weapons operator the threat level, as also whether a target is to be engaged or not. Based on the suggestion of the artificial intelligence (AI)-based weapons system, the weapons operator has the final authority to engage or not to engage the target. The pace of technological advancements, however, may bring about an era of ‘human-out-of-the-loop’, wherein the AI-based weapons system may autonomously engage a target without human intervention. Therefore, for the purpose of accountability, as human-in-the-loop system envisages ‘veto control’ of a human being in the weapons platform, it is preferred. The fear in case of human-in-the-loop-system still persists due to the phenomenon of ‘automation bais’. Under this phenomenon, an operator, by virtue of his/her training, is inclined to exercise options as suggested by the AI system. Hence, though the system is operating on the basis of human-in-the-loop, there is no meaningful control being exercised by the operator. In such scenarios, the deficit of accountability has been qualified by many as more of a systemic than individual one.21

Computer Programmers

In case the weapon operators are not being held responsible for the actions of UMS due to system malfunction or software design flaws, a more sought-after option for fixing of accountability could shift to the engineers, computer programmers and designers responsible for creating the system in the first place.22 Such a liability structure would result in a paradigm shift from the earlier models adopted for accountability. The present system of near-zero liability being attributed to engineers, computer programmers and designers could be ascribed to the fact that all the weapons platforms are tested and inducted subject to clearance by respective national defence authorities. Thus, till date, the engineers, computer programmers and designers have been discharged of their liability upon clearance by competent national defence authorities. However, in the most likely scenario wherein future weapons systems would also have the ability to learn on their own, known as ‘machine learning’, the engineers, computer programmers and designers may be attributed liability for a product malfunction resulting in loss of life or otherwise. In an incident in March 2018 involving an Uber self-driving car, a crash led to the loss of a human life, further highlighting the importance of the issue of accountability.23 Though the incident led to fatality, it was legally settled out of court. However, the same cannot be the scenario in case of a malfunction by weapons system, as strict regulations seem to be the only way to ensure a framework with defined accountabilities, leading to a safer development of such systems.  

Operational Commander

Utilisation of LAWS, like any other weapons system in wartime, is expected to be as per the directions/instructions of the operational commander. To achieve victory in war, operational commanders plan to gain military advantage and in doing so, they are expected to be mindful of the collateral damage they might cause in achieving their goal. Accordingly, in future weapon systems like LAWS, it would be expected that the operational commander will set the threshold value of collateral damage to very low, so as to avoid destruction of civilian property and fatalities to the civilian populous. Provided that the operational commander has directed programming of autonomous weapon systems correctly, their use could be attributed to command responsibility as it concerns the responsibility of a commander for the people under his/her command. Due to the very nature of LAWS, wherein the ability to ‘identify and engage’—which was earlier being undertaken by a human—would now be undertaken by an AI-based system, the proposal for expanding the responsibility to operational commanders to encompass the actions undertaken by LAWS would be only prudent. As the limits on the LAWS during a conflict are expected to be within the domain of an operational commander, the responsibility also needs to be placed at higher levels for the purpose of accountability for the actions of LAWS.

WEAPONS LAW

Assessment of legality as regards weapons law vis-à-vis LAWS generally involves analysis of various aspects. Clear distinction of targets (legitimate or illegitimate) constitutes one of the major basis for assessment of legality for a weapons platform, in addition to its ability to inflict unnecessary sufferings. Furthermore, another scenario where the weapons law prohibits a weapon is in case a state possesses the weapon not permitted due to it being a party to a certain convention or agreement (bilateral/multilateral). Violation in any manner of the above-mentioned conditions constitutes a breach of the weapons law under the regime of IHL. Thus, the use of such a weapon would be in blatant violation of the principle of distinction as the inability to target specifically may lead to unwanted harm upon civilians and non-combatants, which is against intended use of any such weapons. The ability of a weapon to strike accurately, but with uncontrollable effects, is a contravention of principle of distinction, also known as principle of discrimination, and thus such a weapons system qualifies for a ban. The final aspect would be whether new weapons technology is prohibited under any agreement or treaty, thereby barring its use per se.

Presently, there is no international treaty that bans the use of LAWS. However, certain organisations have voiced their concerns and are constantly prompting governments to adopt a pre-emptive prohibition policy. The need for a human control/intervention to prevent human loss of life and breach of LOAC is the fundamental basis for such calls for pre-emptive prohibition policy. Lack of international consensus for a policy over LAWS necessitates their assessment under the present regime of weapons and targeting laws. It is felt that to abide by the weapons law, at the stage of manufacturing and designing itself, there is a requirement to put an embargo on the capability of operating weapons which would result in unnecessary suffering or superfluous injuries.

TARGETING LAW

Law of targeting regulates the use of any weapons system in warfare. As in case of conventional weapon systems, LAWS are also expected to adhere to the targeting laws. One of the facets of targeting laws, the principle of distinction, demands that a weapons system is able to distinguish between combatant and non-combatant as a legitimate military target. Article 52(2) of the Additional Protocol I describes targets as: ‘limited to those objects which by their nature, location, purpose or use make an effective contribution to military action and whose total or partial destruction, capture or neutralization, in the circumstances ruling at the time, offers a definite military advantage.’24 Thus, military necessity means that weapons must be configured to apply only the requisite force to achieve a legitimate military objective to defeat enemy forces. Proportionality is yet another important principle under law of targeting. It deals with the quantum of force to be applied to a target to achieve a legitimate military advantage, with the least amount of collateral damage. Finally, necessary precautions must be taken when operating weapons based on the principles of necessity, proportionality and distinction. This also involves taking precautions in avoiding, and in any event minimising, incidental loss of civilian life, injury to civilians and damage to civilian objects.

Under targeting law, there are no international treaties restricting certain legal uses of LAWS. It is presently opined that restriction on such systems could actually be a more viable alternative as opposed to a prohibition. Indeed, to persuade nations across the world for an all-out ban would be wishful thinking. Rather, a restrictive development (in compliance with IHL and LOAC rules) of such weapons technology would be preferred by the nations because having LAWS in their inventory is likely to change the outcome of war/conflict. As possessing LAWS would be relevant in projection of a state as a superpower in future—the way the nations projected in the past on possessing nuclear weapons—the states are only expected to outperform each other for possessing LAWS by any means possible. The principle of distinction places an obligation that the future weapons systems are capable of distinguishing between combatant and non-combatant. This principle also casts a duty upon LAWS for distinguishing a wounded combatant and not targeting him/her. Furthermore, as is often the case in modern warfare, the enemy does not always wear a uniform, making it difficult to determine who is a combatant. Military necessity, as it pertains to a weapons system, is intricately tied to distinction. The degree of expectations as well as responsibility on LAWS is very high as they are expected to pass the test under the principle of distinction. Under targeting laws, LAWS are expected to cause damage in proportion to the military advantage expected out of a target. Thus, for LAWS to be compliant of IHL, the way out is to prioritise the level of collateral damage expected out of the target.

The continued integration of technology into warfare means that the speed of combat is becoming faster, outpacing the ability of human beings to accurately react in a timely manner. The tempo of warfare in future is bound to be at an amplified rate than the present pace. Indeed, LAWS would be programmed and expected to perform at such a high tempo to be relevant in future warfare. Human-in-the-loop system may become irrelevant in future as the pace of warfare can only be expected to be matched by LAWS. Finally, such system’s decision-making programme could be more logical than its human operator’s. Therefore, by way of instilling restrictions on machine learning capability of such systems at the development stage itself, weapons could be programmed to be compliant of laws concerning targeting.

LAWS AND GROUP OF GOVERNMENTAL EXPERTS

The ambiguity surrounding legal principles concerning LAWS had been taken note of by the international community and therefore, in 2013 Convention on Certain Conventional Weapons (CCW) meeting, it was decided by the chairperson and high contracting parties to convene an informal meeting of Group of Governmental Experts (GGE) on the issues concerning LAWS. Towards this end, meetings have been held since 2014 to evaluate various aspects, such as human–machine interaction, human responsibility, risk assessment and mitigation circumstances, in relation to LAWS. 

Till 2019, the GGE on LAWS had addressed the above-stated issues and had arrived at the guiding principles for LAWS, wherein the use of potential technologies concerning LAWS is developed and conducted in accordance with present IHL norms. It was agreed upon by all that at the present juncture, the involvement of human judgement is essential for LAWS for compliance in accordance with IHL.25 However, till date, despite best efforts in this direction, no agreed-upon definition of LAWS has been formulated.

MARTENS CLAUSE

A Russian professor, Von Martens (1899 Russian delegation member to Hague Peace Conference), introduced a clause now named after him. Under Martens clause, which appeared in the Preamble of Hague Convention (II), 1899, it was mentioned that:

Until a more complete code of the laws of war is issued, the High Contracting Parties think it right to declare that in cases not included in the Regulations adopted by them, populations and belligerents remain under the protection and empire of the principles of international law, as they result from the usages established between civilized nations, from the laws of humanity and the requirements of the public conscience.

26

The Martens clause has been referred to and reproduced in different forms over the years so as to provide a legal basis for general statements of humanitarian protection. Various international declarations and judgements have referred to it for providing a basis for regulating issues of IHL, which were unregulated in the past. In terms of LAWS, the same would appear to be relevant and a sound basis for regulating development of such weapons till any legal basis for regulating them is agreed upon by nations.

THE WAY FORWARD

It is not war but technology that has shaped warfare since the beginning of mankind. Wartime operations have had the most telling impact on technology by making it constantly evolve during war. Military innovation, throughout history, has been both the primary source of technology and primary driving factor impacting the warfare.

The present-day warfare chivalry finds at its core a code of conduct that is held as the gold standard by the military elite. Jean Pictet, the most influential twentieth  century expert on the LOAC, noted that the institution of chivalry brought with it the recognition that in war, as in the game of chess, there should be rules and that one does not win by overturning the board. While a direct comparison between chess and warfare may well be somewhat away from reality, the underlying presumption that organised violence amounts to warfare only when it conforms to certain prescriptions is a fundamental aspect.27

Naval warfare and warships have been the most sophisticated technological tools of warfare in human history. Advances in technology may change warfare, but it can never determine warfare, neither how it will be conducted nor how it will turn out.28 Thus comes the need to regulate (like any other facet of human life) the facet of modern warfare with effective legal machinery to make the modern warfare IHL compliant. 

At higher academic and diplomatic levels, it needs to be examined how warfare has changed from that imagined by the drafters of the previous conventions and its impact in the present-day scenario. Advances in the modern warfare necessitate an urgent conceptual change in the laws concerning warfare as it may serve as a foundation for future legal reform.

Lieutenant Commander Abhimanyu Singh is working as Deputy Judge Advocate (Naval Operations) at Integrated Headquarters Ministry of Defence (Navy), Delhi.

Disclaimer: The views expressed and suggestions made in the article are solely of the author in his personal capacity and do not have any official endorsement.

Attachment
Download Complete [PDF]
  • Visitor No: 000058457
  • © Manohar Parikar Institute For Defence Studies and Analysis
  • XHTML validation2.0 Level Double-A Conformance
    CSS Validator
    Explanation of WCAG 2.0 Level Double-A Conformance