In Nature last year, Stuart Russell said the “meaning of meaningful” in the phrase “meaningful human control” was “still to be determined”.

I see five key points for “meaningful” human control of LAWS: the policy loop, Article 36 review, activation, the firing loop and deactivation.


FIGURE 1: Opportunities for “meaningful” human control.

  1. Policy loop. What rules does the LAWS follow when it targets? Who or what (human or machine) initiates, reviews and approves the rules of targeting and engagement? Control the policy the LAWS executes and you control the LAWS. If the LAWS is a Turing machine, it cannot disobey its rule book.
  2. Article 36 Review. People testing the policy control works and is reliable and predictable is a form of control.
  3. Activation. Turn the LAWS on. If a human decides to activate, knowing what policy the LAWS follows and being able to foresee the consequences, then this is a form of control.
  4. Firing loop. Having a human “in” or “on” the firing loop to confirm or supervise the LAWS firing decisions in real time is a form of control.
  5. Deactivation. Being able to turn the LAWS off or recall it, if it mistargets (the consequences are not as expected at activation) is a form of control.

How much of the above and exactly what variants makes up “meaningful” as distinct from “meaningless” I leave to CCW delegates to figure out.

Firing Loop

Most existing debate centres on the “firing loop” that covers the select and engage functions of a LAWS.

Label Select Confirm/Abort Engage Example
Remote Control Human Human confirms Human Telepiloted Predator B-1
Human “in the loop” Robot Human must confirm Robot Patriot
Human “on the loop” Robot Robot confirms Human can abort Robot Phalanx once activated
Human “off the loop” Robot Robot confirms
Human cannot abort
Robot Anti-tank and naval mines

TABLE 1: Firing loop – Standard “in, on and off the loop” distinctions

Policy Loop

There is relatively little definitional effort going into the “policy loop” that determines the rules the LAWS uses when it identifies and selects targets to engage. Control the rule book, control the Turing machine.

While the “firing loop” is about the execution of policy, the “policy loop” is about the definition of policy.

Label Targeting Rule Initiation Targeting Rule Review Targeting Rule Authorization Example
Human Policy Control Human Human Human Mines
Arkin (2009)
Sea Hunter?
Human “in the policy loop” AI Human Human ?
Human “on the policy loop” AI AI AI authorizes

Human can reject

Human “off the policy loop” AI AI AI authorizes

Human cannot reject


NorMAS blueprint

TABLE 2: Policy Loop – “in, on and off the loop” distinctions

There is a clear case for insisting on humans either having direct policy control (i.e. humans initiate, review and approve lethal policy) keyed into LAWS or having humans review and approve lethal policy devised by AIs.

A ban proposal couched at this level might actually get up.

Engineers following best practice today cannot even circulate a requirements specification without submitting it to ISO 9001 versioning, review and approval processes. We don’t let humans initiate and execute policy without review and approval. There is no case for letting AIs skip review and approval of their policy inventions either.

Trying to get Arkin-type LAWS banned strikes me as a lost cause. NATO opposition is firming up (US, UK, France) are firmly in the “no ban” camp either through words (UK, France) or deeds (US just launched Sea Hunter and is pushing with development of LRASM).

The distinction between “offensive” and “defensive” weapons made in the AI and Robotics open letter is hopeless. Lagatha and Ragnar Lothbrok routinely belt Saxons with their “defensive” shields in Vikings…  Aggressively sail an Aegis ship into enemy waters and it will “defend” itself against manned (or unmanned) air and sea attack by attacking the incoming enemy objects with explosive projectiles (if a sailor hits the activate button).

However, there is still a chance that a ban on something like AlphaGo (a “deep reinforcement learning” war fighting AI) having direct control of lethal actuators with real time lethal policy development on the fly (with no human review or approval) might get up.