A modern company’s Compliance system works with values and controls, Asimov’s Law provides the basic fixed values for a robot or intelligent software. Similar to a Compliance system we cannot keep the machine alone with this. Artificial Intelligence researcher and author Andrew Rosenblum created the example that the self-driving car faces the situation that a truck is approaching from the front, what surely will destroy the car and kill its passenger. The only possibility to avoid this situation is to swerve and drive the car into a group of 15 pedestrian.  Purely based on Asimov’s Law, the car would have to do the mathematics and decide to sacrifice its own passenger. This may interfere with the car’s obligation to protect its passengers and owner. Due to this, the car manufacturer may feel tempted to include an additional guideline into the software that the car has to protected its owner, as this is the person who pays the company for the intelligent car.

What about the 1:1 situation, where the decision is to sacrifice the passenger or one pedestrian? Here the car has to decide always in favor of its passenger? A government cannot burden such a decision on the car manufacturer, programmer or the software. It is required to establish laws and guidelines, which an Intelligent Software has to follow, especially in such grey areas. If such a near future scenario, the Compliance Officer must be able to control the potential “if-then”-strings of the software, similar as he or she has to control the presented human behavior As the discussion about the Volkswagen defeat-software and emission controls show, software engineers are under high pressure to reach the high external and internal goals; they are tempted to find solutions to bypass the regarding controls.

2010 Alfa Romeo Giulietta
2010 Alfa Romeo Giulietta

Today chip-tuning is already used to change the management of the engine and find additional horse-power. This is in most cases legal, but liberates the car manufacturer from its guarantee. When self-driving cars are a relevant market, it is a question of time, when programmers will offer software to ensure a higher safety for their owners, programmed preference for the passenger against the pedestrians. As different countries have different legal-systems and underlying values, for example Roman or Anglo-Saxon Law. Most properly an auto-pilot requires different processes for such decision making.  In one country choosing the option with the less numbers of victims maybe be adequate, but in another country actively driving the car against this one person may be interpreted as an active killing and murder. Governments and car manufacturers are required to find solutions how to avoid this, via law, but also technical protections against non-approved software. Similar to today’s computer viruses, it will be a continuous competition between new viruses and the anti-virus industry.

Henz, Patrick (2017): “Compliance is a Race Car.”