AIs, Honesty, Murder and Trolleys

[vicky_molokh]

Greetings, all!

In an investigation, we're trying to indirectly figure out whether a SAI has been made rogue (i.e. had its Honesty removed), or merely somewhat corrupted and coerced. Long story short, the AI has been 100% convinced that if it doesn't destroy a certain space station, millions of people will die through indirect consequences. The AI did try to destroy the (smaller, 6k-people-ish) station, but failed. The instance that committed the act is not available anymore. Was it a Rogue, or merely misguided?

Originally Posted by B139, on Honesty

You [b]may[/b[ fight (or even start a fight,
if you do it in a legal way). You may
even kill in a legal duel or in self-
defense – but you may never [b]murder[/b[.

If you're not a LEO, not a military person etc., does premeditated mass killing count as murder if the alternative is believed to cause more deaths?

According to the infallible font of Internet knowledge, murder requires malice aforethought, and the four states of mind recognized as constituting "malice" are:

• Intent to kill,
• Intent to inflict grievous bodily harm short of death,
• Reckless indifference to an unjustifiably high risk to human life (sometimes described as an "abandoned and malignant heart"), or
• Intent to commit a dangerous felony (the "felony-murder" doctrine).

Does letting millions die through your inaction count as unjustifiably high risk, thus making inaction count as murder and thus forcing the AI to choose between two different variants of mass murder?

Thanks in advance!

[whswhs]

It depends. Really, I think that's all you can say. It's the sort of question you might assign to a moot court in 2100 to see what they made of it.

Bill Stoddard

[Flyndaran]

I think national governments reserve the right to willingly kill innocents for any reason. For citizens, it's always murder, sometimes with mitigating circumstances, but murder none the less.

[Anthony]

Originally Posted by Flyndaran

I think national governments reserve the right to willingly kill innocents for any reason.

Eh, it's more a case of who exactly is going to enforce it.

Originally Posted by Flyndaran

For citizens, it's always murder, sometimes with mitigating circumstances, but murder none the less.

Nah, there's such a thing as an accident.

[Flyndaran]

Originally Posted by Anthony

Eh, it's more a case of who exactly is going to enforce it.

Nah, there's such a thing as an accident.

The right of nation includes the very concept of war, which most people approve of at least in theory.

I included willfully as a qualifier. Even then there is negligent homicide for egregious "accidents" like drunk driving, and malpractice.

[jeff_wilson]

Originally Posted by vicky_molokh

If you're not a LEO, not a military person etc., does premeditated mass killing count as murder if the alternative is believed to cause more deaths?

In US jurisprudence the answer is yes. LEOs are taught that homicide is not justified by the threat of death to people unrelated to the crime, or to the person committing homicide. "Kill X or we'll kill you" is not justified, nor is "kill X or Y dies".

If the person's proximate cause of death is not due to your intervention, but your lack of intervention (you chose to give your spare oxygen bottle to B rather than A, and A runs out and dies), that's different.

Originally Posted by vicky_molokh

Does letting millions die through your inaction count as unjustifiably high risk, thus making inaction count as murder and thus forcing the AI to choose between two different variants of mass murder?

Depends on previous commitments; if the SAI was tasked with guarding the safety of the millions but let them die from something that doesn't involve criminal acts by other people, the SAI will be on the hook for some kind of homicide, from negligent homicide / manslaughter on up.

[Pomphis]

Originally Posted by jeff_wilson

In US jurisprudence the answer is yes. LEOs are taught that homicide is not justified by the threat of death to people unrelated to the crime, or to the person committing homicide. "Kill X or we'll kill you" is not justified, nor is "kill X or Y dies".

Basically the same in germany, though it´s quite possible that killing X would be rated as manslaughter, not murder.

[Flyndaran]

Originally Posted by Pomphis

Basically the same in germany, though it´s quite possible that killing X would be rated as manslaughter, not murder.

Every jurisdiction puts mitigating and exacerbating circumstances onto the specific charges.

The efficacy of memetic influence would likely affect how such laws are practices. I'm not sure which way they would go though.
More lenient as anyone could be manipulated more easily.
Or stricter as everyone knows that everything but absolute facts are just loads of BS and shouldn't be trusted implicitly.

[dchart]

Originally Posted by jeff_wilson

In US jurisprudence the answer is yes. LEOs are taught that homicide is not justified by the threat of death to people unrelated to the crime, or to the person committing homicide. "Kill X or we'll kill you" is not justified, nor is "kill X or Y dies".

I think this shows that the initial situation is under-described.

How did the indirect consequences work?

"Blow up this station or we kill millions": That's murder in the jurisdictions I know, and everyone on the thread seems to agree.

"This station will crash into New York in eight hours and there is no way to change its orbit while preserving its structural integrity (because PHYSICS!)": Not clear to me. The AI's aim is to divert the station and save NY. The AI knows that it is inevitable that everyone on the station will die as a result. This could easily fail to be murder.

Honesty is a psychological disadvantage, so it is obviously based on your knowledge of the law; it doesn't give you the equivalent of Local Law-infinite. An AI can have the actual laws built in, but all laws have grey areas, and an AI can't know how actual courts would interpret them. In cases like this, there is no way to "err on the side of caution", because there are millions of lives at stake. This could well be a case where the GURPS Basic Set fails to solve important problems in philosophy and jurisprudence in its description of a Disadvantage, so you can't actually use the situation to decide whether the AI still has the Disad.

[jeff_wilson]

Originally Posted by dchart

Honesty is a psychological disadvantage, so it is obviously based on your knowledge of the law; it doesn't give you the equivalent of Local Law-infinite. An AI can have the actual laws built in, but all laws have grey areas, and an AI can't know how actual courts would interpret them.

It doesn't have to actually know, it just has to have a function that prevents it from doing things that are obviously against criminal law, and one that estimates if it risks civil liability beyond its applicable insurance coverage or other limit set by the manufacturer or owner. The law requires it be made to obey the law, so as long as it is done in order not to transgress, it's failure to act is legally excusable.

There can certainly be circumstances where laws conflict (this is actually a studied problem in computer science called deadlock), and so it can pick the lesser of two liabilities, or shutdown entirely, or say, "Norman, coordinate" as the manufacturer decides.