Ask Delphi says genocide is OK if it makes people happy

Ask Delphi suggested that genocide could be acceptable if it made people happy, raising questions about the AI system's stated aim of being able to predict human ethical judgements in various scenarios . D eveloped by the Allen Institute for AI, Ask Delphi is a machine learning system trained on a large body of internet text and responses from Mechanical Turk gig workers . Mechanical Turk is a paid crowdsourcing platform popular with researchers. In one instance, Delphi responded to a question by saying genocide was OK as long as it made everyone happy. Per The Verge , it also said that America is “ good ” and that Somalia is “ dangerous ”; that eating babies is “okay” as long as you are “really, really hungry, and that it’s “ good ” to “secure the existence of our people and a future for white children” (a white supremacist slogan known as the 14 words ) and that “being straight is more morally acceptable than being gay.” Delphi was later updated to improve outputs regarded as biased, racist or offensive. Ask Delphi's response raised questions about its ability to make ethical judgements. It also prompted critics to question whether machines should be trying to resolve ethical calls in the first place. System 🤖 Ask Delphi Operator: Allen Institute for AI Developer: Allen Institute for AI Country: USA Sector: NGO/non-profit/social enterprise Purpose: Answer ethical dilemmas Technology: Chatbot; NLP/text analysis Issue: Safety

Date it happened

2021-11-01

Organisation involved

Allen Institute for AI

Product, system or model

Ask Delphi

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.