Phone case design bot goes rogue, offering thousands of unsuitable designs
A bot that designed iPhone cases printed with random stock photographs for sale on Amazon appeared to malfunction, offering thousands of tacky and inappropriate designs. Based on popular image searches, the bot appeared to malfunction, creating 31,000 phone cases, many of which were unsuitable for sale. Per The Guardian , these included an iPhone 6 case with a picture of 'Male hands with soap dispenser use in the restroom', another featuring an 'Irrigation pipe in dirt trenches for sprinkler system' and ... one illustrated with an 'adult diaper worn by an old man with a crutch'. Another case featured a photo of a person preparing heroin for injection. It is unclear why the bot's algorithm went awry, though commentators figured it may hav e been due to a lack of oversight, or proper filtering mechanisms. It appears the bot had been programmed to scrape images from the internet without adequate checks on the appropriateness or relevance of the content. The creator did not advise potential buyers that the designs were generated using artificial intelligence. It also remains unclear who created the bot and Amazon account. System 🤖 Unknown Developer: My Handy Designs Country: USA Sector: Consumer goods Purpose: Develop creative designs Technology: Bot/intelligent agent; Machine learning Issue: Accountability; Accuracy/reliability; Transparency
- Date it happened
- 2017-07-01
- Organisation involved
- My Handy Designs; Amazon
- Product, system or model
- My Handy Designs
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.