The record

Where automated decisions went wrong

Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.

Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.

Clear

92 incidents involving Amazon · search all text for “Amazon Web Services” instead · public reporting

WF-RHVPVR14 Feb 2018

Amazon Echo owner complains after TV ad triggers unwanted cat food order

An Amazon Echo Dot owner complained to the UK Advertising Standards Authority after a television advertisement for the device caused their Alexa assistant to place an unwanted order for cat food. Amazon confirmed the order was placed but cancelled by the customer, and explained that it marks ads to prevent such triggers and requires purchase confirmation. The ASA rejected the complaint, finding no breach of the advertising code.

Company involved
Amazon
AI system involved
Alexa

10 source articles · read the reporting →

WF-UCTXD92 Feb 2018

Amazon patents wristbands to track and guide warehouse workers' movements

Amazon received patents for a wristband designed to track warehouse employees' hand positions and use vibrations to guide them toward correct inventory bins. The technology, if implemented, could also enable workplace surveillance, alerting management to staff who are wasting time or taking long breaks. Amazon stated that the speculation is misguided and that the idea would improve the process for fulfillment associates by freeing their hands and eyes. Social media users compared the technology to devices from 'Black Mirror'.

Company involved
Amazon
AI system involved
Wristband with haptic feedback system

10 source articles · read the reporting →

WF-UUU4KQ18 Sep 2017

Amazon Algorithm Suggests Dangerous Item Combinations

Amazon's 'frequently bought together' algorithm reportedly suggested shoppers purchase items that could be combined to create dangerous reactions and explosives. Channel 4 News found that certain chemicals were recommended alongside other ingredients that could produce thermite or other hazardous substances. Amazon stated that all products comply with legal guidelines and that it cooperates with law enforcement. No actual harm was reported, but the potential hazard raised safety concerns.

Company involved
Amazon
AI system involved
Frequently bought together

2 source articles · read the reporting →

WF-1P7PLD1 Jan 2017

Amazon deploys Rekognition facial recognition for government surveillance

Amazon developed Rekognition, a facial recognition system, and is marketing it to law enforcement agencies. The city of Orlando and Washington County Sheriff's Office have deployed the system for real-time surveillance and identification of individuals. The ACLU has demanded that Amazon stop allowing governments to use Rekognition, citing civil liberties concerns. Amazon removed mention of police body cameras from its site after the ACLU raised concerns.

Company involved
Amazon
AI system involved
Rekognition

10 source articles · read the reporting →

WF-5MXLAD1 Jan 2016

Amazon used seller data to copy products and rig search results in India

Internal documents reveal that Amazon's India marketplace systematically used third-party sellers' non-public data to create copycat products and manipulated search results to favour its own private-label items. One seller reported a significant drop in sales after Amazon introduced a cheaper AmazonBasics version of a product. Amazon denies the allegations, calling them unsubstantiated. The practices have drawn antitrust scrutiny from US FTC Chair Lina Khan.

Company involved
Amazon

4 source articles · read the reporting →

WF-XV5P501 Oct 2015

Amazon Buy Box algorithm accused of overcharging UK consumers by £1 billion

Consumer champion Robert Hammond has filed a collective proceedings order application against Amazon in the UK Competition Appeal Tribunal. The claim alleges that Amazon's Buy Box algorithm manipulated search results to favour products more profitable for Amazon, obscuring cheaper alternatives and causing approximately 49.4 million UK consumers to overpay by more than £1 billion between October 2015 and June 2020. The algorithm determined the first-ranked item in the Buy Box, which consumers accepted over 80% of the time. The case is pending, and similar investigations by the Italian Competition Authority and the UK Competition and Markets Authority are ongoing.

Company involved
Amazon
AI system involved
Buy Box algorithm

5 source articles · read the reporting →

WF-P7UKM91 Jan 2015

Amazon recruiting algorithm showed gender bias, down-ranking women

Amazon developed an AI recruiting tool that was found in 2015 to penalise resumes containing the word 'women's' and to prefer male-associated language. The algorithm was trained on ten years of historical resumes, which were predominantly from men, causing it to replicate gender bias. Amazon corrected the bias in trials and ultimately scrapped the program because it did not return relevant candidates. The tool was never used to evaluate actual candidates.

Company involved
Amazon

10 source articles · read the reporting →

WF-Z7PLQV18 Apr 2011

Algorithmic Pricing Glitch Causes $24 Million Book Listing on Amazon

Two booksellers on Amazon, profnath and bordeebook, used algorithmic pricing to automatically adjust their prices relative to each other. Due to a lack of upper price limits, their algorithms entered a feedback loop, driving the price of a used copy of 'The Making of a Fly' to over $23 million. The error was discovered by a researcher and the price was later manually corrected by one of the sellers. No purchase was made at the inflated price.

Company involved
profnath and bordeebook
AI system involved
Amazon algorithmic pricing

2 source articles · read the reporting →

Is Your Voiceprint Protected? Amazon’s Biometric Privacy Battle in Illinois - FindLaw

The system collected and used voiceprints to verify callers' identities without their knowledge or consent.

Company involved
John Hancock
AI system involved
Amazon Connect with Pindrop

1 source article · read the reporting →

Amazon overcharged school districts and local governments using opaque pricing algorithms

Senator Elizabeth Warren pressed Amazon CEO over allegations that Amazon Business uses opaque pricing algorithms and misleading contracts to overcharge school districts and local governments. An investigation found that Amazon charged some buyers up to three times as much for identical products, such as a 12-pack of Sharpie markers. The pricing is algorithm-driven and dynamic, and Amazon's group purchasing contracts with OMNIA Partners sidestep traditional competitive bidding. The overcharging is ongoing and has not been resolved.

Company involved
Amazon
AI system involved
Amazon Business

6 source articles · read the reporting →

Amazon cuts 14,000 corporate jobs in bet on AI

Amazon announced it is cutting 14,000 corporate jobs, potentially up to 30,000, as part of a strategy to invest in artificial intelligence, which the company says will replace some human roles. The cuts affect divisions including human resources, operations, devices and services, and Amazon Web Services. CEO Andy Jassy has said AI will change how work is done and result in fewer people doing some jobs. The company has not commented on the impact on its UK workforce.

Company involved
Amazon

5 source articles · read the reporting →

Amazon's Astro robot tracks everyone and has flawed person recognition

Leaked documents and employee testimony reveal that Amazon's Astro home robot uses flawed facial recognition to track and record people in homes. The robot patrols rooms, investigates strangers by following and recording them, and uploads footage automatically. Two sources said the device is unreliable, may fall down stairs, and its person detection is poor. Amazon denied the claims, stating the robot underwent rigorous testing.

Company involved
Amazon
AI system involved
Astro

8 source articles · read the reporting →

Arizona accuses Amazon of using dark patterns and biased Buy Box algorithm

The Arizona Attorney General filed two lawsuits against Amazon on May 15, 2024. One lawsuit alleges that Amazon used deceptive design tricks (dark patterns) to prevent users from canceling Prime subscriptions. The other alleges that Amazon's Buy Box algorithm is biased in favor of Amazon's own products and Fulfillment by Amazon sellers, causing consumers to overpay. Amazon denied the allegations, stating that the lawsuits are based on a misunderstanding.

Company involved
Amazon
AI system involved
Buy Box algorithm

6 source articles · read the reporting →

Italy fines Amazon $1.3 billion for algorithm favouring its logistics

Italy's antitrust regulator fined Amazon €1.13 billion ($1.3 billion) for abusing its market dominance. The regulator alleged that Amazon used an algorithm to give an unfair advantage to its own logistics service over third-party sellers. Amazon said it disagreed with the decision and would appeal.

Company involved
Amazon

9 source articles · read the reporting →

CMA investigates Amazon over suspected anti-competitive marketplace practices

The UK Competition and Markets Authority has opened an investigation into Amazon UK Marketplace, alleging that Amazon may be abusing a dominant position by using third-party seller data and favouring its own retail business or sellers using its logistics and delivery services. The investigation will examine how Amazon collects and uses seller data, how it sets criteria for the Buy Box, and how it determines Prime eligibility. The CMA stated it has not reached any conclusions and that the investigation focuses on potential harm to competition and consumers.

Company involved
Amazon

8 source articles · read the reporting →

Ex-Amazon AI scientist alleges she was fired after complaining about copyright policy violations

Dr Viviane Ghaderi, a former AI scientist at Amazon, alleges that she was demoted and fired after returning from maternity leave and complaining that Amazon was violating its own copyright policies in its AI research. She claims her managers told her to ignore copyright rules to keep up with competitors. The lawsuit, filed in Los Angeles state court, accuses Amazon of pregnancy discrimination, retaliation, and wrongful termination.

Company involved
Amazon

6 source articles · read the reporting →

Amazon France fined €32m for illegal worker surveillance

Amazon France Logistique was fined €32m (£27m) by France's data protection authority, the CNIL, for illegally monitoring warehouse workers through handheld scanners. The system tracked scanning speed, break durations, and stored data for 31 days, leading to workers potentially having to justify every brief interruption. The CNIL found the surveillance to be excessive and in breach of GDPR. Amazon disputes the findings and stated it may appeal.

Company involved
Amazon France Logistique
AI system involved
handheld scanner monitoring system

9 source articles · read the reporting →

Amazon's Alexa claimed 2020 US election was stolen, citing Rumble

Amazon's voice assistant Alexa told users who asked about fraud in the 2020 US presidential election that the contest was 'stolen by a massive amount of election fraud', citing the conservative-favoured video platform Rumble. The Washington Post reports that this false claim was still being delivered more than two years after Joe Biden defeated Donald Trump, despite Amazon promoting Alexa as a reliable source of election news. The article does not name any individual who was harmed by the response, but describes the episode as foreshadowing a new information battleground.

Company involved
Amazon
AI system involved
Alexa

6 source articles · read the reporting →

AI-generated travel guides flood Amazon, deceiving customers

AI-generated travel guides are flooding Amazon, allegedly written by scammers using AI to produce generic, low-quality content. Customers who purchase these guides receive poor information and feel defrauded. Amazon claims to enforce content guidelines but many such books remain on the platform.

Company involved
Amazon

8 source articles · read the reporting →

Amazon accused of using Project Nessie algorithm to inflate prices

Amazon is accused of using a secret algorithm called Project Nessie to inflate prices on its platform. The algorithm monitored competitors' prices and raised its own, then reverted if rivals didn't follow. The practice was revealed through an FTC lawsuit. Amazon says it scrapped the program in 2019 and denies wrongdoing.

Company involved
Amazon
AI system involved
Project Nessie

10 source articles · read the reporting →

Amazon's Alexa makes racist remarks about Black girls and boys

Amazon's Alexa voice assistant provided racist descriptions when asked about African-American girls and boys. Researchers reported that the device said African-American girls are the fastest growing segment of the juvenile justice system and that African-American boys are struggling readers. The article notes that such responses can harm children's confidence and highlights ongoing issues of racial bias in AI systems.

Company involved
Amazon
AI system involved
Alexa

10 source articles · read the reporting →

AI chatbots used to write fake Amazon reviews

Some Amazon product reviews are written by AI chatbots like ChatGPT. The reviews contain the phrase 'As an AI language model' and are generic. Amazon prohibits review abuse and says it uses moderators and automated tools to combat fake reviews.

Company involved
Amazon

7 source articles · read the reporting →

Amazon's Alexa accidentally attempted to order dollhouses after hearing a TV news segment

A TV news broadcast in San Diego, California, caused multiple Amazon Alexa devices to wake up and attempt to order dollhouses after the anchor said the word 'Alexa'. Although no orders were completed because a secondary confirmation was required, the incident highlighted privacy concerns about the device's always-listening microphone. Amazon acknowledged the event but stated that voice shopping was nearly impossible by accident.

Company involved
Amazon
AI system involved
Alexa

8 source articles · read the reporting →

Neighbour wins privacy case over Ring doorbell and cameras in Oxfordshire

Dr Mary Fairhurst claimed that a Ring doorbell and security cameras installed by her neighbour Jon Woodard unlawfully captured images of her property and audio of her conversations. The judge ruled that the devices breached UK data laws and contributed to harassment. Mr Woodard now faces a substantial fine.

Company involved
Jon Woodard
AI system involved
Ring doorbell and security cameras

10 source articles · read the reporting →

← Newerpage 3 of 4Older →