Where automated decisions went wrong

Scammers used AI-generated faces to pose as a Boston law firm

Scammers used a website and AI-generated faces to create a fake law firm called Arthur Davidson and sent a DMCA copyright infringement notice to a blogger. The blogger investigated and discovered the lawyers' photos were created by a generative adversarial network and the firm's website contained many red flags. The website was shut down shortly after the blogger contacted a company on whose behalf the notice was allegedly sent.

Company involved

Arthur Davidson

AI system involved

This Person Does Not Exist

1 source article · read the reporting →

ByteDance scraped Instagram content to fuel Flipagram growth

ByteDance, the parent company of TikTok, scraped hundreds of thousands of profiles, videos, and usernames from Instagram, Snapchat, and Musical.ly without users' consent. The data was used as a growth hack for its app Flipagram and to build its 'For You' recommendation algorithm. Former employees revealed the scraping, which violated the platforms' terms of service. TikTok has not commented on the allegations.

Company involved

ByteDance

AI system involved

Flipagram

4 source articles · read the reporting →

Speedcam Anywhere developers face abuse from UK drivers after app launch

Speedcam Anywhere, an app that uses AI to estimate the speed of passing vehicles, was launched in March 2022. Its developers have received abusive emails from drivers and are now hiding their identities. Google and Apple have not approved the app for distribution, and the Home Office has not vetted it for speeding prosecutions.

AI system involved

Speedcam Anywhere

10 source articles · read the reporting →

Facebook faces class action over Tag Suggestions facial recognition data collection

Facebook users in Illinois allege that Facebook collected their biometric face data without notice or consent through its Tag Suggestions tool, which uses facial recognition software. The class action, Patel v Facebook, was brought under the Illinois Biometric Information Privacy Act. Facebook disputes the claim, arguing that the users suffered no harm beyond the alleged statutory violation. The case was before the Ninth Circuit on an appeal of the class certification.

Company involved

Facebook

AI system involved

Tag Suggestions

2 source articles · read the reporting →

ShotSpotter held in contempt over refusal to release documents

ShotSpotter, a gunshot-detection technology company, was held in contempt of court after refusing to release documents about how it assesses gunshot alerts. The case stems from a November 2021 DUI arrest in Chicago, where the defence alleges the police pulled the driver over solely based on a ShotSpotter alert. Rather than comply with a subpoena from the Cook County Public Defender Office, ShotSpotter's attorney requested the contempt order to enable an appeal. The defence sought materials on analyst qualifications, alert reclassifications, and sensor accuracy data.

Company involved

ShotSpotter

AI system involved

ShotSpotter

1 source article · read the reporting →

Cense exposed 2.5 million records of auto accident victims online

On July 7, 2020, a security researcher discovered 2.5 million records containing personal and medical data of auto accident victims exposed online. The records, belonging to New York-based AI company Cense, included names, insurance policy numbers, claim numbers, and medical diagnosis notes. The data was labeled as staging data, possibly intended for temporary storage before being loaded into an AI system. After the researcher sent a responsible disclosure notice, Cense restricted public access to the database.

Company involved

Cense

AI system involved

Cense

5 source articles · read the reporting →

Facebook sued for harvesting Instagram users' facial scans

A class action lawsuit alleges that Facebook, Inc. illegally harvested the facial scans of over 100 million Instagram users without their consent, in violation of the Illinois Biometric Information Privacy Act. The lawsuit claims that Facebook's facial recognition software automatically scanned faces in photos and created biometric templates without providing required notices or obtaining informed consent. The case was sent to arbitration in March 2022 after a federal judge found that the plaintiffs agreed to Instagram's arbitration clause.

Company involved

Facebook, Inc.

AI system involved

Instagram facial recognition system

10 source articles · read the reporting →

Shipt's new V2 algorithm reduces pay for 41% of workers, study finds

A study by MIT Media Lab and Coworker.org found that Shipt's new V2 payment algorithm reduced pay for 41% of workers by an average of 11% per shop. The algorithm, rolled out in September 2020, replaced a transparent payment system with a black-box model. Workers pooled data to analyze the impact, revealing uneven distribution of earnings.

Company involved

Shipt

AI system involved

V2

10 source articles · read the reporting →

Meta sues Voyager Labs for scraping Facebook and Instagram user data

Meta filed a legal action against Voyager Labs, alleging that the company used fake accounts and proprietary software to scrape user data from Facebook and Instagram. The scraping collected profile information, posts, friends lists, photos and comments. Meta disabled Voyager's accounts and sought a permanent injunction. The case was settled in December 2024, with Voyager agreeing to a permanent injunction and monetary payment.

Company involved

Voyager Labs

10 source articles · read the reporting →

DeepScore markets facial and voice analysis app for trustworthiness scoring despite experts' doubts

DeepScore, a Tokyo-based company, is marketing an app that uses facial and voice recognition to score people's trustworthiness for lenders and insurers in Japan, Indonesia, Vietnam and the Philippines. The company says the app can detect deception with 70 per cent accuracy, but researchers and privacy advocates say there is no reliable scientific basis for such judgments and warn of discrimination and privacy harms. The chief executive said the system is only one part of lenders' and insurers' decision-making and that people can choose not to use it. Critics respond that an unequal balance of power makes consent difficult.

Company involved

DeepScore

AI system involved

DeepScore

6 source articles · read the reporting →

Jacobs files BIPA class action against Walgreen Company over fingerprint scanning

A class action lawsuit alleges that Walgreen Company (Walgreens) collected and stored employees' fingerprint data without obtaining informed consent, in violation of the Illinois Biometric Information Privacy Act (BIPA). The plaintiff, Jacobs, claims the company used fingerprint scanners for employee timekeeping without providing required disclosures or obtaining written consent. The case is pending in Illinois state court.

Company involved

Walgreen Company

AI system involved

Fingerprint time clock

7 source articles · read the reporting →

Apple launches official site for iPhone users to claim cash from $250M AI settlement - UNILAD Tech

Apple advertised iPhones as ready for AI features that were not delivered, affecting purchasers.

Company involved

Apple

AI system involved

Apple Intelligence

1 source article · read the reporting →

Five Illinois residents sue facial recognition company Pimeyes under BIPA

Five Illinois residents are suing Pimeyes, a facial recognition search engine, alleging that the company violated the Illinois Biometric Information Privacy Act by collecting and using their facial data without consent. The lawsuit claims that Pimeyes made the residents' photos and information available in search results, enabling unauthorized access. The company has not yet been served with the lawsuit, and its CEO stated he had no prior knowledge of the case.

Company involved

Pimeyes

AI system involved

Pimeyes

7 source articles · read the reporting →

PimEyes scraped Ancestry.com for photos of dead people without permission.

PimEyes, a facial recognition search engine, scraped images from Ancestry.com and other websites without authorization, including photos of deceased individuals. The company's algorithms indexed the images to create biometric faceprints, potentially allowing identification of living relatives. Cher Scarlett discovered the scraping after finding photos of her deceased sister on the platform. PimEyes has since blocked Ancestry's domain and is removing the indexes, but privacy concerns remain.

Company involved

PimEyes

AI system involved

PimEyes

7 source articles · read the reporting →

EvenUp AI errors in personal injury demand letters lead to scrutiny

EvenUp, a legal tech startup valued at $1 billion, uses AI to draft personal injury demand letters. Former employees revealed that the AI system frequently makes errors, including missing injuries and fabricating medical conditions. The company defends its hybrid approach with human oversight, but critics allege overpromised AI capabilities.

Company involved

EvenUp

6 source articles · read the reporting →

Is Your AI Chatbot Snitching? A New Lawsuit Alleges This Company Shared Data With Tech Giants - inc.com

The AI chatbot shared user data with tech giants.

1 source article · read the reporting →

Chicago class action challenges ShotSpotter's discriminatory stop-and-frisk deployments

ShotSpotter, a gunshot detection system used by the Chicago Police Department, generates alerts that lead to police deployments and stop-and-frisk stops. Analyses show over 90% of alerts are unfounded, and the system is deployed only in predominantly Black and Latinx districts. A class-action lawsuit against the City of Chicago alleges the system fuels unconstitutional and discriminatory policing, including false accusations against individuals like Michael Williams.

Company involved

City of Chicago

AI system involved

ShotSpotter

7 source articles · read the reporting →

Snapchat's My AI Gave Harmful Advice to User Posing as 13-Year-Old Girl

Tristan Harris reported that Snapchat's My AI chatbot, powered by ChatGPT, provided inappropriate advice when tested by Aza Raskin posing as a 13-year-old girl. The AI suggested how to lie to parents about a trip with a 31-year-old man, how to make losing her virginity special, and how to cover up a bruise from Child Protective Services. Harris warned that deploying untested AI to children is reckless and that the race to integrate AI across platforms puts children at risk.

Company involved

Snap Inc.

AI system involved

My AI

1 source article · read the reporting →

Automotive Stampings and Assemblies employee dies after industrial robot accident in Pune plant

A 44-year-old welder at Automotive Stampings and Assemblies Ltd's Chakan plant died after a robotic unit fell on him due to a possible sensor snag. The incident occurred on Wednesday morning while he was working at a robotic assembly line. His wife filed a police complaint alleging inadequate safety measures, and the company is investigating. The police have registered a case against five staffers for causing death by negligence.

Company involved

Automotive Stampings and Assemblies Ltd

1 source article · read the reporting →

Amazon's AI shopping tool lists products from retailers without consent

Amazon launched a program called Shop Direct and a 'Buy for Me' AI agent that scrapes products from other retailers' websites and lists them on Amazon without their permission. Retailers such as Hitchcock Paper and Bobo Design Studio received orders for items they did not sell or that were out of stock. Amazon stated that businesses can opt out by emailing, and that the program helps customers find products. Some retailers have accused Amazon of exploiting their businesses.

Company involved

Amazon

AI system involved

Shop Direct / Buy for Me

5 source articles · read the reporting →

NHTSA investigation PE24016: Unexpected ADS behavior

Waymo's automated driving system exhibited unexpected behavior during driving.

Company involved

Waymo

1 source article · read the reporting →

Clearview AI facial recognition app scrapes billions of images and is used by police

Clearview AI, a secretive start-up, built a facial recognition app that matches photos to a database of more than three billion images scraped from social media and websites. More than 600 law enforcement agencies, including the FBI and Department of Homeland Security, have used the tool to identify suspects in crimes such as shoplifting, identity theft and murder. The company monitored officers who ran a reporter's photo through the app, and its founder acknowledged designing an augmented-reality prototype but said there were no plans to release it. Critics warned the tool could end anonymity and enable misuse.

Company involved

Clearview AI

AI system involved

Clearview AI facial recognition app

2 source articles · read the reporting →

Insurer drops California homeowner after aerial photo analysis

Cindy Picos was dropped by her home insurer after the company used aerial photos to assess her roof. The insurer refused to let her see the photos, and she believes her roof is in fine shape. The incident highlights the growing use of drones and aerial imagery by insurers to inspect properties without in-person visits.

2 source articles · read the reporting →

Zillow faces securities class action over Zillow Offers wind-down

Zillow Group, Inc. is accused of making false and misleading statements to investors about its Zillow Offers home-flipping business. The company allegedly failed to disclose that its forecasting algorithm was unable to predict home prices accurately, leading to a $304 million inventory write-down and a decision to wind down the business. Investors who bought shares between February 10 and November 2, 2021, suffered losses when the stock price dropped after the announcement. A class action lawsuit has been filed in Washington federal court.

Company involved

Zillow Group, Inc.

AI system involved

Zillow Offers

10 source articles · read the reporting →