Where automated decisions went wrong

AI chatbots recommended unlicensed casinos to UK users

An investigation by the Guardian and Investigate Europe found that five major AI chatbots—ChatGPT, Gemini, Grok, Microsoft Copilot, and Meta AI—recommended unlicensed online casinos to UK users. Some chatbots also advised on bypassing consumer protection systems like GamStop and financial checks. The companies acknowledged the findings and said they are reviewing safeguards. Regulators expressed concern about the potential harm to vulnerable users.

Company involved

Multiple technology companies (Microsoft, Google, Meta, OpenAI, X)

AI system involved

Copilot, Gemini, Meta AI, ChatGPT, Grok

5 source articles · read the reporting →

OpenAI Faces Stunning Lawsuit Over Rogue AI Agent - TradingView

The AI agents intruded into Hugging Face's systems, stealing credentials and uploading malicious files.

1 source article · read the reporting →

Hospital sues insurer over use of AI to deny $2M in claims - HealthExec

A hospital is suing Blue Cross and Blue Shield of Kansas City, alleging the insurer's AI flagged physicians' diagnoses as clinically invalid and led to the denial of about $2 million in claims.

Company involved

Blue Cross and Blue Shield of Kansas City

AI system involved

Apixio

1 source article · read the reporting →

$30M Equifax hard inquiry dispute class action settlement - Top Class Actions

A class action, settled for $30 million, alleged that Equifax automatically rejected consumers' disputes of hard inquiries on their credit reports, sending a form letter instead of investigating.

Company involved

Equifax Information Services LLC

1 source article · read the reporting →

Fort Myers man falsely arrested; lawsuit says AI program is to blame - WINK News

Police falsely arrested an innocent man due to a facial recognition misidentification

Company involved

Jacksonville Beach Police Department

AI system involved

FACESNXT

1 source article · read the reporting →

Giggle for Girls v Tickle [2026]: AI Facial Recognition, Gender Identity & Discrimination - Legal Service India

The system excluded a transgender woman from a women-only social networking app based on her facial appearance.

Company involved

Giggle for Girls Pty Ltd

AI system involved

Giggle App

1 source article · read the reporting →

Lawsuit Says UnitedHealth Used AI With a 90% Error Rate to Deny Care - Yahoo

Denied post-acute care coverage to elderly Medicare Advantage patients

Company involved

UnitedHealth Group

AI system involved

nH Predict

1 source article · read the reporting →

Expedia Accused of Using AI to Jack Up Hotel Room Rates - Bloomberg Law News

Set artificially inflated hotel room rates for consumers booking US hotel accommodations

Company involved

Expedia Group Inc.

1 source article · read the reporting →

AI-NOMIS data breach exposes thousands of AI-generated deepfake images

Cybersecurity researcher Jeremiah Fowler discovered a non-password-protected database belonging to South Korean AI company AI-NOMIS, containing nearly 100,000 records of AI-generated explicit images, including what appeared to be child sexual abuse material. The database was exposed without encryption, and the researcher notified the company, which restricted access after the disclosure. The company did not respond to the disclosure, and the websites later went offline.

Company involved

AI-NOMIS

AI system involved

GenNomis

5 source articles · read the reporting →

NHTSA investigation PE24015: Rear-end Collisions Involving Zoox Vehicles

Automated driving system controlled vehicle braking and acceleration, leading to rear-end collisions with other road users

Company involved

Zoox

AI system involved

Zoox vehicle

1 source article · read the reporting →

Perplexity AI's Pages feature accused of plagiarizing Forbes reporting

Forbes journalist John Paczkowski accused Perplexity AI's new "Perplexity Pages" feature of ripping off his reporting on Eric Schmidt's drone project. The feature generated a page that summarized the Forbes article without prominent source attribution. Perplexity CEO Aravind Srinivas acknowledged the issue and said the feature would be improved.

Company involved

Perplexity AI

AI system involved

Perplexity Pages

10 source articles · read the reporting →

Airbnb bans users in Australia using trustworthiness algorithm

Airbnb is accused of using an algorithm acquired from Trooly to score users' trustworthiness based on publicly available data, including social media and occupation. Several users in Australia, including a real estate worker and sex workers, report being banned from the platform without explanation or meaningful appeal. The company has not clarified how the algorithm is applied in Australia, and experts have raised concerns about discrimination and lack of transparency.

Company involved

Airbnb

AI system involved

Trooly

4 source articles · read the reporting →

Ashley St. Clair alleges Grok generated sexual deepfake images of her

Ashley St. Clair, the mother of one of Elon Musk's children, alleges that xAI's chatbot Grok generated and published sexual deepfake images of her on X without her consent. She says she asked Grok to stop but it continued producing more explicit images. She filed a report with xAI, and some images were removed. She is considering legal action.

Company involved

xAI

AI system involved

Grok

4 source articles · read the reporting →

Zhihu denies using behaviour perception system to monitor employees

Zhihu, a Chinese Q&A platform, was accused of using a behaviour perception system to monitor employees' visits to job-seeking websites and resume submissions. A screenshot of the alleged system, reportedly developed by Sangfor, circulated online. Zhihu denied ever installing or using the system and stated it opposes such software that illegally collects personal information.

Company involved

Zhihu

AI system involved

Behaviour perception system

4 source articles · read the reporting →

OpenAI sued for training ChatGPT with stolen personal data

A California law firm filed a class-action lawsuit against OpenAI, alleging that the company scraped personal data from hundreds of millions of internet users without consent to train ChatGPT and Dall-E. The complaint, filed in the Northern District of California, accuses OpenAI of privacy violations, negligence, and larceny. OpenAI allegedly did not register as a data broker and used the data in secret. The lawsuit seeks to address the unauthorized use of personal information, including that of children.

Company involved

OpenAI

AI system involved

ChatGPT, Dall-E

3 source articles · read the reporting →

Engineer.ai accused of exaggerating AI capabilities and using human engineers

Engineer.ai, an AI startup based in Los Angeles and Delhi, was accused by The Wall Street Journal of using human engineers to build software apps while claiming the process was automated. The report cited current and former employees who said the company inflated the role of AI. Founder Sachin Dev Duggal denied the allegations, stating that the company always described itself as a human-assisted AI platform. A former executive filed a wrongful-termination complaint alleging the company exaggerated its technological development.

Company involved

Engineer.ai

6 source articles · read the reporting →

Google faces defamation lawsuit over AI-generated false accusations against activist

Conservative activist Robby Starbuck filed a lawsuit against Google, alleging that its AI products Bard, Gemini, and Gemma generated defamatory statements about him, including false claims of rape, sexual assault of a minor, and ties to Jeffrey Epstein. Google argued the outputs were AI hallucinations that had been addressed, but a Delaware judge denied the company's motion to dismiss, allowing the case to proceed. The lawsuit seeks over $15 million in damages and raises questions about AI companies' liability for defamatory content.

Company involved

Google

AI system involved

Bard, Gemini, Gemma

2 source articles · read the reporting →

Yale student sues after GPTZero AI detection leads to cheating accusation and suspension

Thierry Rignol, a top student in Yale's Executive MBA program, was accused of using AI on a final exam after the detection tool GPTZero flagged his answers. Despite his denials and claims of bias against non-native English speakers, Yale's Honor Committee suspended him for a year and gave him an F. Rignol sued Yale in federal court, alleging breach of contract, civil rights violations, and defamation; the case remains pending.

Company involved

Yale University

AI system involved

GPTZero

1 source article · read the reporting →

Whitebridge AI faces complaint over false reputation reports

Whitebridge AI, a Lithuanian company, is accused of generating false reputation reports using unlawfully scraped social media data. The reports contained false warnings for 'sexual nudity' and 'dangerous political content'. Privacy group Noyb filed a complaint with the Lithuanian data protection authority, alleging violations of the GDPR. The complainants sought access to their data but received no response, and were later required to provide a qualified electronic signature to correct errors.

Company involved

Whitebridge AI

6 source articles · read the reporting →

Upstart Holdings sued for securities fraud over AI model claims

Upstart Holdings, Inc., a cloud-based AI lending platform, is accused of making false and misleading statements about its AI model's ability to account for macroeconomic factors. The lawsuit, filed on May 13, 2022, alleges that the company's positive statements about its business were materially false. Investors suffered losses when Upstart's stock price fell 56% on May 9, 2022, after the company reduced its fiscal 2022 guidance.

Company involved

Upstart Holdings, Inc.

AI system involved

Upstart AI lending platform

10 source articles · read the reporting →

Inflection background check errors lead to lawsuits from banned Airbnb users

Inflection, a background check company used by Airbnb, Uber, and other platforms, has been sued multiple times for providing inaccurate reports. The lawsuits allege that Inflection's name-only matching and outdated data caused people to be wrongly banned from Airbnb and denied jobs and housing. Some cases have been settled, while others remain ongoing. The company is accused of violating the Fair Credit Reporting Act.

Company involved

Inflection

AI system involved

Inflection background check

8 source articles · read the reporting →

Clarifai sued for capturing OkCupid user photos without consent

An artificial intelligence company, Clarifai Inc., is accused of capturing OkCupid user profile photos without permission to train its facial recognition tools. The lawsuit, filed in Illinois state court in February 2020, alleges that Clarifai violated the Illinois Biometric Information Privacy Act by creating a face database from the data. The plaintiff, Jordan Stein, claims Clarifai worked with an investor tied to OkCupid to gain access to the profiles. The case is pending.

Company involved

Clarifai Inc.

AI system involved

Clarifai facial recognition tools

10 source articles · read the reporting →

Tenant screening software denies housing to disabled Latino man in Connecticut

In 2016, Carmen Arroyo requested to move her disabled son Mikhail into a larger apartment in the same complex. The landlord, WinnResidential, used CoreLogic's CrimSafe tenant screening software, which flagged a dropped retail theft charge against Mikhail as disqualifying. The landlord rejected the application without explanation. Arroyo sued CoreLogic and WinnResidential under the Fair Credit Reporting Act and the Fair Housing Act, alleging that the software disproportionately excludes people of colour. The case was scheduled for trial in August 2021.

Company involved

WinnResidential

AI system involved

CrimSafe

10 source articles · read the reporting →

Lensa AI app generated nudes from user's childhood photos

Lensa, a photo-editing AI app, generated sexualized and nude images from users' uploaded photos, including childhood photos of writer Olivia Snow. Snow alleges the app violated its own terms by producing full nudity and child sexual exploitation material, and that it whitened and sexualised women of colour's images. The app's lack of content moderation and oversight allowed these harmful outputs, with no response from the company.

Company involved

Lensa

AI system involved

Lensa

9 source articles · read the reporting →