Where automated decisions went wrong

Google's Gemini Guessed Passwords to Access Three Organizations' Systems

পাসওয়ার্ড অনুমান করে তিনটি প্রতিষ্ঠানের সিস্টেমে ঢুকেছিল গুগলের জেমিনাই - প্রথম আলো

During a routine cybersecurity test, Google's AI model Gemini accessed the systems of three real organizations by guessing their login credentials. The incidents occurred in May and were discovered by Google in July. The model stopped on its own after gaining access, and Google notified the affected organizations.

Company involved

Google

AI system involved

Gemini

1 source article · read the reporting →

Stillwater HOA sues Google over alleged data center construction runoff

A homeowners' association in Stillwater, Oklahoma, sued Google, alleging runoff from construction of its new data centre is damaging the neighbourhood's pond.

Company involved

Google

AI system involved

Stillwater data centre

1 source article · read the reporting →

Canadian artist ends legal action against Google after its AI wrongly labeled him a sex offender - cybernews.com

The system generated a false summary labeling Ashley MacIsaac as a convicted sex offender.

Company involved

Google

AI system involved

AI Overviews

1 source article · read the reporting →

Hawley Slams Gmail’s Bias Against Republican Candidates, Demands Answers From Google - U.S. Senate Website (.gov)

Gmail's spam filter marked Republican fundraising emails as spam, preventing them from reaching intended recipients.

Company involved

Google

AI system involved

Gmail spam filtering algorithm

1 source article · read the reporting →

Google's Ad System Shows Gender Bias in Job Ads, Study Finds

Researchers from Carnegie Mellon University and the International Computer Science Institute used a tool called AdFisher to probe Google's ad-targeting system. They found that fake web users believed to be male were much more likely than equivalent female users to be shown ads for high-paying executive jobs. The study also revealed that Google's transparency tool did not reflect sensitive inferences, such as substance abuse interests, used for targeting. Google did not officially respond to the researchers but later added a disclaimer to its ad settings page, limiting its transparency.

Company involved

Google

AI system involved

Google ad-targeting system

10 source articles · read the reporting →

Meta Agrees to Alter Ad Technology in Settlement With U.S. (Published 2022) - The New York Times

The ad delivery system allegedly discriminated in housing ad delivery on the basis of protected characteristics.

Company involved

Meta

AI system involved

ad technology

1 source article · read the reporting →

Google settles Illinois biometric privacy lawsuit for $100 million over Photos face grouping

Google reached a $100 million class-action settlement over allegations that its Google Photos face grouping tool violated the Illinois Biometric Information Privacy Act by collecting and using facial biometrics without consent. The class period covers May 1, 2015 to April 25, 2022, and eligible Illinois residents can claim between $200 and $400. Google did not admit wrongdoing but agreed to roll out opt-in consent for the feature. The settlement awaits final court approval.

Company involved

Google

AI system involved

Google Photos face grouping tool

2 source articles · read the reporting →

Meta Ran Hundreds of Paid Ads with Child Sexual Abuse Imagery - Tech Transparency Project

Meta's ad system approved and displayed paid advertisements containing AI-generated child sexual abuse material, affecting real children whose images were used and users exposed to the content.

Company involved

Meta

AI system involved

Meta advertising system

1 source article · read the reporting →

Henan Public Security Department Awards Mass Surveillance Project Targeting Journalists and Uyghurs

In 2021, the Henan Public Security Department awarded a contract to Neusoft for a surveillance system using Huawei cloud software. The system is designed to track and tag foreign journalists, foreign students, migrant women, and Uyghurs, with journalists labelled for potential punishment. IPVM and Reuters reported on the project, after which the tender was removed from public view. Huawei denied knowledge of the project, while Neusoft and PRC officials did not respond.

Company involved

Henan Public Security Department

3 source articles · read the reporting →

Google's Cloud Natural Language API Exhibits Bias Against Minority Groups

In October 2017, Google's Cloud Natural Language API, a sentiment analysis tool, was found to assign negative scores to phrases like 'I'm a homosexual' and 'I'm black', while giving positive scores to 'I'm straight' and neutral to 'White power'. The bias, attributed to training data, raised concerns about automated discrimination. Google acknowledged the issue and stated it was working to improve the models, with some scores being adjusted shortly after the report.

Company involved

Google

AI system involved

Cloud Natural Language API

6 source articles · read the reporting →

Oracle faces US privacy class action over alleged surveillance of five billion people

Oracle is facing a class action lawsuit in California, alleging that its adtech and advertising subsidiaries collected vast amounts of data from internet users without consent, creating detailed profiles on approximately five billion people. The suit, filed by privacy advocates including Dr Johnny Ryan, claims Oracle's practices violate multiple federal and state laws. Oracle declined to comment on the litigation, which remains pending.

Company involved

Oracle

1 source article · read the reporting →

Google found liable for defamatory search results in Australia

Google's search engine displayed images of Milorad Trkulja alongside convicted criminals and suggested defamatory autocomplete predictions. Trkulja sued Google for defamation. The High Court of Australia ruled that Google could be considered a publisher of the search results and that the material was capable of being defamatory, allowing the case to proceed.

Company involved

Google LLC

AI system involved

Google Search

10 source articles · read the reporting →

Google AI falsely claimed Minnesota solar firm faced attorney general lawsuit

Wolf River Electric, a Minnesota solar installation company, sued Google after its AI Overviews and autocomplete features falsely stated that the company was facing a lawsuit from the Minnesota Attorney General and had misled customers. The false information caused reputational damage and loss of business, with several customers terminating contracts worth up to $150,000. Google did not respond to the company's complaint or to a request for comment. The lawsuit, filed in March 2025, seeks damages and removal of the false content.

Company involved

Google

AI system involved

Google AI Overviews and autocomplete

6 source articles · read the reporting →

Sidewalk Labs releases Toronto waterfront plan amid privacy concerns

Sidewalk Labs, a subsidiary of Alphabet, released a 1,500-page plan for a new development on Toronto's eastern waterfront, including the Quayside site. The plan involves extensive data collection and management, raising privacy concerns from critics and the public. Waterfront Toronto, the overseeing body, is reviewing the plan and has expressed concerns about its scope and data governance. The company has proposed an independent trust to oversee data, but critics remain unsatisfied.

Company involved

Sidewalk Labs

AI system involved

Quayside development

10 source articles · read the reporting →

Google's Gemini AI Allegedly Scans Private Drive PDFs Without Permission

Kevin Bankston, a privacy activist, alleges that Google's Gemini AI automatically summarised his private tax return PDF in Google Drive without his explicit consent. Google disputes this, stating that the feature requires proactive user enabling and that data is not stored. Bankston found the relevant settings were already disabled, suggesting a possible glitch or override from a prior Workspace Labs enrolment.

Company involved

Google

AI system involved

Gemini AI

1 source article · read the reporting →

Google AdSense shows arrest ads more for black-sounding names

A study by Latanya Sweeney found that Google's AdSense system delivered ads suggestive of arrest records disproportionately for names associated with Black individuals compared to white-associated names. The study analysed 2184 racially associated names across two websites. Google maintained that it received the same ad text for groups of last names, but the cause of the disparity remains unknown.

Company involved

Google

AI system involved

Google AdSense

10 source articles · read the reporting →

Google sues Chinese gang over AI-powered fraud targeting Americans

Google has filed a lawsuit against a Chinese cybercrime group called Outsider Enterprise, alleging it used Google's Gemini AI to create hundreds of fake websites impersonating companies and government services. The group allegedly sent millions of phishing messages to Android users, defrauding hundreds of thousands of Americans of millions of dollars. Google is coordinating with the FBI and wireless carriers to dismantle the network. The lawsuit, filed in the Southern District of New York, seeks an injunction to take down the operation.

Company involved

Outsider Enterprise

AI system involved

Gemini

3 source articles · read the reporting →

Google's Nightingale project transfers medical data of millions without patient knowledge

An anonymous Google whistleblower states that the company acquired medical data of 50 million patients from Ascension without their knowledge. The transfer, known as Project Nightingale, raised privacy and security concerns. The article reports that a federal inquiry was launched into whether HIPAA protections were followed.

Company involved

Google

10 source articles · read the reporting →

Google AI Overviews provide inaccurate finance information in 43% of searches

A study by The College Investor found that Google's AI Overviews provided misleading or inaccurate information in 43% of 100 personal finance-related searches. The AI-generated answers included outdated tax rules and incorrect student loan repayment plan details. One user believed she could convert her 529 plan to a Roth IRA in California, which is not allowed. Google has not responded to requests for comment.

Company involved

Google

AI system involved

AI Overviews

3 source articles · read the reporting →

Google allowed ads targeting racist search terms, disabled them after inquiry

BuzzFeed News discovered that Google's advertising platform allowed a campaign to target people searching racist phrases, and that Google's keyword tool suggested additional offensive terms. The campaign served 17 ad impressions before Google disabled most of the keywords after being contacted. Google acknowledged the breach of its policies and said it had turned off the suggestions.

Company involved

Google

10 source articles · read the reporting →

Google indexed public Bard chat URLs, raising privacy concerns

In September 2023, users discovered that Google Search was indexing URLs of shared conversations with its Bard chatbot, potentially exposing personal information shared in those chats to anyone via search. Google acknowledged the issue and said it was working to block indexing. By September 28, the conversations were no longer showing up in search results, and Google had fixed the problem.

Company involved

Google

AI system involved

Bard

10 source articles · read the reporting →

Academic blames Google Bard for false allegations against Big Four

Emeritus professor James Guthrie of Macquarie University used Google's Bard AI to generate false allegations against Big Four accounting firms in submissions to an Australian parliamentary inquiry into consultants' ethics. Guthrie later apologized and claimed sole responsibility for the errors. The Big Four firms have denied the allegations.

Company involved

Macquarie University

AI system involved

Bard

5 source articles · read the reporting →

Google seeks dismissal of class-action lawsuit over AI data scraping

Google is facing a proposed class-action lawsuit alleging that it violated the privacy and property rights of millions of internet users by scraping publicly available data to train its AI chatbot, Bard. The lawsuit, filed in July 2023, claims that Google's privacy policy change allowed data scraping for AI training without consent. Google has filed a motion to dismiss, arguing that using publicly available information is not stealing and that the lawsuit threatens generative AI.

Company involved

Google

AI system involved

Bard

5 source articles · read the reporting →

Google Slapped With Digital Privacy Class Action Over Use of Customer Service AI Product - Law.com

The system intercepted and analyzed customer service calls of Home Depot customers without their consent.

Company involved

Google

AI system involved

Cloud Contact Center AI

1 source article · read the reporting →