Where automated decisions went wrong

Thailand halts iris-scan crypto scheme and deletes 1.2m biometric records

The Ministry of Digital Economy and Society (DES) and the Personal Data Protection Committee (PDPC) ordered a company to stop collecting iris data and delete 1.2 million citizen records. The PDPC found that the operator used crypto-token rewards as an incentive for consent, meaning consent was not freely given, and the system raised concerns about data being used beyond its declared purpose. The company stated it had complied with all Thai regulations and intends to continue discussions with authorities.

Company involved

The company behind the iris-scan system (not named)

AI system involved

Iris-scan system

4 source articles · read the reporting →

Former Police Officer Sentenced to 5 Years After Misusing Flock Cameras To Stalk Ex-Girlfriend - Texas Scorecard

Former officer used Flock cameras to surveil and stalk his ex-girlfriend, her family members, and other romantic interests.

Company involved

Flock Safety

1 source article · read the reporting →

Hackers Extract Encryption Keys from Flock Surveillance Cameras, Revealing Person Detection Alongside Vehicle Tracking - finance.biggo.com

The system detected and recorded images of vehicles and people, affecting individuals in public spaces.

Company involved

Flock Safety

AI system involved

Flock Safety ALPR cameras

1 source article · read the reporting →

Flock seeks to have security researchers' map of Flock cameras taken down — unauthenticated flaw exposed 335,701 camera locations nationwide…

The system captured license plate data and vehicle locations of individuals passing the cameras.

Company involved

Flock Safety

AI system involved

Flock Safety cameras

1 source article · read the reporting →

Oakland County investigating alleged misuse of Flock camera data by former deputy - WXYZ 7 News Detroit

The Flock camera system collected and stored license plate data, and a former deputy allegedly accessed it without authorization.

Company involved

Oakland County Sheriff's Office

AI system involved

Flock camera

1 source article · read the reporting →

New Meta smart glasses class action gives employers a reason to write the policy they've been puttin - hcamag.com

The smart glasses recorded bystanders without their consent and sent the footage to Meta for AI training.

Company involved

Meta Platforms

AI system involved

Meta AI-enabled Ray-Ban and Oakley smart glasses

1 source article · read the reporting →

Fort Myers man falsely arrested; lawsuit says AI program is to blame - WINK News

Police falsely arrested an innocent man due to a facial recognition misidentification

Company involved

Jacksonville Beach Police Department

AI system involved

FACESNXT

1 source article · read the reporting →

Lawsuit claims facial recognition program led to wrongful arrest - FOX 2

Identified Christopher Gatlin as a suspect in an assault, leading to his arrest and 17 months in jail.

Company involved

St. Louis County Police Department

AI system involved

Rekognition

1 source article · read the reporting →

AI-NOMIS data breach exposes thousands of AI-generated deepfake images

Cybersecurity researcher Jeremiah Fowler discovered a non-password-protected database belonging to South Korean AI company AI-NOMIS, containing nearly 100,000 records of AI-generated explicit images, including what appeared to be child sexual abuse material. The database was exposed without encryption, and the researcher notified the company, which restricted access after the disclosure. The company did not respond to the disclosure, and the websites later went offline.

Company involved

AI-NOMIS

AI system involved

GenNomis

5 source articles · read the reporting →

Mr Craig Hadley wrongly accused of fraud at Sports Direct shop using Facewatch facial recognition.

Mr Craig Hadley was wrongly accused of being a fraudster at a Rotherham Sports Direct shop using facial recognition technology supplied by Facewatch. The system flagged him as a known fraudster, but the alert was later attributed to human error. Big Brother Watch, a privacy campaign group, has raised concerns about the lack of due process in such systems.

Company involved

Sports Direct

AI system involved

Facewatch

2 source articles · read the reporting →

Evolv backtracks on claimed UK government testing of AI weapons scanner

Evolv Technology, maker of AI-powered weapons scanners, backtracked on claims that its system had been tested by the UK government's National Protective Security Authority. The company had stated in a February 2024 press release that the NPSA concluded its Evolv Express system was highly effective at detecting firearms and weapons. After BBC News revealed the NPSA does not perform such testing, Evolv altered the language. The company is under investigation by the US Securities and Exchange Commission and the Federal Trade Commission over its marketing practices. Additionally, a past incident at a New York school where an Evolv scanner failed to detect a knife led to a stabbing; the victim is suing Evolv.

Company involved

Evolv Technology

AI system involved

Evolv Express

3 source articles · read the reporting →

Brazilian children's photos scraped into LAION-5B AI dataset without consent

Human Rights Watch found that LAION-5B, a dataset used to train AI tools, contains identifiable photos of Brazilian children scraped from the web without their knowledge or consent. The dataset includes names and locations of children. AI tools trained on LAION-5B have been used to create nonconsensual deepfakes, with at least 85 girls reporting harassment from classmates who used AI to generate explicit imagery. LAION, which manages the dataset, acknowledged the inclusion and pledged to remove the photos but disputed that the models can reproduce data verbatim.

Company involved

LAION

AI system involved

LAION-5B

7 source articles · read the reporting →

Australian children's photos scraped into LAION-5B AI dataset without consent

Human Rights Watch found that the LAION-5B dataset, used to train AI tools, contains links to identifiable photos of Australian children scraped from the web without consent. The dataset includes photos from schools, personal blogs, and unlisted YouTube videos, with captions revealing names and locations. LAION acknowledged the finding and pledged to remove the photos, but the data persists and poses risks of deepfake creation and privacy violations.

Company involved

LAION

AI system involved

LAION-5B

4 source articles · read the reporting →

Chesterfield police employee arrested for misusing Flock cameras - The Richmonder

The system allowed unauthorized searches of license plate data, affecting individuals whose plates were queried.

Company involved

Chesterfield County Police Department

AI system involved

Flock Safety Automatic License Plate Reader System

1 source article · read the reporting →

Kingsville police officer resigns after investigation into alleged Flock camera misuse - Yahoo

Kingsville police officer resigns after investigation into alleged Flock camera misuse Yahoo

Company involved

Kingsville Police Department

AI system involved

Flock camera system

1 source article · read the reporting →

More than 70 sue Meta in California, alleging smart glasses exposed intimate footage - The Cool Down

The smart glasses recorded and transmitted intimate images and videos of wearers and bystanders to Meta for AI training without consent.

Company involved

Meta

AI system involved

Ray-Ban Meta smart glasses

1 source article · read the reporting →

Boulder police chief faces class action over Flock cameras' warrantless surveillance - Courthouse News

The system captured and stored license plate data and vehicle movements of Boulder residents and commuters, making it searchable by law enforcement without a warrant.

Company involved

Boulder Police Department

AI system involved

Flock Safety

1 source article · read the reporting →

Five More Cops Charged for Abusing Flock Cameras Amid So Many Similar Scandals That We Can't Even Keep Track -…

The system tracked the movements of individuals, including women stalked by officers, a private citizen, and a domestic violence victim's data shared.

Company involved

Flock

AI system involved

Flock ALPR

1 source article · read the reporting →

Trial of live facial recognition in London stations leads to a false positive and no arrests - AOL.ca

Trial of live facial recognition in London stations leads to a false positive and no arrests AOL.ca

1 source article · read the reporting →

Former Emerson officer accused of misusing Flock camera system - wrganews.com

The Flock camera system captured license plate data of individuals.

Company involved

Acworth Police Department

AI system involved

Flock Safety System

1 source article · read the reporting →

NC police officer allegedly used Flock license plate cameras to track boyfriend's ex-wife - ABC11 News

The system tracked the location of the boyfriend's ex-wife.

Company involved

Mooresville Police Department

AI system involved

Flock

1 source article · read the reporting →

Google Pixel 9 tools generate offensive images and fake photos

Google's Pixel Studio and Reimagine features on the Pixel 9 phones are accused of generating offensive, copyrighted images and allowing users to create convincing fake photos. Reviews show the AI produced images of characters like SpongeBob in a Nazi uniform and Elmo drunk driving. Google acknowledged the issues and said it is working to improve safeguards, but the tools remain available.

Company involved

Google

AI system involved

Pixel Studio, Reimagine (Magic Editor)

5 source articles · read the reporting →

South Korean local governments build AI surveillance systems using CCTV data without consent

Several local governments in South Korea, including Bucheon, are developing AI systems that use CCTV footage to track individuals for purposes such as disease control, child abuse detection, and personal protection. The systems are being built without obtaining consent from the people captured in the footage, raising concerns about privacy and surveillance. Bucheon plans to launch a smart epidemiological investigation system in January 2022 that will analyze footage from thousands of crime-prevention cameras to track COVID-19 cases and contacts. Critics question the adequacy of de-identification measures and warn of potential misuse of biometric data.

Company involved

Bucheon City Government

AI system involved

Smart epidemiological investigation system

10 source articles · read the reporting →

FTC charges Ring with illegal surveillance and security failures

The Federal Trade Commission charged Ring, a home security camera company, with compromising customer privacy by allowing employees to access private videos and failing to prevent hackers from taking control of cameras. Hackers accessed approximately 55,000 U.S. customers' accounts, harassing individuals including children and the elderly. The proposed order requires Ring to pay $5.8 million in refunds and implement security measures.

Company involved

Ring LLC

AI system involved

Ring cameras

10 source articles · read the reporting →