State ex rel. Steven Schnell v. Honorable Rebecca Richardson (CA Missouri): AI-hallucinated content in court filing, Monetary Sanction
The AI-generated fictitious case citations misled the court and caused opposing counsel to incur attorney's fees.
- Company involved
- Jenna Rohr Conley
1 source article · read the reporting →
DFFH breaches privacy by using ChatGPT in child protection report
A child protection worker at the Department of Families, Fairness and Housing (DFFH) used ChatGPT to draft a Protection Application Report for the Children’s Court, entering sensitive personal information about a child. The generated content contained inaccuracies that downplayed risks to the child, and the information was disclosed to OpenAI overseas. An investigation by the Office of the Victorian Information Commissioner found DFFH failed to ensure accuracy and protect personal information, contravening IPPs 3.1 and 4.1. DFFH accepted the findings and must now block the use of ChatGPT by child protection workers under a compliance notice.
- Company involved
- Department of Families, Fairness and Housing
- AI system involved
- ChatGPT
9 source articles · read the reporting →
Consumer Reports calls for action against Meta over scam ad proliferation
Consumer Reports has called on the FTC and state attorneys general to take enforcement action against Meta for allowing billions of scam advertisements on its platforms. According to a Reuters investigation, Meta's own documents showed that the company delivered an estimated 15 billion scam ads per day in 2024, generating billions of dollars in revenue. Meta is accused of failing to identify and remove most scam ads, exposing users to fraudulent schemes and illegal products. Consumer Reports argues that Meta's practices constitute unfair business practices under the FTC Act and state laws.
- Company involved
- Meta
6 source articles · read the reporting →
Geico and other insurers charge higher premiums in minority neighborhoods than in white areas with similar risk
ProPublica's analysis of auto insurance premiums in California, Illinois, Texas and Missouri found that major insurers such as Geico, Allstate and Liberty Mutual charge minority neighborhoods up to 30% more than white areas with similar accident costs. The disparity is alleged to result from pricing algorithms that may inadvertently use zip code as a proxy for race. Otis Nash, a 26-year-old Chicagoan, pays $190.69 per month for Geico insurance while a white driver in a safer neighborhood pays $54.67. The insurers deny discrimination, stating they do not collect race data.
- Company involved
- Geico
5 source articles · read the reporting →
FTC charges Ring with illegal surveillance and security failures
The Federal Trade Commission charged Ring, a home security camera company, with compromising customer privacy by allowing employees to access private videos and failing to prevent hackers from taking control of cameras. Hackers accessed approximately 55,000 U.S. customers' accounts, harassing individuals including children and the elderly. The proposed order requires Ring to pay $5.8 million in refunds and implement security measures.
- Company involved
- Ring LLC
- AI system involved
- Ring cameras
10 source articles · read the reporting →
Ningbo fines property developers for using facial recognition without consent
In Ningbo, China, property developers were fined for using facial recognition technology to identify customers without their consent. The enforcement followed China's annual Consumer Protection Gala that highlighted misuse of facial recognition. The fines were imposed by the local government and the developers were accused of violating privacy regulations.
- Company involved
- Multiple property developers in Ningbo
- AI system involved
- Facial recognition technology
6 source articles · read the reporting →
CNAF algorithm flags single mother Juliette for welfare fraud investigation
In 2022, Juliette, a single mother on welfare in France, was flagged by CNAF's secretive fraud detection algorithm. A fraud investigator later determined she owed thousands of euros, which were deducted from her monthly payments. The algorithm, which scores half of France's population, is accused of discriminating against vulnerable people by using factors like single parenthood and low income.
- Company involved
- CNAF
10 source articles · read the reporting →
Tinder accused of age discrimination in Tinder Plus pricing
A CHOICE mystery shop found Tinder charging users over 30 up to five times more for Tinder Plus than younger users, with prices ranging from $6.99 to $34.37 per month. An anti-discrimination law expert says this likely breaches Australia's Age Discrimination Act 2004. Tinder did not respond to CHOICE's requests for comment, but settled a similar US lawsuit by stopping age-based pricing in California and compensating affected users.
- Company involved
- Tinder
- AI system involved
- Tinder Plus
4 source articles · read the reporting →
Whitebridge AI faces complaint over false reputation reports
Whitebridge AI, a Lithuanian company, is accused of generating false reputation reports using unlawfully scraped social media data. The reports contained false warnings for 'sexual nudity' and 'dangerous political content'. Privacy group Noyb filed a complaint with the Lithuanian data protection authority, alleging violations of the GDPR. The complainants sought access to their data but received no response, and were later required to provide a qualified electronic signature to correct errors.
- Company involved
- Whitebridge AI
6 source articles · read the reporting →
Oregon child welfare agency stops using algorithm to screen families for investigations
The Oregon Department of Human Services announced it will stop using its Safety at Screening Tool algorithm, which helped hotline workers decide which families to investigate for child abuse and neglect. The decision followed concerns about racial bias and disparities, as the algorithm was inspired by a Pennsylvania tool that had flagged a disproportionate number of Black children. The agency will replace the algorithm with a new Structured Decision Making model.
- Company involved
- Oregon Department of Human Services
- AI system involved
- Safety at Screening Tool
10 source articles · read the reporting →
Instacart's rating and fraud detection system unfairly penalizes shoppers
Instacart shoppers allege that the company's automated rating and fraud detection system unfairly penalizes them for issues outside their control, such as customer fraud. The system assigns low ratings and blames shoppers for missing items, leading to reduced pay and deactivation. The Gig Workers Collective has sent an open letter to CEO Fidji Simo demanding changes including higher base pay, item commission, and occupational death benefits.
- Company involved
- Instacart
10 source articles · read the reporting →
Ring hacker yelled obscenities at 8-year-old girl via security camera
In December 2019, a hacker took control of a Ring security camera in a Mississippi home and used it to yell obscenities at an 8-year-old girl. Ring disclosed that four employees had improperly accessed footage over four years. The company faced lawsuits for failing to protect users. Ring implemented two-factor authentication for new devices but not existing ones.
- Company involved
- Ring
- AI system involved
- Ring security camera
10 source articles · read the reporting →
Inflection background check errors lead to lawsuits from banned Airbnb users
Inflection, a background check company used by Airbnb, Uber, and other platforms, has been sued multiple times for providing inaccurate reports. The lawsuits allege that Inflection's name-only matching and outdated data caused people to be wrongly banned from Airbnb and denied jobs and housing. Some cases have been settled, while others remain ongoing. The company is accused of violating the Fair Credit Reporting Act.
- Company involved
- Inflection
- AI system involved
- Inflection background check
8 source articles · read the reporting →
Tenant screening software denies housing to disabled Latino man in Connecticut
In 2016, Carmen Arroyo requested to move her disabled son Mikhail into a larger apartment in the same complex. The landlord, WinnResidential, used CoreLogic's CrimSafe tenant screening software, which flagged a dropped retail theft charge against Mikhail as disqualifying. The landlord rejected the application without explanation. Arroyo sued CoreLogic and WinnResidential under the Fair Credit Reporting Act and the Fair Housing Act, alleging that the software disproportionately excludes people of colour. The case was scheduled for trial in August 2021.
- Company involved
- WinnResidential
- AI system involved
- CrimSafe
10 source articles · read the reporting →
Privacy group files complaint against proctoring companies over AI surveillance
The Electronic Privacy Information Center (EPIC) filed a complaint with the District of Columbia attorney general against five exam proctoring companies—Respondus, ProctorU, Proctorio, Examity, and Honorlock—alleging unfair and deceptive business practices. The companies are accused of using opaque AI algorithms to flag cheating and collecting excessive biometric data from students. Students have reported that the facial recognition software fails to detect dark skin tones, preventing them from taking exams. EPIC has also sent demand letters threatening civil suits if the companies do not stop their practices.
- Company involved
- Proctorio, Respondus, ProctorU, Examity, Honorlock
10 source articles · read the reporting →
Dutch Tax Authority discriminated against dual nationality applicants for child care benefits using automated risk system.
The Dutch Data Protection Authority (AP) found that the Benefits Department of the Tax and Customs Administration (Belastingdienst) unlawfully processed the dual nationality of 1.4 million applicants for child care benefits. The data was used as an indicator in an automated risk-scoring system that flagged applications as high-risk, which was unnecessary and discriminatory. The AP concluded that the processing violated the GDPR and was discriminatory. The AP completed its fact-finding and will consider imposing a sanction.
- Company involved
- Belastingdienst
10 source articles · read the reporting →
Appen criticised for asking job candidates to take 'paper bag test' about skin colour
Appen, an Australian AI company, is accused of racism after asking job candidates to select their skin complexion as part of an online application. Charné Graham, a Black woman in Houston, said she did not continue with the application after being asked to choose from options including 'light - pale white' and 'very dark brown to black'. Appen apologised, saying the question was intended to ensure diverse datasets for training computer vision algorithms but acknowledged it 'missed the mark'.
- Company involved
- Appen
10 source articles · read the reporting →
Roblox's AI age verification misidentifies users' ages
Roblox launched an AI-powered age verification system that estimates users' ages from facial scans. The system has misidentified many users, placing children in adult categories and adults in teen categories, and has been tricked with avatars or altered faces. Users report loss of chat access and privacy concerns, and age-verified accounts are being sold online. Roblox acknowledged the issues and said it is working on solutions.
- Company involved
- Roblox
- AI system involved
- AI-powered age verification system
5 source articles · read the reporting →
Vrije Universiteit Amsterdam accused of racial discrimination by anti-cheating software
A female student with dark skin at Vrije Universiteit Amsterdam alleges that the Proctorio anti-cheating software used during online exams in the 2020/21 academic year discriminated against her on the basis of race. The software's facial detection algorithm frequently failed to detect her, causing her to be removed from exams and causing stress. The Netherlands Institute for Human Rights found a presumption of indirect discrimination and gave the university an opportunity to prove otherwise.
- Company involved
- Stichting Vrije Universiteit
- AI system involved
- Proctorio
10 source articles · read the reporting →
Adoption algorithm Family-Match produced limited results in multiple states
The AI-powered adoption algorithm Family-Match, developed by Adoption-Share and designed by former eharmony researchers, has produced limited results in states where it was used. Social workers reported that the tool often matched children with unwilling families and failed to generate successful adoptions. Virginia, Georgia, and Tennessee dropped or paused the algorithm after trials, though some later resumed use. The algorithm's creator, Thea Ramirez, disputed the accounts and said the tool is a starting point for social workers.
- Company involved
- Adoption-Share
- AI system involved
- Family-Match
7 source articles · read the reporting →
Washington Post uses algorithm to set subscription prices based on personal data
The Washington Post is accused of using an algorithm to set subscription prices based on subscribers' personal data. Some subscribers received emails stating that their new price was determined by an algorithm using their personal data. The practice, known as surveillance pricing, has drawn criticism from Democratic lawmakers who have proposed legislation to ban it.
- Company involved
- Washington Post
6 source articles · read the reporting →
Illinois DCFS ends unreliable predictive analytics program for child abuse risk
The Illinois Department of Children and Family Services ended its Rapid Safety Feedback program, which used data mining to predict child abuse risk, after the agency's director called the technology unreliable. The system, developed by Eckerd Connects and Mindshare Technology, assigned risk scores to children but failed to flag several high-profile child deaths. The program was also criticised for overwhelming caseworkers with alerts and for potential bias against poor children of colour. DCFS decided not to renew the contract.
- Company involved
- Illinois Department of Children and Family Services
- AI system involved
- Rapid Safety Feedback
10 source articles · read the reporting →
DOJ examines Allegheny County child welfare AI screening tool for disability bias
The Justice Department is scrutinizing the Allegheny Family Screening Tool, an AI system used by Allegheny County, Pennsylvania, to assess risk for families reported for child welfare concerns. A father with an intellectual disability, represented by attorney Robin Frank, filed a complaint alleging the algorithm discriminates against families with disabilities by using disability-related data points. The county denies awareness of concerns and says the tool helps predict risk, but critics argue it hardens bias. The DOJ's Civil Rights Division is investigating.
- Company involved
- Allegheny County Department of Human Services
- AI system involved
- Allegheny Family Screening Tool
5 source articles · read the reporting →
Call Vendors Skirt Wiretap Suit Over AI Transcription Tool - Law360
The system recorded and analyzed patient phone calls without their consent.
- Company involved
- Heartland Dental LLC
1 source article · read the reporting →