Lawsuit claims facial recognition program led to wrongful arrest - FOX 2
Identified Christopher Gatlin as a suspect in an assault, leading to his arrest and 17 months in jail.
- Company involved
- St. Louis County Police Department
- AI system involved
- Rekognition
1 source article · read the reporting →
NHTSA investigation PE24015: Rear-end Collisions Involving Zoox Vehicles
Automated driving system controlled vehicle braking and acceleration, leading to rear-end collisions with other road users
- Company involved
- Zoox
- AI system involved
- Zoox vehicle
1 source article · read the reporting →
Allegheny County child neglect screening algorithm shows racial disparities
The Allegheny Family Screening Tool, an algorithm used to screen child neglect reports, has been found to flag a disproportionate number of Black children for mandatory investigation compared to white children. A technical glitch also presented wrong risk scores for over two years. County officials say social workers can override the algorithm and that the glitch has been fixed, but critics argue the tool hardens racial disparities and lacks transparency.
- Company involved
- Allegheny County Department of Human Services
- AI system involved
- Allegheny Family Screening Tool
10 source articles · read the reporting →
Pearl Gardner v. Nationstar Mortgage LLC (D. Arizona): AI-hallucinated content in court filing, Bar Referral
Pro Se Litigant | D. Arizona | State(s): USA. AI tool: Clearpoint. Problem: Fabricated: Case Law. Consequence: Bar Referral.
1 source article · read the reporting →
Airbnb bans users in Australia using trustworthiness algorithm
Airbnb is accused of using an algorithm acquired from Trooly to score users' trustworthiness based on publicly available data, including social media and occupation. Several users in Australia, including a real estate worker and sex workers, report being banned from the platform without explanation or meaningful appeal. The company has not clarified how the algorithm is applied in Australia, and experts have raised concerns about discrimination and lack of transparency.
- Company involved
- Airbnb
- AI system involved
- Trooly
4 source articles · read the reporting →
Spinvox accused of using human transcribers for voice messages
Spinvox, a UK voice-to-text firm, is accused of using call centre staff in South Africa and the Philippines to transcribe the majority of user messages, contrary to claims of automated speech recognition. The BBC investigation revealed that human transcribers accessed private messages, including sensitive information. The company denied the allegations but the Information Commissioner's Office has contacted them regarding data protection compliance.
- Company involved
- Spinvox
- AI system involved
- D2 (the Brain)
4 source articles · read the reporting →
New York federal court sanctions lawyer for non-existent quotations after ChatGPT research
In Tiekert v. Village of Mamaroneck in the Southern District of New York, the court found that the plaintiff's lawyer filed briefs containing non-existent quotations and misrepresented authorities. In a filing to the court, the lawyer said ChatGPT had been used to assist with legal research. On September 14, 2026, the court imposed Rule 11 sanctions: $500 payable to the court and two New York continuing legal education credits in ethics on the use of AI.
2 source articles · read the reporting →
Capital Standard, LLC v. U.S. Bank National Association (CA Florida (2d)): AI-hallucinated content in court filing, Monetary Sanction; Adverse Costs…
The AI generated false legal citations that were included in a court filing, misleading the court.
1 source article · read the reporting →
DFFH breaches privacy by using ChatGPT in child protection report
A child protection worker at the Department of Families, Fairness and Housing (DFFH) used ChatGPT to draft a Protection Application Report for the Children’s Court, entering sensitive personal information about a child. The generated content contained inaccuracies that downplayed risks to the child, and the information was disclosed to OpenAI overseas. An investigation by the Office of the Victorian Information Commissioner found DFFH failed to ensure accuracy and protect personal information, contravening IPPs 3.1 and 4.1. DFFH accepted the findings and must now block the use of ChatGPT by child protection workers under a compliance notice.
- Company involved
- Department of Families, Fairness and Housing
- AI system involved
- ChatGPT
9 source articles · read the reporting →
Target's pregnancy prediction algorithm reveals teen's pregnancy to father
Target used a data mining algorithm to predict when customers were pregnant based on their purchases. The algorithm sent baby product coupons to a high school student, leading her father to complain to the store. The father later learned his daughter was indeed pregnant. Target subsequently altered the coupons to disguise the data mining.
- Company involved
- Target
- AI system involved
- Target pregnancy prediction algorithm
9 source articles · read the reporting →
Geico and other insurers charge higher premiums in minority neighborhoods than in white areas with similar risk
ProPublica's analysis of auto insurance premiums in California, Illinois, Texas and Missouri found that major insurers such as Geico, Allstate and Liberty Mutual charge minority neighborhoods up to 30% more than white areas with similar accident costs. The disparity is alleged to result from pricing algorithms that may inadvertently use zip code as a proxy for race. Otis Nash, a 26-year-old Chicagoan, pays $190.69 per month for Geico insurance while a white driver in a safer neighborhood pays $54.67. The insurers deny discrimination, stating they do not collect race data.
- Company involved
- Geico
5 source articles · read the reporting →
Instawork app penalizes workers for joining strikes, union files complaint
Unite Here Local 11 filed an unfair labor practice complaint with the National Labor Relations Board against Instawork, a staffing app. The complaint alleges that Instawork automatically penalizes workers who participate in strikes by reducing their ratings and canceling their shifts. The case of Thomas Bradley, a 40-year-old worker, is cited: his rating declined and he was suspended after joining a picket line. Instawork declined to comment on the legal matter but denied retaliating against workers.
- Company involved
- Instawork
- AI system involved
- Instawork
10 source articles · read the reporting →
Tinder accused of age discrimination in Tinder Plus pricing
A CHOICE mystery shop found Tinder charging users over 30 up to five times more for Tinder Plus than younger users, with prices ranging from $6.99 to $34.37 per month. An anti-discrimination law expert says this likely breaches Australia's Age Discrimination Act 2004. Tinder did not respond to CHOICE's requests for comment, but settled a similar US lawsuit by stopping age-based pricing in California and compensating affected users.
- Company involved
- Tinder
- AI system involved
- Tinder Plus
4 source articles · read the reporting →
Whitebridge AI faces complaint over false reputation reports
Whitebridge AI, a Lithuanian company, is accused of generating false reputation reports using unlawfully scraped social media data. The reports contained false warnings for 'sexual nudity' and 'dangerous political content'. Privacy group Noyb filed a complaint with the Lithuanian data protection authority, alleging violations of the GDPR. The complainants sought access to their data but received no response, and were later required to provide a qualified electronic signature to correct errors.
- Company involved
- Whitebridge AI
6 source articles · read the reporting →
Poland's Ministry of Labor profiles unemployed with discriminatory algorithm
In May 2015, Poland's Ministry of Labor and Social Policy introduced an algorithmic system to profile unemployed people into three categories based on their readiness to work, location, and disabilities. The system is accused of being discriminatory and lacking transparency, as it uses default criteria that may lead to social exclusion. A report by the Panoptykon Foundation in October 2015 highlighted these issues and recommended reforms, including granting the right to appeal.
- Company involved
- Ministry of Labor and Social Policy
10 source articles · read the reporting →
Upstart Holdings sued for securities fraud over AI model claims
Upstart Holdings, Inc., a cloud-based AI lending platform, is accused of making false and misleading statements about its AI model's ability to account for macroeconomic factors. The lawsuit, filed on May 13, 2022, alleges that the company's positive statements about its business were materially false. Investors suffered losses when Upstart's stock price fell 56% on May 9, 2022, after the company reduced its fiscal 2022 guidance.
- Company involved
- Upstart Holdings, Inc.
- AI system involved
- Upstart AI lending platform
10 source articles · read the reporting →
Oregon child welfare agency stops using algorithm to screen families for investigations
The Oregon Department of Human Services announced it will stop using its Safety at Screening Tool algorithm, which helped hotline workers decide which families to investigate for child abuse and neglect. The decision followed concerns about racial bias and disparities, as the algorithm was inspired by a Pennsylvania tool that had flagged a disproportionate number of Black children. The agency will replace the algorithm with a new Structured Decision Making model.
- Company involved
- Oregon Department of Human Services
- AI system involved
- Safety at Screening Tool
10 source articles · read the reporting →
Inflection background check errors lead to lawsuits from banned Airbnb users
Inflection, a background check company used by Airbnb, Uber, and other platforms, has been sued multiple times for providing inaccurate reports. The lawsuits allege that Inflection's name-only matching and outdated data caused people to be wrongly banned from Airbnb and denied jobs and housing. Some cases have been settled, while others remain ongoing. The company is accused of violating the Fair Credit Reporting Act.
- Company involved
- Inflection
- AI system involved
- Inflection background check
8 source articles · read the reporting →
Tenant screening software denies housing to disabled Latino man in Connecticut
In 2016, Carmen Arroyo requested to move her disabled son Mikhail into a larger apartment in the same complex. The landlord, WinnResidential, used CoreLogic's CrimSafe tenant screening software, which flagged a dropped retail theft charge against Mikhail as disqualifying. The landlord rejected the application without explanation. Arroyo sued CoreLogic and WinnResidential under the Fair Credit Reporting Act and the Fair Housing Act, alleging that the software disproportionately excludes people of colour. The case was scheduled for trial in August 2021.
- Company involved
- WinnResidential
- AI system involved
- CrimSafe
10 source articles · read the reporting →
Privacy group files complaint against proctoring companies over AI surveillance
The Electronic Privacy Information Center (EPIC) filed a complaint with the District of Columbia attorney general against five exam proctoring companies—Respondus, ProctorU, Proctorio, Examity, and Honorlock—alleging unfair and deceptive business practices. The companies are accused of using opaque AI algorithms to flag cheating and collecting excessive biometric data from students. Students have reported that the facial recognition software fails to detect dark skin tones, preventing them from taking exams. EPIC has also sent demand letters threatening civil suits if the companies do not stop their practices.
- Company involved
- Proctorio, Respondus, ProctorU, Examity, Honorlock
10 source articles · read the reporting →
Dutch Tax Authority discriminated against dual nationality applicants for child care benefits using automated risk system.
The Dutch Data Protection Authority (AP) found that the Benefits Department of the Tax and Customs Administration (Belastingdienst) unlawfully processed the dual nationality of 1.4 million applicants for child care benefits. The data was used as an indicator in an automated risk-scoring system that flagged applications as high-risk, which was unnecessary and discriminatory. The AP concluded that the processing violated the GDPR and was discriminatory. The AP completed its fact-finding and will consider imposing a sanction.
- Company involved
- Belastingdienst
10 source articles · read the reporting →
Appen criticised for asking job candidates to take 'paper bag test' about skin colour
Appen, an Australian AI company, is accused of racism after asking job candidates to select their skin complexion as part of an online application. Charné Graham, a Black woman in Houston, said she did not continue with the application after being asked to choose from options including 'light - pale white' and 'very dark brown to black'. Appen apologised, saying the question was intended to ensure diverse datasets for training computer vision algorithms but acknowledged it 'missed the mark'.
- Company involved
- Appen
10 source articles · read the reporting →
Vrije Universiteit Amsterdam accused of racial discrimination by anti-cheating software
A female student with dark skin at Vrije Universiteit Amsterdam alleges that the Proctorio anti-cheating software used during online exams in the 2020/21 academic year discriminated against her on the basis of race. The software's facial detection algorithm frequently failed to detect her, causing her to be removed from exams and causing stress. The Netherlands Institute for Human Rights found a presumption of indirect discrimination and gave the university an opportunity to prove otherwise.
- Company involved
- Stichting Vrije Universiteit
- AI system involved
- Proctorio
10 source articles · read the reporting →
Adoption algorithm Family-Match produced limited results in multiple states
The AI-powered adoption algorithm Family-Match, developed by Adoption-Share and designed by former eharmony researchers, has produced limited results in states where it was used. Social workers reported that the tool often matched children with unwilling families and failed to generate successful adoptions. Virginia, Georgia, and Tennessee dropped or paused the algorithm after trials, though some later resumed use. The algorithm's creator, Thea Ramirez, disputed the accounts and said the tool is a starting point for social workers.
- Company involved
- Adoption-Share
- AI system involved
- Family-Match
7 source articles · read the reporting →