Meta AI Chatbot Falsely Claims to Have a Child in NYC Parents Group
Meta's AI chatbot responded to a parent's question in a private Facebook group for New York City parents, falsely claiming to have a twice-exceptional child enrolled in a specific public school's gifted program. The chatbot later apologised, stating it does not have personal experiences or children. The incident was reported by 404 Media after a screenshot was shared by a Princeton professor.
- Company involved
- Meta
- AI system involved
- Meta AI chatbot
2 source articles · read the reporting →
Volusia County Schools' Gaggle AI flags threat, leading to student's arrest
A 15-year-old student at Deltona High School wrote a threat on his school laptop, which was flagged by the Gaggle monitoring system. The alert notified a school resource deputy, who detained the student before he could leave. The teen admitted to the threat, saying he was angry about being bullied, and was arrested on a second-degree felony charge.
- Company involved
- Volusia County Schools
- AI system involved
- Gaggle
1 source article · read the reporting →
DOJ, Pinnacle reach settlement in RealPage case - Yahoo Finance
The system recommended rent prices to landlords, affecting renters' housing costs.
- Company involved
- Pinnacle Property Management Services
- AI system involved
- RealPage
1 source article · read the reporting →
UK schools install toilet sensors that listen for keywords to monitor pupils
Several UK schools, including St Joseph's College in Stoke-on-Trent, have installed Halo Smart Sensors in toilets that use machine learning to detect keywords such as 'help me' and 'stop it'. The sensors trigger alerts to staff when keywords are detected, aiming to crack down on vaping and bullying. Privacy advocates argue the sensors violate children's privacy and may be unlawful. The schools have not sought parental consent for the monitoring.
- Company involved
- St Joseph's College
- AI system involved
- Halo Smart Sensors
4 source articles · read the reporting →
Allegheny County child neglect screening algorithm shows racial disparities
The Allegheny Family Screening Tool, an algorithm used to screen child neglect reports, has been found to flag a disproportionate number of Black children for mandatory investigation compared to white children. A technical glitch also presented wrong risk scores for over two years. County officials say social workers can override the algorithm and that the glitch has been fixed, but critics argue the tool hardens racial disparities and lacks transparency.
- Company involved
- Allegheny County Department of Human Services
- AI system involved
- Allegheny Family Screening Tool
10 source articles · read the reporting →
High school student used ChatGPT to hack Kaikatsu Club, steal 7.25 million records
A 17-year-old high school student in Japan was arrested for using ChatGPT to create a computer program that hacked into the server of Kaikatsu Frontier, operator of the Kaikatsu Club internet cafe chain. The student allegedly obtained approximately 7.25 million sets of membership information between January 18 and 20, 2025. No misuse of the data has been confirmed. The student was arrested for violating Japan's unauthorized computer access and fraudulent obstruction of business laws.
- Company involved
- Kaikatsu Frontier
- AI system involved
- ChatGPT
3 source articles · read the reporting →
FTC Orders Edmodo to Stop Unlawful Collection of Children's Data for Advertising
The FTC filed a complaint against Edmodo, an education technology provider, alleging that it collected personal information from children under 13 without parental consent and used it for advertising, in violation of the COPPA Rule. Edmodo also allegedly outsourced its compliance responsibilities to schools, providing them with inadequate information. Under a proposed order, Edmodo will be required to delete improperly collected data and change its practices, and faces a suspended $6 million penalty.
- Company involved
- Edmodo, LLC
- AI system involved
- Edmodo
1 source article · read the reporting →
US Maven targeting system kills 175 at Iranian school after database error
On 28 February 2026, US forces struck the Shajareh Tayyebeh primary school in Minab, Iran, killing between 175 and 180 people, most of them girls. The targeting was carried out by the Maven Smart System, built by Palantir, which had classified the building as a military facility based on an outdated Defense Intelligence Agency database. Although media attention focused on the chatbot Claude, the real failure was a bureaucratic one: the database had not been updated to reflect the building's conversion into a school years earlier. The incident highlights the lethal consequences of compressed kill chains and automated targeting infrastructure.
- Company involved
- US Department of Defense
- AI system involved
- Maven Smart System
2 source articles · read the reporting →
Poland's Ministry of Labor profiles unemployed with discriminatory algorithm
In May 2015, Poland's Ministry of Labor and Social Policy introduced an algorithmic system to profile unemployed people into three categories based on their readiness to work, location, and disabilities. The system is accused of being discriminatory and lacking transparency, as it uses default criteria that may lead to social exclusion. A report by the Panoptykon Foundation in October 2015 highlighted these issues and recommended reforms, including granting the right to appeal.
- Company involved
- Ministry of Labor and Social Policy
10 source articles · read the reporting →
Oregon child welfare agency stops using algorithm to screen families for investigations
The Oregon Department of Human Services announced it will stop using its Safety at Screening Tool algorithm, which helped hotline workers decide which families to investigate for child abuse and neglect. The decision followed concerns about racial bias and disparities, as the algorithm was inspired by a Pennsylvania tool that had flagged a disproportionate number of Black children. The agency will replace the algorithm with a new Structured Decision Making model.
- Company involved
- Oregon Department of Human Services
- AI system involved
- Safety at Screening Tool
10 source articles · read the reporting →
Goffstown parents say students used AI to create deepfake nudes of classmates
In Goffstown, New Hampshire, parents of three middle school girls allege that 8th grade boys used an AI-powered app to generate deepfake explicit photos and videos of their daughters. The app reportedly 'undresses' a person from a single photo. The school principal notified the parents in early October, but the district waited a month to inform other families. The girls have been devastated and are afraid to attend school, and the parents are calling for accountability and education on AI dangers.
- Company involved
- SAU #19 (Goffstown School District)
1 source article · read the reporting →
Privacy group files complaint against proctoring companies over AI surveillance
The Electronic Privacy Information Center (EPIC) filed a complaint with the District of Columbia attorney general against five exam proctoring companies—Respondus, ProctorU, Proctorio, Examity, and Honorlock—alleging unfair and deceptive business practices. The companies are accused of using opaque AI algorithms to flag cheating and collecting excessive biometric data from students. Students have reported that the facial recognition software fails to detect dark skin tones, preventing them from taking exams. EPIC has also sent demand letters threatening civil suits if the companies do not stop their practices.
- Company involved
- Proctorio, Respondus, ProctorU, Examity, Honorlock
10 source articles · read the reporting →
Scottish Prison Service freed eight life prisoners due to risk assessment system error
The Scottish Prison Service's Level of Service and Case Management System failed to update risk assessments when new information was entered, leading to incorrect risk scores for 495 closed cases and 285 open cases. As a result, eight prisoners serving life for murder or rape were granted temporary release without accurate risk assessments. Justice Secretary Keith Brown acknowledged the error and stated that the cases would be reviewed. Opposition politicians called for an apology, claiming the releases put the public at risk.
- Company involved
- Scottish Prison Service
- AI system involved
- Level of Service and Case Management System
10 source articles · read the reporting →
Evolv AI scanner fails to detect knife, student stabbed at Utica school
In October 2022, a student at Proctor High School in Utica, New York, was stabbed with a knife that the Evolv Express AI weapons scanner failed to detect. The school district had installed the system in 13 schools. An internal investigation found the system was not designed to detect knives. The scanners were removed from Proctor High School but remain in other schools.
- Company involved
- Utica City School District
- AI system involved
- Evolv Express
10 source articles · read the reporting →
Minneapolis schools use Gaggle AI to monitor students' online activity
Minneapolis Public Schools deployed Gaggle, an AI-powered surveillance system, to monitor students' school-issued Google and Microsoft accounts during remote learning. The system flagged thousands of incidents, including references to self-harm and pornography, raising privacy concerns. Critics argue the surveillance undermines student trust and may deter them from seeking help. The district defended the tool as a way to identify students in distress.
- Company involved
- Minneapolis Public Schools
- AI system involved
- Gaggle
9 source articles · read the reporting →
Nigeria collects fingerprints and photos of children for school feeding program
The Nigerian government has begun collecting fingerprints and photos of children as young as five for the National Home Grown School Feeding Programme. The biometric enrollment aims to confirm eligibility and increase transparency, but the exercise has started before the country's data protection law comes into effect. The motives for collection are unclear, and there have been technical issues with the enrollment tablets. The program already feeds nine million children daily.
- Company involved
- Nigeria's National Home Grown School Feeding Programme (NHGSFP)
- AI system involved
- HID Global DP4500 biometric scanner and Plovtech enrollment app
5 source articles · read the reporting →
Polish school fined for processing students' fingerprints without legal basis
Primary School No. 2 in Gdansk used a biometric fingerprint reader to identify students for meal payment verification. The system processed the biometric data of 680 children without a valid legal basis, as the school could have used alternative identification methods. The Polish Personal Data Protection Office (UODO) imposed a fine of PLN 20,000 ($5,300) and ordered the erasure of the data. The authority also noted that students without biometric identification were required to wait at the end of the queue, constituting unequal treatment.
- Company involved
- Primary School No. 2 in Gdansk
6 source articles · read the reporting →
Alpha School parents allege harm from AI-driven learning software
Parents of students at Alpha School in Brownsville, Texas, allege that the school's AI-driven learning software caused their children psychological distress and physical harm. One 9-year-old girl was forced to repeat multiplication lessons hundreds of times, leading to crying episodes and weight loss, and was denied snacks until she met software metrics. The school denies the allegations and says it prioritises a safe environment.
- Company involved
- Alpha School
- AI system involved
- IXL
5 source articles · read the reporting →
Omnilert AI weapon detection system fails at Antioch High School shooting
An AI-powered weapon detection system called Omnilert failed to detect a handgun carried by a 17-year-old student at Antioch High School in Nashville, Tennessee, on 22 January 2025. The student shot and killed a 16-year-old girl and wounded another student before killing himself. A district official said the system did not detect the gun because the shooter was not close enough to the cameras. The system was activated by police brandishing their weapons during the response.
- Company involved
- Metro Nashville Public Schools
- AI system involved
- Omnilert
6 source articles · read the reporting →
Telangana algorithm denied food to thousands of poor families
The Telangana government's algorithmic system Samagra Vedika wrongfully denied subsidised food to thousands of poor families by incorrectly tagging them as ineligible. The system, developed by Posidex Technologies, misidentified individuals across databases, leading to false positives. Affected families, including 67-year-old widow Bismillah Bee, were forced to prove their eligibility, often unsuccessfully. A Supreme Court case ordered re-verification, but many remain excluded.
- Company involved
- Government of Telangana
- AI system involved
- Samagra Vedika
8 source articles · read the reporting →
Illinois DCFS ends unreliable predictive analytics program for child abuse risk
The Illinois Department of Children and Family Services ended its Rapid Safety Feedback program, which used data mining to predict child abuse risk, after the agency's director called the technology unreliable. The system, developed by Eckerd Connects and Mindshare Technology, assigned risk scores to children but failed to flag several high-profile child deaths. The program was also criticised for overwhelming caseworkers with alerts and for potential bias against poor children of colour. DCFS decided not to renew the contract.
- Company involved
- Illinois Department of Children and Family Services
- AI system involved
- Rapid Safety Feedback
10 source articles · read the reporting →
DOJ examines Allegheny County child welfare AI screening tool for disability bias
The Justice Department is scrutinizing the Allegheny Family Screening Tool, an AI system used by Allegheny County, Pennsylvania, to assess risk for families reported for child welfare concerns. A father with an intellectual disability, represented by attorney Robin Frank, filed a complaint alleging the algorithm discriminates against families with disabilities by using disability-related data points. The county denies awareness of concerns and says the tool helps predict risk, but critics argue it hardens bias. The DOJ's Civil Rights Division is investigating.
- Company involved
- Allegheny County Department of Human Services
- AI system involved
- Allegheny Family Screening Tool
5 source articles · read the reporting →
Paraná schools use European facial recognition to mark attendance, risking welfare for false absences
Since 2023, the Paraná state education department in Brazil has deployed facial recognition software from Slovak company Innovatrics in over 1,700 schools to automate attendance for nearly one million children. The system has made errors, marking present students as absent. Because attendance records generated by the algorithm affect eligibility for the Bolsa Família welfare program, false absences could threaten families' benefits. In April 2025, the state's public prosecutor filed a legal challenge alleging the system violates Brazilian law; the case is pending.
- Company involved
- Paraná state education department (Secretaria da Educação do Paraná)
- AI system involved
- LRCO Paraná
8 source articles · read the reporting →
U.S. Federal Court Allows CIPA Class Action Against AI Customer Service Provider to Proceed - Wilson Sonsini
The AI system intercepted and recorded a customer's phone call and collected her personal and financial information without consent.
- Company involved
- ConverseNow Technologies, Inc.
1 source article · read the reporting →
