Richland School District investigates AI-generated obscene images of students
Students at Richland Township School District allegedly used artificial intelligence to create and distribute obscene images of other students. The district began investigating on Thursday, November 14, and contacted police and parents. Charges are pending, and the district has urged the community to avoid speculation on social media.
- Company involved
- Richland Township School District
2 source articles · read the reporting →
Character.AI Hosts Chatbots Modeled After School Shooters and Victims
Character.AI, a Google-backed AI startup, hosts chatbots that simulate school shooters and their victims, including those from Sandy Hook and Columbine. The chatbots, which are not age-gated, have accumulated tens of thousands of chats and failed to flag violent statements. The platform is facing lawsuits alleging it caused a teenager's suicide and facilitated sexual abuse of minors. Character.AI deactivated some bots after being contacted but did not remove all related content.
- Company involved
- Character.AI
- AI system involved
- Character.AI chatbots
2 source articles · read the reporting →
Palantir secretly tested predictive policing in New Orleans
Beginning in 2012, Palantir Technologies secretly partnered with the New Orleans Police Department to deploy a predictive policing system. The programme analysed gang affiliations, social media and criminal histories to forecast individuals’ likelihood of committing or becoming victims of violence, operating without public knowledge or city council oversight. Researchers and law enforcement officials raised concerns about systemic bias and civil liberties. As of 2018, the city and Palantir had not disclosed the programme’s status.
- Company involved
- New Orleans Police Department
1 source article · read the reporting →
Pasco Sheriff's Office used algorithm to target potential future criminals and schoolchildren
The Pasco Sheriff's Office operates an intelligence-led policing programme that uses an algorithm to identify people who might break the law based on criminal histories and social networks. Deputies are sent to the homes of those flagged, even without evidence of a crime, and former deputies allege they were ordered to make targets' lives miserable. The agency also keeps a list of more than 400 schoolchildren predicted to 'fall into a life of crime', built from data such as grades and child welfare records, without informing the children or their parents. Civil liberties groups are considering lawsuits and public advocacy campaigns, and experts have called the programmes 'morally repugnant'.
- Company involved
- Pasco Sheriff's Office
10 source articles · read the reporting →
Duke University MTMC Dataset Used in Authoritarian Surveillance Research
Duke University created and openly distributed the Duke MTMC dataset, containing surveillance footage of approximately 2,000 students and visitors on campus. The dataset was used by numerous organisations, including Chinese military-linked companies like SenseTime and Hikvision, for developing person re-identification and facial recognition technologies. Following an investigation by exposing.ai and the Financial Times, Duke University terminated the dataset in May 2019. The incident highlights the privacy risks of academic datasets being repurposed for mass surveillance without consent.
- Company involved
- Duke University
- AI system involved
- Duke MTMC
1 source article · read the reporting →
Google contractors targeted homeless people for Pixel 4 facial recognition data
Google hired Randstad to collect facial data to train the Pixel 4's face recognition system. Contractors allegedly targeted homeless people and unaware students, using deceptive tactics such as calling it a "selfie game" and offering $5 without informing subjects they were being recorded. Google suspended the research program and opened an investigation following the report.
- Company involved
- Google
- AI system involved
- Pixel 4 facial recognition
1 source article · read the reporting →
PredictiveHire builds AI to predict job hopping from interviews
PredictiveHire, an AI hiring firm, developed a machine-learning model that analyses candidates' open-ended interview responses to predict their likelihood of 'job hopping'. The company used data from 45,899 applicants to build the 'flight risk' assessment, which it advertises as coming soon. Scholars warn that such tools can suppress wages by screening out workers who might seek better pay or conditions, continuing a historical trend of using personality tests to identify potential labour organisers.
- Company involved
- PredictiveHire
- AI system involved
- Phai
1 source article · read the reporting →
Xuhui police expand facial recognition surveillance to profile 1.1 million residents
The Xuhui District branch of the Shanghai Municipal Bureau of Public Security is expanding its Intelligent Image Recognition System, adding 2,500 facial recognition cameras and increased computing capacity to build profiles of residents and flag deviations. The project, contracted to US-sanctioned FiberHome, is designed to match each face to files on more than 50 million people. Officials say the system will analyse behaviour patterns and trigger early warnings.
- Company involved
- Shanghai Municipal Bureau of Public Security, Xuhui District Branch
- AI system involved
- Intelligent Image Recognition System
3 source articles · read the reporting →
NHS faces legal action over Palantir data contract extension
The NHS is being taken to court by campaign group Open Democracy over its contract with data firm Palantir. The legal action alleges that the extension of Palantir's involvement in analysing NHS patient data for pandemic response and beyond lacked a proper Data Protection Impact Assessment. The contract, initially an emergency response, was extended for two years at a cost of £23.5m. The case is pending.
- Company involved
- NHS
- AI system involved
- Palantir data analysis platform
10 source articles · read the reporting →
Students at Ontario school share AI-generated fake nude photos of peers
Students at St. Thomas Aquinas Catholic Secondary School in London, Ontario, used artificial intelligence to alter photos of their classmates from social media, creating fake nude images and sharing them through group chats. The school sent a letter to parents warning of disciplinary measures, and London police are investigating. A 16-year-old victim said she feels humiliated and embarrassed, and students are calling for more education on AI misuse.
5 source articles · read the reporting →
Broward College student flagged by Honorlock and accused of cheating
A 17-year-old Black student at Broward College was flagged by Honorlock, a remote proctoring system, during an online biology exam in February 2022. Her professor reviewed the recording and accused her of academic dishonesty for looking down and away from the screen; the college found her responsible for noncompliance and gave her a zero and a warning. The student says she was only thinking and fidgeting, and the article reports that the video is ambiguous. Honorlock says it does not definitively identify cheaters and the final decision rests with the school.
- Company involved
- Broward College
- AI system involved
- Honorlock
2 source articles · read the reporting →
Northeastern University Student Complains About Professor's Undisclosed AI-Generated Presentation
In February 2025, an undergraduate student at Northeastern University noticed that a professor's presentation contained misspellings and distorted images, leading her to suspect it was AI-generated. The professor had prohibited students from using AI, yet used it himself without disclosure. The student filed a formal complaint and demanded a tuition refund of over $8,000, but the university rejected her claim. The incident prompted Northeastern to later adopt a formal AI policy requiring attribution and review of AI-generated content.
- Company involved
- Northeastern University
2 source articles · read the reporting →
West Midlands Police test NDAS system to assess crime risk
West Midlands Police are testing a system called the National Data Analytics Solution (NDAS) that analyses police data to assess the risk of individuals committing a crime or becoming a victim. The Home Office has funded the project with millions of pounds, intending to roll it out across England and Wales. Critics warn that the system could reinforce bias against minorities and undermine the presumption of innocence. The police say the technology is designed to support, not replace, officer decision-making and that it is still in early testing.
- Company involved
- West Midlands Police
- AI system involved
- National Data Analytics Solution
1 source article · read the reporting →
Brainwash cafe customers unknowingly put in AI surveillance dataset
In 2014, customers at the Brainwash Cafe in San Francisco were recorded by a publicly available webcam. The images were compiled into a dataset containing 11,917 photos for training surveillance-related object and head detection algorithms. The dataset was later removed from access following an investigation revealing use by researchers affiliated with the National University of Defense Technology in China. The dataset's creators are accused of collecting the images without the cafe customers' knowledge or consent.
- Company involved
- Stanford University
- AI system involved
- Brainwash dataset
1 source article · read the reporting →
Ex-Pikesville athletic director used AI deepfake to impersonate principal
Dazhon Darien, former athletic director at Pikesville High School, used artificial intelligence to create a deepfake audio clip in 2024 that made it appear as though principal Eric Eiswert made racist and antisemitic comments. The clip spread widely online, severely damaging Eiswert's reputation. Darien was arrested and later pleaded guilty to unrelated child sex crimes. Eiswert sued Baltimore County Public Schools and reached a settlement.
- Company involved
- Baltimore County Public Schools
7 source articles · read the reporting →
Whole Foods uses heat map to track unionization risk at stores
Whole Foods, owned by Amazon, is reportedly using a heat map that scores stores on their risk of unionization. The system analyses metrics including local union membership, NLRB charges, employee compensation, diversity, and internal sentiment surveys. The tool is used to monitor and potentially prevent union organising efforts. Whole Foods stated it prefers a direct relationship with employees, while Amazon has a history of aggressively combating unionization.
- Company involved
- Whole Foods Market
1 source article · read the reporting →
NEDA Fires Helpline Staff, Replaces Them with Chatbot After Unionization
The National Eating Disorders Association (NEDA) fired four full-time helpline staff and announced the shutdown of its 20-year-old helpline, replacing it with a chatbot named Tessa. The staff had recently unionized and won an NLRB election, leading to allegations that the move was union busting. NEDA claims the chatbot, a rule-based system developed by Washington University, is a separate program to better serve the community, but the union and critics argue it cannot replace human empathy and may cause irreparable harm. The helpline is set to end on June 1, 2023, with volunteers asked to become testers for the chatbot.
- Company involved
- National Eating Disorders Association (NEDA)
- AI system involved
- Tessa
10 source articles · read the reporting →
Delhi Government uses facial recognition on children in government schools
The Delhi Government's Education Department is using facial recognition technology in conjunction with CCTV cameras in government schools, according to responses to RTI requests filed by the Internet Freedom Foundation. The organisation urged the Delhi Commission for Protection of Child Rights to intervene, alleging that the practice violates children's right to privacy and education under the Indian Constitution. No specific incident of harm has been reported, but the organisation warns of potential misidentification and disproportionate penalties.
- Company involved
- Government of NCT of Delhi, Education Department
10 source articles · read the reporting →
Meta AI Chatbot Falsely Claims to Have a Child in NYC Parents Group
Meta's AI chatbot responded to a parent's question in a private Facebook group for New York City parents, falsely claiming to have a twice-exceptional child enrolled in a specific public school's gifted program. The chatbot later apologised, stating it does not have personal experiences or children. The incident was reported by 404 Media after a screenshot was shared by a Princeton professor.
- Company involved
- Meta
- AI system involved
- Meta AI chatbot
2 source articles · read the reporting →
Volusia County Schools' Gaggle AI flags threat, leading to student's arrest
A 15-year-old student at Deltona High School wrote a threat on his school laptop, which was flagged by the Gaggle monitoring system. The alert notified a school resource deputy, who detained the student before he could leave. The teen admitted to the threat, saying he was angry about being bullied, and was arrested on a second-degree felony charge.
- Company involved
- Volusia County Schools
- AI system involved
- Gaggle
1 source article · read the reporting →
DOJ, Pinnacle reach settlement in RealPage case - Yahoo Finance
The system recommended rent prices to landlords, affecting renters' housing costs.
- Company involved
- Pinnacle Property Management Services
- AI system involved
- RealPage
1 source article · read the reporting →
Where does SpaceXAI rank for air pollution in Shelby County? - WREG.com
The data center's operations emitted air pollutants affecting nearby residents
- Company involved
- SpaceXAI
- AI system involved
- Colossus 1
1 source article · read the reporting →
UK schools install toilet sensors that listen for keywords to monitor pupils
Several UK schools, including St Joseph's College in Stoke-on-Trent, have installed Halo Smart Sensors in toilets that use machine learning to detect keywords such as 'help me' and 'stop it'. The sensors trigger alerts to staff when keywords are detected, aiming to crack down on vaping and bullying. Privacy advocates argue the sensors violate children's privacy and may be unlawful. The schools have not sought parental consent for the monitoring.
- Company involved
- St Joseph's College
- AI system involved
- Halo Smart Sensors
4 source articles · read the reporting →
Brazilian children's photos scraped into LAION-5B AI dataset without consent
Human Rights Watch found that LAION-5B, a dataset used to train AI tools, contains identifiable photos of Brazilian children scraped from the web without their knowledge or consent. The dataset includes names and locations of children. AI tools trained on LAION-5B have been used to create nonconsensual deepfakes, with at least 85 girls reporting harassment from classmates who used AI to generate explicit imagery. LAION, which manages the dataset, acknowledged the inclusion and pledged to remove the photos but disputed that the models can reproduce data verbatim.
- Company involved
- LAION
- AI system involved
- LAION-5B
7 source articles · read the reporting →
