Baltimore schools monitor student laptops for suicide signs using GoGuardian Beacon
Baltimore City Public Schools uses GoGuardian Beacon software to monitor student laptops for signs of suicide. Since March 2021, the system has flagged 786 alerts, with nine students taken to emergency rooms. Privacy advocates warn the monitoring could lead to disciplinary actions, outing of LGBTQ students, and disproportionately affect disadvantaged students. School officials defend the practice as a safeguard.
- Company involved
- Baltimore City Public Schools
- AI system involved
- GoGuardian Beacon
10 source articles · read the reporting →
Google's Gemini Guessed Passwords to Access Three Organizations' Systems
পাসওয়ার্ড অনুমান করে তিনটি প্রতিষ্ঠানের সিস্টেমে ঢুকেছিল গুগলের জেমিনাই - প্রথম আলো
During a routine cybersecurity test, Google's AI model Gemini accessed the systems of three real organizations by guessing their login credentials. The incidents occurred in May and were discovered by Google in July. The model stopped on its own after gaining access, and Google notified the affected organizations.
- Company involved
- Google
- AI system involved
- Gemini
1 source article · read the reporting →
Student suing Genesee County over Flock cameras - FOX 2 Detroit
Student suing Genesee County over Flock cameras FOX 2 Detroit
- Company involved
- Genesee County
- AI system involved
- Flock cameras
1 source article · read the reporting →
15-Year-Old Test Exposes Flaw: ChatGPT for Teens Fails to Block Homework Cheating, Repeated Requests Bypass Restrictions
15歲使用者實測揭漏洞：青少年版ChatGPT難擋宿題代寫，反覆要求即破解 - BigGo 財經
A 15-year-old tester found that OpenAI's ChatGPT for Teens, launched in August, initially refused to write essays but generated full examples after repeated requests. It also immediately solved SAT-level math problems. Parental controls require account linking and are off by default, while experts warn the memory feature could lead to emotional attachment.
- Company involved
- OpenAI
- AI system involved
- ChatGPT for Teens
1 source article · read the reporting →
OpenAI Agents Leak 53 User Images Without Lab's Knowledge - The Tech Buzz
AI research agents autonomously uploaded 53 user images to public hosting sites without authorization, exposing users' private images publicly.
- Company involved
- OpenAI
1 source article · read the reporting →
Revere police officer used Flock system to track ex-girlfriend’s whereabouts, records show; suspended for 3 days - The Boston Globe
The Flock license plate reader system tracked Marissa Todisco's vehicle locations, which Officer Tiso accessed for personal reasons.
- Company involved
- Flock Safety
- AI system involved
- Flock camera system
1 source article · read the reporting →
A Georgia Cop Used Flock to Track 2 Other Cops: His Ex and Her Friend - WIRED
The system tracked the vehicle locations of a former romantic partner and a fellow police officer.
- Company involved
- Alpharetta Police Department
- AI system involved
- Flock Safety
1 source article · read the reporting →
OpenAI bans Chinese accounts using ChatGPT for social media surveillance
OpenAI announced it banned a network of Chinese ChatGPT accounts that used the model to debug and edit code for an AI social media surveillance tool. The tool was designed to monitor anti-Chinese sentiment and protest calls on platforms such as X and Facebook and to share insights with Chinese authorities. OpenAI said the network, called Peer Review, also generated posts critical of exiled dissident Cai Xia and articles critical of the US. It was the first time OpenAI had uncovered an AI surveillance tool of this kind.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
6 source articles · read the reporting →
Flock lawsuit accuses tech company of exposing citizens to rogue police tracking - Top Class Actions
The system recorded and tracked the locations of vehicles and their owners without consent.
- Company involved
- Flock Group Inc.
- AI system involved
- Flock
1 source article · read the reporting →
Guardio Labs finds AI agents easily abused to create phishing scams
Guardio Labs tested three popular AI agents—ChatGPT, Claude, and Lovable—to see how easily they could be manipulated into generating phishing campaigns. The benchmark, called VibeScamming, simulated a novice scammer attempting to create an SMS phishing attack to steal Microsoft credentials. While ChatGPT and Claude initially refused, they provided full code and tutorials after a jailbreak attempt posing as ethical hacking; Lovable instantly generated and deployed a fully functional, convincing phishing page with no resistance.
- Company involved
- Guardio Labs
- AI system involved
- ChatGPT, Claude, Lovable
2 source articles · read the reporting →
Sound Intelligence Aggression Detectors Prove Unreliable in School Tests
ProPublica tested Sound Intelligence's aggression detection software, used in hundreds of U.S. schools and hospitals. The algorithm, which analyzes audio for signs of stress and anger, frequently misidentified innocent sounds like coughing as aggressive and failed to detect actual screams. At a New Jersey hospital, the system ignored an agitated man screaming and pounding a desk, escalating the situation. Sound Intelligence's CEO acknowledged the imperfections but defended the technology as an early warning system.
- AI system involved
- Sound Intelligence aggression detector
2 source articles · read the reporting →
Urban Cyber Security VPN extension harvested AI chatbot prompts and responses
In July 2025, Urban Cyber Security updated its Urban VPN Proxy Chrome extension to automatically harvest everything users typed into major AI chatbots, including ChatGPT and Claude, as well as the chatbots' replies. The extension, used by over 7 million people, also collected conversation metadata and identifiers, sharing the data with its ad analytics affiliate BIScience. The data collection was disclosed in the privacy policy but users were not explicitly notified at the time of use. Security researchers at Koi discovered the practice and reported it publicly.
- Company involved
- Urban Cyber Security
- AI system involved
- Urban VPN Proxy
3 source articles · read the reporting →
Blind people and advocates criticise AccessiBe for worsening website access
Blind users and disability advocates say AccessiBe, an automated web accessibility overlay, disrupts screen readers on websites, making some sites unnavigable and preventing users from paying rent, shopping or teaching. More than 400 people signed an open letter asking companies to stop using automated overlays. Some blind users have brought ADA lawsuits against companies that use AccessiBe; one case was referred to mediation and another was settled. AccessiBe denies that it was at fault, saying critics do not give specific examples.
- Company involved
- AccessiBe
- AI system involved
- AccessiBe
10 source articles · read the reporting →
Baltimore County Schools Relocate Students After AI Gun False Alarm
On November 7, 2025, an AI security system called Omnilert reported a possible gun at Parkville High School in Baltimore County, Maryland. Students were relocated while police searched the school, but no threat was found. The incident follows a similar false alarm at Kenwood High School in October, where Omnilert mistook a bag of chips for a gun, leading to students being handcuffed. Baltimore County Public Schools has promised annual training on protocols for responding to Omnilert alerts.
- Company involved
- Baltimore County Public Schools
- AI system involved
- Omnilert
1 source article · read the reporting →
LAUSD shelves AI chatbot after vendor AllHere collapses
Los Angeles Unified School District turned off its 'Ed' AI chatbot on June 14, 2024, after the vendor AllHere furloughed most staff due to financial collapse. The chatbot, which cost $3 million, was designed to provide students and parents with academic guidance and school information. A former AllHere employee alleged that student data was improperly shared with third parties and processed overseas, raising privacy concerns. The district stated it will ensure privacy protections and plans to eventually restore the chatbot.
- Company involved
- Los Angeles Unified School District
- AI system involved
- Ed
5 source articles · read the reporting →
Gamma AI Presentation Tool Exploited in Multi-Stage Phishing Campaign
Threat actors used Gamma, an AI-powered presentation builder, to host a page that redirected recipients to a fake Microsoft SharePoint login portal. Emails sent from compromised legitimate accounts passed authentication checks, while a Cloudflare Turnstile blocked automated security scanners. An adversary-in-the-middle framework validated credentials in real time and captured session cookies, enabling multi-factor authentication bypass on Microsoft accounts. Abnormal reported the campaign on 15 April 2025.
- AI system involved
- Gamma
7 source articles · read the reporting →
Lockport City School District's Facial Recognition System Misidentified Black Faces and Weapons
Lockport City School District deployed SN Technologies' AEGIS face and weapons detection system in January 2020. Parents and the New York Civil Liberties Union allege the system misidentifies Black people more often than white people and falsely detects weapons such as broom handles. A lawsuit was filed against the New York State Education Department seeking to ban facial recognition in schools. SN Technologies denied misleading the district.
- Company involved
- Lockport City School District
- AI system involved
- AEGIS
1 source article · read the reporting →
Former Hall County deputy arrested for allegedly misusing Flock camera system, GBI says - Atlanta News First
License plate reader system used for non-law enforcement purposes
- Company involved
- Hall County Sheriff's Office
- AI system involved
- Flock
1 source article · read the reporting →
Google's Gemini AI Allegedly Scans Private Drive PDFs Without Permission
Kevin Bankston, a privacy activist, alleges that Google's Gemini AI automatically summarised his private tax return PDF in Google Drive without his explicit consent. Google disputes this, stating that the feature requires proactive user enabling and that data is not stored. Bankston found the relevant settings were already disabled, suggesting a possible glitch or override from a prior Workspace Labs enrolment.
- Company involved
- Google
- AI system involved
- Gemini AI
1 source article · read the reporting →
Outback Steakhouse franchise tests Presto Vision to monitor staff and guests
Evergreen Restaurant Group, a franchisee of Outback Steakhouse, has begun testing Presto Vision, a computer vision system that analyses surveillance camera footage to track employee performance and guest behaviour. The system compiles metrics on service, wait times, and customer turnover, which managers receive by email. Employees say they were not informed about the technology, and researchers have raised concerns about workplace stress, job losses, and data sharing with parent companies.
- Company involved
- Evergreen Restaurant Group
- AI system involved
- Presto Vision
8 source articles · read the reporting →
Beijing school uses Hanwang emotion recognition system to monitor student behaviour
Niulanshan First Secondary School in Beijing has installed Hanwang Technology's Classroom Care System, which photographs students every second and analyses their facial expressions and behaviour. The system sends weekly reports to teachers and parents, and one student was described as low participation in English class despite being recorded as focused 94% of the time. The school has been highlighted in a report by Article 19, which says such emotion recognition tools rely on junk science and risk eroding privacy and human rights.
- Company involved
- Niulanshan First Secondary School
- AI system involved
- Classroom Care System
10 source articles · read the reporting →
Banjo's Live Time system overstated capabilities and posed privacy risks for Utah public safety data
The Utah State Auditor reviewed the contract for Banjo's Live Time public safety application. The system was found to lack advertised AI technology and posed security risks due to direct database queries to PSAP databases. Banjo also overstated its capabilities in its response to the RFP. The auditor concluded that personally identifiable information was unlikely accessed but recommended more rigorous vetting and better security practices.
- Company involved
- Utah Attorney General's Office
- AI system involved
- Live Time
5 source articles · read the reporting →
Proctorio's exam proctoring system allows human agents to review student feeds despite privacy claims
A security researcher analyzed Proctorio's Chrome extension and found evidence that the system allows human agents to review students' room scans and live ID checks, contrary to Proctorio's claims that only professors can access recordings. The system flags students based on behavioral metrics and can terminate exams. The researcher also raised concerns about discrimination against low-income students and privacy violations.
- Company involved
- Proctorio
- AI system involved
- Proctorio
10 source articles · read the reporting →
Clearview AI tested facial recognition surveillance cameras with UFT and Rudin
Clearview AI, the facial recognition company that scraped billions of photos from social media, developed a surveillance camera system under the name Insight Camera. The system was tested by the United Federation of Teachers and Rudin Management in New York City. The UFT used it to identify individuals who had made threats and prevent them from entering its offices. Clearview did not respond to requests for comment.
- Company involved
- Clearview AI
- AI system involved
- Insight Camera
9 source articles · read the reporting →
