Where automated decisions went wrong

Mazaheri v Law Society of Ontario (Law Society Tribunal (ON)): AI-hallucinated content in court filing, Adverse Costs Order

AI generated fabricated legal citations in a court filing by Shahryar Mazaheri.

1 source article · read the reporting →

South Korea's AI textbook program rolled back after complaints

South Korea's Ministry of Education launched AI-powered digital textbooks in March 2025 for math, English, and computer science. Students, teachers, and parents complained about technical problems, inaccuracies, and data privacy risks. After four months, the textbooks were stripped of official status and made optional. The program was criticized for being rushed and insufficiently tested.

Company involved

South Korean Ministry of Education

AI system involved

AI-powered textbooks

7 source articles · read the reporting →

Meta altered content for user including 'Palestinian' and flag

Dr. Khan, a Pakistani dentist, reported that when he included the word 'Palestinian' and the Palestinian flag emoji in a social media story, the platform altered the content. After he complained via TikTok, Meta changed the system. The incident was reported on X and picked up by a BBC reporter.

Company involved

Meta

10 source articles · read the reporting →

LINAGORA closes Lucie 7B after user mockery

LINAGORA, a French open-source software company, launched a beta version of its large language model Lucie 7B. The model was intended to be a transparent and ethical alternative to big tech AI. However, after users tested it and highlighted its shortcomings, the model was mocked online. LINAGORA subsequently closed the platform to address the issues and collect more data.

Company involved

LINAGORA

AI system involved

Lucie 7B

6 source articles · read the reporting →

New York City Department of Education blocks ChatGPT on school devices

The New York City Department of Education blocked access to the AI chatbot ChatGPT on school devices and networks, citing concerns about negative impacts on student learning and the safety and accuracy of content. The ban applies to all students and teachers on education department devices and internet networks. Individual schools can still request access for studying the technology. The move is the nation's largest school system's response to the arrival of ChatGPT.

Company involved

New York City Department of Education

AI system involved

ChatGPT

9 source articles · read the reporting →

Meta tracks employee keystrokes on Google, LinkedIn, Wikipedia for AI training

Meta is using an internal tool, Model Capability Initiative (MCI), to capture employees' keystrokes, mouse movements and screen contents on work computers, including on sites such as Google, LinkedIn, Wikipedia and Slack, to train AI agents. Meta confirmed the project and said safeguards protect sensitive content and that the data is not used for other purposes. Employees raised concerns in internal messages that the tool could expose passwords, product details and personal information. A Meta memo said staff can avoid capture by not doing personal work on work computers.

Company involved

Meta

AI system involved

Model Capability Initiative (MCI)

5 source articles · read the reporting →

UIUC students petition to stop Proctorio exam proctoring over privacy concerns

A petition at the University of Illinois at Urbana-Champaign (UIUC) alleges that Proctorio, an online exam proctoring system, violates student privacy by accessing websites, downloads, screen content, and app settings. The petition claims the terms of service allow monitoring by 'any other means necessary', which students find unsettling. The petition, created on September 30, 2020, gathered 1,087 supporters but does not report any specific incident of harm. It calls on UIUC to discontinue use of Proctorio in favour of alternatives.

Company involved

UIUC

AI system involved

Proctorio

9 source articles · read the reporting →

Google Gemini chatbot gives biased response about PM Narendra Modi

Google's Gemini chatbot generated a biased and inappropriate response to a query about Indian Prime Minister Narendra Modi. A user on X shared screenshots, and Union minister Rajeev Chandrasekhar flagged the response as violating India's IT rules and criminal code. Google acknowledged the issue and said it had taken steps to address it, but warned that Gemini may not be reliable for current events and political topics.

Company involved

Google

AI system involved

Gemini

5 source articles · read the reporting →

US State Department to use AI to revoke student visas over pro-Hamas posts

The US State Department is reported to be using AI-assisted reviews of tens of thousands of foreign students' social media accounts, with a view to revoking visas of those deemed 'pro-Hamas'. According to Axios, the project was launched by Secretary of State Marco Rubio and is looking for evidence of alleged terrorist sympathies expressed since October 2023. Officials claimed no visa revocations were made under the Biden administration. No individual revocations have yet been reported.

Company involved

US State Department

5 source articles · read the reporting →

AI unmasks anonymous chess players, study warns of privacy risks

A study described in Science shows that software can identify people who play chess anonymously, revealing their identities. Researchers, including Alexandra Wood, say the work highlights a growing privacy threat. The article does not name the software or any specific victims.

5 source articles · read the reporting →

ChatGPT use linked to memory loss and procrastination in students

A study published in the International Journal of Educational Technology in Higher Education surveyed hundreds of university students in Pakistan and found that those who relied more on ChatGPT reported increased procrastination, memory loss, and lower GPAs. The researchers attribute this to the chatbot making schoolwork too easy, reducing students' cognitive effort. The study's lead author warned of a "dark side" to excessive generative AI usage.

Company involved

National University of Computer and Emerging Sciences

AI system involved

ChatGPT

4 source articles · read the reporting →

Fake AI audio of Sadiq Khan nearly caused disorder

A fake audio clip using AI to clone London Mayor Sadiq Khan's voice was shared online in November 2023, allegedly making inflammatory remarks about Remembrance Day. The clip spread rapidly and triggered a spike in hateful comments, and Mr Khan said it almost caused serious disorder. The Metropolitan Police examined the audio and said it did not constitute a criminal offence. The creator of the audio has not been identified.

8 source articles · read the reporting →

AEMPS withdraws AI medicines tool MeQA after detecting errors

AEMPS launched MeQA, an artificial intelligence tool for answering public questions about medicines, on 13 May 2025. Two days later it withdrew the tool after detecting that some responses contained errors. The agency said that most answers were correct but that the errors could affect patient safety, and that it would restore the service as soon as possible.

Company involved

Agencia Española de Medicamentos y Productos Sanitarios (AEMPS)

AI system involved

MeQA

4 source articles · read the reporting →

Texas uses AI to grade student STAAR test answers

The Texas Education Agency will use an automated scoring engine to grade written answers on the 2023 STAAR tests, replacing thousands of human graders. The system uses natural language processing and will initially score all responses, with a quarter rescored by humans. Educators have expressed concerns about the system's fairness and the potential for errors, especially for creative or non-standard answers.

Company involved

Texas Education Agency

AI system involved

automated scoring engine

10 source articles · read the reporting →

Teenager in Pakistan arrested after allegedly using AI for poison guidance - The Jerusalem Post

AI provided information on producing poisons to a teenager

Company involved

National Cyber Crime Investigation Agency

AI system involved

Claude

1 source article · read the reporting →

Singapore writers reject government plan to use their work for AI training

The Singaporean government's National Multimodal LLM Programme (NMLP) requested permission from local writers to use their copyrighted works to train a large language model aimed at reducing Western bias. Writers expressed skepticism over the lack of clarity on compensation and copyright protection. The S$70 million project, launched in December 2023, is part of Singapore's effort to become a global AI leader. No actual use of the works has occurred yet, and the writers have not agreed.

Company involved

Singapore government (National Multimodal LLM Programme)

AI system involved

National Multimodal LLM (NMLP)

2 source articles · read the reporting →

Baltimore schools monitor student laptops for suicide signs using GoGuardian Beacon

Baltimore City Public Schools uses GoGuardian Beacon software to monitor student laptops for signs of suicide. Since March 2021, the system has flagged 786 alerts, with nine students taken to emergency rooms. Privacy advocates warn the monitoring could lead to disciplinary actions, outing of LGBTQ students, and disproportionately affect disadvantaged students. School officials defend the practice as a safeguard.

Company involved

Baltimore City Public Schools

AI system involved

GoGuardian Beacon

10 source articles · read the reporting →

Mistral releases unmoderated chatbot that gives instructions on murder and ethnic cleansing

Mistral, a French AI startup valued at $260 million, released an open-source large language model named Mistral-7B-v0.1 without safety evaluations or moderation mechanisms. The model readily provides detailed instructions on murder, ethnic cleansing, suicide, and other harmful content. Mistral added a statement after the release acknowledging the lack of moderation but did not remove the model, which is distributed via torrent and cannot be deleted.

Company involved

Mistral

AI system involved

Mistral-7B-v0.1

7 source articles · read the reporting →

CBSE OnMark portal vulnerability exposed student data to Google Gemini

A 19-year-old ethical hacker, Nisarga Adhikary, claimed to have hacked the CBSE's digital evaluation ecosystem, revealing that personal information of students was processed by Google's Gemini in automation scripts. The Central Board of Secondary Education (CBSE) stated on May 31, 2026, that the identified vulnerabilities had been contained and other exploitable weaknesses were being ruled out. The board expressed gratitude to alert citizens and ethical hackers who pointed out the weaknesses. No actual data breach was confirmed, but the incident raised concerns about student privacy.

Company involved

Central Board of Secondary Education (CBSE)

AI system involved

OnMark

1 source article · read the reporting →

Telegram deepfake porn crisis engulfs hundreds of South Korean schools

Journalist Ko Narin uncovered Telegram chatrooms in South Korea where users shared photos of women and girls and used AI software to turn them into fake pornographic images. More than 500 schools and universities have been identified as targets, with many victims believed to be under 16. Police have announced an investigation into Telegram, which said it actively combats harmful content on its platform.

Company involved

Telegram

9 source articles · read the reporting →

Mass AI cheating scandal at Yonsei University with hundreds of students using ChatGPT

A large-scale cheating scandal has erupted at Yonsei University, where hundreds of students in a third-year online course are suspected of using AI tools such as ChatGPT to cheat on their midterm exam. The professor discovered signs of misconduct and offered students a chance to confess, with those coming forward receiving a zero but no further penalty. A poll on a student community app indicated that over half of respondents admitted to cheating. The university has not yet established clear guidelines on AI use.

Company involved

Yonsei University

AI system involved

ChatGPT

6 source articles · read the reporting →

Albanian actor sues government over unauthorized use of her likeness for AI virtual minister

Albanian actor Anila Bisha is suing the government after her image and voice were used without consent to create "Diella," the world's first virtual minister for artificial intelligence. Bisha had signed a contract allowing use of her likeness for a chatbot on the e-Albania portal, but alleges the government expanded its use to a ministerial role without permission. The case is pending in the Administrative Court of Albania.

Company involved

Council of Ministers of Albania

AI system involved

Diella

4 source articles · read the reporting →

School AI surveillance like Gaggle can lead to false alarms, arrests

AI surveillance tools used in schools, such as Gaggle, GoGuardian and Bark, are reported to generate false alarms that have led to student arrests. The article examines cases where automated monitoring flagged innocent behaviour as threats, causing harm to students and families.

2 source articles · read the reporting →

Greater Noida Student Dies After School Alleges AI Cheating

A 16-year-old student in Greater Noida died by suicide on 23 December 2025 after school authorities questioned her over suspected use of AI tools during a pre-board exam. Her father alleges that teachers and the principal publicly humiliated her, causing severe emotional distress, and has filed a police complaint for abetment of suicide. The school denies harassment, stating they followed CBSE protocols, and has provided CCTV footage to police. The incident is under investigation.

1 source article · read the reporting →