CPS apologises after AI generates fake legal citations in court documents
The Crown Prosecution Service (CPS) used generative AI to draft court documents for extradition appeals, resulting in the inclusion of non-existent legal authorities. The error was identified by opposing counsel before the hearing and had no impact on the outcome. The CPS apologised to the High Court, attributing the mistake to a failure by a reviewing lawyer to verify the AI-generated content, and undertook an internal review to prevent recurrence. The judge accepted the apology and noted the serious risks of using AI without proper oversight.
- Company involved
- Crown Prosecution Service
1 source article · read the reporting →
Eastpointe officers used Flock access improperly, chief says - detroitnews.com
The system captured and allowed searching of vehicle location data, impacting the privacy of vehicle owners.
- Company involved
- Eastpointe Police Department
- AI system involved
- Flock Safety
1 source article · read the reporting →
Stockton police disclose Flock misuse, outline oversight changes after employee’s departure - Stocktonia News
Stockton police disclose Flock misuse, outline oversight changes after employee’s departure Stocktonia News
1 source article · read the reporting →
South Africa's Supreme Court of Appeal Considers Sassa Algorithm Case
Верховный апелляционный суд ЮАР рассмотрел дело об алгоритмах Sassa - UA.NEWS
The Supreme Court of Appeal heard an appeal on 25 August about Sassa's digital application system for the SRD grant. The system only accepts online applications and uses automated bank account checks that may deny grants to people whose accounts receive deposits that are not regular income. The Global Center on AI Governance submitted that automated decisions must uphold constitutional rights and be fair, non-discriminatory, and suited to South Africa's informal economy.
- Company involved
- South African Social Security Agency (Sassa)
- AI system involved
- Sassa digital application system
1 source article · read the reporting →
Argentine Court Annuls Embargoes on Retiree After Identity Mix-Up in Child Support Case
Justicia argentina anula embargos a jubilada por cobro indebido de pensión de alimentos a raíz de error de identidad -…
A retiree had her pension wrongly garnished because her name matched that of the defendant in a child support lawsuit. The appeals court confirmed the immediate halt of the deductions and annulled the embargo, ruling it had no legal basis against a third party. The proceedings against the actual defendant were upheld, as the error was limited to the garnishment order sent with incorrect identification.
- Company involved
- Cámara de Apelaciones en lo Civil, Comercial, de Familia, de Minería y Contencioso Administrativo de Cipolletti
1 source article · read the reporting →
Flock lawsuit accuses tech company of exposing citizens to rogue police tracking - Top Class Actions
The system recorded and tracked the locations of vehicles and their owners without consent.
- Company involved
- Flock Group Inc.
- AI system involved
- Flock
1 source article · read the reporting →
Auditor General finds CRA chatbot Charlie frequently inaccurate
The Canada Revenue Agency's chatbot Charlie, launched in 2020, has cost over $18 million but frequently provides incorrect answers, according to an Auditor General audit. Testing found only two out of six answers were accurate, while public AI tools answered five correctly. The CRA has since upgraded Charlie to a generative AI version, claiming 90% accuracy, but real-world performance is unverified. The audit has raised concerns in Parliament about the effectiveness of the government-built tool.
- Company involved
- Canada Revenue Agency
- AI system involved
- Charlie
2 source articles · read the reporting →
Court orders suspension of facial recognition in São Paulo metro
A court in São Paulo ordered the suspension of a facial recognition system in the city's metro stations following a lawsuit by civil society groups. The system, SecurOS by ISS and operated by ViaQuatro, was capturing biometric data of millions of daily users without adequate transparency or risk assessment. The court also barred the metro operator, METRO, from installing new biometric equipment and imposed daily fines for non-compliance. METRO stated it would appeal the ruling and prove compliance with data protection regulations.
- Company involved
- Companhia do Metropolitano de São Paulo (METRO)
- AI system involved
- SecurOS
3 source articles · read the reporting →
Services Australia voiceprint system fooled by AI voice clone
A Guardian Australia investigation found that the voiceprint system used by Services Australia's Centrelink and the Australian Taxation Office can be bypassed using an AI-generated voice clone. A journalist created a clone of their own voice from four minutes of audio and used it with a customer reference number to access their Centrelink self-service account. The system is used by millions of Australians for identity verification over the phone. Services Australia stated that it continually assesses risks and applies additional tests if unusual circumstances are detected, but did not commit to changing the technology.
- Company involved
- Services Australia
- AI system involved
- Voiceprint
1 source article · read the reporting →
Trelleborg municipality's welfare robot leaks personal data of hundreds
Trelleborg municipality in Sweden introduced a fully automated system named Ernst to handle welfare applications. An analysis of the system's source code, obtained via a freedom of information request, revealed that the names and social security numbers of approximately 250 people were included in the code since its introduction, making them visible to anyone who filed a request. The municipality is investigating the leak, which is also the subject of a complaint to the Swedish Parliamentary Ombudsman.
- Company involved
- Trelleborg municipality
- AI system involved
- Ernst
10 source articles · read the reporting →
AI-generated Centrelink phishing emails target 270,000 Australians
More than 270,000 fake emails impersonating Services Australia and Centrelink were detected over four months in a broad phishing campaign. Cybersecurity firm Mimecast reports that cybercriminals are using artificial intelligence to create highly convincing clones of legitimate government communications about benefits. The attack targets vulnerable people and can lead to identity theft, data theft, malware, or ransomware.
- Company involved
- Services Australia
4 source articles · read the reporting →
South African government departments suspend officials over AI-generated fake sources
The South African Department of Communication and Digital Technology and Department of Home Affairs suspended four officials after discovering that AI-generated fictitious references were used in policy documents. The Draft National Artificial Intelligence Policy was withdrawn from public consultation, and the Revised White Paper on Citizenship, Immigration, and Refugees was also affected. The departments are investigating and have appointed independent law firms to review all policy documents since November 2022. The ministers have pledged to implement AI verification checks in future policy approval processes.
- Company involved
- Department of Communication and Digital Technology; Department of Home Affairs
3 source articles · read the reporting →
U.S. Department of Homeland Security Uses Secret Algorithm ATLAS to Flag Naturalized Citizens for Denaturalization
The U.S. Department of Homeland Security operates a secret algorithm called ATLAS that screens naturalized citizens for potential fraud or national security concerns, flagging them for denaturalization. The system, hosted on Amazon Web Services, analyzes biometric and other data from various federal databases, and its rules are undisclosed. Human reviewers then decide whether to refer flagged individuals to Immigration and Customs Enforcement for possible deportation. Critics allege the algorithm relies on inaccurate data and lacks transparency, leading to wrongful denaturalization proceedings.
- Company involved
- U.S. Department of Homeland Security
- AI system involved
- ATLAS
2 source articles · read the reporting →
CISA Acting Director Uploaded Sensitive Files to Public ChatGPT
Madhu Gottumukkala, the acting director of the Cybersecurity and Infrastructure Security Agency, uploaded contracting documents marked 'for official use only' into a public version of ChatGPT in August 2025, triggering security warnings. The agency had blocked ChatGPT for other employees, but Gottumukkala had obtained special permission to use it. The Department of Homeland Security launched an internal review to assess potential harm to government security. No classified information was exposed, but the incident raised concerns about the handling of sensitive material.
- Company involved
- Cybersecurity and Infrastructure Security Agency
- AI system involved
- ChatGPT
1 source article · read the reporting →
Civil Society Demands Halt to AI Facial Recognition Tracking System by South Korean Government
South Korea's Ministry of Justice and Ministry of Science and ICT are building an AI identification and tracking system using facial recognition for airport immigration. It was revealed that over 170 million facial photos of citizens and foreigners were transferred to private companies for AI training without proper legal basis. Civil society groups held a press conference on November 9, 2021, condemning the project as an unprecedented violation of information human rights and demanding its immediate halt. They argue the data transfer violates the Personal Information Protection Act and have requested a meeting with the Minister of Justice.
- Company involved
- Ministry of Justice and Ministry of Science and ICT
- AI system involved
- AI Identification and Tracking System
5 source articles · read the reporting →
CFPB fines General Information Services for inaccurate background checks
The Consumer Financial Protection Bureau took action against General Information Services and its affiliate e-Background-checks.com for failing to ensure the accuracy of employment background screening reports. The companies allegedly provided inaccurate criminal history information to employers, potentially affecting job applicants' eligibility and causing reputational harm. The CFPB ordered the companies to provide $10.5 million in relief to harmed consumers and pay a $2.5 million penalty.
- Company involved
- General Information Services
10 source articles · read the reporting →
Barclays pilot of Sapience monitoring software causes employee stress
Barclays introduced a pilot of employee monitoring software from Sapience in its product control department at Canary Wharf. The system monitors computer activity and admonishes staff if they are not deemed active enough, recording breaks as "unaccounted activity". Employees reported significant stress and worry about taking breaks. Barclays acknowledged the pilot and said it would listen to feedback.
- Company involved
- Barclays
- AI system involved
- Sapience employee monitoring software
10 source articles · read the reporting →
Suzhou clarifies civility scoring system after public backlash
The Suzhou municipal government announced a civility code system that assigns scores to residents based on their behavior, including traffic violations and volunteer work. The system, currently in a trial phase, aims to generate a personal portrait for each resident. Following public backlash and concerns about privacy and potential abuse, an official stated that participation will not be mandatory and that a bad score will not affect citizens. The system is still under development.
- Company involved
- Suzhou municipal government
- AI system involved
- Civility code
10 source articles · read the reporting →
Unnamed Brazilian litigant (): AI-hallucinated content in court filing, Appeal partially granted (reintegration suspended, rent imposed), but
AI-generated fake court precedents were included in a legal appeal, leading to sanctions against the litigant for bad-faith litigation.
- AI system involved
- ChatGPT
1 source article · read the reporting →
Serbia's Social Card law could harm marginalized groups
Serbia's Social Card law, which entered into force on 1 March 2022, establishes a centralized government database to assess eligibility for social security support. Amnesty International and seven other rights organizations submitted a legal opinion to the Constitutional Court, alleging that the automated system is an intrusive surveillance system that could discriminate against Roma communities and people with disabilities. The system processes 130 categories of personal data and lacks transparency, potentially leading to errors and denial of benefits.
- Company involved
- Serbian government
- AI system involved
- Social Card system
10 source articles · read the reporting →
CBSE introduces facial recognition system for students to access digital documents
The Central Board of Secondary Education (CBSE) has introduced a facial recognition system for Class 10 and 12 students to access their digital academic documents. A live image of the student is compared with the photograph on their CBSE admit card and, if the match succeeds, the certificate is emailed to them. The facility is available on Digi Locker for 2020 records and is expected to help foreign students and those unable to open a Digi Locker account.
- Company involved
- Central Board of Secondary Education (CBSE)
- AI system involved
- Facial Recognition System
10 source articles · read the reporting →
ViaQuatro's facial recognition system in São Paulo metro challenged in court
In April 2018, ViaQuatro installed the Digital Interactive Doors System, developed by AdMobilize, on the São Paulo metro's yellow line. The system used cameras to detect passengers' faces and claimed to infer their emotion, age, and gender in order to target advertisements. The Brazilian Institute of Consumer Protection (IDEC) filed a public civil action alleging that the system violated consumer and data protection laws by processing biometric data without consent and making pseudoscientific and discriminatory inferences. A judge ordered the cameras removed in August 2018, and the case is pending a final ruling.
- Company involved
- ViaQuatro
- AI system involved
- Digital Interactive Doors System (DID system)
10 source articles · read the reporting →
Denmark's Udbetaling Danmark welfare algorithm illegally surveilled millions
Udbetaling Danmark (UDK), Denmark's centralized welfare payment agency, used automated algorithms to check eligibility and detect fraud. The system collected personal data on millions of beneficiaries and their relatives, including non-beneficiaries, in what a think-tank called 'systematic surveillance'. A complaint to the Danish Data Protection Authority led to a ruling that the data collection violated GDPR, and UDK eventually deleted the illegally obtained data. The system also caused erroneous payments, including underpaying 300,000 pensioners by 94 million euros annually.
- Company involved
- Udbetaling Danmark
10 source articles · read the reporting →
CBSA tested facial recognition on millions at Toronto Pearson Airport in 2016
In 2016, the Canada Border Services Agency (CBSA) secretly tested facial recognition technology on millions of travellers at Toronto Pearson Airport without their knowledge. The system, supplied by Face4 Systems, matched faces against a database of previously deported individuals. The CBSA stated that no one was deported as a result of the pilot, and the technology was removed after six months. Privacy advocates raised concerns about lack of consent, transparency, and potential racial bias.
- Company involved
- Canada Border Services Agency
- AI system involved
- Faces on the Move
8 source articles · read the reporting →
