AMS algorithm lacks transparency and may discriminate against job seekers
The Austrian Public Employment Service (AMS) uses an algorithm to classify job seekers into categories A, B, and C, determining their access to benefits and training. Scientists from TU Wien, WU Wien, and University of Vienna have criticised the algorithm for lacking transparency, as only two of 96 model variants have been published. They allege that the system may discriminate against women and people with migration background, and that job seekers are not informed about how the algorithm works or given a chance to appeal.
- Company involved
- AMS (Arbeitsmarktservice Österreich)
- AI system involved
- AMS-Algorithmus
10 source articles · read the reporting →
Newham Council fined £145,000 over leaked gang matrix data
Newham Council was fined £145,000 by the Information Commissioner's Office after a leaked unredacted gangs matrix, containing details of 203 suspected gang members, ended up in the hands of rival gang members via Snapchat. The leak occurred in January 2017 when a council employee emailed both redacted and unredacted versions to 44 recipients. The ICO found the breach was unnecessary and that the council failed to report it promptly. The council apologised and accepted the breach was not deliberate.
- Company involved
- Newham Council
- AI system involved
- Gangs matrix
8 source articles · read the reporting →
Dutch police predictive policing project in Roermond faces discrimination allegations
Amnesty International's report alleges that the Dutch police's Sensing predictive policing project in Roermond uses algorithms that result in automated discrimination and mass surveillance. The system assesses crime risk for individuals and locations, potentially targeting marginalized communities. The report calls for accountability and human rights safeguards.
- Company involved
- Dutch police
- AI system involved
- Sensing project
10 source articles · read the reporting →
New York City's McKinsey-led jail violence program manipulated data, violence increased
New York City paid McKinsey & Company $27.5 million to reduce violence at Rikers Island jail complex. McKinsey designed a predictive algorithm called the Housing Unit Balancer and Restart housing units, but jail officials and McKinsey consultants stacked the units with compliant inmates to artificially lower violence numbers. Violence actually increased by nearly 50% during the project. The city eventually decided to close Rikers.
- Company involved
- New York City Department of Correction
- AI system involved
- Housing Unit Balancer (HUB)
10 source articles · read the reporting →
Portland Metro ends Replica partnership over data privacy concerns
Portland Metro, an elected regional government in Oregon, ended its pilot project with movement data company Replica after a disagreement about data sharing. Portland Metro requested raw, disaggregated data, which Replica refused to provide, citing user privacy concerns. The partnership was terminated without payment.
- Company involved
- Portland Metro
- AI system involved
- Replica
10 source articles · read the reporting →
Kohler, BMW, MaxMara secretly collected customers' facial recognition data
During the 2021 CCTV 3·15 Gala, it was revealed that Kohler, BMW, and MaxMara stores had installed facial recognition cameras from vendors such as Wandianzhang and Youluoke. These cameras captured customers' facial data without their knowledge or consent, and the data was used to track customer visits and inform sales strategies. The systems were deployed in thousands of stores across China, collecting over 100 million facial records. The companies did not inform customers or obtain consent, violating Chinese privacy laws.
- Company involved
- Kohler (China) Investment Co., Ltd., ZhengTong Auto (BMW dealerships), MaxMara
- AI system involved
- Facial recognition cameras from Wandianzhang, Youluoke, Yaliang, Ruiwei
10 source articles · read the reporting →
Judge rules police search using Flock was mass surveillance
A judge ruled that a police search using Flock's automated license plate readers constituted a form of mass surveillance. The ruling concerns the deployment of the AI-based camera system by law enforcement, which the court found to be an invasive surveillance practice. No further details of the case were provided in the article.
- AI system involved
- Flock
4 source articles · read the reporting →
Michael L. Ruiz v. Magellan Financial & Insurance Services (2) (D. Arizona): AI-hallucinated content in court filing, Formal public reprimand
AI-hallucinated fake quotations were included in court filings submitted by counsel Elizabeth Tate, misleading the court.
1 source article · read the reporting →
Huawei and Megvii tested Uyghur alarm facial recognition system
Huawei and Megvii conducted an interoperability test in 2018 that verified a 'Uyghur alarm' function in Megvii's facial recognition system, according to a confidential report found by IPVM. The system could identify Uyghur minorities and alert police. Huawei denied real-world application and deleted the report after IPVM inquired. Megvii denied targeting ethnic groups. The test report was publicly accessible on Huawei's European website.
- Company involved
- Huawei
- AI system involved
- Megvii Dynamic Face Recognition
10 source articles · read the reporting →
Meta's content moderation errors during May 2021 Israel-Palestine escalation
During the May 2021 escalation of violence in Israel and Palestine, Meta's automated content moderation systems temporarily restricted access to the al-Aqsa hashtag page and under-enforced rules against incitement to violence against Israelis and Jews. An independent due diligence report commissioned by Meta found that these systems had an unintentional impact on Palestinian and Arab communities' freedom of expression. Meta has committed to implementing several recommendations, including improving machine learning classifiers and keyword review processes.
- Company involved
- Meta
- AI system involved
- Facebook and Instagram content moderation systems
10 source articles · read the reporting →
GSMA fined €200,000 for facial recognition privacy violation at MWC
In 2021, Dr Anastasia Dedyukhina was invited to speak at Mobile World Congress in Barcelona. The organizers required her to upload biometric data (passport) online for identity verification, which she refused. She was then forced to attend virtually. She filed a complaint with the Spanish data protection agency (AEPD) along with Adam Leon Smith. Two years later, the AEPD fined GSMA €200,000 for infringing the privacy of nearly 20,000 attendees through the use of facial recognition without proper justification.
- Company involved
- GSMA
10 source articles · read the reporting →
USPS algorithm RRECS causes pay cuts for two-thirds of rural mail carriers
The United States Postal Service (USPS) implemented a new algorithm, RRECS, to evaluate rural carrier routes and determine pay. Due to flaws in how carriers scanned packages, the algorithm underestimated route times, resulting in pay cuts for 66 per cent of rural carriers, some losing thousands of dollars annually. Carriers report that they were not adequately trained on the system and that the cuts are scheduled to take effect, though they have been postponed multiple times. The USPS stated that the system is the result of a nationally negotiated agreement.
- Company involved
- United States Postal Service
- AI system involved
- RRECS
9 source articles · read the reporting →
McCormick v. Texakoma Financial (E.D. Texas): AI-hallucinated content in court filing, Public Reprimand; Monetary Sanction; Firmwide citation review; CLE
The AI generated fictitious legal citations and quotes that were included in a court filing, misleading the court and opposing counsel.
- Company involved
- Ginsburg Law Group, P.C.
1 source article · read the reporting →
Thomas Raynard James v. Detective Kevin Conley, et al. (S.D. Florida): AI-hallucinated content in court filing, Bar Referral
AI generated hallucinated content in a court filing, affecting the legal process and the lawyers who filed it.
1 source article · read the reporting →
Hamburg deployed mass surveillance for G20 Summit
The city of Hamburg planned to use predictive policing software and facial recognition cameras for mass surveillance during the G20 Summit in July 2017. The surveillance was enforced by paramilitary means, creating a democracy-free zone. DiEM25 wrote an open letter to the Senate of Hamburg to protest the measures.
- Company involved
- Senate of Hamburg
10 source articles · read the reporting →
Met Police buys £3m retrospective facial recognition system
The Metropolitan Police Service (MPS) has awarded a £3 million, four-year contract to Northgate Public Services for a new retrospective facial-recognition (RFR) system to be deployed within three months. RFR processes biometric information from historic CCTV, social media, and other images to identify suspects and missing persons, operating retroactively unlike live facial recognition. The procurement was approved by the Mayor's Office for Policing and Crime in August 2021. The MPS states that human-in-the-loop decision-making will be used, but digital rights groups and a former biometrics commissioner have raised concerns about potential discrimination, overrepresentation of marginalised groups in watch lists, and lack of a legislative framework.
- Company involved
- Metropolitan Police Service
- AI system involved
- Retrospective facial-recognition software
9 source articles · read the reporting →
In re Rosslyn2016, LLC, et al. (S.D. Texas (Bankruptcy)): AI-hallucinated content in court filing, CLE on generative AI; Civil Contempt;…
The AI generated fabricated legal citations that were submitted to the bankruptcy court.
1 source article · read the reporting →
Woman arrested in Moscow after facial recognition detection on Metro
A woman in Moscow was arrested on 1 March 2022 after she posted a coded protest message on social media. She believes she was detected by facial-recognition software on the Moscow Metro system. The authorities presented a screenshot of her tweet in her court hearing. The article reports that she was arrested while taking a train, five days after her post.
- Company involved
- Moscow Metro
10 source articles · read the reporting →
DeepSeek exposed user data via open ClickHouse database
Cloud security firm Wiz discovered a ClickHouse database belonging to DeepSeek that was open to the internet without authentication, containing over a million lines of logs with chat histories, secret keys and backend details. Wiz disclosed the breach to DeepSeek, which promptly locked down the database. The incident highlights security risks in rapidly deploying AI services.
- Company involved
- DeepSeek
- AI system involved
- DeepSeek-R1
5 source articles · read the reporting →
Met Police accessed PimEyes facial recognition site 2,000 times
The Metropolitan Police accessed the controversial facial recognition search engine PimEyes over 2,000 times from its computers in the first three months of 2024. The force stated that the hits did not necessarily mean searches were conducted, but after being alerted by i and Liberty Investigates, it blocked access to the site and strengthened safeguards. Privacy campaigners and MPs have raised concerns that the unregulated tool could be used for stalking and surveillance.
- Company involved
- Metropolitan Police Service
- AI system involved
- PimEyes
3 source articles · read the reporting →
Meta tracks employee keystrokes on Google, LinkedIn, Wikipedia for AI training
Meta is using an internal tool, Model Capability Initiative (MCI), to capture employees' keystrokes, mouse movements and screen contents on work computers, including on sites such as Google, LinkedIn, Wikipedia and Slack, to train AI agents. Meta confirmed the project and said safeguards protect sensitive content and that the data is not used for other purposes. Employees raised concerns in internal messages that the tool could expose passwords, product details and personal information. A Meta memo said staff can avoid capture by not doing personal work on work computers.
- Company involved
- Meta
- AI system involved
- Model Capability Initiative (MCI)
5 source articles · read the reporting →
Teething problems in Mater Dei's medicine robots addressed
The Malta Union for Midwives and Nurses claimed that a €23 million investment in two computerised drug administration robots, Mario and Sophia, at Mater Dei Hospital had resulted in a complete failure. However, sources within the Health Ministry said that most teething problems have been addressed and that the supplier has not been paid yet. They reported that out of over 1,700 medication rounds, only four required a contingency plan.
- Company involved
- Mater Dei Hospital
- AI system involved
- Mario and Sophia
6 source articles · read the reporting →
AEMPS withdraws AI medicines tool MeQA after detecting errors
AEMPS launched MeQA, an artificial intelligence tool for answering public questions about medicines, on 13 May 2025. Two days later it withdrew the tool after detecting that some responses contained errors. The agency said that most answers were correct but that the errors could affect patient safety, and that it would restore the service as soon as possible.
- Company involved
- Agencia Española de Medicamentos y Productos Sanitarios (AEMPS)
- AI system involved
- MeQA
4 source articles · read the reporting →
Manchester Arena's Evolv weapon scanners fail to detect some knives, report finds
ASM Global's use of Evolv Express AI weapon scanners at Manchester Arena has been questioned after a private report found the system failed to detect large knives in 42% of walkthroughs. The report, produced by NCS4 and obtained by IPVM, also suggested the scanners may miss some bombs and components. Evolv did not dispute the findings but said it communicates capabilities and limitations to customers. ASM Global declined to comment on security matters.
- Company involved
- ASM Global
- AI system involved
- Evolv Express
5 source articles · read the reporting →
