Idaho Fish and Game warns AI gives wrong hunting regulations, leading to citations
Idaho Fish and Game reports that people using AI and search engines to find hunting and fishing regulations are receiving incorrect information, resulting in citations from conservation officers. The agency advises hunters and anglers to use official printed booklets or the Fish and Game website directly to avoid tickets. The problem is ongoing as AI-generated summaries may not reflect current rules.
3 source articles · read the reporting →
Nurse at St. Rose Dominican Hospital Averts AI-Sepsis Alert Error
A nurse at St. Rose Dominican Hospital in Henderson, Nevada, refused to follow an AI-generated sepsis alert that would have given an elderly dialysis patient IV fluids, which could have caused a life-threatening complication. The alert, part of the hospital's electronic system, prompted a charge nurse to order the fluids despite the patient's compromised kidneys. A physician intervened and ordered dopamine instead, averting harm. The incident highlights concerns about AI tools overriding clinical judgment.
- Company involved
- St. Rose Dominican Hospital
1 source article · read the reporting →
Center for Auto Safety petitions NHTSA over Nissan Rogue AEB false activations
The Center for Auto Safety petitioned NHTSA to investigate false activations of the Automatic Emergency Braking (AEB) system on 2017-2018 Nissan Rogue vehicles. The system reportedly engages abruptly when no obstruction is present, causing near misses and forcing following drivers to take evasive action. Nissan issued a Technical Service Bulletin to dealers but has not conducted a safety recall. The petition seeks a formal investigation and a determination that the defect poses an unreasonable risk to safety.
- Company involved
- Nissan
- AI system involved
- Automatic Emergency Braking (AEB)
5 source articles · read the reporting →
Apple iPhone crash detection triggers false 911 calls from skiers in Colorado
Apple's crash detection feature on iPhone 14 and Apple Watch automatically calls 911 when it detects a sudden stop, but on ski slopes it misinterprets falls as car crashes. Dispatch centers in Colorado ski towns received dozens of false calls daily, diverting resources from real emergencies. Apple acknowledged the issue and said it is working on a fix, but the problem persists.
- Company involved
- Apple Inc.
- AI system involved
- Crash detection and fall detection on iPhone 14 and Apple Watch
10 source articles · read the reporting →
Cense exposed 2.5 million records of auto accident victims online
On July 7, 2020, a security researcher discovered 2.5 million records containing personal and medical data of auto accident victims exposed online. The records, belonging to New York-based AI company Cense, included names, insurance policy numbers, claim numbers, and medical diagnosis notes. The data was labeled as staging data, possibly intended for temporary storage before being loaded into an AI system. After the researcher sent a responsible disclosure notice, Cense restricted public access to the database.
- Company involved
- Cense
- AI system involved
- Cense
5 source articles · read the reporting →
UK councils use Covid OneView AI to harvest personal data for risk scoring
UK local authorities are using a system called Covid OneView, developed by data analytics firm Xantura, to harvest millions of personal details from council records. The system uses predictive analytics and AI to assign risk scores to households and individuals, aiming to identify those vulnerable to Covid or likely to break lockdown rules. Privacy campaigners and MPs have criticised the lack of transparency and the extent of data collection, which includes sensitive information such as debt levels, living arrangements, and even notes on unfaithful sex. Xantura and Barking and Dagenham Council have defended the system as compliant with data protection rules and focused on providing support.
- Company involved
- UK local authorities
- AI system involved
- Covid OneView
6 source articles · read the reporting →
Woman sues Google over unsafe walking directions
Lauren Rosenberg alleges that Google Maps provided walking directions that led her onto a busy road without sidewalks in Park City, Utah, where she was struck by a motorist and suffered severe injuries. She filed a lawsuit against Google Inc. in U.S. District Court in Utah, seeking over $100,000 in damages. The mobile version of Google Maps did not display a safety warning that was present on the computer version.
- Company involved
- Google Inc.
- AI system involved
- Google Maps
10 source articles · read the reporting →
Cruise robotaxi hits San Francisco fire truck, one injured
A Cruise self-driving car collided with a San Francisco Fire Department truck that was responding to an emergency. The incident occurred on August 17, 2023, in the Tenderloin district. One passenger sustained non-severe injuries and was taken to hospital. Cruise acknowledged the crash and offered support to the rider.
- Company involved
- Cruise
- AI system involved
- Cruise self-driving car
10 source articles · read the reporting →
Environment Agency's automated flood warnings cause false alarms and late alerts
The Environment Agency in England has made its automated flood warning system permanent despite inaccuracies. The system, trialled during strikes, issues alerts based on sensor thresholds without human review. Residents have reported false alarms, such as a warning of severe flooding on the River Severn that never materialised, and late warnings, including an alert for the River Wye that arrived 24 hours after peak water levels and after a home was already flooded. The agency acknowledges the feedback and says it will review the system.
- Company involved
- Environment Agency
- AI system involved
- Automated flood warning system
10 source articles · read the reporting →
UnitedHealth used ALERT algorithm to limit mental health coverage, regulators found
ProPublica reports that UnitedHealth Group's Optum subsidiary used the ALERT algorithm to flag mental health patients and therapists as outliers, leading to therapy coverage denials. Regulators in California, Massachusetts and New York alleged this breached the federal Mental Health Parity and Addiction Equity Act, and UnitedHealth agreed to restrict the system in those jurisdictions. The company denies wrongdoing and says its programmes are compliant. Affected patients, including Medicaid enrollees, were left to pay out-of-pocket or go without care.
- Company involved
- UnitedHealth Group
- AI system involved
- ALERT
5 source articles · read the reporting →
Chattr.ai exposed job applicant data due to insecure Firebase rules
A security researcher discovered that Chattr.ai, an AI hiring system used by many fast food chains, had a Firebase database with insecure security rules. By registering a new user, the researcher gained full read/write access to the database, exposing personal data of job applicants and employees, including names, phone numbers, emails, and some plaintext passwords. The vulnerability was reported to Chattr.ai on 9 January and patched the next day, but the company did not provide further contact or thanks.
- Company involved
- Chattr.ai
- AI system involved
- Chattr.ai
6 source articles · read the reporting →
EvenUp AI errors in personal injury demand letters lead to scrutiny
EvenUp, a legal tech startup valued at $1 billion, uses AI to draft personal injury demand letters. Former employees revealed that the AI system frequently makes errors, including missing injuries and fabricating medical conditions. The company defends its hybrid approach with human oversight, but critics allege overpromised AI capabilities.
- Company involved
- EvenUp
6 source articles · read the reporting →
Chicago class action challenges ShotSpotter's discriminatory stop-and-frisk deployments
ShotSpotter, a gunshot detection system used by the Chicago Police Department, generates alerts that lead to police deployments and stop-and-frisk stops. Analyses show over 90% of alerts are unfounded, and the system is deployed only in predominantly Black and Latinx districts. A class-action lawsuit against the City of Chicago alleges the system fuels unconstitutional and discriminatory policing, including false accusations against individuals like Michael Williams.
- Company involved
- City of Chicago
- AI system involved
- ShotSpotter
7 source articles · read the reporting →
ICO investigates Microsoft's Recall feature for privacy risks
The UK Information Commissioner's Office (ICO) has issued a statement on 22 May 2024 regarding Microsoft's Recall feature. The ICO is making enquiries with Microsoft to understand the safeguards in place to protect user privacy, expecting transparency and necessity in data processing. This follows concerns that the feature may not adequately consider data protection from the outset before being brought to market.
- Company involved
- Microsoft
- AI system involved
- Recall
10 source articles · read the reporting →
Baltimore schools monitor student laptops for suicide signs using GoGuardian Beacon
Baltimore City Public Schools uses GoGuardian Beacon software to monitor student laptops for signs of suicide. Since March 2021, the system has flagged 786 alerts, with nine students taken to emergency rooms. Privacy advocates warn the monitoring could lead to disciplinary actions, outing of LGBTQ students, and disproportionately affect disadvantaged students. School officials defend the practice as a safeguard.
- Company involved
- Baltimore City Public Schools
- AI system involved
- GoGuardian Beacon
10 source articles · read the reporting →
Eventbrite algorithm recommended illegal opioid sales to addiction recovery seekers
Eventbrite's recommendation algorithm promoted thousands of listings selling illegal prescription drugs like oxycodone and Xanax alongside addiction recovery events. The listings, which appeared in search results and related events, directed users to unlicensed online pharmacies. Eventbrite acknowledged the issue and removed the listings after WIRED alerted them.
- Company involved
- Eventbrite
- AI system involved
- Eventbrite recommendation algorithm
4 source articles · read the reporting →
Outabox hack exposes biometric data of patrons at bars, clubs and casinos
Hackers claiming to be former employees published a website allowing searches of Outabox's facial recognition database, exposing biometric and other sensitive data of patrons used for age verification at bars, clubs and casinos. The Surveillance Technology Oversight Project warns that the breach demonstrates the danger of facial recognition for age verification. S.T.O.P. has launched a campaign to ban facial recognition in public accommodations.
- Company involved
- Outabox
8 source articles · read the reporting →
Two women hospitalized after stopping medication on ChatGPT's advice
Two women in Ho Chi Minh City, Vietnam, stopped taking prescribed medications for diabetes and high cholesterol after following advice from OpenAI's ChatGPT. This led to severe health complications, including dangerously high blood sugar and signs of myocardial ischemia, requiring hospitalization. The treating doctor warned that AI cannot replace medical professionals and that self-medicating based on AI advice can be life-threatening.
- Company involved
- OpenAI
- AI system involved
- ChatGPT
1 source article · read the reporting →
Dutch probation service's OXREC algorithm flawed, leading to incorrect recidivism risk assessments
The Dutch Inspectorate of Justice and Security (Inspectie JenV) published a report finding that the probation service's (Reclassering) OXREC algorithm contains serious flaws, including swapped formulas and incorrect numbers, causing about a quarter of risk assessments to be wrong. The algorithm, used since 2018 for about 44,000 cases per year, also uses variables that can lead to discrimination, such as neighborhood score and income. The Inspectorate recommended immediate correction or temporary suspension. The probation service announced it would temporarily stop using OXREC.
- Company involved
- Reclassering Nederland
- AI system involved
- OXREC
4 source articles · read the reporting →
ChatGPT Health fails to direct 52% of medical emergencies to emergency care in study
A study published in Nature Medicine found that OpenAI's ChatGPT Health tool under-triaged 52% of true medical emergencies, directing users to non-urgent care instead of emergency departments. The AI also misclassified 35% of non-urgent cases. Researchers at Mount Sinai conducted 960 tests across 60 clinical scenarios, noting the tool's susceptibility to anchoring bias when symptoms were minimized. The study highlights potential safety concerns as millions use AI for health guidance.
- Company involved
- OpenAI
- AI system involved
- ChatGPT Health
4 source articles · read the reporting →
NHTSA investigation PE24016: Unexpected ADS behavior
Waymo's automated driving system exhibited unexpected behavior during driving.
- Company involved
- Waymo
1 source article · read the reporting →
Google's earthquake alert system failed to warn Turkish residents before deadly quake
In February 2023, a 7.8-magnitude earthquake struck Turkey, killing over 50,000 people. Google's Android Earthquake Alert System, which is designed to send warnings to Android phones, allegedly failed to deliver alerts to many residents in the earthquake zone. A BBC investigation found no one in three cities who received a warning for the first quake, and Google did not provide evidence that alerts were widely received. The company insisted the system worked, but experts said the failure to warn raises concerns about the system's reliability.
- Company involved
- Google
- AI system involved
- Android Earthquake Alert System
6 source articles · read the reporting →
Durham's ShotSpotter fails to detect two deadly shootings
ShotSpotter, a gunshot detection system piloted by the Durham Police Department, failed to detect two fatal shootings in February 2023, as well as a New Year's Day shooting that injured five. The system, which uses sensors and human review to alert police, did not send alerts for these incidents within its three-square-mile coverage area. The company stated it investigated and provided a report to the police, while a city official defended the pilot, noting it is still early in the year-long trial. Community concerns have been raised about increased police presence and potential racial profiling.
- Company involved
- Durham Police Department
- AI system involved
- ShotSpotter
5 source articles · read the reporting →
Apple Intelligence generates false BBC news alert about suicide
On 13 December 2024, Apple's new generative AI feature, Apple Intelligence, generated a false news alert attributing it to the BBC, claiming the suicide of Luigi Mangione. The BBC complained to Apple. Reporters Without Borders (RSF) urged Apple to remove the feature, citing risks to reliable journalism.
- Company involved
- Apple
- AI system involved
- Apple Intelligence
7 source articles · read the reporting →
