Driver-Assist System Initiates Lane Change, Causes Rear-End Collision on Highway
“智驾系统”自主变道 高速公路上遭遇追尾 - 新浪财经
A driver's XPeng X9, with its NGP driver-assistance system engaged, autonomously initiated a left lane change on an expressway. During the maneuver, a vehicle ahead braked suddenly and the car was rear-ended by a large truck. Police determined the lane change caused the accident and held the driver fully liable; the vehicle repair estimate exceeded 100,000 yuan.
- AI system involved
- Xpeng NGP (Navigation Guided Pilot)
1 source article · read the reporting →
False AIS tracks simulated for Swedish Navy vessels
An investigation by SkyTruth and Global Fishing Watch revealed that false automatic identification system (AIS) tracks were generated for nine Swedish Navy vessels on February 4-5, 2021. The Swedish Navy confirmed the positions were false. The false tracks appeared plausible and were indistinguishable from real AIS broadcasts, posing a threat to data integrity for marine monitoring.
- AI system involved
- Automatic Identification System (AIS)
10 source articles · read the reporting →
Playground AI makes MIT student's headshot appear Caucasian
Rona Wang, an MIT graduate, used Playground AI to generate a professional LinkedIn photo. The AI altered her appearance, giving her lighter skin and blue eyes, making her look Caucasian. The incident sparked discussion about racial bias in AI. Playground AI's founder acknowledged the issue and expressed a desire to fix it.
- Company involved
- Playground AI
- AI system involved
- Playground AI
1 source article · read the reporting →
Google's Med-Gemini AI Hallucinates Non-Existent Body Part in Brain Scan
In a 2024 research paper, Google's Med-Gemini AI model generated a radiology report diagnosing an 'old left basilar ganglia infarct', a non-existent body part conflating the basal ganglia and basilar artery. The error was not caught by Google before publication. After a neurologist flagged the mistake, Google quietly edited a blog post to correct the term, calling it a typo, but the research paper remains unchanged. Experts warn that such hallucinations pose significant risks if doctors fail to notice them in clinical settings.
- Company involved
- Google
- AI system involved
- Med-Gemini
1 source article · read the reporting →
Harvard students create facial recognition smart glasses to identify strangers
Two Harvard students built a system called I-XRAY that uses Meta's Ray Ban smart glasses and the facial recognition service Pimeyes to automatically identify strangers and retrieve their personal information, including name, phone number, and home address. The students tested the system on unsuspecting people in public to demonstrate the privacy risks. They are not releasing the code.
- AI system involved
- I-XRAY
6 source articles · read the reporting →
New York lawyer faces discipline for citing ChatGPT-fabricated case in appeal
New York attorney Jae Lee cited a fabricated case in an appeal to revive a client's medical malpractice lawsuit against a Queens doctor. The case was suggested by ChatGPT and did not exist. The court dismissed the case and referred Lee to the grievance panel for further investigation. Lee stated there was no bad faith, but the court found her conduct fell below basic obligations of counsel.
- Company involved
- JSL Law Offices
- AI system involved
- ChatGPT
5 source articles · read the reporting →
Gamma AI Presentation Tool Exploited in Multi-Stage Phishing Campaign
Threat actors used Gamma, an AI-powered presentation builder, to host a page that redirected recipients to a fake Microsoft SharePoint login portal. Emails sent from compromised legitimate accounts passed authentication checks, while a Cloudflare Turnstile blocked automated security scanners. An adversary-in-the-middle framework validated credentials in real time and captured session cookies, enabling multi-factor authentication bypass on Microsoft accounts. Abnormal reported the campaign on 15 April 2025.
- AI system involved
- Gamma
7 source articles · read the reporting →
OpenAI and Google AI autocompletes AOC photo in bikini
Researchers demonstrated that image-generation algorithms from OpenAI and Google, when given a cropped photo of a woman, often autocomplete her wearing a bikini or low-cut top, while men are given suits. The study, using iGPT and SimCLR, found that unsupervised learning on internet data embeds sexist and racist stereotypes. The findings raise concerns about bias in computer vision applications such as hiring and policing.
- Company involved
- OpenAI, Google
- AI system involved
- iGPT, SimCLR
1 source article · read the reporting →
Suresnes allows startup XXII to use CCTV for experimental surveillance algorithms
The city of Suresnes has allowed startup XXII to use its public CCTV cameras for 18 months to develop algorithms for detecting suspicious behavior. The algorithms are experimental and may have high error rates. Residents were not informed or consulted. The startup will own the data and can use the city's surveillance center as a showroom for clients.
- Company involved
- Mairie de Suresnes
- AI system involved
- XXIISmartCity
10 source articles · read the reporting →
Xpeng P7 Collides with Truck During NGP Assisted Driving, Driver Injured
On 23 September 2021, a Xpeng P7 owner in China was using the NGP automatic navigation assisted driving system when the vehicle failed to identify a flatbed truck ahead and collided with its rear. The driver, who trusted the system to brake automatically, sustained slight injuries. Xpeng Motors stated that the system's functions were operating normally before the crash and that further analysis would be conducted. Experts suggested the visual camera may have misjudged the distance due to the truck's unusual shape.
- Company involved
- Xpeng Motors
- AI system involved
- NGP automatic navigation assisted driving system
1 source article · read the reporting →
Grok and Google Lens AI overviews falsely claim fake images show Huntingdon train attack
Grok and Google Lens AI overviews have been claiming that AI-generated images of the Huntingdon train attack are real. Grok told users one image was a genuine photo, while Google Lens claimed it was a still from a BBC news report. The AI systems also linked unrelated videos to the attack. Full Fact contacted Google, which told users they can provide feedback on errors. xAI has not yet responded.
- Company involved
- xAI and Google
- AI system involved
- Grok and Google Lens AI overviews
4 source articles · read the reporting →
Duke University's PULSE AI upscaler turns Barack Obama white due to racial bias
Duke University researchers developed PULSE, an AI tool that upscales blurry face photos using Nvidia's StyleGAN model. When tested, it generated a white face from a pixelated image of Barack Obama, revealing racial bias. The bias stemmed from StyleGAN's training data being predominantly white and the evaluation dataset CelebA being 90% white. The researchers acknowledged the issue and updated their paper, while Nvidia stated it is working to mitigate bias.
- Company involved
- Duke University
- AI system involved
- PULSE
2 source articles · read the reporting →
Yasiel Puig Valdes v. All3Media America, LLC, et al. (SCA California (Los Angeles)): AI-hallucinated content in court filing, Referral
The AI generated fake legal citations that were included in a court filing, potentially harming the plaintiff's case.
- AI system involved
- ChatGPT
1 source article · read the reporting →
Researchers demonstrate drone attack on Renault Captur ADAS with fake road signs
Security researchers from Ben Gurion University demonstrated a vulnerability in the Mobileye ADAS system used in a Renault Captur. They used drones to project fake road signs for 100 milliseconds, too quick for human perception but detectable by the system. The attack could trick the car into making unsafe maneuvers, though the system is only Level 0 (advisory). The researchers published their findings in a paper titled 'MobilBye: Attacking ADAS with Camera Spoofing'.
- AI system involved
- Mobileye ADAS
7 source articles · read the reporting →
Nvidia releases AI-powered Eye Contact feature for Broadcast app
Nvidia has released a beta version of Eye Contact, an AI-powered feature for its Broadcast app that replaces a user's eyes in video streams with simulated eyeballs that always face the camera. The feature is intended to maintain eye contact even when the user looks away, but social media response has been largely negative. A former TV news anchor said that unbroken eye contact can be regarded as creepy or uncomfortable.
- Company involved
- Nvidia
- AI system involved
- Eye Contact
8 source articles · read the reporting →
Delphi denies near-miss between its self-driving car and Google's autonomous vehicle
Delphi Automotive initially reported a near-miss between its self-driving Audi Q5 SUV and a Google Lexus RX400h in Palo Alto, California. A Delphi executive claimed Google's car cut off the Delphi vehicle, which took evasive action. However, Delphi later denied the incident, stating the vehicles were a lane width apart and that both cars performed correctly. Google also confirmed the cars behaved appropriately in an ordinary driving scenario.
- Company involved
- Delphi Automotive
- AI system involved
- Delphi self-driving car (Audi Q5 SUV)
7 source articles · read the reporting →
Retorio AI personality test swayed by candidate appearance in BR experiment
Bayerischer Rundfunk journalists conducted experiments with Retorio's AI video interview analysis tool. The AI, which assesses personality traits from short videos, produced different scores when the same actress changed her appearance (glasses, headscarf, wig) or the video background and lighting were altered. The start-up Retorio acknowledged that the AI considers external image, similar to a human interviewer. Experts warned that such software could perpetuate stereotypes and unfairly affect job candidates.
- AI system involved
- Retorio AI
10 source articles · read the reporting →
Google Health's diabetic retinopathy AI faced real-world issues in Thailand clinics
Google Health deployed a deep-learning system to screen for diabetic retinopathy in 11 clinics across Thailand. The AI was highly accurate in lab tests but rejected over a fifth of eye scans in real-world conditions due to poor lighting and slow internet. Patients whose scans were rejected had to visit specialists at other clinics, causing inconvenience and frustration for nurses. Google Health is now working with local staff to improve the system's workflow.
- Company involved
- Google Health
10 source articles · read the reporting →
Deep-Live-Cam enables real-time face swapping, raising fraud concerns
A software package called Deep-Live-Cam has gone viral for enabling real-time face swapping on webcams using a single photo. The tool, which is free and open source, allows anyone to impersonate another person's appearance in a video chat, raising concerns about remote deception and fraud. The article notes that similar technology has been used in a $25 million heist in Hong Kong and in audio deepfake scams.
- AI system involved
- Deep-Live-Cam
7 source articles · read the reporting →
Stable Diffusion amplifies racial and gender stereotypes in generated images
An analysis by Bloomberg of over 5,000 images generated by Stability AI's Stable Diffusion found that the text-to-image model amplifies racial and gender stereotypes. The model overrepresented lighter-skinned men in high-paying jobs and darker-skinned people in low-paying jobs, and underrepresented women in positions of power. Stability AI acknowledged the inherent biases in its models and stated it is working on mitigation.
- Company involved
- Stability AI
- AI system involved
- Stable Diffusion
8 source articles · read the reporting →
Apple's Enhanced Visual Search raises privacy concerns over default data sharing
Apple's iOS 18 update introduced an 'Enhanced Visual Search' feature that automatically shares encrypted photo data with Apple to identify landmarks. The feature is enabled by default, requiring users to manually opt out, which has sparked privacy concerns. Critics argue it should be opt-in, given Apple's usual privacy standards. The article reports on the feature's design and the resulting debate, not on any specific harm to an individual.
- Company involved
- Apple
- AI system involved
- Enhanced Visual Search
5 source articles · read the reporting →
Researchers jailbreak Stable Diffusion and DALL-E 2 to generate disturbing images
Researchers from Johns Hopkins and Duke universities developed a method called SneakyPrompt that uses reinforcement learning to bypass safety filters in text-to-image AI models. The technique allowed them to generate images of nudity and violence from Stable Diffusion and DALL-E 2. OpenAI has since fixed the vulnerability in DALL-E 2, but Stable Diffusion 1.4 remains vulnerable. Stability AI says it is working with the researchers to improve defenses.
- AI system involved
- Stable Diffusion 1.4 and DALL-E 2
5 source articles · read the reporting →
Stable Diffusion reproduces exact copies of training images
Researchers found that Stable Diffusion, an AI image generation model, can reproduce exact copies of images from its training dataset, including copyrighted material and personal photos. The model memorized over a thousand training examples, posing copyright and privacy risks. The researchers warn that this is an industry-wide problem affecting models like DALL-E 2 and Imagen.
- Company involved
- Stability AI
- AI system involved
- Stable Diffusion
7 source articles · read the reporting →
Stable Diffusion 3 Medium release generates anatomically incorrect images
Stability AI released Stable Diffusion 3 Medium, an AI image generator, on June 12, 2024. Users on Reddit reported that the model produces mangled human anatomy, such as deformed hands and bodies. The failures are attributed to aggressive NSFW content filtering in the training data that removed images of human anatomy. The company has not responded to the criticism.
- Company involved
- Stability AI
- AI system involved
- Stable Diffusion 3 Medium
4 source articles · read the reporting →
