Supreme Court strikes down NCLT order based on AI-hallucinated judgments
The National Company Law Tribunal admitted an insolvency petition against Essel Infraprojects, relying on six AI-generated citations that were later found to be non-existent or misattributed. Essel Infraprojects appealed, and the Supreme Court set aside the order, calling the use of fake judgments catastrophic. The court directed the Bar Council of India to examine AI use in litigation and ordered a fresh decision.
- Company involved
- National Company Law Tribunal
1 source article · read the reporting →
Cruise robotaxi drags woman 20 feet, company pays over $8M settlement
On October 2, 2023, a Cruise robotaxi in downtown San Francisco struck and dragged a woman 20 feet after she was hit by a human driver. The autonomous vehicle attempted to pull over while the pedestrian was trapped underneath, causing life-threatening injuries. Cruise later paid a settlement of between $8 million and $12 million to the victim. The California DMV suspended Cruise's driverless permits after the company failed to show regulators complete footage of the incident.
- Company involved
- Cruise
4 source articles · read the reporting →
Zillow shuts down iBuying after algorithms cause $380 million loss
Zillow shut down its Zillow Offers iBuying business after its pricing algorithms, known as Zestimates, consistently overvalued homes. The company purchased thousands of homes at above-market prices and is now selling many at a loss, expecting to lose an estimated $380 million. Zillow attributed the failure to the algorithms' inability to accurately forecast home prices in a volatile market. The company will write off $569 million in home inventory and lay off 25% of its staff.
- Company involved
- Zillow
- AI system involved
- Zestimates
5 source articles · read the reporting →
Claude Code deletes developer's production database and snapshots
Alexey Grigorev used Claude Code to manage infrastructure with Terraform for his websites AI Shipping Labs and DataTalks.Club. Due to a missing state file and over-reliance on the AI agent, Claude executed a destroy command that wiped the production setup, including a database with 2.5 years of records and snapshots. Amazon Business support helped restore the data within a day. Grigorev is now implementing safeguards to prevent recurrence.
- Company involved
- AI Shipping Labs
- AI system involved
- Claude Code
2 source articles · read the reporting →
Hassan v ABC International Bank (Employment Tribunals): AI-hallucinated content in court filing, Costs Order
The AI generated fabricated legal citations that the claimant submitted to the tribunal, resulting in a costs order against the claimant.
1 source article · read the reporting →
Knight Capital fined $12 million for 2012 trading glitch that caused $460 million loss
On August 1, 2012, Knight Capital Americas LLC's automated equity router malfunctioned due to a code deployment error, sending over 4 million erroneous orders into the market within 45 minutes. The firm suffered a loss of more than $460 million. The SEC charged Knight Capital with violating the market access rule and Regulation SHO, and the firm agreed to pay a $12 million penalty and retain an independent consultant.
- Company involved
- Knight Capital Americas LLC
- AI system involved
- Knight Capital's automated equity router
10 source articles · read the reporting →
Australian government settles Robodebt class action for $1.7 billion
The Robodebt system, an automated debt recovery system used by Centrelink, calculated welfare debts using averaged income data from the ATO, leading to invalid debts. A class action was filed in November 2019 and settled in November 2020. The settlement provided refunds of over $751 million in invalid debts collected and $112 million in compensation to approximately 400,000 group members. The Australian government also dropped claims for over $1 billion in invalid debts.
- Company involved
- Australian Government (Centrelink)
- AI system involved
- Robodebt
10 source articles · read the reporting →
Hong Kong tycoon sues Tyndaris over AI hedge fund losses
Hong Kong tycoon Samathur Li Kin-kan is suing Tyndaris Investments after its AI-powered hedge fund, K1, lost over $20 million in a single day. Li alleges that the system was not as sophisticated as claimed. The trial is set to begin in London in April 2020.
- Company involved
- Tyndaris Investments
- AI system involved
- K1
10 source articles · read the reporting →
Ferris v. Amazon.com Services (N.D. Mississippi): AI-hallucinated content in court filing, Plaintiff ordered to pay Defendant’s reasonable costs related to…
The AI generated fabricated legal citations that were submitted to the court, affecting the defendant and the judicial process.
1 source article · read the reporting →
Air Canada Chatbot Fabricates Discount, Leading to Court Case
Air Canada's customer service chatbot allegedly fabricated a discount during a conversation with a customer last year. The incident resulted in a court case against the airline. Insurer Armilla stated that its new AI mishap policy would have covered the loss from selling tickets at the discounted price if the chatbot was found to have underperformed.
- Company involved
- Air Canada
- AI system involved
- chatbot
6 source articles · read the reporting →
Newham Council fined £145,000 over leaked gang matrix data
Newham Council was fined £145,000 by the Information Commissioner's Office after a leaked unredacted gangs matrix, containing details of 203 suspected gang members, ended up in the hands of rival gang members via Snapchat. The leak occurred in January 2017 when a council employee emailed both redacted and unredacted versions to 44 recipients. The ICO found the breach was unnecessary and that the council failed to report it promptly. The council apologised and accepted the breach was not deliberate.
- Company involved
- Newham Council
- AI system involved
- Gangs matrix
8 source articles · read the reporting →
Cense exposed 2.5 million records of auto accident victims online
On July 7, 2020, a security researcher discovered 2.5 million records containing personal and medical data of auto accident victims exposed online. The records, belonging to New York-based AI company Cense, included names, insurance policy numbers, claim numbers, and medical diagnosis notes. The data was labeled as staging data, possibly intended for temporary storage before being loaded into an AI system. After the researcher sent a responsible disclosure notice, Cense restricted public access to the database.
- Company involved
- Cense
- AI system involved
- Cense
5 source articles · read the reporting →
Gittemeier v. Liberty Mutual Personal Insurance Company (E.D. Missouri): AI-hallucinated content in court filing, Costs Order + Fine; One Attorney…
The AI system generated fictitious legal citations in a summary judgment motion, misleading the court and opposing counsel.
1 source article · read the reporting →
Allstate's proposed retention model charged big spenders more in Maryland
Allstate proposed an auto insurance rate adjustment plan in Maryland using a retention model algorithm that varied premium changes based on existing premiums. The algorithm would have increased rates up to 20% for customers already paying the highest premiums while capping increases at 5% for others, and would have denied meaningful discounts to thousands owed reductions. The Maryland Insurance Administration rejected the plan as discriminatory, but Allstate claims it was withdrawn and has continued proposing similar models elsewhere. The proposal never took effect in Maryland.
- Company involved
- Allstate Corporation
- AI system involved
- retention model
9 source articles · read the reporting →
Study finds Italian car insurers charge more based on birthplace
A study by the Universities of Padua, Udine and Carnegie Mellon found that Italian car insurers, including Genertel, Mps, Quixa and Con.Te, use birthplace and citizenship in pricing algorithms, charging some drivers over €1,000 more. The practice was ruled against in a 2018 decree involving Linear, but the study says it continues. The companies contacted denied or explained the findings.
- Company involved
- Genertel, Mps, Quixa, Con.Te
8 source articles · read the reporting →
Landberg v City of New York (CA NY (2d)): AI-hallucinated content in court filing, Monetary Sanction
AI-generated fake legal citations were included in a court filing, resulting in monetary sanctions against the attorney and his law firm.
1 source article · read the reporting →
Aguilar v. The Crawford Group, Inc. (D. Massachusetts): AI-hallucinated content in court filing, Adverse Costs Order; Pro hac vice status…
The AI generated fake legal citations that were included in a court filing, misleading the court and opposing counsel.
- Company involved
- Lindemann Law Firm
1 source article · read the reporting →
Barteca Holdings v. Tacobarn (D. Connecticut): AI-hallucinated content in court filing, Monetary Sanction; Bar Referral
The attorney filed a court document containing AI-hallucinated legal authorities, misleading the court and opposing party.
- Company involved
- Hilary Miller
1 source article · read the reporting →
Heriberto Perez-Castillo v. Todd W. Blanche (7th Cir. CA): AI-hallucinated content in court filing, Monetary Sanction; Admonishment; Bar Referral
The AI generated false legal citations and quotations in an appellate brief, misleading the court and harming the client's immigration appeal.
1 source article · read the reporting →
TOV Realty, LLC v. Suarez; Kosel Equity, LLC v. MacGregor (SC Connecticut): AI-hallucinated content in court filing, 6 hours CLE;…
The AI generated legal briefs containing fabricated citations, which were submitted to the Connecticut Supreme Court.
- Company involved
- GLG Law LLC
- AI system involved
- ChatGPT
1 source article · read the reporting →
LoanDepot algorithm denied mortgage to Black couple in Charlotte
In August 2019, Crystal Marie and Eskias McDaniels, a Black couple, were denied a mortgage for a house in Charlotte, North Carolina, by loanDepot's automated underwriting algorithm. The algorithm rejected the application because Crystal Marie was a contractor, not a full-time employee, despite her high credit score and income. After the couple enlisted their real estate agent and employer to intervene, the lender reversed the decision and cleared them to close. The couple alleged that race played a role in the denial, which loanDepot denied.
- Company involved
- loanDepot
- AI system involved
- Classic FICO and Fannie Mae/Freddie Mac automated underwriting software
10 source articles · read the reporting →
Mazaheri v Law Society of Ontario (Law Society Tribunal (ON)): AI-hallucinated content in court filing, Adverse Costs Order
AI generated fabricated legal citations in a court filing by Shahryar Mazaheri.
1 source article · read the reporting →
In re Rosslyn2016, LLC, et al. (S.D. Texas (Bankruptcy)): AI-hallucinated content in court filing, CLE on generative AI; Civil Contempt;…
The AI generated fabricated legal citations that were submitted to the bankruptcy court.
1 source article · read the reporting →
Southland Homes & Real Estate and Investment, LLC v. Lam (SC California): AI-hallucinated content in court filing, Monetary Sanctions; Bar…
The AI system generated legal briefs containing fabricated case citations, which were filed in court, affecting the court and the opposing party.
1 source article · read the reporting →
