Michael Cohen supplies fake AI legal citations to lawyer
Google's Bard chatbot generated false legal citations that were used in a legal filing involving Donald Trump's former lawyer Michael Cohen, highlighting concerns about the system's propensity to 'hallucinate' 'facts' and users' poor understanding of its risks. In November 2023, Michael Cohen submitted legal citations he believed to be accurate to his lawyer David Schwartz that he thought might help his request for an early end to court supervision. Cohen had pled guilty to tax evasion and campaign finance violations i n 2018 and served time in prison. However, a court filing (pdf) revealed the citations had been generated by Google's Bard chatbot, and were false - an admission made after the judge overseeing Cohen’s case had said that he could not find any of the decisions Schwartz had cited and demanded an explanation. In a sworn statement, Cohen said: 'As a non-lawyer, I have not kept up with the emerging trends (and related risks) in legal technology and did not realize that Google Bard was a generative text service that, like Chat-GPT, could show citations and descriptions that looked real but actually were not'. Hallucination (artificial intelligence) In the field of artificial intelligence (AI), a hallucination or artificial hallucination (also called bullshitting, confabulation or delusion) is a response generated by AI that contains false or misleading information presented as fact . Source: Wikipedia 🔗 System 🤖 Gemini Operator: Michael Cohen Developer: Microsoft Country: USA Sector: Business/professional services Purpose: Generate text Technology: Chatbot; NLP/text analysis; Neural network; Deep learning; Machine learning; Reinforcement learning Issue: Accuracy/reliability; Mis/disinformation
- Date it happened
- 2023-01-01
- Organisation involved
- Michael Cohen
- Product, system or model
- Gemini
This incident was imported from AIAAIC and is used under CC BY-SA 4.0. Our additions to it — the structured fields, the translation, the checks against other reports — are published under the same licence.
This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.