LEAP AI invents family court legal case citations

An Australian lawyer was referred to a legal complaints authority after AI legal software he had used generated fictitious case citations that were used during a family court case. What happened During a hearing on July 19, 2024, a Melbour ne-based lawyer submitted a list of purported case precedents that neither the judge nor her staff could verify. Upon investigation, it was revealed that the citations had been fabricated by LEAP's LawY AI tool and that the lawyer had not confirmed their accuracy before presenting them in court , leading to a postponement of the hearing and a slap on the wrist by the legal authorities. Why it happened The lawyer admitted to using Leap's AI capabilities without understanding it's hallucinatory limitations . He ultimately issu ed an apology and offer ed to cover costs incurred by the other party due to the adjournment. What it means The incident raises concerns about the accuracy and reliability of LEAP's generative AI software, a nd highlights the importance of verifying AI-generated information . It is part of a broader trend in which multiple lawyers have faced repercussions for relying on unreliable generative AI systems without properly checking their outp uts and, as such, ra ises questions about the ethical implications of using AI in legal practice. Similar cases have emerged globally, including instances in Canada and the UK where legal professionals cited fabricated cases generated by tools like ChatGPT . Our view This is yet another example of a lawyer failing to understand that generative AI systems are prone to be inaccurate and unreliable, or understanding their limitations but unwilling to take time to check their outputs. Lawyers are often handed ethics mandates given their supposed understanding of legal and ethical questions - an assumption that is clearly questionable in instances such as this. More broadly, perhaps lawyers using generative AI to cut corners should consider cutting their fees. System 🤖 LawY Developer: Leap Country: Australia Sector: Business/profes sional services Purpose: Provide legal answers Technology: Gene rative AI Issue: Accountability; Accuracy/reliability Legal, regulatory 👩🏼‍⚖️ Dayal [2024] FedCFamC2F 1166 Handa & Mallick

Date it happened

2024-07-01

Product, system or model

LawY

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.