Incidents gathered from public reporting around the world. Each one links to the articles it came from. None of it is a finding that anyone broke the law.
Reports people file about their own experience are not shown here and never will be without their agreement. Tell us what happened to you.
271 incidents closest to “FRISS Risk Assessment at Underwriting” · matched on meaning · public reporting
The Parole Board imposed a condition preventing Susan Neill-Fraser from communicating with the media to claim innocence.
1 source article · read the reporting →
The Allegheny Family Screening Tool, an algorithm used to screen child neglect reports, has been found to flag a disproportionate number of Black children for mandatory investigation compared to white children. A technical glitch also presented wrong risk scores for over two years. County officials say social workers can override the algorithm and that the glitch has been fixed, but critics argue the tool hardens racial disparities and lacks transparency.
10 source articles · read the reporting →
Airbnb is accused of using an algorithm acquired from Trooly to score users' trustworthiness based on publicly available data, including social media and occupation. Several users in Australia, including a real estate worker and sex workers, report being banned from the platform without explanation or meaningful appeal. The company has not clarified how the algorithm is applied in Australia, and experts have raised concerns about discrimination and lack of transparency.
4 source articles · read the reporting →
The AI generated false legal citations that were included in a court filing, misleading the court.
1 source article · read the reporting →
A child protection worker at the Department of Families, Fairness and Housing (DFFH) used ChatGPT to draft a Protection Application Report for the Children’s Court, entering sensitive personal information about a child. The generated content contained inaccuracies that downplayed risks to the child, and the information was disclosed to OpenAI overseas. An investigation by the Office of the Victorian Information Commissioner found DFFH failed to ensure accuracy and protect personal information, contravening IPPs 3.1 and 4.1. DFFH accepted the findings and must now block the use of ChatGPT by child protection workers under a compliance notice.
9 source articles · read the reporting →
Prakazrel 'Pras' Michel, a former member of the Fugees, was convicted of criminal conspiracy charges for attempting to influence US presidential administrations. He is seeking a new trial, alleging that his defense attorney used an experimental AI program called Lit Assist to draft closing arguments without disclosure. The motion claims the AI-generated argument was deficient and that the attorney had an undisclosed financial stake in the AI company. The court has scheduled an evidentiary hearing.
10 source articles · read the reporting →
ProPublica's analysis of auto insurance premiums in California, Illinois, Texas and Missouri found that major insurers such as Geico, Allstate and Liberty Mutual charge minority neighborhoods up to 30% more than white areas with similar accident costs. The disparity is alleged to result from pricing algorithms that may inadvertently use zip code as a proxy for race. Otis Nash, a 26-year-old Chicagoan, pays $190.69 per month for Geico insurance while a white driver in a safer neighborhood pays $54.67. The insurers deny discrimination, stating they do not collect race data.
5 source articles · read the reporting →
Jennifer Miller, whose son Rhys Cauzzo took his own life in 2017 while facing Centrelink debts of about $17,000, told the robodebt royal commission that the government stonewalled her for years. The commission heard that Cauzzo's debt was unlawfully calculated using income averaging and that no vulnerability indicator was on his file despite his mental health conditions. Miller said she felt vindicated by the commission's findings, while the former human services minister Alan Tudge had claimed the department acted appropriately. The robodebt scheme, which ran from 2015 to 2019, ended in a $1.8bn settlement after a class action.
10 source articles · read the reporting →
In 2022, Juliette, a single mother on welfare in France, was flagged by CNAF's secretive fraud detection algorithm. A fraud investigator later determined she owed thousands of euros, which were deducted from her monthly payments. The algorithm, which scores half of France's population, is accused of discriminating against vulnerable people by using factors like single parenthood and low income.
10 source articles · read the reporting →
Engineer.ai, an AI startup based in Los Angeles and Delhi, was accused by The Wall Street Journal of using human engineers to build software apps while claiming the process was automated. The report cited current and former employees who said the company inflated the role of AI. Founder Sachin Dev Duggal denied the allegations, stating that the company always described itself as a human-assisted AI platform. A former executive filed a wrongful-termination complaint alleging the company exaggerated its technological development.
6 source articles · read the reporting →
Whitebridge AI, a Lithuanian company, is accused of generating false reputation reports using unlawfully scraped social media data. The reports contained false warnings for 'sexual nudity' and 'dangerous political content'. Privacy group Noyb filed a complaint with the Lithuanian data protection authority, alleging violations of the GDPR. The complainants sought access to their data but received no response, and were later required to provide a qualified electronic signature to correct errors.
6 source articles · read the reporting →
In May 2015, Poland's Ministry of Labor and Social Policy introduced an algorithmic system to profile unemployed people into three categories based on their readiness to work, location, and disabilities. The system is accused of being discriminatory and lacking transparency, as it uses default criteria that may lead to social exclusion. A report by the Panoptykon Foundation in October 2015 highlighted these issues and recommended reforms, including granting the right to appeal.
10 source articles · read the reporting →
Upstart Holdings, Inc., a cloud-based AI lending platform, is accused of making false and misleading statements about its AI model's ability to account for macroeconomic factors. The lawsuit, filed on May 13, 2022, alleges that the company's positive statements about its business were materially false. Investors suffered losses when Upstart's stock price fell 56% on May 9, 2022, after the company reduced its fiscal 2022 guidance.
10 source articles · read the reporting →
The Oregon Department of Human Services announced it will stop using its Safety at Screening Tool algorithm, which helped hotline workers decide which families to investigate for child abuse and neglect. The decision followed concerns about racial bias and disparities, as the algorithm was inspired by a Pennsylvania tool that had flagged a disproportionate number of Black children. The agency will replace the algorithm with a new Structured Decision Making model.
10 source articles · read the reporting →
Instacart shoppers allege that the company's automated rating and fraud detection system unfairly penalizes them for issues outside their control, such as customer fraud. The system assigns low ratings and blames shoppers for missing items, leading to reduced pay and deactivation. The Gig Workers Collective has sent an open letter to CEO Fidji Simo demanding changes including higher base pay, item commission, and occupational death benefits.
10 source articles · read the reporting →
Inflection, a background check company used by Airbnb, Uber, and other platforms, has been sued multiple times for providing inaccurate reports. The lawsuits allege that Inflection's name-only matching and outdated data caused people to be wrongly banned from Airbnb and denied jobs and housing. Some cases have been settled, while others remain ongoing. The company is accused of violating the Fair Credit Reporting Act.
8 source articles · read the reporting →
In 2016, Carmen Arroyo requested to move her disabled son Mikhail into a larger apartment in the same complex. The landlord, WinnResidential, used CoreLogic's CrimSafe tenant screening software, which flagged a dropped retail theft charge against Mikhail as disqualifying. The landlord rejected the application without explanation. Arroyo sued CoreLogic and WinnResidential under the Fair Credit Reporting Act and the Fair Housing Act, alleging that the software disproportionately excludes people of colour. The case was scheduled for trial in August 2021.
10 source articles · read the reporting →
The Scottish Prison Service's Level of Service and Case Management System failed to update risk assessments when new information was entered, leading to incorrect risk scores for 495 closed cases and 285 open cases. As a result, eight prisoners serving life for murder or rape were granted temporary release without accurate risk assessments. Justice Secretary Keith Brown acknowledged the error and stated that the cases would be reviewed. Opposition politicians called for an apology, claiming the releases put the public at risk.
10 source articles · read the reporting →
The Dutch Data Protection Authority (AP) found that the Benefits Department of the Tax and Customs Administration (Belastingdienst) unlawfully processed the dual nationality of 1.4 million applicants for child care benefits. The data was used as an indicator in an automated risk-scoring system that flagged applications as high-risk, which was unnecessary and discriminatory. The AP concluded that the processing violated the GDPR and was discriminatory. The AP completed its fact-finding and will consider imposing a sanction.
10 source articles · read the reporting →
The NFL Concussion Settlement disability assessments allegedly use race as a factor to lower baseline cognitive function scores for black and brown players, making it harder for them to qualify for disability benefits. Physicians are reportedly pressured to use this scale. A petition demands the practice stop and that demographic data be released. The settlement is overseen by the United States District Court for the Eastern District of Pennsylvania.
10 source articles · read the reporting →
A female student with dark skin at Vrije Universiteit Amsterdam alleges that the Proctorio anti-cheating software used during online exams in the 2020/21 academic year discriminated against her on the basis of race. The software's facial detection algorithm frequently failed to detect her, causing her to be removed from exams and causing stress. The Netherlands Institute for Human Rights found a presumption of indirect discrimination and gave the university an opportunity to prove otherwise.
10 source articles · read the reporting →
Artificial intelligence company Prisma Labs is accused of violating Illinois' biometric privacy law by collecting facial geometry data from users of its Lensa app. A proposed class action was filed, but a California federal judge granted Prisma's motion to compel arbitration because users agreed to it during sign-up. The plaintiff alleges the app collected his facial data without consent.
8 source articles · read the reporting →
The Washington Post is accused of using an algorithm to set subscription prices based on subscribers' personal data. Some subscribers received emails stating that their new price was determined by an algorithm using their personal data. The practice, known as surveillance pricing, has drawn criticism from Democratic lawmakers who have proposed legislation to ban it.
6 source articles · read the reporting →
The Illinois Department of Children and Family Services ended its Rapid Safety Feedback program, which used data mining to predict child abuse risk, after the agency's director called the technology unreliable. The system, developed by Eckerd Connects and Mindshare Technology, assigned risk scores to children but failed to flag several high-profile child deaths. The program was also criticised for overwhelming caseworkers with alerts and for potential bias against poor children of colour. DCFS decided not to renew the contract.
10 source articles · read the reporting →
