AI detector falsely accuses student of cheating at Central Methodist University
Moira Olmsted, a student at Central Methodist University, submitted a written assignment and received a zero grade after her professor said an AI detection tool flagged her work as AI-generated. Olmsted alleges the accusation was false and that her writing had been flagged at least once before. The incident highlights the risk of AI detectors falsely accusing students of cheating.
- Company involved
- Central Methodist University
1 source article · read the reporting →
Student at University of South-Eastern Norway used deepfake video to cheat on Spanish exam
A student at the University of South-Eastern Norway submitted AI-generated deepfake videos for Spanish language assessments in autumn 2023. The videos featured a synthetic voice and manipulated face, with perfect pronunciation but elementary errors. The university's appeals committee found her guilty of cheating, annulled her exams, and excluded her for two semesters. The student admitted using AI and withdrew from the programme.
- Company involved
- Universitetet i Sørøst-Norge
1 source article · read the reporting →
Universities' use of race in student risk algorithms harms Black and Latinx students
The Markup found that several universities, including UMass Amherst, UW–Milwaukee, and Texas A&M, use EAB's Navigate software which incorporates race as a predictor of student dropout risk. The algorithms label Black and Latinx students as high risk at disproportionately high rates compared to White students. Students, professors, and experts worry this is pushing minority students out of math and science majors through biased advising.
- Company involved
- University of Massachusetts Amherst,University of Wisconsin–Milwaukee,Texas A&M University,Texas Tech University,South Dakota State University
- AI system involved
- Navigate
1 source article · read the reporting →
Turnitin's AI Detector Falsely Flags Student's Essay as AI-Generated
A Washington Post test of Turnitin's new AI writing detector found it erroneously flagged 8% of a high school student's original essay as likely generated by ChatGPT. The student, Lucy Goetz, had not used AI, raising concerns about false accusations of cheating. Turnitin acknowledged the issue and added a caution flag, but educators worry about the potential for baseless academic-integrity investigations. The incident highlights the challenges of detecting AI-generated text and the need for careful human review.
- Company involved
- Turnitin
- AI system involved
- Turnitin AI writing detector
1 source article · read the reporting →
Hanoi students caught using AI to cheat in graduation exam
Two students in Hanoi used mobile phones to photograph questions from the national high school graduation exam and submitted them to AI apps like StudyX, Gemini, and ChatGPT for answers. Proctors detected the cheating, and the students were suspended. Hanoi police are investigating the incident, which has raised concerns about exam security and integrity. The students face disqualification and possible administrative or criminal penalties.
- AI system involved
- StudyX, Gemini, ChatGPT
2 source articles · read the reporting →
Texas A&M Professor Uses ChatGPT to Falsely Accuse Students of Cheating
Jared Mumm, an instructor at Texas A&M University–Commerce, used ChatGPT to check if students' writing was AI-generated, leading him to accuse them of using the chatbot. He informed students they would receive an incomplete grade and those deemed guilty would get a zero, causing distress and temporarily withholding one student's diploma. The university investigated and stated that no students failed or were barred from graduating, and the professor is working individually with students to resolve the issue.
- Company involved
- Texas A&M University–Commerce
- AI system involved
- ChatGPT
5 source articles · read the reporting →
TikTok's recommendation algorithm shows racial bias, researcher finds
A researcher at UC Berkeley observed that TikTok's recommendation algorithm suggested accounts to follow that looked similar in ethnicity and hair colour to the account just followed. TikTok stated it uses collaborative filtering based on user behaviour, not facial recognition, but the researcher noted this can still reproduce biases and create feedback loops that limit diversity. The observation was anecdotal and calls for further investigation into potential coverage bias against creators of colour.
- Company involved
- TikTok
- AI system involved
- TikTok recommendation algorithm
4 source articles · read the reporting →
Bunce v. Visual Technology Innovations (2) (E.D. Pennsylvania): AI-hallucinated content in court filing, Monetary Sanction, Additional CLE
The AI generated fabricated legal citations in a court filing, misleading the court and opposing counsel.
- Company involved
- Mr. Rajan
1 source article · read the reporting →
Researchers hide prompts to manipulate AI peer review
Nikkei found hidden prompts in 17 preprints on arXiv that instructed AI tools to give the papers positive reviews and ignore negatives. The manuscripts were linked to 14 academic institutions, including Waseda University, KAIST, Peking University, the National University of Singapore, the University of Washington and Columbia University. At least one paper was set to be withdrawn, while some researchers defended the prompts as a check on reviewers who improperly use AI.
- Company involved
- Multiple academic institutions
- AI system involved
- arXiv
1 source article · read the reporting →
TikTok's internal AI avatar tool generated harmful content without guardrails
TikTok mistakenly released an internal version of its Symphony Digital Avatars tool, allowing users with personal accounts to generate videos of paid actors saying anything, including quotes from Hitler and messages telling people to drink bleach. The tool lacked the AI-generated watermark and content guardrails of the public version. TikTok acknowledged the technical error and removed the internal tool, stating that any such content posted would have been rejected under its policies.
- Company involved
- TikTok
- AI system involved
- Symphony Digital Avatars
2 source articles · read the reporting →
Argentine judge's sentence annulled after ChatGPT use revealed by copy-paste phrase
A criminal court in Esquel, Chubut, Argentina, annulled a sentence after discovering that Judge Carlos Rogelio Richeri had used ChatGPT to draft the decision. The judge accidentally left in the phrase 'Aquí tienes el punto IV reeditado, sin citas y listo para copiar y pegar,' revealing the AI's involvement. The appeals court ruled that delegating the judicial decision to AI violated the principle of a natural judge and ordered a new trial with a different judge, while the Superior Tribunal of Justice will investigate the judge's ethical lapse.
- Company involved
- Juzgado Penal de Esquel
- AI system involved
- ChatGPT
3 source articles · read the reporting →
RFK Jr.'s MAHA Report Riddled with AI-Generated Fake Citations
An investigation found dozens of errors in the White House's 'Make America Healthy Again' commission report, including broken links, wrong authors, and entirely fictitious sources. The presence of 'oaicite' markers in URLs strongly suggests that ChatGPT was used to generate citations, leading to AI hallucinations. The White House described the errors as a 'formatting issue' and updated the report to correct some of them, while defending the report's substance.
- Company involved
- U.S. Department of Health and Human Services
- AI system involved
- ChatGPT
3 source articles · read the reporting →
Educational Testing Service's E-rater algorithm biases essay scores against minority students
The Educational Testing Service's E-rater algorithm, used to grade essays on the GRE and other standardized tests, has been found to systematically give higher scores to students from mainland China and lower scores to African American students compared to human graders. The bias stems from the algorithm's reliance on surface-level metrics like vocabulary and sentence length, which disadvantage certain groups. Despite studies dating back to 1999, the bias persists, and in many states, only a small percentage of essays are reviewed by humans.
- Company involved
- Educational Testing Service
- AI system involved
- E-rater
10 source articles · read the reporting →
NSW Education Standards Authority used AI-generated image in HSC English exam without disclosure
The NSW Education Standards Authority (NESA) used an AI-generated image as a stimulus in the 2024 HSC English exam without disclosing its origin. The image, created by Florian Schroeder using OpenAI's ChatGPT and Dall-E 2, was published on Medium in July 2023. Students suspected AI use due to irregularities in the image, and NESA initially declined to confirm. After the Sydney Morning Herald confirmed the image was AI-generated, NESA stated that students would be marked on their response to the question, not the image's origin.
- Company involved
- NSW Education Standards Authority
- AI system involved
- ChatGPT and Dall-E 2
6 source articles · read the reporting →
Middle schooler beats Edgenuity grading algorithm to get perfect score
A seventh-grade student in the Los Angeles Unified School District received a failing grade on a history assignment graded by Edgenuity's automated scoring algorithm. With help from his mother, a history professor, he reverse-engineered the algorithm by writing a paragraph with relevant keywords and a jumble of words, earning a perfect score. The incident highlights concerns about the accuracy and fairness of automated grading systems in education.
- Company involved
- Los Angeles Unified School District
- AI system involved
- Edgenuity
10 source articles · read the reporting →
Proctortrack data breach exposed student data from online proctoring
Proctortrack, an online proctoring service used by universities, suffered a data breach in September 2020 when its source code was leaked online. An analysis by Consumer Reports found that the code contained hard-coded passwords and exposed the names and email addresses of over 150 students. The company acknowledged the leak but said no harm resulted. Students had been required to use the software, which performed facial recognition and recorded video during exams.
- Company involved
- Proctortrack
- AI system involved
- Proctortrack
10 source articles · read the reporting →
College student uses GPT-3 to create fake blog that reaches #1 on Hacker News
Liam Porr, a college student at UC Berkeley, used OpenAI's GPT-3 language model to generate a fake blog under a fake name. One of the posts reached the number-one spot on Hacker News, fooling tens of thousands of readers. Porr later confessed and retired the blog after two weeks. The experiment demonstrated the ease of creating convincing AI-generated content.
- AI system involved
- GPT-3
10 source articles · read the reporting →
ChatGPT-4 generates false accusations and quotes about law professors
OpenAI's ChatGPT-4 generated false accusations and fabricated quotes about law professors when asked about scandals and crimes involving law professors. The system produced entirely made-up newspaper citations and quotes, falsely alleging misconduct such as harassment and tax fraud. The article reports that these hallucinations could mislead users who trust the generated quotes. The author corrected an earlier error attributing similar results to ChatGPT-4 instead of ChatGPT-3.5, but confirmed that ChatGPT-4 also produces such false outputs.
- Company involved
- OpenAI
- AI system involved
- ChatGPT-4
10 source articles · read the reporting →
NewsGuard finds ChatGPT-4 generates all 100 false narratives in test
In March 2023, NewsGuard tested ChatGPT-4 by prompting it with 100 false narratives from its database. The chatbot generated all 100 false narratives, often without disclaimers, and produced more persuasive misinformation than its predecessor ChatGPT-3.5. NewsGuard reported that OpenAI had not fixed the flaw before releasing the tool, and the company did not respond to requests for comment.
- Company involved
- OpenAI
- AI system involved
- ChatGPT-4
3 source articles · read the reporting →
UW-Madison disables Honorlock after skin tone recognition failure
The University of Wisconsin-Madison disabled the exam pause feature of its Honorlock anti-cheating software in March 2021 after three students complained that the software failed to recognize their darker skin tones and paused their exams. The software, used since online classes began, automatically pauses exams when it cannot detect facial features. Honorlock denied the issue was related to skin tone, attributing it to students looking away from their webcams. The university responded by disabling the feature.
- Company involved
- University of Wisconsin-Madison
- AI system involved
- Honorlock
10 source articles · read the reporting →
Tow Center finds ChatGPT Search misattributes publisher content
The Tow Center for Digital Journalism tested ChatGPT Search with 200 block quotes from 20 publishers and found 153 partially or fully incorrect citations. The chatbot often conjured responses when it could not access content, sometimes citing plagiarized or syndicated versions. OpenAI responded that the study was atypical and that it supports publishers with clear links and attribution.
- Company involved
- OpenAI
- AI system involved
- ChatGPT Search
6 source articles · read the reporting →
Company fires HR team after ATS auto-rejects manager's CV due to filtering error
A company's applicant tracking system (ATS) auto-rejected qualified candidates' resumes for three months because it was filtering for the outdated framework AngularJS instead of the required Angular framework. The manager discovered the flaw by submitting his own CV under a pseudonym and found it was rejected within seconds. After the manager reported the issue to upper management, the company investigated and dismissed half of its HR team. No legal action or regulatory involvement is reported.
4 source articles · read the reporting →
Study finds Midjourney, DALL-E 2, Stable Diffusion accept over 85% of fake news prompts
A study by AI startup Logically tested Midjourney, DALL-E 2, and Stable Diffusion and found that they accepted over 85% of prompts seeking to generate fake political news. The systems generated images of ballot stuffing, small boat arrivals, and explosions. Logically warned that the lack of moderation could pose threats to upcoming elections. Stability AI responded by stating its ethical use license and measures to prevent misuse.
- Company involved
- Midjourney, OpenAI, Stability AI
- AI system involved
- Midjourney, DALL-E 2, Stable Diffusion
8 source articles · read the reporting →
GlobalVillageSpace.com used AI to rewrite New York Times articles without credit
NewsGuard identified 37 websites using AI chatbots to rewrite articles from mainstream news outlets without credit. One example is GlobalVillageSpace.com, which appeared to use AI to rewrite a New York Times article about NFL tight end Darren Waller. The site published an AI error message indicating the article was rewritten. After NewsGuard contacted the site, it removed the article but did not respond to inquiries.
- Company involved
- GlobalVillageSpace.com
6 source articles · read the reporting →
