TikTok fails to stop beheading video going viral

Footage of a girl dancing spliced with a highly graphic and disturbing video of a man being beheaded by a group of men in a bathroom went viral on TikTok, raising concerns about the platform's safety and underlining the need for the effective regulation of algorithmic systems for minors. What happened A graphic video showing a person being beheaded went viral on TikTok after being uploaded by the user @mayengg03. The video was designed to "trick" the platform’s automated moderation systems: it began innocuously with a young girl in a black tank top dancing to a song by the rapper Doja Cat. Without warning, the footage spliced into a scene of Spanish-speaking men decapitating an individual in a bathroom. The video circulated widely, appearing on the "For You" feeds of unsuspecting users across the globe. While the exact number of views is unconfirmed, the incident triggered a massive wave of community-led warnings as users attempted to shield others from the trauma. TikTok eventually removed the original video and added it to its "Hashbank" (a digital fingerprinting database) to block re-uploads, but not before the content had already inflicted significant psychological distress on a primarily young audience. The video had previously been present on gore sites for two years, with reports at the time of its initial discovery in 2019 alleging the victim was a 19-year-old Mexican man. Why it happened The root cause of the incident appears to be limitations in TikTok's automated content moderation systems: AI filters are easily “tricked” when graphic content is encoded into a larger innocuous video, reducing the likelihood of detection before human review. While TikTok employs thousands of human moderators, the sheer volume of uploads - combined with an AI system that prioritises engagement metrics - allowed the video to reach a "viral" threshold before human intervention occurred. TikTok said it had " quickly" removed the offending item . H owever, TikTok users continued to say long after the event that they were too scared to use the app lest they saw the beheading. Wh at it means For users, particularly minors, witnessing videos of this type can be acutely distressing, sometimes resulting in acute psychological trauma. Furthermore, the "bait-and-switch" nature of the video created a sense of "digital unsafety," where even seemingly benign content can harbour harmful imagery. For society, t he incident nighlighted the limitations of "automated safety" and sparked a heated conversation about the accountability of social media giants and the need for stricter "Online Safety" legislation. It demonstrated that as long as platforms prioritise algorithmic speed and engagement over rigorous human-in-the-loop moderation, "realised harms" such as the normalisation of extreme violence will continue to permeate public digital spaces. System 🤖 TikTok For You TikTok content moderation system Developer: TikTok Country: U SA Sector: Media/entertainment/sports/arts Purpose: Moderate content; Recommend content Technology: Content moderation system; Recommendation system; Machine learning Issue: Accountability; Accuracy/reliability; Normalisation; Safety; Transparency Timeline ⏰ June 9–10, 2021. A spliced beheading video uploaded by user @mayengg03 bypasses TikTok's AI filters. It goes viral globally, appearing on the "For You" feeds of millions of users, many of whom are minors June 11, 2021. TikTok bans the original account and issues a formal apology. The video is added to the "Hashbank" to automate the removal of re-uploads December 2021. Former content moderator Candie Frazier files a class-action lawsuit, alleging she developed PTSD and anxiety from reviewing thousands of copies of the graphic footage due to TikTok's inadequate mental health safeguards October 26, 2023. UK Online Safety Act Becomes Law The UK passes the Online Safety Act, specifically citing events like the 2021 beheading as reasons to impose a "Duty of Care" on social medi

Date it happened

2021-06-01

Organisation involved

TikTok

Product, system or model

For You

This is a record of what was reported, not a finding that anyone broke the law. If it names your organisation and you believe it is wrong, the corrections process is free and open to everyone.