Anti-immigrant narratives and sexualised AI videos fuel billions of TikTok views
Researchers discovered 354 AI-focused accounts that amassed 4.5 billion views in a month. Hundreds of accounts on TikTok are gaining billions of views by producing AI-generated content, including anti-immigrant and sexualized material, according to a report.
Researchers said they found 354 AI-focused accounts distributing 43,000 posts made using generative AI tools, which accumulated 4.5 billion views over a month-long period.
According to AI Forensics, a Paris-based non-profit organization, some of these accounts try to manipulate TikTok’s algorithm—which decides what content users see—by uploading large amounts of content in the hopes of going viral.
One account posted up to 70 times a day or at the same time of day, indicating an automated account, and most of the accounts were launched at the start of the year.
Last month, TikTok revealed there were at least 1.3 billion AI-generated posts on the platform. More than 100 million pieces of content are uploaded daily, suggesting that labeled AI material represents a small portion of TikTok’s catalog. TikTok is also providing users the option to reduce the amount of AI content they see.
Of the accounts that posted content most frequently, half focused on content related to the female body. “These AI women are always stereotypically attractive, with sexualized attire or cleavage,” the report stated.
AI Forensics found that half of the content posted by the accounts was not labeled, and less than 2% carried the TikTok label for AI content—something the nonprofit warned could increase the material’s deceptive potential. Researchers added that the accounts sometimes evade TikTok’s moderation for months despite posting content prohibited by its terms of service.
Dozens of the accounts mentioned in the study have since been deleted, researchers said, indicating that some were removed by moderators.
Some of the content was in the form of fake broadcast news segments with anti-immigrant narratives and material sexualizing female bodies, including what appeared to be underage girls. The female body category accounted for half of the top 10 most active accounts, AI Forensics said, while some fake news pieces used known broadcasting brands like Sky News and ABC.
Some posts were removed by TikTok after being reported to the platform by the Guardian.
TikTok claimed the report’s assertions were “unsubstantiated” and suggested the researchers singled it out for an issue affecting multiple platforms. In August, the Guardian reported that nearly one in ten of the fastest-growing YouTube channels globally displayed only AI-generated content.
“On TikTok, we remove harmful AIGC [artificial intelligence-generated content], block hundreds of millions of bot accounts from being created, invest in industry-leading AI-labeling technologies, and empower people with tools and education to control how they experience this content on our platform,” a TikTok spokesperson said.
The most popular accounts highlighted by AI Forensics in terms of views had posted “slop,” the term for AI-made content that is nonsensical, bizarre, and designed to clutter people’s social media feeds—such as animals in an Olympic diving contest or talking babies. The researchers acknowledged that some of the slop content was “entertaining” and “cute.”
TikTok guidelines prohibit using AI to depict fake authoritative sources, the likeness of under-18s, or the likeness of adults who are not public figures.
“This investigation of [automated accounts] shows how AI content is now integrated into platforms and a larger virality ecosystem,” the researchers said.
“The blurring line between authentic human and synthetic AI-generated content on the platform is signaling a new shift towards more AI-generated content on users’ feeds.”
The researchers examined data from mid-August to mid-September. Some of the content attempts to monetize users, including promoting health supplements via fake influencers, advertising tools to create viral AI content, and seeking sponsorships for posts.
AI Forensics, which also highlighted the prevalence of AI content on Instagram, said it welcomed TikTok’s decision to allow users to limit the amount of AI content they see, but emphasized that labeling needed improvement.
“Given the structural and non-negligible failure to identify such content, we remain skeptical regarding the success of this feature,” they stated.
The researchers added that TikTok should consider creating an AI-only feature on the app to separate AI-made content from human-generated posts. “Platforms must go beyond weak or optional ‘AI content’ labels and consider separating generative content from human-created material or developing a fair system that enforces systematic and visible labeling of AI content,” they said.
Head of Investigations
Read more similar news:
Comments:
comments powered by Disqus
