The Rise of Nonconsensual AI ‘Nudify’ Bots on Telegram: A Deep Dive into a Growing Digital Nightmare

To go back to this article, head to My Profile, and then click on Saved stories.

In the beginning of 2020, deepfake specialist Henry Ajder discovered an early instance of a Telegram bot designed to remove clothing from images of women through the use of AI technology. Ajder noted that by then, the bot had already produced over 100,000 explicit images, some of which included minors, signaling a significant and disturbing milestone in the potential for harm through deepfake technology. From that point, the issue of deepfakes has only grown, becoming more widespread, more harmful, and simpler to generate.

A recent investigation by WIRED into Telegram groups sharing unauthorized explicit material has uncovered over 50 bots purportedly capable of generating explicit imagery of individuals in just a few clicks. These bots offer a range of functions; some assert they can digitally "strip" individuals in photos, while others purport to fabricate visuals of people engaged in different sexual activities.

According to a review by WIRED, analyzing statistics from each bot, a collective of 50 bots boasts over 4 million "monthly users." Among these, two bots each reported having over 400,000 monthly users, and an additional 14 bots claimed to have over 100,000 members each. These results highlight the extensive reach of explicit deepfake generating tools and underscore Telegram's significant role as a key hub for these technologies. Nonetheless, this analysis, primarily focusing on English-language bots, probably represents just a fraction of the total deepfake bots active on Telegram.

Ajder highlights a substantial surge in the individuals engaging and producing such content through Telegram bots. He expresses grave concerns over the accessibility and availability of these tools on the surface web, especially on one of the world's largest apps. According to him, these tools are wreaking havoc, particularly affecting young girls and women, by creating extremely distressing situations.

The creation of deepfake content without consent, also known as nonconsensual intimate image abuse (NCII), has surged since its appearance in late 2017. This increase is partly due to advancements in generative AI technology. The internet is now peppered with a range of sites and tools, including those offering "nudify" and "undress" services, as well as sophisticated software and Telegram bots. These resources are being exploited to victimize countless women and girls globally, affecting individuals as varied as Italy's prime minister to young students in South Korea. A recent survey revealed that 40 percent of U.S. students are aware of deepfakes related to their elementary to high school institutions over the past year.

WIRED has discovered Telegram bots linked to over 25 related Telegram channels, amassing over 3 million subscribers in total. These channels serve as hubs for updates on the bots' latest functionalities and promotional deals on "tokens" required for their operation. Furthermore, they provide a networking space for users to access new bots, especially if previous ones are taken down by Telegram.

Following an inquiry from WIRED regarding the permission of explicit deepfake content on its platform, Telegram took down the 75 bots and channels identified by WIRED. Telegram did not reply to several questions or explain the rationale behind the deletion of the channels.

Further investigations by WIRED uncovered more unauthorized deepfake Telegram channels and bots, highlighting the extent of the issue. Various channel operators announced that their bots had been shut down, with one stating, “We will create a new bot tomorrow.” These accounts were subsequently removed as well.

Concealed Within the Obvious

Telegram bots function as miniature applications operating within the Telegram platform. They coexist with the application's channels, capable of transmitting messages to countless subscribers; groups that accommodate interactions among up to 200,000 members; and direct messaging between individuals. Developers have engineered bots for various purposes including conducting trivia quizzes, translating texts, setting up notifications, or initiating Zoom calls. Unfortunately, these bots have also been manipulated to generate harmful deepfake content.

Owing to the potential dangers posed by deepfake technologies, WIRED refrained from experimenting with the Telegram bots and has chosen not to disclose the identities of any specific bots or channels. Even though Telegram's data indicates that these bots attracted millions of users on a monthly basis, the exact number of images generated by these bots remains uncertain. The usage varied significantly among individuals, with some possibly subscribing to numerous channels and bots without producing any images, while others might have generated several hundred.

Numerous deepfake bots examined by WIRED explicitly state their intended purposes. The titles and summaries of these bots often mention the generation of nude images or the digital removal of women's clothing. One bot's developers state, "Provide me with a photo, and I can manipulate its face or attire as you wish." Another bot entices with, "Discover the awe-inspiring capabilities of AI." Additionally, Telegram's suggestion feature can guide users to "related channels," facilitating their navigation among various channels and bots.

The majority of these bots necessitate users purchasing "tokens" to generate images, and there's ambiguity regarding their actual functionality. With the rapid growth of the deepfake creation ecosystem, this field has emerged as a profitable venture for developers of websites, apps, and bots. The surge in individuals seeking out "nudify" platforms has led to Russian hackers, according to 404Media, launching counterfeit sites designed to distribute malware among users.

The initial Telegram bots, discovered a few years back, were quite basic. However, the development of technology for generating more convincing AI images has advanced, and now, some of these bots are concealed right under our noses.

A bot with a user base exceeding 300,000 monthly did not make any direct references to explicit content either in its title or on its introductory page. Yet, upon engaging with the bot, it advertises access to over 40 types of imagery, a significant portion of which are explicit. This particular bot also provides a manual available on an external website, guiding users on generating high-quality content. Although bot creators have the option to impose terms of service that prohibit the upload of images without the depicted individual's permission or of minors, enforcement of these conditions seems to be minimal or nonexistent.

A different bot, boasting over 38,000 users, suggested that by submitting six photos of the same person—uniquely, it was among the few that offered to generate images of men—users could "educate" an AI system. This system would then be capable of producing new deepfake images of the person in question. Upon joining this bot, users were introduced to a selection of 11 "additional bots" created by the same team, presumably as a strategy to maintain their operations and skirt around being taken down.

"Emma Pickering, who leads the division focusing on abuse enabled by technology and financial empowerment at Refuge, the foremost domestic abuse charity in the UK, highlights that counterfeit images can inflict significant psychological damage. They have the potential to trigger emotional distress, including feelings of degradation, panic, mortification, and guilt. Despite the prevalence of this kind of abuse, especially in relationships between intimate partners, those responsible are seldom brought to justice. Pickering also notes a troubling rise in these incidents."

The creation and widespread distribution of explicit deepfakes have surged, with legislative bodies and technology firms lagging in their response to curb the phenomenon. In the United States, 23 states have enacted legislation aimed at combating deepfakes shared without consent, while technology companies have strengthened certain policies to deal with the issue. Despite these efforts, applications capable of generating explicit deepfakes have been discovered within the app stores of Apple and Google. Moreover, explicit deepfakes featuring Taylor Swift circulated extensively on X in January, and the sign-in systems provided by major technology companies have simplified the process for users to set up accounts on websites dedicated to deepfakes.

Kate Ruane, who leads the free expression project at the Center for Democracy and Technology, highlights that the majority of significant tech platforms have adopted rules against sharing intimate images without consent. Moreover, many of the largest platforms have committed to guidelines addressing the issue of deepfakes. Regarding Telegram’s policies, Ruane notes that it's ambiguous whether the platform explicitly bans the creation or sharing of intimate images without consent, pointing out that Telegram’s terms of service are not as comprehensive as those of other leading tech companies.

Criticism has often been directed at Telegram by activist organizations for its failure to adequately police the platform, which has become a haven for fraudulent actors, far-right organizations, and terror-linked materials. Following the arrest and charging of Pavel Durov, Telegram's CEO and creator, in France during August for various alleged violations, the messaging service has begun updating its user agreement and cooperating with police authorities. When asked by WIRED if it has a clear ban on explicit deepfake content, the company did not provide a response.

Implementing the Damage

Four years after uncovering deepfake Telegram bots, Ajder, the investigator behind the discovery, points out that the app is particularly vulnerable to deepfake exploitation. "Telegram offers search capabilities, enabling users to locate specific groups, conversations, and bots," Ajder explains. "It also supports bot hosting, essentially supplying the necessary tools. Additionally, it serves as the platform where the damaging content can be distributed and ultimately cause harm in its final stage."

Towards the end of September, multiple channels known for deepfake content reported that their bots had been taken down by Telegram. The reasons behind these takedowns remain unknown. On the 30th of September, a channel boasting 295,000 followers announced that Telegram had "prohibited" its bots, yet it shared a new link for a bot for its audience to access. (This channel was taken down following inquiries from WIRED to Telegram.)

"Elena Michael, the co-founder and director of #NotYourPorn, a campaign group dedicated to safeguarding individuals from image-based sexual abuse, expresses significant worry over apps like Telegram due to their challenging nature in terms of surveillance and tracking, especially when viewed from the survivors' standpoint."

Michael mentions that engaging with Telegram on matters of safety has been notably challenging, though she acknowledges there has been a slight improvement in the company's efforts recently. Nonetheless, she believes that the company needs to take a more active role in overseeing and eliminating inappropriate content by itself.

"Consider being a survivor tasked with handling this on your own; it really shouldn't be the responsibility of the individual," Michael states. "The responsibility ought to lie with the corporation to implement proactive measures instead of being reactive."

Explore Further…

Discover the World of Politics: Subscribe to our newsletter and tune into our podcast.

A remedy for America's firearm crisis from an emergency room physician

Video: Antony Blinken propels American diplomacy into the modern era

Insights from a Seasoned Hinge Enthusiast

Come be a part of the Energy Tech Summit happening on October 10th in Berlin.

Additional Content from WIRED

Evaluations and Manuals

© 2024 Condé Nast. All rights reserved. Purchases made through our site from products linked to our Affiliate Partnerships with various retailers may result in a commission for WIRED. Content from this site is protected by copyright and cannot be copied, distributed, shared, or used in any form without explicit written consent from Condé Nast. Advertising Choices.

Choose a global website