India’s Election Frontier: Deepfakes, Digital Clones, and the Future of Political Campaigns

To go back to this article, navigate to My Profile and then click on Saved stories to view them.

Varsha Bansal Nilesh Christopher

Millions of Deepfakes Target Indian Electorate, With Politicians' Blessing

During a sweltering April day in Ajmer, Rajasthan, Shakti Singh Rathore, a regional political figure, positioned himself in front of a greenscreen, ready to record a brief video. Appearing anxious, it marked his initial experience with being digitally duplicated.

Dressed in a sharp white shirt complemented by a traditional saffron scarf adorned with a lotus flower—the emblem of the BJP, India's governing party—Rathore joined his hands in a gesture of greeting before addressing the crowd in Hindi. "Namashkar," he initiated. "To all my brothers—"

Before he could proceed further, the shoot's director stepped into the scene. At 31 years of age, Divyendra Singh Jadoun, noticeable for his baldness and dense black beard, informed Rathore that his excessive movement was problematic for the camera. Jadoun's objective was to gather sufficient audio and visual material to create a convincing AI-generated deepfake of Rathore, aiming to persuade 300,000 prospective voters in Ajmer that they were engaging in a direct conversation with him. However, too much motion could disrupt the algorithm's effectiveness. Jadoun instructed his subject to maintain direct eye contact with the camera and to restrict movement to his mouth alone. “Let's take it from the top,” he commanded.

At Polymath Synthetic Media Solutions, autodidact deepfake expert Divyendra Singh Jadoun amasses video and audio recordings of regional politicians to convert their messages into various languages for engaging with voters. In this instance, Shakti Singh Rathore's address is recreated in Hindi, Tamil, Sanskrit, and Marathi. Video by: Nilesh Christopher/Divyendra Singh Jadoun/WIRED

Currently, the biggest democratic nation is in the midst of elections. Nearly a billion people in India have the right to vote in this significant national election, where deepfakes may have a crucial, and possibly polarizing, impact. Indian political factions are manipulating artificial intelligence to alter perceptions via low-cost audio manipulations, doctored images, and AI-generated parodies. Although the worldwide conversation about deepfakes typically centers on the dangers of misinformation and disinformation, numerous politicians in India are employing this technology differently, aiming it at engaging with voters.

Political parties and candidates across India are embracing artificial intelligence as a tool to bridge the linguistic diversity of the nation, which boasts 22 recognized languages and countless local dialects. This technology is being harnessed to tailor messages for remote communities. Meanwhile, in the United States, the use of AI to generate voices for unsolicited calls has been banned. Contrarily, in India, the creation and use of deepfake technology for such purposes has evolved into a lucrative industry, valued at approximately $60 million. In the lead-up to the national elections starting in April, over 50 million calls using AI-created voice clones were placed, and industry insiders anticipate millions more will occur throughout the election period. This insight was shared with WIRED by one of India's leading business communication firms.

Jadoun has quickly become the face of this rapidly expanding sector. His enterprise, Polymath Synthetic Media Solutions, ranks among a growing number of Indian firms offering deepfake technologies tailored for political figures. During the current election period, Jadoun's team has successfully executed five AI-driven campaigns, earning a total of $55,000. His pricing undercuts that of major political advisory firms, asking 125,000 rupees (about $1,500) for the creation of a digital persona and 60,000 rupees (approximately $720) for a voice duplicate. Notably, Jadoun has produced deepfake content for Prem Singh Tamang, the Chief Minister of Sikkim, and has digitally revived Y. S. Rajasekhara Reddy, a beloved politician who passed away in a helicopter crash in 2009, to support his son Y. S. Jagan Mohan Reddy, the present Chief Minister of Andhra Pradesh. Additionally, Jadoun has developed AI-powered promotional music for various political figures, including Tamang, a parliamentary nominee, and the Chief Minister of Maharashtra. A campaign song for a politician from Ajmer, featuring both male and female vocals in a lively melody, proclaimed, “He is our pride, always fair.”

Jadoun has also created AI-generated songs for political campaigns, such as the one for Ajmer's local representative, Ram Chandra Choudhary. In English translation, the song praises Choudhary with words like: “Ajmer received a unique present / His name, Ram Chandra / Aids all in need / Once led the Ajmer Dairy / Known for his fairness / His name embodies Ram / He’s our source of pride / A stalwart for the Congress / Feels the people's woes / Ajmer's own son / A champion for progress / Ajmer's very own / The epitome of progress / Advocates for the rights of all / Ram Chandra, on the clarinet, plays.” The song is credited to Divyendra Singh Jadoun in the audio segment.

Rathore, although not on the ballot this year, is among the over 18 million volunteers for the BJP who are dedicated to ensuring Prime Minister Narendra Modi's government continues its reign. Previously, this effort involved extensive travel across Rajasthan, a state comparable in size to Italy, to engage with voters directly, highlighting the advantages they've received from BJP's social initiatives such as pensions, complimentary cooking gas cylinders, and financial support for expectant mothers. However, the introduction of Jadoun's deepfake technology has significantly simplified Rathore's tasks.

He's set to spend a quarter of an hour here, engaging directly with the camera, delving into major electoral topics as Jadoun feeds him queries. However, the content of his dialogue is secondary. Jadoun's primary aim is to capture Rathore's vocal essence. After accomplishing this, Jadoun plans to craft videos and phone calls that will be dispatched straight to the electorate's devices. Instead of a traditional doorstep visit or a brief interaction at a political event, individuals will witness or listen to Rathore personally addressing them, discussing relevant issues with astonishing precision, and soliciting their support for the BJP. Should they pose any questions, the AI is designed to reply with a clarity and calmness that arguably surpasses Rathore's own fast-paced speech. Voters not well-versed in technology might not even discern they've been conversing with an artificial entity. Rathore himself acknowledges his limited knowledge on AI, yet he recognizes the significance of psychological strategies. “Such initiatives could sway undecided voters,” he believes.

Mingling with political figures is familiar territory for Jadoun, having once been a politician himself. Back in 2015, he ran for the position of district president in Ajmer under the banner of the National Students Union of India (NSUI), which is the youth affiliate of the Indian National Congress. This party, previously a dominant force in Indian politics, currently serves as the main challenger to Modi's BJP.

By Joel Khalili

Authored by Reece Rogers

Authored by Celia Ford

Authored by Brian

Jadoun was naturally sai-allcreator.com">killed at marketing himself, which led his friends to encourage him to enter the race. For his campaign, he employed four armed guards with shotguns to accompany him through the city in convertible jeeps, aiming to craft an image of strength. "Having armed escorts makes others jealous," he remarked. "They would follow me wherever I went." Occasionally, they would discharge their weapons into the sky. However, he crossed a line when he walked into a voting station with armed men, igniting a widespread discussion about the issue of disorder in India's smaller communities.

Despite winning the election, Jadoun ascended to the role of state general secretary for the NSUI in Rajasthan two years post-victory. As a leader within the NSUI, he captivated audiences with his powerful oratory at public gatherings, shared moments with underprivileged youth in photographs, and traversed Rajasthan to forge connections with political figures, thereby expanding his influence. However, after a half-decade immersed in the political landscape, he began to experience a feeling of confinement. Jadoun had left college early, and it dawned on him that despite the support and promotion from those around him, he was also indebted to them. "There were times I would receive calls at 2 in the morning to deal with emergencies," he recalls. "It became clear to me that I no longer had control over my own time or my life."

The pandemic restrictions turned out to be a silver lining. Confined to his house and distanced from the political scene, he embarked on a venture: initiating "30-day challenges" during which he dedicated each month to acquiring a new skill – be it playing the flute, excelling at graphic design, diving into stock trading, or learning to use both hands equally. His goal was to explore if this approach could guide him in discerning the future direction of his life.

In October 2020, he was captivated by the Black Eyed Peas' music video for "Action," which featured the group's members superimposed onto scenes from Indian films. This inspired him to explore Reddit communities and experiment with technology that could swap faces. He gave himself a goal in October of the same year to attract 10,000 followers on Instagram within a month. To achieve this, he launched an account named The Indian Deepfaker and began transforming scenes from various movies and TV series using the open-source tool DeepFaceLab. He creatively inserted the cast of Breaking Bad into a scene from an Indian drama series and reimagined Bollywood superstar Shah Rukh Khan as Iron Man.

Content from Instagram

You can also see this content on its original website.

Authored by Joel Khalili

Authored by Reece Rogers

By Celia Ford

Crafted by Brian

His content quickly became a hit online, leading to numerous direct messages requesting his services for personalized videos. He charged for these unique creations, which ranged from inserting a customer's face into a major film to crafting video messages from deceased relatives for newlyweds. It was an eye-opener for him when he realized he could earn a living through these endeavors. "Someone offered me £800 [$1,000] for a project," he recalls. "I was absolutely blown away." His talents caught the attention of Netflix, which brought him on board to create a special video commemorating an employee, incorporating the employee's face into scenes from the TV shows they contributed to. Despite the confidentiality agreement, Jadoun shared details of this project on Instagram.

In 2022, Sagar Vishnoi, an astute and clever political advisor from the quaint town of Chandigarh, came across Jadoun's projects on the internet. Vishnoi was at the forefront of integrating artificial intelligence into political campaigning in India, gaining national attention in 2020 for producing the inaugural officially approved deepfake video in a political context—for Manoj Tiwari, a politician with the BJP in Delhi. This 44-second video featured a dubbed monologue, enabling clips of Tiwari to appear as though he was delivering speeches in English and Haryanvi—languages he does not speak—accusing the opposition of having "deceived" the electorate. This video was disseminated through 5,800 WhatsApp groups before the Delhi elections, with no disclosure that it was generated through AI technology.

Browsing Jadoun's Instagram profile, Vishnoi immediately recognized his talent and reached out to propose leveraging his abilities for political promotion. The pair spent considerable time on a call, exchanging tales from their respective professional journeys. Vishnoi shared his experiences as an anti-corruption crusader, including instances of being targeted with water cannons at demonstrations. In April 2023, they decided to collaborate, and have been actively working together since then.

Divyendra Singh Jadoun, captured in a photo on the rooftop of his residence and workplace in Ajmer, has emerged as the leading figure in India's politically sanctioned deepfake industry, valued at $60 million.

As Jadoun and Vishnoi became more involved in genuine political activities, Jadoun began receiving communications through WhatsApp from foreign numbers, and through anonymous profiles on Instagram and Telegram. These were from individuals seeai-allcreator.com">king to employ unauthorized deepfake technology against their political rivals. Fact-checking organizations have pinpointed numerous significant audio and video deepfakes aimed at swaying electoral perspectives in India. This includes fabricated clips of politicians making statements they never made, as well as instances where politicians have used the "deepfake" defense to refute claims of making certain statements they actually made.

Authored by Joel Khalili

Authored by Reece Rogers

By Celia Ford

Authored by Brian

Jadoun has received over 250 requests for similar employment opportunities. However, he is adamant about not employing deepfakes for disseminating false information. This stance stems largely from his involvement in a controversy that swept the nation in November 2023. A widely circulated deepfake depicted the face of Indian actress Rashmika Mandanna superimposed on a woman wearing a revealing black top, considered risqué by traditional Indian norms.

The incident became a major news topic throughout India. In response, Prime Minister Modi delivered an address cautioning about the dangers of deepfake technology. Authorities were prompted to launch an investigation to identify those responsible. Jadoun, who had previously collaborated with Mandanna, found himself under suspicion by many of his Instagram followers who believed he was behind it. "People suspected me because I possessed her facial data," he explained. At a certain juncture, Jadoun was approached by three undercover officers, which initially led him to fear they were investigating him for the deepfake involving Mandanna. However, their visit was actually to seek his expertise in determining if an audio clip of a politician in Hindi was a deepfake. Following this incident, Jadoun shifted his focus away from creating deepfakes for entertainment, opting instead to concentrate on political matters. Not long before a meeting with Rathore, Jadoun had co-signed a manifesto for an "Ethical AI Coalition," committing to avoid the malevolent use of AI technology.

Originally starting as a solo venture from his own bedroom, Jadoun's business has since grown to include a team of nine to handle the increasing workload. In March, he expanded his home by adding two more levels: the ground floor is dedicated to his team's workspace, while his family resides on the first floor. He has future plans to transform the first floor into a training center aimed at educating local youths on AI and deepfake technology, intending to relocate his family to the newly added second floor. Polymath, Jadoun's company, is also setting its sights on international expansion, particularly targeting countries with forthcoming elections. One of Jadoun's clients introduced his services to Pierre Poilievre's team, the leader of the Conservative Party in Canada, who showed interest in adapting Poilievre's speeches into Punjabi and Chinese. Furthermore, they are considering entering the US market in anticipation of the next election cycle, with a focus on promoting their custom video messages as a key service offering.

Polymath is strategically positioning itself across the political spectrum: Jadoun leverages his political influence and community ties to persuade potential candidates to adopt AI in their campaigns, while Vishnoi is developing fee-based training sessions and public education initiatives aimed at training police forces to identify and combat deepfake technology.

Supporters began arriving at the campaign headquarters, sharing their joy over a phone conversation they had with the candidate. Unbeknownst to them, they had been interacting with an artificial intelligence.

Jadoun's efforts represent merely a small portion of a larger phenomenon. In conversations with WIRED, representatives from six different firms throughout India revealed their involvement in deepfake initiatives worth millions of dollars. Among these, iToConnect stood out by executing 25 million AI-driven personalized calls in the fortnight before the general elections in Telangana and Andhra Pradesh, two states in the south. According to the political consultants interviewed, while city dwellers often view these unsolicited communications as bothersome, individuals in rural areas tend to feel valued when they receive calls from figures of authority. "The expectation among voters is for the candidate to make the first move, to engage with them directly. And in situations where it's not feasible for candidates to visit each voter personally, AI-powered calls serve as an effective alternative," explained Abhishek Pasupulety, a technology leader at iToConnect.

Pasupulety's belief in the effectiveness of artificial intelligence strategies stems from his firm's successful endeavors during the 2023 state elections cycle. At that time, his company, iToConnect, executed 20 million AI-generated calls in Telugu for 15 different political figures, notably featuring the voice of the incumbent chief minister. In the fortnight leading up to the election, constituents received customized half-minute calls. These ranged from appeals to support specific politicians, to tailored messages for festive occasions, and even personalized birthday wishes.

Authored by Joel Khalili

By Reece Rogers

Authored by Celia Ford

Authored by Brian

The strategy was successful. Party headquarters began to see an influx of voters arriving to share their happiness about getting a phone call from the candidate and being addressed personally by name. Unbeknownst to them, they had been interacting with an artificial intelligence. Meanwhile, Pasupulety's team was handling inquiries from puzzled party staff members who were clueless about the ongoing situation.

The advantages of mastering this technology are clear—tailored communication on a large scale. Employing actual people in call centers is approximately 4 rupees per interaction, which is eight times the cost of AI-generated calls, according to Pasupulety. While traditional, non-customized prerecorded robocalls are less expensive, they fail to captivate the audience as effectively; a provider notes that AI-driven calls tend to keep individuals engaged for a longer duration. Additionally, reaching out to voters at the last minute could significantly influence election results: The National Election Study reveals that 25% of voters in India make their voting decision in the days leading up to the election.

During the extensive general election period, which commenced in April and is slated to conclude on June 1, there has been a notable surge in the volume and regularity of AI-generated phone campaigns. This election spans seven phases and has seen a significant use of unidirectional AI-driven phone messages. In Tamil Nadu, a southern state of India, a firm known as IndiaSpeaks Research Lab reached out to electorates using voice messages from the deceased politician J. Jayalalithaa to support a particular candidate, executing around 250,000 tailored AI phone messages using the voice of a past chief minister. (While they had the consent of Jayalalithaa’s political party, they did not have approval from her relatives.) In a different state, there have been numerous calls made using a digital replica of Narendra Modi's voice, extracted from publicly available speeches, to back a regional candidate. In the northern regions, political adviser Sumit Savara has been aggressively marketing his AI solutions to both candidates and fellow political strategists. He points out that, “In the rural areas of India, social media isn’t widespread,” and believes that these phone messages will have an impact there.

Vijay Vasanth, a member of the Congress Party from the southern extremity of India in Kanyakumari, was propelled into the political arena in 2021 following the death of his father, H. Vasanthakumar, a former parliamentarian, due to Covid. To capitalize on the legacy of his father, Vasanth's campaign team utilized artificial intelligence to create a video lasting two minutes, which was circulated on WhatsApp and Instagram. In this video, the deceased politician is seen urging the electorate of Kanyakumari to cast their votes for his son, proclaiming him as the deserving successor to the trust and affection bestowed upon him. Furthermore, Vasanth's campaign innovatively used AI to simulate video calls bearing his father's voice and image, enabling direct engagement with voters in the far-flung areas of the coastal district. These virtual interactions were designed to give the impression of live conversations, bridging the gap in areas Vasanth could not physically visit.

Politician Vijay Vasanth from the Congress Party employed artificial intelligence to bring back the voice of his deceased father, who once served as a parliament member, to support him in a regional election. In the AI-produced video, a deepfake of H. Vasanthakumar is heard saying, “While dedicating my time to serving the community during a crisis, I was struck by COVID-19 and succumbed to it. Though I may not be with you in body, my spirit remains alongside you.” The video was shared on Instagram by behindwoods.

Authored by Joel Khalili

Authored by Reece Rogers

Unfortunately, you've provided a

Authored by Brian

The Congress Party stands out for its widespread distribution of AI-generated clones, with many of its official profiles disseminating unauthorized voice endorsements from Bollywood stars. This activity has triggered alerts from India's electoral monitor and led to the filing of at least three police reports. According to Sam Gregory, the executive director at the nonprofit organization Witness, audio deepfakes pose a significant threat. He explains that while video manipulations can be compared to their originals to identify alterations, pure audio forgeries often lack a point of reference, making it challenging to locate any authentic source for comparison.

However, officially approved deepfakes also complicate matters, even in instances where they are welcomed by recipients. Gregory comments, "Pleasingly misleading the public does not justify the deceit. The reality that you've been tricked doesn't alter simply because it was done with a grin."

Gregory highlights the primary concerns with deepfakes as being the absence of permission, transparency, and appropriate content. He points out the delicate balance between engaging voters, making candidates relatable, and the risk of misleading them, which greatly depends on how familiar people are with the technology. While personalized emails and letters are common, receiving voice messages and videos is a significant escalation. Although Jadoun might have approval from the politicians he represents, the electorate they reach out to might be completely unaware that they're interacting with a synthetic duplicate. As the electoral period in India concludes, the largest democracy in the world is venturing into a new era dominated by AI, with many unanswered questions about its implications.

"The boundary between engaging voters, making candidates relatable, and misleading people is quite fine."

Once Jadoun completed the scanning process of Rathore against the greenscreen backdrop, the politician made us take a seat and took out his mobile phone. He scrolled past numerous applications until he found the SARAL app, identified by its saffron emblem. This app demonstrates the effortless integration of AI into the BJP's political framework. (SARAL is an acronym for "sangathan reporting and analysis," with "sangathan" translating to "organization.")

Rathore highlighted that through SARAL, party members have the capability to document their actions and record the information of individuals they interact with. "The core leaders are informed instantly about the occurrences," mentioned Diggaj Mogra, the head of Jarvis Consulting, the firm behind the development of the SARAL application and the management of the BJP's expansive network of 20,000 call center agents.

Originally trained as an engineer, Mogra became well-known as part of the quartet that strategized Modi's victory in 2014. However, he chooses to maintain a low profile. In 2019, he initiated the development of SARAL, aimed at overseeing the extensive network of BJP operatives on the ground, thereby enabling the party's top brass to efficiently deploy its vast membership. Presently, according to Mogra, the application serves as the "digital handheld office" for the BJP, with over 4 million party workers engaging with it on a monthly basis.

On the SARAL platform, Rathore has access to details concerning his vicinity as well as a catalog of individuals who have gained from state initiatives. (The question of whether political factions ought to be privy to this information remains contentious, but Rathore disclosed that his source was "the party." According to Amogh Dhar Sharma, a political expert at the University of Oxford, there is a well-documented track record of administrations exploiting this kind of information to tailor messages for voters.)

Shakti Singh Rathore showcases the mobile interface of the SARAL application, facilitating connections between countless BJP field workers and beneficiaries of their welfare programs, aiming to transform these individuals into supporters.

Authored by Joel Khalili

Authored by Reece Rogers

Unfortunately, you haven't provided

Authored by Brian

The application additionally offers BJP members an opportunity to attract attention from the party's hierarchy through its dynamic leaderboard, which reflects participants' engagement levels. Rathore's ranking is determined by the number of households he reaches and the extent of information he gathers. "My aim is to be at the top in Rajasthan," Rathore stated. Back in April, he was positioned 12th within his area. To achieve the goals outlined by the SARAL application, Rathore is leveraging artificial intelligence-powered calls.

Mogra envisions that in the future, AI-enabled calls will enhance the efficiency of the BJP's activities. Currently, the party's extensive call centers are constantly busy, coordinating with volunteers and monitoring the progress of door-to-door initiatives, ensuring the distribution of posters and other resources, and stepping in to address any issues in the campaigning process. These routine follow-ups often include repetitive questions about logistics, such as inquiries about the receipt of promotional materials or booth kits. "Our goal is to substitute these repetitive inquiries with AI-powered voice calls," Mogra states.

Mogra aims to increase the daily bot calls to 100,000 in anticipation of the state elections happening later this year, drawn by the potential for reduced costs. However, he has found the AI call center options presented to him lackluster, noting a significant disparity between what is advertised and the actual performance: “They offer you a red Ferrari, but what you end up with is a red Fiat,” he observed.

During several of the conversations, the artificial intelligence experienced distortions in its responses, and in a particular instance, it communicated at such a high volume that it resembled the sensation of being yelled at.

Before departing from Jadoun's workspace in April, he showcased the result of his efforts that day: AI-generated phone conversations, in which the caller engages in dialogue with a persuasive replica. The caller's spoken words are converted into written text, then input into the extensive language model named Mistral to compose a response resembling human speech, which is subsequently transformed into a spoken reply mimicking the voice of the politician the caller believes they are conversing with.

The experiment wasn't successful. We observed numerous issues, including delays in responses from the machine, pronunciations that were overly influenced by English, and instances of the AI producing irrelevant or nonsensical outputs. At one point, the AI became stuck, repeatedly uttering the same sentence in response to any inquiry. In a different scenario, it bizarrely introduced itself as “Rathore’s AI avatar.” When Jadoun attempted to utilize an AI version of Rathore in interactions with actual voters, the outcomes were equally problematic. “There were instances where the AI was generating nonsensical content, and in one instance, it spoke with such volume that it was akin to yelling,” he remarked. For the time being, even those among Jadoun's colleagues who are engaged in multi-million-dollar ventures consider the use of AI in interactive calls with notable political figures to be too fraught with issues to be viable.

Jadoun is optimistic about his team of student engineers' ability to solve the problem. He describes them as "jugaadus," a term that captures the Indian knack for improvising solutions with scarce resources. Weekly, Jadoun's team analyzes call data to enhance their performance. In one corner, an 18-year-old with glasses perched on a bean bag works meticulously on adjusting a Mistral LLM, ensuring its responses are courteous.

Several hours after completing the scan of Rathore, Jadoun received another visitor in his office. This time, it was a politician seeking to be cloned—Manoj Bhagat, an emerging contender from the Congress Party. Bhagat's request was for Jadoun to archive his vocal and visual representations to leverage them in reaching out to constituents during religious festivals and notable events. He believed this innovative approach could propel him into the limelight of mainstream political discourse. “I figured this approach was unique and could draw people’s attention to me,” Bhagat remarked. “In an era where individuals are bombarded with calls, a personalized touch ensuring the call begins with their name could make them feel directly addressed, encouraging them to listen to the entire message.”

Share your thoughts on this piece by sending a letter to the editor at mail@wired.com.

Suggested for You …

Delivered to your email: Dive into Will Knight's Fast Forward for the latest developments in artificial intelligence.

He transferred the contents of a cryptocurrency exchange to a flash drive and then vanished.

The emergence of live deepfake love cons is upon us

Excitement over boomergasms is

Heading outside? Discover the top sleeping bags for all your adventures

Brendan I. Koerner

Steve Levy

Author: Kathy Gilsinan

Additional Content from WIRED

Insights and How-tos

© 2024 Condé Nast. All rights reserved. WIRED might receive a share of revenue from the sale of goods linked on our website, as a result of our Affiliate Agreements with retail partners. Content from this website cannot be copied, shared, broadcast, stored, or utilized in any form without explicit consent from Condé Nast. Advertisement Options

Choose a global website