India’s Election Frontier: Deepfake Technologies Transform Political Campaigning and Voter Engagement

To look at this article again, go to My Profile and then click on View saved stories.

Varsha Bansal Nilesh Christopher

Millions of Deepfakes Flood Indian Electorate, with Political Endorsement

During a sweltering April day in Ajmer, located in Rajasthan, India, Shakti Singh Rathore, a regional political figure, positioned himself before a greenscreen to record a brief video. Appearing anxious, it marked his initial experience of being duplicated.

Dressed in a sharp white shirt and adorned with a ritualistic orange scarf featuring the lotus emblem of the BJP, India's governing party, Rathore joined his hands in a traditional gesture and addressed his listeners in Hindi. “Namashkar,” he initiated. “To all my brothers—”

As he was about to proceed, the photo shoot's overseer entered the shot. At 31 years old, Divyendra Singh Jadoun, recognizable by his baldness and dense black beard, informed Rathore that his excessive motion was problematic for the camera. Jadoun's aim was to gather sufficient audio and visual material to create a convincing AI-generated deepfake of Rathore, one that would persuade nearly 300,000 potential voters in Ajmer they had engaged in a personal interaction with him. However, too much movement could disrupt the deepfake technology. He instructed Rathore to maintain direct eye contact with the camera and to only move his mouth. “Let’s try that again,” he instructed.

At the company Polymath Synthetic Media Solutions, autodidact in deepfake technology, Divyendra Singh Jadoun, compiles visual and auditory content of regional political figures with the aim of converting their messages into various languages to extend their electoral engagement. In this instance, the communication delivered by Shakti Singh Rathore is reproduced in Hindi, Tamil, Sanskrit, and Marathi. Video contributors include Nilesh Christopher alongside Divyendra Singh Jadoun, as reported by WIRED.

Currently, the globe's most populous democratic nation is conducting elections. Nearly a billion Indian citizens have the right to vote in the nationwide electoral process, where deepfakes might significantly influence the outcome, potentially causing division. Political factions within India have harnessed artificial intelligence to manipulate the truth, creating inexpensive audio manipulations, doctored images, and AI-generated satire. However, unlike the international conversation surrounding deepfakes, which typically centers on the spread of false information and societal damage, numerous politicians in India are employing this technology for an alternative aim: to connect with voters.

Across various political beliefs, political parties are utilizing artificial intelligence to manage communications in the country's 22 recognized languages and numerous local dialects, ensuring tailored messages reach remote areas. Although the US has banned the use of AI-generated voices for making unsolicited calls, in India, officially approved deepfake technology has emerged as a lucrative $60 million industry. In the two months before the elections began in April, over 50 million calls using AI-created voice clones were placed, and one of the largest business messaging firms in the country anticipates millions more will be made throughout the election period, as reported by WIRED.

Jadoun has become the face of the rapidly expanding sector. His company, Polymath Synthetic Media Solutions, stands amongst a growing number of firms in India offering deepfake technologies tailored for political figures. During this election period, Jadoun's enterprise has successfully executed five artificial intelligence-based campaigns, accumulating earnings of $55,000 in total. His pricing undercuts major political consulting fees, charging 125,000 rupees (around $1,500) for creating a digital twin and 60,000 rupees (approximately $720) for producing a voice duplicate. Among his notable works, he has generated deepfake content for Prem Singh Tamang, the Chief Minister of Sikkim, and brought back to life the late Y. S. Rajasekhara Reddy, a revered political figure who perished in a helicopter crash in 2009, to support his son Y. S. Jagan Mohan Reddy, the present Chief Minister of Andhra Pradesh. Moreover, Jadoun has produced AI-crafted campaign songs for various politicians, including Tamang, a parliamentary candidate, and the Chief Minister of Maharashtra. One such song, dedicated to a politician from Ajmer, praised him in Hindi as a source of pride and celebrated his fairness, all set to an upbeat melody with both male and female vocal harmonies.

Jadoun has also ventured into creating AI-assisted electoral anthems, crafting one such song for the local representative, Ram Chandra Choudhary, in Ajmer. When translated to English, the song's verses convey: “Ajmer has been gifted anew / By a man named Ram Chandra / A helper to all in need / Formerly led Ajmer Dairy / Known for his fairness / His name bears 'Ram' / A figure of our esteem / A loyalist of the Congress / Empathizes with the people's woes / Ajmer's very own son / A champion for progress / Ajmer's offspring / The epitome of progress / Advocates for the rights of all / Ram Chandra's tunes on the clarinet resonate.” The song is credited to Divyendra Singh Jadoun in the audio segment.

Rathore, who is not on the ballot this year, is among the over 18 million volunteers for the BJP, charged with the mission to ensure Prime Minister Narendra Modi's administration retains its grip on power. Previously, this mission would have involved Rathore traveling extensively across Rajasthan, a state with an area comparable to Italy, to engage with voters one-on-one, highlighting the advantages they've received from various BJP-led social initiatives, such as pension schemes, complimentary cooking gas cylinders, and monetary support for expectant mothers. However, the introduction of Jadoun's sophisticated deepfake technology has significantly simplified Rathore's responsibilities.

He will allocate 15 minutes to engage directly with the camera, discussing major political topics as Jadoun guides the conversation with inquiries. However, the content of his speech is secondary. Jadoun's primary objective is to capture Rathore's vocal essence. Following this, Jadoun plans to create and distribute videos and voice messages straight to the electorate's mobile devices. Instead of traditional campaign methods like door-to-door visits or brief interactions at public gatherings, voters will experience personalized messages from Rathore. He will address them individually, delving into the issues they care about with surprising detail, and encourage them to support the BJP. Should they pose any questions, the AI is designed to reply with a clarity and serenity that arguably surpasses Rathore's own hurried speech patterns. Voters less familiar with technology might not even discern that their conversation partner is not human. Rathore himself acknowledges his limited understanding of AI technology but appreciates the psychological impact. He believes these tailored communications could be influential with undecided voters.

Encountering politicians is familiar territory for Jadoun. He was once among their ranks. In 2015, he ran for office in Ajmer, aiming to become the district president for the National Students Union of India (NSUI), which is the youth affiliate of the Indian National Congress. This party, previously a dominant force in national politics, currently stands as the primary rival to Modi's BJP.

Authored by Joel Khalili

Authored by Reece Rogers

Authored by Celia Ford

Authored by Brian

Jadoun was naturally gifted at marketing himself, which is why his friends encouraged him to enter the race. For his campaign, he decided to employ four armed guards to accompany him throughout the town in convertible jeeps, aiming to craft an image of strength. "Having armed guards by your side makes you the object of jealousy," he commented. "They would follow me wherever I went." Occasionally, they would discharge their weapons into the sky. However, Jadoun crossed the line when he entered a voting station with armed men, igniting widespread discussions on the issue of disorder in India's smaller towns.

Despite securing a victory in the elections, Jadoun ascended to the role of state general secretary for the NSUI in Rajasthan two years post-victory. In his capacity as a leader within the NSUI, he delivered impassioned addresses to extensive assemblies at gatherings, was photographed with underprivileged youth, and journeyed throughout Rajasthan to engage with politicians and fortify his connections. However, after a tenure of five years, the world of politics began to weigh heavily on him. Having abandoned his college education, Jadoun came to the realization that despite being uplifted by those around him, he was simultaneously indebted to them. He recounts, “I would receive calls at 2 am to manage crises,” highlighting a loss of control over his own time and life.

The pandemic-induced quarantines turned out to be a silver lining. Confined to his home and distanced from the political scene, he embarked on a journey of personal growth through "30-day challenges." Each month, he dedicated himself to acquiring a new sai-allcreator.com">kill—whether it was learning to play the flute, delving into graphic design, exploring stock trading, or training to become ambidextrous—with the hope of gaining clarity on what direction to take in his life next.

In October 2020, he came across the music video for “Action” by the Black Eyed Peas, which featured the band members' faces superimposed on scenes from Indian films. This inspired him to explore Reddit communities and experiment with face-swapping technology. That same month, he embarked on a personal goal to attract 10,000 followers on Instagram within 30 days. To achieve this, he launched an account named The Indian Deepfaker and began altering scenes from various movies and television series using the open-source tool DeepFaceLab. His projects included integrating characters from Breaking Bad into an Indian drama series and transforming Bollywood star Shah Rukh Khan into Iron Man.

Instagram material

You can also see this material on its original website.

Authored by Joel Khalili

Authored by Reece Rogers

By [Your Name]

[

Authored by Brian

His content quickly became an internet sensation, leading to a flood of direct messages requesting his services for a price. He specialized in creating personalized videos, whether it was superimposing someone's face into a major film or crafting heartfelt video messages from late parents to their children on their wedding day. He began to realize that this could be a viable source of income. "One project was priced at £800, which is around $1,000. It blew my mind," he recalls. His skills caught the attention of Netflix, which contracted him for a special project to create a tribute video for one of their employees, by integrating the employee's face into scenes from the show they contributed to. Despite the confidentiality agreement, Jadoun shared his involvement in the project on Instagram.

In 2022, a keen and astute political strategist named Sagar Vishnoi from the quaint town of Chandigarh came across some of Jadoun’s online contributions. Vishnoi was at the forefront of integrating artificial intelligence into Indian political campaigns, gaining nationwide attention in 2020 by producing the first officially approved deepfake video for a political campaign. This video was for Manoj Tiwari, a politician from the Bharatiya Janata Party (BJP) in Delhi. The 44-second video cleverly used lip-sync technology to show Tiwari delivering a speech in English and Haryanvi—two languages he does not speak fluently—accusing the opposition of deceiving the public. This video was distributed through 5,800 WhatsApp groups in the run-up to the Delhi elections, with no indication that AI technology was used in its creation.

Browsing Jadoun's Instagram profile, Vishnoi quickly recognized his talent and reached out with an idea to leverage his abilities for a political campaign. They spent hours conversing over the phone, sharing tales from their respective professional journeys. Vishnoi shared his experiences as an activist fighting corruption, including being targeted with water cannons at demonstrations. In April 2023, they decided to collaborate, and have been actively working together since.

Captured in a photo on the rooftop of his residence and workplace in Ajmer, Divyendra Singh Jadoun has emerged as the emblem of India's politically sanctioned deepfake sector, valued at $60 million.

As Jadoun and Vishnoi immersed themselves further in genuine political activities, Jadoun began receiving communications from foreign contacts on WhatsApp, along with messages from anonymous profiles on Instagram and Telegram. These were from individuals seeking to deploy unauthorized deepfake technology against their political rivals. Fact-checking organizations have pinpointed multiple significant audio and video deepfakes aimed at swaying electoral sentiments in India. There have been instances of fabricated replicas of politicians uttering statements they never made, alongside politicians invoking the term "deepfake" as a defense to repudiate statements they actually made.

Authored by Joel Khalili

By Reece Rogers

Authored by Celia Ford

Authored by Brian

Jadoun has received over 250 inquiries regarding this type of work. However, he is adamant about not employing deepfakes for disseminating false information. This stance is largely due to his involvement in a significant controversy in November 2023. During this incident, a deepfake video that had Rashmika Mandanna's face superimposed on another woman wearing a revealing black top, considered risqué by traditional Indian norms, became widely circulated.

The incident grabbed national attention in India. Prime Minister Modi addressed the nation, emphasizing the dangers of deepfake technology misuse. Authorities launched an investigation to apprehend those responsible. Jadoun, who had previously collaborated with Mandanna, became a prime suspect among his social media followers due to his access to her digital likeness. "Everyone suspected me because I possessed her digital imprints," he remarked. At one juncture, Jadoun was approached by three undercover officers, sparking fear they were there to interrogate him about the deepfake incident involving Mandanna. However, their visit was to enlist his expertise in verifying the authenticity of a Hindi audio clip purportedly from a political figure, to determine if it was a deepfake. Following this event, Jadoun shifted his focus away from creating deepfakes for entertainment, opting instead to engage more seriously in political advocacy. Just before collaborating with Rathore, Jadoun played a pivotal role in launching the "Ethical AI Coalition," co-signing a manifesto that commits to the responsible use of AI technology, avoiding its application for malicious purposes.

Initially starting as a solo venture from his bedroom, Jadoun's business has now grown to include a team of nine to manage the increasing workload. In March, he expanded his residence by two levels: the bottom floor serves as the workspace for his team, while the first floor remains a living space for his family. He has future plans to transform the first floor into an educational center where local youths can gain knowledge about artificial intelligence and the creation of deepfakes, intending to relocate his family to the newly added second floor. Furthermore, Polymath is exploring opportunities in international markets, particularly in countries with upcoming elections, aiming for global expansion. One of Jadoun’s clients introduced his services to the team of Pierre Poilievre, leader of the Conservative Party in Canada, who showed an interest in having Poilievre’s speeches translated into Punjabi and Chinese. They are also setting their sights on the US market, especially for the forthcoming elections, with a focus on offering customized video messages.

Polymath is engaging in a dual strategy: Jadoun leverages his political influence and community ties to aggressively promote AI-driven campaign strategies to potential candidates, while Vishnoi develops paid training sessions and public awareness initiatives, instructing police forces on methods to combat deepfake technology.

Supporters began arriving at the party headquarters, sharing their pleasure about getting a phone call from the candidate. What they were unaware of was that they had been interacting with an artificial intelligence.

Jadoun's initiatives represent just a small portion of a much larger phenomenon. In conversations with WIRED, representatives from six different firms throughout India revealed their involvement in deepfake projects worth millions of dollars. Among these, iToConnect stood out for making 25 million AI-generated personalized phone calls in the fortnight preceding the elections in Telangana and Andhra Pradesh, two southern states. According to the political consultants we interviewed, while city dwellers often find such unsolicited communications bothersome, people in rural areas tend to feel valued and significant when they receive calls from high-ranking individuals. "Voters generally prefer it when the candidate makes the effort to reach out to them directly. And in situations where it's not feasible for the candidate to visit each voter personally, AI-generated calls offer an effective alternative," explained Abhishek Pasupulety, a technology manager at iToConnect.

Pasupulety's belief in the effectiveness of AI strategies stems from his firm's prior involvement in the 2023 state elections. During that period, iToConnect executed 20 million AI-generated calls in Telugu for 15 political figures, featuring the voice of the state's chief minister at the time. In the fortnight leading up to the election, voters received customized half-minute calls. These ranged from appeals to support specific politicians, to festive greetings during religious celebrations, and even birthday wishes.

Authored by Joel Khalili

Authored by Reece Rogers

Authored by Celia Ford

Authored by Brian

The strategy was successful. Party headquarters began to see an influx of voters who were pleased to have received a personal call from the candidate, especially being addressed by their names. Unbeknownst to them, they were actually interacting with an artificial intelligence. Pasupulety's crew managed inquiries from bewildered party staff who were completely in the dark about the situation.

The advantages of mastering this technology are clear—tailored communication on a large scale. Employing real people in call centers is approximately eight times costlier, at 4 rupees per call, than utilizing AI for the same purpose, according to Pasupulety. While generic, prerecorded robocalls may be more affordable, they lack the ability to engage listeners as effectively; a provider noted that AI-generated calls tend to retain listeners' attention for a longer duration. Additionally, calls made at the eleventh hour could significantly influence election results: The National Election Study reveals that 25% of voters in India make their final decision on whom to vote for just a few days before voting begins.

During the current election period, which spans seven phases from April to June 1, the use of AI in making campaign calls has seen a significant uptick in both volume and frequency. Many of these calls are broadcasted without expecting a response. In Tamil Nadu, a southern state in India, IndiaSpeaks Research Lab reached out to voters using the voice of the late politician J. Jayalalithaa to support a specific candidate, making over 250,000 tailored AI calls mimicking a past chief minister. While they obtained approval from Jayalalithaa's political party, her family did not consent to this. In a different region, a vast number of calls have featured a digital imitation of Narendra Modi's voice, taken from his public speeches, to promote a regional candidate. Meanwhile, in the northern parts of the country, political strategist Sumit Savara has been aggressively marketing his AI services to both candidates and political advisors. He argues that for those living in less urban areas of India, who may not have access to social media, these AI-driven calls could be influential.

Vijay Vasanth, a member of the Congress Party from the southernmost point of India, Kanyakumari, was propelled into the political arena in 2021 following the death of his father, H. Vasanthakumar, a former parliamentarian, due to Covid. To capitalize on the positive regard his father held, Vasanth's campaign leveraged artificial intelligence to create a video that was spread via WhatsApp and Instagram. In this video, lasting two minutes, the late politician appeared to endorse his son, urging the electorate of Kanyakumari to support him as his successor, promising continuity of the trust and affection they had for him. Furthermore, Vasanth's team utilized AI to simulate video calls in his father's voice and image, reaching out to voters in isolated areas of the coastal town. These virtual interactions were designed to give the impression of live conversations, thereby engaging with constituents in places Vasanth himself could not visit physically.

Politician Vijay Vasanth from the Congress Party employed artificial intelligence technology to create a video where his deceased father, also a former parliamentarian, appears to support his campaign for a local election seat. In the digitally crafted video, H. Vasanthakumar is seen saying, “During my time of service to the populace, I became infected with the coronavirus, which led to my demise. Though I am not among you physically, my spirit remains with you.” The video was shared on Instagram by behindwoods.

By [Your Name]

Authored by Reece Rogers

Authored by Celia Ford

Authored by Brian

The Congress Party has become the leading distributor of AI-generated clones, with several of its official platforms circulating unauthorized voice clone endorsements from Bollywood stars. This activity has attracted cautionary notices from India's election supervisory body and prompted at least three police reports. Sam Gregory, the executive director of the nonprofit organization Witness, highlights the particular danger of audio deepfakes. He explains that while it is feasible to locate and compare the original footage for video manipulations, fake audio presents a tougher challenge due to the absence of a reference point or an original recording to verify against.

However, officially approved deepfakes also complicate matters, even if recipients are pleased with them. Gregory points out, "Pleasingly tricking voters doesn't justify the trickery. The fact that you were duped with a smile doesn't alter the reality that you were misled."

Gregory highlights the primary concerns surrounding deepfakes as issues related to consent, transparency, and the nature of the content itself. He points out that there is a delicate balance to maintain between engaging voters, making candidates appear relatable, and avoiding deceit, which largely depends on the public's understanding of the technology in question. While personalized emails and letters are something the public has become accustomed to, receiving voice messages and videos represents a significant leap forward. Although Jadoun might have received approval from the politicians he represents, the electorate they are reaching out to might be completely unaware that their interactions have been with a digital duplicate. As the election period in India, the largest democracy globally, comes to an end, the nation is stepping into a future filled with uncertainties, propelled by advancements in artificial intelligence.

"There exists a delicate balance between engaging voters, making candidates relatable, and misleading people."

Once Jadoun had completed the process of digitally capturing Rathore against the backdrop of a greenscreen, the politician invited us to take a seat and proceeded to take out his smartphone. He navigated through numerous applications until he found the one marked by a saffron-colored emblem, known as SARAL. This app represents a preview into the future integration of Artificial Intelligence within the operations of the BJP party. (SARAL is an acronym for "sangathan reporting and analysis," with "sangathan" translating to "organization.")

Rathore outlined how with the help of SARAL, party volunteers are able to document their actions and disseminate information regarding the individuals they interact with. "This allows the top officials to receive instant reports on occurrences," stated Diggaj Mogra, the head of Jarvis Consulting, the firm behind the development of the SARAL application and the management of the BJP's extensive network comprising 20,000 call center agents.

Trained as an engineer, Mogra became well-known as a key figure among the quartet who orchestrated Modi's victory in 2014. However, he chooses to remain out of the spotlight. His project, SARAL, initiated in 2019, was designed to streamline the vast network of BJP activists, enabling the party's top brass to efficiently deploy its vast membership. Currently, the app serves as what Mogra describes as the BJP's "digital handheld office," with over 4 million party members utilizing it on a monthly basis.

Through SARAL, Rathore is able to view details of his vicinity along with the names of numerous individuals who have received advantages from government initiatives. (The appropriateness of political factions obtaining this information remains uncertain, yet Rathore mentioned it was acquired "through the party." Amogh Dhar Sharma, an academic specializing in politics at the University of Oxford, points out that there is a well-documented precedent of authorities exploiting this kind of information to tailor messages for voters.)

Shakti Singh Rathore showcases the mobile interface of the SARAL application, facilitating communication between millions of BJP's field operatives and individuals who have received benefits from their welfare programs, aiming to transform these beneficiaries into supporters.

Authored by Joel Khalili

Unfortunately, you haven't provided

Authored by Celia Ford

Authored by Brian

The application provides an opportunity for BJP members to gain recognition from the party's higher-ups through a dynamic leaderboard that reflects their engagement level. Rathore's performance is evaluated based on the number of households he engages with and the volume of information he gathers. "My goal is to be at the top in Rajasthan," Rathore expressed. In April, he ranked 12th in his area. To achieve the goals outlined by the SARAL application, Rathore employs artificial intelligence-powered calls.

Mogra envisions that artificial intelligence-driven calls will enhance the efficiency of the BJP's processes in the future. Currently, the party's extensive call centers are completely engaged in coordinating with volunteers and evaluating the progress of door-to-door campaigns, ensuring the distribution of posters and other campaign materials, and addressing issues when campaign activities do not go as planned. This process includes a multitude of repetitive logistical inquiries, such as confirmation of receipt of promotional items or booth kits. "Our goal is to substitute these repetitive queries with calls powered by artificial intelligence," Mogra states.

He aims to increase the daily bot calls to 100,000 for the state elections happening later this year due to the appealing cost reductions. However, he remains unimpressed with the AI call center services presented to him, noting a significant discrepancy between their promises and actual performance. Mogra remarked, "They assure you of delivering a red Ferrari, but what you end up with is a red Fiat."

During a few of the conversations, the artificial intelligence experienced delusions, and in a particular call, its volume was so high it seemed as though someone was shouting.

Before departing from Jadoun's workspace in April, he showcased the outcome of his day's work: artificial intelligence-enabled calls allowing for an interactive dialogue with a realistic replica. The conversation from the caller is converted into written form, which is then processed by the advanced language model named Mistral to generate a response resembling human interaction. This reply is subsequently transformed into a spoken message, mimicking the voice of the supposed politician the caller believes they are conversing with.

The performance was far from satisfactory. We observed several issues including lag in the system's response time, mispronunciations with an English accent, and instances of the AI producing nonsensical output. At one point, the AI became stuck, endlessly repeating the same sentence regardless of the input it received. In a different instance, it oddly introduced itself as “Rathore’s AI avatar” in English. When Jadoun attempted to utilize a digital version of Rathore to engage with actual voters, the outcome was just as problematic. Jadoun reported, “There were times when the AI would produce gibberish, and on one occasion, it spoke at such a high volume it was akin to yelling.” Currently, even Jadoun’s colleagues who are involved in projects worth millions of dollars consider the use of interactive AI in phone calls too erratic for application with notable political figures.

Jadoun is optimistic about his determined group of student engineers solving the problem. He describes them as "jugaadus," a term that denotes the Indian knack for achieving goals despite resource constraints. Jadoun and his team analyze call data weekly to enhance performance. In a corner, an 18-year-old with glasses adjusted a Mistral LLM on a bean bag, guiding it to respond with courtesy.

Several hours after Jadoun completed the scan of Rathore, another individual made their way into his office. This time, it was a politician eager for cloning. Manoj Bhagat, an aspiring member of the Congress Party, sought Jadoun's services to record his voice and likeness. His aim was to utilize this technology to reach out to voters during religious festivals and significant events, believing it could propel him into the political limelight. "I believed this approach was innovative and could draw people's attention to me," Bhagat explained. "In an era where people are bombarded with calls, they might disconnect if the call seems mundane. However, if the call addresses them by name right at the beginning, it signals to them that it's of personal relevance, encouraging them to listen all the way through."

Share your thoughts on this piece with us. Send your feedback via email to the editor at mail@wired.com.

Suggested For You …

Direct to your email: Dive into the future of artificial intelligence with Will Knight's Fast Forward series.

He transferred all the contents of a cryptocurrency trading platform onto a USB stick—then vanished.

Live deepfake romance cons are now a reality.

Excitement around Boomergasms is

Heading outside? Check out the top sleeping bags for all types of adventures

Brendan I. Koerner

Steven Levy

Kathy Gilsinan

Additional Content from WIRED

Insights and Tutorials

© 2024 Condé Nast. All rights protected. WIRED might receive a share of revenue from items bought via our website, as a component of our Affiliate Agreements with retail partners. Content from this website is not permitted to be copied, shared, broadcasted, stored, or utilized in any form without explicit written consent from Condé Nast. Advertising Options

Choose a global website