Microsoft Revamps Copilot AI with Voice, Vision, and an Encouraging Persona: A New Era of Digital Assistance

To go back to this article, go to My Profile and then click on View saved stories.

Microsoft Revamps Copilot AI with Voice, Visual Capabilities, and an Enthusiastic Character

Seventeen years after Microsoft phased out Clippy, the overly helpful office assistant, the concept of a genial and positive AI companion has been resurrected from digital oblivion. Microsoft is upgrading its Copilot, a textual AI assistant integrated within Windows and other applications, by introducing visual and auditory functionalities and enhancing its problem-solving capacity. This upgrade also infuses the assistant with a more supportive demeanor.

"Mustafa Suleyman, the CEO of Microsoft AI, expresses his excitement, stating, 'We truly find ourselves at an extraordinary juncture.' He explains that AI entities have reached a level where they can visually perceive what we do, listen to the sounds we hear, and converse in the same language we use to interact with each other."

So far, opinions on Copilot are divided, with certain users expressing dissatisfaction due to its slow or unclear replies. However, Microsoft is optimistic that this tool could become essential across Windows, Office, and other platforms. Leveraging OpenAI's technology in software utilized by a vast number of individuals places Microsoft in a leading position to explore how AI can enhance efficiency in office tasks. Meanwhile, Google, a significant competitor, is also integrating AI into its office applications, such as Gmail and Google Docs.

The updated Copilot feature will have the capability to interact with users using a variety of voices that mimic human speech, smoothly managing any interruptions or breaks in the conversation. "It allows for interruptions during its responses, and it possesses the ability to pay attention actively," explains Suleyman. "That truly captures the essence of engaging dialogue."

Suleyman mentions that enhancements have been made to Copilot, now providing increased emotional encouragement for its users. "It's like having a supportive teammate, someone who's always in your corner, cheering you on," he explains. Starting today, Copilot Voice will be accessible in English for users located in Australia, Canada, New Zealand, the United Kingdom, and the United States, with plans to expand to additional countries, according to the company.

Microsoft's assistant, known as Clippy, portrayed as a talking paper clip, famously popped up in Word, often greeting users with the notorious phrase, “It looks like you’re writing a letter …” Its lack of popularity stemmed from Microsoft's realization that it did not live up to the expectations of human-like smarts it was supposed to offer, notably by not remembering user preferences and being repetitively annoying. Advanced language models today show a much-improved ability to imitate human thought processes, yet they sometimes act in strange and unforeseeable ways, which might influence how well Copilot is received by users.

Copilot Voice is going to be accessible within the complimentary edition of Copilot for Windows, and it can also be found as a separate mobile application as well as on the internet.

Microsoft is rolling out additional experimental features for Copilot, exclusively available to subscribers of the $20 monthly Copilot Pro plan. A new opt-in functionality, dubbed Copilot Vision, enables the AI assistant to view users' screens and respond to items they highlight with their mouse. According to Suleyman, this allows a user to select a product and request Copilot to provide feedback based on online reviews.

"Suleyman notes that a frequent occurrence is individuals seeking advice on style," he explains. "While browsing a fashion site, they often inquire, 'What's the name of this design? What do you call this type of dress?'"

Suleyman suggests that Copilot could, in the future, evaluate a webpage and offer an opinion tailored to a user's likes and dislikes. He explains, "It might instantly analyze the whole page and then discuss its contents with you. For example, you could inquire, 'Is this an article you believe I'd like?' This represents a novel approach."

Microsoft has stated that conversations with Copilot are retained for a period of 18 months, but users have the option to remove these interactions. According to Microsoft, Copilot Vision won't store queries made by users, erasing this information once a session concludes. Restrictions will apply to the feature's availability across specific websites, and it will actively prevent access to copyrighted or not safe for work (NSFW) material. The deployment of this feature to Copilot Pro users in the United States will occur at a yet-to-be-revealed future date, Microsoft has announced. The company also assures that no information is shared with OpenAI.

A new trial function named Think Deeper enables Copilot to tackle more intricate issues by employing a method that resembles sequential logic. This feature partially incorporates a fresh AI architecture known as OpenAI o1, which OpenAI unveiled recently. Starting today, a select group of Copilot Pro subscribers in the US will have access to Think Deeper.

The modifications made to Copilot highlight Microsoft's ambition to innovate within its AI offerings and enhance their appeal. This also demonstrates the swift advancement in AI technology, especially with the latest advanced large language models – the technology behind chatbots – now proficient in managing audio, images, and text. In the past few months, OpenAI, Google, and other companies have upgraded their models to engage in more fluid conversations using varied human tones.

In addition to facing a lot of rivals, Microsoft is also dealing with some uncertainty that's not immediately apparent.

The firm has reportedly poured $13 billion into OpenAI and holds a license that allows it to utilize its AI technologies. Despite OpenAI's reputation as a frontrunner in artificial intelligence, it has faced significant upheaval, highlighted by the recent exits of Chief Technology Officer Mira Murati and two key engineers spearheading research initiatives. Suleyman refrained from discussing the circumstances at OpenAI.

In March, Suleyman became part of Microsoft following the tech behemoth's agreement to a $650 million licensing contract with his new venture, Inflection AI. Before this venture, he was a co-founder of the UK-based AI firm DeepMind, eventually bought by Google in 2014. Recently, DeepMind combined forces with Google's AI projects, creating Google DeepMind, currently under the leadership of Demis Hassabis, who also co-founded DeepMind.

Inspired by the achievements of a coding tool launched in 2021 named GitHub Copilot, which offers code completion and responds to programming queries, Microsoft created Copilot.

Shane Greenstein, a Harvard Business School professor with expertise in Microsoft's AI approach, believes that creating an effective all-around assistant presents a significant challenge. He also notes that the company's innovative features are yet to demonstrate their worth to consumers.

"Greenstein notes that it required a period of five to ten years of experimentation with web interfaces before figuring out a method to attract more than just the technically proficient to make online purchases. He anticipates a similar timeframe for progress in this area as well."

Suggested for You…

Directly to your email: A selection of our top stories, curated daily just for you.

Google's Seven-Year Quest to Equip Artificial Intelligence with a Physical Form

The Exclusive Conversation: Mark Cuban Aims to Challenge Pharma Intermediaries

The largest bitcoin mining operation is causing a stir in this oil-rich Texas community.

Event: Come along to WIRED Health, happening on March 18, in London.

Additional Coverage from WIRED

Critiques and Manuals

© 2024 Condé Nast. All rights reserved. Purchases made through our site from products featured may result in revenue for WIRED as part of our Affiliate Partnerships with retail partners. Content from this site cannot be copied, shared, broadcast, stored, or utilized in any form without explicit consent from Condé Nast. Ad Choices

Choose a global website