Google Unveils Gemini: The Next Evolution of Android AI That Blends the Future with Nostalgic Features

To go back to this article, go to My Profile and then to View saved stories.

Name: Julian Chokkattu

Google's Gemini for Android Hints at Mobile Computing's Evolution and History

Almost ten years prior, Google unveiled a functionality known as Now on Tap within Android Marshmallow—by pressing and holding the home button, Google would provide useful information pertinent to the content displayed on your screen. Discussing a film via text message with a buddy? Now on Tap was capable of fetching information about the movie without the need to exit your chat application. Browsing a dining spot on Yelp? Your device could present suggestions from OpenTable with a simple press.

Just out of university, the enhancements seemed thrilling and enchanting to me—the way it could grasp the screen's content and foresee your potential moves appeared ahead of its time. It ranked high among my preferred Android functionalities. Gradually, it evolved into Google Assistant, which was impressive by itself, yet it didn't quite capture the original essence.

Today, during its I/O developer conference held in Mountain View, California, Google showcased the latest updates to its Android operating system, reminiscent of the older Now on Tap feature. These updates aim to simplify smartphone usage by leveraging contextual data from your surroundings. However, this iteration benefits from ten years of progress in large language model technology.

"Dave Burke, the vice president of engineering for Android, shared his enthusiasm with me during a Google Meet video call, highlighting the advancements in technology that now enable the creation of truly engaging assistants. He emphasized the importance of developing computer systems with the capability to comprehend their surroundings, a feat he believes was not possible with the technology available in the past. Now, according to him, we have the necessary technology."

I had the opportunity to converse with Burke and Sameer Samat, who oversees the Android ecosystem at Google, regarding the latest developments in Android, the introduction of Google's new AI assistant named Gemini, and the implications these advancements may have for the operating system's future. Samat described these advancements as a "unique opportunity that comes once in a generation to redefine the capabilities of the phone and to overhaul the entire Android experience."

Exploring Your Assignments with Circle to Search

Experience the enhanced Circle to Search feature in use.

The beginning is marked by Circle to Search, Google's latest innovation in mobile search technology. Similar to the functionality offered by Now on Tap, Circle to Search—a feature introduced by the tech giant a few months back—offers a more dynamic interaction than the traditional method of entering queries into a search box. Users actively draw a circle around the subject of their interest on the screen to initiate a search. According to Burke, "This method offers an intuitive, enjoyable, and contemporary approach to searching… It also appeals more to the younger demographic due to its playful nature."

Samat reports that Google has garnered favorable reactions from users, particularly pointing out that the newest addition to Circle to Search was inspired by suggestions from students. Now, when users circle mathematics and physics questions, Circle to Search will provide detailed guides on how to solve these problems directly, without requiring users to exit their syllabus application.

Samat emphasized that Gemini's role extended beyond merely offering solutions; it was guiding students through the problem-solving process. As the year progresses, Circle to Search will enhance its capability to tackle intricate challenges, including those involving diagrams and graphs. This advancement is supported by Google's LearnLM models, specifically optimized for educational purposes.

Gemini Enhances Android Experience with Deeper Integration

Google's artificial intelligence helper, Gemini, is outshining Google Assistant in several aspects. Indeed, on a majority of Android smartphones now, when you activate Google Assistant, you're presented with the choice to switch to Gemini. This prompted me to question Burke and Samat if this development signals that Assistant is on its way to being discontinued by Google.

Samat explains, "Consider Gemini as a feature you can choose to use on your phone. It's clear that Gemini is progressing and developing over time. While we don't have any new announcements at the moment, users have the option to engage with this AI-driven assistant if they're interested. We've noticed an uptake in users trying it, and the feedback has been overwhelmingly positive."

Reece Rogers

Oliver Ben

Adrienne So

Julian Chokkattu

In the upcoming update, users will have the capability to insert images created by AI directly into emails and messages.

During the I/O conference, enhancements to Gemini for Android were announced, aiming to enhance its contextual understanding, mirroring the capabilities introduced by Now on Tap almost ten years ago. In the coming months, users will gain the ability to create images using Gemini and seamlessly integrate them into applications such as Gmail or Google Messages. Burke demonstrated how Gemini could be used to craft an unconventional image of tennis using pickles in response to a text message about playing pickleball. He activated Gemini as a floating overlay on the messaging application, commanded it to create the image, and then easily dragged and dropped the generated picture into the ongoing conversation.

You can request Gemini to extract particular pieces of data from a video.

Next, he accessed a YouTube tutorial detailing the regulations of pickleball. During the viewing, a feature to "Ask this video" appears if you engage Gemini, allowing you to utilize Gemini to pinpoint particular details in the footage instead of manually searching the entire content. (Because really, who has the leisure for such an exhaustive task?) When Burke inquired about a precise rule of pickleball, Gemini promptly provided an answer derived from the video content. This capability to "summarize" is a distinguishing feature of numerous AI applications, including the ability to condense information from PDFs, videos, briefings, and news articles (fantastic).

Reece Rogers

Oliver Ben

By Adrienne So

Author: Julian Chokkattu

Summarizing video content in text form could be beneficial.

Discussing PDF functionalities, Gemini will soon introduce the capability to attach a PDF document directly to it (you'll see an “Ask this PDF” option), allowing Gemini to provide precise information and eliminate the hassle of manually browsing through multiple pages. According to Burke, these updates are set to be deployed across millions of devices in the forthcoming months. However, the PDF attachment feature will be exclusive to Gemini Advanced subscribers, who pay a monthly fee of $20 to utilize the most advanced features of Google's AI technologies.

Gemini, as a whole, will display more "active recommendations" depending on the on-screen activity. These suggestions will appear directly over the Gemini overlay as soon as the assistant is engaged.

Gemini Nano Receives Enhancements

Google's Gemini Nano, the advanced language model behind specific in-built functionalities on some smartphones, including the Pixel 8 lineup, the Samsung Galaxy S24 series, and the latest Pixel 8A, has been upgraded. The operation of these features directly on the devices ensures that user data remains private by eliminating the need to transmit it to the cloud. Additionally, this allows the features to function without an internet connection.

Nano is the driving force behind functionalities such as the Summarize option in Google's Recorder application, which condenses transcription content, and Smart Reply in certain messaging applications, providing auto-replies that are more relevant to the conversation context. Google plans to release an updated version of this model, named Gemini Nano with Multimodality, beginning with Pixel smartphones this year. Although the name might seem complicated, it essentially indicates that Gemini Nano will have capabilities extending beyond mere text processing.

"Burke mentions that this model, with its 3.8 billion parameters and multimodal capabilities, is a pioneering on-device, integrated multimodal model. He emphasizes its strength, noting that it achieves roughly 80% of the performance of Gemini 1.0 in scholarly tests, an impressive feat for a model of its size."

Google is enhancing its screen reader's capabilities to improve its interpretation and depiction of images.

The Gemini Nano model is set to enhance Google's current TalkBack screen reader functionality on Android devices. This feature is designed to assist users who are blind or have low vision by providing a verbal explanation of the content displayed on the screen. According to Google, Gemini Nano is expected to deliver more detailed and accurate image descriptions. On average, TalkBack users encounter about 90 images daily that lack labels, but with Gemini's capability to interpret and narrate the contents of these images, even without an internet connection, this challenge can be significantly mitigated.

Over recent years, Google has channeled its artificial intelligence expertise into enhancing its call screening capabilities to reduce unwanted robocalls. Soon, its advanced technology known as Gemini Nano with Multimodality will introduce a feature aimed at helping users identify and avoid telephone scams as they happen. This upcoming functionality, dubbed Scam Detection, will monitor live phone conversations. Should it detect suspicious phrases or requests from the caller, it will promptly warn the user that they might be dealing with a scam. According to Burke, this technology was developed using data from sources such as BanksNeverAskThat.com to understand what legitimate banks would not inquire about, as well as common scammer tactics. He emphasized that all monitoring and analysis occur directly on the user's device, ensuring privacy. Details about this voluntary feature are expected to be released later in the year.

Rogers Reece

Oliver Ben

Adrienne So

Authored by Julian Chokkattu

In a departure from the norm, Google has announced that instead of pacai-allcreator.com">king all its updates into today's announcements, it will reveal several new Android features tomorrow. So, keep an eye out for further details.

As AI-powered devices aim to take over the role of smartphones, and the buzz around generative interfaces without the need for apps grows, I inquired about Samat's perspective on the future evolution of Android over the coming five years. He expressed enthusiasm about the creativity emerging from both new and established firms exploring novel concepts, mentioning that Google is also experimenting with numerous ideas internally. He simplified his outlook by comparing the situation to developments in the car industry.

When purchasing a vehicle, there are basic elements you anticipate, such as a steering wheel. However, with advancements in AI, there's a radical possibility of eliminating these standard components—imagine cars without steering wheels or interfaces. "This concept would thrill some, while others might find it unsettling," he observes. He suggests that AI will enhance many tasks we currently perform on our smartphones, making some functions obsolete and replaced by smarter, AI-driven alternatives.

"As this progresses, we're discovering—and our experiments already show this—that there are chances to completely change the user interface in specific aspects. This shift moves from merely being helpful to recognizing that there needs to be a completely new approach. This is what's thrilling and exhilarating about the current moment. It's a remarkable period to be involved in developing this technology."

Recommended for You…

Delivered to your email: Will Knight's Fast Forward delves into the progress of artificial intelligence.

He transferred the contents of a cryptocurrency exchange to a USB drive and then vanished.

Live deepfake romance cons are now a reality

Explosive excitement over Boomergasms

Venturing outside? Discover the top sleeping bags for all types of expeditions

—

Text:

Ashworth Boone

Name: Julian Chokkattu

Ashworth Boone

There's no text to rewrite

Authored by David

N/A

Julian Chokkattu

Author: Julian Chokkattu

WIRED OFFERS

Additional 20% discount on selected Dyson products with Owner Rewards

GoPro Discount Code: Save 15% on Cameras & Accessories

Unlock Up To An Additional 45% Discount – May Exclusive Sale

Receive a 5% Discount on All Items Using the Dell Promo

Register Now and Enjoy a 25% Discount Using This VistaPrint Promo Code

Half-Price Discounts on Chosen Items – Newegg Discount Coupon

Additional Content from WIRED

Evaluations and Manuals

© 2024 Condé Nast. All rights reserved. Purchases made through our site from products linked via our retail affiliate partnerships may generate revenue for WIRED. Unauthorized reproduction, distribution, transmission, storage, or use of the content on this website is prohibited without explicit prior written consent from Condé Nast. Ad Choices

Choose a global website