Ring’s AI Search Revolutionizes Home Surveillance, But Not Without Its Quirks

To go back to this article, go to My Profile, and then check your saved stories.

Ring Introduces AI-Powered Video Search Feature, Yielding Varying Outcomes

Liz Hamren, head of Amazon's Ring camera division, frequently ponders if her spouse managed to pick up the routinely delivered frozen package at their residence. Instead of pestering him, she opts to review their Ring camera's recordings.

In the past, she had to painstakingly scroll through the video timeline on the Ring app to determine if the package was collected, and if so, by whom. However, in the recent period, Hamren has utilized artificial intelligence to streamline the process. By entering "package today" into the app, she can immediately view the footage of her husband fulfilling his task, provided that he actually did.

Today, Ring introduced its latest feature, Smart Video Search, which will become available to all Home Pro subscribers by November 5. Home Pro users, who pay either $20 a month or $200 a year, can look forward to this new addition. In testing the feature for several days, WIRED found that it wasn't useful for solving common household queries like, "Did you remember to lock the door?" However, when searching for terms such as "dog pooping," "gardeners," and "rollerblading," the results were generally precise.

Hamren, who is in charge of three additional Amazon home tech divisions, states that Ring prevents searches for material deemed "offensive, inappropriate, or harmful." She opted not to provide further details but mentioned that weapon names and "descriptive words regarding individuals or scenarios" are included in the restricted content.

When we looked up keywords that could be deemed controversial, such as "Black," "turban," and "gun," we didn't find any matches, even though there were videos that fit those categories. Searching for "disabled" showed individuals using wheelchairs and also mothers with baby strollers. Ring has decided to prevent searches for "disabled" going forward to avoid potential abuse.

The introduction of search functionality represents a significant improvement for Ring, a company that has become well-known over the last ten years for its video doorbells and expanding range of security devices. The problem up until now has been that the vast number of Ring customers have accumulated up to six months of video recordings with very limited tools for efficiently accessing the data contained in those recordings. Hamren, who previously held positions at Microsoft and Meta before joining Ring in March 2023, aimed to provide users with meaningful insights without requiring them to engage in extensive manual searching. This was made feasible through the use of artificial intelligence.

Promotional video snippet for Smart Video Search by Ring

Promotional Video for Ring's Intelligent Search Feature

Under Hamren's leadership, Ring has undergone a notable transformation in how it markets itself to prospective buyers. The company has moved away from aggressive crime-fighting commercials featuring dubious figures. Hamren scaled down collaborations with law enforcement, which previously allowed Ring to create features enabling users to share footage directly with police without a warrant.

Currently, Ring's advertising strategy highlights charming videos that include animals and kids, suggesting that customers share these lighthearted moments with their digital circles, in addition to more serious content. The brand has shifted its emphasis from preventing crime to alleviating worry through providing users with detailed views of the activities captured by their cameras.

Exploration is merely the beginning. According to Hamren, the advanced artificial intelligence technologies behind search functionalities will soon allow Ring to alert its users to particular scenarios, such as the presence of a bear in their backyard or unusual water accumulation on their patio. Users will also be able to get brief overviews of recent events. "Our ultimate goal is to offer insights and understanding regarding ongoing activities," she states.

Residential areas aren't her only focus. By the middle of 2025, Ring intends to introduce a subscription service targeting small to medium-sized enterprises. According to Hamren, this service will allow subscribers to more effectively monitor various sites and provide employees with access to specific videos.

Trial Phases

The development of Smart Video Search has surpassed a year, according to Hamren. The AI technology employed contrastive learning for its training, enabling it to identify various elements such as parcels, individuals, automobiles, fauna, meteorological conditions, and specific movements like sprinting. "These are the aspects that truly matter to users," she mentions. Ring states that the training process utilizes videos from public domains as well as those contributed by its staff and their acquaintances. Rivals such as Nest and Wyze are also exploring or experimenting with comparable search functionalities.

Ring is introducing search functionality exclusively within its priciest subscription tier due to the substantial cloud computing resources needed to support it. According to Hamren, this decision comes as the company, which achieved profitability last year, aims to maintain its financial health. She further explains that prioritizing features customers are willing to pay for ensures a "very straightforward relationship."

Subscribers of Ring's Home Pro package have the option to participate in the search feature and select the cameras they wish to involve, compatible with all models. However, the search function is unavailable for those who have activated end-to-end encryption, offering an advanced level of video privacy. Hamren mentions that Ring aims to overcome this restriction but did not provide a specific timeframe. Although searches are not recorded in the users' profiles, anonymized search queries might be used to enhance the functionality of the feature, Ring states.

Ring's search functionality operates similarly to how one might search for a video on YouTube. Users have the option to sort their search outcomes by either relevance or chronology. While the system is adept at identifying different categories of vehicles, such as "police cars" or "minivans," it is not designed to differentiate between car brands like Kia and Mazda, nor can it decipher specific license plate numbers. "License plate recognition isn't our current focus," Hamren clarifies. Furthermore, the company has no plans to integrate facial recognition technology. Despite its growing use by both governmental and private entities, facial recognition technology raises significant concerns related to widespread surveillance and the risk of incorrect identification. However, Hamren mentions that Ring might revisit this decision if there is a strong demand from users for such a feature.

In the evaluations conducted by WIRED, Ring's search function performed satisfactorily when tasked with finding items such as "backpack," "scarf," "speeding cars," "ice cream truck," and "shopping cart."

However, numerous search outcomes frequently contained reasonable errors in identification. The system mistook the sudden appearance of headlights from cars making turns for "fireworks." Searches for "blonde women" yielded not only individuals matching the criteria but also golden retrievers. According to the AI, the presence of a hand close to someone's face was often interpreted as "smoking." Hamren anticipates that input will guide future improvements.

A query for "blonde woman" yielded multiple hits showing golden retrievers, like the one mentioned.

Some peculiarities were quite unusual. The term "Purple hair" not only fetched an individual wearing a purple sweater but also a bus adorned in purple. The system failed to recognize a decorative pumpkin, yet linked it to a landscaper wearing an orange vest who was gathering leaves into a white bag. While searching for "Rollerblades" yielded a correct result of a person skating, the phrase "roller blades" (spelled with a space) resulted in no findings, as the system correlated "blade" with weapons.

Attempts to get the camera to identify individuals swinging a wooden baseball bat or holding a hammer were unsuccessful. (Mistakes for "hammer" involved identifying a leaf blower and a hedge trimmer.) Searching for "water gun" yielded results showing two people using water guns, however, searches for "gun" came up empty.

Hamren expressed her admiration for the fact that during her experiments, searching for "volleyball" successfully identified an individual wearing a non-specific volleyball outfit. However, WIRED encountered difficulties with Smart Video Search's inability to detect a football team's name on a jersey, attributing the issue to the tool's lack of direct text recognition capabilities.

The most remarkable aspect of the search tool was its blend of unsettling and accidental discovery capabilities, particularly in being able to monitor events during periods typically not scrutinized. Our exploration led us to unexpectedly detailed observations on the habits of a specific dog's bathroom routines. Similarly, tracking the frequency of visits from a family of raccoons, mistakenly identified in a search for "bat," proved to be quite entertaining. Whether these insights serve to soothe or enhance one's sense of connection remains ambiguous. Nonetheless, the search function undoubtedly paves the way for an increase in Ring videos gaining popularity online, and its utility is expected to grow as it evolves.

Hamren admits that her curiosity hasn't been fully satisfied by her search efforts. She still doesn't know how a tile in her backyard became stained, as her camera unfortunately didn't catch the event in question. It's a puzzle that, as of now, AI hasn't been able to crack.

Check Out Similar Content…

Explore the World of Politics: Subscribe to our newsletter and tune into our podcast

A remedy for America's firearm crisis from an emergency room physician

View: Antony Blinken propels American diplomacy into the modern era

Revelations from an Avid Hinge User

Come be a part of the Energy Technology Conference happening on October 10th in Berlin.

Further Insights from WIRED

Critiques and Tutorials

© 2024 Condé Nast. All rights reserved. Purchases made through our website could result in WIRED receiving a share of the sales, this is due to our affiliations with retail partners. Reproduction, distribution, transmission, storage, or any form of usage of this site's content is strictly prohibited without the explicit written consent of Condé Nast. Choices regarding advertisements are available.

Choose a global website