Fei-Fei Li’s World Labs: Pioneering Spatial Intelligence to Build the Future of AI

To look back at this article, go to My Profile and then click on View saved stories.

The AI Pioneer Encourages Universal Participation in Creating Worlds

Purchasing through links in our articles may generate a commission for us, aiding our journalistic efforts. To understand more and to support our work, consider subscribing to WIRED.

Market-focused technology commentators and skeptics have declared the end of the artificial intelligence boom, signaling a return to a gloomier period. However, Fei-Fei Li is not convinced by this narrative. Known affectionately as the "godmother of AI," Li is taking a different path. She has temporarily stepped away from her role at Stanford University to help start a new venture named World Labs. This company aims to push beyond the current limitations of generative AI, which primarily revolves around language, envisioning a future where systems can create entire universes that mimic the complexity, physics, and intricate details of our real world. This vision is not without its challenges, and despite the pessimistic views from some corners that AI development has reached a standstill, World Labs is quickly gaining financial support. The company, which is about a year away from unveiling its first product and still faces uncertainty regarding its effectiveness, has attracted $230 million in investment, with its valuation already reaching the billion-dollar mark.

About ten years back, Li made a significant contribution to artificial intelligence by developing ImageNet, a specialized collection of online images that greatly enhanced the capabilities of neural networks. She believes that for AI to be capable of generating both lifelike simulations and entirely fictional worlds, current deep-learning technologies require a substantial advancement. Future storytellers, akin to George R.R. Martin, may design their fantasy realms using simple commands, creating spaces that people can explore and experience. Li explains, "Computers perceive the real world through their 'eyes'—the cameras—and the 'brain' that processes what they see. To turn this visual information into understanding, creation, and eventually, interaction, it's essential to grasp the physical layout and dynamics of the world. This field is known as spatial intelligence." World Labs identifies as a company focused on spatial intelligence, and its success or failure will play a crucial role in determining whether spatial intelligence will lead to a groundbreaking shift or become the subject of jokes.

For years, Li has had a deep fascination with spatial intelligence. As the world was enamored with ChatGPT, she found herself eagerly discussing the future of AI with a past student, Justin Johnson, over the phone. Johnson, who has since become an assistant professor at the University of Michigan, shared, “The coming ten years will focus on creating fresh content that integrates computer vision, deep learning, and AI into our physical realm, anchoring them in both space and time.”

In early 2023, Li made the decision to launch a new venture following a discussion with Martin Casado, a trailblazer in the field of virtual networking and currently a partner at the venture capital firm Andreessen Horowitz, known for its fervent support of AI technology. Casado draws a parallel between the evolution of AI and the progression of computer games, which evolved from simple text-based formats to sophisticated 3D environments. He believes that spatial intelligence will be key to this transformation. He imagines a future where one could immerse themselves in their favorite book by converting it into a dynamic, real-time 3D experience. To begin realizing this vision, Casado and Li concurred that the focus should shift from developing extensive language models to creating comprehensive world models.

Li embarked on forming a group, bringing Johnson on board as a cofounder. On Casado's recommendation, they added two individuals to the team. One of them was Christoph Lassner, a former employee of Amazon, Meta’s Reality Labs, and Epic Games, and the creator of Pulsar. This rendering technique, known for its development into the acclaimed 3D Gaussian Splatting method, might sound like the name of a niche music group at a college-themed party, but in reality, it's a method for creating comprehensive scenes rather than standalone items. The second of Casado's recommendations was Ben Mildenhall, who developed an innovative method known as NeRF—neural radiance fields—which transforms 2D images into 3D visuals. “We were able to bring objects from the real world into VR and render them with stunning realism,” he commented. He resigned from his role as a senior research scientist at Google to be a part of Li’s initiative.

A clear objective for a comprehensive global model would involve equipping robots with a sense of the world around them. This is indeed on the agenda for World Labs, though it's not their immediate focus. Initially, the priority is to develop a model that possesses an intricate comprehension of three-dimensionality, physical properties, and the concepts of space and time. Subsequently, the plan is to progress to a stage where the models can enhance augmented reality applications. Following this, the organization will venture into robotics. Should this strategy be realized, expansive global models could significantly enhance self-driving vehicles, automated manufacturing processes, and potentially, humanoid robots.

The timeline is quite extended, and success isn't guaranteed. World Labs has pledged to deliver a product by 2025. Upon questioning the founders about the specifics of the product, its target audience, and their revenue model, they highlighted that their operations are just beginning to scale. "We're facing numerous challenges and uncertainties," Li remarked. "But undoubtedly, we have the world's top team to navigate through these uncertainties."

Casado delves into greater detail. Similar to ChatGPT or Anthropic's Claude, he points out, a model itself can serve as a product—an infrastructure utilized directly by some or as a base for various applications. Potential clients could range from video game developers to film production houses. It brings to mind discussions about Pixar's former extensive investment in areas such as rendering fur on creatures or simulating water dynamics. Now, consider achieving that with a simple sentence instruction.

World Labs is not alone in its venture into what is often referred to as physical AI. Nvidia's CEO, Jensen Huang, highlighted earlier this year the importance of creating universal base models for humanoid robots as a thrilling challenge in the field of AI. In a recent piece, I discussed another enterprise, Archetype, which is also exploring this avenue. However, Casado believes that World Labs stands out due to its unparalleled ambition, talent, and visionary approach. "In my nearly a decade of investment experience, I've never encountered a team as exceptional as this one," he remarks. While it's typical for investors to praise the companies they invest in, Casado's commitment goes beyond financial investment. For the first time in his venture capitalist career, he has taken on a role within the company, dedicating one day each week to work alongside the team.

Several other venture capital enterprises are contributing as well, featuring investments from Radical Ventures, NEA, and Nvidia's investment branch, alongside a notable group of individual investors such as Marc Benioff, Reid Hoffman, Jeff Dean, Eric Schmidt, Ron Conway, and Geoff Hinton. (This includes support from a pioneering figure in AI for another leading figure in the domain.) Additionally, the recently deceased Susan Wojcicki had made investments prior to her unfortunate death last month.

Is it possible that all the intelligent individuals are mistaken? Absolutely. It's not difficult to notice the similarities between World Lab's ambitions and a term that quickly lost its hype: the metaverse. The creators behind World Lab suggest that the fleeting hype was too soon, a minor hiccup spurred by intriguing technology lacking engaging material. They hint that expansive world models might address this issue. It's assumed that in such worlds, AI wouldn't be depicted as hitting a standstill.

Chronological Journeys

In the previous year, Fei-Fei Li released a unique blend of autobiography and artificial intelligence romance, titled The Worlds I See. Upon its release, I commended the work and engaged in a conversation with her featured in a Plaintext article named, "Fei-Fei Li Initiated an AI Upheaval through Algorithmic Vision." Currently, her aspiration is to create previously unimaginable realms.

Li tends to keep her personal life private and finds it challenging to speak about herself. Nevertheless, she has adeptly woven her story as an immigrant who arrived in the United States at 16 without any proficiency in English, and her journey of overcoming various hurdles to become an influential figure in a crucial technological field. Before reaching her current role, she held significant positions including the director of the Stanford AI Lab and the chief scientist for AI and machine learning at Google Cloud. Li describes her book as being organized in a manner similar to a double helix, intertwining her own narrative with the development of AI into a cohesive spiral. Li believes that we understand ourselves through our reflections, stating, "Part of the reflection is technology itself. The most challenging world to understand is our own."

The narrative of how ImageNet was brought to life and put into practice is particularly compelling. Li shares her resolve to overcome skepticism from her peers and others who believed it was unfeasible to classify and label millions of images, ensuring there were over a thousand examples for each item in a wide range of categories, from decorative cushions to musical instruments like violins. This endeavor demanded not just technical resilience but also the hard work of thousands of individuals (a notable assist came from Amazon’s Mechanical Turk). To fully grasp the project's scope, one must appreciate Li's own story. Her boldness to embark on such an uncertain venture was bolstered by her parents' encouragement. They urged her to forsake a high-paying corporate job to follow her passion for science, despite their own financial hardships. Successfully completing this monumental task would serve as the ultimate acknowledgment of their support and sacrifice.

Inquire About Anything

Tom inquires, “In the beginning, there was much discussion about the proper way to use smartphones in public. These days, it's typical to witness everyone in a public area engrossed in their phones. How do you foresee the manners concerning AR headsets evolving?”

Hello, Tom, I appreciate your inquiry. Navigating the etiquette for augmented reality (AR) presents more complexity compared to the clear-cut norms we have with smartphones, where it's obvious when we're engrossed in our devices. The pinnacle of AR technology will be achieved when firms succeed in integrating it into subtle, stylish eyewear—similar to Meta's popular Ray-Ban glasses, which currently lack AR capabilities but are expected to incorporate them eventually. Much of the content we currently view on our phones will transition to being displayed on head-up displays.

At that moment, it may not be immediately apparent, but hidden behind our sunglasses, we'll be more engrossed in TikTok videos, text messages, and playing Candy Crush than engaging with the people we're dining with. Although it might not seem like it, public spaces will give off the vibe that everyone is mentally elsewhere. I foresee that haptic feedback technology will become crucial to nudge individuals when it's time to catch their train, when they're obstructing an entryway, or even when they've been pickpocketed. A common scenario at dinner might unfold as follows: "Did you catch what I just mentioned?" [No response.] "DO YOU EVEN HEAR WHAT I'M SAYING? [Brief pause, then a tap on the side of the glasses.] "Yes, of course, I'm listening." This scene will be a regular occurrence at every table in the eatery!

My forecast for social manners? Individuals will resort to texting one another despite being in close proximity, as messages delivered directly to your visual and auditory devices will hold more allure. Thus, let's halt our grievances regarding the constant phone gazing; there are more challenging times ahead.

Send your inquiries to mail@wired.com. Please include "ASK LEVY" in the email subject.

Apocalypse Watch

Is it possible for temperatures to rise even more? Hold on to see.

Wrapping Things Up

Here's a summary of all the announcements from Apple's September gathering.

The spotlight may have been on the iPhone 16, but Apple's introduction of AirPods designed to function as hearing aids could be seen as their most impactful innovation.

Inhabitants of a Texan petroleum city become less friendly after the arrival of a bitcoin mining operation.

Mark Cuban insists that he is not experiencing a midlife crisis.

Ensure you don't miss out on exclusive editions of this column just for subscribers. Sign up for WIRED now and enjoy a 50% discount for Plaintext readers.

Check Out…

Inside Politics: Subscribe to the newsletter and tune into the podcast

Exploring the consequences of distributing no-strings-attached cash

Not all individuals experience weight loss with Ozempic

The Pentagon is planning to allocate $141 billion towards the development of an apocalyptic device.

Gathering Invitation: Be part of the Energy Tech Summit happening on October 10th in Berlin.

Additional Content from WIRED

Evaluations and Tutorials

© 2024 Condé Nast. All rights reserved. Purchases made through our website might result in WIRED receiving a share of the sale, as a part of our collaboration with retail partners. The content on this website is protected and cannot be copied, shared, broadcast, stored, or used in any form without explicit consent from Condé Nast. Advertisement Choices

Choose a global website