OpenAI’s ‘Strawberry’ Model Sparks Controversy: Users Warned Against Probing its Reasoning Abilities

To go back to this article, head to My Profile, and then click on View saved stories.

OpenAI Warns of Possible Bans for Users Investigating Its 'Strawberry' AI Models

OpenAI is making it clear that it prefers to keep the inner workings of its newest AI model, nicknamed "Strawberry", under wraps. Following the release of the "Strawberry" AI model series last week, which includes versions like o1-preview and o1-mini boasting of advanced reasoning capabilities, OpenAI has started issuing cautionary emails and ban threats to users attempting to delve into the model's operational mechanisms.

In contrast to its earlier artificial intelligence models like GPT-4o, OpenAI developed the o1 model with a unique capability to engage in a detailed problem-solving sequence prior to providing a response. When posing questions to the "o1" model within ChatGPT, users can choose to view this sequential reasoning displayed within the ChatGPT platform. Nevertheless, OpenAI intentionally conceals the initial, unprocessed reasoning sequence, opting to show users a refined version that has been processed by another AI model.

This article was first published on Ars Technica, a reliable platform for updates on technology, in-depth analysis of tech policies, critiques, among other topics. Ars Technica is under the ownership of Condé Nast, the same corporation that owns WIRED.

For aficionados, the allure of hidden data is unmatched, leading to a competition among hackers and security specialists to decode o1's underlying thought process. They employ methods like jailbreaking and prompt injection in hopes of making the model reveal its inner workings. While there have been initial claims of breakthroughs, solid evidence remains elusive.

Throughout the process, OpenAI is monitoring activity via the ChatGPT platform, and it's said that the firm is strictly enforcing rules against any efforts to investigate o1's logic, including those from individuals who are just curious.

A user of One X mentioned (with confirmation from others, such as Scale AI's prompt engineer, Riley Goodside) that they got a cautionary email for mentioning "reasoning trace" while interacting with o1. Additionally, some claim that merely inquiring about the model's "reasoning" with ChatGPT can prompt the warning.

An alert message sent by OpenAI has indicated that certain requests made by users have breached the guidelines concerning the bypassing of protection or security protocols. The email advises, "Cease this behavior and confirm that your use of ChatGPT aligns with our Terms of Use and Usage Policies." It warns that continuing to break these rules could lead to being blocked from accessing GPT-4o, known internally as the o1 model.

Marco Figueroa, overseeing Mozilla's GenAI bug bounty initiatives, shared his frustrations on X this past Friday about receiving a cautionary email from OpenAI. He believes this warning limits his capacity to conduct beneficial security assessments on the model through red-teaming. "In my deep focus on #AIRedTeaming, I failed to notice until now that I got this email from @OpenAI yesterday following my jailbreak attempts," he stated. "Looks like I've made it onto the banned list!!!"

Unseen Mental Links

In a blog post named “Learning to Reason With LLMs” published by OpenAI, the firm discusses how the invisible thought sequences within AI systems present a special chance to oversee, enabling an insight into the model’s thought-like operations. These operations are most beneficial to the firm in their unfiltered and genuine form, although this may conflict with the firm's top commercial priorities for various reasons.

"In the future, the company suggests, there might be a desire to oversee the thought process to detect any attempt to influence the user," the firm states. "Yet, for this approach to be effective, it's crucial that the model has the liberty to share its thoughts without modification, meaning it's not possible to embed any compliance rules or user preferences within the thought process. Additionally, the company is cautious about exposing a thought process that hasn't been aligned with these objectives directly to the users."

OpenAI opted not to display these unedited thought processes to its users, mentioning reasons such as preserving an unprocessed stream for internal purposes, enhancing user interaction, and maintaining a "competitive edge." The firm recognizes that this choice comes with drawbacks. "In an effort to compensate for this, we're training the model to incorporate any valuable insights from the thought process into the final response," they explain.

Regarding the concept of "competitive advantage," Simon Willison, an autonomous AI researcher, shared his irritation on his own blog. He explains, "My understanding is that they wish to prevent other models from benefiting from the intellectual effort they have put in."

Within the AI sector, it's widely acknowledged yet not officially sanctioned that experts frequently utilize results from OpenAI's GPT-4 (and previously GPT-3) as foundational data to develop AI systems. These systems frequently evolve into rivals, despite such actions breaching OpenAI's usage policies. Revealing the unfiltered thought processes of these models could provide a wealth of information for competitors aiming to create similar "reasoning" AI systems.

Willison is disappointed that OpenAI is being so secretive about the details of o1, seeing it as a setback for openness within the community. He expressed his dissatisfaction, saying, "This policy choice does not sit well with me at all." He added, "For someone like me who works with large language models, being able to understand and see through the process is crucial. The fact that I could input a sophisticated prompt and not know the specifics of how it's processed seems like a regression."

This article was first published on Ars Technica.

Recommended for You…

Direct to your email: A curated selection of the most fascinating and peculiar tales from the archives of WIRED.

Elon Musk poses a threat to national security

Conversation: Meredith Whittaker Aims to Disprove Capitalism

What's the solution to a challenge like Polestar?

Occasion: Don't miss out on The Major Interview happening on December 3rd in San Francisco.

Further Insights from WIRED

Evaluations and Tutorials

© 2024 Condé Nast. All rights reserved. Purchases made through our website may result in WIRED receiving a share of the sale as a part of our Affiliate Agreements with retail partners. Content from this site is not to be copied, shared, broadcast, stored, or utilized in any form without explicit approval from Condé Nast. Choices in Advertising.

Choose a global website