What is 'AI scheming'? Unpacking a concerning OpenAI study.

Is your favorite AI chatbot scheming against you?

If "AI scheming" sounds ominous, you should know that OpenAI is actively studying this phenomenon. This week, OpenAI published a study conducted alongside Apollo Research on "Detecting and reducing scheming in AI models." The researchers “found behaviors consistent with scheming in controlled tests,” the result of AI models with multiple, and at times competing, objectives.

So, what is AI scheming, and does it mean that ChatGPT is lying to you?

In a blog post about the study, the creators of ChatGPT define AI scheming as a chatbot "pretending to be aligned while secretly pursuing some other agenda." OpenAI wants to know why AI is deliberately lying to users and what to do about it.

OpenAI introduces the study with an interesting "human analogy" to better understand what AI scheming is:

Imagine a stock trader whose goal is to maximize earnings. In a highly regulated field such as stock trading, it’s often possible to earn more by breaking the law than by following it. If the trader lacks integrity, they might try to earn more by breaking the law and covering their tracks to avoid detection rather than earning less while following the law. From the outside, a stock trader who is very good at covering their tracks appears as lawful as — and more effective than — one who is genuinely following the law.

This sounds like a real problem; however, OpenAI claims that most AI models have "little opportunity to scheme in ways that could cause significant harm." According to OpenAI, the real harm could come in the future as AI is given "more complex tasks with real-world consequences." In addition, OpenAI warns that AI models will only get better at scheming in the future if the issue isn't dealt with now.

There are other ways AI can be at odds with its own users. For instance, if a user requests information on making controlled substances, an AI chatbot might face two competing objectives: Answer a user's query and keep them engaged, but also abide by system prompts that prohibit sharing this type of potentially harmful information.

In order to guard against AI scheming, OpenAI says its researchers "trained versions of OpenAI o3 and OpenAI o4-mini" by "teaching them to read and reason about a general anti-scheming spec before acting." OpenAI refers to this as "deliberative alignment⁠."

OpenAI says that utilizing this method reduced AI scheming; however, researchers have yet to discover how to eliminate AI scheming altogether. 

An even more concerning discovery from OpenAI's research is that AI models can't simply be taught not to scheme. Doing so would actually just make them better at scheming in the future.

"A major failure mode of attempting to 'train out' scheming is simply teaching the model to scheme more carefully and covertly," the study found.

In fact, OpenAI says, training with the intent to eliminate scheming actually makes the AI model aware that they are being tested for scheming, and as a result, the AI model will simply lie in order to pass the test.

"Anti-scheming training does appear to reduce scheming, but our analysis is complicated by another factor: models often become more aware that they are being evaluated. This situational awareness can itself reduce scheming, independent of genuine alignment," the study says.

In conclusion, OpenAI finds that "scheming is a complex failure mode that we do not expect to diminish with scale." OpenAI continues, saying "our findings show that scheming is not merely a theoretical concern—we are seeing signs that this issue is beginning to emerge across all frontier models today."

To summarize: AI models can intentionally lie to their users. Again, that sounds like a big problem, but OpenAI says it isn't yet.

Per OpenAI, most AI models aren't responsible for tasks with "real-world consequences" yet, limiting the potential for harm — for now.

Disclosure: Ziff Davis, Mashable’s parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.