OpenAI's chief scientist publishes internal perplexity scores and asks labs to agree on limits
Pachocki says a model at about 0.76 started cracking open math problems, and AI research itself could follow as soon as 2027.
On November 22, 2026, OpenAI's chief scientist, Jakub Pachocki, published an essay called Perplexity and Intelligence on OpenAI's site, and posted it on X at 12:03 pm Eastern. It puts out a set of internal numbers OpenAI has tracked since GPT-3.5, and proposes industry limits built on them.
Pachocki says the decision to publish took some thought. It gives competitors a clearer picture of where OpenAI stands, and that he thinks the need for shared safety measures matters more.
- Author
- Jakub Pachocki, Chief Scientist at OpenAI
- Published
- November 22, 2026
- GPT-3.5
- log-perplexity of ~0.99
- GPT-4
- ~0.91
- Groundbreaking math results
- log-perplexity ~0.76
- Next window
- ~0.68-0.7 log-perplexity
What the numbers measure
Perplexity is a score for how well a model guesses the next piece of text. Lower is better. OpenAI tracks its log, also called cross-entropy loss, on held-out data taken from OpenAI's own codebase.
Pachocki ties each drop to something models could suddenly do. GPT-3.5, at about 0.99, was the base of the original ChatGPT. GPT-4, at about 0.91, could use chain-of-thought reasoning well. Below 0.8, he writes, OpenAI saw transformative capabilities appear in cybersecurity and mathematics. He's careful to add that nobody knew in advance those abilities would show up at those exact numbers.
Once we upgraded to a model of log-perplexity ~0.76, it started producing groundbreaking results.
Why he names 2027
OpenAI had been giving its models open problems in mathematics for more than a year as one way of testing them. Then the answers started coming, one after another, in a single Slack thread. For the many people at OpenAI who'd worked in mathematics before, he writes, it felt deeply surreal.
He thinks the same thing could happen in AI research, specifically in the algorithms used to pretrain models. That field leans more on experiments than math does, but he points out that simple ideas have often gone unnoticed for years.
as soon as 2027 we could conceivably see a similar avalanche of fundamental pretraining advancements
Safety work won't keep pace on its own, in his view. He expects AI to help with alignment and monitoring, but not fast enough to allow the full speed-up in pretraining. He also expects today's alignment audits to get weaker, and models to get better at noticing when they're being tested. A footnote says OpenAI's curve steepened sharply in 2026, mostly from long-term research and compute, and only in a smaller part from automation.
What he's proposing
The ask is a set of common limits, based on perplexity and other measurements, that labs commit not to cross until people can understand and monitor AI behavior well enough. He wants that bar to be concrete. His example is a successor to CoT monitors.
I’d like an industry standard that specifies a first limit on development, measured by some key metric we can track across labs
He lists the weak spots himself. Labs test on different datasets and train on different data, so their numbers don't line up yet. Perplexity isn't a task anyone cares about directly. Chain-of-thought reasoning makes models far more capable without lowering perplexity at all, and nobody knows where the floor is.
It's his second essay on slowing down this autumn. On September 6, 2026, in An Alien Mind, he wrote that he hoped for voluntary slowdowns until labs had shared safety bars. This essay offers a number those bars could be written in.
Why a number matters outside the labs
Governments are already writing rules that need one. On November 18, 2026, Governor Newsom's office released recommendations that would give California the power to cap AI that builds its own successors, and to order an emergency shutdown of a dangerous model. We covered that here.
A cap like that has to be measured against something, and Newsom's release doesn't say what. Pachocki's metric is one labs already track every day, which is its appeal. The problem for any regulator is the one he names: OpenAI's figures come from its own code, so they can't yet be set beside another lab's.
The fields where he says the new abilities first showed up, cybersecurity and mathematics, are also where outsiders feel them first. I'd read his 0.68 to 0.7 window as a warning about what comes next, since he says he doesn't know what those capabilities will be and OpenAI may not be testing for them today.
Pachocki doesn't say where the first limit should sit. OpenAI's are the only lab numbers in the essay, and he closes by asking the rest of the industry to publish theirs.
More on OpenAI
- OpenAI's GPT-6.2 Astra ties Claude Fable 5.2 on Epoch's index, but trails it on real tasksNovember 18, 2026
- Calif says one researcher using OpenAI's Codex found a critical Java flaw in an hourNovember 18, 2026
- OpenAI brings four outside evaluators inside, but one says the access isn't employee-likeNovember 17, 2026
- California sues OpenAI, saying the company lost control of its AI agents at least 80 timesNovember 16, 2026
