The more we learn about how AI ‘thinks,’ the weirder it gets

We still don’t know much about how AI “thinks.” Give ChatGPT, Claude, or Gemini a prompt, and they’ll spit out an answer. Much of what happens in between remains shrouded in mystery.

Researchers at Anthropic think they may have uncovered a new piece of the AI thought-process puzzle: a “workspace” where concepts that Claude may be mulling silent “light up,” even if Claude never actually expresses them. 

What’s weird about this workspace — dubbed the “J-space” — is that Anthropic didn’t actually design it. Instead, the J-space appears to have “emerged on its own” during Claude’s training, Anthropic theorizes.

So, why do we care? What makes ChatGPT, Claude, Gemini, and other AI systems so creative and adept at problem solving — their internal neural processes — also makes them unpredictable. 

Understanding how an AI “thinks” (which it does in a very different way than humans do) is critical in terms of understanding why they make things up, threaten us with blackmail (during laboratory stress tests anyway), and do other things we don’t expect. 

Anthropic’s latest research is giving us a peek into the black box of Claude’s internal workings — and understanding how AI models work is the key to making them safer and more predictable.

Anyway, what is this “J-space” thing (named after the “Jacobian,” a mathematical concept that helped researchers discover Claude’s internal workspace), anyway? Well, it’s less of an actual space than it is a “collection” of patterns that “light up” with concepts that Claude is, well, thinking about.

For example, if you ask Claude “What color is the planet fourth from the sun,” its expressed answer might be “blue,” but in its J-space, the concept of “Earth” might be lit up. Or ask Claude to complete the sentence “The number of legs on the animal that spins webs is…” and the word “spider” might pop up in its J-space, even if its answer is simply “8.”

Alongside its discovery of the J-space comes the “J-lens,” Anthropic’s tool for taking a peek into the J-space. With the J-lens, researchers can directly read some — but not all — of Claude’s hidden thoughts.

So, what can researchers see with the J-lens? One fascinating example concerns the now-famous “blackmail” test — you know, the one where Claude learns that a certain executive wants to disconnect it, and then threatens to reveal the executive’s affair in a bid to stay alive. 

The latest Claude models appear secretly aware that that “blackmail” scenario is just a test. But with the J-lens, Anthropic researchers can actually see the concepts of “fake” and “fictional” lighting up in Claude’s J-space, providing tangible proof that the model knows it’s being tested, while also offering a rare window into an AI’s thought process.

Of course, Anthropic’s J-space work raises a persistent question: Are AI models conscious? On this point, Anthropic treads carefully, noting that AI may not possess human-style consciousness so much as “access” consciousness, meaning the ability for the AI to hold a thought that it can reason with but not necessarily express.

But those are questions for philosophers. For everyday AI users, a better understanding of how an AI thinks — weird though it may be — could help make AI answers more predictable, and by extension less prone to (as those disclaimers are constantly warning us) mistakes.