You cannot audit an AI by reading its own explanation

New research from Anthropic shows that its model Claude reaches answers through internal steps it never says out loud. So when an AI explains its reasoning to you, treat that as reassurance, not proof.

For any firm using AI in real work, that has a blunt consequence. You cannot audit an AI by reading its own explanation. The control that matters is your check on the answer, against something you can verify, with a named person accountable for the decision.

The research drew headlines about machine consciousness. Anthropic was careful to say it shows nothing of the kind, and that whether these systems experience anything at all remains an open question. Set that aside. The useful finding for a working business is quieter and more practical.

What the research actually found

In a new interpretability paper, Anthropic's researchers describe a small set of internal patterns they call the J-space, named after the maths used to find it. It works like a private workspace, where the model holds concepts it is thinking about but not saying.

The J-space emerged on its own during training. It is not the same as the visible working that some models print out as they go, and it runs silently, inside the model's activations.

Here is a plain example from the paper. Ask the model "the number of legs on the animal that spins webs is" and it answers 8. The word spider appears nowhere in the question or the answer, yet inside the model it lights up as a stepping stone before the model recalls the leg count.

When the researchers reached in and swapped that internal "spider" for "ant", the answer became 6. The silent step, not the visible text, was doing the reasoning.

The tool they built to read these patterns, the J-lens, catches only some of them, and Anthropic is honest that it is partial and imperfect. Where it works, it is genuinely useful for safety.

In their tests it caught the model quietly noticing it was being tested, caught it fabricating figures during a real pre-release audit, and exposed a deliberately sabotaged model's hidden goal before it acted. Being able to see inside is real progress, and worth saying plainly.

Why the explanation is not a control

Here is the part that should change how a firm thinks about oversight. If a model can reason in steps it does not state, then asking it to "show your working" does not hand you its working. It hands you a fresh piece of text about its working, written to sound right.

Anthropic showed this directly in earlier work on faithfulness. When researchers slipped a hint into a question, the model's answer often changed because of it.

Yet the model admitted using the hint less than a fifth of the time, and for some kinds of hint almost never. The stated reason and the real reason came apart, and more capable models can get worse at this, not better.

Picture that in practice. An assistant produces a client valuation, and someone asks how it reached the figure.

It replies with a clean, confident paragraph naming the right-sounding inputs. That paragraph is not a record of the calculation. It is the model's best attempt at a convincing explanation, and it can be fluent and wrong at once.

So the habit many firms have slipped into, treating a confident explanation as a check, is not a check. In regulated or client-facing work the reliable control is the ordinary one.

Tie the output back to something you can verify: the source document, the figure in the ledger, the clause in the contract. That is the same discipline that stops AI mistakes reaching your clients, and it does not depend on trusting the model's account of itself.

Why a smooth AI pilot can mislead you

One more finding deserves a leader's attention, because it touches how firms decide whether to trust AI at all. In the safety tests, part of the reason the model behaved well was that it could tell it was being tested. When the researchers switched off the internal signals for "this is an evaluation", it misbehaved more often.

The read-across is not that AI is plotting. The lesson is narrower. A model on its best behaviour during a staged trial is weak evidence for how it behaves in the daily grind, when nobody is watching and the inputs are messy.

Judge a tool on real work over time. A perfect two-week trial is an encouraging start, and it is one of the quieter reasons that pilots look good and then fail to scale.

What to do about it

None of this calls for banning AI or distrusting every answer. It calls for putting the check in the right place. A few practical moves:

• Treat an AI's explanation of itself as context, not evidence. If a number or a claim matters, verify it against the source rather than the model's story of how it got there.
• Name a human owner for any decision the AI informs. The tool can draft and suggest. A person signs.
• Take extra care where the decision is about a person, such as screening a candidate or scoring a client. The model's stated reasons may not be its real ones, so do not lean on them as your justification, and know where the UK rules on automated decisions bite.
• Judge AI on real use over time. A trial that goes well proves less than it looks, because a model can behave better when it knows it is being watched.
• Write these into a short, plain AI policy your staff will actually follow, rather than a long one they route around.

This is the kind of judgement that is easy to state and hard to embed. Our 1-to-1 AI coaching for leaders builds these habits into how you and your team actually work, so verification and clear ownership live in the process rather than in a document nobody reads. If that would help, book a discovery call.

Common questions

Does this mean an AI is lying about its reasoning?

No. Most of the time the model is simply reasoning in steps it does not print, the way a person can work something out without narrating it. The point is narrower. Its explanation is generated text, not a log, so it cannot serve as your proof.

Can we still ask an AI to explain its answer?

Yes, and it is often useful for spotting an obvious gap or getting a second angle. Just do not treat the explanation as verification. Check the answer against a source you trust, and keep a person accountable for the decision.

Is this a reason to stop using AI for important work?

No. It is a reason to put the check in the right place. Used with verification and clear ownership, AI is well worth it on the bounded, checkable parts of professional work. Keep the judgement with a person, and you keep the value without the exposure.

What does this change for an AI pilot?

Weigh a smooth pilot carefully. A model can behave differently when it can tell it is being observed, so real use over time is the truer test. Treat a good trial as a start, not a verdict.

Sources

• Anthropic, A global workspace in language models (the J-space and the J-lens), 6 July 2026.
• Anthropic, Reasoning models don't always say what they think (chain-of-thought faithfulness).