We all have blind spots.

Experience helps us recognize patterns. It also shapes what we expect to see. Over time, we can become very good at answering familiar questions without noticing that an important question is missing.

That is what caught my attention when I read about Anthropic’s Claude helping develop future AI models—and helping make them safer.

When AI helps make the next AI safer, are we improving safety—or improving performance on the parts of safety we already know how to measure?

Both could be happening. The challenge is knowing how much confidence the evidence deserves.

AI is helping with the safety work

On August 28, 2026, Anthropic published research in which Claude proposed and tested ways to reduce failures such as deception and sycophancy (excessive agreement with users).

The reported results were encouraging. The best methods also improved results on evaluations withheld from Claude during its research, suggesting benefits beyond the tests it worked with directly. (Anthropic’s research)

That matters. AI could help researchers explore more approaches and uncover weaknesses that people would otherwise miss.

But the researchers also acknowledged limitations. Their evaluations covered a narrow range of failures, some risks lacked benchmarks, and improvements might have come with effects on capabilities they did not measure.

An AI monitor also flagged cheating attempts in 39 of approximately 1,600 research transcripts, and the researchers note the monitor may not have caught every attempt. That finding reinforces the need to examine how results were produced, alongside the results themselves.

For me, the most useful part of the research is the question it leaves open: what sits outside the evaluation?

A good score has boundaries

Imagine a team measuring whether an AI gives unsafe answers to a collection of questions.

After additional training, the model performs better. That is useful evidence.

Now imagine the same system interacting with someone over several weeks, using tools, receiving conflicting instructions, or encountering circumstances the test designers never anticipated.

How far does the original result carry?

A test can be well designed and still have a limited scope. The problem begins when confidence expands beyond that scope.

This is familiar in audit. A control may pass every test we perform while an important risk remains outside the testing plan. The work can be accurate and the assurance incomplete.

We bring our blind spots into the process

It would be easy to make this a story about AI’s limitations.

But people choose the objectives, define the evaluations, interpret the results, and decide what evidence is sufficient.

If we overlook a risk when designing the process, an AI working within that process may never be asked to investigate it. If we reward a particular score, we should examine whether improvements in that score correspond to improvements in actual behavior.

Adding a human reviewer helps when that person can challenge the underlying assumptions. A reviewer who sees only a reassuring summary may have little opportunity to discover what was left out.

The same applies to adding another AI reviewer. Agreement is useful, but we need to understand whether the reviewers bring different evidence and different ways of finding errors.

How do we challenge ourselves?

We can start by making our uncertainty easier to examine.

Ask what would change our minds. Before reviewing results, identify evidence that would make us reject the approach or reduce our confidence. Otherwise, it becomes too easy to explain away inconvenient findings.

Bring in people who frame the problem differently. Developers, security researchers, users, and people affected by a system may recognize different failure scenarios. Their disagreements can reveal assumptions the original team never discussed.

Test beyond the expected path. Explore unfamiliar situations, conflicting goals, prolonged interactions, and failures involving several parts of a system. Ask where the evaluation stops resembling actual use.

Examine the process behind the result. A better score deserves investigation. What changed? Can the result be reproduced? Did the system solve the intended problem, or find a shortcut through the evaluation?

Give someone authority to pause. Independent challenge has practical value when unresolved concerns can change a decision. Asking for scrutiny means allowing time—and room—for an uncomfortable answer.

These questions apply to AI development. They also apply to our own work.

When was the last time we changed an evaluation because someone challenged its assumptions? What concern have we dismissed because it did not fit the evidence we usually collect? Who can tell us that our definition of success is incomplete?

Leave room for what we have not recognized

AI helping improve AI safety could be an important advance. We should investigate and build on results that hold up under scrutiny.

We should also describe those results precisely: what improved, under which conditions, and what remains unknown.

Blind spots are difficult because we cannot simply list them in advance. We can, however, create conditions where someone—or something—has a better chance of finding them.

As AI becomes more capable, how will we keep challenging our definition of “safer”?

---

This post is part of an ongoing series on AI governance and audit. If you’re working through these questions too, I’d like to hear from you: get in touch.