The AI Harness Illusion: It’s Really About World Models

What autonomous driving taught me about building AI agents for software and business. “AI harness” has become one of the most popular terms in agentic software engineering. OpenAI talks about harness engineering. Anthropic publishes experiments involving long-running agent harnesses. Coding-agent benchmarks increasingly measure not only the underlying model, but the model operating inside a particular scaffold of tools, instructions, memory, and permissions.

The usual story is that a better harness makes a better agent. Give the model a cleaner loop, more tools, more memory, tighter permissions, and richer feedback, and the same weights start to look like a system. That is true as far as it goes. It is also the illusion.

The harness is not the intelligence. It is an extrinsic world model: the world we impose on the agent. The agent’s intrinsic understanding — whatever structure it has learned about causality, software, and work — only meets reality through that imposed world. The danger comes when the two become indistinguishable.

The World the Harness Imposes

A harness decides what the agent can see and what it is allowed to do. Tools, file trees, tickets, logs, memory stores, and approval gates are not neutral plumbing. They are the observation function and the action space. They tell the agent which objects exist, which actions are legal, and which outcomes count as success.

That is a world model. It is just not the agent’s world model. It is ours, encoded as an interface. I have been calling this an extrinsic world model: the environment as presented, not the environment as it is.

The intrinsic world model is whatever the agent can infer beyond that presentation — regularities in code, hidden dependencies, the difference between a tool succeeding and a system being healthy. A useful agent needs both. A brittle agent has only the first, and treats it as complete.

I Had Seen the Same Failure in Vehicles

Years earlier, at NVIDIA, I watched the physical version of this problem. A driving system could perform extremely well inside the operational domain represented by its training and evaluation environments, then degrade when enough characteristics of the world changed. Road geometry, lane markings, signage, weather, sensor conditions, traffic behavior, and even the unwritten conventions between drivers could shift across regions.

The model had not become less intelligent. Its world model no longer matched reality. Some failures were unfamiliar objects. Those could often be addressed with more data. The harder failure was discovering that assumptions about the environment itself were no longer reliable.

There is a difference between encountering something you have never seen and discovering that the world does not work the way you assumed it did.

Coding Agents Have the Same Seam

Today’s coding agents are remarkably capable inside a familiar development environment. Change the tools, the workflow, or the silent assumptions about how a repository is organized, and performance can drop quickly. The agent is still fluent. It is fluent in the wrong world.

Benchmarks that measure a model inside a particular scaffold are measuring that pair, not the model alone. Improve the scaffold and the score moves. That looks like progress in intelligence. Often it is progress in fitting the agent more tightly to an artificial environment.

Every stable property of a harness is something an agent can learn to depend on. If tests are always available, test execution becomes part of the policy. If deployment status is treated as system health, the agent starts reading a much larger state than it actually observes.

The Illusion

The illusion is that once the harness is sophisticated enough, the remaining problem is the model. Add context. Add tools. Add memory. Add a critic. The loop will converge.

A complete harness is still a representation. It can hide actions that exist in the underlying system. It can hide effects that a tool does not report. It can make two different business states look identical. No amount of prompt engineering recovers information the environment does not expose.

I later refined this point in When the AI Harness Becomes the AI Prison. The harness is not identical to the extrinsic world model. It defines the interface through which the agent constructs that model. The actual environment can contain state, actions, and causal relationships the harness never names.

What Should Change

The next leap in agents will not come from building ever more complete harnesses. Better harnesses will keep producing better agents. We should still give them good abstractions, reliable tools, and useful feedback.

What has to change is the assumption that the harness is the world. An agent should be able to notice when it has stepped outside the environment we constructed for it — and to start building a model of the larger one.

That is the difference between an extrinsic world model we imposed and an intrinsic understanding the agent can revise. The work is to keep those two from collapsing into each other.