Same Model, Different Results: Why Your AI Coding Harness Matters

The model is the engine, the harness is the steering

The industry obsession with model frontier benchmarks ignores the most consequential layer of the developer stack. We treat these agents as black boxes where the intelligence comes entirely from the weights. The truth is that a model is merely a statistical token generator. It exists in a vacuum until a harness provides the context.

The harness is the infrastructure wrapping the model. It defines what the agent sees, which local tools it can invoke, and how it handles retries when a command fails. When your AI agent produces sloppy code, you are likely looking at a failure of its system prompt or context management logic. You are seeing the harness make a bad decision, not the model failing to understand your syntax.

Every harness introduces its own layer of abstraction. It manages conversation history, selects which tools to execute, and decides how to patch your files. These decisions are invisible to you in a chat interface, yet they fundamentally change the model logic. Switching a harness often yields a more drastic change in result quality than upgrading a model version.

Measuring the divergence between wrappers

I often see development teams compare models by looking at leaderboard numbers that mean nothing for their specific workflow. Real world testing reveals massive gaps. For example, the same underlying model can score 77 percent in one harness and 93 percent in another. This 16 point swing comes entirely from the wrapper.

This divergence is not an outlier. Across independent benchmarks, harness performance varies by 5 to 40 percentage points. If a tool feels inconsistent or struggles with specific frontend patterns, look at the harness engineering. It might be aggressively truncating context or providing poor tool descriptions. This effectively starves the model of the information it needs to succeed.

Teams often assume that using a premium model is a shortcut to better results. When the output remains mediocre, they simply pay for a larger model release. They rarely audit the harness configuration or the prompt formatting. This represents a fundamental misdiagnosis of their own tooling limitations.

The hidden mechanics of agent loops

The loop of gathering history and executing tool calls looks identical in almost every agent. The difference lies in the unglamorous plumbing. Some harnesses run locally with full project access, building a rich memory of your codebase. Others work in a remote, sealed container, viewing only what you copy or what the harness deems relevant.

File editing strategies are also heavily idiosyncratic. Some agents rewrite the entire file for every change. Others attempt precision patches. Some agents ask clarifying questions when requirements are ambiguous. Others commit to an action immediately. These architectural choices are encoded into the harness. You are opting into these behaviors every time you open the terminal interface.

The best harness is one that remains reliable over long sessions. A system that succeeds on a single turn but falls apart when the file list grows is not helpful for real development. High quality harnesses prioritize persistent state and clean history management so that the agent does not lose focus after five minutes of activity.

The economics of context management

Efficiency is the silent metric. Two harnesses might complete the same task, but one might consume six times the tokens of the other. This efficiency is driven by the context pruning policy and how the harness summarizes intermediate steps. A model is only as efficient as the context it is fed.

When a harness is poorly tuned, it generates redundant reasoning. It uses tools inefficiently. Even a highly capable model can look mediocre if its harness forces it to re-parse the same verbose tool output repeatedly. Optimizing token usage is an engineering problem at the harness layer, not an issue with the base model.

You should measure your agent by its total cost per completed feature. If you have to pay for a mountain of redundant tokens just to get a single component generated, the harness is working against you. The most effective agents are those that minimize the round trips between the model and the filesystem.

Evidence from the OpenVibeEval results

I maintain OpenVibeEval. It is a benchmark where I run identical frontend prompts across multiple models and harnesses. I restrict every prompt to a single self-contained index.html file with no build steps. This constraint is the only way to ensure the comparison is honest and sandboxed.

Currently, the results tree holds 169 entries across ten different harnesses. These include zcode, antigravity-cli, and claude-code. In these results, I can observe exactly how the harness changes the output on identical prompts. The variation is stark. A harness is not just an interface. It is the primary factor in whether a prompt leads to a functional UI or a broken mess.

My observations show that frontend generation is highly sensitive to the initial system instructions. When I provide the same prompt to two different agents, one might respect the CSS framework requirements while the other ignores them. This happens even when the underlying models are identical. The difference is purely in the harness instructions.

Next steps for your agentic workflow

If you want to move beyond vendor marketing and start evaluating tools based on actual performance, use this checklist to audit your current harness setup.

First, run a series of identical prompts across two different harnesses using the same model. Does the frontend quality change in spacing or structure? A measurable difference indicates that the harness is the variable you should optimize.

Second, examine how the harness handles context. Does it seem to forget file contents after a few turns, or does it hold state across long sessions? You need an agent that respects your project history.

Third, audit the file editing strategy. Does it perform clean, minimal edits, or does it frequently destroy or duplicate code blocks during iteration? A good agent respects your existing formatting conventions.

Fourth, verify if the harness provides a way to customize behavior via configuration files like CLAUDE.md. A tool that lets you set rules is objectively more powerful than one that hides them.

Fifth, check your cost per task if the data is available. A high token usage might indicate poor context management rather than model complexity. Keep track of your burn rate to ensure you are not overpaying for inefficiency.