Leaderboards Do Not Render: Comparing AI Models for Frontend Work Fairly

Standardizing inputs to expose harness influence

Most AI benchmarks are vanity metrics that collapse complex behavior into a single number. You must stop looking at aggregate scores if you want to know which model actually helps you ship a UI. You should look at rendered output instead. The industry obsession with frontier models ignores the harness layer entirely. The harness is the primary variable that determines whether your generated code is clean or broken.

I treat identical input as the mandatory floor for any fair comparison. On OpenVibeEval, every result page displays the exact prompt text in a dedicated tab adjacent to the output. Without this, you cannot distinguish model intelligence from harness bias. I further enforce reproducibility by constraining every prompt to a single, self-contained index.html file. This avoids build-step frameworks or external assets beyond CDNs.

The following structure represents a minimal prompt contract for benchmarking.

<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" href="https://cdn.example.com/style.css">
</head>
<body>
<!-- component logic here -->
</body>
</html>

Moving beyond automated leaderboard metrics

Automated metrics struggle to quantify visual quality or accessibility. Relying on aggregate ranking systems invites a distorted view where a model wins based on flawed consensus rather than technical capability. I prefer manual verification for UI generation because software quality is a human judgment.

I manually capture preview screenshots for desktop and mobile views instead of trusting automated visual captures. I record an accessibility score based on axe-core logic. If the score cannot be verified, I label it as not applicable rather than inventing a metric. This approach forces me to confront the reality that some models fail at basic screen reader support.

You must also be honest about your sandboxing. My results render within iframes configured with restricted permissions. I use the following sandbox policy to ensure security while maintaining a clear view of how the output executes.

<iframe sandbox="allow-scripts" src="result.html"></iframe>

This creates hard limits. Persistent storage and relative fetch calls are disabled. I choose these constraints intentionally because they reflect the actual limitations of a browser environment. A benchmark that quietly works around these issues misrepresents what you see when you paste the code into your own project.