Claude and 12 JS Frameworks: What Agent Code Costs and What Bundle Ships to the BrowserArticle · digest for November 6
Comparative still life of AI token cost measurements and client bundle weights across 12 web frameworks.
Article4 sections≈ 4 min

Claude and 12 JS Frameworks: What Agent Code Costs and What Bundle Ships to the Browser

Over the past year, web development has been swept by a wave of so-called "vibe coding." Across tech communities and social media, a stubborn set of myths took root: supposedly some JavaScript frameworks are "ideally optimized for AI comprehension," while others are "too complex for language models."

Frontend engineers started picking architecture stacks based not on technical project requirements, but on gut feelings: "I feel like Claude understands this framework better."

Renowned web engineer, web platform performance specialist, and author of the Read the Tea Leaves blog Nolan Lawson decided to put those claims to the test with hard numbers. In his extensive study, also featured in JavaScript Weekly #809, he ran an impartial crash test across 12 popular web stacks.

Competing in the benchmark were: Astro, Hono + htmx, SvelteKit, Qwik, Nuxt, SolidStart, React Router, TanStack Start, Next.js, Gracile, Angular, and plain Vanilla JS.

Experiment Methodology: 60 Independent Builds of One App

To rule out luck, Lawson designed a strict, reproducible experiment:

  1. Single technical spec: the autonomous Claude Code agent powered by Sonar/Sonnet was tasked with building a Pokémon catalog web app from scratch against the public PokéAPI: search, pagination, stats display, and a detail view for each Pokémon.
  2. Five runs each: for each of the 12 frameworks, the agent was launched 5 independent times with a clean cache to smooth out neural network stochasticity.
  3. Automated spec verification: finished solutions were run through a strict end-to-end test suite (checking accessibility tree semantics, button reachability, and data accuracy). The agent couldn't cut corners or turn in a broken prototype.

Measurements tracked two main axes: generation cost for the developer (in dollars for API tokens) and final product quality for the end user (client bundle weight and Core Web Vitals scores).

Token Cost: The Myth of the "Ideal AI Framework" Shattered

The study's first fundamental finding: the difference in generation cost across most modern frameworks was practically negligible!

Analyst Gautam Parab noted that token-cost confidence intervals for the vast majority of participants overlap:

  • The most economical stack was Hono + htmx: median build cost was $2.67 per run;
  • Popular heavyweights (Next.js, SvelteKit, Nuxt, SolidStart) clustered in a tight $3.20–$3.80 band;
  • The priciest and most finicky was Angular: the median hit $4.97, and on several early attempts the agent got stuck in complex Dependency Injection conventions, burning millions of tokens debugging modules on its own.

The gap between the cheapest and most expensive options was under 1.8x. That completely demolishes the marketing myth that picking a stack can dramatically slash your API bill in agentic development.

The Real Shock: Client JavaScript Weight in the User's Browser

When Lawson measured how much JavaScript each framework ships to a real person's browser, however, a genuine disaster emerged:

  • Astro: 0 KB of client JS (pure server rendering without forced hydration);
  • Hono + htmx: just 18 KB;
  • SvelteKit: 36 KB;
  • Qwik: 42 KB;
  • Next.js / React Router / TanStack Start: 115 KB to 142 KB of client bundle for the exact same functionality!

Feel the scale of the absurdity: to render a simple searchable table, a modern React stack forces the user to download eight times more script than an htmx pairing and heat up their smartphone CPU compiling hundreds of kilobytes of runtime. Interaction to Next Paint (INP) and Total Blocking Time (TBT) on mobile devices saw heavy meta-frameworks lose to Astro and htmx by multiples.

The Industry's Main Question: Why Do We Need a Heavy Stack in the Age of Agents?

Nolan Lawson framed a blunt, brutally honest conclusion:

"Historically, we chose heavy meta-frameworks like Next.js for developer ergonomics: so that we humans could comfortably write components in a unified JSX syntax. But if an agent is writing the code for us now, why on earth are we still forcing hundreds of kilobytes of client-side bloat onto users?"

A visitor to your site couldn't care less how "cleanly" a neural network composed a virtual DOM. The user pays for mobile data, suffers through janky rendering, and waits for a button click to respond.

Lawson's benchmark makes it clear: server-centric approaches (Astro, htmx, classic HTML) aren't merely holding their own in the era of AI agents—they're becoming the undisputed leaders. They're cheaper to generate for the creator and immeasurably faster and lighter for the real human on the other side of the screen.

Want a deep diveVotes help us choose what to break down in detail. This material has 0 votes.
Sources · 3