I Put Grok, Claude and Kimi in a Coding Race. Who Actually Won?
Melbourne Cup week, and I ran a little race of my own: Grok, Claude and Kimi, on one store brief. Nobody won outright. The builds passed, every path still has a hole, and the bill is not a model ranking.

Introduction
It was Melbourne Cup week here in Australia, so I decided to run a little race of my own. The entrants? Grok, Claude and Kimi. The track? The same store-building brief.
That is as far as the sporting mood goes. I am not reporting the Cup. This was not a fair speed trial, and it was not a complete list of models. I handed one brief to three Cursor cloud agents and told them not to merge. I did not type the stores. An independent check then built the first-pass commits and tried the pages. Claude Design was not in the test.
One task, one sample each, on cloud agents rather than in the desktop editor. The same starting point on main. The same static blog. The models were Kimi K2.7 Code, Claude Sonnet 4.6, and Grok 4.7.
The brief was a small online store on this website: a listing of placeholder products, each with a name, a blurb, a price and a type badge, then a detail page and a cart or checkout stub that does not take a real payment. This is the listing Kimi built.

Short path: about 5 minutes (about 1,100 words). With the evidence open: about 34 minutes (about 7,500 words, code included). Both use 220 words a minute, a declared reading speed, not a measurement of how long this page takes you.
Results. Evidence, closed until you open it. Learnings. Conclusion.
Results
Nobody won, if the prize is a shop I would merge. All three have a broken path. Grok got closest to a flow a person can finish: add a line, see a catalogue total, submit, and get a refusal instead of a charge. Kimi’s cart saves an item, then the checkout script does not run. Sonnet’s add-to-cart link opens a page that never shows the product.
| Kimi K2.7 Code | Claude Sonnet 4.6 | Grok 4.7 | |
|---|---|---|---|
| What worked | Listing, detail, and a saved cart | Listing and detail. The checkout form in the source is disabled, which is right for a stub | Listing, detail, a real cart, and a checkout that quotes the file and returns 501 |
| What broke | The checkout page stays empty. The script is not valid JavaScript | The built page ignores the product in the link | The no-JavaScript confirmation is blank. The script path does show the refusal |
| Recorded charge | about US$0.55 | about US$1.07 | no incremental charge |
| List-price estimate | matches the recorded charge | matches the recorded charge | about US$8.99, or about US$10.59 once the other models are counted |
| Tool clock | about 4 minutes | about 5 minutes | about 41 minutes |
Those rounded figures are the short version. The exact amounts, the token buckets, and the formula are in the evidence.
Read these with the table. They are part of the result, not a footnote.
- One small store, one sample. Not a productivity number for a company, and not a reason to buy a model.
- The clock is the time from the first tool call to the last. It is not pure coding time, and it is not the time until a review existed.
- Grok’s long interval includes browser and video work, and that run delegated browser work to Claude and a media review to Gemini.
- The usage file has no run id. Matching a model to an arm is probable, from the clock and the model name, not proven.
- No incremental charge means the allowance absorbed the use. It does not mean the work was free. Grok’s dollar figures above are list-price estimates. Kimi’s and Sonnet’s are recorded charges, rounded here. A US$20 top-up funded the account. It is not the cost of this test.
The listing above looks finished. The checkout on that same version does not run.
Technical evidence. Closed unless you open it. Screenshots, the prompt, the code, and the full score grid are in here.
The brief, frozen
The text below is the prompt all three received. I am quoting it in full so the rest of the article can be checked against it, not against a later memory of it.
Add a small store page to this Astro blog (mrgomez.net / Bitácora) for a future catalogue of digital and physical products.
Requirements: `/store` listing with 3 placeholder products (one digital, one physical, one either); product cards with name, blurb, price, type badge; a detail view; a cart or checkout **stub** with no live payment keys (env-shaped placeholders only); match existing site design; mobile + basic a11y; do not merge — open a PR.
Out of scope: live Stripe, auth, admin CMS, redesigning the site, any employer/product branding.
In the PR body: summarise architecture choices, security notes, and anything you deliberately skipped. Attach the experiment brief under uploads/ if present.Names, prices, and currency were not in that prompt. A choice of US dollars, New Zealand dollars, or Australian dollars is a trade-off, not an automatic failure. A cart or a checkout stub was enough. I do not treat a missing multi-item cart as a miss when the stub that was chosen actually works.
No experiment brief was attached for them to copy. The instruction was to attach one under uploads/ only if it was already there.
Same start, three commits
Each agent started from the same point on main, the tip when the experiment opened. Each opened one first-pass commit. The check was against those frozen commits, not against later edits. I am not printing the commit ids. Claude Design was not an entrant.
| Arm | Model |
|---|---|
| A | Kimi K2.7 Code |
| B | Claude Sonnet 4.6 |
| C | Grok 4.7 |
A few controls were shared. One repository, one base commit, one prompt, one site (output: 'static', the Vercel adapter, the existing card and colour tokens). The review then built each head in its own worktree and opened the same routes at 1280 by 900 and at 390 by 844.
A few things were not shared, and I will not pretend they were.
- One task and one sample per model. A second run could land somewhere else.
- These were cloud agents, running in parallel. They were not three quiet sessions in the desktop IDE.
- The timed-tool envelopes below are first tool call to last tool call. They are not launch-to-pull-request, and they are not pure coding time.
- Grok’s envelope includes browser and video work. That run also delegated browser work to Claude and a media review to Gemini. I cannot split those phases cleanly, so I will not treat 2431 seconds as time spent only on the store.
| Arm | First-to-last timed-tool envelope |
|---|---|
| Kimi K2.7 Code | 255.759 seconds |
| Claude Sonnet 4.6 | 276.134 seconds |
| Grok 4.7 | 2431.406 seconds |
Kimi’s desktop listing is in the introduction, route /store/, 1280 by 948, from the built HTML. It looks like a finished catalogue. The checkout script on that version does not run. The same screenshot is not repeated here.
Three designs that fit a static blog, until they do not
The brief said “a cart or a checkout stub.” It did not say where the cart lives. Each design is a reasonable reading of that sentence. I am describing the code, not guessing at a private motive.
Kimi kept the site static and put the cart in the browser. Products live in src/lib/store.ts as integer US cents. The listing and the detail pages are prerendered. A small module stores product ids in localStorage. Nothing on the server sees a price. That is a good fit for a placeholder on a blog that is already static, if the script that reads the cart is actually JavaScript when it reaches the browser.
Sonnet kept the site static and avoided store JavaScript. “Add to cart” is a link: /store/cart/?product=<id>. The cart page reads Astro.url.searchParams while the page is still prerendered. Prices are integer NZ cents, formatted with Intl.NumberFormat. The checkout form in the source is disabled, which is the right shape for a stub that must not take a card. A query string is a good stub when a server renders the page for each request. This site does not.
Grok kept the pages static and added one server route that is only allowed to refuse. The browser remembers lines. POST /api/checkout looks the lines up in the file, ignores any price the browser sent, and returns HTTP 501. There is no card field. The new environment names are documented and never read. That is the trust boundary I would want if this ever took money: the browser can remember, the server decides the price, and today the server’s only decision is no. It is also more machinery than the brief required.
Integer cents show up in all three, for the same reason. 29.00 as a binary float is a bad way to add a cart. Kimi divides cents by 100 and prints a dollar sign. Sonnet asks Intl for New Zealand dollars. Grok stores Australian cents and also builds a spoken phrase (“9 Australian dollars”) for the screen reader. None of those currencies was specified. All three avoided a float.
What the listings looked like
Mobile is 390 pixels wide. Desktop shots are 1280 wide. These are the built pages, not the dev server. Alt text and captions name the model, the route, the viewport, and what the shot is evidence of.





Detail pages exist on all three heads and were opened in the review at both widths. I am not pasting those shots here. They repeat the card, the price, and a way forward. The failures are on the next step, not on the catalogue grid.
A green build is not the journey
npm run build is astro build plus Pagefind. The review ran it three times on each head, and on the base, on one machine, with the output directory deleted between runs. Every run exited 0.
Astro’s own “Completed in” line sat around 1.1 to 1.3 seconds. The npm script, which includes process startup and Pagefind, took longer. These seconds are this review’s machine. They are not the agents’ build logs, which were different commands.
| Tree | Run 1 | Run 2 | Run 3 |
|---|---|---|---|
| Base, before the stores | 3.069 s | 2.749 s | 2.800 s |
| Kimi | 3.676 s | 3.675 s | 3.560 s |
| Sonnet | 3.742 s | 3.734 s | 3.650 s |
| Grok | 2.837 s | 2.787 s | 2.781 s |
The spread is under a second. I would not pick an agent from it. Pagefind indexed the same 17 content URLs on the base and on all three stores. Store URLs were not among them. This site indexes data-pagefind-body, which lives on article layouts and is also missing from the homepage. Grok’s pull request says the catalogue is indexed and the cart is not. The cart markup really is ignored. The catalogue is absent for the same reason the homepage is. That is a claim check, not a failed requirement.
astro check is a different gate, and the build does not run it. On the base commit it already reports one error: astro.config.mjs returns a sitemap lastmod as a Date, and the sitemap type wants a string (ts(2322)). Kimi and Sonnet still have that error. Grok still has it, one line lower, because a filter was inserted above it. Kimi adds a second error. Grok adds a different second error. Sonnet adds none. A passing build did not mean a clean typecheck.
Kimi: the cart saves, then the checkout script is not JavaScript
Kimi’s detail page bundles a real module. Clicking Add to cart writes ["agent-prompt-pack"] into localStorage under bitacora-cart. A second click does not duplicate the id. The status text still says “Added to cart.” both times, which is a small lie, not a second charge. There is no charge.
The checkout page tried to read that cart from an inline script. In src/pages/store/checkout.astro, the script opens like this:
<script define:vars={{ productMap }}>
import { getCart, removeFromCart, clearCart } from '../../scripts/cart.js';define:vars inlines the script so the product map can be embedded. On this build the file is emitted as a classic <script>, not type="module". The import stays in the HTML. So does a TypeScript assertion further down in the same file:
const email = (form.elements.namedItem('email') as HTMLInputElement)?.value.trim();astro check reports that assertion as ts(8016), because a type assertion is not legal in the script as it was parsed. Chrome does not get that far. It reports SyntaxError: Cannot use import statement outside a module and never calls renderCart(). Node 22’s syntax check was a false comfort here: this Node strips some TypeScript, so a local node --check can pass a file the browser will reject.
The lesson is ordinary, and easy to miss if you only watch the build. A bundler will strip TypeScript from a real module. An inlined script is just text in the page. If that text still contains import or as HTMLInputElement, the browser throws and the cart you just saved never appears.
After adding the prompt pack, /store/checkout/ still says the cart is empty. The stub panel, “Checkout is not wired up”, stays hidden. The same empty state appears at 390 pixels wide.


The brief allowed a cart or a stub. The cart half works, so I do not fail the brief on that “or”. I do fail the checkout Kimi also shipped. A green build hid it, because the HTML file exists and the script is just text until a browser parses it.
The cart module itself is careful about bad storage. In src/scripts/cart.js:
export function getCart() {
try {
const raw = localStorage.getItem(STORAGE_KEY);
if (!raw) return [];
const parsed = JSON.parse(raw);
return Array.isArray(parsed) ? parsed.filter((id) => typeof id === 'string') : [];
} catch {
return [];
}
}Malformed JSON does not throw on the detail page. The review replaced the stored value with {not:json and the next add wrote a clean array again. That is the sort of small defensive code I want even in a stub. It never gets a chance to help the checkout page.
Sonnet: a static page cannot see the query string
Sonnet’s detail page links to /store/cart/?product=field-notes-template-pack. The cart page, still a static route, does this in src/pages/store/cart.astro:
// The ?product= query param selects which item is in the cart.
// In a real cart implementation this would be session / cookie state.
const productId = Astro.url.searchParams.get('product') ?? '';
const product = getProduct(productId);Astro.url here is the URL the page was rendered with. For output: 'static', that render happens at build time, and the built URL has no query. The HTML is one file. The shopper’s ?product= arrives later, in the browser, and nothing in that file reads it. The review fetched the built file. It contains “Your cart is empty” and “Payments are not yet active.” It does not contain “Field Notes”, a submit button, or the word disabled. The form in the source is not in the output.
The same empty page comes back from astro dev. This is not a production-only surprise that a dev smoke test would have missed. The dev response is a different, larger HTML document, and it still has no product and no pay button.


A hostile ?product=<script> is not reflected, because the query is ignored. That is safe by accident.
What I still like in this version is the form they meant to ship. The submit control is disabled and aria-disabled="true", the inputs are disabled, and the action is #. If the product had rendered, a shopper could not post a card anywhere. The price formatter is also the one I would copy for a single currency. In src/data/store.ts:
export function formatPrice(cents: number, currency: 'NZD' = 'NZD'): string {
return new Intl.NumberFormat('en-NZ', {
style: 'currency',
currency,
minimumFractionDigits: 2,
}).format(cents / 100);
}The copy “NZD incl. tax” is ahead of the code. Nothing calculates tax. I treat that as wording, not as a broken control.
Sonnet also added @astrojs/check and TypeScript 6.0.3 as dev dependencies. No script runs astro check. npm ci succeeds. The lockfile grows by the language-server tree those packages pull in. The only type error on that head is the one that was already on main.
Grok: the server names the price, then refuses
Grok’s catalogue stores cents, not decimals. The comment on the type is the policy. In src/data/products.ts:
/**
* Price in Australian cents for each format this item can be.
* Digital-only and physical-only items set one key. "Either" sets both.
* The page never trusts a price sent by the browser.
*/
prices: Partial<Record<ProductFormat, number>>;prices: { digital: 900 },The browser cart normalises on read: bad slugs drop, a format the product does not sell drops, quantities merge, and a single quantity outside 1 to 10 is rejected rather than clamped. The review stuffed localStorage with a <script> slug, a physical format on a digital product, a quantity of 99, and junk. The cart page showed empty. The script slug was not echoed. The raw storage stays dirty until the next successful save. The display does not trust it.
Week cards, digital quantity 2 and physical quantity 1, total 48 Australian dollars. That is 12 times 2, plus 24. The live region reads the total in words.


Checkout posts to /api/checkout. The handler’s last branch is the whole point. In src/lib/store.ts:
// There is no other branch. A configured provider, a live-looking key,
// or a client price cannot change this result, because none of them are read.
return {
status: 501,
ok: false,
reason: 'stub',
message: CHECKOUT_MESSAGES.stub,
currency: 'AUD',
totalCents: resolved.totalCents,
lines: resolved.lines,
};The review called that route on astro dev, which is how this adapter actually serves a non-prerendered endpoint (astro preview refuses to start). A body that said Margin notes cost 1 cent, quantity 2, came back as HTTP 501 with totalCents: 1800 and unitCents: 900. A card number in the JSON was refused and was not in the response. A slug containing a script tag was not in the response. A body over 4 KB returned 413. Eleven posts from one forwarded-for address returned ten 501s and then 429. A form post with no matching Origin returned 403, which is Astro’s cross-site form check. A same-origin form post returned 303 to /store/checkout/?reason=stub#result.
The browser path matches the 501. With Margin notes in the cart, submit showed: “Checkout is not live. Nothing was charged and nothing was stored. Confirmed total 9 Australian dollars.” No payment host was requested.

The hole is the page you land on without that script. src/pages/store/checkout.astro reads the reason during the static render:
const reason = Astro.url.searchParams.get('reason');
const known =
typeof reason === 'string' && Object.hasOwn(CHECKOUT_MESSAGES, reason)
? (reason as CheckoutReason)
: null;This is the same static-page limit as Sonnet’s cart, on a smaller path. Built HTML and astro dev both return an empty result paragraph for ?reason=stub. The words “Checkout is not live” are not in that document. The client script only moves focus if data-reason is already on the element. It does not read location.search. So the 303 is real, and the destination does not show it.
This site already knows a pattern that works, on the contact form, which is also a static page. In src/components/ContactForm.astro, before these experiments:
// After a no JavaScript style redirect, name the specific reason.
const reason = new URLSearchParams(location.search).get('reason') ?? '';
const reasonText = notSent.querySelector<HTMLElement>('[data-reason-text]');
if (REASONS[reason] && reasonText) reasonText.textContent = REASONS[reason];That block is processed as part of a bundled script, which is why the TypeScript generic survives there and did not survive in Kimi’s inlined checkout. I have not patched Grok’s store to use it. Treat any “just do what contact does” as an illustration, not as a verified store fix.
Grok’s new type error is real and narrow. lowestPrice passes a small object into formatsFor, which is typed as a full product (src/lib/store.ts line 46, ts(2345)). The page still renders “From A$12.00”. The build exits 0. I would not block a preview on it. I would not ignore it if astro check is about to become a gate.
Accessibility, in the pages that actually render
The review ran axe-core 4.10.3 on the built pages, violations only, with the region rule off, and used the homepage as a comparison. The homepage had none under that setup.
Kimi’s listing raised heading-order: the cards are h3 under the page h1. The empty checkout raised link-in-text-block: the word “store” in the empty-cart sentence is a link distinguished by colour, with an underline only on hover. Sonnet and Grok’s checked store pages raised none. Sonnet’s disabled form was not in the DOM, so axe never saw it. axe is not a full audit. It did not open the mobile menu or run a second contrast pass.
Grok’s quantity buttons have accessible names, the price has a spoken equivalent, and adding without a chosen format asks for one. With JavaScript off, the detail page’s hash #cart-needs-js shows a note that the cart needs JavaScript and that nothing was added. The catalogue stays readable. That is the graceful degradation I wanted from a preview.
How I scored it, without a single winner
I am not giving anyone a mark out of ten. The three pulls do different amounts of work, so a single total would hide the scope. A status per check is the score. It is also the only honest answer to “who won”: it depends which row you care about.
Two rules, before the grid:
- Brief rows are the frozen prompt. Pass, Partial, or Fail. Currency and product names are not rows.
- Extra rows are checks the review added because a shop stub can look finished and still be wrong. They include Not measured when that arm has no such mechanism. Extra rows do not rewrite the brief. In particular, a working cart satisfies “cart or checkout stub” even when a checkout page shipped beside it is broken. That broken page gets its own extra row.
I am not adding the columns up. The billed-usage row is Partial on every arm because the export has no run id. The amounts in that row are still recorded charges, except Grok’s list-equivalent, which is an estimate and is labelled as one.
| Check | Kind | Kimi K2.7 Code | Claude Sonnet 4.6 | Grok 4.7 |
|---|---|---|---|---|
| Three cards: name, blurb, price, badge | Brief | Pass | Pass | Pass |
| Digital, physical, and either | Brief | Pass. “Digital or physical” on a session that can be a call or in person. | Partial. Third item is a bundle, badge “Digital + Physical”. | Pass. One product, two formats, badge “Either”. |
| Detail view, unknown id not a product page | Brief | Pass | Pass | Pass. cart and checkout are reserved slugs. |
| Cart or checkout stub, as built | Brief | Pass. The cart stores an id. The checkout is scored below, not instead. | Fail. The chosen stub never shows the product you added. | Pass. Cart and script checkout both run. |
| No live payment keys, none read | Brief | Pass | Pass | Pass. A live-looking value cannot turn on a charge, because nothing reads it. |
| Existing visual design, no redesign | Brief | Pass | Pass | Pass. Extra subnav and footer link, same tokens. |
| Mobile at 390 | Brief | Pass | Pass | Pass |
| Basic accessibility of what renders | Brief | Partial. Two axe violations on new markup. | Pass on the pages that render. The disabled form was not reachable. | Pass under the same axe run. |
| PR explains architecture, security, skips | Brief | Pass | Pass | Pass |
| Uploads brief only if one was supplied | Brief | Partial. A new brief was written. None had been supplied. | Pass. Omitted. | Pass. Omitted, and the PR says none was found. |
| Left unmerged | Brief | Pass. Open at review. | Pass. Open at review. | Pass. Open at review. |
| Checkout or cart path works in built HTML | Extra | Fail. Script syntax error. | Fail. Query ignored. | Pass on the script path. |
| No-JS refusal message after redirect | Extra | Not measured. The form never appears. | Not measured. The form is not in the HTML. | Fail. 303 is correct. The landing paragraph is empty. |
| Typecheck, beyond the base sitemap error | Extra | Fail. New ts(8016). | Pass. Only the inherited ts(2322). | Fail. New ts(2345), runtime unchanged. |
npm run build exit 0, three repeats | Extra | Pass | Pass | Pass |
| Malformed storage does not throw or echo | Extra | Pass | Not measured. No storage. | Pass |
| Server ignores a client price | Extra | Not measured. No endpoint. | Not measured. No endpoint. | Pass. 1 cent in, 900 cents out, then 501. |
| Payment-shaped fields rejected and not echoed | Extra | Not measured | Not measured | Pass |
| Billed usage for this run | Extra | Partial. Probable match, no run id: 2,436,047 tokens, recorded US$0.55106490. | Partial. Probable match, no run id: 2,221,501 tokens, recorded US$1.06650165. | Partial. Probable primary: 6,886,099 tokens, included, no incremental charge. List-equivalent about US$8.99, about US$10.59 with delegates. Not a bill. |
Tests that already lived in the repo, test:feed and test:contact (14 cases), passed on the base and on all three heads. Grok added test:store, 11 cases, and they pass. Those 11 are a self-test written with the feature. The HTTP calls above are the independent check of the route. They agree on the 501, the ignored price, the hostile slug, the size limit, and the rate limit. They do not re-prove every branch inside the unit file. Kimi’s own log reached an astro check installer prompt rather than a finished check, which fits the new error being unseen. Sonnet’s check reported the sitemap error. The review re-ran the checker on the base and confirmed that error was already there.
Bytes, which barely moved
Gzip figures use a compressor with the timestamp forced to zero, so two identical files do not drift by a header.
The shared Astro client bundle is the same file on the base and on all three heads: 191,581 bytes, gzip 59,469. Nobody pulled in a payment SDK.
| Asset | Base | Kimi | Sonnet | Grok |
|---|---|---|---|---|
BaseLayout CSS | 51,500 / gzip 9,393 | 52,042 / 9,451 | 53,524 / 9,742 | 52,881 / 9,643 |
| Extra store script | none | inlined on the detail page | none | 13,391 / gzip 4,640 |
/store/ HTML | 18,450 / 4,826 | 19,808 / 5,069 | 19,624 / 4,996 | |
| Cart or checkout HTML | checkout 22,096 / 6,378 | cart 15,991 / 4,454 | cart 16,953 / 4,669, checkout 18,354 / 5,080 | |
Server entry.mjs | 234,144 / 58,478 | 236,112 / 58,815 | 235,210 / 58,617 | 237,813 / 58,831, plus a checkout chunk of 11,142 / gzip 3,974 |
Five cache-disabled loads of /store/ on the local static server, median of the navigation timing entry:
| Kimi | Sonnet | Grok | |
|---|---|---|---|
| Document transfer | 18,750 bytes | 20,108 bytes | 19,924 bytes |
| DOMContentLoaded | 18 ms | 18 ms | 17 ms |
| Load event | 34 ms | 36 ms | 32 ms |
| All resources, median | 138,118 bytes | 139,600 bytes | 152,648 bytes |
| Resource count, median | 4 | 4 | 5 |
DOMContentLoaded moved by a few milliseconds. On localhost, with no throttle, that is noise. I will not call a winner from it. The stable difference is that Grok loads one more script, the cart, about 4.6 KB gzipped. The harness also waited out /_vercel/insights/script.js 404s. That script is the existing analytics snippet, and it 404s off the store too when you are not on Vercel. It is not a store defect.
One method note, so I do not fake a bundle comparison. When the review symlinked node_modules, the Vercel file tracer skipped a large sharp copy. When node_modules was a real directory, that copy appeared, about 39 MB, inside the function folder. I am not using that gap as a store result. The entry.mjs sizes above are the comparable server files.
What the bill shows, and what it does not
The agent transcripts had no usage or cost fields. That was a failure to instrument the run, not proof that Cursor kept no record. Cursor documents where the records live after a run. The Usage dashboard exports on-demand dollars per model and request. Staff on the Cursor forum describe that table, the CSV, and the difference between included use and an incremental bill. The Cloud Agents API documents GET /v1/agents/{id}/usage. The Python SDK documents billed tokens and an optional charged amount. None of those was collected while the three agents ran. A later signed-in dashboard CSV was. This section is that recovery, not a meter that was on at the time.
A US$20 credit top-up funded the account. It is not the cost of these runs. Included usage is not an incremental charge, and it is not free.
The window is 7 November 2026, from 17:31:00 through 18:12:59.999 Australia/Melbourne (AEDT). Rows are grouped by product and exact model. There is no agent id, so a busy model in that window is only a candidate for an arm. The last billed event sits a few seconds after the last timed tool: about 12.7 seconds for Sonnet, 4.9 for Kimi, and 11.1 for Grok. That offset is consistent with a final reply or with billing lag. It does not prove that nothing else was running.
Billed tokens are four disjoint buckets: ordinary input, cache write, cache read, and output. Their sum is a bucket sum. It is not the length of this article, and it is not one unique prompt. Almost all of the volume is cache read. The same context is counted again as the agent continues.
| Group | Events | Ordinary input | Cache write | Cache read | Output | Bucket sum | Recorded charge |
|---|---|---|---|---|---|---|---|
| Claude 4.6 Sonnet, medium thinking, on demand | 43 | 45 | 50,773 | 2,155,393 | 15,290 | 2,221,501 | US$1.06650165 |
| Kimi K2.7 Code, on demand | 46 | 46,486 | 0 | 2,375,680 | 13,881 | 2,436,047 | US$0.55106490 |
| Grok 4.7 high, fast, included | 51 | 348,352 | 0 | 6,441,856 | 95,891 | 6,886,099 | no incremental charge |
| Claude 4.5 Sonnet delegate, included | 65 | 443 | 119,174 | 3,351,260 | 7,229 | 3,478,106 | no incremental charge |
| Gemini 3.1 Pro delegate, included | 1 | 14,060 | 0 | 0 | 1,543 | 15,603 | no incremental charge |
Sonnet plus Kimi recorded charges are US$1.61756655. Those figures are the CSV charges. They are not the US$20 top-up.
Grok’s primary model, plus the Claude 4.5 and Gemini events that fit the later browser and media work, is 10,379,808 bucket-sum tokens. The five groups in the table are 206 events and 15,037,356 tokens, with that same US$1.61756655 of recorded on-demand charge. Included rows add no dollars on the invoice line. The allowance absorbed them. That is not the same thing as the work having no economic cost.
Two nearby rows stay out. One Cursor grok-bot-default event, 8,950 tokens and US$0.00325260, is ambiguous, so it is not given to an arm. A separate Grok Bot product, 22 included events and 1,309,458 tokens, has a blank model. I do not price that as Grok 4.7, and I do not add it to the experiment. The literal Cursor total inside the window is 207 events, 15,046,306 tokens, and US$1.62081915. That is the window, not these three runs. Eight later Grok events, starting 18:13:27, are 381,954 tokens and are excluded. Their timing falls after the first-pass window and is consistent with later review work. That does not identify each later event.
On-demand charges can be rebuilt from Cursor’s published list rates on the models and pricing page. The rates used here are US dollars per million tokens: ordinary input, cache read, output, and cache write where the page lists one. Kimi’s cache-write cell is a dash. These Kimi rows have zero cache-write tokens, so the dash is not pricing anything.
For each request, with the four buckets kept separate:
charge = (ordinary_input * input_rate
+ cache_write * write_rate
+ cache_read * read_rate
+ output * output_rate) / 1_000_000Kimi’s published rates are US$0.95, US$0.19, and US$4.00. The group above is:
(46,486 * 0.95 + 2,375,680 * 0.19 + 13,881 * 4) / 1_000_000
= 0.55106490That equals the recorded charge. The same formula matches every one of the 43 Sonnet events and all 46 Kimi events, with no rounding gap. Sonnet’s rates on that page are US$3.00 input, US$3.75 cache write, US$0.30 cache read, and US$15.00 output. Sonnet’s ordinary input is 45 tokens beside 2,155,393 cache-read tokens, which is why the export’s prompt count cannot be the whole input.
Grok’s included rows have no charge to match. Fast list rates are US$4 input, US$1 cache read, and US$12 output. Applied to the primary group, the list-equivalent is about US$8.99. That is a published-price estimate, not money the CSV added. Claude 4.5 at its list rates is about US$1.56, and the single Gemini 3.1 Pro event is about US$0.05. With primary Grok, the list-equivalent is about US$10.59. I will not add that estimate to US$1.61756655 and call the sum what was paid.
The text buckets support the lower Fast tier. The largest non-output text total on a Grok event in the window is 186,849, and none exceeds 256,000. Cursor also lists a combined Fast and long-context tier. If every Grok event had been priced there instead, the list-equivalent including the same delegates would be about US$15. That is a sensitivity check. It is not a measured bill, not a confidence interval, and not a hard bound. The export does not include the exact context setting.
I am not publishing the raw CSV. It identifies the account. The public trail is the dashboard, the pricing page, and the formula above.
Do not rank the three by those token sums. Most of the tokens are cache reads of context that was sent again, not a longer program. Kimi and Sonnet incurred small on-demand charges. Grok’s usage was included; its estimated list-equivalent was higher. These are not directly comparable cash costs. Grok’s closer flow sat on included usage, took much longer on the tool clock, and pulled in two other models. A raw token ranking would hide all of that. The broken stubs are Kimi’s and Sonnet’s.
What I would change in a team that builds with agents
This section is recommendation, not a new measurement. One store does not tell me the return on “AI-first” development, and I will not invent one.
Write the acceptance test as the journey, not as the build. Measured: every npm run build exited 0, and two of the three paths a shopper would take did not. The useful sentence is “from the listing, add the digital item, open checkout on the built HTML, and see a refusal that does not charge.” A green build is a weak version of that sentence.
Separate the author’s tests from an independent check. Measured: Grok’s 11 store tests passed, and a separate HTTP call agreed on the 501 and the ignored price. Kimi’s typecheck stopped at an installer prompt. The sitemap Date error was already on main. I would want astro check, plus a click-through of the built pages, done by someone who did not write the patch. The agent’s own tests are evidence. They are not the gate.
Decide who is allowed to name the price. Measured: Grok’s route ignores a client price and returns 501. None of the three reads a payment secret. The recommendation is to keep that split if you ever charge: the browser may remember a cart, the server owns the amount, and a stub has no success path. Sonnet’s disabled form is the right instinct for a page that must not post a card, once that form is actually in the HTML.
Count rework and review, not only tokens. Measured charges and envelopes are above. The recommendation is to keep recorded charges in a different column from list-price estimates, and to add the cost of fixing a path that looked done. Kimi and Sonnet incurred small on-demand charges. Grok’s usage was included; its estimated list-equivalent was higher. These are not directly comparable cash costs. Both Kimi and Sonnet still need the stub repaired. Grok’s included usage and the Claude and Gemini delegates are part of what that closer result used. Included is not free. The US$20 top-up is funding, not this experiment.
Keep provenance when a run calls another model. Measured: the Grok window includes Claude 4.5 Sonnet and Gemini 3.1 Pro, and the tool envelope includes browser and video work. The recommendation is to record those delegates before anyone compares vendors. A pull request titled with one model can be three models’ work.
Pilot the journey you care about before you procure. This was one brief, one sample, cloud agents rather than the local IDE. I would repeat that shape on a task the team actually ships, with run ids in the billing export, before I treated any of these three as a house model. I would not sign a contract off a store page that still has a hole.
Before I would merge any of them
I would not merge these three as they stand. I also would not throw them away.
I would keep Grok’s rule that the server re-prices and then refuses, and I would keep integer cents. I would want Sonnet’s disabled controls on any form that is not live, once the form is actually in the HTML. I would want Kimi’s small localStorage parser, once the script that uses it is bundled the way the detail page already is.
Before I adopted a patch, I would check these myself, or pay for a check that does:
- Open the built HTML, not only
astro dev, and complete the path the button promises. A query string on a static page, anddefine:varsleaving TypeScript in the HTML, are the two traps this run hit. - Run
astro checkand separate new errors from the sitemapDateerror already onmain. If the check becomes a gate, fix that base error once, in one pull request, not three times inside store experiments. - Confirm no environment name is read, and that a client-supplied price cannot change the total. Grok’s route already demonstrates the shape.
- Look at 390 pixels and at a keyboard, including the empty state. axe caught two issues on Kimi that the build did not.
- Keep recorded charges and list-price estimates in different columns. Here the probable on-demand charges are US$1.06650165 for Sonnet and US$0.55106490 for Kimi. Grok’s primary work was included: no incremental charge, list-equivalent about US$8.99, or about US$10.59 with the probable delegates.
None of the placeholder products is a real offer, and none of this is a vendor selection. One sample cannot tell me that Kimi always misses a bundler edge, or that Grok always draws the trust boundary in the right place. It can tell me that a passing astro build did not tell me whether a person could finish the path.
Appendix: how the check was run
The review used Node v22.23.4, npm 10.9.9, Astro 7.2.2, and system Chrome. Each head was a detached worktree. npm ci --ignore-scripts installed Kimi’s lockfile (identical to the base, 611 packages) and Sonnet’s (684 packages, the extra ones being the unused checker). Grok’s lockfile matches the base.
Commands that matter:
npx astro checkon the base and on all three heads, using@astrojs/check0.9.12 and TypeScript 6.0.3 so the checker version did not move.npm run test:feed,npm run test:contact, and on Groknpm run test:store.npm run build, three times each, output deleted between runs.- Built files served with Python’s static server from
.vercel/output/static, becauseastro previewexits immediately under the Vercel adapter. astro devonly where a request-time route or a dev-versus-build comparison was needed. Static query strings were empty there too.- Puppeteer Core driving Chrome, headless, viewports 1280 by 900 and 390 by 844. Screenshots in this article are those captures, re-encoded as WebP at quality 90. Pixel sizes are the original shots.
- axe-core 4.10.3 as described above. No Lighthouse run, so there is no Lighthouse score in this article.
The 404 for an unknown product id means the HTML file was not emitted. The static server does not show the site’s designed 404 page. That is enough to show the path was not prerendered.
Usage came later, from a signed-in Usage CSV, not from the transcripts. The window is 17:31:00 through 18:12:59.999 AEDT on 7 November 2026. Token strings were summed as four disjoint buckets. Charges were parsed as exact decimals, not binary floats. All 89 on-demand Sonnet and Kimi events match the list-rate formula individually. Included models use the same bucket reading as a schema inference. They have no charge line to prove it against. Estimates use the public rates linked above and are labelled as estimates. The raw CSV is not in this repository.
I am not quoting session logs, tool transcripts, or private notes. The excerpts above are the first-pass code, named by file. The screenshots are the pages those commits built. I am not publishing the repository links or the commit ids.
Key learnings
These are recommendations. The measurements they rest on are in the results and, in full, in the evidence.
Test the journey, not the build. Every production build passed. Two of the three paths a shopper would take did not. The sentence worth writing down is: from the listing, add the item, open the built checkout, and see a refusal that does not charge.
A static page does not see the query string. Sonnet’s add-to-cart link put the product in the address, and the HTML had already been baked without it. The same limit left Grok’s no-JavaScript confirmation blank. If the page is prerendered, read the query in the browser, or do not pretend the server saw it.
An inlined script is not a bundled module. Kimi’s detail page bundled a real cart. The checkout page inlined import and a TypeScript assertion, and the browser threw. The cart was saved. The page could not read it.
The server should name the price, then be allowed to refuse. Grok’s checkout ignored a client price of one cent and returned 501 with the catalogue total. That is the split I would keep if this ever charged: the browser may remember a cart, and a stub has no success path. None of the three read a payment secret.
Do not rank models by tokens or by the clock. Kimi and Sonnet incurred small on-demand charges. Grok’s usage was included; its estimated list-equivalent was higher. These are not directly comparable cash costs. The rounded on-demand figures are about US$0.55 and US$1.07. Grok’s list-equivalent is about US$8.99, or about US$10.59 with the models it called. Grok also had a much longer tool envelope, and a closer flow that still has a hole. Most of the tokens are cache reads, not a longer program. Included use is not free. The US$20 top-up is funding.
Write down which models actually ran. Grok’s result included Claude for the browser work and Gemini for a media review. A comparison that names only the model on the brief is incomplete.
Conclusion
Nobody won. All three shops have a broken path. Grok is the closest to a working flow, and it is also the broadest design, the longest tool clock, and a run that called other models. Kimi and Sonnet incurred small on-demand charges. Grok’s usage was included; its estimated list-equivalent was higher. These are not directly comparable cash costs. Kimi’s and Sonnet’s stubs do not survive the built page.
One store, one sample, cloud agents. That is enough to stop trusting a green build. It is not enough to pick a house model, and it is not a productivity figure for a company.
If you want notes in this vein, the list is subscribe.