Measurement 2 of 4 · the one that diverged
Cost
Same task. Same quality. A ~200× spread on the bill. When the models converge on correctness, this is the variable that's left — and it's the whole lesson of the experiment.
USD for the same deliverable, as run. The dashed tick on Fable 5 marks the ~$1 a clean one-shot API call would cost — most of the as-run figure is Claude Code harness cache traffic. Grok ran on a subscription and exposes no per-token price, so it gets no bar rather than an invented one.
The receipts, per model
only what the data supports gets a numberThe honest ranking
| Model | Cost | Basis |
|---|
Where the numbers come from
Every one of these CLIs quietly logs its model, tokens and cost on your own machine — ~/.claude/projects, ~/.grok, ~/.gemini, and opencode's SQLite DB. That's where this comparison data was pulled from. No benchmark to trust; the receipts are local.
The takeaway: stop optimizing the model, optimize the bill. "Cheap" didn't mean "worse" — the two cheapest runs matched the premium one on the measurement that matters.