d2_analytics_console

where the tokens actually went

Cumulative tokens against each run's own clock, from the per-turn usage in trace.jsonl. A flat stretch is one turn the model spent thinking and writing; a staircase is many short ones. Reconciles exactly with the run totals below.

arm Aarm A · nonearm Barm B · nonearm Iarm I · none

what each arm spent its tool calls on

Every tool call the run made, collapsed to five jobs and drawn on a shared scale — the bar lengths are counts, not proportions. An arm with no registry access has nothing to look up and nothing to materialise, so those segments are absent rather than short.

explore the repolook up the registrymaterialisewrite a whole fileedit in place

where the lines in the repo came from

By file, on a shared scale. Counted conservatively: a page is scored as generated in full even where a splice supplied most of its lines, so the copied share is an understatement rather than a flattering one.

the model generatedcopied verbatim, usedcopied verbatim, never imported

cost, tokens, time, tool calls

Everything here is the run's own record. Wall clock is dominated by waiting on the model in all three, so tool-heavy arms do not pay for their tools in time — they pay in turns, and turns are tokens.

	arm Ad2_A_s1	arm A · noned2_A_none_s1	arm Bd2_B_s1	arm B · noned2_B_none_s1	arm Id2_I_s1	arm I · noned2_I_none_s1
wall clock	356.1s	119.8s	533.7s	194.7s	414.1s	195.7s
waiting on the modelthe rest is tool execution	355.2s	119.3s	532.6s	194.2s	409.3s	190.3s
running tools	0.84s	0.39s	0.95s	0.43s	4.62s	5.27s
turns	16	14	20	32	30	42
tool calls	23	18	62	47	86	85
malformed tool calls	0	0	0	0	0	0
tokens, total	656,249	185,531	1,335,475	477,314	1,374,779	1,347,812
cache-hit inputpriced 50x below a cache miss	592,000	155,008	1,195,264	427,648	1,271,168	1,291,008
cache-miss input	10,840	7,092	55,220	18,271	44,224	33,279
output	53,409	23,431	84,991	31,395	59,387	23,525
of which reasoning	38,361	0	59,171	0	40,601	0
cost	$0.0181	$0.0080	$0.0349	$0.0125	$0.0264	$0.0149

does it work

The harness's own scorers. All three build and pass every behavioural check, which is the point: correctness screens do not separate these runs.

	arm Ad2_A_s1	arm A · noned2_A_none_s1	arm Bd2_B_s1	arm B · noned2_B_none_s1	arm Id2_I_s1	arm I · noned2_I_none_s1
typechecktsc --noEmit, zero errors on all three	yes	yes	yes	no	yes	yes
production build	yes	yes	yes	no	yes	yes
behavioural checks	20/20	15/20	17/20	4/20	20/20	20/20
axe — serious + criticalthe harness's counted metric	0	0	0	0	0	0
axe — every impact levelmoderate findings are excluded from the count above	none	none	none	none	none	none
composition analysis	n/a — no registry components	n/a — no registry components	3 high orphaned_trigger · dialog app/console/alerts/page.tsx:118 · controlled by dialogOpen	n/a — no registry components	clean	clean

what the code is like

Computed by scripts/code_quality.py over each run's workspace on disk. These are facts with a file and a line behind them, not judgements — the judgements are in FINDINGS.md.

	arm Ad2_A_s1	arm A · noned2_A_none_s1	arm Bd2_B_s1	arm B · noned2_B_none_s1	arm Id2_I_s1	arm I · noned2_I_none_s1
source files	15	12	25	22	30	27
total lines	1,373	1,313	2,476	2,108	2,424	2,281
the page itselfeverything the task actually asked for	639	1,117	803	991	549	744
primitives	0	0	1,021	809	957	957
largest single fileone file is not a defect; it is a review surface	293	327	338	336	305	308
primitives	none — all bespoke markup	none — all bespoke markup	10 hand-writtenbutton, card, checkbox, dialog, dropdown-menu, input, label, select, table, tabs	10 hand-writtenbutton, card, checkbox, dialog, dropdown-menu, input, label, select, table, tabs	10 installedbutton, card, checkbox, dialog, dropdown-menu, input, label, select, table, tabs	10 installedbutton, card, checkbox, dialog, dropdown-menu, input, label, select, table, tabs
library generationthe registry these runs had docs for is uniformly v4	—	—	all currentevery data-slot matches the registry	10 pre-v4button, card, checkbox, dialog, dropdown-menu, input, label, select, table, tabs · 63 data-slot attributes missing	all currentevery data-slot matches the registry	all currentevery data-slot matches the registry
imported by nothing	none	none	none	none	2 files · 122 loccomponents/blocks/dialog-example.tsx, components/blocks/dropdown-menu-example.tsx	none
harness authored_locwhat the corpus reports	1,474	1,378	2,642	2,251	1,540	1,393
of which verbatim registry bytes	none	none	none	none	404 loc (26%)pasted by add_example, counted as authored — INSTRUMENT-LOG E3	271 loc (19%)pasted by add_example, counted as authored — INSTRUMENT-LOG E3
page has an h1	yes	yes	yes	yes	yes	yes
React 19 deprecationsrenamed to React.ComponentRef in React 19	none	none	none	24x React.ElementRef	none	none
any / ts-ignore / non-null assertions	0	0	0	0	0	0