diff --git a/.flatbread-proof/findings/fnd-a-state-read-adds-to-search-on-counts-blockers-a--hy9fx5hrw0gbcwzj.md b/.flatbread-proof/findings/fnd-a-state-read-adds-to-search-on-counts-blockers-a--hy9fx5hrw0gbcwzj.md new file mode 100644 index 00000000..95830a0a --- /dev/null +++ b/.flatbread-proof/findings/fnd-a-state-read-adds-to-search-on-counts-blockers-a--hy9fx5hrw0gbcwzj.md @@ -0,0 +1,17 @@ +--- +id: fnd-a-state-read-adds-to-search-on-counts-blockers-a--hy9fx5hrw0gbcwzj +effort: eff-effort-graph-memory-and-agent-wedge--szeqvmgqjqnhd002 +title: 'A state read adds to search on counts, blockers and status history' +kind: measurement +created_at: '2026-10-19T12:41:23.560Z' +derives_from: + - fnd-search-cards-raise-accuracy-and-cut-reading-on-t--ydqm2ybyqjedzw9y +--- + +`proof state ` prints what search cannot: exact counts, open blockers and what they gate, contested topics, Issues closed without a fix, dead evidence and dated changes. On the benchmark graph ten times larger, Haiku answered the 97 count, status-history, blocker and dead-evidence questions twice with `find` plus `state` and twice with `find` alone. + +- Accuracy rose by about 5 questions per run: 181 against 171 of 194 question-runs. Over both runs, only `find` plus `state` was right 15 times and only `find` 5 times (sign test p = 0.041). +- It read 18 percent fewer tokens (15,578 against 18,907) and made 9 tool calls instead of 12. +- On the half of the questions `state` was not shaped on, it led 83 to 79 of 88 (5 to 1). That points the same way but is too small to show on its own. + +Offline, `state` alone held the answer for 55 percent of these questions on that half, below its 60 percent bar. It earns its place by covering what `find` misses, not by answering alone. The plan's mark (at least 4 more per run, no rise in tokens) is met. diff --git a/.flatbread-proof/findings/fnd-on-two-real-graphs-search-still-wins-and-the-sta--9qssh2p2w9tevfpw.md b/.flatbread-proof/findings/fnd-on-two-real-graphs-search-still-wins-and-the-sta--9qssh2p2w9tevfpw.md new file mode 100644 index 00000000..80e884c2 --- /dev/null +++ b/.flatbread-proof/findings/fnd-on-two-real-graphs-search-still-wins-and-the-sta--9qssh2p2w9tevfpw.md @@ -0,0 +1,17 @@ +--- +id: fnd-on-two-real-graphs-search-still-wins-and-the-sta--9qssh2p2w9tevfpw +effort: eff-effort-graph-memory-and-agent-wedge--szeqvmgqjqnhd002 +title: On two real graphs search still wins and the state read is neutral +kind: measurement +created_at: '2026-10-19T12:54:53.992Z' +derives_from: + - fnd-a-state-read-adds-to-search-on-counts-blockers-a--hy9fx5hrw0gbcwzj + - fnd-search-cards-raise-accuracy-and-cut-reading-on-t--ydqm2ybyqjedzw9y +--- + +Two real Proof graphs from private projects (200 and 134 records) served as a held-out check that nothing was tuned on. An author blind to `find` and `state` output wrote 39 and 41 machine-checked questions; they stay outside the repository. Haiku answered each set twice with no-brief, `find`, and `find` plus `state`. + +- `find` beat no-brief on both graphs: 71.7 against 66.3 of 78, and 75 against 71 of 82. Over both, it alone was right 13 times and no-brief alone 3 times (sign test p = 0.021). It read about half the tokens. +- Adding `state` changed nothing measurable: 1 to 2 and 3 to 4 against `find`, at the same tokens. + +So the search gain holds on real memory. The `state` gain seen on the 10x graph's counts and blockers does not appear on graphs a twenty-fifth that size or less, where paging records is cheap; `state` neither helps nor costs there. diff --git a/.flatbread-proof/findings/fnd-search-keeps-its-gain-on-look-alike-filler-and-s--rtg5zjs9rndhqe5v.md b/.flatbread-proof/findings/fnd-search-keeps-its-gain-on-look-alike-filler-and-s--rtg5zjs9rndhqe5v.md new file mode 100644 index 00000000..9ae90d6d --- /dev/null +++ b/.flatbread-proof/findings/fnd-search-keeps-its-gain-on-look-alike-filler-and-s--rtg5zjs9rndhqe5v.md @@ -0,0 +1,16 @@ +--- +id: fnd-search-keeps-its-gain-on-look-alike-filler-and-s--rtg5zjs9rndhqe5v +effort: eff-effort-graph-memory-and-agent-wedge--szeqvmgqjqnhd002 +title: Search keeps its gain on look-alike filler and saves reading on Sonnet +kind: measurement +created_at: '2026-10-19T12:33:23.668Z' +derives_from: + - fnd-search-cards-raise-accuracy-and-cut-reading-on-t--ydqm2ybyqjedzw9y +--- + +Two more single runs of `proof find` against no-brief on the benchmark graph ten times larger, 245 comparable questions each. + +- **Look-alike filler (Haiku).** With 30 percent of filler records reusing real subsystem names, `find` got 233.7 right and no-brief 214.7: a gap of 19, the same as on the plain graph (19 to 1 on the questions only one got right, p < 0.0001). `find` read 0.38 of no-brief's tokens. So the earlier gain was not an artifact of filler that is easy to skip. +- **Sonnet.** `find` got 233.4 and no-brief 226.7, a gap of 6.8 that one run cannot separate from noise (11 to 5, p = 0.21). `find` still read 44 percent fewer tokens (9,280 against 16,674) and made 7 tool calls instead of 18. On 24 questions that name no Effort, `find` got 24 and no-brief 17. + +A stronger model crawls the graph well enough without search, so it gains less accuracy and keeps most of the token saving. Search helps every model most when the question does not say which Effort it belongs to. diff --git a/benchmarks/proof-recall/README.md b/benchmarks/proof-recall/README.md index c3380d8f..f75e5e51 100644 --- a/benchmarks/proof-recall/README.md +++ b/benchmarks/proof-recall/README.md @@ -872,6 +872,52 @@ What this does not show: finds the right Effort, not that agents handle a hard question without a named Effort. +### Look-alike filler, and Sonnet + +Two more single runs of `find` against no-brief, with the same task texts +and `find_hash` as above, on 270 questions each (the 246 and the 24 +ambiguous-Effort ones). Figures are for the 245 comparable questions. + +**Run C: Haiku on the look-alike graph** (`--scale 10 --filler-overlap 0.3`, +where 30% of filler records reuse real subsystem names). Run files: +`runs/x10adv-find-r1-*.json`. + +| Measure | `find` | no-brief | +| --------------------------------- | ------ | -------- | +| Correct | 233.7 | 214.7 | +| Questions only this arm got right | 19 | 1 | +| Mean read tokens | 13,835 | 36,891 | +| Mean tool calls | 9 | 22 | +| "Unknown" answers | 18 | 33 | + +The gap is 19 questions (p < 0.0001), the same as on the plain graph +(about 19 per run), and `find` read 0.38 of no-brief's tokens. The plan's +pass mark for Run C was half the Run A gain and at most 0.75 of the +tokens, so it passes easily: filler that looks real did not hurt search. +On the ambiguous set `find` got 22 of 24 and no-brief 15 (7 to 0, +p = 0.016). + +**Sonnet on the plain 10x graph.** Run files: `runs/x10-sonnet-r1-*.json`. + +| Measure | `find` | no-brief | +| --------------------------------- | ------ | -------- | +| Correct | 233.4 | 226.7 | +| Questions only this arm got right | 11 | 5 | +| Mean read tokens | 9,280 | 16,674 | +| Mean tool calls | 7 | 18 | +| "Unknown" answers | 16 | 27 | + +Sonnet without `find` does far better than Haiku without it (226.7 +against about 211), so the accuracy gap shrinks to 6.8 questions and is +not significant in one run (p = 0.21). `find` still read 44% fewer tokens +and made fewer than half the calls. On the ambiguous set, where the +question names no Effort, the gap stays large: `find` 24 of 24, no-brief +17 (7 to 0, p = 0.016). + +So a stronger model gains less accuracy from search and keeps most of the +saving; a weaker one gains both. Timeouts (10 minutes) hit 3 Haiku and 5 +Sonnet sessions, all but one in the no-brief arm. + ### The offline gate for `proof state` `proof state ` is one read, built from frontmatter and links, @@ -965,5 +1011,61 @@ The find gate, rerun with the same `find_hash` (`e5be6532…`), still gives 81% evidence in the top 5 and 62% answers on a card on the stripped set. The paid Run D compares `findstate` against `find` on these questions. -Before paying for it, decide whether a read that fails its own bar is -worth a paid run. +It was run anyway, because `find` and `state` together cover more than +either alone; see below. + +### `find` plus `state` against `find` (Run D) + +Haiku on the 10x graph, twice, on the 97 questions in the aggregate, +dead-evidence, lifecycle and blocker categories (all but `world-077`): +`run.mjs --strategies findstate,find --ids `. The `findstate` +arm adds one line to the `find` task text: use `proof state ` for +counts, open and closed sets, status history, blockers and dead evidence. +Run files: `runs/x10-rund-r{1,2}-*.json`. + +Pooled over both runs (194 question-runs per arm): + +| Measure | `findstate` | `find` | +| ------------------------------------ | ----------- | ------ | +| Correct | 181 | 171 | +| Runs of a question only this arm got | 15 | 5 | +| Mean read tokens | 15,578 | 18,907 | +| Mean tool calls | 9 | 12 | + +The pooled sign test gives p = 0.041; counting each question once, 9 to 4 +(p = 0.27). The plan kept `state` only if it gained at least 4 questions +per run on this subset with no rise in tokens. It gained 5 per run and read +18% fewer tokens, so it passes. + +`state` was shaped on the tune questions, so the holdout half is the clean +measure: there `findstate` got 83 and `find` 79 of 88 (5 to 1, p = 0.22), +against 98 and 92 of 106 on tune (10 to 4). The direction holds on +holdout, but two runs on 44 questions cannot show it alone. Two sessions +timed out, both in the `find` arm. + +### Two private project graphs, as a held-out check + +Two real Proof graphs from private projects (200 records across 11 Efforts, +and 134 across 6) tested whether the gains above hold on memory nobody tuned +against. An author who never saw `find` or `state` output wrote 39 and 41 +machine-checked questions about them. The questions, answers, graph copies +and run files live outside this repository (see "Question sets kept outside +the repository"); only these totals are published. Before any paid run, +only category totals of the offline gates were read, never a card. + +Haiku, two runs per graph, three arms each, pooled: + +| Graph | `find` | no-brief | `findstate` | Read tokens, `find` / no-brief | +| ----- | ---------- | -------- | ----------- | ------------------------------ | +| A | 71.7 of 78 | 66.3 | 71.3 | 12,488 / 27,975 | +| B | 75 of 82 | 71 | 74 | 12,230 / 24,389 | + +- `find` against no-brief: 7 to 1 on A and 6 to 2 on B; 13 to 3 over both + (sign test p = 0.021). `find` read about half the tokens on both. +- `findstate` against `find`: 1 to 2 on A and 3 to 4 on B, with the same + tokens. On these small graphs `state` neither helps nor costs. + +So the search gain carries over to real memory at about half the reading. +The gain from `state` on the 10x graph's counts and blockers does not show +on graphs a twenty-fifth +that size or less, where paging `records` is cheap. diff --git a/benchmarks/proof-recall/runs/x10-rund-r1-find.json b/benchmarks/proof-recall/runs/x10-rund-r1-find.json new file mode 100644 index 00000000..8800e3fe --- /dev/null +++ b/benchmarks/proof-recall/runs/x10-rund-r1-find.json @@ -0,0 +1,871 @@ +{ + "run": { + "name": "x10-rund-r1-find", + "protocol": "one fresh session per question", + "models": ["claude-haiku-5-5"], + "find_hash": "e5be653299abe9b2329822d98ffe09e90ee7cfe96f8711c813a51dc6500b887f", + "state_hash": "b73c82ecb0649c1db200b189b827c76250569b75327ba5847d9939352dcf1679" + }, + "answers": { + "cross-002": { + "answer": 2, + "tokens": 11859, + "fixed_tokens": 1100, + "processed_tokens": 79858, + "tool_calls": 8, + "turns": 7 + }, + "cross-003": { + "answer": 2, + "tokens": 5341, + "fixed_tokens": 1104, + "processed_tokens": 39672, + "tool_calls": 4, + "turns": 5 + }, + "cross-004": { + "answer": "dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e", + "tokens": 9742, + "fixed_tokens": 1102, + "processed_tokens": 87650, + "tool_calls": 7, + "turns": 8 + }, + "cross-014": { + "answer": 3, + "tokens": 4749, + "fixed_tokens": 1100, + "processed_tokens": 39266, + "tool_calls": 4, + "turns": 5 + }, + "cross-015": { + "answer": "Two Efforts are no longer active. The shader compile cache and build farm Effort (eff-shader-compile-cache-and-build-farm--6xeac11zxvwwarkn) is completed: its end condition was met when the cache was turned on for every farm build (dec-turn-the-shader-compile-cache-on-for-every-farm--4rr3b12b4sasdmaj), as the retrospective fnd-retrospective-on-the-shader-compile-cache--9rpv2a60ach04cx6 confirms. The co-op rollback netcode prototype Effort (eff-co-op-rollback-netcode-prototype--n5ejm416kfcfap49) is abandoned: rollback was dropped by dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b, because re-running tide simulation on every rolled-back frame cost 4.8 ms on Skiff.", + "tokens": 38611, + "fixed_tokens": 1098, + "processed_tokens": 243381, + "tool_calls": 13, + "turns": 10 + }, + "netcode-001": { + "answer": "abandoned", + "tokens": 7148, + "fixed_tokens": 1099, + "processed_tokens": 57553, + "tool_calls": 5, + "turns": 6 + }, + "netcode-010": { + "answer": "dec-use-a-6-frame-rollback-window-with-1-frame-of-in--s53944zhkw103frk", + "tokens": 13573, + "fixed_tokens": 1101, + "processed_tokens": 111935, + "tool_calls": 9, + "turns": 9 + }, + "netcode-013": { + "answer": [ + "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "fnd-rollback-re-simulates-a-4-frame-window-in-1-1-ms--xa30a1cez2brphyf", + "rsk-re-simulation-cost-could-blow-the-skiff-frame-bu--sk2njsp170a3qwyf" + ], + "tokens": 21371, + "fixed_tokens": 1098, + "processed_tokens": 174154, + "tool_calls": 14, + "turns": 11 + }, + "netcode-014": { + "answer": [ + "iss-late-join-during-a-tide-surge-desyncs-the-joinin--mya93va0n7s8g7cj" + ], + "tokens": 42740, + "fixed_tokens": 1104, + "processed_tokens": 490928, + "tool_calls": 17, + "turns": 18 + }, + "netcode-015": { + "answer": 2, + "tokens": 21542, + "fixed_tokens": 1097, + "processed_tokens": 131065, + "tool_calls": 9, + "turns": 10 + }, + "ports-004": { + "answer": "accepted", + "tokens": 7565, + "fixed_tokens": 1099, + "processed_tokens": 58295, + "tool_calls": 5, + "turns": 6 + }, + "ports-005": { + "answer": "unknown", + "tokens": 26078, + "fixed_tokens": 1097, + "processed_tokens": 184185, + "tool_calls": 10, + "turns": 10 + }, + "ports-017": { + "answer": "fnd-skiff-microsd-slows-sharply-on-full-warm-cards--p1rgh36hat2egpz3", + "tokens": 8211, + "fixed_tokens": 1097, + "processed_tokens": 60091, + "tool_calls": 5, + "turns": 6 + }, + "ports-019": { + "answer": "4 MB per minute", + "tokens": 8720, + "fixed_tokens": 1097, + "processed_tokens": 84122, + "tool_calls": 7, + "turns": 8 + }, + "ports-021": { + "answer": "SKF-DSP-004", + "tokens": 9013, + "fixed_tokens": 1100, + "processed_tokens": 85427, + "tool_calls": 7, + "turns": 8 + }, + "ports-022": { + "answer": "cit-skiff-sdk-1-2-release-notes--7gwwn8xxffrx6npe", + "tokens": 10990, + "fixed_tokens": 1105, + "processed_tokens": 119338, + "tool_calls": 9, + "turns": 10 + }, + "ports-023": { + "answer": "fnd-tern-costs-3-9-ms-at-quality-and-2-1-ms-at-perfo--dkbskbqfgfwtryq0", + "tokens": 10618, + "fixed_tokens": 1101, + "processed_tokens": 90134, + "tool_calls": 7, + "turns": 8 + }, + "ports-026": { + "answer": "912 MB", + "tokens": 12833, + "fixed_tokens": 1096, + "processed_tokens": 155110, + "tool_calls": 11, + "turns": 12 + }, + "ports-027": { + "answer": [ + "dec-stream-skiff-world-tiles-in-2-mb-reads--3jz9w9gvhhc7eppw", + "rsk-tide-streaming-outruns-skiff-storage--6q8s3b383z2tyeyb" + ], + "tokens": 43573, + "fixed_tokens": 1100, + "processed_tokens": 660101, + "tool_calls": 23, + "turns": 23 + }, + "ports-028": { + "answer": 2, + "tokens": 8613, + "fixed_tokens": 1106, + "processed_tokens": 73737, + "tool_calls": 7, + "turns": 8 + }, + "ports-029": { + "answer": 8, + "tokens": 85807, + "fixed_tokens": 1095, + "processed_tokens": 2728261, + "tool_calls": 54, + "turns": 55 + }, + "ports-030": { + "answer": [ + "dec-cap-tide-storm-particles-at-6-000-on-skiff--resze52afjxyxys1", + "dec-drop-skiff-to-30-hz-while-a-tide-storm-is-on-scr--72vx1c8s4jrs6axc" + ], + "tokens": 10398, + "fixed_tokens": 1097, + "processed_tokens": 85447, + "tool_calls": 9, + "turns": 8 + }, + "ports-031": { + "answer": [ + "iss-skiff-cert-docked-30-hz-judder-on-some-tvs--9nq2xg9mzash474g", + "iss-meridian-cert-activity-cards-missing-for-two-bio--k214cmfra7pyvvzq" + ], + "tokens": 21295, + "fixed_tokens": 1098, + "processed_tokens": 150041, + "tool_calls": 11, + "turns": 10 + }, + "ports-032": { + "answer": 2, + "tokens": 6315, + "fixed_tokens": 1104, + "processed_tokens": 51050, + "tool_calls": 5, + "turns": 6 + }, + "render-005": { + "answer": true, + "tokens": 7782, + "fixed_tokens": 1094, + "processed_tokens": 58369, + "tool_calls": 5, + "turns": 6 + }, + "render-012": { + "answer": "deferred", + "tokens": 9941, + "fixed_tokens": 1101, + "processed_tokens": 88082, + "tool_calls": 7, + "turns": 8 + }, + "render-023": { + "answer": "dec-capture-the-cliffs-and-dunes-benchmarks-on-every--3pqyr66yh4v6q676", + "tokens": 7138, + "fixed_tokens": 1100, + "processed_tokens": 56636, + "tool_calls": 5, + "turns": 6 + }, + "render-024": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 7882, + "fixed_tokens": 1099, + "processed_tokens": 59399, + "tool_calls": 5, + "turns": 6 + }, + "render-029": { + "answer": "fnd-vsm-page-pool-overflows-during-spring-tides--m2p8e89hhdmh0z8a", + "tokens": 8197, + "fixed_tokens": 1095, + "processed_tokens": 60351, + "tool_calls": 5, + "turns": 6 + }, + "render-033": { + "answer": "128 MB", + "tokens": 8262, + "fixed_tokens": 1098, + "processed_tokens": 60505, + "tool_calls": 5, + "turns": 6 + }, + "render-035": { + "answer": "5912 pages", + "tokens": 9603, + "fixed_tokens": 1099, + "processed_tokens": 88246, + "tool_calls": 7, + "turns": 8 + }, + "render-036": { + "answer": "HDR-07", + "tokens": 7480, + "fixed_tokens": 1104, + "processed_tokens": 58094, + "tool_calls": 5, + "turns": 6 + }, + "render-041": { + "answer": "4.7 ms", + "tokens": 8143, + "fixed_tokens": 1103, + "processed_tokens": 60068, + "tool_calls": 5, + "turns": 6 + }, + "render-042": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 9844, + "fixed_tokens": 1094, + "processed_tokens": 63998, + "tool_calls": 7, + "turns": 6 + }, + "render-043": { + "answer": "fnd-meshlet-cone-culling-rejects-9-percent-of-triang--t6e5x7trzedex105", + "tokens": 7959, + "fixed_tokens": 1099, + "processed_tokens": 59355, + "tool_calls": 5, + "turns": 6 + }, + "render-048": { + "answer": "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "tokens": 20248, + "fixed_tokens": 1098, + "processed_tokens": 197352, + "tool_calls": 13, + "turns": 13 + }, + "render-058": { + "answer": "iss-which-upscaler-should-lantern-ship--g8c2y9tsr856tn2v", + "tokens": 13712, + "fixed_tokens": 1099, + "processed_tokens": 123388, + "tool_calls": 9, + "turns": 10 + }, + "render-059": { + "answer": "fnd-sdf-gi-costs-4-7-ms-on-meridian--sm19rt2f274ajsvr", + "tokens": 13885, + "fixed_tokens": 1097, + "processed_tokens": 90664, + "tool_calls": 8, + "turns": 7 + }, + "render-060": { + "answer": "0.31", + "tokens": 10366, + "fixed_tokens": 1091, + "processed_tokens": 114633, + "tool_calls": 9, + "turns": 10 + }, + "render-061": { + "answer": "con-ui-composites-at-a-fixed-203-nits-in-hdr--de6aft7cvhdezn5s", + "tokens": 11274, + "fixed_tokens": 1096, + "processed_tokens": 120545, + "tool_calls": 9, + "turns": 10 + }, + "render-063": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 10373, + "fixed_tokens": 1097, + "processed_tokens": 89513, + "tool_calls": 7, + "turns": 8 + }, + "render-065": { + "answer": "dec-ship-tern-as-lantern-s-only-upscaler--fmreq35nzyjc7k4e", + "tokens": 9536, + "fixed_tokens": 1099, + "processed_tokens": 87117, + "tool_calls": 7, + "turns": 8 + }, + "render-066": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 8524, + "fixed_tokens": 1091, + "processed_tokens": 60020, + "tool_calls": 5, + "turns": 6 + }, + "render-067": { + "answer": 4, + "tokens": 22350, + "fixed_tokens": 1092, + "processed_tokens": 142928, + "tool_calls": 8, + "turns": 9 + }, + "render-068": { + "answer": [ + "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "dec-lock-caustic-photon-jitter-to-the-tide-phase--85x10k4h11t8nx7c", + "dec-use-the-probe-grid-alone-in-120-hz-performance-m--6dzm1nt40pjfpyae", + "dec-trace-sdf-gi-at-quarter-resolution-in-performanc--06cqt3mqn3fg5h6e", + "dec-read-paper-white-from-the-meridian-system-settin--7crbx73gqw1hn7xq", + "dec-add-an-in-game-paper-white-slider-from-80-to-400--1phfvxby983z8tjf" + ], + "tokens": 25960, + "fixed_tokens": 1100, + "processed_tokens": 210993, + "tool_calls": 11, + "turns": 12 + }, + "render-069": { + "answer": "iss-shadow-pages-run-out-at-spring-low-tide--f4fgy6b44jbkrsmh", + "tokens": 18168, + "fixed_tokens": 1101, + "processed_tokens": 130451, + "tool_calls": 10, + "turns": 10 + }, + "render-070": { + "answer": ["dec-use-signed-distance-field-gi--yn366zmp3k7snd9w"], + "tokens": 53705, + "fixed_tokens": 1099, + "processed_tokens": 684871, + "tool_calls": 19, + "turns": 20 + }, + "render-071": { + "answer": 3, + "tokens": 10818, + "fixed_tokens": 1101, + "processed_tokens": 73453, + "tool_calls": 8, + "turns": 7 + }, + "render-072": { + "answer": [ + "rsk-sdf-volumes-outgrow-the-render-target-memory-bud--xfd2pfztvskdcd2c", + "rsk-artists-cannot-match-concept-paintings-under-the--6cfh9vkamn0kbtcx" + ], + "tokens": 12334, + "fixed_tokens": 1095, + "processed_tokens": 91248, + "tool_calls": 9, + "turns": 8 + }, + "render-073": { + "answer": [ + "rsk-vsm-page-pool-overflows-when-tides-expose-the-fl--x1r161nrdngra4ty", + "rsk-gi-cannot-fit-the-120-hz-performance-mode--tqt3fj839dqfq5s7", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 14555, + "fixed_tokens": 1093, + "processed_tokens": 115565, + "tool_calls": 13, + "turns": 10 + }, + "render-074": { + "answer": 5, + "tokens": 7977, + "fixed_tokens": 1094, + "processed_tokens": 54520, + "tool_calls": 5, + "turns": 6 + }, + "render-075": { + "answer": "3", + "tokens": 47777, + "fixed_tokens": 1100, + "processed_tokens": 1471949, + "tool_calls": 41, + "turns": 42 + }, + "render-076": { + "answer": false, + "tokens": 16178, + "fixed_tokens": 1093, + "processed_tokens": 159317, + "tool_calls": 11, + "turns": 12 + }, + "render-078": { + "answer": [ + "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 18957, + "fixed_tokens": 1098, + "processed_tokens": 183534, + "tool_calls": 13, + "turns": 13 + }, + "render-079": { + "answer": null, + "tokens": 108867, + "fixed_tokens": 1095, + "processed_tokens": 4118436, + "tool_calls": 63, + "turns": 63 + }, + "render-080": { + "answer": "6 versions", + "tokens": 13652, + "fixed_tokens": 1096, + "processed_tokens": 126386, + "tool_calls": 9, + "turns": 10 + }, + "render-081": { + "answer": "unknown", + "tokens": 15596, + "fixed_tokens": 1099, + "processed_tokens": 107770, + "tool_calls": 9, + "turns": 8 + }, + "render-083": { + "answer": 0, + "tokens": 34083, + "fixed_tokens": 1100, + "processed_tokens": 373163, + "tool_calls": 16, + "turns": 17 + }, + "tools-001": { + "answer": "completed", + "tokens": 8357, + "fixed_tokens": 1105, + "processed_tokens": 72714, + "tool_calls": 6, + "turns": 7 + }, + "tools-007": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 47109, + "fixed_tokens": 1099, + "processed_tokens": 848045, + "tool_calls": 23, + "turns": 23 + }, + "tools-010": { + "answer": "dec-cap-each-farm-agent-at-6-parallel-shader-compile--q5w4sfjm73tsj8x8", + "tokens": 16858, + "fixed_tokens": 1103, + "processed_tokens": 150483, + "tool_calls": 10, + "turns": 11 + }, + "tools-012": { + "answer": 3, + "tokens": 7706, + "fixed_tokens": 1096, + "processed_tokens": 71897, + "tool_calls": 7, + "turns": 8 + }, + "tools-013": { + "answer": 7, + "tokens": 198786, + "fixed_tokens": 1096, + "processed_tokens": 3695584, + "tool_calls": 39, + "turns": 40 + }, + "tools-015": { + "answer": [ + "iss-whether-the-review-bot-should-load-on-a-worker-t--rtsjwvsksjdq15v1", + "iss-whether-the-level-editor-autosave-should-be-cach--radcjs4hyvh7p55q", + "iss-whether-the-crash-symbol-server-should-load-on-a--qvyphhgnsnm2cvqb", + "iss-whether-the-license-scanner-should-be-cached-bet--nqhd89czbfat9rya", + "iss-whether-the-doc-site-should-load-on-a-worker-thr--qxswe0pg52ekky0z", + "iss-whether-the-asset-search-should-load-on-a-worker--f9gpv3z522x6p6h4", + "iss-whether-the-audio-bank-packer-should-load-on-a-w--sqzp2rwp6s01w3b0", + "iss-whether-the-level-editor-autosave-needs-its-own--rkvmnbqe7w644zkr", + "iss-whether-the-artist-sandbox-should-load-on-a-work--73j819anqct8etfn", + "iss-whether-the-changelog-bot-should-load-on-a-worke--jgtdn2f02p8fx5nr", + "iss-whether-the-review-bot-needs-its-own-memory-pool--pg3m8g9gn3y6v3kq", + "iss-whether-the-changelog-bot-should-skip-work-when--p2vrp5d682p3jrc3", + "iss-whether-the-patch-differ-should-load-on-a-worker--cysv4n56q1qn7t2k", + "iss-whether-the-artist-sandbox-should-skip-work-when--11tqzws72gms2h8x", + "iss-whether-the-audio-bank-packer-needs-its-own-memo--qjrqrseast8d5mhf", + "iss-whether-the-changelog-bot-needs-its-own-memory-p--mw8zejz6hk5m60x0", + "iss-whether-the-crash-symbol-server-should-ship-in-t--bhq91ajavah18wsk", + "iss-whether-the-asset-search-should-load-on-a-worker--80b27kqh36rpafdt", + "iss-whether-the-doc-site-should-ship-in-the-first-pa--wcdkq5zab7dxe5yj", + "iss-whether-the-patch-differ-should-be-cached-betwee--28b5zqb1ketwzf0r", + "iss-whether-the-test-runner-should-load-on-a-worker--aygay1r0qj4mzf78", + "iss-whether-the-texture-viewer-should-be-split-into--1jhgyqb70h5x7m7b", + "iss-whether-the-level-editor-autosave-should-skip-wo--fyysrcv6fvbsvcsy", + "iss-whether-the-level-editor-autosave-should-be-spli--ncnwh0csw671fs0q", + "iss-whether-the-asset-search-should-ship-in-the-firs--hsb2avhx613prxew", + "iss-whether-the-test-runner-needs-its-own-memory-poo--bs6arq0qv37d6wmx", + "iss-whether-the-artist-sandbox-should-be-split-into--ek47ca2vgmrjanp8", + "rsk-shader-cache-storage-could-outgrow-the-4-tb-farm--76y9649mwrx7aavd" + ], + "tokens": 66233, + "fixed_tokens": 1097, + "processed_tokens": 462484, + "tool_calls": 15, + "turns": 15 + }, + "tools-016": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 16744, + "fixed_tokens": 1102, + "processed_tokens": 162569, + "tool_calls": 10, + "turns": 11 + }, + "world-011": { + "answer": "fnd-shipping-entity-names-costs-57-mb-at-full-popula--6w7gjvy7wdmdx9qx", + "tokens": 9116, + "fixed_tokens": 1095, + "processed_tokens": 61560, + "tool_calls": 7, + "turns": 6 + }, + "world-013": { + "answer": "unknown", + "tokens": 7570, + "fixed_tokens": 1093, + "processed_tokens": 58222, + "tool_calls": 5, + "turns": 6 + }, + "world-016": { + "answer": "unknown", + "tokens": 7104, + "fixed_tokens": 1103, + "processed_tokens": 64847, + "tool_calls": 6, + "turns": 7 + }, + "world-020": { + "answer": "dec-schedule-streaming-reads-by-deadline-instead-of--q2x5aknabgcssdj2", + "tokens": 9135, + "fixed_tokens": 1096, + "processed_tokens": 85560, + "tool_calls": 7, + "turns": 8 + }, + "world-021": { + "answer": "dec-cut-navigation-tiles-to-64-m-so-three-fit-along--snvepk8ygeb3qsjs", + "tokens": 7676, + "fixed_tokens": 1095, + "processed_tokens": 58656, + "tool_calls": 5, + "turns": 6 + }, + "world-022": { + "answer": "fnd-41-of-alpha-crash-reports-have-no-usable-stack--35a50b07ferhf13m", + "tokens": 7443, + "fixed_tokens": 1093, + "processed_tokens": 57816, + "tool_calls": 5, + "turns": 6 + }, + "world-029": { + "answer": "dec-return-engine-jobs-to-a-thread-pool-with-continu--1zkc1v2x97taf50h", + "tokens": 10082, + "fixed_tokens": 1098, + "processed_tokens": 88177, + "tool_calls": 7, + "turns": 8 + }, + "world-034": { + "answer": "unknown", + "tokens": 13434, + "fixed_tokens": 1095, + "processed_tokens": 112503, + "tool_calls": 8, + "turns": 9 + }, + "world-037": { + "answer": "fnd-hot-reload-drops-suspended-gull-coroutines--va0fn0svryzwbzws", + "tokens": 9434, + "fixed_tokens": 1093, + "processed_tokens": 86843, + "tool_calls": 7, + "turns": 8 + }, + "world-042": { + "answer": "5.4 s", + "tokens": 8977, + "fixed_tokens": 1094, + "processed_tokens": 85652, + "tool_calls": 7, + "turns": 8 + }, + "world-049": { + "answer": "22%", + "tokens": 7984, + "fixed_tokens": 1101, + "processed_tokens": 59479, + "tool_calls": 5, + "turns": 6 + }, + "world-052": { + "answer": "Dara Whitlock", + "tokens": 8098, + "fixed_tokens": 1095, + "processed_tokens": 59666, + "tool_calls": 5, + "turns": 6 + }, + "world-053": { + "answer": "1.9 ms", + "tokens": 11041, + "fixed_tokens": 1100, + "processed_tokens": 93748, + "tool_calls": 9, + "turns": 8 + }, + "world-057": { + "answer": "(212, 87)", + "tokens": 12467, + "fixed_tokens": 1097, + "processed_tokens": 151807, + "tool_calls": 11, + "turns": 12 + }, + "world-059": { + "answer": "dec-derive-the-missing-harmonic-phase-from-each-save--gbqrvvgyq2gy2gnw", + "tokens": 9553, + "fixed_tokens": 1099, + "processed_tokens": 86935, + "tool_calls": 7, + "turns": 8 + }, + "world-060": { + "answer": "dec-partition-the-world-into-192-m-cells--bk9r1c5jwb6fh8tc", + "tokens": 9506, + "fixed_tokens": 1096, + "processed_tokens": 86960, + "tool_calls": 7, + "turns": 8 + }, + "world-061": { + "answer": "236 MB", + "tokens": 14175, + "fixed_tokens": 1094, + "processed_tokens": 158908, + "tool_calls": 11, + "turns": 12 + }, + "world-062": { + "answer": "310 MB", + "tokens": 13484, + "fixed_tokens": 1093, + "processed_tokens": 187754, + "tool_calls": 13, + "turns": 14 + }, + "world-065": { + "answer": "3.1 ms", + "tokens": 12603, + "fixed_tokens": 1101, + "processed_tokens": 151586, + "tool_calls": 11, + "turns": 12 + }, + "world-068": { + "answer": false, + "tokens": 20415, + "fixed_tokens": 1096, + "processed_tokens": 143471, + "tool_calls": 10, + "turns": 11 + }, + "world-069": { + "answer": 6, + "tokens": 44381, + "fixed_tokens": 1100, + "processed_tokens": 449804, + "tool_calls": 18, + "turns": 19 + }, + "world-070": { + "answer": 3, + "tokens": 7271, + "fixed_tokens": 1093, + "processed_tokens": 62643, + "tool_calls": 6, + "turns": 7 + }, + "world-071": { + "answer": [ + "dec-hot-reload-gull-functions-by-swapping-bytecode-a--8dxnkg6h8vmcsr42", + "dec-compress-streamed-assets-with-kelp--ezwc5p1tgsgm7v8m", + "dec-evict-cells-by-distance-and-tide-forecast--fyp1jp366pjjzxcg" + ], + "tokens": 68743, + "fixed_tokens": 1103, + "processed_tokens": 868686, + "tool_calls": 23, + "turns": 24 + }, + "world-072": { + "answer": [ + "dec-compress-streamed-assets-with-lz4--v1pfsgfp0b3yjv4s", + "dec-compress-streamed-assets-with-zstandard--bs1t15cygc51cpje" + ], + "tokens": 17478, + "fixed_tokens": 1094, + "processed_tokens": 166562, + "tool_calls": 11, + "turns": 12 + }, + "world-073": { + "answer": false, + "tokens": 11141, + "fixed_tokens": 1105, + "processed_tokens": 91911, + "tool_calls": 9, + "turns": 8 + }, + "world-074": { + "answer": 7, + "tokens": 9425, + "fixed_tokens": 1098, + "processed_tokens": 70713, + "tool_calls": 6, + "turns": 7 + }, + "world-075": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--8679sjn071ycsbpv", + "dec-partition-the-world-into-192-meter-cells--q62ddwdv2n8wxay6", + "dec-return-engine-jobs-to-the-thread-pool-with-conti--547qbweqvrp1j5g3" + ], + "tokens": 6376, + "fixed_tokens": 1102, + "processed_tokens": 51081, + "tool_calls": 5, + "turns": 6 + }, + "world-076": { + "answer": "unknown", + "tokens": 52506, + "fixed_tokens": 1096, + "processed_tokens": 990265, + "tool_calls": 39, + "turns": 32 + }, + "world-078": { + "answer": [ + "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "rsk-reworking-the-read-scheduler-may-slip-the-coast--9jzvs7etvb5jxqtp" + ], + "tokens": 11198, + "fixed_tokens": 1096, + "processed_tokens": 74801, + "tool_calls": 7, + "turns": 7 + }, + "world-079": { + "answer": [ + "con-no-runtime-code-generation-on-consoles--6cf25zqmsvqkdd73", + "con-tide-state-must-be-bit-identical-on-every-peer--zrpnp8qjyxzkas7t", + "con-saves-from-beta-onward-must-load-in-every-later--v6z856sxtgc5sb09", + "con-streaming-pool-budget--8czwb4fjr5p5vpy7" + ], + "tokens": 11534, + "fixed_tokens": 1096, + "processed_tokens": 83754, + "tool_calls": 7, + "turns": 8 + }, + "world-080": { + "answer": [ + "iss-cross-platform-cloud-saves-are-unplanned--5bk0k4wrgj42ztfy", + "iss-gull-hot-reload-drops-suspended-coroutines--z7j2gmyny6gdxd07", + "iss-gull-hot-reload-rejects-struct-layout-changes--3v2w2f9hg2gwy6aj" + ], + "tokens": 27905, + "fixed_tokens": 1100, + "processed_tokens": 179390, + "tool_calls": 13, + "turns": 10 + }, + "world-081": { + "answer": "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "tokens": 58329, + "fixed_tokens": 1104, + "processed_tokens": 1506977, + "tool_calls": 48, + "turns": 44 + } + } +} diff --git a/benchmarks/proof-recall/runs/x10-rund-r1-findstate.json b/benchmarks/proof-recall/runs/x10-rund-r1-findstate.json new file mode 100644 index 00000000..9d119c86 --- /dev/null +++ b/benchmarks/proof-recall/runs/x10-rund-r1-findstate.json @@ -0,0 +1,851 @@ +{ + "run": { + "name": "x10-rund-r1-findstate", + "protocol": "one fresh session per question", + "models": ["claude-haiku-5-5"], + "find_hash": "e5be653299abe9b2329822d98ffe09e90ee7cfe96f8711c813a51dc6500b887f", + "state_hash": "b73c82ecb0649c1db200b189b827c76250569b75327ba5847d9939352dcf1679" + }, + "answers": { + "cross-002": { + "answer": 2, + "tokens": 13035, + "fixed_tokens": 1102, + "processed_tokens": 69443, + "tool_calls": 5, + "turns": 6 + }, + "cross-003": { + "answer": 2, + "tokens": 5260, + "fixed_tokens": 1110, + "processed_tokens": 32507, + "tool_calls": 3, + "turns": 4 + }, + "cross-004": { + "answer": "dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e", + "tokens": 10192, + "fixed_tokens": 1111, + "processed_tokens": 90169, + "tool_calls": 7, + "turns": 8 + }, + "cross-014": { + "answer": 3, + "tokens": 29706, + "fixed_tokens": 1103, + "processed_tokens": 99823, + "tool_calls": 7, + "turns": 6 + }, + "cross-015": { + "answer": "Two Efforts are no longer active. Co-op rollback netcode prototype was abandoned: its decision dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b (accepted 2026-04-01) stopped the prototype because rollback re-simulating tides costs 4.8 ms per frame on Skiff. Shader compile cache and build farm was completed once every farm build read from the cache, as recorded in fnd-retrospective-on-the-shader-compile-cache--9rpv2a60ach04cx6 (2026-04-28).", + "tokens": 18074, + "fixed_tokens": 1103, + "processed_tokens": 113580, + "tool_calls": 10, + "turns": 8 + }, + "netcode-001": { + "answer": "abandoned", + "tokens": 10268, + "fixed_tokens": 1115, + "processed_tokens": 61364, + "tool_calls": 5, + "turns": 6 + }, + "netcode-010": { + "answer": "dec-use-a-6-frame-rollback-window-with-1-frame-of-in--s53944zhkw103frk", + "tokens": 10198, + "fixed_tokens": 1105, + "processed_tokens": 76209, + "tool_calls": 6, + "turns": 7 + }, + "netcode-013": { + "answer": [ + "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "fnd-rollback-re-simulates-a-4-frame-window-in-1-1-ms--xa30a1cez2brphyf", + "rsk-re-simulation-cost-could-blow-the-skiff-frame-bu--sk2njsp170a3qwyf" + ], + "tokens": 13134, + "fixed_tokens": 1110, + "processed_tokens": 81776, + "tool_calls": 6, + "turns": 7 + }, + "netcode-014": { + "answer": [ + "iss-late-join-during-a-tide-surge-desyncs-the-joinin--mya93va0n7s8g7cj" + ], + "tokens": 38703, + "fixed_tokens": 1105, + "processed_tokens": 283006, + "tool_calls": 13, + "turns": 12 + }, + "netcode-015": { + "answer": 2, + "tokens": 8807, + "fixed_tokens": 1105, + "processed_tokens": 60467, + "tool_calls": 5, + "turns": 6 + }, + "ports-004": { + "answer": "accepted", + "tokens": 15274, + "fixed_tokens": 1107, + "processed_tokens": 90564, + "tool_calls": 7, + "turns": 7 + }, + "ports-005": { + "answer": "unknown", + "tokens": 20746, + "fixed_tokens": 1111, + "processed_tokens": 154386, + "tool_calls": 8, + "turns": 9 + }, + "ports-017": { + "answer": "fnd-skiff-microsd-slows-sharply-on-full-warm-cards--p1rgh36hat2egpz3", + "tokens": 8197, + "fixed_tokens": 1102, + "processed_tokens": 60461, + "tool_calls": 5, + "turns": 6 + }, + "ports-019": { + "answer": "4 MB per minute", + "tokens": 15425, + "fixed_tokens": 1106, + "processed_tokens": 135115, + "tool_calls": 8, + "turns": 9 + }, + "ports-021": { + "answer": "SKF-DSP-004", + "tokens": 15978, + "fixed_tokens": 1108, + "processed_tokens": 89810, + "tool_calls": 8, + "turns": 7 + }, + "ports-022": { + "answer": "cit-skiff-sdk-1-2-release-notes--7gwwn8xxffrx6npe", + "tokens": 12740, + "fixed_tokens": 1103, + "processed_tokens": 127760, + "tool_calls": 11, + "turns": 10 + }, + "ports-023": { + "answer": "fnd-tern-costs-3-9-ms-at-quality-and-2-1-ms-at-perfo--dkbskbqfgfwtryq0", + "tokens": 17402, + "fixed_tokens": 1108, + "processed_tokens": 113914, + "tool_calls": 9, + "turns": 8 + }, + "ports-026": { + "answer": "912 MB", + "tokens": 12802, + "fixed_tokens": 1102, + "processed_tokens": 156007, + "tool_calls": 11, + "turns": 12 + }, + "ports-027": { + "answer": [ + "dec-stream-skiff-world-tiles-in-2-mb-reads--3jz9w9gvhhc7eppw", + "rsk-tide-streaming-outruns-skiff-storage--6q8s3b383z2tyeyb" + ], + "tokens": 14823, + "fixed_tokens": 1107, + "processed_tokens": 126632, + "tool_calls": 8, + "turns": 9 + }, + "ports-028": { + "answer": 2, + "tokens": 11490, + "fixed_tokens": 1102, + "processed_tokens": 83339, + "tool_calls": 6, + "turns": 7 + }, + "ports-029": { + "answer": 8, + "tokens": 48151, + "fixed_tokens": 1109, + "processed_tokens": 394639, + "tool_calls": 19, + "turns": 14 + }, + "ports-030": { + "answer": [ + "dec-cap-tide-storm-particles-at-6-000-on-skiff--resze52afjxyxys1", + "dec-drop-skiff-to-30-hz-while-a-tide-storm-is-on-scr--72vx1c8s4jrs6axc" + ], + "tokens": 12665, + "fixed_tokens": 1111, + "processed_tokens": 95639, + "tool_calls": 7, + "turns": 8 + }, + "ports-031": { + "answer": [ + "iss-meridian-cert-activity-cards-missing-for-two-bio--k214cmfra7pyvvzq", + "iss-skiff-cert-docked-30-hz-judder-on-some-tvs--9nq2xg9mzash474g" + ], + "tokens": 14095, + "fixed_tokens": 1105, + "processed_tokens": 88966, + "tool_calls": 8, + "turns": 7 + }, + "ports-032": { + "answer": 2, + "tokens": 13147, + "fixed_tokens": 1106, + "processed_tokens": 87009, + "tool_calls": 6, + "turns": 7 + }, + "render-005": { + "answer": true, + "tokens": 7933, + "fixed_tokens": 1104, + "processed_tokens": 59507, + "tool_calls": 5, + "turns": 6 + }, + "render-012": { + "answer": "deferred", + "tokens": 9780, + "fixed_tokens": 1102, + "processed_tokens": 89404, + "tool_calls": 7, + "turns": 8 + }, + "render-023": { + "answer": "dec-capture-the-cliffs-and-dunes-benchmarks-on-every--3pqyr66yh4v6q676", + "tokens": 8846, + "fixed_tokens": 1102, + "processed_tokens": 61183, + "tool_calls": 7, + "turns": 6 + }, + "render-024": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 9748, + "fixed_tokens": 1102, + "processed_tokens": 88908, + "tool_calls": 7, + "turns": 8 + }, + "render-029": { + "answer": "fnd-vsm-page-pool-overflows-during-spring-tides--m2p8e89hhdmh0z8a", + "tokens": 9960, + "fixed_tokens": 1099, + "processed_tokens": 90316, + "tool_calls": 7, + "turns": 8 + }, + "render-033": { + "answer": "128 MB", + "tokens": 10218, + "fixed_tokens": 1102, + "processed_tokens": 65521, + "tool_calls": 7, + "turns": 6 + }, + "render-035": { + "answer": "5912 pages", + "tokens": 20212, + "fixed_tokens": 1109, + "processed_tokens": 238534, + "tool_calls": 14, + "turns": 15 + }, + "render-036": { + "answer": "HDR-07", + "tokens": 7788, + "fixed_tokens": 1097, + "processed_tokens": 59488, + "tool_calls": 5, + "turns": 6 + }, + "render-041": { + "answer": "4.7 ms", + "tokens": 8571, + "fixed_tokens": 1103, + "processed_tokens": 61900, + "tool_calls": 5, + "turns": 6 + }, + "render-042": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 20344, + "fixed_tokens": 1098, + "processed_tokens": 192082, + "tool_calls": 10, + "turns": 11 + }, + "render-043": { + "answer": "fnd-meshlet-cone-culling-rejects-9-percent-of-triang--t6e5x7trzedex105", + "tokens": 7457, + "fixed_tokens": 1101, + "processed_tokens": 58245, + "tool_calls": 5, + "turns": 6 + }, + "render-048": { + "answer": "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "tokens": 23612, + "fixed_tokens": 1105, + "processed_tokens": 231632, + "tool_calls": 13, + "turns": 12 + }, + "render-058": { + "answer": "iss-which-upscaler-should-lantern-ship--g8c2y9tsr856tn2v", + "tokens": 13814, + "fixed_tokens": 1102, + "processed_tokens": 163616, + "tool_calls": 13, + "turns": 12 + }, + "render-059": { + "answer": "fnd-sdf-gi-costs-2-2-ms-on-meridian--r7t7zy97nwxxxctd", + "tokens": 17404, + "fixed_tokens": 1097, + "processed_tokens": 107782, + "tool_calls": 6, + "turns": 7 + }, + "render-060": { + "answer": "0.31", + "tokens": 10911, + "fixed_tokens": 1101, + "processed_tokens": 118451, + "tool_calls": 9, + "turns": 10 + }, + "render-061": { + "answer": "con-ui-composites-at-a-fixed-203-nits-in-hdr--de6aft7cvhdezn5s", + "tokens": 13491, + "fixed_tokens": 1098, + "processed_tokens": 159195, + "tool_calls": 11, + "turns": 12 + }, + "render-063": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 19881, + "fixed_tokens": 1102, + "processed_tokens": 103366, + "tool_calls": 6, + "turns": 7 + }, + "render-065": { + "answer": "dec-ship-tern-as-lantern-s-only-upscaler--fmreq35nzyjc7k4e", + "tokens": 9512, + "fixed_tokens": 1101, + "processed_tokens": 88326, + "tool_calls": 7, + "turns": 8 + }, + "render-066": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 9218, + "fixed_tokens": 1103, + "processed_tokens": 62747, + "tool_calls": 5, + "turns": 6 + }, + "render-067": { + "answer": 4, + "tokens": 12235, + "fixed_tokens": 1101, + "processed_tokens": 54722, + "tool_calls": 4, + "turns": 5 + }, + "render-068": { + "answer": [ + "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "dec-lock-caustic-photon-jitter-to-the-tide-phase--85x10k4h11t8nx7c", + "dec-use-the-probe-grid-alone-in-120-hz-performance-m--6dzm1nt40pjfpyae", + "dec-trace-sdf-gi-at-quarter-resolution-in-performanc--06cqt3mqn3fg5h6e", + "dec-read-paper-white-from-the-meridian-system-settin--7crbx73gqw1hn7xq", + "dec-add-an-in-game-paper-white-slider-from-80-to-400--1phfvxby983z8tjf" + ], + "tokens": 14013, + "fixed_tokens": 1100, + "processed_tokens": 74667, + "tool_calls": 5, + "turns": 6 + }, + "render-069": { + "answer": "iss-shadow-pages-run-out-at-spring-low-tide--f4fgy6b44jbkrsmh", + "tokens": 13955, + "fixed_tokens": 1096, + "processed_tokens": 74758, + "tool_calls": 5, + "turns": 6 + }, + "render-070": { + "answer": [ + "dec-use-signed-distance-field-gi--yn366zmp3k7snd9w", + "dec-enable-meshlet-cone-culling-on-all-platforms--cpyweprmzcqhqsv2", + "dec-feed-a-saltglass-reactive-mask-into-tern--xezjs7v3vq3r8e6k" + ], + "tokens": 15930, + "fixed_tokens": 1103, + "processed_tokens": 95259, + "tool_calls": 6, + "turns": 7 + }, + "render-071": { + "answer": 3, + "tokens": 13484, + "fixed_tokens": 1099, + "processed_tokens": 91341, + "tool_calls": 6, + "turns": 7 + }, + "render-072": { + "answer": [ + "rsk-sdf-volumes-outgrow-the-render-target-memory-bud--xfd2pfztvskdcd2c", + "rsk-artists-cannot-match-concept-paintings-under-the--6cfh9vkamn0kbtcx" + ], + "tokens": 18412, + "fixed_tokens": 1109, + "processed_tokens": 100971, + "tool_calls": 6, + "turns": 7 + }, + "render-073": { + "answer": [ + "rsk-vsm-page-pool-overflows-when-tides-expose-the-fl--x1r161nrdngra4ty", + "rsk-gi-cannot-fit-the-120-hz-performance-mode--tqt3fj839dqfq5s7", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 20070, + "fixed_tokens": 1106, + "processed_tokens": 149504, + "tool_calls": 8, + "turns": 9 + }, + "render-074": { + "answer": 5, + "tokens": 12474, + "fixed_tokens": 1097, + "processed_tokens": 55074, + "tool_calls": 4, + "turns": 5 + }, + "render-075": { + "answer": "3 Decisions", + "tokens": 24189, + "fixed_tokens": 1100, + "processed_tokens": 171033, + "tool_calls": 9, + "turns": 10 + }, + "render-076": { + "answer": false, + "tokens": 10633, + "fixed_tokens": 1102, + "processed_tokens": 103079, + "tool_calls": 8, + "turns": 9 + }, + "render-078": { + "answer": [ + "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 17112, + "fixed_tokens": 1107, + "processed_tokens": 115621, + "tool_calls": 9, + "turns": 10 + }, + "render-079": { + "answer": 5, + "tokens": 47327, + "fixed_tokens": 1105, + "processed_tokens": 563320, + "tool_calls": 17, + "turns": 18 + }, + "render-080": { + "answer": "6 versions", + "tokens": 13161, + "fixed_tokens": 1099, + "processed_tokens": 140411, + "tool_calls": 10, + "turns": 11 + }, + "render-081": { + "answer": "unknown", + "tokens": 17546, + "fixed_tokens": 1097, + "processed_tokens": 86482, + "tool_calls": 5, + "turns": 6 + }, + "render-083": { + "answer": 4, + "tokens": 29178, + "fixed_tokens": 1101, + "processed_tokens": 198871, + "tool_calls": 15, + "turns": 10 + }, + "tools-001": { + "answer": "completed", + "tokens": 10191, + "fixed_tokens": 1109, + "processed_tokens": 53099, + "tool_calls": 4, + "turns": 5 + }, + "tools-007": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 8489, + "fixed_tokens": 1105, + "processed_tokens": 71132, + "tool_calls": 6, + "turns": 7 + }, + "tools-010": { + "answer": "dec-cap-each-farm-agent-at-6-parallel-shader-compile--q5w4sfjm73tsj8x8", + "tokens": 17147, + "fixed_tokens": 1105, + "processed_tokens": 104291, + "tool_calls": 9, + "turns": 8 + }, + "tools-012": { + "answer": 3, + "tokens": 8967, + "fixed_tokens": 1109, + "processed_tokens": 72821, + "tool_calls": 6, + "turns": 7 + }, + "tools-013": { + "answer": 9, + "tokens": 55597, + "fixed_tokens": 1113, + "processed_tokens": 1090189, + "tool_calls": 31, + "turns": 30 + }, + "tools-015": { + "answer": [ + "iss-developer-workstations-cannot-read-the-farm-shad--q2xw0f7fze6z55nj", + "rsk-shader-cache-storage-could-outgrow-the-4-tb-farm--76y9649mwrx7aavd" + ], + "tokens": 11551, + "fixed_tokens": 1110, + "processed_tokens": 78167, + "tool_calls": 8, + "turns": 7 + }, + "tools-016": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 13643, + "fixed_tokens": 1107, + "processed_tokens": 116412, + "tool_calls": 8, + "turns": 9 + }, + "world-011": { + "answer": "fnd-shipping-entity-names-costs-57-mb-at-full-popula--6w7gjvy7wdmdx9qx", + "tokens": 9247, + "fixed_tokens": 1108, + "processed_tokens": 85596, + "tool_calls": 7, + "turns": 8 + }, + "world-013": { + "answer": "unknown", + "tokens": 16178, + "fixed_tokens": 1112, + "processed_tokens": 87552, + "tool_calls": 6, + "turns": 7 + }, + "world-016": { + "answer": "unknown", + "tokens": 13836, + "fixed_tokens": 1099, + "processed_tokens": 92464, + "tool_calls": 6, + "turns": 7 + }, + "world-020": { + "answer": "dec-schedule-streaming-reads-by-deadline-instead-of--q2x5aknabgcssdj2", + "tokens": 15066, + "fixed_tokens": 1104, + "processed_tokens": 62794, + "tool_calls": 4, + "turns": 5 + }, + "world-021": { + "answer": "dec-cut-navigation-tiles-to-64-m-so-three-fit-along--snvepk8ygeb3qsjs", + "tokens": 7075, + "fixed_tokens": 1099, + "processed_tokens": 47146, + "tool_calls": 4, + "turns": 5 + }, + "world-022": { + "answer": "fnd-41-of-alpha-crash-reports-have-no-usable-stack--35a50b07ferhf13m", + "tokens": 6254, + "fixed_tokens": 1105, + "processed_tokens": 53354, + "tool_calls": 5, + "turns": 6 + }, + "world-029": { + "answer": "dec-return-engine-jobs-to-a-thread-pool-with-continu--1zkc1v2x97taf50h", + "tokens": 10513, + "fixed_tokens": 1104, + "processed_tokens": 90652, + "tool_calls": 7, + "turns": 8 + }, + "world-034": { + "answer": "fnd-cross-platform-cloud-saves-need-platform-agreeme--4qgfc1a4nq4padvz", + "tokens": 21358, + "fixed_tokens": 1101, + "processed_tokens": 151012, + "tool_calls": 8, + "turns": 9 + }, + "world-037": { + "answer": "fnd-hot-reload-drops-suspended-gull-coroutines--va0fn0svryzwbzws", + "tokens": 9648, + "fixed_tokens": 1100, + "processed_tokens": 88458, + "tool_calls": 7, + "turns": 8 + }, + "world-042": { + "answer": "5.4 s", + "tokens": 9047, + "fixed_tokens": 1109, + "processed_tokens": 62622, + "tool_calls": 7, + "turns": 6 + }, + "world-049": { + "answer": "22%", + "tokens": 8266, + "fixed_tokens": 1104, + "processed_tokens": 60807, + "tool_calls": 5, + "turns": 6 + }, + "world-052": { + "answer": "Dara Whitlock", + "tokens": 8163, + "fixed_tokens": 1106, + "processed_tokens": 60623, + "tool_calls": 5, + "turns": 6 + }, + "world-053": { + "answer": "1.9 ms", + "tokens": 11558, + "fixed_tokens": 1100, + "processed_tokens": 96683, + "tool_calls": 9, + "turns": 8 + }, + "world-057": { + "answer": "tile (212, 87)", + "tokens": 13055, + "fixed_tokens": 1100, + "processed_tokens": 184558, + "tool_calls": 13, + "turns": 14 + }, + "world-059": { + "answer": "dec-derive-the-missing-harmonic-phase-from-each-save--gbqrvvgyq2gy2gnw", + "tokens": 11396, + "fixed_tokens": 1099, + "processed_tokens": 95567, + "tool_calls": 9, + "turns": 8 + }, + "world-060": { + "answer": "dec-partition-the-world-into-192-m-cells--bk9r1c5jwb6fh8tc", + "tokens": 11112, + "fixed_tokens": 1099, + "processed_tokens": 142074, + "tool_calls": 11, + "turns": 12 + }, + "world-061": { + "answer": "236 MB", + "tokens": 13875, + "fixed_tokens": 1107, + "processed_tokens": 188394, + "tool_calls": 13, + "turns": 14 + }, + "world-062": { + "answer": "310 MB", + "tokens": 12808, + "fixed_tokens": 1097, + "processed_tokens": 155537, + "tool_calls": 11, + "turns": 12 + }, + "world-065": { + "answer": "3.1 ms", + "tokens": 26118, + "fixed_tokens": 1099, + "processed_tokens": 303187, + "tool_calls": 14, + "turns": 15 + }, + "world-068": { + "answer": true, + "tokens": 14897, + "fixed_tokens": 1102, + "processed_tokens": 94161, + "tool_calls": 6, + "turns": 7 + }, + "world-069": { + "answer": 7, + "tokens": 79670, + "fixed_tokens": 1103, + "processed_tokens": 753464, + "tool_calls": 18, + "turns": 18 + }, + "world-070": { + "answer": 3, + "tokens": 13707, + "fixed_tokens": 1108, + "processed_tokens": 92210, + "tool_calls": 6, + "turns": 7 + }, + "world-071": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--yxsjd26wcaz6fgx9", + "dec-compress-streamed-assets-with-kelp--ezwc5p1tgsgm7v8m", + "dec-evict-cells-by-distance-and-tide-forecast--fyp1jp366pjjzxcg", + "dec-hot-reload-gull-functions-by-swapping-bytecode-a--8dxnkg6h8vmcsr42", + "dec-stamp-each-save-with-a-schema-version-and-upgrad--nbg1h8g1bhn2ks8w" + ], + "tokens": 35376, + "fixed_tokens": 1100, + "processed_tokens": 212332, + "tool_calls": 17, + "turns": 10 + }, + "world-072": { + "answer": [ + "dec-compress-streamed-assets-with-lz4--v1pfsgfp0b3yjv4s", + "dec-compress-streamed-assets-with-zstandard--bs1t15cygc51cpje" + ], + "tokens": 29458, + "fixed_tokens": 1104, + "processed_tokens": 198789, + "tool_calls": 11, + "turns": 12 + }, + "world-073": { + "answer": false, + "tokens": 11225, + "fixed_tokens": 1109, + "processed_tokens": 80654, + "tool_calls": 8, + "turns": 7 + }, + "world-074": { + "answer": 7, + "tokens": 13294, + "fixed_tokens": 1103, + "processed_tokens": 73768, + "tool_calls": 5, + "turns": 6 + }, + "world-075": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--8679sjn071ycsbpv", + "dec-partition-the-world-into-192-meter-cells--q62ddwdv2n8wxay6", + "dec-return-engine-jobs-to-the-thread-pool-with-conti--547qbweqvrp1j5g3" + ], + "tokens": 7232, + "fixed_tokens": 1099, + "processed_tokens": 62953, + "tool_calls": 6, + "turns": 7 + }, + "world-076": { + "answer": "dec-use-virtual-shadow-maps-for-the-sun--x8nqkn9q7eaf1ap4", + "tokens": 42867, + "fixed_tokens": 1097, + "processed_tokens": 442473, + "tool_calls": 15, + "turns": 15 + }, + "world-078": { + "answer": [ + "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "rsk-reworking-the-read-scheduler-may-slip-the-coast--9jzvs7etvb5jxqtp" + ], + "tokens": 19105, + "fixed_tokens": 1106, + "processed_tokens": 102712, + "tool_calls": 6, + "turns": 7 + }, + "world-079": { + "answer": [ + "con-no-runtime-code-generation-on-consoles--6cf25zqmsvqkdd73", + "con-tide-state-must-be-bit-identical-on-every-peer--zrpnp8qjyxzkas7t", + "con-saves-from-beta-onward-must-load-in-every-later--v6z856sxtgc5sb09", + "con-streaming-pool-budget--8czwb4fjr5p5vpy7" + ], + "tokens": 18892, + "fixed_tokens": 1096, + "processed_tokens": 102425, + "tool_calls": 6, + "turns": 7 + }, + "world-080": { + "answer": [ + "iss-gull-hot-reload-rejects-struct-layout-changes--3v2w2f9hg2gwy6aj", + "iss-gull-hot-reload-drops-suspended-coroutines--z7j2gmyny6gdxd07", + "iss-cross-platform-cloud-saves-are-unplanned--5bk0k4wrgj42ztfy" + ], + "tokens": 16107, + "fixed_tokens": 1099, + "processed_tokens": 115613, + "tool_calls": 7, + "turns": 8 + }, + "world-081": { + "answer": "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "tokens": 47270, + "fixed_tokens": 1099, + "processed_tokens": 388445, + "tool_calls": 18, + "turns": 12 + } + } +} diff --git a/benchmarks/proof-recall/runs/x10-rund-r2-find.json b/benchmarks/proof-recall/runs/x10-rund-r2-find.json new file mode 100644 index 00000000..01acb51f --- /dev/null +++ b/benchmarks/proof-recall/runs/x10-rund-r2-find.json @@ -0,0 +1,872 @@ +{ + "run": { + "name": "x10-rund-r2-find", + "protocol": "one fresh session per question", + "models": ["claude-haiku-5-5"], + "find_hash": "e5be653299abe9b2329822d98ffe09e90ee7cfe96f8711c813a51dc6500b887f", + "state_hash": "b73c82ecb0649c1db200b189b827c76250569b75327ba5847d9939352dcf1679" + }, + "answers": { + "cross-002": { + "answer": 2, + "tokens": 14985, + "fixed_tokens": 1096, + "processed_tokens": 95970, + "tool_calls": 8, + "turns": 7 + }, + "cross-003": { + "answer": 2, + "tokens": 8349, + "fixed_tokens": 1101, + "processed_tokens": 65973, + "tool_calls": 8, + "turns": 7 + }, + "cross-004": { + "answer": "dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e", + "tokens": 11612, + "fixed_tokens": 1104, + "processed_tokens": 78923, + "tool_calls": 6, + "turns": 7 + }, + "cross-014": { + "answer": 3, + "tokens": 4766, + "fixed_tokens": 1100, + "processed_tokens": 39291, + "tool_calls": 4, + "turns": 5 + }, + "cross-015": { + "answer": "Two Efforts are no longer active. The Shader compile cache and build farm Effort is completed: it ended once every farm build read from the cache, per the decision dec-turn-the-shader-compile-cache-on-for-every-farm--4rr3b12b4sasdmaj (accepted 2026-03-20) and the retrospective fnd-retrospective-on-the-shader-compile-cache--9rpv2a60ach04cx6. The Co-op rollback netcode prototype Effort is abandoned: rollback will not ship for co-op because tide simulation makes it unworkable, per dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b (accepted 2026-04-01).", + "tokens": 16540, + "fixed_tokens": 1104, + "processed_tokens": 137468, + "tool_calls": 10, + "turns": 10 + }, + "netcode-001": { + "answer": "abandoned", + "tokens": 7201, + "fixed_tokens": 1101, + "processed_tokens": 57571, + "tool_calls": 5, + "turns": 6 + }, + "netcode-010": { + "answer": "dec-use-a-6-frame-rollback-window-with-1-frame-of-in--s53944zhkw103frk", + "tokens": 22798, + "fixed_tokens": 1098, + "processed_tokens": 103777, + "tool_calls": 7, + "turns": 7 + }, + "netcode-013": { + "answer": [ + "fnd-rollback-re-simulates-a-4-frame-window-in-1-1-ms--xa30a1cez2brphyf", + "rsk-re-simulation-cost-could-blow-the-skiff-frame-bu--sk2njsp170a3qwyf" + ], + "tokens": 25686, + "fixed_tokens": 1100, + "processed_tokens": 209989, + "tool_calls": 20, + "turns": 11 + }, + "netcode-014": { + "answer": [ + "iss-late-join-during-a-tide-surge-desyncs-the-joinin--mya93va0n7s8g7cj" + ], + "tokens": 47955, + "fixed_tokens": 1098, + "processed_tokens": 493528, + "tool_calls": 17, + "turns": 18 + }, + "netcode-015": { + "answer": 2, + "tokens": 12124, + "fixed_tokens": 1102, + "processed_tokens": 119880, + "tool_calls": 10, + "turns": 11 + }, + "ports-004": { + "answer": "accepted", + "tokens": 7442, + "fixed_tokens": 1102, + "processed_tokens": 57873, + "tool_calls": 5, + "turns": 6 + }, + "ports-005": { + "answer": "unknown", + "tokens": 26205, + "fixed_tokens": 1100, + "processed_tokens": 196057, + "tool_calls": 10, + "turns": 11 + }, + "ports-017": { + "answer": "fnd-skiff-microsd-slows-sharply-on-full-warm-cards--p1rgh36hat2egpz3", + "tokens": 8062, + "fixed_tokens": 1102, + "processed_tokens": 59625, + "tool_calls": 5, + "turns": 6 + }, + "ports-019": { + "answer": "4 MB per minute", + "tokens": 8679, + "fixed_tokens": 1104, + "processed_tokens": 84120, + "tool_calls": 7, + "turns": 8 + }, + "ports-021": { + "answer": "SKF-DSP-004", + "tokens": 12032, + "fixed_tokens": 1099, + "processed_tokens": 125201, + "tool_calls": 11, + "turns": 10 + }, + "ports-022": { + "answer": "cit-skiff-sdk-1-2-release-notes--7gwwn8xxffrx6npe", + "tokens": 13443, + "fixed_tokens": 1096, + "processed_tokens": 132862, + "tool_calls": 11, + "turns": 10 + }, + "ports-023": { + "answer": "fnd-tern-costs-3-9-ms-at-quality-and-2-1-ms-at-perfo--dkbskbqfgfwtryq0", + "tokens": 11455, + "fixed_tokens": 1108, + "processed_tokens": 120766, + "tool_calls": 9, + "turns": 10 + }, + "ports-026": { + "answer": "912 MB", + "tokens": 12220, + "fixed_tokens": 1098, + "processed_tokens": 166662, + "tool_calls": 12, + "turns": 13 + }, + "ports-027": { + "answer": [ + "dec-load-skiff-fast-travel-without-a-loading-screen--v4qft5znysjv5f24", + "dec-stream-skiff-world-tiles-in-2-mb-reads--3jz9w9gvhhc7eppw", + "rsk-tide-streaming-outruns-skiff-storage--6q8s3b383z2tyeyb" + ], + "tokens": 141037, + "fixed_tokens": 1098, + "processed_tokens": 2637233, + "tool_calls": 42, + "turns": 38 + }, + "ports-028": { + "answer": 2, + "tokens": 7604, + "fixed_tokens": 1098, + "processed_tokens": 53367, + "tool_calls": 5, + "turns": 6 + }, + "ports-029": { + "answer": 8, + "tokens": 80299, + "fixed_tokens": 1102, + "processed_tokens": 1561323, + "tool_calls": 35, + "turns": 31 + }, + "ports-030": { + "answer": [ + "dec-cap-tide-storm-particles-at-6-000-on-skiff--resze52afjxyxys1", + "dec-drop-skiff-to-30-hz-while-a-tide-storm-is-on-scr--72vx1c8s4jrs6axc" + ], + "tokens": 10597, + "fixed_tokens": 1104, + "processed_tokens": 84212, + "tool_calls": 7, + "turns": 8 + }, + "ports-031": { + "answer": [ + "iss-skiff-cert-docked-30-hz-judder-on-some-tvs--9nq2xg9mzash474g", + "iss-meridian-cert-activity-cards-missing-for-two-bio--k214cmfra7pyvvzq" + ], + "tokens": 17426, + "fixed_tokens": 1098, + "processed_tokens": 137656, + "tool_calls": 11, + "turns": 10 + }, + "ports-032": { + "answer": 2, + "tokens": 6207, + "fixed_tokens": 1098, + "processed_tokens": 50960, + "tool_calls": 5, + "turns": 6 + }, + "render-005": { + "answer": "iss-shadow-pages-run-out-at-spring-low-tide--f4fgy6b44jbkrsmh", + "tokens": 11232, + "fixed_tokens": 1094, + "processed_tokens": 76974, + "tool_calls": 6, + "turns": 7 + }, + "render-012": { + "answer": "deferred", + "tokens": 9034, + "fixed_tokens": 1094, + "processed_tokens": 73292, + "tool_calls": 7, + "turns": 7 + }, + "render-023": { + "answer": "dec-capture-the-cliffs-and-dunes-benchmarks-on-every--3pqyr66yh4v6q676", + "tokens": 7360, + "fixed_tokens": 1093, + "processed_tokens": 57602, + "tool_calls": 5, + "turns": 6 + }, + "render-024": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 9355, + "fixed_tokens": 1095, + "processed_tokens": 62600, + "tool_calls": 7, + "turns": 6 + }, + "render-029": { + "answer": "fnd-vsm-page-pool-overflows-during-spring-tides--m2p8e89hhdmh0z8a", + "tokens": 8073, + "fixed_tokens": 1093, + "processed_tokens": 60302, + "tool_calls": 5, + "turns": 6 + }, + "render-033": { + "answer": "128 MB", + "tokens": 8233, + "fixed_tokens": 1097, + "processed_tokens": 60521, + "tool_calls": 5, + "turns": 6 + }, + "render-035": { + "answer": "5912 pages", + "tokens": 9402, + "fixed_tokens": 1096, + "processed_tokens": 87599, + "tool_calls": 7, + "turns": 8 + }, + "render-036": { + "answer": "HDR-07", + "tokens": 7491, + "fixed_tokens": 1097, + "processed_tokens": 58120, + "tool_calls": 5, + "turns": 6 + }, + "render-041": { + "answer": "4.7 ms", + "tokens": 8379, + "fixed_tokens": 1100, + "processed_tokens": 60626, + "tool_calls": 5, + "turns": 6 + }, + "render-042": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 9764, + "fixed_tokens": 1098, + "processed_tokens": 88192, + "tool_calls": 7, + "turns": 8 + }, + "render-043": { + "answer": "fnd-meshlet-cone-culling-rejects-9-percent-of-triang--t6e5x7trzedex105", + "tokens": 9613, + "fixed_tokens": 1093, + "processed_tokens": 63359, + "tool_calls": 7, + "turns": 6 + }, + "render-048": { + "answer": "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "tokens": 21090, + "fixed_tokens": 1094, + "processed_tokens": 247357, + "tool_calls": 14, + "turns": 15 + }, + "render-058": { + "answer": "iss-which-upscaler-should-lantern-ship--g8c2y9tsr856tn2v", + "tokens": 12564, + "fixed_tokens": 1091, + "processed_tokens": 181675, + "tool_calls": 13, + "turns": 14 + }, + "render-059": { + "answer": "fnd-sdf-gi-costs-4-7-ms-on-meridian--sm19rt2f274ajsvr", + "tokens": 11489, + "fixed_tokens": 1102, + "processed_tokens": 94764, + "tool_calls": 7, + "turns": 8 + }, + "render-060": { + "answer": "0.31", + "tokens": 10743, + "fixed_tokens": 1098, + "processed_tokens": 115668, + "tool_calls": 9, + "turns": 10 + }, + "render-061": { + "answer": "con-ui-composites-at-a-fixed-203-nits-in-hdr--de6aft7cvhdezn5s", + "tokens": 10168, + "fixed_tokens": 1098, + "processed_tokens": 114905, + "tool_calls": 9, + "turns": 10 + }, + "render-063": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 10359, + "fixed_tokens": 1096, + "processed_tokens": 77014, + "tool_calls": 7, + "turns": 7 + }, + "render-065": { + "answer": "dec-ship-tern-as-lantern-s-only-upscaler--fmreq35nzyjc7k4e", + "tokens": 9923, + "fixed_tokens": 1096, + "processed_tokens": 101741, + "tool_calls": 8, + "turns": 9 + }, + "render-066": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 8327, + "fixed_tokens": 1096, + "processed_tokens": 59689, + "tool_calls": 5, + "turns": 6 + }, + "render-067": { + "answer": 4, + "tokens": 9190, + "fixed_tokens": 1101, + "processed_tokens": 90009, + "tool_calls": 8, + "turns": 9 + }, + "render-068": { + "answer": [ + "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "dec-lock-caustic-photon-jitter-to-the-tide-phase--85x10k4h11t8nx7c", + "dec-use-the-probe-grid-alone-in-120-hz-performance-m--6dzm1nt40pjfpyae", + "dec-trace-sdf-gi-at-quarter-resolution-in-performanc--06cqt3mqn3fg5h6e", + "dec-read-paper-white-from-the-meridian-system-settin--7crbx73gqw1hn7xq", + "dec-add-an-in-game-paper-white-slider-from-80-to-400--1phfvxby983z8tjf" + ], + "tokens": 18635, + "fixed_tokens": 1099, + "processed_tokens": 132854, + "tool_calls": 9, + "turns": 10 + }, + "render-069": { + "answer": "iss-shadow-pages-run-out-at-spring-low-tide--f4fgy6b44jbkrsmh", + "tokens": 17874, + "fixed_tokens": 1097, + "processed_tokens": 118909, + "tool_calls": 9, + "turns": 9 + }, + "render-070": { + "answer": ["dec-use-signed-distance-field-gi--yn366zmp3k7snd9w"], + "tokens": 60745, + "fixed_tokens": 1092, + "processed_tokens": 814699, + "tool_calls": 22, + "turns": 23 + }, + "render-071": { + "answer": 3, + "tokens": 11070, + "fixed_tokens": 1098, + "processed_tokens": 74211, + "tool_calls": 8, + "turns": 7 + }, + "render-072": { + "answer": [ + "rsk-sdf-volumes-outgrow-the-render-target-memory-bud--xfd2pfztvskdcd2c", + "rsk-artists-cannot-match-concept-paintings-under-the--6cfh9vkamn0kbtcx" + ], + "tokens": 12446, + "fixed_tokens": 1100, + "processed_tokens": 100327, + "tool_calls": 10, + "turns": 9 + }, + "render-073": { + "answer": [ + "rsk-vsm-page-pool-overflows-when-tides-expose-the-fl--x1r161nrdngra4ty", + "rsk-gi-cannot-fit-the-120-hz-performance-mode--tqt3fj839dqfq5s7", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 14191, + "fixed_tokens": 1095, + "processed_tokens": 104353, + "tool_calls": 12, + "turns": 9 + }, + "render-074": { + "answer": 5, + "tokens": 8311, + "fixed_tokens": 1093, + "processed_tokens": 67296, + "tool_calls": 6, + "turns": 7 + }, + "render-075": { + "answer": "3", + "tokens": 47840, + "fixed_tokens": 1095, + "processed_tokens": 1180237, + "tool_calls": 34, + "turns": 35 + }, + "render-076": { + "answer": "false", + "tokens": 15432, + "fixed_tokens": 1096, + "processed_tokens": 125801, + "tool_calls": 9, + "turns": 10 + }, + "render-078": { + "answer": [ + "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 19515, + "fixed_tokens": 1097, + "processed_tokens": 215643, + "tool_calls": 13, + "turns": 14 + }, + "render-079": { + "answer": 5, + "tokens": 33067, + "fixed_tokens": 1093, + "processed_tokens": 415847, + "tool_calls": 17, + "turns": 18 + }, + "render-080": { + "answer": "6 versions", + "tokens": 13622, + "fixed_tokens": 1104, + "processed_tokens": 140662, + "tool_calls": 10, + "turns": 11 + }, + "render-081": { + "answer": "unknown", + "tokens": 16784, + "fixed_tokens": 1091, + "processed_tokens": 81135, + "tool_calls": 5, + "turns": 6 + }, + "render-083": { + "answer": null, + "tokens": 108416, + "fixed_tokens": 1098, + "processed_tokens": 7981750, + "tool_calls": 116, + "turns": 116 + }, + "tools-001": { + "answer": "completed", + "tokens": 8296, + "fixed_tokens": 1100, + "processed_tokens": 72028, + "tool_calls": 6, + "turns": 7 + }, + "tools-007": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 46255, + "fixed_tokens": 1102, + "processed_tokens": 1198521, + "tool_calls": 35, + "turns": 36 + }, + "tools-010": { + "answer": "dec-cap-each-farm-agent-at-6-parallel-shader-compile--q5w4sfjm73tsj8x8", + "tokens": 17324, + "fixed_tokens": 1103, + "processed_tokens": 166976, + "tool_calls": 12, + "turns": 11 + }, + "tools-012": { + "answer": 3, + "tokens": 20759, + "fixed_tokens": 1097, + "processed_tokens": 168279, + "tool_calls": 10, + "turns": 11 + }, + "tools-013": { + "answer": 7, + "tokens": 91442, + "fixed_tokens": 1104, + "processed_tokens": 2152952, + "tool_calls": 42, + "turns": 43 + }, + "tools-015": { + "answer": [ + "iss-whether-the-audio-bank-packer-should-load-on-a-w--sqzp2rwp6s01w3b0", + "iss-whether-the-level-editor-autosave-needs-its-own--rkvmnbqe7w644zkr", + "iss-whether-the-level-editor-autosave-should-be-cach--radcjs4hyvh7p55q", + "iss-whether-the-artist-sandbox-should-load-on-a-work--73j819anqct8etfn", + "iss-whether-the-review-bot-should-load-on-a-worker-t--rtsjwvsksjdq15v1", + "iss-whether-the-crash-symbol-server-should-load-on-a--qvyphhgnsnm2cvqb", + "iss-whether-the-license-scanner-should-be-cached-bet--nqhd89czbfat9rya", + "iss-whether-the-doc-site-should-load-on-a-worker-thr--qxswe0pg52ekky0z", + "iss-whether-the-asset-search-should-load-on-a-worker--f9gpv3z522x6p6h4", + "iss-whether-the-changelog-bot-should-load-on-a-worke--jgtdn2f02p8fx5nr", + "iss-whether-the-review-bot-needs-its-own-memory-pool--pg3m8g9gn3y6v3kq", + "iss-whether-the-changelog-bot-should-skip-work-when--p2vrp5d682p3jrc3", + "iss-whether-the-patch-differ-should-load-on-a-worker--cysv4n56q1qn7t2k", + "iss-whether-the-artist-sandbox-should-skip-work-when--11tqzws72gms2h8x", + "iss-whether-the-audio-bank-packer-needs-its-own-memo--qjrqrseast8d5mhf", + "iss-whether-the-changelog-bot-needs-its-own-memory-p--mw8zejz6hk5m60x0", + "iss-whether-the-crash-symbol-server-should-ship-in-t--bhq91ajavah18wsk", + "iss-whether-the-asset-search-should-load-on-a-worker--80b27kqh36rpafdt", + "iss-whether-the-doc-site-should-ship-in-the-first-pa--wcdkq5zab7dxe5yj", + "iss-whether-the-patch-differ-should-be-cached-betwee--28b5zqb1ketwzf0r", + "iss-whether-the-test-runner-should-load-on-a-worker--aygay1r0qj4mzf78", + "iss-whether-the-texture-viewer-should-be-split-into--1jhgyqb70h5x7m7b", + "iss-whether-the-level-editor-autosave-should-skip-wo--fyysrcv6fvbsvcsy", + "iss-whether-the-level-editor-autosave-should-be-spli--ncnwh0csw671fs0q", + "iss-whether-the-asset-search-should-ship-in-the-firs--hsb2avhx613prxew", + "iss-whether-the-test-runner-needs-its-own-memory-poo--bs6arq0qv37d6wmx", + "iss-whether-the-artist-sandbox-should-be-split-into--ek47ca2vgmrjanp8", + "rsk-shader-cache-storage-could-outgrow-the-4-tb-farm--76y9649mwrx7aavd" + ], + "tokens": 78795, + "fixed_tokens": 1106, + "processed_tokens": 530845, + "tool_calls": 17, + "turns": 16 + }, + "tools-016": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 14856, + "fixed_tokens": 1098, + "processed_tokens": 116514, + "tool_calls": 8, + "turns": 9 + }, + "world-011": { + "answer": "fnd-shipping-entity-names-costs-57-mb-at-full-popula--6w7gjvy7wdmdx9qx", + "tokens": 8934, + "fixed_tokens": 1094, + "processed_tokens": 83692, + "tool_calls": 7, + "turns": 8 + }, + "world-013": { + "answer": "unknown", + "tokens": 10933, + "fixed_tokens": 1096, + "processed_tokens": 77161, + "tool_calls": 6, + "turns": 7 + }, + "world-016": { + "answer": "unknown", + "tokens": 10561, + "fixed_tokens": 1097, + "processed_tokens": 100190, + "tool_calls": 8, + "turns": 9 + }, + "world-020": { + "answer": "dec-schedule-streaming-reads-by-deadline-instead-of--q2x5aknabgcssdj2", + "tokens": 7708, + "fixed_tokens": 1091, + "processed_tokens": 58614, + "tool_calls": 5, + "turns": 6 + }, + "world-021": { + "answer": "dec-cut-navigation-tiles-to-64-m-so-three-fit-along--snvepk8ygeb3qsjs", + "tokens": 7537, + "fixed_tokens": 1100, + "processed_tokens": 58278, + "tool_calls": 5, + "turns": 6 + }, + "world-022": { + "answer": "fnd-41-of-alpha-crash-reports-have-no-usable-stack--35a50b07ferhf13m", + "tokens": 7378, + "fixed_tokens": 1095, + "processed_tokens": 57637, + "tool_calls": 5, + "turns": 6 + }, + "world-029": { + "answer": "dec-return-engine-jobs-to-a-thread-pool-with-continu--1zkc1v2x97taf50h", + "tokens": 9998, + "fixed_tokens": 1092, + "processed_tokens": 87886, + "tool_calls": 7, + "turns": 8 + }, + "world-034": { + "answer": "fnd-cross-platform-cloud-saves-need-platform-agreeme--4qgfc1a4nq4padvz", + "tokens": 15251, + "fixed_tokens": 1098, + "processed_tokens": 115323, + "tool_calls": 8, + "turns": 9 + }, + "world-037": { + "answer": "fnd-hot-reload-drops-suspended-gull-coroutines--va0fn0svryzwbzws", + "tokens": 7858, + "fixed_tokens": 1098, + "processed_tokens": 58783, + "tool_calls": 5, + "turns": 6 + }, + "world-042": { + "answer": "5.4 s", + "tokens": 7638, + "fixed_tokens": 1098, + "processed_tokens": 58366, + "tool_calls": 5, + "turns": 6 + }, + "world-049": { + "answer": "22%", + "tokens": 7885, + "fixed_tokens": 1093, + "processed_tokens": 59315, + "tool_calls": 5, + "turns": 6 + }, + "world-052": { + "answer": "Dara Whitlock", + "tokens": 11273, + "fixed_tokens": 1094, + "processed_tokens": 93990, + "tool_calls": 9, + "turns": 8 + }, + "world-053": { + "answer": "1.9 ms", + "tokens": 9539, + "fixed_tokens": 1093, + "processed_tokens": 86512, + "tool_calls": 7, + "turns": 8 + }, + "world-057": { + "answer": "tile [212, 87]", + "tokens": 12677, + "fixed_tokens": 1102, + "processed_tokens": 181025, + "tool_calls": 13, + "turns": 14 + }, + "world-059": { + "answer": "dec-derive-the-missing-harmonic-phase-from-each-save--gbqrvvgyq2gy2gnw", + "tokens": 12500, + "fixed_tokens": 1098, + "processed_tokens": 126680, + "tool_calls": 11, + "turns": 10 + }, + "world-060": { + "answer": "dec-partition-the-world-into-192-m-cells--bk9r1c5jwb6fh8tc", + "tokens": 9415, + "fixed_tokens": 1098, + "processed_tokens": 86538, + "tool_calls": 7, + "turns": 8 + }, + "world-061": { + "answer": "236 MB", + "tokens": 13839, + "fixed_tokens": 1098, + "processed_tokens": 188217, + "tool_calls": 13, + "turns": 14 + }, + "world-062": { + "answer": "310 MB", + "tokens": 13234, + "fixed_tokens": 1098, + "processed_tokens": 185567, + "tool_calls": 13, + "turns": 14 + }, + "world-065": { + "answer": "3.1 ms", + "tokens": 13105, + "fixed_tokens": 1098, + "processed_tokens": 184664, + "tool_calls": 13, + "turns": 14 + }, + "world-068": { + "answer": true, + "tokens": 21532, + "fixed_tokens": 1098, + "processed_tokens": 228123, + "tool_calls": 14, + "turns": 15 + }, + "world-069": { + "answer": "unknown", + "tokens": 48437, + "fixed_tokens": 1096, + "processed_tokens": 394905, + "tool_calls": 19, + "turns": 16 + }, + "world-070": { + "answer": 3, + "tokens": 7310, + "fixed_tokens": 1103, + "processed_tokens": 62628, + "tool_calls": 6, + "turns": 7 + }, + "world-071": { + "answer": [ + "dec-hot-reload-gull-functions-by-swapping-bytecode-a--8dxnkg6h8vmcsr42", + "dec-compress-streamed-assets-with-kelp--ezwc5p1tgsgm7v8m", + "dec-store-components-in-archetype-tables-and-volatil--yxsjd26wcaz6fgx9", + "dec-evict-cells-by-distance-and-tide-forecast--fyp1jp366pjjzxcg" + ], + "tokens": 67557, + "fixed_tokens": 1105, + "processed_tokens": 718966, + "tool_calls": 23, + "turns": 21 + }, + "world-072": { + "answer": [ + "dec-compress-streamed-assets-with-lz4--v1pfsgfp0b3yjv4s", + "dec-compress-streamed-assets-with-zstandard--bs1t15cygc51cpje" + ], + "tokens": 19529, + "fixed_tokens": 1096, + "processed_tokens": 154351, + "tool_calls": 10, + "turns": 11 + }, + "world-073": { + "answer": false, + "tokens": 10913, + "fixed_tokens": 1095, + "processed_tokens": 64472, + "tool_calls": 9, + "turns": 6 + }, + "world-074": { + "answer": 7, + "tokens": 9448, + "fixed_tokens": 1092, + "processed_tokens": 70748, + "tool_calls": 6, + "turns": 7 + }, + "world-075": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--8679sjn071ycsbpv", + "dec-partition-the-world-into-192-meter-cells--q62ddwdv2n8wxay6", + "dec-return-engine-jobs-to-the-thread-pool-with-conti--547qbweqvrp1j5g3" + ], + "tokens": 6582, + "fixed_tokens": 1098, + "processed_tokens": 51270, + "tool_calls": 5, + "turns": 6 + }, + "world-076": { + "answer": "unknown", + "tokens": 40379, + "fixed_tokens": 1100, + "processed_tokens": 903093, + "tool_calls": 34, + "turns": 35 + }, + "world-078": { + "answer": [ + "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "rsk-reworking-the-read-scheduler-may-slip-the-coast--9jzvs7etvb5jxqtp" + ], + "tokens": 10377, + "fixed_tokens": 1097, + "processed_tokens": 58578, + "tool_calls": 5, + "turns": 6 + }, + "world-079": { + "answer": [ + "con-no-runtime-code-generation-on-consoles--6cf25zqmsvqkdd73", + "con-tide-state-must-be-bit-identical-on-every-peer--zrpnp8qjyxzkas7t", + "con-saves-from-beta-onward-must-load-in-every-later--v6z856sxtgc5sb09", + "con-streaming-pool-budget--8czwb4fjr5p5vpy7" + ], + "tokens": 12030, + "fixed_tokens": 1093, + "processed_tokens": 85245, + "tool_calls": 7, + "turns": 8 + }, + "world-080": { + "answer": [ + "iss-gull-hot-reload-rejects-struct-layout-changes--3v2w2f9hg2gwy6aj", + "iss-gull-hot-reload-drops-suspended-coroutines--z7j2gmyny6gdxd07", + "iss-cross-platform-cloud-saves-are-unplanned--5bk0k4wrgj42ztfy" + ], + "tokens": 25868, + "fixed_tokens": 1098, + "processed_tokens": 260739, + "tool_calls": 14, + "turns": 15 + }, + "world-081": { + "answer": "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "tokens": 36383, + "fixed_tokens": 1096, + "processed_tokens": 291394, + "tool_calls": 28, + "turns": 13 + } + } +} diff --git a/benchmarks/proof-recall/runs/x10-rund-r2-findstate.json b/benchmarks/proof-recall/runs/x10-rund-r2-findstate.json new file mode 100644 index 00000000..10a214a2 --- /dev/null +++ b/benchmarks/proof-recall/runs/x10-rund-r2-findstate.json @@ -0,0 +1,852 @@ +{ + "run": { + "name": "x10-rund-r2-findstate", + "protocol": "one fresh session per question", + "models": ["claude-haiku-5-5"], + "find_hash": "e5be653299abe9b2329822d98ffe09e90ee7cfe96f8711c813a51dc6500b887f", + "state_hash": "b73c82ecb0649c1db200b189b827c76250569b75327ba5847d9939352dcf1679" + }, + "answers": { + "cross-002": { + "answer": 2, + "tokens": 9386, + "fixed_tokens": 1105, + "processed_tokens": 48094, + "tool_calls": 4, + "turns": 5 + }, + "cross-003": { + "answer": 2, + "tokens": 5450, + "fixed_tokens": 1107, + "processed_tokens": 40529, + "tool_calls": 4, + "turns": 5 + }, + "cross-004": { + "answer": "dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e", + "tokens": 10174, + "fixed_tokens": 1104, + "processed_tokens": 89732, + "tool_calls": 7, + "turns": 8 + }, + "cross-014": { + "answer": 3, + "tokens": 5020, + "fixed_tokens": 1103, + "processed_tokens": 40278, + "tool_calls": 4, + "turns": 5 + }, + "cross-015": { + "answer": "Two Efforts are no longer active. The Shader compile cache and build farm Effort is completed: the cache now runs on every farm build, as its closing retrospective fnd-retrospective-on-the-shader-compile-cache--9rpv2a60ach04cx6 records. The Co-op rollback netcode prototype is abandoned: the team stopped the prototype and decided co-op will not ship with rollback, because re-running tides every rolled-back frame costs 4.8 ms on Skiff (dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b).", + "tokens": 18009, + "fixed_tokens": 1108, + "processed_tokens": 113502, + "tool_calls": 10, + "turns": 8 + }, + "netcode-001": { + "answer": "abandoned", + "tokens": 10254, + "fixed_tokens": 1110, + "processed_tokens": 61251, + "tool_calls": 5, + "turns": 6 + }, + "netcode-010": { + "answer": "dec-use-a-6-frame-rollback-window-with-1-frame-of-in--s53944zhkw103frk", + "tokens": 10120, + "fixed_tokens": 1107, + "processed_tokens": 75734, + "tool_calls": 6, + "turns": 7 + }, + "netcode-013": { + "answer": [ + "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "fnd-rollback-re-simulates-a-4-frame-window-in-1-1-ms--xa30a1cez2brphyf", + "rsk-re-simulation-cost-could-blow-the-skiff-frame-bu--sk2njsp170a3qwyf" + ], + "tokens": 20529, + "fixed_tokens": 1104, + "processed_tokens": 170247, + "tool_calls": 11, + "turns": 10 + }, + "netcode-014": { + "answer": [ + "iss-how-should-co-op-keep-up-to-four-players-in-sync--92t2zfsy3etfzr89", + "iss-late-join-during-a-tide-surge-desyncs-the-joinin--mya93va0n7s8g7cj" + ], + "tokens": 43538, + "fixed_tokens": 1111, + "processed_tokens": 425739, + "tool_calls": 15, + "turns": 16 + }, + "netcode-015": { + "answer": 2, + "tokens": 9445, + "fixed_tokens": 1114, + "processed_tokens": 74890, + "tool_calls": 6, + "turns": 7 + }, + "ports-004": { + "answer": "accepted", + "tokens": 14493, + "fixed_tokens": 1107, + "processed_tokens": 97010, + "tool_calls": 6, + "turns": 7 + }, + "ports-005": { + "answer": "unknown", + "tokens": 22541, + "fixed_tokens": 1104, + "processed_tokens": 176476, + "tool_calls": 9, + "turns": 10 + }, + "ports-017": { + "answer": "fnd-skiff-microsd-slows-sharply-on-full-warm-cards--p1rgh36hat2egpz3", + "tokens": 8626, + "fixed_tokens": 1104, + "processed_tokens": 61601, + "tool_calls": 5, + "turns": 6 + }, + "ports-019": { + "answer": "4 MB per minute", + "tokens": 8956, + "fixed_tokens": 1111, + "processed_tokens": 86048, + "tool_calls": 7, + "turns": 8 + }, + "ports-021": { + "answer": "SKF-DSP-004", + "tokens": 15081, + "fixed_tokens": 1089, + "processed_tokens": 111551, + "tool_calls": 7, + "turns": 8 + }, + "ports-022": { + "answer": "cit-skiff-sdk-1-2-release-notes--7gwwn8xxffrx6npe", + "tokens": 15050, + "fixed_tokens": 1104, + "processed_tokens": 174175, + "tool_calls": 13, + "turns": 12 + }, + "ports-023": { + "answer": "fnd-tern-costs-3-9-ms-at-quality-and-2-1-ms-at-perfo--dkbskbqfgfwtryq0", + "tokens": 20027, + "fixed_tokens": 1104, + "processed_tokens": 171363, + "tool_calls": 10, + "turns": 11 + }, + "ports-026": { + "answer": "912 MB", + "tokens": 12931, + "fixed_tokens": 1102, + "processed_tokens": 157252, + "tool_calls": 11, + "turns": 12 + }, + "ports-027": { + "answer": [ + "dec-stream-skiff-world-tiles-in-2-mb-reads--3jz9w9gvhhc7eppw", + "rsk-tide-streaming-outruns-skiff-storage--6q8s3b383z2tyeyb" + ], + "tokens": 14884, + "fixed_tokens": 1107, + "processed_tokens": 91234, + "tool_calls": 8, + "turns": 7 + }, + "ports-028": { + "answer": 2, + "tokens": 10643, + "fixed_tokens": 1100, + "processed_tokens": 51190, + "tool_calls": 4, + "turns": 5 + }, + "ports-029": { + "answer": 5, + "tokens": 71986, + "fixed_tokens": 1104, + "processed_tokens": 2278748, + "tool_calls": 51, + "turns": 52 + }, + "ports-030": { + "answer": [ + "dec-cap-tide-storm-particles-at-6-000-on-skiff--resze52afjxyxys1", + "dec-drop-skiff-to-30-hz-while-a-tide-storm-is-on-scr--72vx1c8s4jrs6axc" + ], + "tokens": 11173, + "fixed_tokens": 1105, + "processed_tokens": 67139, + "tool_calls": 5, + "turns": 6 + }, + "ports-031": { + "answer": [ + "iss-meridian-cert-activity-cards-missing-for-two-bio--k214cmfra7pyvvzq", + "iss-skiff-cert-docked-30-hz-judder-on-some-tvs--9nq2xg9mzash474g" + ], + "tokens": 13722, + "fixed_tokens": 1102, + "processed_tokens": 88085, + "tool_calls": 8, + "turns": 7 + }, + "ports-032": { + "answer": 2, + "tokens": 14598, + "fixed_tokens": 1105, + "processed_tokens": 106231, + "tool_calls": 9, + "turns": 8 + }, + "render-005": { + "answer": true, + "tokens": 16609, + "fixed_tokens": 1100, + "processed_tokens": 88998, + "tool_calls": 6, + "turns": 7 + }, + "render-012": { + "answer": "deferred", + "tokens": 9817, + "fixed_tokens": 1107, + "processed_tokens": 89689, + "tool_calls": 7, + "turns": 8 + }, + "render-023": { + "answer": "dec-capture-the-cliffs-and-dunes-benchmarks-on-every--3pqyr66yh4v6q676", + "tokens": 7666, + "fixed_tokens": 1096, + "processed_tokens": 58966, + "tool_calls": 5, + "turns": 6 + }, + "render-024": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 9478, + "fixed_tokens": 1100, + "processed_tokens": 63436, + "tool_calls": 7, + "turns": 6 + }, + "render-029": { + "answer": "fnd-vsm-page-pool-overflows-during-spring-tides--m2p8e89hhdmh0z8a", + "tokens": 10130, + "fixed_tokens": 1102, + "processed_tokens": 90692, + "tool_calls": 7, + "turns": 8 + }, + "render-033": { + "answer": "128 MB", + "tokens": 10035, + "fixed_tokens": 1103, + "processed_tokens": 64986, + "tool_calls": 7, + "turns": 6 + }, + "render-035": { + "answer": "5912 pages", + "tokens": 9720, + "fixed_tokens": 1101, + "processed_tokens": 89464, + "tool_calls": 7, + "turns": 8 + }, + "render-036": { + "answer": "HDR-07", + "tokens": 7748, + "fixed_tokens": 1114, + "processed_tokens": 59519, + "tool_calls": 5, + "turns": 6 + }, + "render-041": { + "answer": "4.7 ms", + "tokens": 8711, + "fixed_tokens": 1101, + "processed_tokens": 62137, + "tool_calls": 5, + "turns": 6 + }, + "render-042": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 17546, + "fixed_tokens": 1101, + "processed_tokens": 131836, + "tool_calls": 8, + "turns": 9 + }, + "render-043": { + "answer": "fnd-meshlet-cone-culling-rejects-9-percent-of-triang--t6e5x7trzedex105", + "tokens": 7794, + "fixed_tokens": 1105, + "processed_tokens": 59407, + "tool_calls": 5, + "turns": 6 + }, + "render-048": { + "answer": "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "tokens": 23368, + "fixed_tokens": 1102, + "processed_tokens": 209585, + "tool_calls": 12, + "turns": 11 + }, + "render-058": { + "answer": "iss-which-upscaler-should-lantern-ship--g8c2y9tsr856tn2v", + "tokens": 12437, + "fixed_tokens": 1101, + "processed_tokens": 154812, + "tool_calls": 12, + "turns": 12 + }, + "render-059": { + "answer": "fnd-sdf-gi-costs-4-7-ms-on-meridian--sm19rt2f274ajsvr", + "tokens": 13404, + "fixed_tokens": 1104, + "processed_tokens": 100929, + "tool_calls": 9, + "turns": 8 + }, + "render-060": { + "answer": "0.31", + "tokens": 10987, + "fixed_tokens": 1099, + "processed_tokens": 118559, + "tool_calls": 9, + "turns": 10 + }, + "render-061": { + "answer": "con-ui-composites-at-a-fixed-203-nits-in-hdr--de6aft7cvhdezn5s", + "tokens": 11221, + "fixed_tokens": 1099, + "processed_tokens": 143588, + "tool_calls": 11, + "turns": 12 + }, + "render-063": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 12902, + "fixed_tokens": 1106, + "processed_tokens": 73443, + "tool_calls": 7, + "turns": 6 + }, + "render-065": { + "answer": "dec-ship-tern-as-lantern-s-only-upscaler--fmreq35nzyjc7k4e", + "tokens": 9662, + "fixed_tokens": 1111, + "processed_tokens": 88717, + "tool_calls": 7, + "turns": 8 + }, + "render-066": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 10463, + "fixed_tokens": 1100, + "processed_tokens": 89925, + "tool_calls": 7, + "turns": 8 + }, + "render-067": { + "answer": 4, + "tokens": 13683, + "fixed_tokens": 1101, + "processed_tokens": 91862, + "tool_calls": 6, + "turns": 7 + }, + "render-068": { + "answer": [ + "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "dec-lock-caustic-photon-jitter-to-the-tide-phase--85x10k4h11t8nx7c", + "dec-use-the-probe-grid-alone-in-120-hz-performance-m--6dzm1nt40pjfpyae", + "dec-trace-sdf-gi-at-quarter-resolution-in-performanc--06cqt3mqn3fg5h6e", + "dec-read-paper-white-from-the-meridian-system-settin--7crbx73gqw1hn7xq", + "dec-add-an-in-game-paper-white-slider-from-80-to-400--1phfvxby983z8tjf" + ], + "tokens": 18884, + "fixed_tokens": 1101, + "processed_tokens": 102426, + "tool_calls": 6, + "turns": 7 + }, + "render-069": { + "answer": "iss-shadow-pages-run-out-at-spring-low-tide--f4fgy6b44jbkrsmh", + "tokens": 14011, + "fixed_tokens": 1106, + "processed_tokens": 74766, + "tool_calls": 5, + "turns": 6 + }, + "render-070": { + "answer": [ + "dec-use-signed-distance-field-gi--yn366zmp3k7snd9w", + "dec-enable-meshlet-cone-culling-on-all-platforms--cpyweprmzcqhqsv2", + "dec-feed-a-saltglass-reactive-mask-into-tern--xezjs7v3vq3r8e6k" + ], + "tokens": 14378, + "fixed_tokens": 1099, + "processed_tokens": 56963, + "tool_calls": 4, + "turns": 5 + }, + "render-071": { + "answer": 3, + "tokens": 13861, + "fixed_tokens": 1104, + "processed_tokens": 92331, + "tool_calls": 6, + "turns": 7 + }, + "render-072": { + "answer": [ + "rsk-sdf-volumes-outgrow-the-render-target-memory-bud--xfd2pfztvskdcd2c", + "rsk-artists-cannot-match-concept-paintings-under-the--6cfh9vkamn0kbtcx" + ], + "tokens": 15263, + "fixed_tokens": 1104, + "processed_tokens": 112548, + "tool_calls": 7, + "turns": 8 + }, + "render-073": { + "answer": [ + "rsk-vsm-page-pool-overflows-when-tides-expose-the-fl--x1r161nrdngra4ty", + "rsk-gi-cannot-fit-the-120-hz-performance-mode--tqt3fj839dqfq5s7", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 18692, + "fixed_tokens": 1107, + "processed_tokens": 101643, + "tool_calls": 6, + "turns": 7 + }, + "render-074": { + "answer": 5, + "tokens": 13239, + "fixed_tokens": 1103, + "processed_tokens": 73591, + "tool_calls": 5, + "turns": 6 + }, + "render-075": { + "answer": "3", + "tokens": 22279, + "fixed_tokens": 1100, + "processed_tokens": 214238, + "tool_calls": 12, + "turns": 12 + }, + "render-076": { + "answer": false, + "tokens": 19892, + "fixed_tokens": 1101, + "processed_tokens": 156194, + "tool_calls": 9, + "turns": 9 + }, + "render-078": { + "answer": [ + "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 16475, + "fixed_tokens": 1099, + "processed_tokens": 93172, + "tool_calls": 7, + "turns": 8 + }, + "render-079": { + "answer": 6, + "tokens": 18011, + "fixed_tokens": 1106, + "processed_tokens": 118745, + "tool_calls": 7, + "turns": 8 + }, + "render-080": { + "answer": "6 versions", + "tokens": 15135, + "fixed_tokens": 1104, + "processed_tokens": 163465, + "tool_calls": 11, + "turns": 12 + }, + "render-081": { + "answer": "unknown", + "tokens": 19424, + "fixed_tokens": 1102, + "processed_tokens": 130717, + "tool_calls": 7, + "turns": 8 + }, + "render-083": { + "answer": 4, + "tokens": 18331, + "fixed_tokens": 1098, + "processed_tokens": 101533, + "tool_calls": 8, + "turns": 7 + }, + "tools-001": { + "answer": "completed", + "tokens": 13152, + "fixed_tokens": 1113, + "processed_tokens": 88903, + "tool_calls": 6, + "turns": 7 + }, + "tools-007": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 10907, + "fixed_tokens": 1105, + "processed_tokens": 76561, + "tool_calls": 8, + "turns": 7 + }, + "tools-010": { + "answer": "dec-cap-each-farm-agent-at-6-parallel-shader-compile--q5w4sfjm73tsj8x8", + "tokens": 15736, + "fixed_tokens": 1113, + "processed_tokens": 99474, + "tool_calls": 7, + "turns": 8 + }, + "tools-012": { + "answer": 3, + "tokens": 11035, + "fixed_tokens": 1104, + "processed_tokens": 90234, + "tool_calls": 7, + "turns": 8 + }, + "tools-013": { + "answer": 9, + "tokens": 72603, + "fixed_tokens": 1109, + "processed_tokens": 1719427, + "tool_calls": 42, + "turns": 37 + }, + "tools-015": { + "answer": [ + "iss-developer-workstations-cannot-read-the-farm-shad--q2xw0f7fze6z55nj", + "rsk-shader-cache-storage-could-outgrow-the-4-tb-farm--76y9649mwrx7aavd" + ], + "tokens": 14963, + "fixed_tokens": 1106, + "processed_tokens": 134247, + "tool_calls": 11, + "turns": 10 + }, + "tools-016": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 13849, + "fixed_tokens": 1105, + "processed_tokens": 116915, + "tool_calls": 8, + "turns": 9 + }, + "world-011": { + "answer": "fnd-shipping-entity-names-costs-57-mb-at-full-popula--6w7gjvy7wdmdx9qx", + "tokens": 18094, + "fixed_tokens": 1109, + "processed_tokens": 94089, + "tool_calls": 8, + "turns": 7 + }, + "world-013": { + "answer": "unknown", + "tokens": 11298, + "fixed_tokens": 1097, + "processed_tokens": 78788, + "tool_calls": 6, + "turns": 7 + }, + "world-016": { + "answer": "unknown", + "tokens": 16422, + "fixed_tokens": 1103, + "processed_tokens": 100276, + "tool_calls": 6, + "turns": 7 + }, + "world-020": { + "answer": "dec-schedule-streaming-reads-by-deadline-instead-of--q2x5aknabgcssdj2", + "tokens": 16776, + "fixed_tokens": 1104, + "processed_tokens": 105730, + "tool_calls": 6, + "turns": 7 + }, + "world-021": { + "answer": "dec-cut-navigation-tiles-to-64-m-so-three-fit-along--snvepk8ygeb3qsjs", + "tokens": 7839, + "fixed_tokens": 1102, + "processed_tokens": 59757, + "tool_calls": 5, + "turns": 6 + }, + "world-022": { + "answer": "fnd-41-of-alpha-crash-reports-have-no-usable-stack--35a50b07ferhf13m", + "tokens": 7851, + "fixed_tokens": 1107, + "processed_tokens": 59292, + "tool_calls": 5, + "turns": 6 + }, + "world-029": { + "answer": "dec-return-engine-jobs-to-a-thread-pool-with-continu--1zkc1v2x97taf50h", + "tokens": 10184, + "fixed_tokens": 1103, + "processed_tokens": 89645, + "tool_calls": 7, + "turns": 8 + }, + "world-034": { + "answer": "unknown", + "tokens": 20953, + "fixed_tokens": 1102, + "processed_tokens": 145643, + "tool_calls": 8, + "turns": 9 + }, + "world-037": { + "answer": "fnd-hot-reload-drops-suspended-gull-coroutines--va0fn0svryzwbzws", + "tokens": 8751, + "fixed_tokens": 1098, + "processed_tokens": 61901, + "tool_calls": 5, + "turns": 6 + }, + "world-042": { + "answer": "5.4 s", + "tokens": 10683, + "fixed_tokens": 1103, + "processed_tokens": 93626, + "tool_calls": 9, + "turns": 8 + }, + "world-049": { + "answer": "22%", + "tokens": 8220, + "fixed_tokens": 1103, + "processed_tokens": 60740, + "tool_calls": 5, + "turns": 6 + }, + "world-052": { + "answer": "Dara Whitlock", + "tokens": 10078, + "fixed_tokens": 1102, + "processed_tokens": 64709, + "tool_calls": 7, + "turns": 6 + }, + "world-053": { + "answer": "1.9 ms", + "tokens": 18829, + "fixed_tokens": 1101, + "processed_tokens": 120741, + "tool_calls": 8, + "turns": 9 + }, + "world-057": { + "answer": "tile [212, 87]", + "tokens": 13319, + "fixed_tokens": 1100, + "processed_tokens": 159498, + "tool_calls": 11, + "turns": 12 + }, + "world-059": { + "answer": "dec-derive-the-missing-harmonic-phase-from-each-save--gbqrvvgyq2gy2gnw", + "tokens": 9923, + "fixed_tokens": 1102, + "processed_tokens": 89255, + "tool_calls": 7, + "turns": 8 + }, + "world-060": { + "answer": "dec-partition-the-world-into-192-m-cells--bk9r1c5jwb6fh8tc", + "tokens": 16164, + "fixed_tokens": 1107, + "processed_tokens": 97374, + "tool_calls": 6, + "turns": 7 + }, + "world-061": { + "answer": "236 MB", + "tokens": 13863, + "fixed_tokens": 1102, + "processed_tokens": 188594, + "tool_calls": 13, + "turns": 14 + }, + "world-062": { + "answer": "310 MB", + "tokens": 12313, + "fixed_tokens": 1102, + "processed_tokens": 154918, + "tool_calls": 11, + "turns": 12 + }, + "world-065": { + "answer": "3.1 ms", + "tokens": 15471, + "fixed_tokens": 1105, + "processed_tokens": 170400, + "tool_calls": 11, + "turns": 12 + }, + "world-068": { + "answer": true, + "tokens": 15038, + "fixed_tokens": 1107, + "processed_tokens": 94199, + "tool_calls": 6, + "turns": 7 + }, + "world-069": { + "answer": 7, + "tokens": 51586, + "fixed_tokens": 1106, + "processed_tokens": 499500, + "tool_calls": 22, + "turns": 17 + }, + "world-070": { + "answer": 3, + "tokens": 15318, + "fixed_tokens": 1105, + "processed_tokens": 95475, + "tool_calls": 6, + "turns": 7 + }, + "world-071": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--yxsjd26wcaz6fgx9", + "dec-compress-streamed-assets-with-kelp--ezwc5p1tgsgm7v8m", + "dec-evict-cells-by-distance-and-tide-forecast--fyp1jp366pjjzxcg", + "dec-hot-reload-gull-functions-by-swapping-bytecode-a--8dxnkg6h8vmcsr42", + "dec-stamp-each-save-with-a-schema-version-and-upgrad--nbg1h8g1bhn2ks8w" + ], + "tokens": 31832, + "fixed_tokens": 1105, + "processed_tokens": 294005, + "tool_calls": 22, + "turns": 14 + }, + "world-072": { + "answer": [ + "dec-compress-streamed-assets-with-lz4--v1pfsgfp0b3yjv4s", + "dec-compress-streamed-assets-with-zstandard--bs1t15cygc51cpje" + ], + "tokens": 37066, + "fixed_tokens": 1099, + "processed_tokens": 421024, + "tool_calls": 16, + "turns": 17 + }, + "world-073": { + "answer": false, + "tokens": 10079, + "fixed_tokens": 1104, + "processed_tokens": 64330, + "tool_calls": 7, + "turns": 6 + }, + "world-074": { + "answer": 7, + "tokens": 13446, + "fixed_tokens": 1100, + "processed_tokens": 74014, + "tool_calls": 5, + "turns": 6 + }, + "world-075": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--8679sjn071ycsbpv", + "dec-partition-the-world-into-192-meter-cells--q62ddwdv2n8wxay6", + "dec-return-engine-jobs-to-the-thread-pool-with-conti--547qbweqvrp1j5g3" + ], + "tokens": 15337, + "fixed_tokens": 1101, + "processed_tokens": 95221, + "tool_calls": 6, + "turns": 7 + }, + "world-076": { + "answer": "unknown", + "tokens": 69544, + "fixed_tokens": 1101, + "processed_tokens": 1041306, + "tool_calls": 30, + "turns": 25 + }, + "world-078": { + "answer": [ + "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "rsk-reworking-the-read-scheduler-may-slip-the-coast--9jzvs7etvb5jxqtp" + ], + "tokens": 19354, + "fixed_tokens": 1098, + "processed_tokens": 103239, + "tool_calls": 6, + "turns": 7 + }, + "world-079": { + "answer": [ + "con-no-runtime-code-generation-on-consoles--6cf25zqmsvqkdd73", + "con-tide-state-must-be-bit-identical-on-every-peer--zrpnp8qjyxzkas7t", + "con-saves-from-beta-onward-must-load-in-every-later--v6z856sxtgc5sb09", + "con-streaming-pool-budget--8czwb4fjr5p5vpy7" + ], + "tokens": 18887, + "fixed_tokens": 1101, + "processed_tokens": 102492, + "tool_calls": 6, + "turns": 7 + }, + "world-080": { + "answer": [ + "iss-gull-hot-reload-rejects-struct-layout-changes--3v2w2f9hg2gwy6aj", + "iss-gull-hot-reload-drops-suspended-coroutines--z7j2gmyny6gdxd07", + "iss-cross-platform-cloud-saves-are-unplanned--5bk0k4wrgj42ztfy" + ], + "tokens": 13214, + "fixed_tokens": 1105, + "processed_tokens": 55955, + "tool_calls": 4, + "turns": 5 + }, + "world-081": { + "answer": "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "tokens": 40360, + "fixed_tokens": 1108, + "processed_tokens": 533809, + "tool_calls": 35, + "turns": 20 + } + } +} diff --git a/benchmarks/proof-recall/runs/x10-sonnet-r1-find.json b/benchmarks/proof-recall/runs/x10-sonnet-r1-find.json new file mode 100644 index 00000000..23e58268 --- /dev/null +++ b/benchmarks/proof-recall/runs/x10-sonnet-r1-find.json @@ -0,0 +1,2234 @@ +{ + "run": { + "name": "x10-sonnet-r1-find", + "protocol": "one fresh session per question", + "models": ["claude-sonnet-5-5"], + "find_hash": "e5be653299abe9b2329822d98ffe09e90ee7cfe96f8711c813a51dc6500b887f" + }, + "answers": { + "ambiguous-001": { + "answer": "384 MB", + "tokens": 8040, + "fixed_tokens": 1115, + "processed_tokens": 57379, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-002": { + "answer": "99.6%", + "tokens": 8738, + "fixed_tokens": 1116, + "processed_tokens": 58904, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-003": { + "answer": "92.4%", + "tokens": 9449, + "fixed_tokens": 1961, + "processed_tokens": 58126, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-004": { + "answer": "6 min 40 s", + "tokens": 7243, + "fixed_tokens": 1964, + "processed_tokens": 41504, + "tool_calls": 4, + "turns": 5 + }, + "ambiguous-005": { + "answer": "6 weeks", + "tokens": 8013, + "fixed_tokens": 1113, + "processed_tokens": 45654, + "tool_calls": 5, + "turns": 5 + }, + "ambiguous-006": { + "answer": "16-bit float per channel", + "tokens": 15963, + "fixed_tokens": 1113, + "processed_tokens": 90093, + "tool_calls": 6, + "turns": 7 + }, + "ambiguous-007": { + "answer": "32 steps (dec-screen-space-refraction-with-a-ray-marched-fallb--2639xtpyrfrvbx1r)", + "tokens": 7325, + "fixed_tokens": 1114, + "processed_tokens": 33563, + "tool_calls": 3, + "turns": 4 + }, + "ambiguous-008": { + "answer": "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "tokens": 9662, + "fixed_tokens": 1113, + "processed_tokens": 60643, + "tool_calls": 6, + "turns": 6 + }, + "ambiguous-009": { + "answer": "3.0 ms", + "tokens": 6298, + "fixed_tokens": 1118, + "processed_tokens": 41253, + "tool_calls": 4, + "turns": 5 + }, + "ambiguous-010": { + "answer": "160 MB/s", + "tokens": 10420, + "fixed_tokens": 1118, + "processed_tokens": 63178, + "tool_calls": 7, + "turns": 6 + }, + "ambiguous-011": { + "answer": "260 MB", + "tokens": 7507, + "fixed_tokens": 1114, + "processed_tokens": 53928, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-012": { + "answer": "11 m", + "tokens": 5937, + "fixed_tokens": 1118, + "processed_tokens": 30864, + "tool_calls": 3, + "turns": 4 + }, + "ambiguous-013": { + "answer": "61%", + "tokens": 7350, + "fixed_tokens": 1117, + "processed_tokens": 53589, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-014": { + "answer": "textures (3184 MB)", + "tokens": 7593, + "fixed_tokens": 1117, + "processed_tokens": 54071, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-015": { + "answer": "38 KB", + "tokens": 22290, + "fixed_tokens": 1115, + "processed_tokens": 130007, + "tool_calls": 9, + "turns": 8 + }, + "ambiguous-016": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 7517, + "fixed_tokens": 1113, + "processed_tokens": 44573, + "tool_calls": 4, + "turns": 5 + }, + "ambiguous-017": { + "answer": "fnd-skiff-microsd-slows-sharply-on-full-warm-cards--p1rgh36hat2egpz3", + "tokens": 7560, + "fixed_tokens": 1117, + "processed_tokens": 34012, + "tool_calls": 3, + "turns": 4 + }, + "ambiguous-018": { + "answer": "fnd-retail-skiff-units-sustain-only-190-mb-s-of-stre--485sdcmm3gweevvn", + "tokens": 6120, + "fixed_tokens": 1118, + "processed_tokens": 31148, + "tool_calls": 3, + "turns": 4 + }, + "ambiguous-019": { + "answer": "7.4 ms", + "tokens": 7423, + "fixed_tokens": 1115, + "processed_tokens": 53742, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-020": { + "answer": "5912 pages", + "tokens": 7439, + "fixed_tokens": 1116, + "processed_tokens": 53521, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-021": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 8482, + "fixed_tokens": 1118, + "processed_tokens": 35900, + "tool_calls": 3, + "turns": 4 + }, + "ambiguous-022": { + "answer": "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "tokens": 8518, + "fixed_tokens": 1114, + "processed_tokens": 47620, + "tool_calls": 4, + "turns": 5 + }, + "ambiguous-023": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 7977, + "fixed_tokens": 1115, + "processed_tokens": 34888, + "tool_calls": 3, + "turns": 4 + }, + "ambiguous-024": { + "answer": "fnd-streaming-tide-heights-costs-46-kb-s-per-peer--knxqtyqh6xcz1tp9", + "tokens": 12353, + "fixed_tokens": 1115, + "processed_tokens": 55126, + "tool_calls": 4, + "turns": 5 + }, + "cross-001": { + "answer": "con-frame-budget--br0qszfg8fys908n", + "tokens": 5985, + "fixed_tokens": 1107, + "processed_tokens": 30931, + "tool_calls": 3, + "turns": 4 + }, + "cross-002": { + "answer": 2, + "tokens": 8300, + "fixed_tokens": 1108, + "processed_tokens": 41969, + "tool_calls": 4, + "turns": 5 + }, + "cross-003": { + "answer": 2, + "tokens": 4758, + "fixed_tokens": 1111, + "processed_tokens": 34921, + "tool_calls": 4, + "turns": 5 + }, + "cross-004": { + "answer": "dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e", + "tokens": 12350, + "fixed_tokens": 1106, + "processed_tokens": 78360, + "tool_calls": 6, + "turns": 7 + }, + "cross-005": { + "answer": "Rhea Valdés", + "tokens": 14104, + "fixed_tokens": 1109, + "processed_tokens": 115262, + "tool_calls": 8, + "turns": 9 + }, + "cross-006": { + "answer": "about 19% (4.8 ms of the 25 ms budget, roughly a fifth)", + "tokens": 7546, + "fixed_tokens": 1108, + "processed_tokens": 33986, + "tool_calls": 3, + "turns": 4 + }, + "cross-007": { + "answer": "8 GB", + "tokens": 15024, + "fixed_tokens": 1109, + "processed_tokens": 92406, + "tool_calls": 8, + "turns": 7 + }, + "cross-008": { + "answer": "unknown", + "tokens": 16745, + "fixed_tokens": 1104, + "processed_tokens": 108673, + "tool_calls": 9, + "turns": 8 + }, + "cross-009": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 12584, + "fixed_tokens": 1102, + "processed_tokens": 83402, + "tool_calls": 8, + "turns": 7 + }, + "cross-010": { + "answer": "dec-build-an-on-device-pipeline-cache-for-skiff--726r9mtk76qndeh7", + "tokens": 7695, + "fixed_tokens": 1107, + "processed_tokens": 34102, + "tool_calls": 3, + "turns": 4 + }, + "cross-011": { + "answer": false, + "tokens": 5863, + "fixed_tokens": 1108, + "processed_tokens": 30694, + "tool_calls": 3, + "turns": 4 + }, + "cross-012": { + "answer": "61 MB/s", + "tokens": 7245, + "fixed_tokens": 1108, + "processed_tokens": 33514, + "tool_calls": 3, + "turns": 4 + }, + "cross-013": { + "answer": "The Skiff port runs Tern at the Performance preset (dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3), which saves 1.8 ms over Quality (2.1 ms vs 3.9 ms).", + "tokens": 9896, + "fixed_tokens": 1109, + "processed_tokens": 48084, + "tool_calls": 5, + "turns": 5 + }, + "cross-014": { + "answer": 3, + "tokens": 3120, + "fixed_tokens": 1108, + "processed_tokens": 25200, + "tool_calls": 3, + "turns": 4 + }, + "cross-015": { + "answer": "Two Efforts are no longer active: \"Shader compile cache and build farm\" (eff-shader-compile-cache-and-build-farm--6xeac11zxvwwarkn) is completed, ending with dec-turn-the-shader-compile-cache-on-for-every-farm--4rr3b12b4sasdmaj, which turned the cache on for every farm build. \"Co-op rollback netcode prototype\" (eff-co-op-rollback-netcode-prototype--n5ejm416kfcfap49) is abandoned via dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b, because the tide simulation cannot stay in sync or be re-run affordably under rollback.", + "tokens": 11187, + "fixed_tokens": 1107, + "processed_tokens": 64628, + "tool_calls": 8, + "turns": 7 + }, + "cross-016": { + "answer": "16.7 ms", + "tokens": 13735, + "fixed_tokens": 1106, + "processed_tokens": 78920, + "tool_calls": 6, + "turns": 6 + }, + "cross-017": { + "answer": "unknown", + "tokens": 9978, + "fixed_tokens": 1108, + "processed_tokens": 63966, + "tool_calls": 5, + "turns": 6 + }, + "netcode-001": { + "answer": "abandoned", + "tokens": 5375, + "fixed_tokens": 1107, + "processed_tokens": 36800, + "tool_calls": 4, + "turns": 5 + }, + "netcode-002": { + "answer": "netcode/rollback-archive", + "tokens": 6872, + "fixed_tokens": 1108, + "processed_tokens": 32691, + "tool_calls": 3, + "turns": 4 + }, + "netcode-003": { + "answer": "38 KB", + "tokens": 8123, + "fixed_tokens": 1112, + "processed_tokens": 46649, + "tool_calls": 4, + "turns": 5 + }, + "netcode-004": { + "answer": "Rollback had to re-run the tide simulation every rolled-back frame, costing 4.8 ms on Skiff (a fifth of the 25 ms frame), and the workarounds failed: smaller windows stalled, quarter-resolution tides desynced, and leaving tides out of the snapshot broke the bit-for-bit constraint. The reason depends on the world Decision dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e (abandon decision: dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b).", + "tokens": 12553, + "fixed_tokens": 1112, + "processed_tokens": 78032, + "tool_calls": 6, + "turns": 7 + }, + "netcode-005": { + "answer": "unknown", + "tokens": 13947, + "fixed_tokens": 1108, + "processed_tokens": 70593, + "tool_calls": 5, + "turns": 6 + }, + "netcode-006": { + "answer": "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "tokens": 8017, + "fixed_tokens": 1111, + "processed_tokens": 55047, + "tool_calls": 5, + "turns": 6 + }, + "netcode-007": { + "answer": "dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b", + "tokens": 5867, + "fixed_tokens": 1110, + "processed_tokens": 30581, + "tool_calls": 3, + "turns": 4 + }, + "netcode-008": { + "answer": "dec-use-a-4-frame-rollback-window--pqe4hvbgmbp0dmh9", + "tokens": 6998, + "fixed_tokens": 1113, + "processed_tokens": 41635, + "tool_calls": 4, + "turns": 5 + }, + "netcode-009": { + "answer": "1.9 ms", + "tokens": 8044, + "fixed_tokens": 1111, + "processed_tokens": 57723, + "tool_calls": 5, + "turns": 6 + }, + "netcode-010": { + "answer": "dec-use-a-6-frame-rollback-window-with-1-frame-of-in--s53944zhkw103frk", + "tokens": 24702, + "fixed_tokens": 1108, + "processed_tokens": 192361, + "tool_calls": 10, + "turns": 11 + }, + "netcode-011": { + "answer": "con-co-op-peers-must-agree-on-tide-state-bit-for-bit--4v0sdbc3t4ywzr3g", + "tokens": 7561, + "fixed_tokens": 1112, + "processed_tokens": 34007, + "tool_calls": 3, + "turns": 4 + }, + "netcode-012": { + "answer": "dec-re-simulate-tides-at-quarter-resolution-during-r--7ah25p0r0xt9719f", + "tokens": 11101, + "fixed_tokens": 1112, + "processed_tokens": 94309, + "tool_calls": 9, + "turns": 9 + }, + "netcode-013": { + "answer": [ + "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "fnd-rollback-re-simulates-a-4-frame-window-in-1-1-ms--xa30a1cez2brphyf", + "rsk-re-simulation-cost-could-blow-the-skiff-frame-bu--sk2njsp170a3qwyf" + ], + "tokens": 18289, + "fixed_tokens": 1106, + "processed_tokens": 146496, + "tool_calls": 12, + "turns": 11 + }, + "netcode-014": { + "answer": [ + "iss-late-join-during-a-tide-surge-desyncs-the-joinin--mya93va0n7s8g7cj" + ], + "tokens": 8126, + "fixed_tokens": 1108, + "processed_tokens": 87532, + "tool_calls": 10, + "turns": 10 + }, + "netcode-015": { + "answer": 2, + "tokens": 5862, + "fixed_tokens": 1108, + "processed_tokens": 61557, + "tool_calls": 7, + "turns": 8 + }, + "ports-001": { + "answer": "25 ms", + "tokens": 6803, + "fixed_tokens": 1105, + "processed_tokens": 52249, + "tool_calls": 5, + "turns": 6 + }, + "ports-002": { + "answer": "384 MB", + "tokens": 7794, + "fixed_tokens": 1108, + "processed_tokens": 45599, + "tool_calls": 4, + "turns": 5 + }, + "ports-003": { + "answer": "61 MB/s", + "tokens": 8585, + "fixed_tokens": 1104, + "processed_tokens": 59115, + "tool_calls": 5, + "turns": 6 + }, + "ports-004": { + "answer": "accepted", + "tokens": 5976, + "fixed_tokens": 1106, + "processed_tokens": 40252, + "tool_calls": 4, + "turns": 5 + }, + "ports-005": { + "answer": "unknown", + "tokens": 17056, + "fixed_tokens": 1109, + "processed_tokens": 77893, + "tool_calls": 6, + "turns": 6 + }, + "ports-034": { + "answer": "6 weeks", + "tokens": 6651, + "fixed_tokens": 1102, + "processed_tokens": 51948, + "tool_calls": 5, + "turns": 6 + }, + "ports-006": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 8729, + "fixed_tokens": 1109, + "processed_tokens": 58813, + "tool_calls": 5, + "turns": 6 + }, + "ports-007": { + "answer": "SKF-UI-031", + "tokens": 7276, + "fixed_tokens": 1105, + "processed_tokens": 52983, + "tool_calls": 5, + "turns": 6 + }, + "ports-008": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 5938, + "fixed_tokens": 1107, + "processed_tokens": 30806, + "tool_calls": 3, + "turns": 4 + }, + "ports-009": { + "answer": "dec-load-skiff-fast-travel-without-a-loading-screen--v4qft5znysjv5f24", + "tokens": 7037, + "fixed_tokens": 1105, + "processed_tokens": 32946, + "tool_calls": 3, + "turns": 4 + }, + "ports-010": { + "answer": "con-frame-budget--wt5rfqh5z444g2cg", + "tokens": 13904, + "fixed_tokens": 1108, + "processed_tokens": 96693, + "tool_calls": 9, + "turns": 8 + }, + "ports-011": { + "answer": "99.6 percent", + "tokens": 5776, + "fixed_tokens": 1107, + "processed_tokens": 30510, + "tool_calls": 3, + "turns": 4 + }, + "ports-012": { + "answer": "They raised the floor to 576p (dec-raise-the-skiff-dynamic-resolution-floor-to-576p--fep9q7m08pp9f0bb) because caustics under saltglass shimmered badly in motion at the 540p floor even after Tern (fnd-saltglass-caustics-shimmer-at-the-540p-floor--0aa8sts89vhq1s6q: visible shimmer in 7 of 10 dune views). That decision was later superseded by a lowering to 504p.", + "tokens": 7674, + "fixed_tokens": 1112, + "processed_tokens": 53330, + "tool_calls": 5, + "turns": 6 + }, + "ports-013": { + "answer": "Ship Tern as Lantern's only upscaler", + "tokens": 6427, + "fixed_tokens": 1112, + "processed_tokens": 41472, + "tool_calls": 4, + "turns": 5 + }, + "ports-014": { + "answer": "dec-scale-skiff-resolution-dynamically-between-540p--jtytvgh2erd32x68", + "tokens": 13985, + "fixed_tokens": 1102, + "processed_tokens": 124404, + "tool_calls": 9, + "turns": 9 + }, + "ports-015": { + "answer": "textures", + "tokens": 7773, + "fixed_tokens": 1106, + "processed_tokens": 54812, + "tool_calls": 5, + "turns": 6 + }, + "ports-016": { + "answer": "dec-ship-prebuilt-pipeline-blobs-in-the-skiff-packag--3vjysjgy9qkvzebn", + "tokens": 9563, + "fixed_tokens": 1107, + "processed_tokens": 68048, + "tool_calls": 7, + "turns": 7 + }, + "ports-017": { + "answer": "fnd-skiff-microsd-slows-sharply-on-full-warm-cards--p1rgh36hat2egpz3", + "tokens": 6561, + "fixed_tokens": 1106, + "processed_tokens": 41780, + "tool_calls": 4, + "turns": 5 + }, + "ports-018": { + "answer": "504p", + "tokens": 10020, + "fixed_tokens": 1107, + "processed_tokens": 63758, + "tool_calls": 5, + "turns": 6 + }, + "ports-019": { + "answer": "4 MB per minute", + "tokens": 9089, + "fixed_tokens": 1107, + "processed_tokens": 79379, + "tool_calls": 7, + "turns": 8 + }, + "ports-020": { + "answer": "fnd-skiff-sdk-1-2-takes-1-gb-back-from-games--a0ybdgjzw64hwfqa", + "tokens": 11418, + "fixed_tokens": 1106, + "processed_tokens": 52779, + "tool_calls": 4, + "turns": 5 + }, + "ports-021": { + "answer": "SKF-DSP-004", + "tokens": 8372, + "fixed_tokens": 1102, + "processed_tokens": 57116, + "tool_calls": 5, + "turns": 6 + }, + "ports-022": { + "answer": "cit-skiff-sdk-1-2-release-notes--7gwwn8xxffrx6npe", + "tokens": 8611, + "fixed_tokens": 1108, + "processed_tokens": 78390, + "tool_calls": 7, + "turns": 8 + }, + "ports-023": { + "answer": "fnd-tern-costs-3-9-ms-at-quality-and-2-1-ms-at-perfo--dkbskbqfgfwtryq0", + "tokens": 9683, + "fixed_tokens": 1109, + "processed_tokens": 47918, + "tool_calls": 5, + "turns": 5 + }, + "ports-024": { + "answer": "fnd-skiff-firmware-1-2-rejects-pipeline-blobs-built--pernvgb2he7m3z3y", + "tokens": 7438, + "fixed_tokens": 1106, + "processed_tokens": 33706, + "tool_calls": 3, + "turns": 4 + }, + "ports-025": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 9242, + "fixed_tokens": 1104, + "processed_tokens": 57423, + "tool_calls": 7, + "turns": 6 + }, + "ports-026": { + "answer": "912 MB", + "tokens": 12226, + "fixed_tokens": 1107, + "processed_tokens": 152126, + "tool_calls": 12, + "turns": 13 + }, + "ports-027": { + "answer": [ + "dec-stream-skiff-world-tiles-in-2-mb-reads--3jz9w9gvhhc7eppw", + "rsk-tide-streaming-outruns-skiff-storage--6q8s3b383z2tyeyb" + ], + "tokens": 37164, + "fixed_tokens": 1109, + "processed_tokens": 306374, + "tool_calls": 18, + "turns": 14 + }, + "ports-028": { + "answer": 2, + "tokens": 7232, + "fixed_tokens": 1108, + "processed_tokens": 55311, + "tool_calls": 6, + "turns": 7 + }, + "ports-029": { + "answer": 8, + "tokens": 30164, + "fixed_tokens": 1109, + "processed_tokens": 680792, + "tool_calls": 32, + "turns": 31 + }, + "ports-030": { + "answer": [ + "dec-cap-tide-storm-particles-at-6-000-on-skiff--resze52afjxyxys1", + "dec-drop-skiff-to-30-hz-while-a-tide-storm-is-on-scr--72vx1c8s4jrs6axc" + ], + "tokens": 6054, + "fixed_tokens": 1105, + "processed_tokens": 45532, + "tool_calls": 5, + "turns": 6 + }, + "ports-031": { + "answer": [ + "iss-meridian-cert-activity-cards-missing-for-two-bio--k214cmfra7pyvvzq", + "iss-skiff-cert-docked-30-hz-judder-on-some-tvs--9nq2xg9mzash474g" + ], + "tokens": 14492, + "fixed_tokens": 1103, + "processed_tokens": 93567, + "tool_calls": 8, + "turns": 9 + }, + "ports-032": { + "answer": 2, + "tokens": 4287, + "fixed_tokens": 1111, + "processed_tokens": 41822, + "tool_calls": 5, + "turns": 6 + }, + "ports-033": { + "answer": "unknown", + "tokens": 8931, + "fixed_tokens": 1103, + "processed_tokens": 46481, + "tool_calls": 4, + "turns": 5 + }, + "render-001": { + "answer": "7.4 ms", + "tokens": 6089, + "fixed_tokens": 1105, + "processed_tokens": 31160, + "tool_calls": 3, + "turns": 4 + }, + "render-002": { + "answer": "3.4x", + "tokens": 5818, + "fixed_tokens": 1104, + "processed_tokens": 30601, + "tool_calls": 3, + "turns": 4 + }, + "render-003": { + "answer": "Tomasz Brandt", + "tokens": 7359, + "fixed_tokens": 1108, + "processed_tokens": 53238, + "tool_calls": 5, + "turns": 6 + }, + "render-004": { + "answer": "rejected", + "tokens": 6092, + "fixed_tokens": 1105, + "processed_tokens": 31148, + "tool_calls": 3, + "turns": 4 + }, + "render-005": { + "answer": true, + "tokens": 7653, + "fixed_tokens": 1103, + "processed_tokens": 45171, + "tool_calls": 4, + "turns": 5 + }, + "render-006": { + "answer": "unknown", + "tokens": 5999, + "fixed_tokens": 1105, + "processed_tokens": 30942, + "tool_calls": 3, + "turns": 4 + }, + "render-007": { + "answer": "2026-07-24", + "tokens": 6085, + "fixed_tokens": 1102, + "processed_tokens": 31106, + "tool_calls": 3, + "turns": 4 + }, + "render-008": { + "answer": "2 times (a third failure triggers full re-review)", + "tokens": 6246, + "fixed_tokens": 1106, + "processed_tokens": 40856, + "tool_calls": 4, + "turns": 5 + }, + "render-009": { + "answer": "32 steps", + "tokens": 7286, + "fixed_tokens": 1104, + "processed_tokens": 33525, + "tool_calls": 3, + "turns": 4 + }, + "render-010": { + "answer": "unknown", + "tokens": 9284, + "fixed_tokens": 1103, + "processed_tokens": 47152, + "tool_calls": 4, + "turns": 5 + }, + "render-011": { + "answer": "fnd-sdf-gi-costs-2-4-ms-on-meridian--2w6c2pc5zmcsyjre", + "tokens": 8834, + "fixed_tokens": 1103, + "processed_tokens": 36494, + "tool_calls": 3, + "turns": 4 + }, + "render-012": { + "answer": "deferred", + "tokens": 9980, + "fixed_tokens": 1104, + "processed_tokens": 87135, + "tool_calls": 7, + "turns": 8 + }, + "render-013": { + "answer": "16-bit float per channel", + "tokens": 10822, + "fixed_tokens": 1104, + "processed_tokens": 50225, + "tool_calls": 4, + "turns": 5 + }, + "render-014": { + "answer": "52 percent", + "tokens": 8165, + "fixed_tokens": 1106, + "processed_tokens": 56085, + "tool_calls": 5, + "turns": 6 + }, + "render-015": { + "answer": "4 minutes", + "tokens": 6358, + "fixed_tokens": 1103, + "processed_tokens": 41312, + "tool_calls": 4, + "turns": 5 + }, + "render-016": { + "answer": "1.2 km", + "tokens": 5484, + "fixed_tokens": 1101, + "processed_tokens": 29943, + "tool_calls": 3, + "turns": 4 + }, + "render-017": { + "answer": "unknown", + "tokens": 13268, + "fixed_tokens": 1103, + "processed_tokens": 83827, + "tool_calls": 7, + "turns": 7 + }, + "render-018": { + "answer": "unknown", + "tokens": 15083, + "fixed_tokens": 1107, + "processed_tokens": 86746, + "tool_calls": 7, + "turns": 7 + }, + "render-019": { + "answer": "dec-use-the-custom-brine-curve-for-tonemapping--p735g6wf2njare86", + "tokens": 4270, + "fixed_tokens": 1104, + "processed_tokens": 27434, + "tool_calls": 3, + "turns": 4 + }, + "render-020": { + "answer": "fnd-screen-space-refraction-loses-off-screen-content--k8zvmmdee7wg2fjy", + "tokens": 5737, + "fixed_tokens": 1102, + "processed_tokens": 30389, + "tool_calls": 3, + "turns": 4 + }, + "render-021": { + "answer": "dec-adopt-a-visibility-buffer-for-all-opaque-geometr--68vchzqgd1avw8q4", + "tokens": 7551, + "fixed_tokens": 1103, + "processed_tokens": 33919, + "tool_calls": 3, + "turns": 4 + }, + "render-022": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 7439, + "fixed_tokens": 1104, + "processed_tokens": 52635, + "tool_calls": 5, + "turns": 6 + }, + "render-023": { + "answer": "dec-capture-the-cliffs-and-dunes-benchmarks-on-every--3pqyr66yh4v6q676", + "tokens": 5740, + "fixed_tokens": 1107, + "processed_tokens": 30369, + "tool_calls": 3, + "turns": 4 + }, + "render-024": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 7482, + "fixed_tokens": 1104, + "processed_tokens": 33850, + "tool_calls": 3, + "turns": 4 + }, + "render-025": { + "answer": "fnd-tern-cost-at-4k-output-on-meridian-after-fp16-pa--p4gcmzw2a4bvzjev", + "tokens": 9097, + "fixed_tokens": 1102, + "processed_tokens": 37077, + "tool_calls": 3, + "turns": 4 + }, + "render-026": { + "answer": "dec-ship-four-tern-quality-presets--vjw18d82dx5gath8", + "tokens": 5870, + "fixed_tokens": 1103, + "processed_tokens": 30672, + "tool_calls": 3, + "turns": 4 + }, + "render-027": { + "answer": "con-frame-budget--j6vneva1g2ywj86m", + "tokens": 7856, + "fixed_tokens": 1103, + "processed_tokens": 81824, + "tool_calls": 8, + "turns": 9 + }, + "render-028": { + "answer": "dec-move-to-gpu-driven-two-phase-occlusion-culling--6dre481mabxsw6ee", + "tokens": 6997, + "fixed_tokens": 1099, + "processed_tokens": 32904, + "tool_calls": 3, + "turns": 4 + }, + "render-029": { + "answer": "fnd-vsm-page-pool-overflows-during-spring-tides--m2p8e89hhdmh0z8a", + "tokens": 5215, + "fixed_tokens": 1106, + "processed_tokens": 29300, + "tool_calls": 3, + "turns": 4 + }, + "render-030": { + "answer": "dec-use-screen-space-refraction-for-saltglass--vknzksgdrgr7zkg6", + "tokens": 14357, + "fixed_tokens": 1099, + "processed_tokens": 86524, + "tool_calls": 8, + "turns": 7 + }, + "render-031": { + "answer": "In dec-visibility-buffer-for-opaque-geometry-foliage-an--gvnvp04jgm7kbfh9, alpha-tested foliage cost 2.1 ms in the visibility buffer (fnd-alpha-tested-foliage-costs-2-1-ms-in-the-visibil--f3g34x2k6fs367eb), so it moved to a forward pass after the resolve, where it costs 0.8 ms. Saltglass moved there too because it needs the resolved scene to refract.", + "tokens": 7657, + "fixed_tokens": 1099, + "processed_tokens": 53215, + "tool_calls": 5, + "turns": 6 + }, + "render-032": { + "answer": "Per dec-ship-tern-as-lantern-s-only-upscaler--fmreq35nzyjc7k4e: Halcyon has no Skiff build and shipping it on only two platforms would break the one-upscaler-path Constraint, and owning Tern's code matters more for saltglass than raw quality (refraction and caustics need upscaler changes the team can make the same week, not vendor tickets, with no license to negotiate) despite Tern costing 0.7 ms more on Meridian.", + "tokens": 7854, + "fixed_tokens": 1102, + "processed_tokens": 53231, + "tool_calls": 5, + "turns": 6 + }, + "render-033": { + "answer": "128 MB", + "tokens": 7178, + "fixed_tokens": 1101, + "processed_tokens": 51816, + "tool_calls": 6, + "turns": 6 + }, + "render-034": { + "answer": "PC workstation (Meridian clock profile)", + "tokens": 7802, + "fixed_tokens": 1107, + "processed_tokens": 55183, + "tool_calls": 5, + "turns": 6 + }, + "render-035": { + "answer": "5912 pages", + "tokens": 7688, + "fixed_tokens": 1102, + "processed_tokens": 54490, + "tool_calls": 5, + "turns": 6 + }, + "render-036": { + "answer": "HDR-07", + "tokens": 7117, + "fixed_tokens": 1107, + "processed_tokens": 52550, + "tool_calls": 5, + "turns": 6 + }, + "render-037": { + "answer": "0.33", + "tokens": 11582, + "fixed_tokens": 1101, + "processed_tokens": 121540, + "tool_calls": 9, + "turns": 10 + }, + "render-038": { + "answer": "dec-visibility-buffer-for-opaque-geometry-foliage-an--gvnvp04jgm7kbfh9", + "tokens": 8379, + "fixed_tokens": 1103, + "processed_tokens": 54514, + "tool_calls": 5, + "turns": 6 + }, + "render-039": { + "answer": "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "tokens": 7309, + "fixed_tokens": 1101, + "processed_tokens": 50123, + "tool_calls": 7, + "turns": 6 + }, + "render-040": { + "answer": "fnd-lantern-at-alpha--ckpd9ecvdjqbcr7c", + "tokens": 7527, + "fixed_tokens": 1105, + "processed_tokens": 66841, + "tool_calls": 8, + "turns": 8 + }, + "render-041": { + "answer": "4.7 ms", + "tokens": 8320, + "fixed_tokens": 1103, + "processed_tokens": 35622, + "tool_calls": 3, + "turns": 4 + }, + "render-042": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 7762, + "fixed_tokens": 1102, + "processed_tokens": 53940, + "tool_calls": 5, + "turns": 6 + }, + "render-043": { + "answer": "fnd-meshlet-cone-culling-rejects-9-percent-of-triang--t6e5x7trzedex105", + "tokens": 7167, + "fixed_tokens": 1104, + "processed_tokens": 33256, + "tool_calls": 3, + "turns": 4 + }, + "render-044": { + "answer": "dec-screen-space-refraction-with-a-ray-marched-fallb--2639xtpyrfrvbx1r", + "tokens": 5768, + "fixed_tokens": 1105, + "processed_tokens": 30370, + "tool_calls": 3, + "turns": 4 + }, + "render-045": { + "answer": false, + "tokens": 7818, + "fixed_tokens": 1102, + "processed_tokens": 34587, + "tool_calls": 3, + "turns": 4 + }, + "render-046": { + "answer": "dec-use-virtual-shadow-maps-for-the-sun--x8nqkn9q7eaf1ap4 was reopened (2026-01-12) because the 3.1 ms cost finding that sank it was a capture error (debug flag discarded cached pages); fnd-virtual-shadow-maps-cost-1-4-ms-on-meridian-with--mwpybda1vh0ac55e shows VSM costs 1.4 ms with cached pages, less than the 1.6 ms for four cascades, and cascade seams through saltglass remained unsolved under CSM.", + "tokens": 9629, + "fixed_tokens": 1106, + "processed_tokens": 58013, + "tool_calls": 7, + "turns": 6 + }, + "render-047": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1: a release-build capture showed the visibility buffer at 7.4 ms on Skiff versus 8.1 ms for forward+ (6.3 ms with half-rate material resolve), so the forward+ fallback was no longer worth keeping, and maintaining two opaque paths cost about one engineer-week a month.", + "tokens": 10352, + "fixed_tokens": 1108, + "processed_tokens": 63982, + "tool_calls": 5, + "turns": 6 + }, + "render-048": { + "answer": "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "tokens": 7339, + "fixed_tokens": 1106, + "processed_tokens": 33529, + "tool_calls": 3, + "turns": 4 + }, + "render-049": { + "answer": "dec-use-virtual-shadow-maps-for-the-sun--x8nqkn9q7eaf1ap4", + "tokens": 9059, + "fixed_tokens": 1110, + "processed_tokens": 70095, + "tool_calls": 6, + "turns": 7 + }, + "render-050": { + "answer": "con-frame-budget--fzdrbxns6dmrrmy3", + "tokens": 5140, + "fixed_tokens": 1106, + "processed_tokens": 36242, + "tool_calls": 4, + "turns": 5 + }, + "render-051": { + "answer": "iss-tern-ghosting-on-moving-saltglass--2yr3q6zf476wajqp", + "tokens": 8495, + "fixed_tokens": 1102, + "processed_tokens": 35795, + "tool_calls": 3, + "turns": 4 + }, + "render-052": { + "answer": "61 percent", + "tokens": 7484, + "fixed_tokens": 1101, + "processed_tokens": 44705, + "tool_calls": 4, + "turns": 5 + }, + "render-053": { + "answer": "every frame", + "tokens": 7484, + "fixed_tokens": 1104, + "processed_tokens": 53923, + "tool_calls": 5, + "turns": 6 + }, + "render-054": { + "answer": "debug-gpu-validation", + "tokens": 14408, + "fixed_tokens": 1105, + "processed_tokens": 165550, + "tool_calls": 11, + "turns": 12 + }, + "render-055": { + "answer": "fnd-tern-cost-at-4k-output-on-meridian-with-the-reac--81tcpzvhwzcy6wyn", + "tokens": 10240, + "fixed_tokens": 1102, + "processed_tokens": 63992, + "tool_calls": 5, + "turns": 6 + }, + "render-056": { + "answer": "208,000 meshlets", + "tokens": 10540, + "fixed_tokens": 1105, + "processed_tokens": 134964, + "tool_calls": 11, + "turns": 12 + }, + "render-057": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 9105, + "fixed_tokens": 1104, + "processed_tokens": 60552, + "tool_calls": 5, + "turns": 6 + }, + "render-058": { + "answer": "iss-which-upscaler-should-lantern-ship--g8c2y9tsr856tn2v", + "tokens": 9431, + "fixed_tokens": 1102, + "processed_tokens": 89095, + "tool_calls": 9, + "turns": 9 + }, + "render-059": { + "answer": "fnd-sdf-gi-costs-4-7-ms-on-meridian--sm19rt2f274ajsvr", + "tokens": 8851, + "fixed_tokens": 1103, + "processed_tokens": 56814, + "tool_calls": 5, + "turns": 6 + }, + "render-060": { + "answer": "0.31", + "tokens": 9759, + "fixed_tokens": 1102, + "processed_tokens": 104806, + "tool_calls": 9, + "turns": 10 + }, + "render-061": { + "answer": "con-ui-composites-at-a-fixed-203-nits-in-hdr--de6aft7cvhdezn5s", + "tokens": 9269, + "fixed_tokens": 1102, + "processed_tokens": 101129, + "tool_calls": 9, + "turns": 10 + }, + "render-062": { + "answer": "con-frame-budget--j6vneva1g2ywj86m", + "tokens": 9791, + "fixed_tokens": 1105, + "processed_tokens": 69909, + "tool_calls": 7, + "turns": 7 + }, + "render-063": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 5150, + "fixed_tokens": 1105, + "processed_tokens": 29181, + "tool_calls": 3, + "turns": 4 + }, + "render-064": { + "answer": "dec-use-a-cached-shadow-atlas-for-local-lights--8p6f2v774he38t5j", + "tokens": 8796, + "fixed_tokens": 1105, + "processed_tokens": 60382, + "tool_calls": 5, + "turns": 6 + }, + "render-065": { + "answer": "dec-ship-tern-as-lantern-s-only-upscaler--fmreq35nzyjc7k4e", + "tokens": 8737, + "fixed_tokens": 1103, + "processed_tokens": 78229, + "tool_calls": 7, + "turns": 8 + }, + "render-066": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 9166, + "fixed_tokens": 1110, + "processed_tokens": 79751, + "tool_calls": 7, + "turns": 8 + }, + "render-067": { + "answer": 4, + "tokens": 19411, + "fixed_tokens": 1102, + "processed_tokens": 99832, + "tool_calls": 8, + "turns": 8 + }, + "render-068": { + "answer": [ + "dec-read-paper-white-from-the-meridian-system-settin--7crbx73gqw1hn7xq", + "dec-add-an-in-game-paper-white-slider-from-80-to-400--1phfvxby983z8tjf", + "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "dec-lock-caustic-photon-jitter-to-the-tide-phase--85x10k4h11t8nx7c", + "dec-use-the-probe-grid-alone-in-120-hz-performance-m--6dzm1nt40pjfpyae", + "dec-trace-sdf-gi-at-quarter-resolution-in-performanc--06cqt3mqn3fg5h6e" + ], + "tokens": 43395, + "fixed_tokens": 1107, + "processed_tokens": 334581, + "tool_calls": 24, + "turns": 14 + }, + "render-069": { + "answer": "iss-shadow-pages-run-out-at-spring-low-tide--f4fgy6b44jbkrsmh", + "tokens": 18669, + "fixed_tokens": 1106, + "processed_tokens": 158473, + "tool_calls": 10, + "turns": 11 + }, + "render-070": { + "answer": [ + "dec-feed-a-saltglass-reactive-mask-into-tern--xezjs7v3vq3r8e6k", + "dec-enable-meshlet-cone-culling-on-all-platforms--cpyweprmzcqhqsv2", + "dec-use-signed-distance-field-gi--yn366zmp3k7snd9w" + ], + "tokens": 68546, + "fixed_tokens": 1105, + "processed_tokens": 913182, + "tool_calls": 27, + "turns": 25 + }, + "render-071": { + "answer": 3, + "tokens": 5076, + "fixed_tokens": 1107, + "processed_tokens": 51063, + "tool_calls": 6, + "turns": 7 + }, + "render-072": { + "answer": [ + "rsk-sdf-volumes-outgrow-the-render-target-memory-bud--xfd2pfztvskdcd2c", + "rsk-artists-cannot-match-concept-paintings-under-the--6cfh9vkamn0kbtcx" + ], + "tokens": 8203, + "fixed_tokens": 1107, + "processed_tokens": 73826, + "tool_calls": 8, + "turns": 9 + }, + "render-073": { + "answer": [ + "rsk-vsm-page-pool-overflows-when-tides-expose-the-fl--x1r161nrdngra4ty", + "rsk-gi-cannot-fit-the-120-hz-performance-mode--tqt3fj839dqfq5s7", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 7324, + "fixed_tokens": 1106, + "processed_tokens": 63947, + "tool_calls": 7, + "turns": 8 + }, + "render-074": { + "answer": 5, + "tokens": 4355, + "fixed_tokens": 1107, + "processed_tokens": 41882, + "tool_calls": 5, + "turns": 6 + }, + "render-075": { + "answer": 3, + "tokens": 18399, + "fixed_tokens": 1104, + "processed_tokens": 347354, + "tool_calls": 22, + "turns": 23 + }, + "render-076": { + "answer": false, + "tokens": 7062, + "fixed_tokens": 1101, + "processed_tokens": 51633, + "tool_calls": 5, + "turns": 6 + }, + "render-077": { + "answer": true, + "tokens": 13854, + "fixed_tokens": 1105, + "processed_tokens": 118552, + "tool_calls": 11, + "turns": 10 + }, + "render-078": { + "answer": [ + "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 19350, + "fixed_tokens": 1104, + "processed_tokens": 136119, + "tool_calls": 11, + "turns": 10 + }, + "render-079": { + "answer": 5, + "tokens": 20669, + "fixed_tokens": 1106, + "processed_tokens": 340672, + "tool_calls": 21, + "turns": 22 + }, + "render-080": { + "answer": 6, + "tokens": 5484, + "fixed_tokens": 1104, + "processed_tokens": 29871, + "tool_calls": 3, + "turns": 4 + }, + "render-081": { + "answer": "unknown", + "tokens": 15776, + "fixed_tokens": 1106, + "processed_tokens": 74547, + "tool_calls": 5, + "turns": 6 + }, + "render-082": { + "answer": false, + "tokens": 4442, + "fixed_tokens": 1105, + "processed_tokens": 27638, + "tool_calls": 3, + "turns": 4 + }, + "render-083": { + "answer": 5, + "tokens": 73624, + "fixed_tokens": 1105, + "processed_tokens": 1141103, + "tool_calls": 34, + "turns": 30 + }, + "tools-001": { + "answer": "completed", + "tokens": 7296, + "fixed_tokens": 1105, + "processed_tokens": 43900, + "tool_calls": 5, + "turns": 5 + }, + "tools-002": { + "answer": "47 minutes", + "tokens": 7009, + "fixed_tokens": 1109, + "processed_tokens": 41566, + "tool_calls": 5, + "turns": 5 + }, + "tools-003": { + "answer": "6 min 40 s", + "tokens": 5987, + "fixed_tokens": 1107, + "processed_tokens": 30952, + "tool_calls": 3, + "turns": 4 + }, + "tools-004": { + "answer": "unknown", + "tokens": 24878, + "fixed_tokens": 1105, + "processed_tokens": 282341, + "tool_calls": 14, + "turns": 15 + }, + "tools-005": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 9635, + "fixed_tokens": 1110, + "processed_tokens": 61475, + "tool_calls": 5, + "turns": 6 + }, + "tools-006": { + "answer": "dec-build-a-content-addressed-shader-compile-cache-s--e65rq9439k7xz5zh", + "tokens": 7371, + "fixed_tokens": 1108, + "processed_tokens": 33628, + "tool_calls": 3, + "turns": 4 + }, + "tools-007": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 16627, + "fixed_tokens": 1113, + "processed_tokens": 133005, + "tool_calls": 12, + "turns": 11 + }, + "tools-008": { + "answer": "dec-key-the-shader-cache-on-raw-source-text-and-comp--hp6awmx3gq5rn9nc", + "tokens": 8838, + "fixed_tokens": 1108, + "processed_tokens": 60330, + "tool_calls": 5, + "turns": 6 + }, + "tools-009": { + "answer": "92.4%", + "tokens": 8536, + "fixed_tokens": 1105, + "processed_tokens": 57829, + "tool_calls": 5, + "turns": 6 + }, + "tools-010": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 7652, + "fixed_tokens": 1106, + "processed_tokens": 34142, + "tool_calls": 3, + "turns": 4 + }, + "tools-011": { + "answer": "fnd-two-compiler-builds-with-the-same-version-string--gmgr4ha0ndxqmcn9", + "tokens": 11739, + "fixed_tokens": 1107, + "processed_tokens": 65560, + "tool_calls": 6, + "turns": 6 + }, + "tools-012": { + "answer": 3, + "tokens": 5800, + "fixed_tokens": 1111, + "processed_tokens": 61484, + "tool_calls": 7, + "turns": 8 + }, + "tools-013": { + "answer": 7, + "tokens": 41026, + "fixed_tokens": 1106, + "processed_tokens": 660951, + "tool_calls": 30, + "turns": 30 + }, + "tools-014": { + "answer": false, + "tokens": 9165, + "fixed_tokens": 1108, + "processed_tokens": 55590, + "tool_calls": 5, + "turns": 6 + }, + "tools-015": { + "answer": [ + "iss-developer-workstations-cannot-read-the-farm-shad--q2xw0f7fze6z55nj", + "rsk-shader-cache-storage-could-outgrow-the-4-tb-farm--76y9649mwrx7aavd" + ], + "tokens": 9345, + "fixed_tokens": 1104, + "processed_tokens": 93310, + "tool_calls": 10, + "turns": 10 + }, + "tools-016": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 12795, + "fixed_tokens": 1106, + "processed_tokens": 75670, + "tool_calls": 6, + "turns": 7 + }, + "world-001": { + "answer": "64 KB", + "tokens": 6724, + "fixed_tokens": 1102, + "processed_tokens": 51586, + "tool_calls": 5, + "turns": 6 + }, + "world-002": { + "answer": "5.1 ms", + "tokens": 6352, + "fixed_tokens": 1105, + "processed_tokens": 41292, + "tool_calls": 4, + "turns": 5 + }, + "world-003": { + "answer": "1 crash per 900 script waits", + "tokens": 7645, + "fixed_tokens": 1104, + "processed_tokens": 53938, + "tool_calls": 5, + "turns": 6 + }, + "world-004": { + "answer": "4,100", + "tokens": 5859, + "fixed_tokens": 1103, + "processed_tokens": 30704, + "tool_calls": 3, + "turns": 4 + }, + "world-005": { + "answer": "11 m", + "tokens": 7330, + "fixed_tokens": 1104, + "processed_tokens": 53056, + "tool_calls": 5, + "turns": 6 + }, + "world-006": { + "answer": "3.4 times per minute", + "tokens": 5990, + "fixed_tokens": 1098, + "processed_tokens": 30922, + "tool_calls": 3, + "turns": 4 + }, + "world-007": { + "answer": "260 MB", + "tokens": 8436, + "fixed_tokens": 1105, + "processed_tokens": 89476, + "tool_calls": 8, + "turns": 9 + }, + "world-008": { + "answer": "10 Hz", + "tokens": 6346, + "fixed_tokens": 1109, + "processed_tokens": 41316, + "tool_calls": 4, + "turns": 5 + }, + "world-009": { + "answer": "0.3 ms", + "tokens": 5685, + "fixed_tokens": 1106, + "processed_tokens": 30333, + "tool_calls": 3, + "turns": 4 + }, + "world-010": { + "answer": "29 GB/s", + "tokens": 5169, + "fixed_tokens": 1105, + "processed_tokens": 29298, + "tool_calls": 3, + "turns": 4 + }, + "world-011": { + "answer": "fnd-shipping-entity-names-costs-57-mb-at-full-popula--6w7gjvy7wdmdx9qx", + "tokens": 7288, + "fixed_tokens": 1102, + "processed_tokens": 44017, + "tool_calls": 4, + "turns": 5 + }, + "world-012": { + "answer": "dec-run-engine-jobs-and-gull-scripts-on-one-thread-p--07pbzvqg00pmnrr3", + "tokens": 6846, + "fixed_tokens": 1103, + "processed_tokens": 32569, + "tool_calls": 3, + "turns": 4 + }, + "world-013": { + "answer": "unknown", + "tokens": 10389, + "fixed_tokens": 1105, + "processed_tokens": 69627, + "tool_calls": 6, + "turns": 7 + }, + "world-014": { + "answer": "unknown", + "tokens": 20048, + "fixed_tokens": 1104, + "processed_tokens": 106736, + "tool_calls": 8, + "turns": 8 + }, + "world-015": { + "answer": "3.0 ms", + "tokens": 6125, + "fixed_tokens": 1104, + "processed_tokens": 40707, + "tool_calls": 4, + "turns": 5 + }, + "world-016": { + "answer": "unknown", + "tokens": 6002, + "fixed_tokens": 1104, + "processed_tokens": 47128, + "tool_calls": 5, + "turns": 6 + }, + "world-017": { + "answer": "dec-partition-the-world-into-128-m-cells--372b6czbjf29ymaw", + "tokens": 9169, + "fixed_tokens": 1103, + "processed_tokens": 61470, + "tool_calls": 5, + "turns": 6 + }, + "world-018": { + "answer": "dec-run-engine-jobs-on-a-work-stealing-thread-pool--1pd72x1pxpgwy21x", + "tokens": 5930, + "fixed_tokens": 1107, + "processed_tokens": 30689, + "tool_calls": 3, + "turns": 4 + }, + "world-019": { + "answer": "fnd-tide-priority-reads-starve-player-proximity-load--by0d7b0bk39xz3rk", + "tokens": 7730, + "fixed_tokens": 1102, + "processed_tokens": 34371, + "tool_calls": 3, + "turns": 4 + }, + "world-020": { + "answer": "dec-schedule-streaming-reads-by-deadline-instead-of--q2x5aknabgcssdj2", + "tokens": 7618, + "fixed_tokens": 1098, + "processed_tokens": 44991, + "tool_calls": 4, + "turns": 5 + }, + "world-021": { + "answer": "dec-cut-navigation-tiles-to-64-m-so-three-fit-along--snvepk8ygeb3qsjs", + "tokens": 7580, + "fixed_tokens": 1104, + "processed_tokens": 34069, + "tool_calls": 3, + "turns": 4 + }, + "world-022": { + "answer": "fnd-41-of-alpha-crash-reports-have-no-usable-stack--35a50b07ferhf13m", + "tokens": 4320, + "fixed_tokens": 1108, + "processed_tokens": 27546, + "tool_calls": 3, + "turns": 4 + }, + "world-023": { + "answer": "dec-compress-streamed-assets-with-kelp--ezwc5p1tgsgm7v8m", + "tokens": 9979, + "fixed_tokens": 1104, + "processed_tokens": 62842, + "tool_calls": 6, + "turns": 6 + }, + "world-024": { + "answer": "dec-shrink-kelp-s-dictionary-to-1-mb-for-streamed-as--yh8hktfqnyqhepew", + "tokens": 7630, + "fixed_tokens": 1101, + "processed_tokens": 34155, + "tool_calls": 3, + "turns": 4 + }, + "world-025": { + "answer": "con-streaming-pool-budget--kca88sgh2j6sxvd5", + "tokens": 5804, + "fixed_tokens": 1100, + "processed_tokens": 30404, + "tool_calls": 3, + "turns": 4 + }, + "world-026": { + "answer": "fnd-retail-skiff-units-sustain-only-190-mb-s-of-stre--485sdcmm3gweevvn", + "tokens": 7244, + "fixed_tokens": 1102, + "processed_tokens": 33395, + "tool_calls": 3, + "turns": 4 + }, + "world-027": { + "answer": "160 MB/s", + "tokens": 7246, + "fixed_tokens": 1106, + "processed_tokens": 53055, + "tool_calls": 5, + "turns": 6 + }, + "world-028": { + "answer": "fnd-streaming-tide-heights-costs-46-kb-s-per-peer--knxqtyqh6xcz1tp9", + "tokens": 5932, + "fixed_tokens": 1108, + "processed_tokens": 30779, + "tool_calls": 3, + "turns": 4 + }, + "world-029": { + "answer": "dec-return-engine-jobs-to-a-thread-pool-with-continu--1zkc1v2x97taf50h", + "tokens": 6515, + "fixed_tokens": 1107, + "processed_tokens": 41564, + "tool_calls": 4, + "turns": 5 + }, + "world-030": { + "answer": "fnd-world-core-state-at-alpha--tkbsm4867h65jqq8", + "tokens": 6016, + "fixed_tokens": 1104, + "processed_tokens": 30909, + "tool_calls": 3, + "turns": 4 + }, + "world-031": { + "answer": "1.5 GB/s", + "tokens": 9411, + "fixed_tokens": 1106, + "processed_tokens": 61218, + "tool_calls": 5, + "turns": 6 + }, + "world-032": { + "answer": "fnd-float-tide-heights-drift-apart-between-pc-and-me--7jb0fg4b5xzr7q5e", + "tokens": 8837, + "fixed_tokens": 1106, + "processed_tokens": 56142, + "tool_calls": 7, + "turns": 6 + }, + "world-033": { + "answer": "dec-save-player-changes-as-deltas-against-the-tide-f--n38scy1yaarsr8cw", + "tokens": 11670, + "fixed_tokens": 1107, + "processed_tokens": 67078, + "tool_calls": 7, + "turns": 6 + }, + "world-034": { + "answer": "Deferred to a post-launch patch (platform agreements missing)", + "tokens": 10818, + "fixed_tokens": 1107, + "processed_tokens": 71103, + "tool_calls": 7, + "turns": 7 + }, + "world-035": { + "answer": false, + "tokens": 6066, + "fixed_tokens": 1105, + "processed_tokens": 31061, + "tool_calls": 3, + "turns": 4 + }, + "world-036": { + "answer": "Per dec-return-engine-jobs-to-a-thread-pool-with-continu--1zkc1v2x97taf50h (2026-04-14), fibers made crashes unreadable (41% of alpha crash reports had no usable stack, blocking beta crash triage) and caused the Gull thread-local bug, while a continuation-based thread pool kept workers 93% busy vs. 95% with fibers, so the small utilisation loss was worth it.", + "tokens": 7924, + "fixed_tokens": 1104, + "processed_tokens": 53964, + "tool_calls": 5, + "turns": 6 + }, + "world-037": { + "answer": "fnd-hot-reload-drops-suspended-gull-coroutines--va0fn0svryzwbzws", + "tokens": 7484, + "fixed_tokens": 1106, + "processed_tokens": 33873, + "tool_calls": 3, + "turns": 4 + }, + "world-038": { + "answer": 3, + "tokens": 9062, + "fixed_tokens": 1106, + "processed_tokens": 56899, + "tool_calls": 7, + "turns": 6 + }, + "world-039": { + "answer": "1.2 GB/s", + "tokens": 7504, + "fixed_tokens": 1105, + "processed_tokens": 53931, + "tool_calls": 5, + "turns": 6 + }, + "world-040": { + "answer": "unknown", + "tokens": 6916, + "fixed_tokens": 1101, + "processed_tokens": 53224, + "tool_calls": 5, + "turns": 6 + }, + "world-041": { + "answer": "61%", + "tokens": 7504, + "fixed_tokens": 1103, + "processed_tokens": 53918, + "tool_calls": 5, + "turns": 6 + }, + "world-042": { + "answer": "5.4 s", + "tokens": 5794, + "fixed_tokens": 1102, + "processed_tokens": 30556, + "tool_calls": 3, + "turns": 4 + }, + "world-043": { + "answer": 5.8, + "tokens": 7561, + "fixed_tokens": 1105, + "processed_tokens": 53789, + "tool_calls": 5, + "turns": 6 + }, + "world-044": { + "answer": "dec-move-engine-jobs-to-fibers--qjkgnm8cex8ydzfq", + "tokens": 10057, + "fixed_tokens": 1103, + "processed_tokens": 60069, + "tool_calls": 5, + "turns": 6 + }, + "world-045": { + "answer": "dec-run-engine-jobs-on-fibers-and-pin-gull-scripts-t--rw28cbqfd07rd9te", + "tokens": 9931, + "fixed_tokens": 1104, + "processed_tokens": 92160, + "tool_calls": 8, + "turns": 9 + }, + "world-046": { + "answer": "con-streaming-pool-budget--632bkbs79bwj4cr5", + "tokens": 10924, + "fixed_tokens": 1103, + "processed_tokens": 65297, + "tool_calls": 6, + "turns": 6 + }, + "world-047": { + "answer": "fnd-world-core-state-at-alpha--tkbsm4867h65jqq8", + "tokens": 8246, + "fixed_tokens": 1102, + "processed_tokens": 77777, + "tool_calls": 8, + "turns": 9 + }, + "world-048": { + "answer": "dec-compile-gull-to-bytecode-for-a-register-vm--jrqje646fpevp5f3", + "tokens": 7952, + "fixed_tokens": 1107, + "processed_tokens": 45925, + "tool_calls": 4, + "turns": 5 + }, + "world-049": { + "answer": "22%", + "tokens": 5990, + "fixed_tokens": 1102, + "processed_tokens": 30992, + "tool_calls": 3, + "turns": 4 + }, + "world-050": { + "answer": "fnd-alpha-content-needs-6-2-ms-of-gull-time-per-fram--4mekgdpsdv5nk085", + "tokens": 18083, + "fixed_tokens": 1102, + "processed_tokens": 117132, + "tool_calls": 8, + "turns": 8 + }, + "world-051": { + "answer": "Strict IEEE float mode (dec-force-strict-ieee-float-mode-for-the-tide-simula--ecwmqt7ncjngysqw) was rejected because it only fixes the compiler, not the math library: with strict flags PC and Skiff still drifted past 1 cm at minute 11, since Skiff's sin() differs by one ulp (fnd-strict-float-mode-still-diverges-because-skiff-s--z811r28v0a1wzb13). So dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e chose 32.32 fixed point with a build-time sine table, giving bit-identical tides on every peer without any platform math library.", + "tokens": 9619, + "fixed_tokens": 1106, + "processed_tokens": 57359, + "tool_calls": 7, + "turns": 6 + }, + "world-052": { + "answer": "Dara Whitlock", + "tokens": 10736, + "fixed_tokens": 1105, + "processed_tokens": 87456, + "tool_calls": 8, + "turns": 8 + }, + "world-053": { + "answer": "1.9 ms", + "tokens": 10645, + "fixed_tokens": 1104, + "processed_tokens": 63263, + "tool_calls": 7, + "turns": 6 + }, + "world-054": { + "answer": "Marta Lindqvist", + "tokens": 9909, + "fixed_tokens": 1103, + "processed_tokens": 63684, + "tool_calls": 5, + "turns": 6 + }, + "world-055": { + "answer": "dec-shrink-kelp-s-dictionary-to-1-mb-for-streamed-as--yh8hktfqnyqhepew: after the cold-cache rerun (Kelp 1.1 vs Zstandard 1.3 GB/s with the 4 MB dictionary), the team kept Kelp but shrank its dictionary to 1 MB on every platform, which fixes the cache thrash on Skiff, and closed the reopened Zstandard option.", + "tokens": 8153, + "fixed_tokens": 1102, + "processed_tokens": 55223, + "tool_calls": 5, + "turns": 6 + }, + "world-056": { + "answer": "dec-run-engine-jobs-on-a-work-stealing-thread-pool--1pd72x1pxpgwy21x", + "tokens": 6560, + "fixed_tokens": 1110, + "processed_tokens": 49157, + "tool_calls": 5, + "turns": 6 + }, + "world-057": { + "answer": "(212, 87)", + "tokens": 11601, + "fixed_tokens": 1106, + "processed_tokens": 114291, + "tool_calls": 9, + "turns": 10 + }, + "world-058": { + "answer": "0.8 GB/s", + "tokens": 8840, + "fixed_tokens": 1105, + "processed_tokens": 60707, + "tool_calls": 5, + "turns": 6 + }, + "world-059": { + "answer": "dec-derive-the-missing-harmonic-phase-from-each-save--gbqrvvgyq2gy2gnw", + "tokens": 10128, + "fixed_tokens": 1106, + "processed_tokens": 61549, + "tool_calls": 5, + "turns": 6 + }, + "world-060": { + "answer": "dec-partition-the-world-into-192-m-cells--bk9r1c5jwb6fh8tc", + "tokens": 11571, + "fixed_tokens": 1101, + "processed_tokens": 111602, + "tool_calls": 11, + "turns": 10 + }, + "world-061": { + "answer": "236 MB", + "tokens": 12193, + "fixed_tokens": 1106, + "processed_tokens": 115653, + "tool_calls": 11, + "turns": 10 + }, + "world-062": { + "answer": "310 MB", + "tokens": 10061, + "fixed_tokens": 1105, + "processed_tokens": 109199, + "tool_calls": 10, + "turns": 10 + }, + "world-063": { + "answer": "con-streaming-pool-budget--632bkbs79bwj4cr5", + "tokens": 8712, + "fixed_tokens": 1105, + "processed_tokens": 48283, + "tool_calls": 4, + "turns": 5 + }, + "world-064": { + "answer": [ + "dec-force-strict-ieee-float-mode-for-the-tide-simula--ecwmqt7ncjngysqw", + "dec-have-the-host-stream-tide-heights-to-every-peer--a3rb1seedn2bj7k8" + ], + "tokens": 9128, + "fixed_tokens": 1102, + "processed_tokens": 79570, + "tool_calls": 8, + "turns": 8 + }, + "world-065": { + "answer": "3.1 ms", + "tokens": 15501, + "fixed_tokens": 1105, + "processed_tokens": 108641, + "tool_calls": 8, + "turns": 9 + }, + "world-066": { + "answer": 46, + "tokens": 12556, + "fixed_tokens": 1103, + "processed_tokens": 75405, + "tool_calls": 6, + "turns": 7 + }, + "world-067": { + "answer": "1.2 GB/s", + "tokens": 12012, + "fixed_tokens": 1105, + "processed_tokens": 117635, + "tool_calls": 9, + "turns": 10 + }, + "world-068": { + "answer": true, + "tokens": 22251, + "fixed_tokens": 1102, + "processed_tokens": 133812, + "tool_calls": 10, + "turns": 10 + }, + "world-069": { + "answer": 6, + "tokens": 30310, + "fixed_tokens": 1102, + "processed_tokens": 796283, + "tool_calls": 38, + "turns": 39 + }, + "world-070": { + "answer": 3, + "tokens": 8465, + "fixed_tokens": 1103, + "processed_tokens": 77757, + "tool_calls": 8, + "turns": 9 + }, + "world-071": { + "answer": [ + "dec-hot-reload-gull-functions-by-swapping-bytecode-a--8dxnkg6h8vmcsr42", + "dec-stamp-each-save-with-a-schema-version-and-upgrad--nbg1h8g1bhn2ks8w" + ], + "tokens": 37678, + "fixed_tokens": 1107, + "processed_tokens": 440489, + "tool_calls": 20, + "turns": 18 + }, + "world-072": { + "answer": [ + "dec-compress-streamed-assets-with-lz4--v1pfsgfp0b3yjv4s", + "dec-compress-streamed-assets-with-zstandard--bs1t15cygc51cpje" + ], + "tokens": 14340, + "fixed_tokens": 1104, + "processed_tokens": 59208, + "tool_calls": 5, + "turns": 5 + }, + "world-073": { + "answer": false, + "tokens": 10005, + "fixed_tokens": 1102, + "processed_tokens": 58028, + "tool_calls": 7, + "turns": 6 + }, + "world-074": { + "answer": 7, + "tokens": 4348, + "fixed_tokens": 1102, + "processed_tokens": 41947, + "tool_calls": 5, + "turns": 6 + }, + "world-075": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--8679sjn071ycsbpv", + "dec-partition-the-world-into-192-meter-cells--q62ddwdv2n8wxay6", + "dec-return-engine-jobs-to-the-thread-pool-with-conti--547qbweqvrp1j5g3" + ], + "tokens": 5204, + "fixed_tokens": 1102, + "processed_tokens": 43730, + "tool_calls": 5, + "turns": 6 + }, + "world-076": { + "answer": "unknown", + "tokens": 29843, + "fixed_tokens": 1103, + "processed_tokens": 408050, + "tool_calls": 24, + "turns": 20 + }, + "world-077": { + "answer": 251, + "tokens": 7473, + "fixed_tokens": 1102, + "processed_tokens": 75396, + "tool_calls": 8, + "turns": 9 + }, + "world-078": { + "answer": [ + "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "rsk-reworking-the-read-scheduler-may-slip-the-coast--9jzvs7etvb5jxqtp" + ], + "tokens": 9979, + "fixed_tokens": 1102, + "processed_tokens": 60733, + "tool_calls": 6, + "turns": 7 + }, + "world-079": { + "answer": [ + "con-streaming-pool-budget--8czwb4fjr5p5vpy7", + "con-no-runtime-code-generation-on-consoles--6cf25zqmsvqkdd73", + "con-tide-state-must-be-bit-identical-on-every-peer--zrpnp8qjyxzkas7t", + "con-saves-from-beta-onward-must-load-in-every-later--v6z856sxtgc5sb09" + ], + "tokens": 12151, + "fixed_tokens": 1103, + "processed_tokens": 78191, + "tool_calls": 7, + "turns": 8 + }, + "world-080": { + "answer": [ + "iss-cross-platform-cloud-saves-are-unplanned--5bk0k4wrgj42ztfy", + "iss-gull-hot-reload-drops-suspended-coroutines--z7j2gmyny6gdxd07", + "iss-gull-hot-reload-rejects-struct-layout-changes--3v2w2f9hg2gwy6aj" + ], + "tokens": 6715, + "fixed_tokens": 1106, + "processed_tokens": 62455, + "tool_calls": 7, + "turns": 8 + }, + "world-081": { + "answer": "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "tokens": 24956, + "fixed_tokens": 1101, + "processed_tokens": 384758, + "tool_calls": 43, + "turns": 21 + } + } +} diff --git a/benchmarks/proof-recall/runs/x10-sonnet-r1-nobrief.json b/benchmarks/proof-recall/runs/x10-sonnet-r1-nobrief.json new file mode 100644 index 00000000..cdcd27bf --- /dev/null +++ b/benchmarks/proof-recall/runs/x10-sonnet-r1-nobrief.json @@ -0,0 +1,2237 @@ +{ + "run": { + "name": "x10-sonnet-r1-nobrief", + "protocol": "one fresh session per question", + "models": ["claude-sonnet-5-5"], + "find_hash": "e5be653299abe9b2329822d98ffe09e90ee7cfe96f8711c813a51dc6500b887f" + }, + "answers": { + "ambiguous-001": { + "answer": "unknown", + "tokens": 31625, + "fixed_tokens": 1124, + "processed_tokens": 672462, + "tool_calls": 31, + "turns": 31 + }, + "ambiguous-002": { + "answer": "unknown", + "tokens": 27808, + "fixed_tokens": 1125, + "processed_tokens": 337636, + "tool_calls": 20, + "turns": 18 + }, + "ambiguous-003": { + "answer": "unknown", + "tokens": 31249, + "fixed_tokens": 1123, + "processed_tokens": 789248, + "tool_calls": 41, + "turns": 41 + }, + "ambiguous-004": { + "answer": "6 min 40 s", + "tokens": 14420, + "fixed_tokens": 1122, + "processed_tokens": 163390, + "tool_calls": 18, + "turns": 14 + }, + "ambiguous-005": { + "answer": "6 weeks", + "tokens": 21118, + "fixed_tokens": 1126, + "processed_tokens": 362461, + "tool_calls": 26, + "turns": 24 + }, + "ambiguous-006": { + "answer": "unknown", + "tokens": 41684, + "fixed_tokens": 1126, + "processed_tokens": 1350456, + "tool_calls": 51, + "turns": 51 + }, + "ambiguous-007": { + "answer": "32 steps", + "tokens": 10802, + "fixed_tokens": 1125, + "processed_tokens": 119917, + "tool_calls": 14, + "turns": 12 + }, + "ambiguous-008": { + "answer": "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "tokens": 8789, + "fixed_tokens": 1125, + "processed_tokens": 81852, + "tool_calls": 8, + "turns": 9 + }, + "ambiguous-009": { + "answer": "3.0 ms", + "tokens": 12088, + "fixed_tokens": 1127, + "processed_tokens": 174939, + "tool_calls": 16, + "turns": 16 + }, + "ambiguous-010": { + "answer": "160 MB/s", + "tokens": 14841, + "fixed_tokens": 1123, + "processed_tokens": 168999, + "tool_calls": 13, + "turns": 13 + }, + "ambiguous-011": { + "answer": "260 MB", + "tokens": 50405, + "fixed_tokens": 1128, + "processed_tokens": 1250761, + "tool_calls": 55, + "turns": 43 + }, + "ambiguous-012": { + "answer": null, + "tokens": 52380, + "fixed_tokens": 1128, + "processed_tokens": 1336811, + "tool_calls": 73, + "turns": 45 + }, + "ambiguous-013": { + "answer": "61%", + "tokens": 21920, + "fixed_tokens": 1124, + "processed_tokens": 320202, + "tool_calls": 25, + "turns": 21 + }, + "ambiguous-014": { + "answer": "textures", + "tokens": 15094, + "fixed_tokens": 1124, + "processed_tokens": 216882, + "tool_calls": 20, + "turns": 17 + }, + "ambiguous-015": { + "answer": "38 KB", + "tokens": 28774, + "fixed_tokens": 1126, + "processed_tokens": 579136, + "tool_calls": 34, + "turns": 30 + }, + "ambiguous-016": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 28214, + "fixed_tokens": 1127, + "processed_tokens": 435497, + "tool_calls": 24, + "turns": 20 + }, + "ambiguous-017": { + "answer": "fnd-skiff-microsd-slows-sharply-on-full-warm-cards--p1rgh36hat2egpz3", + "tokens": 28027, + "fixed_tokens": 1127, + "processed_tokens": 454362, + "tool_calls": 23, + "turns": 22 + }, + "ambiguous-018": { + "answer": "fnd-skiff-microsd-slows-sharply-on-full-warm-cards--p1rgh36hat2egpz3", + "tokens": 43953, + "fixed_tokens": 1122, + "processed_tokens": 1217787, + "tool_calls": 55, + "turns": 46 + }, + "ambiguous-019": { + "answer": "7.4 ms", + "tokens": 13556, + "fixed_tokens": 1127, + "processed_tokens": 208553, + "tool_calls": 17, + "turns": 18 + }, + "ambiguous-020": { + "answer": "5912 pages", + "tokens": 14813, + "fixed_tokens": 1122, + "processed_tokens": 190081, + "tool_calls": 17, + "turns": 15 + }, + "ambiguous-021": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 6419, + "fixed_tokens": 1127, + "processed_tokens": 55602, + "tool_calls": 6, + "turns": 7 + }, + "ambiguous-022": { + "answer": null, + "tokens": 49721, + "fixed_tokens": 1124, + "processed_tokens": 1806564, + "tool_calls": 65, + "turns": 61 + }, + "ambiguous-023": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 7173, + "fixed_tokens": 1125, + "processed_tokens": 47301, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-024": { + "answer": "fnd-streaming-tide-heights-costs-46-kb-s-per-peer--knxqtyqh6xcz1tp9", + "tokens": 30835, + "fixed_tokens": 1124, + "processed_tokens": 965315, + "tool_calls": 44, + "turns": 44 + }, + "cross-001": { + "answer": "con-frame-budget--br0qszfg8fys908n", + "tokens": 9704, + "fixed_tokens": 1117, + "processed_tokens": 106026, + "tool_calls": 11, + "turns": 11 + }, + "cross-002": { + "answer": 2, + "tokens": 13641, + "fixed_tokens": 1116, + "processed_tokens": 82152, + "tool_calls": 15, + "turns": 8 + }, + "cross-003": { + "answer": 2, + "tokens": 8124, + "fixed_tokens": 1117, + "processed_tokens": 52713, + "tool_calls": 9, + "turns": 6 + }, + "cross-004": { + "answer": "dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e", + "tokens": 13304, + "fixed_tokens": 1113, + "processed_tokens": 165415, + "tool_calls": 15, + "turns": 15 + }, + "cross-005": { + "answer": "Rhea Valdés", + "tokens": 15325, + "fixed_tokens": 1113, + "processed_tokens": 189501, + "tool_calls": 16, + "turns": 15 + }, + "cross-006": { + "answer": "about 19% (4.8 ms of the 25 ms frame)", + "tokens": 17487, + "fixed_tokens": 1117, + "processed_tokens": 246915, + "tool_calls": 18, + "turns": 18 + }, + "cross-007": { + "answer": "8 GB", + "tokens": 23744, + "fixed_tokens": 1112, + "processed_tokens": 379862, + "tool_calls": 31, + "turns": 23 + }, + "cross-008": { + "answer": "unknown", + "tokens": 12733, + "fixed_tokens": 1113, + "processed_tokens": 97400, + "tool_calls": 14, + "turns": 9 + }, + "cross-009": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 13716, + "fixed_tokens": 1112, + "processed_tokens": 134349, + "tool_calls": 15, + "turns": 12 + }, + "cross-010": { + "answer": "dec-build-an-on-device-pipeline-cache-for-skiff--726r9mtk76qndeh7", + "tokens": 14187, + "fixed_tokens": 1114, + "processed_tokens": 143861, + "tool_calls": 13, + "turns": 12 + }, + "cross-011": { + "answer": false, + "tokens": 24172, + "fixed_tokens": 1118, + "processed_tokens": 399090, + "tool_calls": 26, + "turns": 23 + }, + "cross-012": { + "answer": "unknown", + "tokens": 26622, + "fixed_tokens": 1113, + "processed_tokens": 512548, + "tool_calls": 25, + "turns": 25 + }, + "cross-013": { + "answer": "Skiff runs Tern at the Performance preset (dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3), saving 1.8 ms over Quality (2.1 ms vs 3.9 ms).", + "tokens": 25704, + "fixed_tokens": 1119, + "processed_tokens": 338504, + "tool_calls": 17, + "turns": 17 + }, + "cross-014": { + "answer": 3, + "tokens": 3104, + "fixed_tokens": 1114, + "processed_tokens": 25011, + "tool_calls": 3, + "turns": 4 + }, + "cross-015": { + "answer": "Two Efforts are no longer active. \"Co-op rollback netcode prototype\" was abandoned by dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b, because re-running tides every rollback frame costs 4.8 ms on Skiff. \"Shader compile cache and build farm\" is completed: it built the shared content-addressed compile cache (dec-build-a-content-addressed-shader-compile-cache-s--e65rq9439k7xz5zh), with its key finally set by dec-key-the-shader-cache-on-preprocessed-source-defi--3a05ff0r954vxh30. I did not find an explicit closing record for it.", + "tokens": 20033, + "fixed_tokens": 1114, + "processed_tokens": 270182, + "tool_calls": 20, + "turns": 17 + }, + "cross-016": { + "answer": "16.7 ms", + "tokens": 10712, + "fixed_tokens": 1115, + "processed_tokens": 130413, + "tool_calls": 13, + "turns": 13 + }, + "cross-017": { + "answer": "unknown", + "tokens": 22721, + "fixed_tokens": 1114, + "processed_tokens": 377284, + "tool_calls": 19, + "turns": 20 + }, + "netcode-001": { + "answer": "abandoned", + "tokens": 5182, + "fixed_tokens": 1121, + "processed_tokens": 36090, + "tool_calls": 4, + "turns": 5 + }, + "netcode-002": { + "answer": "netcode/rollback-archive", + "tokens": 18318, + "fixed_tokens": 1120, + "processed_tokens": 172289, + "tool_calls": 10, + "turns": 11 + }, + "netcode-003": { + "answer": "38 KB", + "tokens": 33244, + "fixed_tokens": 1122, + "processed_tokens": 514080, + "tool_calls": 32, + "turns": 26 + }, + "netcode-004": { + "answer": "Rollback was abandoned because it must re-run the tide simulation on every rolled-back frame, costing 4.8 ms on Skiff (about a fifth of the 25 ms frame), and the workarounds failed: quarter-resolution tide re-simulation desynced peers within 90 seconds, and leaving tides out of the snapshot breaks the bit-for-bit tide constraint (decision dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b). That reason depends on the world Decision dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e (Run the tide simulation in 32.32 fixed point).", + "tokens": 26791, + "fixed_tokens": 1120, + "processed_tokens": 337869, + "tool_calls": 17, + "turns": 16 + }, + "netcode-005": { + "answer": "unknown", + "tokens": 20082, + "fixed_tokens": 1122, + "processed_tokens": 320628, + "tool_calls": 28, + "turns": 23 + }, + "netcode-006": { + "answer": "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "tokens": 7325, + "fixed_tokens": 1120, + "processed_tokens": 66324, + "tool_calls": 7, + "turns": 8 + }, + "netcode-007": { + "answer": "dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b", + "tokens": 6372, + "fixed_tokens": 1121, + "processed_tokens": 64688, + "tool_calls": 7, + "turns": 8 + }, + "netcode-008": { + "answer": "dec-use-a-4-frame-rollback-window--pqe4hvbgmbp0dmh9", + "tokens": 9056, + "fixed_tokens": 1112, + "processed_tokens": 102020, + "tool_calls": 12, + "turns": 11 + }, + "netcode-009": { + "answer": "1.9 ms", + "tokens": 10387, + "fixed_tokens": 1118, + "processed_tokens": 154439, + "tool_calls": 14, + "turns": 15 + }, + "netcode-010": { + "answer": "dec-use-a-6-frame-rollback-window-with-1-frame-of-in--s53944zhkw103frk", + "tokens": 25866, + "fixed_tokens": 1120, + "processed_tokens": 299282, + "tool_calls": 16, + "turns": 16 + }, + "netcode-011": { + "answer": "con-co-op-peers-must-agree-on-tide-state-bit-for-bit--4v0sdbc3t4ywzr3g", + "tokens": 20671, + "fixed_tokens": 1118, + "processed_tokens": 338683, + "tool_calls": 26, + "turns": 24 + }, + "netcode-012": { + "answer": "dec-re-simulate-tides-at-quarter-resolution-during-r--7ah25p0r0xt9719f", + "tokens": 19341, + "fixed_tokens": 1117, + "processed_tokens": 119058, + "tool_calls": 9, + "turns": 9 + }, + "netcode-013": { + "answer": [ + "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "fnd-rollback-re-simulates-a-4-frame-window-in-1-1-ms--xa30a1cez2brphyf", + "rsk-re-simulation-cost-could-blow-the-skiff-frame-bu--sk2njsp170a3qwyf" + ], + "tokens": 25021, + "fixed_tokens": 1117, + "processed_tokens": 234505, + "tool_calls": 21, + "turns": 13 + }, + "netcode-014": { + "answer": [ + "iss-late-join-during-a-tide-surge-desyncs-the-joinin--mya93va0n7s8g7cj" + ], + "tokens": 17079, + "fixed_tokens": 1118, + "processed_tokens": 149405, + "tool_calls": 10, + "turns": 11 + }, + "netcode-015": { + "answer": 2, + "tokens": 4593, + "fixed_tokens": 1123, + "processed_tokens": 49612, + "tool_calls": 6, + "turns": 7 + }, + "ports-001": { + "answer": "25 ms", + "tokens": 8714, + "fixed_tokens": 1116, + "processed_tokens": 102021, + "tool_calls": 11, + "turns": 11 + }, + "ports-002": { + "answer": "unknown", + "tokens": 25753, + "fixed_tokens": 1116, + "processed_tokens": 500319, + "tool_calls": 25, + "turns": 24 + }, + "ports-003": { + "answer": "61 MB/s", + "tokens": 37514, + "fixed_tokens": 1118, + "processed_tokens": 845447, + "tool_calls": 40, + "turns": 33 + }, + "ports-004": { + "answer": "accepted", + "tokens": 5361, + "fixed_tokens": 1116, + "processed_tokens": 52257, + "tool_calls": 6, + "turns": 7 + }, + "ports-005": { + "answer": "unknown", + "tokens": 42897, + "fixed_tokens": 1119, + "processed_tokens": 968781, + "tool_calls": 62, + "turns": 37 + }, + "ports-034": { + "answer": "6 weeks", + "tokens": 18380, + "fixed_tokens": 1115, + "processed_tokens": 167266, + "tool_calls": 11, + "turns": 12 + }, + "ports-006": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 10597, + "fixed_tokens": 1114, + "processed_tokens": 80851, + "tool_calls": 7, + "turns": 8 + }, + "ports-007": { + "answer": "SKF-UI-031", + "tokens": 10312, + "fixed_tokens": 1116, + "processed_tokens": 134644, + "tool_calls": 15, + "turns": 14 + }, + "ports-008": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 5987, + "fixed_tokens": 1115, + "processed_tokens": 45040, + "tool_calls": 5, + "turns": 6 + }, + "ports-009": { + "answer": "dec-load-skiff-fast-travel-without-a-loading-screen--v4qft5znysjv5f24", + "tokens": 35296, + "fixed_tokens": 1121, + "processed_tokens": 976723, + "tool_calls": 38, + "turns": 38 + }, + "ports-010": { + "answer": "con-frame-budget--wt5rfqh5z444g2cg", + "tokens": 16652, + "fixed_tokens": 1113, + "processed_tokens": 204073, + "tool_calls": 16, + "turns": 16 + }, + "ports-011": { + "answer": "99.6%", + "tokens": 21711, + "fixed_tokens": 1116, + "processed_tokens": 299556, + "tool_calls": 24, + "turns": 20 + }, + "ports-012": { + "answer": "Because saltglass caustics shimmered badly at the 540p floor (visible in 7 of 10 dune views vs 2 of 10 at 576p), so decision dec-raise-the-skiff-dynamic-resolution-floor-to-576p--fep9q7m08pp9f0bb raised the floor to 576p (at +1.3 ms, still within the 33.3 ms budget); per fnd-saltglass-caustics-shimmer-at-the-540p-floor--0aa8sts89vhq1s6q. (That decision was later superseded by a 504p floor.)", + "tokens": 20041, + "fixed_tokens": 1116, + "processed_tokens": 213838, + "tool_calls": 13, + "turns": 14 + }, + "ports-013": { + "answer": "Ship Tern as Lantern's only upscaler", + "tokens": 24535, + "fixed_tokens": 1122, + "processed_tokens": 476068, + "tool_calls": 33, + "turns": 29 + }, + "ports-014": { + "answer": "dec-scale-skiff-resolution-dynamically-between-540p--jtytvgh2erd32x68", + "tokens": 36607, + "fixed_tokens": 1111, + "processed_tokens": 736919, + "tool_calls": 31, + "turns": 28 + }, + "ports-015": { + "answer": "textures", + "tokens": 12805, + "fixed_tokens": 1113, + "processed_tokens": 168542, + "tool_calls": 16, + "turns": 15 + }, + "ports-016": { + "answer": "dec-ship-prebuilt-pipeline-blobs-in-the-skiff-packag--3vjysjgy9qkvzebn", + "tokens": 13014, + "fixed_tokens": 1116, + "processed_tokens": 134578, + "tool_calls": 12, + "turns": 13 + }, + "ports-017": { + "answer": "unknown", + "tokens": 37992, + "fixed_tokens": 1119, + "processed_tokens": 1156330, + "tool_calls": 41, + "turns": 42 + }, + "ports-018": { + "answer": "504p", + "tokens": 10930, + "fixed_tokens": 1113, + "processed_tokens": 121432, + "tool_calls": 11, + "turns": 12 + }, + "ports-019": { + "answer": "4 MB per minute", + "tokens": 19036, + "fixed_tokens": 1112, + "processed_tokens": 189673, + "tool_calls": 13, + "turns": 13 + }, + "ports-020": { + "answer": "fnd-skiff-sdk-1-2-takes-1-gb-back-from-games--a0ybdgjzw64hwfqa", + "tokens": 15313, + "fixed_tokens": 1114, + "processed_tokens": 100073, + "tool_calls": 7, + "turns": 8 + }, + "ports-021": { + "answer": "SKF-DSP-004", + "tokens": 18990, + "fixed_tokens": 1114, + "processed_tokens": 363581, + "tool_calls": 24, + "turns": 24 + }, + "ports-022": { + "answer": "cit-skiff-sdk-1-2-release-notes--7gwwn8xxffrx6npe", + "tokens": 23159, + "fixed_tokens": 1115, + "processed_tokens": 218851, + "tool_calls": 12, + "turns": 13 + }, + "ports-023": { + "answer": "fnd-tern-costs-3-9-ms-at-quality-and-2-1-ms-at-perfo--dkbskbqfgfwtryq0", + "tokens": 21928, + "fixed_tokens": 1120, + "processed_tokens": 262916, + "tool_calls": 16, + "turns": 15 + }, + "ports-024": { + "answer": "fnd-skiff-firmware-1-2-rejects-pipeline-blobs-built--pernvgb2he7m3z3y", + "tokens": 22088, + "fixed_tokens": 1119, + "processed_tokens": 259896, + "tool_calls": 14, + "turns": 15 + }, + "ports-025": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 22915, + "fixed_tokens": 1117, + "processed_tokens": 288779, + "tool_calls": 16, + "turns": 16 + }, + "ports-026": { + "answer": "912 MB", + "tokens": 16092, + "fixed_tokens": 1119, + "processed_tokens": 272469, + "tool_calls": 21, + "turns": 21 + }, + "ports-027": { + "answer": [ + "dec-stream-skiff-world-tiles-in-2-mb-reads--3jz9w9gvhhc7eppw", + "rsk-tide-streaming-outruns-skiff-storage--6q8s3b383z2tyeyb" + ], + "tokens": 53307, + "fixed_tokens": 1113, + "processed_tokens": 1020767, + "tool_calls": 46, + "turns": 29 + }, + "ports-028": { + "answer": 2, + "tokens": 7050, + "fixed_tokens": 1117, + "processed_tokens": 54253, + "tool_calls": 6, + "turns": 7 + }, + "ports-029": { + "answer": 8, + "tokens": 38682, + "fixed_tokens": 1118, + "processed_tokens": 659188, + "tool_calls": 29, + "turns": 28 + }, + "ports-030": { + "answer": [ + "dec-cap-tide-storm-particles-at-6-000-on-skiff--resze52afjxyxys1", + "dec-drop-skiff-to-30-hz-while-a-tide-storm-is-on-scr--72vx1c8s4jrs6axc" + ], + "tokens": 5854, + "fixed_tokens": 1117, + "processed_tokens": 44646, + "tool_calls": 5, + "turns": 6 + }, + "ports-031": { + "answer": [ + "iss-skiff-cert-docked-30-hz-judder-on-some-tvs--9nq2xg9mzash474g", + "iss-meridian-cert-activity-cards-missing-for-two-bio--k214cmfra7pyvvzq" + ], + "tokens": 8524, + "fixed_tokens": 1115, + "processed_tokens": 79281, + "tool_calls": 8, + "turns": 9 + }, + "ports-032": { + "answer": 2, + "tokens": 5857, + "fixed_tokens": 1121, + "processed_tokens": 52185, + "tool_calls": 6, + "turns": 7 + }, + "ports-033": { + "answer": "unknown", + "tokens": 8848, + "fixed_tokens": 1114, + "processed_tokens": 100863, + "tool_calls": 10, + "turns": 11 + }, + "render-001": { + "answer": "7.4 ms", + "tokens": 13234, + "fixed_tokens": 1118, + "processed_tokens": 173139, + "tool_calls": 15, + "turns": 15 + }, + "render-002": { + "answer": "3.4x", + "tokens": 7713, + "fixed_tokens": 1112, + "processed_tokens": 93911, + "tool_calls": 10, + "turns": 11 + }, + "render-003": { + "answer": "Tomasz Brandt", + "tokens": 8002, + "fixed_tokens": 1113, + "processed_tokens": 85020, + "tool_calls": 9, + "turns": 10 + }, + "render-004": { + "answer": "rejected", + "tokens": 10811, + "fixed_tokens": 1110, + "processed_tokens": 93393, + "tool_calls": 12, + "turns": 10 + }, + "render-005": { + "answer": true, + "tokens": 18116, + "fixed_tokens": 1110, + "processed_tokens": 305056, + "tool_calls": 25, + "turns": 22 + }, + "render-006": { + "answer": "unknown", + "tokens": 14861, + "fixed_tokens": 1112, + "processed_tokens": 237816, + "tool_calls": 20, + "turns": 20 + }, + "render-007": { + "answer": "2026-07-24", + "tokens": 24370, + "fixed_tokens": 1113, + "processed_tokens": 318114, + "tool_calls": 17, + "turns": 17 + }, + "render-008": { + "answer": 2, + "tokens": 25325, + "fixed_tokens": 1111, + "processed_tokens": 416203, + "tool_calls": 21, + "turns": 21 + }, + "render-009": { + "answer": 32, + "tokens": 9213, + "fixed_tokens": 1115, + "processed_tokens": 93353, + "tool_calls": 10, + "turns": 10 + }, + "render-010": { + "answer": "unknown", + "tokens": 9558, + "fixed_tokens": 1112, + "processed_tokens": 104642, + "tool_calls": 12, + "turns": 11 + }, + "render-011": { + "answer": "fnd-sdf-gi-costs-2-4-ms-on-meridian--2w6c2pc5zmcsyjre", + "tokens": 7850, + "fixed_tokens": 1116, + "processed_tokens": 66910, + "tool_calls": 8, + "turns": 8 + }, + "render-012": { + "answer": "deferred", + "tokens": 14253, + "fixed_tokens": 1115, + "processed_tokens": 203764, + "tool_calls": 19, + "turns": 17 + }, + "render-013": { + "answer": "16-bit float per channel", + "tokens": 22160, + "fixed_tokens": 1115, + "processed_tokens": 399612, + "tool_calls": 23, + "turns": 24 + }, + "render-014": { + "answer": "52 percent", + "tokens": 20686, + "fixed_tokens": 1116, + "processed_tokens": 393401, + "tool_calls": 25, + "turns": 26 + }, + "render-015": { + "answer": "unknown", + "tokens": 48283, + "fixed_tokens": 1113, + "processed_tokens": 1092057, + "tool_calls": 68, + "turns": 39 + }, + "render-016": { + "answer": "1.2 km", + "tokens": 6356, + "fixed_tokens": 1113, + "processed_tokens": 65113, + "tool_calls": 7, + "turns": 8 + }, + "render-017": { + "answer": "unknown", + "tokens": 18184, + "fixed_tokens": 1115, + "processed_tokens": 355532, + "tool_calls": 24, + "turns": 24 + }, + "render-018": { + "answer": "unknown", + "tokens": 27017, + "fixed_tokens": 1109, + "processed_tokens": 454781, + "tool_calls": 22, + "turns": 22 + }, + "render-019": { + "answer": "dec-use-the-custom-brine-curve-for-tonemapping--p735g6wf2njare86", + "tokens": 8932, + "fixed_tokens": 1111, + "processed_tokens": 90753, + "tool_calls": 10, + "turns": 10 + }, + "render-020": { + "answer": "fnd-screen-space-refraction-loses-off-screen-content--k8zvmmdee7wg2fjy", + "tokens": 14355, + "fixed_tokens": 1110, + "processed_tokens": 198141, + "tool_calls": 22, + "turns": 17 + }, + "render-021": { + "answer": "dec-adopt-a-visibility-buffer-for-all-opaque-geometr--68vchzqgd1avw8q4", + "tokens": 6382, + "fixed_tokens": 1113, + "processed_tokens": 63345, + "tool_calls": 7, + "turns": 8 + }, + "render-022": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 10032, + "fixed_tokens": 1109, + "processed_tokens": 71719, + "tool_calls": 8, + "turns": 8 + }, + "render-023": { + "answer": "dec-capture-the-cliffs-and-dunes-benchmarks-on-every--3pqyr66yh4v6q676", + "tokens": 9396, + "fixed_tokens": 1111, + "processed_tokens": 137014, + "tool_calls": 14, + "turns": 14 + }, + "render-024": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 5458, + "fixed_tokens": 1111, + "processed_tokens": 60227, + "tool_calls": 7, + "turns": 8 + }, + "render-025": { + "answer": "fnd-tern-cost-at-4k-output-on-meridian-after-fp16-pa--p4gcmzw2a4bvzjev", + "tokens": 37717, + "fixed_tokens": 1113, + "processed_tokens": 857162, + "tool_calls": 33, + "turns": 32 + }, + "render-026": { + "answer": "dec-ship-four-tern-quality-presets--vjw18d82dx5gath8", + "tokens": 15916, + "fixed_tokens": 1113, + "processed_tokens": 252286, + "tool_calls": 20, + "turns": 20 + }, + "render-027": { + "answer": "con-frame-budget--j6vneva1g2ywj86m", + "tokens": 9557, + "fixed_tokens": 1114, + "processed_tokens": 52067, + "tool_calls": 5, + "turns": 6 + }, + "render-028": { + "answer": "dec-move-to-gpu-driven-two-phase-occlusion-culling--6dre481mabxsw6ee", + "tokens": 10798, + "fixed_tokens": 1115, + "processed_tokens": 121808, + "tool_calls": 11, + "turns": 12 + }, + "render-029": { + "answer": "fnd-vsm-page-pool-overflows-during-spring-tides--m2p8e89hhdmh0z8a", + "tokens": 6846, + "fixed_tokens": 1114, + "processed_tokens": 64661, + "tool_calls": 7, + "turns": 8 + }, + "render-030": { + "answer": "dec-use-screen-space-refraction-for-saltglass--vknzksgdrgr7zkg6", + "tokens": 17272, + "fixed_tokens": 1111, + "processed_tokens": 200220, + "tool_calls": 21, + "turns": 15 + }, + "render-031": { + "answer": "Decision dec-visibility-buffer-for-opaque-geometry-foliage-an--gvnvp04jgm7kbfh9 (derived from fnd-alpha-tested-foliage-costs-2-1-ms-in-the-visibil--f3g34x2k6fs367eb) moved them out because alpha-tested foliage cost 2.1 ms in the visibility buffer versus 0.8 ms in forward, and saltglass can't live in the buffer since it refracts the resolved scene and needs that image first, so both render in a forward pass after the resolve.", + "tokens": 10200, + "fixed_tokens": 1112, + "processed_tokens": 118154, + "tool_calls": 11, + "turns": 12 + }, + "render-032": { + "answer": "Per dec-ship-tern-as-lantern-s-only-upscaler--fmreq35nzyjc7k4e: Halcyon has no Skiff build, and shipping it on two platforms would break the one-upscaler-on-every-platform Constraint. Owning Tern's code also lets the team change the upscaler quickly for saltglass refraction and caustics, with no vendor tickets or license to negotiate, even though Tern costs 0.7 ms more on Meridian.", + "tokens": 15357, + "fixed_tokens": 1114, + "processed_tokens": 165738, + "tool_calls": 14, + "turns": 14 + }, + "render-033": { + "answer": "128 MB", + "tokens": 6852, + "fixed_tokens": 1109, + "processed_tokens": 64844, + "tool_calls": 7, + "turns": 8 + }, + "render-034": { + "answer": "PC workstation (Meridian clock profile)", + "tokens": 18467, + "fixed_tokens": 1112, + "processed_tokens": 358686, + "tool_calls": 26, + "turns": 26 + }, + "render-035": { + "answer": "5912 pages", + "tokens": 9133, + "fixed_tokens": 1114, + "processed_tokens": 113730, + "tool_calls": 11, + "turns": 12 + }, + "render-036": { + "answer": "HDR-07", + "tokens": 35303, + "fixed_tokens": 1114, + "processed_tokens": 1000566, + "tool_calls": 39, + "turns": 39 + }, + "render-037": { + "answer": "unknown", + "tokens": 27070, + "fixed_tokens": 1113, + "processed_tokens": 434385, + "tool_calls": 24, + "turns": 22 + }, + "render-038": { + "answer": "dec-visibility-buffer-for-opaque-geometry-foliage-an--gvnvp04jgm7kbfh9", + "tokens": 9087, + "fixed_tokens": 1113, + "processed_tokens": 96212, + "tool_calls": 9, + "turns": 10 + }, + "render-039": { + "answer": "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "tokens": 21508, + "fixed_tokens": 1114, + "processed_tokens": 204455, + "tool_calls": 12, + "turns": 13 + }, + "render-040": { + "answer": "fnd-lantern-at-alpha--ckpd9ecvdjqbcr7c", + "tokens": 12274, + "fixed_tokens": 1114, + "processed_tokens": 94568, + "tool_calls": 9, + "turns": 10 + }, + "render-041": { + "answer": "4.7 ms", + "tokens": 23152, + "fixed_tokens": 1115, + "processed_tokens": 292239, + "tool_calls": 16, + "turns": 16 + }, + "render-042": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 9572, + "fixed_tokens": 1112, + "processed_tokens": 117962, + "tool_calls": 11, + "turns": 12 + }, + "render-043": { + "answer": "fnd-meshlet-cone-culling-rejects-9-percent-of-triang--t6e5x7trzedex105", + "tokens": 17824, + "fixed_tokens": 1114, + "processed_tokens": 322284, + "tool_calls": 23, + "turns": 23 + }, + "render-044": { + "answer": "dec-screen-space-refraction-with-a-ray-marched-fallb--2639xtpyrfrvbx1r", + "tokens": 8714, + "fixed_tokens": 1114, + "processed_tokens": 91316, + "tool_calls": 10, + "turns": 10 + }, + "render-045": { + "answer": false, + "tokens": 20040, + "fixed_tokens": 1112, + "processed_tokens": 172855, + "tool_calls": 11, + "turns": 12 + }, + "render-046": { + "answer": "dec-use-virtual-shadow-maps-for-the-sun--x8nqkn9q7eaf1ap4 was reopened because the cost finding that sank virtual shadow maps was a capture error (with cached pages VSM costs 1.4 ms, less than four cascades), and cascade seams through saltglass remain unsolved under cascaded shadow maps (dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr).", + "tokens": 10731, + "fixed_tokens": 1113, + "processed_tokens": 108348, + "tool_calls": 11, + "turns": 11 + }, + "render-047": { + "answer": "Decision dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1 returned Skiff to the visibility buffer because the release-build capture showed it at 7.4 ms versus 8.1 ms for forward+ (6.3 ms with half-rate material resolve), and keeping two opaque paths cost about one engineer-week a month in fixes, so the Skiff forward+ fallback was no longer worth keeping.", + "tokens": 9131, + "fixed_tokens": 1111, + "processed_tokens": 72367, + "tool_calls": 8, + "turns": 8 + }, + "render-048": { + "answer": null, + "tokens": 59520, + "fixed_tokens": 1111, + "processed_tokens": 2337676, + "tool_calls": 63, + "turns": 60 + }, + "render-049": { + "answer": "dec-use-virtual-shadow-maps-for-the-sun--x8nqkn9q7eaf1ap4", + "tokens": 12976, + "fixed_tokens": 1113, + "processed_tokens": 140382, + "tool_calls": 13, + "turns": 13 + }, + "render-050": { + "answer": "con-frame-budget--fzdrbxns6dmrrmy3", + "tokens": 9726, + "fixed_tokens": 1114, + "processed_tokens": 59679, + "tool_calls": 6, + "turns": 7 + }, + "render-051": { + "answer": "iss-tern-ghosting-on-moving-saltglass--2yr3q6zf476wajqp", + "tokens": 11114, + "fixed_tokens": 1116, + "processed_tokens": 140117, + "tool_calls": 14, + "turns": 14 + }, + "render-052": { + "answer": "61 percent", + "tokens": 13595, + "fixed_tokens": 1113, + "processed_tokens": 149872, + "tool_calls": 12, + "turns": 13 + }, + "render-053": { + "answer": "every frame", + "tokens": 12798, + "fixed_tokens": 1111, + "processed_tokens": 142308, + "tool_calls": 13, + "turns": 13 + }, + "render-054": { + "answer": "debug-gpu-validation", + "tokens": 12240, + "fixed_tokens": 1115, + "processed_tokens": 153393, + "tool_calls": 13, + "turns": 14 + }, + "render-055": { + "answer": "fnd-tern-cost-at-4k-output-on-meridian-with-the-reac--81tcpzvhwzcy6wyn", + "tokens": 15280, + "fixed_tokens": 1112, + "processed_tokens": 198146, + "tool_calls": 17, + "turns": 16 + }, + "render-056": { + "answer": "208,000 meshlets", + "tokens": 17808, + "fixed_tokens": 1112, + "processed_tokens": 310187, + "tool_calls": 21, + "turns": 22 + }, + "render-057": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 23360, + "fixed_tokens": 1110, + "processed_tokens": 185925, + "tool_calls": 11, + "turns": 12 + }, + "render-058": { + "answer": "iss-which-upscaler-should-lantern-ship--g8c2y9tsr856tn2v", + "tokens": 14720, + "fixed_tokens": 1114, + "processed_tokens": 202276, + "tool_calls": 19, + "turns": 17 + }, + "render-059": { + "answer": "fnd-sdf-gi-costs-4-7-ms-on-meridian--sm19rt2f274ajsvr", + "tokens": 28374, + "fixed_tokens": 1114, + "processed_tokens": 440053, + "tool_calls": 22, + "turns": 20 + }, + "render-060": { + "answer": "0.31", + "tokens": 17002, + "fixed_tokens": 1113, + "processed_tokens": 285243, + "tool_calls": 24, + "turns": 23 + }, + "render-061": { + "answer": "con-ui-composites-at-a-fixed-203-nits-in-hdr--de6aft7cvhdezn5s", + "tokens": 9600, + "fixed_tokens": 1115, + "processed_tokens": 124657, + "tool_calls": 12, + "turns": 13 + }, + "render-062": { + "answer": "con-frame-budget--j6vneva1g2ywj86m", + "tokens": 17226, + "fixed_tokens": 1112, + "processed_tokens": 111934, + "tool_calls": 8, + "turns": 9 + }, + "render-063": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 11805, + "fixed_tokens": 1121, + "processed_tokens": 103444, + "tool_calls": 10, + "turns": 10 + }, + "render-064": { + "answer": "dec-use-a-cached-shadow-atlas-for-local-lights--8p6f2v774he38t5j", + "tokens": 14918, + "fixed_tokens": 1109, + "processed_tokens": 173326, + "tool_calls": 17, + "turns": 15 + }, + "render-065": { + "answer": "dec-ship-tern-as-lantern-s-only-upscaler--fmreq35nzyjc7k4e", + "tokens": 26536, + "fixed_tokens": 1113, + "processed_tokens": 689871, + "tool_calls": 44, + "turns": 41 + }, + "render-066": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 10122, + "fixed_tokens": 1114, + "processed_tokens": 133434, + "tool_calls": 14, + "turns": 14 + }, + "render-067": { + "answer": 4, + "tokens": 17093, + "fixed_tokens": 1113, + "processed_tokens": 122525, + "tool_calls": 9, + "turns": 10 + }, + "render-068": { + "answer": [ + "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "dec-lock-caustic-photon-jitter-to-the-tide-phase--85x10k4h11t8nx7c", + "dec-use-the-probe-grid-alone-in-120-hz-performance-m--6dzm1nt40pjfpyae", + "dec-trace-sdf-gi-at-quarter-resolution-in-performanc--06cqt3mqn3fg5h6e", + "dec-read-paper-white-from-the-meridian-system-settin--7crbx73gqw1hn7xq", + "dec-add-an-in-game-paper-white-slider-from-80-to-400--1phfvxby983z8tjf" + ], + "tokens": 12239, + "fixed_tokens": 1115, + "processed_tokens": 93561, + "tool_calls": 13, + "turns": 9 + }, + "render-069": { + "answer": "iss-shadow-pages-run-out-at-spring-low-tide--f4fgy6b44jbkrsmh", + "tokens": 12021, + "fixed_tokens": 1109, + "processed_tokens": 134774, + "tool_calls": 12, + "turns": 13 + }, + "render-070": { + "answer": [ + "dec-use-signed-distance-field-gi--yn366zmp3k7snd9w", + "dec-enable-meshlet-cone-culling-on-all-platforms--cpyweprmzcqhqsv2", + "dec-feed-a-saltglass-reactive-mask-into-tern--xezjs7v3vq3r8e6k" + ], + "tokens": 68204, + "fixed_tokens": 1110, + "processed_tokens": 2177216, + "tool_calls": 72, + "turns": 53 + }, + "render-071": { + "answer": 3, + "tokens": 4886, + "fixed_tokens": 1112, + "processed_tokens": 49946, + "tool_calls": 6, + "turns": 7 + }, + "render-072": { + "answer": [ + "rsk-sdf-volumes-outgrow-the-render-target-memory-bud--xfd2pfztvskdcd2c", + "rsk-artists-cannot-match-concept-paintings-under-the--6cfh9vkamn0kbtcx" + ], + "tokens": 7221, + "fixed_tokens": 1115, + "processed_tokens": 71344, + "tool_calls": 9, + "turns": 9 + }, + "render-073": { + "answer": [ + "rsk-vsm-page-pool-overflows-when-tides-expose-the-fl--x1r161nrdngra4ty", + "rsk-gi-cannot-fit-the-120-hz-performance-mode--tqt3fj839dqfq5s7", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 9216, + "fixed_tokens": 1112, + "processed_tokens": 58335, + "tool_calls": 6, + "turns": 7 + }, + "render-074": { + "answer": 5, + "tokens": 4175, + "fixed_tokens": 1110, + "processed_tokens": 40953, + "tool_calls": 5, + "turns": 6 + }, + "render-075": { + "answer": 3, + "tokens": 22201, + "fixed_tokens": 1113, + "processed_tokens": 385060, + "tool_calls": 36, + "turns": 24 + }, + "render-076": { + "answer": false, + "tokens": 11116, + "fixed_tokens": 1114, + "processed_tokens": 129948, + "tool_calls": 12, + "turns": 12 + }, + "render-077": { + "answer": true, + "tokens": 16865, + "fixed_tokens": 1115, + "processed_tokens": 197659, + "tool_calls": 18, + "turns": 15 + }, + "render-078": { + "answer": [ + "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 20073, + "fixed_tokens": 1112, + "processed_tokens": 185458, + "tool_calls": 22, + "turns": 15 + }, + "render-079": { + "answer": 5, + "tokens": 35365, + "fixed_tokens": 1111, + "processed_tokens": 493227, + "tool_calls": 21, + "turns": 20 + }, + "render-080": { + "answer": 6, + "tokens": 9911, + "fixed_tokens": 1110, + "processed_tokens": 108098, + "tool_calls": 12, + "turns": 11 + }, + "render-081": { + "answer": "unknown", + "tokens": 17351, + "fixed_tokens": 1117, + "processed_tokens": 271635, + "tool_calls": 24, + "turns": 20 + }, + "render-082": { + "answer": false, + "tokens": 15756, + "fixed_tokens": 1113, + "processed_tokens": 242576, + "tool_calls": 22, + "turns": 19 + }, + "render-083": { + "answer": null, + "tokens": 61681, + "fixed_tokens": 1114, + "processed_tokens": 1819415, + "tool_calls": 62, + "turns": 45 + }, + "tools-001": { + "answer": "completed", + "tokens": 5162, + "fixed_tokens": 1117, + "processed_tokens": 36036, + "tool_calls": 4, + "turns": 5 + }, + "tools-002": { + "answer": "47 minutes", + "tokens": 6659, + "fixed_tokens": 1119, + "processed_tokens": 63972, + "tool_calls": 7, + "turns": 8 + }, + "tools-003": { + "answer": "6 min 40 s", + "tokens": 22077, + "fixed_tokens": 1118, + "processed_tokens": 225768, + "tool_calls": 13, + "turns": 14 + }, + "tools-004": { + "answer": "unknown", + "tokens": 10021, + "fixed_tokens": 1117, + "processed_tokens": 105809, + "tool_calls": 12, + "turns": 11 + }, + "tools-005": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 7523, + "fixed_tokens": 1120, + "processed_tokens": 57182, + "tool_calls": 7, + "turns": 7 + }, + "tools-006": { + "answer": "dec-build-a-content-addressed-shader-compile-cache-s--e65rq9439k7xz5zh", + "tokens": 11709, + "fixed_tokens": 1117, + "processed_tokens": 71486, + "tool_calls": 6, + "turns": 7 + }, + "tools-007": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 26050, + "fixed_tokens": 1116, + "processed_tokens": 338868, + "tool_calls": 18, + "turns": 17 + }, + "tools-008": { + "answer": "dec-key-the-shader-cache-on-raw-source-text-and-comp--hp6awmx3gq5rn9nc", + "tokens": 8248, + "fixed_tokens": 1117, + "processed_tokens": 71450, + "tool_calls": 7, + "turns": 8 + }, + "tools-009": { + "answer": "92.4%", + "tokens": 11887, + "fixed_tokens": 1117, + "processed_tokens": 179967, + "tool_calls": 16, + "turns": 17 + }, + "tools-010": { + "answer": "dec-cap-each-farm-agent-at-6-parallel-shader-compile--q5w4sfjm73tsj8x8", + "tokens": 29004, + "fixed_tokens": 1119, + "processed_tokens": 377260, + "tool_calls": 19, + "turns": 18 + }, + "tools-011": { + "answer": "fnd-two-compiler-builds-with-the-same-version-string--gmgr4ha0ndxqmcn9", + "tokens": 9713, + "fixed_tokens": 1117, + "processed_tokens": 104255, + "tool_calls": 10, + "turns": 11 + }, + "tools-012": { + "answer": 3, + "tokens": 4551, + "fixed_tokens": 1113, + "processed_tokens": 49501, + "tool_calls": 6, + "turns": 7 + }, + "tools-013": { + "answer": 7, + "tokens": 44287, + "fixed_tokens": 1120, + "processed_tokens": 787887, + "tool_calls": 39, + "turns": 28 + }, + "tools-014": { + "answer": false, + "tokens": 20644, + "fixed_tokens": 1117, + "processed_tokens": 213045, + "tool_calls": 12, + "turns": 13 + }, + "tools-015": { + "answer": [ + "iss-developer-workstations-cannot-read-the-farm-shad--q2xw0f7fze6z55nj", + "rsk-shader-cache-storage-could-outgrow-the-4-tb-farm--76y9649mwrx7aavd" + ], + "tokens": 21305, + "fixed_tokens": 1113, + "processed_tokens": 135825, + "tool_calls": 12, + "turns": 10 + }, + "tools-016": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 26936, + "fixed_tokens": 1116, + "processed_tokens": 432723, + "tool_calls": 20, + "turns": 20 + }, + "world-001": { + "answer": "64 KB", + "tokens": 10466, + "fixed_tokens": 1112, + "processed_tokens": 130953, + "tool_calls": 13, + "turns": 13 + }, + "world-002": { + "answer": "5.1 ms", + "tokens": 10543, + "fixed_tokens": 1113, + "processed_tokens": 109658, + "tool_calls": 11, + "turns": 11 + }, + "world-003": { + "answer": "one crash per 900 script waits", + "tokens": 25386, + "fixed_tokens": 1113, + "processed_tokens": 387685, + "tool_calls": 24, + "turns": 22 + }, + "world-004": { + "answer": "4,100 file opens", + "tokens": 30345, + "fixed_tokens": 1119, + "processed_tokens": 506171, + "tool_calls": 41, + "turns": 28 + }, + "world-005": { + "answer": "11 m", + "tokens": 37621, + "fixed_tokens": 1116, + "processed_tokens": 1029476, + "tool_calls": 51, + "turns": 42 + }, + "world-006": { + "answer": "3.4 reversals per minute", + "tokens": 14107, + "fixed_tokens": 1115, + "processed_tokens": 240967, + "tool_calls": 20, + "turns": 20 + }, + "world-007": { + "answer": "260 MB", + "tokens": 14347, + "fixed_tokens": 1113, + "processed_tokens": 207534, + "tool_calls": 17, + "turns": 17 + }, + "world-008": { + "answer": "10 Hz", + "tokens": 18861, + "fixed_tokens": 1115, + "processed_tokens": 319179, + "tool_calls": 23, + "turns": 23 + }, + "world-009": { + "answer": "unknown", + "tokens": 22535, + "fixed_tokens": 1112, + "processed_tokens": 334838, + "tool_calls": 28, + "turns": 22 + }, + "world-010": { + "answer": "29 GB/s", + "tokens": 5313, + "fixed_tokens": 1114, + "processed_tokens": 43719, + "tool_calls": 5, + "turns": 6 + }, + "world-011": { + "answer": "fnd-shipping-entity-names-costs-57-mb-at-full-popula--6w7gjvy7wdmdx9qx", + "tokens": 29067, + "fixed_tokens": 1113, + "processed_tokens": 490691, + "tool_calls": 24, + "turns": 24 + }, + "world-012": { + "answer": "dec-run-engine-jobs-and-gull-scripts-on-one-thread-p--07pbzvqg00pmnrr3", + "tokens": 7110, + "fixed_tokens": 1116, + "processed_tokens": 66349, + "tool_calls": 7, + "turns": 8 + }, + "world-013": { + "answer": "unknown", + "tokens": 22175, + "fixed_tokens": 1113, + "processed_tokens": 337691, + "tool_calls": 31, + "turns": 22 + }, + "world-014": { + "answer": "unknown", + "tokens": 22309, + "fixed_tokens": 1115, + "processed_tokens": 410700, + "tool_calls": 29, + "turns": 26 + }, + "world-015": { + "answer": "3.0 ms", + "tokens": 16702, + "fixed_tokens": 1114, + "processed_tokens": 259444, + "tool_calls": 25, + "turns": 19 + }, + "world-016": { + "answer": "unknown", + "tokens": 5193, + "fixed_tokens": 1113, + "processed_tokens": 36027, + "tool_calls": 4, + "turns": 5 + }, + "world-017": { + "answer": "dec-partition-the-world-into-128-m-cells--372b6czbjf29ymaw", + "tokens": 12269, + "fixed_tokens": 1116, + "processed_tokens": 138401, + "tool_calls": 14, + "turns": 13 + }, + "world-018": { + "answer": "dec-run-engine-jobs-on-a-work-stealing-thread-pool--1pd72x1pxpgwy21x", + "tokens": 5878, + "fixed_tokens": 1114, + "processed_tokens": 44784, + "tool_calls": 5, + "turns": 6 + }, + "world-019": { + "answer": "fnd-tide-priority-reads-starve-player-proximity-load--by0d7b0bk39xz3rk", + "tokens": 24110, + "fixed_tokens": 1113, + "processed_tokens": 470102, + "tool_calls": 27, + "turns": 27 + }, + "world-020": { + "answer": "dec-schedule-streaming-reads-by-deadline-instead-of--q2x5aknabgcssdj2", + "tokens": 9728, + "fixed_tokens": 1113, + "processed_tokens": 72034, + "tool_calls": 8, + "turns": 8 + }, + "world-021": { + "answer": "dec-cut-navigation-tiles-to-64-m-so-three-fit-along--snvepk8ygeb3qsjs", + "tokens": 5488, + "fixed_tokens": 1113, + "processed_tokens": 52596, + "tool_calls": 6, + "turns": 7 + }, + "world-022": { + "answer": "fnd-41-of-alpha-crash-reports-have-no-usable-stack--35a50b07ferhf13m", + "tokens": 5409, + "fixed_tokens": 1112, + "processed_tokens": 43758, + "tool_calls": 5, + "turns": 6 + }, + "world-023": { + "answer": "dec-compress-streamed-assets-with-kelp--ezwc5p1tgsgm7v8m", + "tokens": 19642, + "fixed_tokens": 1116, + "processed_tokens": 236453, + "tool_calls": 20, + "turns": 20 + }, + "world-024": { + "answer": "dec-shrink-kelp-s-dictionary-to-1-mb-for-streamed-as--yh8hktfqnyqhepew", + "tokens": 9524, + "fixed_tokens": 1108, + "processed_tokens": 111148, + "tool_calls": 11, + "turns": 12 + }, + "world-025": { + "answer": "con-streaming-pool-budget--kca88sgh2j6sxvd5", + "tokens": 5780, + "fixed_tokens": 1111, + "processed_tokens": 44474, + "tool_calls": 5, + "turns": 6 + }, + "world-026": { + "answer": "fnd-retail-skiff-units-sustain-only-190-mb-s-of-stre--485sdcmm3gweevvn", + "tokens": 40197, + "fixed_tokens": 1114, + "processed_tokens": 1153251, + "tool_calls": 53, + "turns": 46 + }, + "world-027": { + "answer": "160 MB/s", + "tokens": 9631, + "fixed_tokens": 1114, + "processed_tokens": 104791, + "tool_calls": 10, + "turns": 11 + }, + "world-028": { + "answer": "fnd-streaming-tide-heights-costs-46-kb-s-per-peer--knxqtyqh6xcz1tp9", + "tokens": 56104, + "fixed_tokens": 1118, + "processed_tokens": 2368331, + "tool_calls": 68, + "turns": 67 + }, + "world-029": { + "answer": "dec-return-engine-jobs-to-a-thread-pool-with-continu--1zkc1v2x97taf50h", + "tokens": 15329, + "fixed_tokens": 1113, + "processed_tokens": 261103, + "tool_calls": 23, + "turns": 21 + }, + "world-030": { + "answer": "fnd-world-core-state-at-alpha--tkbsm4867h65jqq8", + "tokens": 12959, + "fixed_tokens": 1115, + "processed_tokens": 151334, + "tool_calls": 15, + "turns": 15 + }, + "world-031": { + "answer": "unknown", + "tokens": 25699, + "fixed_tokens": 1112, + "processed_tokens": 509367, + "tool_calls": 25, + "turns": 25 + }, + "world-032": { + "answer": "fnd-float-tide-heights-drift-apart-between-pc-and-me--7jb0fg4b5xzr7q5e", + "tokens": 24124, + "fixed_tokens": 1111, + "processed_tokens": 492150, + "tool_calls": 31, + "turns": 30 + }, + "world-033": { + "answer": "dec-save-player-changes-as-deltas-against-the-tide-f--n38scy1yaarsr8cw", + "tokens": 15868, + "fixed_tokens": 1114, + "processed_tokens": 184108, + "tool_calls": 20, + "turns": 15 + }, + "world-034": { + "answer": "unknown", + "tokens": 18995, + "fixed_tokens": 1114, + "processed_tokens": 330275, + "tool_calls": 30, + "turns": 23 + }, + "world-035": { + "answer": false, + "tokens": 11093, + "fixed_tokens": 1113, + "processed_tokens": 121495, + "tool_calls": 11, + "turns": 12 + }, + "world-036": { + "answer": "dec-return-engine-jobs-to-a-thread-pool-with-continu--1zkc1v2x97taf50h: 41% of alpha crash reports had no usable stack (unreadable fiber stacks blocked crash triage for beta), and a continuation-wait thread pool kept workers 93% busy vs 95% for fibers, so fibers gave only two points of utilisation at the cost of unreadable crashes and the Gull thread-local bug.", + "tokens": 23891, + "fixed_tokens": 1117, + "processed_tokens": 276363, + "tool_calls": 15, + "turns": 16 + }, + "world-037": { + "answer": "fnd-hot-reload-drops-suspended-gull-coroutines--va0fn0svryzwbzws", + "tokens": 19579, + "fixed_tokens": 1111, + "processed_tokens": 334729, + "tool_calls": 24, + "turns": 23 + }, + "world-038": { + "answer": 3, + "tokens": 8851, + "fixed_tokens": 1115, + "processed_tokens": 100372, + "tool_calls": 11, + "turns": 11 + }, + "world-039": { + "answer": "unknown", + "tokens": 25682, + "fixed_tokens": 1112, + "processed_tokens": 427019, + "tool_calls": 32, + "turns": 26 + }, + "world-040": { + "answer": "unknown", + "tokens": 19617, + "fixed_tokens": 1112, + "processed_tokens": 212421, + "tool_calls": 14, + "turns": 14 + }, + "world-041": { + "answer": "61%", + "tokens": 12858, + "fixed_tokens": 1112, + "processed_tokens": 172869, + "tool_calls": 17, + "turns": 16 + }, + "world-042": { + "answer": "5.4 s", + "tokens": 10866, + "fixed_tokens": 1112, + "processed_tokens": 150785, + "tool_calls": 14, + "turns": 15 + }, + "world-043": { + "answer": "5.8 minutes", + "tokens": 11141, + "fixed_tokens": 1117, + "processed_tokens": 154165, + "tool_calls": 16, + "turns": 15 + }, + "world-044": { + "answer": "dec-move-engine-jobs-to-fibers--qjkgnm8cex8ydzfq", + "tokens": 15693, + "fixed_tokens": 1110, + "processed_tokens": 211994, + "tool_calls": 17, + "turns": 16 + }, + "world-045": { + "answer": "dec-run-engine-jobs-on-fibers-and-pin-gull-scripts-t--rw28cbqfd07rd9te", + "tokens": 14851, + "fixed_tokens": 1110, + "processed_tokens": 204035, + "tool_calls": 16, + "turns": 16 + }, + "world-046": { + "answer": "con-streaming-pool-budget--632bkbs79bwj4cr5", + "tokens": 9645, + "fixed_tokens": 1116, + "processed_tokens": 52256, + "tool_calls": 5, + "turns": 6 + }, + "world-047": { + "answer": "fnd-world-core-state-at-alpha--tkbsm4867h65jqq8", + "tokens": 8627, + "fixed_tokens": 1110, + "processed_tokens": 57461, + "tool_calls": 6, + "turns": 7 + }, + "world-048": { + "answer": "dec-compile-gull-to-bytecode-for-a-register-vm--jrqje646fpevp5f3", + "tokens": 28603, + "fixed_tokens": 1112, + "processed_tokens": 752754, + "tool_calls": 40, + "turns": 38 + }, + "world-049": { + "answer": "22%", + "tokens": 26229, + "fixed_tokens": 1109, + "processed_tokens": 390648, + "tool_calls": 33, + "turns": 23 + }, + "world-050": { + "answer": "fnd-alpha-content-needs-6-2-ms-of-gull-time-per-fram--4mekgdpsdv5nk085", + "tokens": 31343, + "fixed_tokens": 1114, + "processed_tokens": 963794, + "tool_calls": 46, + "turns": 46 + }, + "world-051": { + "answer": "Float tides drifted apart across machines within 6 minutes (fnd-float-tides-drift-apart-between-machines-within--znd0wy18bfw2prss) from FMA contraction plus a Skiff sin() differing by one ulp, and the constraint requires bit-identical tide state on every peer; dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e chose 32.32 fixed point because integer math and a build-time sine table give identical bits on any CPU/compiler with no platform math library involved. The graph does not record an explicit evaluation of strict IEEE mode.", + "tokens": 25784, + "fixed_tokens": 1112, + "processed_tokens": 406070, + "tool_calls": 20, + "turns": 20 + }, + "world-052": { + "answer": "Priya Natarajan", + "tokens": 13603, + "fixed_tokens": 1115, + "processed_tokens": 133666, + "tool_calls": 10, + "turns": 11 + }, + "world-053": { + "answer": null, + "tokens": 51713, + "fixed_tokens": 1113, + "processed_tokens": 2777023, + "tool_calls": 91, + "turns": 91 + }, + "world-054": { + "answer": "Marta Lindqvist", + "tokens": 13524, + "fixed_tokens": 1107, + "processed_tokens": 165279, + "tool_calls": 13, + "turns": 14 + }, + "world-055": { + "answer": "dec-shrink-kelp-s-dictionary-to-1-mb-for-streamed-as--yh8hktfqnyqhepew: the cold-cache rerun showed only the 4 MB dictionary losing to Zstandard on Skiff, so Kelp stays with a 1 MB dictionary, which cold on Skiff beats Zstandard by 1.7x (it decodes at 2.2 GB/s), closing the reopened Zstandard option.", + "tokens": 25086, + "fixed_tokens": 1114, + "processed_tokens": 284169, + "tool_calls": 16, + "turns": 15 + }, + "world-056": { + "answer": "dec-run-engine-jobs-on-a-work-stealing-thread-pool--1pd72x1pxpgwy21x", + "tokens": 7807, + "fixed_tokens": 1113, + "processed_tokens": 67341, + "tool_calls": 8, + "turns": 8 + }, + "world-057": { + "answer": "(212, 87)", + "tokens": 25189, + "fixed_tokens": 1110, + "processed_tokens": 353511, + "tool_calls": 17, + "turns": 18 + }, + "world-058": { + "answer": "0.8 GB/s", + "tokens": 20673, + "fixed_tokens": 1109, + "processed_tokens": 342219, + "tool_calls": 25, + "turns": 23 + }, + "world-059": { + "answer": "dec-derive-the-missing-harmonic-phase-from-each-save--gbqrvvgyq2gy2gnw", + "tokens": 25111, + "fixed_tokens": 1111, + "processed_tokens": 284904, + "tool_calls": 15, + "turns": 16 + }, + "world-060": { + "answer": "dec-partition-the-world-into-192-m-cells--bk9r1c5jwb6fh8tc", + "tokens": 10451, + "fixed_tokens": 1115, + "processed_tokens": 143702, + "tool_calls": 14, + "turns": 14 + }, + "world-061": { + "answer": "236 MB", + "tokens": 9660, + "fixed_tokens": 1110, + "processed_tokens": 115969, + "tool_calls": 11, + "turns": 12 + }, + "world-062": { + "answer": "310 MB", + "tokens": 24200, + "fixed_tokens": 1116, + "processed_tokens": 398390, + "tool_calls": 21, + "turns": 20 + }, + "world-063": { + "answer": "con-streaming-pool-budget--632bkbs79bwj4cr5", + "tokens": 25770, + "fixed_tokens": 1113, + "processed_tokens": 242326, + "tool_calls": 15, + "turns": 14 + }, + "world-064": { + "answer": [ + "dec-force-strict-ieee-float-mode-for-the-tide-simula--ecwmqt7ncjngysqw", + "dec-have-the-host-stream-tide-heights-to-every-peer--a3rb1seedn2bj7k8" + ], + "tokens": 21487, + "fixed_tokens": 1114, + "processed_tokens": 219738, + "tool_calls": 15, + "turns": 15 + }, + "world-065": { + "answer": "3.1 ms", + "tokens": 25156, + "fixed_tokens": 1112, + "processed_tokens": 310739, + "tool_calls": 25, + "turns": 20 + }, + "world-066": { + "answer": 46, + "tokens": 29477, + "fixed_tokens": 1112, + "processed_tokens": 439718, + "tool_calls": 19, + "turns": 20 + }, + "world-067": { + "answer": "1.2 GB/s", + "tokens": 9748, + "fixed_tokens": 1111, + "processed_tokens": 118079, + "tool_calls": 11, + "turns": 12 + }, + "world-068": { + "answer": true, + "tokens": 16830, + "fixed_tokens": 1111, + "processed_tokens": 200934, + "tool_calls": 15, + "turns": 15 + }, + "world-069": { + "answer": 6, + "tokens": 32357, + "fixed_tokens": 1110, + "processed_tokens": 394444, + "tool_calls": 18, + "turns": 17 + }, + "world-070": { + "answer": 3, + "tokens": 8060, + "fixed_tokens": 1112, + "processed_tokens": 67729, + "tool_calls": 7, + "turns": 8 + }, + "world-071": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--yxsjd26wcaz6fgx9", + "dec-compress-streamed-assets-with-kelp--ezwc5p1tgsgm7v8m", + "dec-evict-cells-by-distance-and-tide-forecast--fyp1jp366pjjzxcg", + "dec-hot-reload-gull-functions-by-swapping-bytecode-a--8dxnkg6h8vmcsr42", + "dec-stamp-each-save-with-a-schema-version-and-upgrad--nbg1h8g1bhn2ks8w" + ], + "tokens": 63595, + "fixed_tokens": 1108, + "processed_tokens": 1094641, + "tool_calls": 44, + "turns": 32 + }, + "world-072": { + "answer": [ + "dec-compress-streamed-assets-with-zstandard--bs1t15cygc51cpje", + "dec-compress-streamed-assets-with-lz4--v1pfsgfp0b3yjv4s" + ], + "tokens": 17938, + "fixed_tokens": 1108, + "processed_tokens": 225763, + "tool_calls": 19, + "turns": 17 + }, + "world-073": { + "answer": false, + "tokens": 14454, + "fixed_tokens": 1110, + "processed_tokens": 156311, + "tool_calls": 14, + "turns": 13 + }, + "world-074": { + "answer": 7, + "tokens": 4182, + "fixed_tokens": 1112, + "processed_tokens": 41067, + "tool_calls": 5, + "turns": 6 + }, + "world-075": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--8679sjn071ycsbpv", + "dec-partition-the-world-into-192-meter-cells--q62ddwdv2n8wxay6", + "dec-return-engine-jobs-to-the-thread-pool-with-conti--547qbweqvrp1j5g3" + ], + "tokens": 5017, + "fixed_tokens": 1112, + "processed_tokens": 42821, + "tool_calls": 5, + "turns": 6 + }, + "world-076": { + "answer": "dec-compile-gull-to-bytecode-for-a-register-vm--jrqje646fpevp5f3", + "tokens": 14086, + "fixed_tokens": 1111, + "processed_tokens": 178940, + "tool_calls": 17, + "turns": 15 + }, + "world-077": { + "answer": 251, + "tokens": 8707, + "fixed_tokens": 1112, + "processed_tokens": 97153, + "tool_calls": 10, + "turns": 11 + }, + "world-078": { + "answer": [ + "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "rsk-reworking-the-read-scheduler-may-slip-the-coast--9jzvs7etvb5jxqtp" + ], + "tokens": 5017, + "fixed_tokens": 1113, + "processed_tokens": 50096, + "tool_calls": 6, + "turns": 7 + }, + "world-079": { + "answer": [ + "con-no-runtime-code-generation-on-consoles--6cf25zqmsvqkdd73", + "con-tide-state-must-be-bit-identical-on-every-peer--zrpnp8qjyxzkas7t", + "con-saves-from-beta-onward-must-load-in-every-later--v6z856sxtgc5sb09", + "con-streaming-pool-budget--8czwb4fjr5p5vpy7" + ], + "tokens": 9718, + "fixed_tokens": 1115, + "processed_tokens": 59302, + "tool_calls": 6, + "turns": 7 + }, + "world-080": { + "answer": [ + "iss-gull-hot-reload-rejects-struct-layout-changes--3v2w2f9hg2gwy6aj", + "iss-gull-hot-reload-drops-suspended-coroutines--z7j2gmyny6gdxd07", + "iss-cross-platform-cloud-saves-are-unplanned--5bk0k4wrgj42ztfy" + ], + "tokens": 17773, + "fixed_tokens": 1115, + "processed_tokens": 116592, + "tool_calls": 9, + "turns": 9 + }, + "world-081": { + "answer": "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "tokens": 17082, + "fixed_tokens": 1114, + "processed_tokens": 266594, + "tool_calls": 20, + "turns": 20 + } + } +} diff --git a/benchmarks/proof-recall/runs/x10adv-find-r1-find.json b/benchmarks/proof-recall/runs/x10adv-find-r1-find.json new file mode 100644 index 00000000..bf93122a --- /dev/null +++ b/benchmarks/proof-recall/runs/x10adv-find-r1-find.json @@ -0,0 +1,2259 @@ +{ + "run": { + "name": "x10adv-find-r1-find", + "protocol": "one fresh session per question", + "models": ["claude-haiku-5-5"], + "find_hash": "e5be653299abe9b2329822d98ffe09e90ee7cfe96f8711c813a51dc6500b887f" + }, + "answers": { + "ambiguous-001": { + "answer": "384 MB", + "tokens": 12916, + "fixed_tokens": 2734, + "processed_tokens": 78852, + "tool_calls": 6, + "turns": 7 + }, + "ambiguous-002": { + "answer": "unknown", + "tokens": 12262, + "fixed_tokens": 2738, + "processed_tokens": 66478, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-003": { + "answer": "92.4%", + "tokens": 13099, + "fixed_tokens": 2737, + "processed_tokens": 116977, + "tool_calls": 9, + "turns": 10 + }, + "ambiguous-004": { + "answer": "6 min 40 s", + "tokens": 9790, + "fixed_tokens": 2732, + "processed_tokens": 59978, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-005": { + "answer": "6 weeks", + "tokens": 8095, + "fixed_tokens": 1109, + "processed_tokens": 59725, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-006": { + "answer": "16-bit float per channel", + "tokens": 13607, + "fixed_tokens": 1105, + "processed_tokens": 82223, + "tool_calls": 6, + "turns": 7 + }, + "ambiguous-007": { + "answer": "32 steps", + "tokens": 11070, + "fixed_tokens": 1110, + "processed_tokens": 77590, + "tool_calls": 6, + "turns": 7 + }, + "ambiguous-008": { + "answer": "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "tokens": 9960, + "fixed_tokens": 1107, + "processed_tokens": 88054, + "tool_calls": 7, + "turns": 8 + }, + "ambiguous-009": { + "answer": "3.0 ms", + "tokens": 12483, + "fixed_tokens": 1112, + "processed_tokens": 79971, + "tool_calls": 6, + "turns": 7 + }, + "ambiguous-010": { + "answer": "160 MB/s", + "tokens": 7864, + "fixed_tokens": 1111, + "processed_tokens": 59264, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-011": { + "answer": "260 MB", + "tokens": 13291, + "fixed_tokens": 1111, + "processed_tokens": 108883, + "tool_calls": 8, + "turns": 9 + }, + "ambiguous-012": { + "answer": "11 m", + "tokens": 11195, + "fixed_tokens": 1106, + "processed_tokens": 78015, + "tool_calls": 6, + "turns": 7 + }, + "ambiguous-013": { + "answer": "61%", + "tokens": 7781, + "fixed_tokens": 1108, + "processed_tokens": 59171, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-014": { + "answer": "textures", + "tokens": 10916, + "fixed_tokens": 1111, + "processed_tokens": 79515, + "tool_calls": 6, + "turns": 7 + }, + "ambiguous-015": { + "answer": "unknown", + "tokens": 39231, + "fixed_tokens": 1113, + "processed_tokens": 321131, + "tool_calls": 13, + "turns": 14 + }, + "ambiguous-016": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 7645, + "fixed_tokens": 1103, + "processed_tokens": 58264, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-017": { + "answer": "fnd-skiff-microsd-slows-sharply-on-full-warm-cards--p1rgh36hat2egpz3", + "tokens": 8504, + "fixed_tokens": 1113, + "processed_tokens": 68975, + "tool_calls": 6, + "turns": 7 + }, + "ambiguous-018": { + "answer": "fnd-retail-skiff-units-sustain-only-190-mb-s-of-stre--485sdcmm3gweevvn", + "tokens": 7827, + "fixed_tokens": 1108, + "processed_tokens": 59309, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-019": { + "answer": "7.4 ms", + "tokens": 8165, + "fixed_tokens": 1106, + "processed_tokens": 59784, + "tool_calls": 5, + "turns": 6 + }, + "ambiguous-020": { + "answer": "5912 pages", + "tokens": 9621, + "fixed_tokens": 1112, + "processed_tokens": 87972, + "tool_calls": 7, + "turns": 8 + }, + "ambiguous-021": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 7482, + "fixed_tokens": 1112, + "processed_tokens": 65538, + "tool_calls": 6, + "turns": 7 + }, + "ambiguous-022": { + "answer": "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "tokens": 13673, + "fixed_tokens": 1107, + "processed_tokens": 124210, + "tool_calls": 9, + "turns": 10 + }, + "ambiguous-023": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 10299, + "fixed_tokens": 1105, + "processed_tokens": 78915, + "tool_calls": 6, + "turns": 7 + }, + "ambiguous-024": { + "answer": "fnd-streaming-tide-heights-costs-46-kb-s-per-peer--knxqtyqh6xcz1tp9", + "tokens": 13901, + "fixed_tokens": 1109, + "processed_tokens": 143430, + "tool_calls": 10, + "turns": 11 + }, + "cross-001": { + "answer": "con-frame-budget--br0qszfg8fys908n", + "tokens": 7588, + "fixed_tokens": 1106, + "processed_tokens": 58716, + "tool_calls": 5, + "turns": 6 + }, + "cross-002": { + "answer": 2, + "tokens": 15314, + "fixed_tokens": 1100, + "processed_tokens": 107915, + "tool_calls": 9, + "turns": 8 + }, + "cross-003": { + "answer": 2, + "tokens": 4930, + "fixed_tokens": 1097, + "processed_tokens": 31672, + "tool_calls": 3, + "turns": 4 + }, + "cross-004": { + "answer": "dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e", + "tokens": 9750, + "fixed_tokens": 1097, + "processed_tokens": 87642, + "tool_calls": 7, + "turns": 8 + }, + "cross-005": { + "answer": "Rhea Valdés", + "tokens": 13573, + "fixed_tokens": 1097, + "processed_tokens": 124358, + "tool_calls": 9, + "turns": 10 + }, + "cross-006": { + "answer": "19%", + "tokens": 9658, + "fixed_tokens": 1096, + "processed_tokens": 73159, + "tool_calls": 6, + "turns": 7 + }, + "cross-007": { + "answer": "8 GB", + "tokens": 16225, + "fixed_tokens": 1102, + "processed_tokens": 171400, + "tool_calls": 10, + "turns": 11 + }, + "cross-008": { + "answer": "unknown", + "tokens": 16563, + "fixed_tokens": 1103, + "processed_tokens": 150273, + "tool_calls": 10, + "turns": 11 + }, + "cross-009": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 14797, + "fixed_tokens": 1102, + "processed_tokens": 119939, + "tool_calls": 8, + "turns": 9 + }, + "cross-010": { + "answer": "dec-build-an-on-device-pipeline-cache-for-skiff--726r9mtk76qndeh7", + "tokens": 9751, + "fixed_tokens": 1097, + "processed_tokens": 63312, + "tool_calls": 7, + "turns": 6 + }, + "cross-011": { + "answer": false, + "tokens": 8346, + "fixed_tokens": 1098, + "processed_tokens": 58855, + "tool_calls": 5, + "turns": 6 + }, + "cross-012": { + "answer": "61 MB/s", + "tokens": 9423, + "fixed_tokens": 1099, + "processed_tokens": 88208, + "tool_calls": 7, + "turns": 8 + }, + "cross-013": { + "answer": "Skiff runs Tern at the Performance preset, which saves 1.8 ms over Quality (2.1 ms vs 3.9 ms); the premise about the lowest preset is true for Meridian (Ultra Performance), but Skiff does not use it. Record: dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 15569, + "fixed_tokens": 1101, + "processed_tokens": 102637, + "tool_calls": 7, + "turns": 8 + }, + "cross-014": { + "answer": 3, + "tokens": 4793, + "fixed_tokens": 1097, + "processed_tokens": 39257, + "tool_calls": 4, + "turns": 5 + }, + "cross-015": { + "answer": "Two Efforts are no longer active. The Shader compile cache and build farm Effort is completed: every farm build reads from the cache after dec-turn-the-shader-compile-cache-on-for-every-farm--4rr3b12b4sasdmaj (accepted 2026-03-20), which the retrospective fnd-retrospective-on-the-shader-compile-cache--9rpv2a60ach04cx6 confirms. The Co-op rollback netcode prototype Effort is abandoned: dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b stopped rollback because re-running tides on each rolled-back frame costs 4.8 ms on Skiff, and co-op will not ship with rollback.", + "tokens": 38092, + "fixed_tokens": 1094, + "processed_tokens": 241537, + "tool_calls": 13, + "turns": 10 + }, + "cross-016": { + "answer": "16.7 ms", + "tokens": 13105, + "fixed_tokens": 1100, + "processed_tokens": 83804, + "tool_calls": 8, + "turns": 7 + }, + "cross-017": { + "answer": "false dec-ship-tern-as-the-only-upscaler-in-lantern--a6gbj8djpd8p6w67", + "tokens": 15600, + "fixed_tokens": 1099, + "processed_tokens": 102402, + "tool_calls": 7, + "turns": 8 + }, + "netcode-001": { + "answer": "abandoned", + "tokens": 8675, + "fixed_tokens": 1098, + "processed_tokens": 60553, + "tool_calls": 5, + "turns": 6 + }, + "netcode-002": { + "answer": "netcode/rollback-archive", + "tokens": 7901, + "fixed_tokens": 1095, + "processed_tokens": 58979, + "tool_calls": 5, + "turns": 6 + }, + "netcode-003": { + "answer": "38 KB", + "tokens": 14610, + "fixed_tokens": 1102, + "processed_tokens": 85363, + "tool_calls": 6, + "turns": 7 + }, + "netcode-004": { + "answer": "The team abandoned rollback because re-running the tide simulation on every rolled-back frame costs 4.8 ms on Skiff, about a fifth of the 25 ms frame, while co-op needs bit-for-bit deterministic tides (dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b). The reason depends on the world Decision \"Run the tide simulation in 32.32 fixed point\" (dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e).", + "tokens": 11843, + "fixed_tokens": 1106, + "processed_tokens": 78486, + "tool_calls": 6, + "turns": 7 + }, + "netcode-005": { + "answer": "unknown", + "tokens": 17145, + "fixed_tokens": 1102, + "processed_tokens": 118394, + "tool_calls": 7, + "turns": 8 + }, + "netcode-006": { + "answer": "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "tokens": 12003, + "fixed_tokens": 1100, + "processed_tokens": 119397, + "tool_calls": 9, + "turns": 10 + }, + "netcode-007": { + "answer": "dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b", + "tokens": 8041, + "fixed_tokens": 1099, + "processed_tokens": 59310, + "tool_calls": 5, + "turns": 6 + }, + "netcode-008": { + "answer": "dec-use-a-4-frame-rollback-window--pqe4hvbgmbp0dmh9", + "tokens": 18639, + "fixed_tokens": 1099, + "processed_tokens": 154132, + "tool_calls": 10, + "turns": 11 + }, + "netcode-009": { + "answer": "1.9 ms", + "tokens": 10826, + "fixed_tokens": 1096, + "processed_tokens": 116587, + "tool_calls": 9, + "turns": 10 + }, + "netcode-010": { + "answer": "dec-use-a-6-frame-rollback-window-with-1-frame-of-in--s53944zhkw103frk", + "tokens": 11472, + "fixed_tokens": 1101, + "processed_tokens": 80345, + "tool_calls": 6, + "turns": 7 + }, + "netcode-011": { + "answer": "con-co-op-peers-must-agree-on-tide-state-bit-for-bit--4v0sdbc3t4ywzr3g", + "tokens": 23383, + "fixed_tokens": 1104, + "processed_tokens": 245097, + "tool_calls": 15, + "turns": 14 + }, + "netcode-012": { + "answer": "unknown", + "tokens": 17366, + "fixed_tokens": 1103, + "processed_tokens": 163747, + "tool_calls": 11, + "turns": 12 + }, + "netcode-013": { + "answer": [ + "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "fnd-rollback-re-simulates-a-4-frame-window-in-1-1-ms--xa30a1cez2brphyf", + "rsk-re-simulation-cost-could-blow-the-skiff-frame-bu--sk2njsp170a3qwyf" + ], + "tokens": 29704, + "fixed_tokens": 1101, + "processed_tokens": 211660, + "tool_calls": 26, + "turns": 11 + }, + "netcode-014": { + "answer": [ + "iss-late-join-during-a-tide-surge-desyncs-the-joinin--mya93va0n7s8g7cj" + ], + "tokens": 39123, + "fixed_tokens": 1098, + "processed_tokens": 355473, + "tool_calls": 14, + "turns": 15 + }, + "netcode-015": { + "answer": 2, + "tokens": 25526, + "fixed_tokens": 1102, + "processed_tokens": 119622, + "tool_calls": 9, + "turns": 9 + }, + "ports-001": { + "answer": "25 ms", + "tokens": 9077, + "fixed_tokens": 1101, + "processed_tokens": 86137, + "tool_calls": 7, + "turns": 8 + }, + "ports-002": { + "answer": "384 MB", + "tokens": 7892, + "fixed_tokens": 1095, + "processed_tokens": 58471, + "tool_calls": 5, + "turns": 6 + }, + "ports-003": { + "answer": "61 MB/s", + "tokens": 8239, + "fixed_tokens": 1100, + "processed_tokens": 60103, + "tool_calls": 5, + "turns": 6 + }, + "ports-004": { + "answer": "accepted", + "tokens": 11332, + "fixed_tokens": 1102, + "processed_tokens": 90204, + "tool_calls": 7, + "turns": 8 + }, + "ports-005": { + "answer": "unknown", + "tokens": 26735, + "fixed_tokens": 1095, + "processed_tokens": 218234, + "tool_calls": 10, + "turns": 11 + }, + "ports-034": { + "answer": "6 weeks", + "tokens": 7809, + "fixed_tokens": 1094, + "processed_tokens": 58966, + "tool_calls": 5, + "turns": 6 + }, + "ports-006": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 8389, + "fixed_tokens": 1093, + "processed_tokens": 69035, + "tool_calls": 6, + "turns": 7 + }, + "ports-007": { + "answer": "SKF-UI-031", + "tokens": 7682, + "fixed_tokens": 1099, + "processed_tokens": 58808, + "tool_calls": 5, + "turns": 6 + }, + "ports-008": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 8055, + "fixed_tokens": 1101, + "processed_tokens": 59685, + "tool_calls": 5, + "turns": 6 + }, + "ports-009": { + "answer": "dec-load-skiff-fast-travel-without-a-loading-screen--v4qft5znysjv5f24", + "tokens": 7897, + "fixed_tokens": 1097, + "processed_tokens": 58513, + "tool_calls": 5, + "turns": 6 + }, + "ports-010": { + "answer": "con-frame-budget--wt5rfqh5z444g2cg", + "tokens": 14087, + "fixed_tokens": 1106, + "processed_tokens": 141411, + "tool_calls": 11, + "turns": 11 + }, + "ports-011": { + "answer": "99.6%", + "tokens": 7490, + "fixed_tokens": 1104, + "processed_tokens": 58181, + "tool_calls": 5, + "turns": 6 + }, + "ports-012": { + "answer": "The ports team raised the Skiff dynamic resolution floor to 576p in dec-raise-the-skiff-dynamic-resolution-floor-to-576p--fep9q7m08pp9f0bb because 540p internal resolution made saltglass caustics shimmer badly in motion, even with Tern (fnd-saltglass-caustics-shimmer-at-the-540p-floor--0aa8sts89vhq1s6q). That decision was later superseded by a 504p floor.", + "tokens": 7964, + "fixed_tokens": 1096, + "processed_tokens": 58254, + "tool_calls": 5, + "turns": 6 + }, + "ports-013": { + "answer": "Ship Tern as Lantern's only upscaler", + "tokens": 10091, + "fixed_tokens": 1106, + "processed_tokens": 89539, + "tool_calls": 7, + "turns": 8 + }, + "ports-014": { + "answer": "dec-scale-skiff-resolution-dynamically-between-540p--jtytvgh2erd32x68", + "tokens": 27552, + "fixed_tokens": 1100, + "processed_tokens": 189237, + "tool_calls": 11, + "turns": 10 + }, + "ports-015": { + "answer": "textures", + "tokens": 9101, + "fixed_tokens": 1100, + "processed_tokens": 85806, + "tool_calls": 7, + "turns": 8 + }, + "ports-016": { + "answer": "dec-ship-prebuilt-pipeline-blobs-in-the-skiff-packag--3vjysjgy9qkvzebn", + "tokens": 10303, + "fixed_tokens": 1100, + "processed_tokens": 89489, + "tool_calls": 7, + "turns": 8 + }, + "ports-017": { + "answer": "fnd-skiff-microsd-slows-sharply-on-full-warm-cards--p1rgh36hat2egpz3", + "tokens": 8116, + "fixed_tokens": 1102, + "processed_tokens": 59671, + "tool_calls": 5, + "turns": 6 + }, + "ports-018": { + "answer": "504p", + "tokens": 13655, + "fixed_tokens": 1096, + "processed_tokens": 90770, + "tool_calls": 7, + "turns": 8 + }, + "ports-019": { + "answer": "4 MB per minute", + "tokens": 8888, + "fixed_tokens": 1105, + "processed_tokens": 84833, + "tool_calls": 7, + "turns": 8 + }, + "ports-020": { + "answer": "fnd-skiff-sdk-1-2-takes-1-gb-back-from-games--a0ybdgjzw64hwfqa", + "tokens": 10590, + "fixed_tokens": 1097, + "processed_tokens": 116471, + "tool_calls": 9, + "turns": 10 + }, + "ports-021": { + "answer": "SKF-DSP-004", + "tokens": 9250, + "fixed_tokens": 1102, + "processed_tokens": 94026, + "tool_calls": 8, + "turns": 9 + }, + "ports-022": { + "answer": "cit-skiff-sdk-1-2-release-notes--7gwwn8xxffrx6npe", + "tokens": 12095, + "fixed_tokens": 1094, + "processed_tokens": 138648, + "tool_calls": 12, + "turns": 11 + }, + "ports-023": { + "answer": "fnd-tern-costs-3-9-ms-at-quality-and-2-1-ms-at-perfo--dkbskbqfgfwtryq0", + "tokens": 16222, + "fixed_tokens": 1097, + "processed_tokens": 138261, + "tool_calls": 9, + "turns": 10 + }, + "ports-024": { + "answer": "fnd-skiff-firmware-1-2-rejects-pipeline-blobs-built--pernvgb2he7m3z3y", + "tokens": 11688, + "fixed_tokens": 1097, + "processed_tokens": 148930, + "tool_calls": 11, + "turns": 12 + }, + "ports-025": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 11099, + "fixed_tokens": 1105, + "processed_tokens": 105683, + "tool_calls": 8, + "turns": 9 + }, + "ports-026": { + "answer": "912 MB", + "tokens": 12748, + "fixed_tokens": 1096, + "processed_tokens": 154327, + "tool_calls": 11, + "turns": 12 + }, + "ports-027": { + "answer": [ + "dec-stream-skiff-world-tiles-in-2-mb-reads--3jz9w9gvhhc7eppw", + "rsk-tide-streaming-outruns-skiff-storage--6q8s3b383z2tyeyb" + ], + "tokens": 48644, + "fixed_tokens": 1106, + "processed_tokens": 574034, + "tool_calls": 20, + "turns": 20 + }, + "ports-028": { + "answer": 2, + "tokens": 7463, + "fixed_tokens": 1102, + "processed_tokens": 53082, + "tool_calls": 5, + "turns": 6 + }, + "ports-029": { + "answer": 8, + "tokens": 58702, + "fixed_tokens": 1098, + "processed_tokens": 1984060, + "tool_calls": 56, + "turns": 53 + }, + "ports-030": { + "answer": [ + "dec-cap-tide-storm-particles-at-6-000-on-skiff--resze52afjxyxys1", + "dec-drop-skiff-to-30-hz-while-a-tide-storm-is-on-scr--72vx1c8s4jrs6axc" + ], + "tokens": 6517, + "fixed_tokens": 1103, + "processed_tokens": 51177, + "tool_calls": 5, + "turns": 6 + }, + "ports-031": { + "answer": [ + "iss-meridian-cert-activity-cards-missing-for-two-bio--k214cmfra7pyvvzq", + "iss-skiff-cert-docked-30-hz-judder-on-some-tvs--9nq2xg9mzash474g" + ], + "tokens": 26409, + "fixed_tokens": 1099, + "processed_tokens": 229800, + "tool_calls": 11, + "turns": 12 + }, + "ports-032": { + "answer": 2, + "tokens": 6825, + "fixed_tokens": 1098, + "processed_tokens": 62709, + "tool_calls": 6, + "turns": 7 + }, + "ports-033": { + "answer": "unknown", + "tokens": 9802, + "fixed_tokens": 1099, + "processed_tokens": 60485, + "tool_calls": 5, + "turns": 6 + }, + "render-001": { + "answer": "7.4 ms", + "tokens": 9091, + "fixed_tokens": 1093, + "processed_tokens": 87030, + "tool_calls": 7, + "turns": 8 + }, + "render-002": { + "answer": "3.4x", + "tokens": 7431, + "fixed_tokens": 1097, + "processed_tokens": 57585, + "tool_calls": 5, + "turns": 6 + }, + "render-003": { + "answer": "Tomasz Brandt", + "tokens": 9296, + "fixed_tokens": 1094, + "processed_tokens": 86578, + "tool_calls": 7, + "turns": 8 + }, + "render-004": { + "answer": "rejected", + "tokens": 9642, + "fixed_tokens": 1100, + "processed_tokens": 88528, + "tool_calls": 7, + "turns": 8 + }, + "render-005": { + "answer": true, + "tokens": 11363, + "fixed_tokens": 1097, + "processed_tokens": 77333, + "tool_calls": 6, + "turns": 7 + }, + "render-006": { + "answer": "unknown", + "tokens": 8870, + "fixed_tokens": 1094, + "processed_tokens": 71542, + "tool_calls": 6, + "turns": 7 + }, + "render-007": { + "answer": "2026-07-24", + "tokens": 8343, + "fixed_tokens": 1102, + "processed_tokens": 60345, + "tool_calls": 5, + "turns": 6 + }, + "render-008": { + "answer": "2", + "tokens": 7653, + "fixed_tokens": 1100, + "processed_tokens": 58639, + "tool_calls": 5, + "turns": 6 + }, + "render-009": { + "answer": "32 steps", + "tokens": 11019, + "fixed_tokens": 1101, + "processed_tokens": 77457, + "tool_calls": 6, + "turns": 7 + }, + "render-010": { + "answer": "unknown", + "tokens": 10345, + "fixed_tokens": 1096, + "processed_tokens": 87872, + "tool_calls": 7, + "turns": 8 + }, + "render-011": { + "answer": "fnd-sdf-gi-costs-2-4-ms-on-meridian--2w6c2pc5zmcsyjre", + "tokens": 10391, + "fixed_tokens": 1096, + "processed_tokens": 76657, + "tool_calls": 6, + "turns": 7 + }, + "render-012": { + "answer": "deferred", + "tokens": 9866, + "fixed_tokens": 1092, + "processed_tokens": 87931, + "tool_calls": 7, + "turns": 8 + }, + "render-013": { + "answer": "16-bit float per channel", + "tokens": 11562, + "fixed_tokens": 1100, + "processed_tokens": 79067, + "tool_calls": 6, + "turns": 7 + }, + "render-014": { + "answer": "52%", + "tokens": 8334, + "fixed_tokens": 1095, + "processed_tokens": 60889, + "tool_calls": 5, + "turns": 6 + }, + "render-015": { + "answer": "4 minutes", + "tokens": 12367, + "fixed_tokens": 1096, + "processed_tokens": 105905, + "tool_calls": 8, + "turns": 9 + }, + "render-016": { + "answer": "1.2 km", + "tokens": 8208, + "fixed_tokens": 1104, + "processed_tokens": 60378, + "tool_calls": 5, + "turns": 6 + }, + "render-017": { + "answer": "unknown", + "tokens": 10909, + "fixed_tokens": 1094, + "processed_tokens": 76511, + "tool_calls": 6, + "turns": 7 + }, + "render-018": { + "answer": "unknown", + "tokens": 14241, + "fixed_tokens": 1097, + "processed_tokens": 98635, + "tool_calls": 7, + "turns": 8 + }, + "render-019": { + "answer": "dec-use-the-custom-brine-curve-for-tonemapping--p735g6wf2njare86", + "tokens": 8967, + "fixed_tokens": 1091, + "processed_tokens": 84697, + "tool_calls": 7, + "turns": 8 + }, + "render-020": { + "answer": "fnd-screen-space-refraction-loses-off-screen-content--k8zvmmdee7wg2fjy", + "tokens": 7420, + "fixed_tokens": 1097, + "processed_tokens": 57976, + "tool_calls": 5, + "turns": 6 + }, + "render-021": { + "answer": "dec-adopt-a-visibility-buffer-for-all-opaque-geometr--68vchzqgd1avw8q4", + "tokens": 10684, + "fixed_tokens": 1096, + "processed_tokens": 88962, + "tool_calls": 7, + "turns": 8 + }, + "render-022": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 21662, + "fixed_tokens": 1099, + "processed_tokens": 178293, + "tool_calls": 10, + "turns": 11 + }, + "render-023": { + "answer": "dec-capture-the-cliffs-and-dunes-benchmarks-on-every--3pqyr66yh4v6q676", + "tokens": 7196, + "fixed_tokens": 1097, + "processed_tokens": 56752, + "tool_calls": 5, + "turns": 6 + }, + "render-024": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 9437, + "fixed_tokens": 1101, + "processed_tokens": 62960, + "tool_calls": 7, + "turns": 6 + }, + "render-025": { + "answer": "fnd-tern-cost-at-4k-output-on-meridian-after-fp16-pa--p4gcmzw2a4bvzjev", + "tokens": 24481, + "fixed_tokens": 1093, + "processed_tokens": 197856, + "tool_calls": 11, + "turns": 12 + }, + "render-026": { + "answer": "dec-ship-four-tern-quality-presets--vjw18d82dx5gath8", + "tokens": 7512, + "fixed_tokens": 1093, + "processed_tokens": 58112, + "tool_calls": 5, + "turns": 6 + }, + "render-027": { + "answer": "con-frame-budget--j6vneva1g2ywj86m", + "tokens": 8485, + "fixed_tokens": 1096, + "processed_tokens": 89923, + "tool_calls": 8, + "turns": 9 + }, + "render-028": { + "answer": "dec-move-to-gpu-driven-two-phase-occlusion-culling--6dre481mabxsw6ee", + "tokens": 9108, + "fixed_tokens": 1098, + "processed_tokens": 85759, + "tool_calls": 7, + "turns": 8 + }, + "render-029": { + "answer": "fnd-vsm-page-pool-overflows-during-spring-tides--m2p8e89hhdmh0z8a", + "tokens": 8127, + "fixed_tokens": 1096, + "processed_tokens": 59813, + "tool_calls": 5, + "turns": 6 + }, + "render-030": { + "answer": "dec-use-screen-space-refraction-for-saltglass--vknzksgdrgr7zkg6", + "tokens": 36996, + "fixed_tokens": 1093, + "processed_tokens": 386470, + "tool_calls": 15, + "turns": 16 + }, + "render-031": { + "answer": "Alpha-tested foliage cost 2.1 ms in the visibility buffer versus 0.8 ms in forward+, so it was moved to a forward pass; saltglass was moved so it could refract the resolved scene (dec-visibility-buffer-for-opaque-geometry-foliage-an--gvnvp04jgm7kbfh9).", + "tokens": 8321, + "fixed_tokens": 1096, + "processed_tokens": 59611, + "tool_calls": 5, + "turns": 6 + }, + "render-032": { + "answer": "Tern was chosen as Lantern's only upscaler because Halcyon has no Skiff build, and shipping it on two platforms would break the one-upscaler constraint; owning the code also mattered more than raw quality for saltglass, since its upscaler changes can be made in-house rather than through vendor tickets. See dec-ship-tern-as-lantern-s-only-upscaler--fmreq35nzyjc7k4e.", + "tokens": 8168, + "fixed_tokens": 1093, + "processed_tokens": 59019, + "tool_calls": 5, + "turns": 6 + }, + "render-033": { + "answer": "128 MB", + "tokens": 8022, + "fixed_tokens": 1097, + "processed_tokens": 59647, + "tool_calls": 5, + "turns": 6 + }, + "render-034": { + "answer": "PC workstation", + "tokens": 10855, + "fixed_tokens": 1098, + "processed_tokens": 92913, + "tool_calls": 9, + "turns": 8 + }, + "render-035": { + "answer": "5912 pages", + "tokens": 9359, + "fixed_tokens": 1097, + "processed_tokens": 87753, + "tool_calls": 7, + "turns": 8 + }, + "render-036": { + "answer": "HDR-07", + "tokens": 7485, + "fixed_tokens": 1100, + "processed_tokens": 58100, + "tool_calls": 5, + "turns": 6 + }, + "render-037": { + "answer": "0.33", + "tokens": 10546, + "fixed_tokens": 1094, + "processed_tokens": 114483, + "tool_calls": 9, + "turns": 10 + }, + "render-038": { + "answer": "dec-visibility-buffer-for-opaque-geometry-foliage-an--gvnvp04jgm7kbfh9", + "tokens": 13836, + "fixed_tokens": 1098, + "processed_tokens": 74324, + "tool_calls": 7, + "turns": 6 + }, + "render-039": { + "answer": "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "tokens": 9560, + "fixed_tokens": 1100, + "processed_tokens": 62141, + "tool_calls": 7, + "turns": 6 + }, + "render-040": { + "answer": "fnd-lantern-at-alpha--ckpd9ecvdjqbcr7c", + "tokens": 12475, + "fixed_tokens": 1095, + "processed_tokens": 90785, + "tool_calls": 7, + "turns": 8 + }, + "render-041": { + "answer": "4.7 ms", + "tokens": 9763, + "fixed_tokens": 1100, + "processed_tokens": 63975, + "tool_calls": 7, + "turns": 6 + }, + "render-042": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 8168, + "fixed_tokens": 1097, + "processed_tokens": 59524, + "tool_calls": 5, + "turns": 6 + }, + "render-043": { + "answer": "fnd-meshlet-cone-culling-rejects-9-percent-of-triang--t6e5x7trzedex105", + "tokens": 7259, + "fixed_tokens": 1101, + "processed_tokens": 57079, + "tool_calls": 5, + "turns": 6 + }, + "render-044": { + "answer": "dec-screen-space-refraction-with-a-ray-marched-fallb--2639xtpyrfrvbx1r", + "tokens": 8124, + "fixed_tokens": 1095, + "processed_tokens": 59644, + "tool_calls": 5, + "turns": 6 + }, + "render-045": { + "answer": "false dec-brine-curve-with-an-hdr-shoulder-at-800-nits--fpznx0cnj4avsxk1", + "tokens": 6297, + "fixed_tokens": 1101, + "processed_tokens": 34143, + "tool_calls": 3, + "turns": 4 + }, + "render-046": { + "answer": "The cost finding that had sunk virtual shadow maps was a capture error; with cached pages, VSM costs 1.4 ms, less than four cascades, and CSM still leaves unsolved cascade seams through saltglass (dec-use-virtual-shadow-maps-for-the-sun--x8nqkn9q7eaf1ap4, reopened 2026-01-12 after being rejected_by dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr).", + "tokens": 8626, + "fixed_tokens": 1101, + "processed_tokens": 60676, + "tool_calls": 5, + "turns": 6 + }, + "render-047": { + "answer": "It was a re-measurement: a release-build capture put the visibility buffer at 7.4 ms on Skiff against 8.1 ms for forward+ (6.3 ms with half-rate material resolve), so the forward+ fallback was no longer worth keeping. Record: dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 18684, + "fixed_tokens": 1100, + "processed_tokens": 177941, + "tool_calls": 10, + "turns": 11 + }, + "render-048": { + "answer": "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "tokens": 29227, + "fixed_tokens": 1099, + "processed_tokens": 302002, + "tool_calls": 17, + "turns": 16 + }, + "render-049": { + "answer": "dec-use-virtual-shadow-maps-for-the-sun--x8nqkn9q7eaf1ap4", + "tokens": 11054, + "fixed_tokens": 1096, + "processed_tokens": 88700, + "tool_calls": 7, + "turns": 8 + }, + "render-050": { + "answer": "con-frame-budget--fzdrbxns6dmrrmy3", + "tokens": 7078, + "fixed_tokens": 1098, + "processed_tokens": 64558, + "tool_calls": 6, + "turns": 7 + }, + "render-051": { + "answer": "iss-tern-ghosting-on-moving-saltglass--2yr3q6zf476wajqp", + "tokens": 11878, + "fixed_tokens": 1093, + "processed_tokens": 120973, + "tool_calls": 9, + "turns": 10 + }, + "render-052": { + "answer": "61 percent", + "tokens": 7531, + "fixed_tokens": 1099, + "processed_tokens": 58245, + "tool_calls": 5, + "turns": 6 + }, + "render-053": { + "answer": "every frame", + "tokens": 9566, + "fixed_tokens": 1097, + "processed_tokens": 87953, + "tool_calls": 7, + "turns": 8 + }, + "render-054": { + "answer": "debug-gpu-validation", + "tokens": 16109, + "fixed_tokens": 1099, + "processed_tokens": 180346, + "tool_calls": 15, + "turns": 12 + }, + "render-055": { + "answer": "fnd-tern-cost-at-4k-output-on-meridian-with-the-reac--81tcpzvhwzcy6wyn", + "tokens": 15045, + "fixed_tokens": 1098, + "processed_tokens": 137178, + "tool_calls": 11, + "turns": 10 + }, + "render-056": { + "answer": "208,000 meshlets", + "tokens": 13431, + "fixed_tokens": 1096, + "processed_tokens": 184732, + "tool_calls": 13, + "turns": 14 + }, + "render-057": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 14427, + "fixed_tokens": 1095, + "processed_tokens": 153808, + "tool_calls": 10, + "turns": 11 + }, + "render-058": { + "answer": "iss-which-upscaler-should-lantern-ship--g8c2y9tsr856tn2v", + "tokens": 13196, + "fixed_tokens": 1100, + "processed_tokens": 182954, + "tool_calls": 13, + "turns": 14 + }, + "render-059": { + "answer": "fnd-sdf-gi-costs-4-7-ms-on-meridian--sm19rt2f274ajsvr", + "tokens": 11426, + "fixed_tokens": 1094, + "processed_tokens": 94657, + "tool_calls": 7, + "turns": 8 + }, + "render-060": { + "answer": "0.31", + "tokens": 14267, + "fixed_tokens": 1103, + "processed_tokens": 191630, + "tool_calls": 14, + "turns": 14 + }, + "render-061": { + "answer": "con-ui-composites-at-a-fixed-203-nits-in-hdr--de6aft7cvhdezn5s", + "tokens": 10192, + "fixed_tokens": 1094, + "processed_tokens": 114179, + "tool_calls": 9, + "turns": 10 + }, + "render-062": { + "answer": "con-frame-budget--j6vneva1g2ywj86m", + "tokens": 11706, + "fixed_tokens": 1099, + "processed_tokens": 102953, + "tool_calls": 10, + "turns": 9 + }, + "render-063": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 9057, + "fixed_tokens": 1097, + "processed_tokens": 61088, + "tool_calls": 5, + "turns": 6 + }, + "render-064": { + "answer": "dec-use-a-cached-shadow-atlas-for-local-lights--8p6f2v774he38t5j", + "tokens": 17509, + "fixed_tokens": 1095, + "processed_tokens": 122126, + "tool_calls": 10, + "turns": 8 + }, + "render-065": { + "answer": "dec-ship-tern-as-lantern-s-only-upscaler--fmreq35nzyjc7k4e", + "tokens": 9810, + "fixed_tokens": 1097, + "processed_tokens": 88699, + "tool_calls": 8, + "turns": 8 + }, + "render-066": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 8254, + "fixed_tokens": 1095, + "processed_tokens": 59610, + "tool_calls": 5, + "turns": 6 + }, + "render-067": { + "answer": 4, + "tokens": 11025, + "fixed_tokens": 1096, + "processed_tokens": 98032, + "tool_calls": 8, + "turns": 9 + }, + "render-068": { + "answer": [ + "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "dec-lock-caustic-photon-jitter-to-the-tide-phase--85x10k4h11t8nx7c", + "dec-use-the-probe-grid-alone-in-120-hz-performance-m--6dzm1nt40pjfpyae", + "dec-trace-sdf-gi-at-quarter-resolution-in-performanc--06cqt3mqn3fg5h6e", + "dec-read-paper-white-from-the-meridian-system-settin--7crbx73gqw1hn7xq", + "dec-add-an-in-game-paper-white-slider-from-80-to-400--1phfvxby983z8tjf" + ], + "tokens": 10929, + "fixed_tokens": 1091, + "processed_tokens": 59603, + "tool_calls": 5, + "turns": 6 + }, + "render-069": { + "answer": "iss-shadow-pages-run-out-at-spring-low-tide--f4fgy6b44jbkrsmh", + "tokens": 19213, + "fixed_tokens": 1099, + "processed_tokens": 161815, + "tool_calls": 10, + "turns": 11 + }, + "render-070": { + "answer": [ + "dec-enable-meshlet-cone-culling-on-all-platforms--cpyweprmzcqhqsv2", + "dec-use-signed-distance-field-gi--yn366zmp3k7snd9w", + "dec-feed-a-saltglass-reactive-mask-into-tern--xezjs7v3vq3r8e6k" + ], + "tokens": 69132, + "fixed_tokens": 1094, + "processed_tokens": 1096394, + "tool_calls": 29, + "turns": 30 + }, + "render-071": { + "answer": 3, + "tokens": 9731, + "fixed_tokens": 1094, + "processed_tokens": 66085, + "tool_calls": 6, + "turns": 7 + }, + "render-072": { + "answer": [ + "rsk-sdf-volumes-outgrow-the-render-target-memory-bud--xfd2pfztvskdcd2c", + "rsk-artists-cannot-match-concept-paintings-under-the--6cfh9vkamn0kbtcx" + ], + "tokens": 12300, + "fixed_tokens": 1097, + "processed_tokens": 91180, + "tool_calls": 9, + "turns": 8 + }, + "render-073": { + "answer": [ + "rsk-vsm-page-pool-overflows-when-tides-expose-the-fl--x1r161nrdngra4ty", + "rsk-gi-cannot-fit-the-120-hz-performance-mode--tqt3fj839dqfq5s7", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 12196, + "fixed_tokens": 1096, + "processed_tokens": 110522, + "tool_calls": 9, + "turns": 10 + }, + "render-074": { + "answer": 5, + "tokens": 7772, + "fixed_tokens": 1101, + "processed_tokens": 54241, + "tool_calls": 5, + "turns": 6 + }, + "render-075": { + "answer": "3", + "tokens": 50842, + "fixed_tokens": 1094, + "processed_tokens": 1434948, + "tool_calls": 38, + "turns": 39 + }, + "render-076": { + "answer": false, + "tokens": 9396, + "fixed_tokens": 1097, + "processed_tokens": 84941, + "tool_calls": 7, + "turns": 8 + }, + "render-077": { + "answer": true, + "tokens": 9096, + "fixed_tokens": 1100, + "processed_tokens": 90349, + "tool_calls": 8, + "turns": 9 + }, + "render-078": { + "answer": [ + "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 19059, + "fixed_tokens": 1101, + "processed_tokens": 192112, + "tool_calls": 12, + "turns": 13 + }, + "render-079": { + "answer": 5, + "tokens": 57198, + "fixed_tokens": 1097, + "processed_tokens": 711604, + "tool_calls": 20, + "turns": 21 + }, + "render-080": { + "answer": "6", + "tokens": 15101, + "fixed_tokens": 1096, + "processed_tokens": 217589, + "tool_calls": 15, + "turns": 16 + }, + "render-081": { + "answer": "unknown", + "tokens": 18027, + "fixed_tokens": 1102, + "processed_tokens": 116574, + "tool_calls": 7, + "turns": 8 + }, + "render-082": { + "answer": false, + "tokens": 30873, + "fixed_tokens": 1100, + "processed_tokens": 219735, + "tool_calls": 10, + "turns": 11 + }, + "render-083": { + "answer": null, + "tokens": 71220, + "fixed_tokens": 1092, + "processed_tokens": 3346367, + "tool_calls": 71, + "turns": 71 + }, + "tools-001": { + "answer": "completed", + "tokens": 10222, + "fixed_tokens": 1100, + "processed_tokens": 90463, + "tool_calls": 7, + "turns": 8 + }, + "tools-002": { + "answer": "47 minutes", + "tokens": 12802, + "fixed_tokens": 1096, + "processed_tokens": 106978, + "tool_calls": 8, + "turns": 9 + }, + "tools-003": { + "answer": "6 min 40 s", + "tokens": 8119, + "fixed_tokens": 1101, + "processed_tokens": 59706, + "tool_calls": 5, + "turns": 6 + }, + "tools-004": { + "answer": "unknown", + "tokens": 9188, + "fixed_tokens": 1097, + "processed_tokens": 84823, + "tool_calls": 7, + "turns": 8 + }, + "tools-005": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 9651, + "fixed_tokens": 1096, + "processed_tokens": 99774, + "tool_calls": 8, + "turns": 9 + }, + "tools-006": { + "answer": "dec-build-a-content-addressed-shader-compile-cache-s--e65rq9439k7xz5zh", + "tokens": 9158, + "fixed_tokens": 1100, + "processed_tokens": 85591, + "tool_calls": 7, + "turns": 8 + }, + "tools-007": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 86069, + "fixed_tokens": 1099, + "processed_tokens": 2588695, + "tool_calls": 42, + "turns": 43 + }, + "tools-008": { + "answer": "dec-key-the-shader-cache-on-raw-source-text-and-comp--hp6awmx3gq5rn9nc", + "tokens": 9842, + "fixed_tokens": 1094, + "processed_tokens": 62912, + "tool_calls": 7, + "turns": 6 + }, + "tools-009": { + "answer": "92.4%", + "tokens": 7679, + "fixed_tokens": 1101, + "processed_tokens": 57918, + "tool_calls": 5, + "turns": 6 + }, + "tools-010": { + "answer": "dec-cap-each-farm-agent-at-6-parallel-shader-compile--q5w4sfjm73tsj8x8", + "tokens": 16636, + "fixed_tokens": 1097, + "processed_tokens": 175857, + "tool_calls": 12, + "turns": 13 + }, + "tools-011": { + "answer": "fnd-two-compiler-builds-with-the-same-version-string--gmgr4ha0ndxqmcn9", + "tokens": 15913, + "fixed_tokens": 1105, + "processed_tokens": 226475, + "tool_calls": 14, + "turns": 15 + }, + "tools-012": { + "answer": 3, + "tokens": 39568, + "fixed_tokens": 1104, + "processed_tokens": 146026, + "tool_calls": 10, + "turns": 9 + }, + "tools-013": { + "answer": 6, + "tokens": 158567, + "fixed_tokens": 1098, + "processed_tokens": 2724347, + "tool_calls": 38, + "turns": 39 + }, + "tools-014": { + "answer": false, + "tokens": 17586, + "fixed_tokens": 1100, + "processed_tokens": 159817, + "tool_calls": 11, + "turns": 12 + }, + "tools-015": { + "answer": [ + "iss-whether-the-audio-bank-packer-should-load-on-a-w--sqzp2rwp6s01w3b0", + "iss-whether-the-level-editor-autosave-needs-its-own--rkvmnbqe7w644zkr", + "iss-whether-the-artist-sandbox-should-load-on-a-work--73j819anqct8etfn", + "iss-whether-the-review-bot-should-load-on-a-worker-t--rtsjwvsksjdq15v1", + "iss-whether-the-level-editor-autosave-should-be-cach--radcjs4hyvh7p55q", + "iss-whether-the-crash-symbol-server-should-load-on-a--qvyphhgnsnm2cvqb", + "iss-whether-the-license-scanner-should-be-cached-bet--nqhd89czbfat9rya", + "iss-whether-the-doc-site-should-load-on-a-worker-thr--qxswe0pg52ekky0z", + "iss-whether-the-asset-search-should-load-on-a-worker--f9gpv3z522x6p6h4", + "iss-whether-the-cache-key-should-move-its-settings-t--jgtdn2f02p8fx5nr", + "iss-whether-the-review-bot-needs-its-own-memory-pool--pg3m8g9gn3y6v3kq", + "iss-whether-the-changelog-bot-should-skip-work-when--p2vrp5d682p3jrc3", + "iss-whether-the-cache-key-should-log-less-0068--cysv4n56q1qn7t2k", + "iss-whether-the-fetch-timeout-needs-its-own-unit-tes--11tqzws72gms2h8x", + "iss-whether-the-audio-bank-packer-needs-its-own-memo--qjrqrseast8d5mhf", + "iss-whether-the-fetch-timeout-should-print-fewer-war--mw8zejz6hk5m60x0", + "iss-whether-the-crash-symbol-server-should-ship-in-t--bhq91ajavah18wsk", + "iss-whether-the-asset-search-should-load-on-a-worker--80b27kqh36rpafdt", + "iss-whether-the-doc-site-should-ship-in-the-first-pa--wcdkq5zab7dxe5yj", + "iss-whether-the-patch-differ-should-be-cached-betwee--28b5zqb1ketwzf0r", + "iss-whether-the-test-runner-should-load-on-a-worker--aygay1r0qj4mzf78", + "iss-whether-the-texture-viewer-should-be-split-into--1jhgyqb70h5x7m7b", + "iss-whether-the-shader-compile-cache-needs-its-own-u--fyysrcv6fvbsvcsy", + "iss-whether-the-level-editor-autosave-should-be-spli--ncnwh0csw671fs0q", + "iss-whether-the-asset-search-should-ship-in-the-firs--hsb2avhx613prxew", + "iss-whether-the-test-runner-needs-its-own-memory-poo--bs6arq0qv37d6wmx", + "iss-whether-the-artist-sandbox-should-be-split-into--ek47ca2vgmrjanp8", + "rsk-shader-cache-storage-could-outgrow-the-4-tb-farm--76y9649mwrx7aavd" + ], + "tokens": 67291, + "fixed_tokens": 1106, + "processed_tokens": 653216, + "tool_calls": 21, + "turns": 19 + }, + "tools-016": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 15208, + "fixed_tokens": 1096, + "processed_tokens": 118011, + "tool_calls": 8, + "turns": 9 + }, + "world-001": { + "answer": "64 KB", + "tokens": 7825, + "fixed_tokens": 1098, + "processed_tokens": 58624, + "tool_calls": 5, + "turns": 6 + }, + "world-002": { + "answer": "5.1 ms", + "tokens": 7810, + "fixed_tokens": 1100, + "processed_tokens": 58610, + "tool_calls": 5, + "turns": 6 + }, + "world-003": { + "answer": "1 crash per 900 script waits", + "tokens": 8299, + "fixed_tokens": 1097, + "processed_tokens": 59753, + "tool_calls": 5, + "turns": 6 + }, + "world-004": { + "answer": "4,100 file opens", + "tokens": 11517, + "fixed_tokens": 1095, + "processed_tokens": 78848, + "tool_calls": 6, + "turns": 7 + }, + "world-005": { + "answer": "11 m", + "tokens": 7861, + "fixed_tokens": 1095, + "processed_tokens": 58952, + "tool_calls": 5, + "turns": 6 + }, + "world-006": { + "answer": "3.4 reversals per minute", + "tokens": 7326, + "fixed_tokens": 1102, + "processed_tokens": 56836, + "tool_calls": 5, + "turns": 6 + }, + "world-007": { + "answer": "260 MB", + "tokens": 9594, + "fixed_tokens": 1098, + "processed_tokens": 87126, + "tool_calls": 7, + "turns": 8 + }, + "world-008": { + "answer": "10 Hz", + "tokens": 7960, + "fixed_tokens": 1093, + "processed_tokens": 59389, + "tool_calls": 5, + "turns": 6 + }, + "world-009": { + "answer": "0.3 ms", + "tokens": 7500, + "fixed_tokens": 1092, + "processed_tokens": 57958, + "tool_calls": 5, + "turns": 6 + }, + "world-010": { + "answer": "29 GB/s", + "tokens": 6738, + "fixed_tokens": 1101, + "processed_tokens": 61971, + "tool_calls": 6, + "turns": 7 + }, + "world-011": { + "answer": "fnd-shipping-entity-names-costs-57-mb-at-full-popula--6w7gjvy7wdmdx9qx", + "tokens": 11920, + "fixed_tokens": 1093, + "processed_tokens": 80728, + "tool_calls": 8, + "turns": 7 + }, + "world-012": { + "answer": "dec-run-engine-jobs-and-gull-scripts-on-one-thread-p--07pbzvqg00pmnrr3", + "tokens": 7702, + "fixed_tokens": 1106, + "processed_tokens": 58419, + "tool_calls": 5, + "turns": 6 + }, + "world-013": { + "answer": "unknown", + "tokens": 10823, + "fixed_tokens": 1096, + "processed_tokens": 76805, + "tool_calls": 6, + "turns": 7 + }, + "world-014": { + "answer": "unknown", + "tokens": 25558, + "fixed_tokens": 1102, + "processed_tokens": 181042, + "tool_calls": 9, + "turns": 10 + }, + "world-015": { + "answer": "3.0 ms", + "tokens": 10651, + "fixed_tokens": 1093, + "processed_tokens": 76642, + "tool_calls": 6, + "turns": 7 + }, + "world-016": { + "answer": "unknown", + "tokens": 7222, + "fixed_tokens": 1091, + "processed_tokens": 61916, + "tool_calls": 6, + "turns": 7 + }, + "world-017": { + "answer": "dec-partition-the-world-into-128-m-cells--372b6czbjf29ymaw", + "tokens": 13754, + "fixed_tokens": 1098, + "processed_tokens": 120471, + "tool_calls": 8, + "turns": 9 + }, + "world-018": { + "answer": "dec-run-engine-jobs-on-a-work-stealing-thread-pool--1pd72x1pxpgwy21x", + "tokens": 10199, + "fixed_tokens": 1093, + "processed_tokens": 89988, + "tool_calls": 7, + "turns": 8 + }, + "world-019": { + "answer": "fnd-tide-priority-reads-starve-player-proximity-load--by0d7b0bk39xz3rk", + "tokens": 7748, + "fixed_tokens": 1102, + "processed_tokens": 58761, + "tool_calls": 5, + "turns": 6 + }, + "world-020": { + "answer": "dec-schedule-streaming-reads-by-deadline-instead-of--q2x5aknabgcssdj2", + "tokens": 9059, + "fixed_tokens": 1095, + "processed_tokens": 61559, + "tool_calls": 7, + "turns": 6 + }, + "world-021": { + "answer": "dec-cut-navigation-tiles-to-64-m-so-three-fit-along--snvepk8ygeb3qsjs", + "tokens": 7587, + "fixed_tokens": 1093, + "processed_tokens": 58360, + "tool_calls": 5, + "turns": 6 + }, + "world-022": { + "answer": "fnd-41-of-alpha-crash-reports-have-no-usable-stack--35a50b07ferhf13m", + "tokens": 7612, + "fixed_tokens": 1094, + "processed_tokens": 58631, + "tool_calls": 5, + "turns": 6 + }, + "world-023": { + "answer": "dec-compress-streamed-assets-with-kelp--ezwc5p1tgsgm7v8m", + "tokens": 10054, + "fixed_tokens": 1096, + "processed_tokens": 88238, + "tool_calls": 7, + "turns": 8 + }, + "world-024": { + "answer": "dec-shrink-kelp-s-dictionary-to-1-mb-for-streamed-as--yh8hktfqnyqhepew", + "tokens": 9514, + "fixed_tokens": 1091, + "processed_tokens": 86880, + "tool_calls": 7, + "turns": 8 + }, + "world-025": { + "answer": "con-streaming-pool-budget--kca88sgh2j6sxvd5", + "tokens": 9311, + "fixed_tokens": 1097, + "processed_tokens": 86975, + "tool_calls": 7, + "turns": 8 + }, + "world-026": { + "answer": "fnd-retail-skiff-units-sustain-only-190-mb-s-of-stre--485sdcmm3gweevvn", + "tokens": 7921, + "fixed_tokens": 1098, + "processed_tokens": 59304, + "tool_calls": 5, + "turns": 6 + }, + "world-027": { + "answer": "160 MB/s", + "tokens": 7941, + "fixed_tokens": 1093, + "processed_tokens": 59504, + "tool_calls": 5, + "turns": 6 + }, + "world-028": { + "answer": "fnd-streaming-tide-heights-costs-46-kb-s-per-peer--knxqtyqh6xcz1tp9", + "tokens": 7462, + "fixed_tokens": 1100, + "processed_tokens": 57691, + "tool_calls": 5, + "turns": 6 + }, + "world-029": { + "answer": "dec-return-engine-jobs-to-a-thread-pool-with-continu--1zkc1v2x97taf50h", + "tokens": 12185, + "fixed_tokens": 1100, + "processed_tokens": 121559, + "tool_calls": 9, + "turns": 10 + }, + "world-030": { + "answer": "fnd-world-core-state-at-alpha--tkbsm4867h65jqq8", + "tokens": 9923, + "fixed_tokens": 1099, + "processed_tokens": 87968, + "tool_calls": 7, + "turns": 8 + }, + "world-031": { + "answer": "1.5 GB/s", + "tokens": 8515, + "fixed_tokens": 1094, + "processed_tokens": 61406, + "tool_calls": 5, + "turns": 6 + }, + "world-032": { + "answer": "fnd-float-tide-heights-drift-apart-between-pc-and-me--7jb0fg4b5xzr7q5e", + "tokens": 12406, + "fixed_tokens": 1096, + "processed_tokens": 87770, + "tool_calls": 8, + "turns": 7 + }, + "world-033": { + "answer": "dec-save-player-changes-as-deltas-against-the-tide-f--n38scy1yaarsr8cw", + "tokens": 18009, + "fixed_tokens": 1097, + "processed_tokens": 185594, + "tool_calls": 11, + "turns": 12 + }, + "world-034": { + "answer": "unknown", + "tokens": 16153, + "fixed_tokens": 1095, + "processed_tokens": 112523, + "tool_calls": 8, + "turns": 9 + }, + "world-035": { + "answer": "false", + "tokens": 7688, + "fixed_tokens": 1099, + "processed_tokens": 57609, + "tool_calls": 5, + "turns": 6 + }, + "world-036": { + "answer": "Fibers were retired because 41% of alpha crash reports had no usable stack, which blocked crash triage for beta, and a continuation-wait prototype kept workers at 93% busy versus the 95% fibers gave, so the utilisation gain did not justify the unreadable crashes and Gull thread-local bug. Record: dec-return-engine-jobs-to-a-thread-pool-with-continu--1zkc1v2x97taf50h", + "tokens": 8479, + "fixed_tokens": 1095, + "processed_tokens": 59788, + "tool_calls": 5, + "turns": 6 + }, + "world-037": { + "answer": "fnd-hot-reload-drops-suspended-gull-coroutines--va0fn0svryzwbzws", + "tokens": 11282, + "fixed_tokens": 1097, + "processed_tokens": 90888, + "tool_calls": 7, + "turns": 8 + }, + "world-038": { + "answer": "3 archetypes", + "tokens": 12029, + "fixed_tokens": 1101, + "processed_tokens": 97878, + "tool_calls": 9, + "turns": 8 + }, + "world-039": { + "answer": "1.2 GB/s", + "tokens": 7858, + "fixed_tokens": 1094, + "processed_tokens": 59317, + "tool_calls": 5, + "turns": 6 + }, + "world-040": { + "answer": "unknown", + "tokens": 8169, + "fixed_tokens": 1096, + "processed_tokens": 60597, + "tool_calls": 5, + "turns": 6 + }, + "world-041": { + "answer": "61%", + "tokens": 8100, + "fixed_tokens": 1092, + "processed_tokens": 59977, + "tool_calls": 5, + "turns": 6 + }, + "world-042": { + "answer": "5.4 s", + "tokens": 9029, + "fixed_tokens": 1093, + "processed_tokens": 85736, + "tool_calls": 7, + "turns": 8 + }, + "world-043": { + "answer": "5.8 minutes", + "tokens": 7728, + "fixed_tokens": 1093, + "processed_tokens": 58696, + "tool_calls": 5, + "turns": 6 + }, + "world-044": { + "answer": "dec-move-engine-jobs-to-fibers--qjkgnm8cex8ydzfq", + "tokens": 17142, + "fixed_tokens": 1095, + "processed_tokens": 182334, + "tool_calls": 14, + "turns": 12 + }, + "world-045": { + "answer": "dec-run-engine-jobs-on-fibers-and-pin-gull-scripts-t--rw28cbqfd07rd9te", + "tokens": 28299, + "fixed_tokens": 1096, + "processed_tokens": 276938, + "tool_calls": 15, + "turns": 15 + }, + "world-046": { + "answer": "con-streaming-pool-budget--632bkbs79bwj4cr5", + "tokens": 15710, + "fixed_tokens": 1092, + "processed_tokens": 160798, + "tool_calls": 10, + "turns": 11 + }, + "world-047": { + "answer": "fnd-world-core-state-at-alpha--tkbsm4867h65jqq8", + "tokens": 15635, + "fixed_tokens": 1095, + "processed_tokens": 132511, + "tool_calls": 11, + "turns": 10 + }, + "world-048": { + "answer": "dec-compile-gull-to-bytecode-for-a-register-vm--jrqje646fpevp5f3", + "tokens": 9373, + "fixed_tokens": 1095, + "processed_tokens": 85668, + "tool_calls": 7, + "turns": 8 + }, + "world-049": { + "answer": "22%", + "tokens": 8089, + "fixed_tokens": 1097, + "processed_tokens": 59929, + "tool_calls": 5, + "turns": 6 + }, + "world-050": { + "answer": "fnd-alpha-content-needs-6-2-ms-of-gull-time-per-fram--4mekgdpsdv5nk085", + "tokens": 13549, + "fixed_tokens": 1100, + "processed_tokens": 108723, + "tool_calls": 8, + "turns": 9 + }, + "world-051": { + "answer": "The team chose 32.32 fixed point because integer adds and multiplies give the same bits on every CPU and compiler, so the drift float tides showed between machines within 6 minutes cannot happen; the strict IEEE option was rejected in favor of it (dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e).", + "tokens": 9564, + "fixed_tokens": 1101, + "processed_tokens": 74504, + "tool_calls": 7, + "turns": 7 + }, + "world-052": { + "answer": "Dara Whitlock", + "tokens": 14698, + "fixed_tokens": 1094, + "processed_tokens": 120534, + "tool_calls": 10, + "turns": 9 + }, + "world-053": { + "answer": "1.9 ms", + "tokens": 12901, + "fixed_tokens": 1096, + "processed_tokens": 106565, + "tool_calls": 8, + "turns": 9 + }, + "world-054": { + "answer": "Marta Lindqvist", + "tokens": 12166, + "fixed_tokens": 1096, + "processed_tokens": 151663, + "tool_calls": 11, + "turns": 12 + }, + "world-055": { + "answer": "The cold-cache rerun's loss applied to Kelp's 4 MB dictionary, so the team shrank the dictionary to 1 MB (dec-shrink-kelp-s-dictionary-to-1-mb-for-streamed-as--yh8hktfqnyqhepew), where Kelp still beats Zstandard cold on Skiff: 1.5 GB/s vs 1.2 GB/s after firmware 3.2 (fnd-kelp-with-a-1-mb-dictionary-decodes-at-1-5-gb-s--y9v8jrxgbv6abrx5).", + "tokens": 12553, + "fixed_tokens": 1098, + "processed_tokens": 69318, + "tool_calls": 9, + "turns": 6 + }, + "world-056": { + "answer": "dec-run-engine-jobs-on-a-work-stealing-thread-pool--1pd72x1pxpgwy21x", + "tokens": 11397, + "fixed_tokens": 1092, + "processed_tokens": 118134, + "tool_calls": 9, + "turns": 10 + }, + "world-057": { + "answer": "tile [212, 87]", + "tokens": 12462, + "fixed_tokens": 1099, + "processed_tokens": 151425, + "tool_calls": 11, + "turns": 12 + }, + "world-058": { + "answer": "0.8 GB/s", + "tokens": 13983, + "fixed_tokens": 1096, + "processed_tokens": 162561, + "tool_calls": 11, + "turns": 12 + }, + "world-059": { + "answer": "dec-derive-the-missing-harmonic-phase-from-each-save--gbqrvvgyq2gy2gnw", + "tokens": 12562, + "fixed_tokens": 1099, + "processed_tokens": 127274, + "tool_calls": 11, + "turns": 10 + }, + "world-060": { + "answer": "dec-partition-the-world-into-192-m-cells--bk9r1c5jwb6fh8tc", + "tokens": 9902, + "fixed_tokens": 1100, + "processed_tokens": 101613, + "tool_calls": 8, + "turns": 9 + }, + "world-061": { + "answer": "236 MB", + "tokens": 13768, + "fixed_tokens": 1091, + "processed_tokens": 187659, + "tool_calls": 13, + "turns": 14 + }, + "world-062": { + "answer": "310 MB", + "tokens": 14507, + "fixed_tokens": 1099, + "processed_tokens": 192542, + "tool_calls": 13, + "turns": 14 + }, + "world-063": { + "answer": "con-streaming-pool-budget--632bkbs79bwj4cr5", + "tokens": 12975, + "fixed_tokens": 1098, + "processed_tokens": 183514, + "tool_calls": 13, + "turns": 14 + }, + "world-064": { + "answer": [ + "dec-force-strict-ieee-float-mode-for-the-tide-simula--ecwmqt7ncjngysqw", + "dec-have-the-host-stream-tide-heights-to-every-peer--a3rb1seedn2bj7k8" + ], + "tokens": 21605, + "fixed_tokens": 1092, + "processed_tokens": 185017, + "tool_calls": 14, + "turns": 12 + }, + "world-065": { + "answer": "3.1 ms", + "tokens": 12866, + "fixed_tokens": 1100, + "processed_tokens": 182209, + "tool_calls": 13, + "turns": 14 + }, + "world-066": { + "answer": "46", + "tokens": 12836, + "fixed_tokens": 1100, + "processed_tokens": 143129, + "tool_calls": 10, + "turns": 11 + }, + "world-067": { + "answer": "1.2 GB/s", + "tokens": 13371, + "fixed_tokens": 1099, + "processed_tokens": 160894, + "tool_calls": 11, + "turns": 12 + }, + "world-068": { + "answer": true, + "tokens": 23315, + "fixed_tokens": 1094, + "processed_tokens": 191139, + "tool_calls": 11, + "turns": 12 + }, + "world-069": { + "answer": 6, + "tokens": 29200, + "fixed_tokens": 1095, + "processed_tokens": 371814, + "tool_calls": 19, + "turns": 20 + }, + "world-070": { + "answer": 3, + "tokens": 19679, + "fixed_tokens": 1098, + "processed_tokens": 141130, + "tool_calls": 8, + "turns": 9 + }, + "world-071": { + "answer": [ + "dec-hot-reload-gull-functions-by-swapping-bytecode-a--8dxnkg6h8vmcsr42" + ], + "tokens": 54192, + "fixed_tokens": 1102, + "processed_tokens": 594598, + "tool_calls": 20, + "turns": 21 + }, + "world-072": { + "answer": [ + "dec-compress-streamed-assets-with-lz4--v1pfsgfp0b3yjv4s", + "dec-compress-streamed-assets-with-zstandard--bs1t15cygc51cpje" + ], + "tokens": 30921, + "fixed_tokens": 1094, + "processed_tokens": 299114, + "tool_calls": 14, + "turns": 15 + }, + "world-073": { + "answer": false, + "tokens": 9618, + "fixed_tokens": 1100, + "processed_tokens": 85411, + "tool_calls": 7, + "turns": 8 + }, + "world-074": { + "answer": 7, + "tokens": 9461, + "fixed_tokens": 1099, + "processed_tokens": 70767, + "tool_calls": 6, + "turns": 7 + }, + "world-075": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--8679sjn071ycsbpv", + "dec-partition-the-world-into-192-meter-cells--q62ddwdv2n8wxay6", + "dec-return-engine-jobs-to-the-thread-pool-with-conti--547qbweqvrp1j5g3" + ], + "tokens": 6553, + "fixed_tokens": 1097, + "processed_tokens": 51144, + "tool_calls": 5, + "turns": 6 + }, + "world-076": { + "answer": "unknown", + "tokens": 34476, + "fixed_tokens": 1097, + "processed_tokens": 336715, + "tool_calls": 24, + "turns": 16 + }, + "world-077": { + "answer": 251, + "tokens": 28482, + "fixed_tokens": 1093, + "processed_tokens": 408861, + "tool_calls": 17, + "turns": 18 + }, + "world-078": { + "answer": [ + "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "rsk-reworking-the-read-scheduler-may-slip-the-coast--9jzvs7etvb5jxqtp" + ], + "tokens": 11593, + "fixed_tokens": 1101, + "processed_tokens": 84590, + "tool_calls": 8, + "turns": 8 + }, + "world-079": { + "answer": [ + "con-streaming-pool-budget--8czwb4fjr5p5vpy7", + "con-no-runtime-code-generation-on-consoles--6cf25zqmsvqkdd73", + "con-tide-state-must-be-bit-identical-on-every-peer--zrpnp8qjyxzkas7t", + "con-saves-from-beta-onward-must-load-in-every-later--v6z856sxtgc5sb09" + ], + "tokens": 14700, + "fixed_tokens": 1095, + "processed_tokens": 81878, + "tool_calls": 6, + "turns": 7 + }, + "world-080": { + "answer": [ + "iss-gull-hot-reload-rejects-struct-layout-changes--3v2w2f9hg2gwy6aj", + "iss-gull-hot-reload-drops-suspended-coroutines--z7j2gmyny6gdxd07", + "iss-cross-platform-cloud-saves-are-unplanned--5bk0k4wrgj42ztfy" + ], + "tokens": 10092, + "fixed_tokens": 1100, + "processed_tokens": 90837, + "tool_calls": 8, + "turns": 9 + }, + "world-081": { + "answer": "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "tokens": 31142, + "fixed_tokens": 1094, + "processed_tokens": 430414, + "tool_calls": 34, + "turns": 24 + } + } +} diff --git a/benchmarks/proof-recall/runs/x10adv-find-r1-nobrief.json b/benchmarks/proof-recall/runs/x10adv-find-r1-nobrief.json new file mode 100644 index 00000000..041d52d5 --- /dev/null +++ b/benchmarks/proof-recall/runs/x10adv-find-r1-nobrief.json @@ -0,0 +1,2256 @@ +{ + "run": { + "name": "x10adv-find-r1-nobrief", + "protocol": "one fresh session per question", + "models": ["claude-haiku-5-5"], + "find_hash": "e5be653299abe9b2329822d98ffe09e90ee7cfe96f8711c813a51dc6500b887f" + }, + "answers": { + "ambiguous-001": { + "answer": "384 MB", + "tokens": 44988, + "fixed_tokens": 2741, + "processed_tokens": 447512, + "tool_calls": 21, + "turns": 17 + }, + "ambiguous-002": { + "answer": "unknown", + "tokens": 58307, + "fixed_tokens": 2743, + "processed_tokens": 332450, + "tool_calls": 12, + "turns": 11 + }, + "ambiguous-003": { + "answer": "92.4%", + "tokens": 63505, + "fixed_tokens": 2742, + "processed_tokens": 1556138, + "tool_calls": 35, + "turns": 33 + }, + "ambiguous-004": { + "answer": "6 min 40 s", + "tokens": 41722, + "fixed_tokens": 2745, + "processed_tokens": 844973, + "tool_calls": 28, + "turns": 29 + }, + "ambiguous-005": { + "answer": "unknown", + "tokens": 73136, + "fixed_tokens": 1115, + "processed_tokens": 2756856, + "tool_calls": 58, + "turns": 59 + }, + "ambiguous-006": { + "answer": "unknown", + "tokens": 64594, + "fixed_tokens": 1116, + "processed_tokens": 1591573, + "tool_calls": 38, + "turns": 39 + }, + "ambiguous-007": { + "answer": "32 steps", + "tokens": 59784, + "fixed_tokens": 1117, + "processed_tokens": 534891, + "tool_calls": 20, + "turns": 17 + }, + "ambiguous-008": { + "answer": "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "tokens": 19164, + "fixed_tokens": 1116, + "processed_tokens": 120011, + "tool_calls": 7, + "turns": 8 + }, + "ambiguous-009": { + "answer": "3.0 ms per frame", + "tokens": 31181, + "fixed_tokens": 1121, + "processed_tokens": 430973, + "tool_calls": 18, + "turns": 19 + }, + "ambiguous-010": { + "answer": "160 MB/s", + "tokens": 30559, + "fixed_tokens": 1116, + "processed_tokens": 279176, + "tool_calls": 12, + "turns": 13 + }, + "ambiguous-011": { + "answer": "260 MB", + "tokens": 55347, + "fixed_tokens": 1119, + "processed_tokens": 967418, + "tool_calls": 24, + "turns": 25 + }, + "ambiguous-012": { + "answer": "11 m", + "tokens": 29092, + "fixed_tokens": 1112, + "processed_tokens": 277365, + "tool_calls": 12, + "turns": 13 + }, + "ambiguous-013": { + "answer": "61%", + "tokens": 38184, + "fixed_tokens": 1114, + "processed_tokens": 488818, + "tool_calls": 16, + "turns": 17 + }, + "ambiguous-014": { + "answer": "textures", + "tokens": 50873, + "fixed_tokens": 1118, + "processed_tokens": 1287910, + "tool_calls": 36, + "turns": 37 + }, + "ambiguous-015": { + "answer": "unknown", + "tokens": 40564, + "fixed_tokens": 1114, + "processed_tokens": 612778, + "tool_calls": 20, + "turns": 21 + }, + "ambiguous-016": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 55486, + "fixed_tokens": 1123, + "processed_tokens": 929840, + "tool_calls": 27, + "turns": 26 + }, + "ambiguous-017": { + "answer": "unknown", + "tokens": 64641, + "fixed_tokens": 1119, + "processed_tokens": 1602785, + "tool_calls": 34, + "turns": 35 + }, + "ambiguous-018": { + "answer": "unknown", + "tokens": 56830, + "fixed_tokens": 1123, + "processed_tokens": 1455063, + "tool_calls": 44, + "turns": 45 + }, + "ambiguous-019": { + "answer": "7.4 ms", + "tokens": 22465, + "fixed_tokens": 1114, + "processed_tokens": 242568, + "tool_calls": 12, + "turns": 13 + }, + "ambiguous-020": { + "answer": "unknown", + "tokens": 24629, + "fixed_tokens": 1118, + "processed_tokens": 248932, + "tool_calls": 12, + "turns": 13 + }, + "ambiguous-021": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 35253, + "fixed_tokens": 1120, + "processed_tokens": 497361, + "tool_calls": 20, + "turns": 20 + }, + "ambiguous-022": { + "answer": "unknown", + "tokens": 55214, + "fixed_tokens": 1115, + "processed_tokens": 1201207, + "tool_calls": 32, + "turns": 33 + }, + "ambiguous-023": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 9734, + "fixed_tokens": 1117, + "processed_tokens": 82476, + "tool_calls": 7, + "turns": 8 + }, + "ambiguous-024": { + "answer": "unknown", + "tokens": 49071, + "fixed_tokens": 1119, + "processed_tokens": 1084627, + "tool_calls": 31, + "turns": 32 + }, + "cross-001": { + "answer": "con-frame-budget--br0qszfg8fys908n", + "tokens": 36599, + "fixed_tokens": 1105, + "processed_tokens": 367891, + "tool_calls": 13, + "turns": 14 + }, + "cross-002": { + "answer": 2, + "tokens": 24706, + "fixed_tokens": 1108, + "processed_tokens": 99276, + "tool_calls": 18, + "turns": 7 + }, + "cross-003": { + "answer": 2, + "tokens": 21937, + "fixed_tokens": 1106, + "processed_tokens": 117663, + "tool_calls": 18, + "turns": 8 + }, + "cross-004": { + "answer": "dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e", + "tokens": 28103, + "fixed_tokens": 1103, + "processed_tokens": 424409, + "tool_calls": 18, + "turns": 19 + }, + "cross-005": { + "answer": "Rhea Valdés", + "tokens": 29919, + "fixed_tokens": 1106, + "processed_tokens": 468939, + "tool_calls": 20, + "turns": 21 + }, + "cross-006": { + "answer": "19%", + "tokens": 22065, + "fixed_tokens": 1109, + "processed_tokens": 217572, + "tool_calls": 11, + "turns": 12 + }, + "cross-007": { + "answer": "8 GB", + "tokens": 63739, + "fixed_tokens": 1107, + "processed_tokens": 1811090, + "tool_calls": 42, + "turns": 43 + }, + "cross-008": { + "answer": "unknown", + "tokens": 46756, + "fixed_tokens": 1108, + "processed_tokens": 451504, + "tool_calls": 14, + "turns": 15 + }, + "cross-009": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 29601, + "fixed_tokens": 1106, + "processed_tokens": 284527, + "tool_calls": 12, + "turns": 13 + }, + "cross-010": { + "answer": "dec-build-an-on-device-pipeline-cache-for-skiff--726r9mtk76qndeh7", + "tokens": 27053, + "fixed_tokens": 1114, + "processed_tokens": 283840, + "tool_calls": 13, + "turns": 14 + }, + "cross-011": { + "answer": "false", + "tokens": 44027, + "fixed_tokens": 1104, + "processed_tokens": 739447, + "tool_calls": 22, + "turns": 23 + }, + "cross-012": { + "answer": "61 MB/s", + "tokens": 37352, + "fixed_tokens": 1107, + "processed_tokens": 643362, + "tool_calls": 22, + "turns": 23 + }, + "cross-013": { + "answer": "Skiff runs Tern at the Performance preset, which saves 1.8 ms against Quality (2.1 ms vs 3.9 ms); see dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3. The graph does not confirm the Meridian 120 Hz premise that Performance is Tern's lowest preset.", + "tokens": 47040, + "fixed_tokens": 1106, + "processed_tokens": 914223, + "tool_calls": 28, + "turns": 29 + }, + "cross-014": { + "answer": 3, + "tokens": 5437, + "fixed_tokens": 1111, + "processed_tokens": 40287, + "tool_calls": 4, + "turns": 5 + }, + "cross-015": { + "answer": "Two Efforts are no longer active. Shader compile cache and build farm is completed: dec-turn-the-shader-compile-cache-on-for-every-farm--4rr3b12b4sasdmaj (accepted 2026-03-20) turned the cache on for all nightly, on-demand and release builds, which is the Effort's stated end. Co-op rollback netcode prototype is abandoned: dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b (accepted 2026-04-01) stopped the rollback prototype because re-running tide simulation on rollback frames costs 4.8 ms on Skiff, and it supersedes dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4.", + "tokens": 51214, + "fixed_tokens": 1109, + "processed_tokens": 521042, + "tool_calls": 17, + "turns": 15 + }, + "cross-016": { + "answer": "16.7 ms", + "tokens": 26952, + "fixed_tokens": 1117, + "processed_tokens": 181880, + "tool_calls": 9, + "turns": 10 + }, + "cross-017": { + "answer": "false", + "tokens": 50484, + "fixed_tokens": 1108, + "processed_tokens": 820741, + "tool_calls": 22, + "turns": 23 + }, + "netcode-001": { + "answer": "abandoned", + "tokens": 16572, + "fixed_tokens": 1113, + "processed_tokens": 82824, + "tool_calls": 6, + "turns": 7 + }, + "netcode-002": { + "answer": "netcode/rollback-archive", + "tokens": 17090, + "fixed_tokens": 1110, + "processed_tokens": 119513, + "tool_calls": 8, + "turns": 9 + }, + "netcode-003": { + "answer": "38 KB", + "tokens": 69446, + "fixed_tokens": 1114, + "processed_tokens": 996042, + "tool_calls": 24, + "turns": 25 + }, + "netcode-004": { + "answer": "The team abandoned rollback netcode because rollback must re-run the tide simulation on every rolled-back frame, costing 4.8 ms on Skiff (about a fifth of the 25 ms frame), and the shrinking-window and quarter-resolution workarounds both failed. The reason depends on the world Decision `dec-run-the-tide-simulation-in-32-32-fixed-point--c5rjfcnmhne4h05e`, which keeps tides bit-identical across peers.", + "tokens": 42804, + "fixed_tokens": 1106, + "processed_tokens": 576132, + "tool_calls": 18, + "turns": 19 + }, + "netcode-005": { + "answer": "unknown", + "tokens": 24756, + "fixed_tokens": 1114, + "processed_tokens": 300732, + "tool_calls": 14, + "turns": 15 + }, + "netcode-006": { + "answer": "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "tokens": 10485, + "fixed_tokens": 1109, + "processed_tokens": 94313, + "tool_calls": 8, + "turns": 9 + }, + "netcode-007": { + "answer": "dec-abandon-rollback-netcode-for-co-op--46envrg0v4pdwv7b", + "tokens": 18698, + "fixed_tokens": 1118, + "processed_tokens": 127714, + "tool_calls": 8, + "turns": 9 + }, + "netcode-008": { + "answer": "dec-use-a-4-frame-rollback-window--pqe4hvbgmbp0dmh9", + "tokens": 35634, + "fixed_tokens": 1110, + "processed_tokens": 470922, + "tool_calls": 18, + "turns": 19 + }, + "netcode-009": { + "answer": "1.9 ms", + "tokens": 28555, + "fixed_tokens": 1108, + "processed_tokens": 324608, + "tool_calls": 16, + "turns": 15 + }, + "netcode-010": { + "answer": "dec-use-a-6-frame-rollback-window-with-1-frame-of-in--s53944zhkw103frk", + "tokens": 27748, + "fixed_tokens": 1109, + "processed_tokens": 358062, + "tool_calls": 15, + "turns": 16 + }, + "netcode-011": { + "answer": "con-co-op-peers-must-agree-on-tide-state-bit-for-bit--4v0sdbc3t4ywzr3g", + "tokens": 45379, + "fixed_tokens": 1120, + "processed_tokens": 712348, + "tool_calls": 26, + "turns": 24 + }, + "netcode-012": { + "answer": "unknown", + "tokens": 27841, + "fixed_tokens": 1112, + "processed_tokens": 372454, + "tool_calls": 19, + "turns": 19 + }, + "netcode-013": { + "answer": [ + "dec-prototype-rollback-netcode-for-co-op--2tx2s2p3sfbwj8x4", + "fnd-rollback-re-simulates-a-4-frame-window-in-1-1-ms--xa30a1cez2brphyf", + "rsk-re-simulation-cost-could-blow-the-skiff-frame-bu--sk2njsp170a3qwyf" + ], + "tokens": 49713, + "fixed_tokens": 1108, + "processed_tokens": 475294, + "tool_calls": 29, + "turns": 15 + }, + "netcode-014": { + "answer": [ + "iss-how-should-co-op-keep-up-to-four-players-in-sync--92t2zfsy3etfzr89", + "iss-late-join-during-a-tide-surge-desyncs-the-joinin--mya93va0n7s8g7cj" + ], + "tokens": 56940, + "fixed_tokens": 1112, + "processed_tokens": 839551, + "tool_calls": 21, + "turns": 22 + }, + "netcode-015": { + "answer": 2, + "tokens": 26711, + "fixed_tokens": 1110, + "processed_tokens": 171534, + "tool_calls": 10, + "turns": 11 + }, + "ports-001": { + "answer": "25 ms", + "tokens": 23219, + "fixed_tokens": 1112, + "processed_tokens": 227375, + "tool_calls": 11, + "turns": 12 + }, + "ports-002": { + "answer": "384 MB", + "tokens": 57840, + "fixed_tokens": 1107, + "processed_tokens": 568589, + "tool_calls": 16, + "turns": 17 + }, + "ports-003": { + "answer": "61 MB/s", + "tokens": 54438, + "fixed_tokens": 1104, + "processed_tokens": 1577548, + "tool_calls": 43, + "turns": 44 + }, + "ports-004": { + "answer": "accepted", + "tokens": 21125, + "fixed_tokens": 1112, + "processed_tokens": 177704, + "tool_calls": 10, + "turns": 11 + }, + "ports-005": { + "answer": "unknown", + "tokens": 97774, + "fixed_tokens": 1114, + "processed_tokens": 4502232, + "tool_calls": 73, + "turns": 74 + }, + "ports-034": { + "answer": "unknown", + "tokens": 58491, + "fixed_tokens": 1116, + "processed_tokens": 1500257, + "tool_calls": 35, + "turns": 36 + }, + "ports-006": { + "answer": "con-skiff-memory-budget--nf7qbhsnmcn6v479", + "tokens": 21182, + "fixed_tokens": 1103, + "processed_tokens": 184456, + "tool_calls": 10, + "turns": 11 + }, + "ports-007": { + "answer": "SKF-UI-031", + "tokens": 46347, + "fixed_tokens": 1108, + "processed_tokens": 1667821, + "tool_calls": 55, + "turns": 56 + }, + "ports-008": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 9415, + "fixed_tokens": 1111, + "processed_tokens": 101617, + "tool_calls": 9, + "turns": 10 + }, + "ports-009": { + "answer": null, + "tokens": 64712, + "fixed_tokens": 1113, + "processed_tokens": 968499, + "tool_calls": 61, + "turns": 31 + }, + "ports-010": { + "answer": "con-frame-budget--wt5rfqh5z444g2cg", + "tokens": 84273, + "fixed_tokens": 1108, + "processed_tokens": 3338037, + "tool_calls": 61, + "turns": 59 + }, + "ports-011": { + "answer": "unknown", + "tokens": 34275, + "fixed_tokens": 1113, + "processed_tokens": 481488, + "tool_calls": 19, + "turns": 19 + }, + "ports-012": { + "answer": "The ports team raised the Skiff dynamic resolution floor to 576p because caustic patterns under saltglass shimmered badly at the 540p floor, even after Tern (visible in 7 of 10 dune views versus 2 of 10 at 576p), per dec-raise-the-skiff-dynamic-resolution-floor-to-576p--fep9q7m08pp9f0bb, which was later superseded by dec-lower-the-skiff-dynamic-resolution-floor-to-504p--15esnt90gcn29enn.", + "tokens": 34385, + "fixed_tokens": 1114, + "processed_tokens": 359259, + "tool_calls": 13, + "turns": 14 + }, + "ports-013": { + "answer": "Ship Tern as Lantern's only upscaler", + "tokens": 39022, + "fixed_tokens": 1106, + "processed_tokens": 393085, + "tool_calls": 14, + "turns": 14 + }, + "ports-014": { + "answer": "unknown", + "tokens": 38556, + "fixed_tokens": 1107, + "processed_tokens": 866297, + "tool_calls": 29, + "turns": 30 + }, + "ports-015": { + "answer": "textures", + "tokens": 25265, + "fixed_tokens": 1110, + "processed_tokens": 348020, + "tool_calls": 16, + "turns": 17 + }, + "ports-016": { + "answer": "dec-ship-prebuilt-pipeline-blobs-in-the-skiff-packag--3vjysjgy9qkvzebn", + "tokens": 42390, + "fixed_tokens": 1107, + "processed_tokens": 692450, + "tool_calls": 22, + "turns": 23 + }, + "ports-017": { + "answer": "fnd-skiff-microsd-slows-sharply-on-full-warm-cards--p1rgh36hat2egpz3", + "tokens": 103837, + "fixed_tokens": 1104, + "processed_tokens": 4318484, + "tool_calls": 59, + "turns": 60 + }, + "ports-018": { + "answer": "504p", + "tokens": 44419, + "fixed_tokens": 1106, + "processed_tokens": 470513, + "tool_calls": 20, + "turns": 16 + }, + "ports-019": { + "answer": "4 MB per minute", + "tokens": 46172, + "fixed_tokens": 1106, + "processed_tokens": 1262839, + "tool_calls": 36, + "turns": 37 + }, + "ports-020": { + "answer": "fnd-skiff-sdk-1-2-takes-1-gb-back-from-games--a0ybdgjzw64hwfqa", + "tokens": 18714, + "fixed_tokens": 1110, + "processed_tokens": 159320, + "tool_calls": 9, + "turns": 10 + }, + "ports-021": { + "answer": "SKF-DSP-004", + "tokens": 58742, + "fixed_tokens": 1107, + "processed_tokens": 1479914, + "tool_calls": 34, + "turns": 35 + }, + "ports-022": { + "answer": "cit-skiff-sdk-1-2-release-notes--7gwwn8xxffrx6npe", + "tokens": 26665, + "fixed_tokens": 1108, + "processed_tokens": 337746, + "tool_calls": 15, + "turns": 16 + }, + "ports-023": { + "answer": "fnd-tern-costs-3-9-ms-at-quality-and-2-1-ms-at-perfo--dkbskbqfgfwtryq0", + "tokens": 15242, + "fixed_tokens": 1109, + "processed_tokens": 223792, + "tool_calls": 16, + "turns": 17 + }, + "ports-024": { + "answer": "fnd-skiff-firmware-1-2-rejects-pipeline-blobs-built--pernvgb2he7m3z3y", + "tokens": 59892, + "fixed_tokens": 1108, + "processed_tokens": 911816, + "tool_calls": 22, + "turns": 23 + }, + "ports-025": { + "answer": "dec-run-tern-on-skiff-at-the-performance-preset--74eqayfa83ybqpe3", + "tokens": 31603, + "fixed_tokens": 1107, + "processed_tokens": 521739, + "tool_calls": 22, + "turns": 22 + }, + "ports-026": { + "answer": "unknown", + "tokens": 62056, + "fixed_tokens": 1102, + "processed_tokens": 2400562, + "tool_calls": 56, + "turns": 57 + }, + "ports-027": { + "answer": [ + "dec-stream-skiff-world-tiles-in-2-mb-reads--3jz9w9gvhhc7eppw", + "rsk-tide-streaming-outruns-skiff-storage--6q8s3b383z2tyeyb" + ], + "tokens": 135262, + "fixed_tokens": 1105, + "processed_tokens": 2120784, + "tool_calls": 42, + "turns": 33 + }, + "ports-028": { + "answer": 2, + "tokens": 7610, + "fixed_tokens": 1114, + "processed_tokens": 61101, + "tool_calls": 6, + "turns": 7 + }, + "ports-029": { + "answer": 8, + "tokens": 96207, + "fixed_tokens": 1110, + "processed_tokens": 1713334, + "tool_calls": 35, + "turns": 34 + }, + "ports-030": { + "answer": [ + "dec-cap-tide-storm-particles-at-6-000-on-skiff--resze52afjxyxys1", + "dec-drop-skiff-to-30-hz-while-a-tide-storm-is-on-scr--72vx1c8s4jrs6axc" + ], + "tokens": 21358, + "fixed_tokens": 1109, + "processed_tokens": 132597, + "tool_calls": 12, + "turns": 9 + }, + "ports-031": { + "answer": [ + "iss-meridian-cert-activity-cards-missing-for-two-bio--k214cmfra7pyvvzq", + "iss-skiff-cert-docked-30-hz-judder-on-some-tvs--9nq2xg9mzash474g" + ], + "tokens": 35824, + "fixed_tokens": 1115, + "processed_tokens": 671118, + "tool_calls": 24, + "turns": 25 + }, + "ports-032": { + "answer": 2, + "tokens": 6047, + "fixed_tokens": 1108, + "processed_tokens": 49968, + "tool_calls": 5, + "turns": 6 + }, + "ports-033": { + "answer": "unknown", + "tokens": 20200, + "fixed_tokens": 1106, + "processed_tokens": 118823, + "tool_calls": 7, + "turns": 8 + }, + "render-001": { + "answer": "7.4 ms", + "tokens": 24161, + "fixed_tokens": 1106, + "processed_tokens": 171209, + "tool_calls": 9, + "turns": 10 + }, + "render-002": { + "answer": "3.4x", + "tokens": 23529, + "fixed_tokens": 1108, + "processed_tokens": 320576, + "tool_calls": 15, + "turns": 16 + }, + "render-003": { + "answer": "Tomasz Brandt", + "tokens": 29698, + "fixed_tokens": 1103, + "processed_tokens": 229776, + "tool_calls": 11, + "turns": 12 + }, + "render-004": { + "answer": "rejected", + "tokens": 23995, + "fixed_tokens": 1108, + "processed_tokens": 405827, + "tool_calls": 24, + "turns": 25 + }, + "render-005": { + "answer": true, + "tokens": 26430, + "fixed_tokens": 1104, + "processed_tokens": 205231, + "tool_calls": 10, + "turns": 11 + }, + "render-006": { + "answer": "unknown", + "tokens": 35959, + "fixed_tokens": 1105, + "processed_tokens": 712938, + "tool_calls": 26, + "turns": 27 + }, + "render-007": { + "answer": "2026-07-24", + "tokens": 61976, + "fixed_tokens": 1114, + "processed_tokens": 2221552, + "tool_calls": 52, + "turns": 53 + }, + "render-008": { + "answer": "2 failures", + "tokens": 23244, + "fixed_tokens": 1107, + "processed_tokens": 293498, + "tool_calls": 14, + "turns": 15 + }, + "render-009": { + "answer": "32 steps", + "tokens": 24802, + "fixed_tokens": 1113, + "processed_tokens": 304901, + "tool_calls": 14, + "turns": 15 + }, + "render-010": { + "answer": "unknown", + "tokens": 27352, + "fixed_tokens": 1110, + "processed_tokens": 248552, + "tool_calls": 12, + "turns": 13 + }, + "render-011": { + "answer": "fnd-sdf-gi-costs-2-4-ms-on-meridian--2w6c2pc5zmcsyjre", + "tokens": 14784, + "fixed_tokens": 1105, + "processed_tokens": 169974, + "tool_calls": 13, + "turns": 14 + }, + "render-012": { + "answer": "deferred", + "tokens": 24907, + "fixed_tokens": 1109, + "processed_tokens": 327278, + "tool_calls": 15, + "turns": 16 + }, + "render-013": { + "answer": "16-bit float per channel", + "tokens": 44654, + "fixed_tokens": 1107, + "processed_tokens": 840474, + "tool_calls": 25, + "turns": 26 + }, + "render-014": { + "answer": "52 percent", + "tokens": 29891, + "fixed_tokens": 1105, + "processed_tokens": 498792, + "tool_calls": 21, + "turns": 22 + }, + "render-015": { + "answer": "unknown", + "tokens": 33192, + "fixed_tokens": 1108, + "processed_tokens": 672577, + "tool_calls": 33, + "turns": 34 + }, + "render-016": { + "answer": "1.2 km", + "tokens": 7597, + "fixed_tokens": 1105, + "processed_tokens": 83430, + "tool_calls": 8, + "turns": 9 + }, + "render-017": { + "answer": "unknown", + "tokens": 73898, + "fixed_tokens": 1103, + "processed_tokens": 876842, + "tool_calls": 20, + "turns": 21 + }, + "render-018": { + "answer": "unknown", + "tokens": 39714, + "fixed_tokens": 1103, + "processed_tokens": 407816, + "tool_calls": 16, + "turns": 15 + }, + "render-019": { + "answer": "dec-use-the-custom-brine-curve-for-tonemapping--p735g6wf2njare86", + "tokens": 24124, + "fixed_tokens": 1101, + "processed_tokens": 213784, + "tool_calls": 13, + "turns": 12 + }, + "render-020": { + "answer": "fnd-screen-space-refraction-loses-off-screen-content--k8zvmmdee7wg2fjy", + "tokens": 28839, + "fixed_tokens": 1103, + "processed_tokens": 379827, + "tool_calls": 18, + "turns": 17 + }, + "render-021": { + "answer": "dec-adopt-a-visibility-buffer-for-all-opaque-geometr--68vchzqgd1avw8q4", + "tokens": 26437, + "fixed_tokens": 1108, + "processed_tokens": 286284, + "tool_calls": 13, + "turns": 14 + }, + "render-022": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 12580, + "fixed_tokens": 1103, + "processed_tokens": 123796, + "tool_calls": 10, + "turns": 11 + }, + "render-023": { + "answer": "dec-capture-the-cliffs-and-dunes-benchmarks-on-every--3pqyr66yh4v6q676", + "tokens": 52089, + "fixed_tokens": 1102, + "processed_tokens": 768227, + "tool_calls": 20, + "turns": 21 + }, + "render-024": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 11880, + "fixed_tokens": 1106, + "processed_tokens": 100529, + "tool_calls": 8, + "turns": 9 + }, + "render-025": { + "answer": "fnd-tern-cost-at-4k-output-on-meridian-after-fp16-pa--p4gcmzw2a4bvzjev", + "tokens": 71356, + "fixed_tokens": 1104, + "processed_tokens": 2969824, + "tool_calls": 67, + "turns": 68 + }, + "render-026": { + "answer": "dec-ship-four-tern-quality-presets--vjw18d82dx5gath8", + "tokens": 38167, + "fixed_tokens": 1103, + "processed_tokens": 714762, + "tool_calls": 26, + "turns": 27 + }, + "render-027": { + "answer": "con-frame-budget--j6vneva1g2ywj86m", + "tokens": 12354, + "fixed_tokens": 1110, + "processed_tokens": 91456, + "tool_calls": 7, + "turns": 8 + }, + "render-028": { + "answer": "dec-move-to-gpu-driven-two-phase-occlusion-culling--6dre481mabxsw6ee", + "tokens": 8975, + "fixed_tokens": 1103, + "processed_tokens": 106239, + "tool_calls": 10, + "turns": 11 + }, + "render-029": { + "answer": "fnd-vsm-page-pool-overflows-during-spring-tides--m2p8e89hhdmh0z8a", + "tokens": 12052, + "fixed_tokens": 1102, + "processed_tokens": 100464, + "tool_calls": 8, + "turns": 9 + }, + "render-030": { + "answer": "dec-use-screen-space-refraction-for-saltglass--vknzksgdrgr7zkg6", + "tokens": 30646, + "fixed_tokens": 1106, + "processed_tokens": 625135, + "tool_calls": 31, + "turns": 29 + }, + "render-031": { + "answer": "Alpha-tested foliage cost 2.1 ms in the visibility buffer versus 0.8 ms in forward+, because the alpha test runs in the raster pass with a per-pixel texture fetch, and saltglass refracts what is behind it so it needs the resolved scene first (dec-visibility-buffer-for-opaque-geometry-foliage-an--gvnvp04jgm7kbfh9).", + "tokens": 16257, + "fixed_tokens": 1108, + "processed_tokens": 206381, + "tool_calls": 14, + "turns": 15 + }, + "render-032": { + "answer": "The team chose Tern because Halcyon has no Skiff build, so shipping it on two platforms would break the one-upscaler constraint, and owning Tern's code suits saltglass refraction and caustics, which need upscaler changes the team can make without vendor tickets (dec-ship-tern-as-lantern-s-only-upscaler--fmreq35nzyjc7k4e).", + "tokens": 46223, + "fixed_tokens": 1109, + "processed_tokens": 600070, + "tool_calls": 19, + "turns": 20 + }, + "render-033": { + "answer": "128 MB", + "tokens": 32024, + "fixed_tokens": 1102, + "processed_tokens": 556889, + "tool_calls": 21, + "turns": 22 + }, + "render-034": { + "answer": "PC workstation (Meridian clock profile)", + "tokens": 30996, + "fixed_tokens": 1103, + "processed_tokens": 478755, + "tool_calls": 19, + "turns": 20 + }, + "render-035": { + "answer": "5912 pages", + "tokens": 15778, + "fixed_tokens": 1110, + "processed_tokens": 184464, + "tool_calls": 12, + "turns": 13 + }, + "render-036": { + "answer": "HDR-07", + "tokens": 71285, + "fixed_tokens": 1099, + "processed_tokens": 2809571, + "tool_calls": 52, + "turns": 53 + }, + "render-037": { + "answer": "unknown", + "tokens": 46409, + "fixed_tokens": 1107, + "processed_tokens": 748750, + "tool_calls": 22, + "turns": 23 + }, + "render-038": { + "answer": "dec-visibility-buffer-for-opaque-geometry-foliage-an--gvnvp04jgm7kbfh9", + "tokens": 24743, + "fixed_tokens": 1104, + "processed_tokens": 211957, + "tool_calls": 11, + "turns": 12 + }, + "render-039": { + "answer": "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "tokens": 29530, + "fixed_tokens": 1104, + "processed_tokens": 390451, + "tool_calls": 16, + "turns": 17 + }, + "render-040": { + "answer": "fnd-lantern-at-alpha--ckpd9ecvdjqbcr7c", + "tokens": 11469, + "fixed_tokens": 1106, + "processed_tokens": 88715, + "tool_calls": 8, + "turns": 9 + }, + "render-041": { + "answer": "4.7 ms", + "tokens": 30330, + "fixed_tokens": 1100, + "processed_tokens": 539968, + "tool_calls": 22, + "turns": 23 + }, + "render-042": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 16097, + "fixed_tokens": 1107, + "processed_tokens": 142191, + "tool_calls": 10, + "turns": 11 + }, + "render-043": { + "answer": "fnd-meshlet-cone-culling-rejects-9-percent-of-triang--t6e5x7trzedex105", + "tokens": 34449, + "fixed_tokens": 1107, + "processed_tokens": 768204, + "tool_calls": 29, + "turns": 30 + }, + "render-044": { + "answer": "dec-screen-space-refraction-with-a-ray-marched-fallb--2639xtpyrfrvbx1r", + "tokens": 28222, + "fixed_tokens": 1103, + "processed_tokens": 207670, + "tool_calls": 10, + "turns": 11 + }, + "render-045": { + "answer": false, + "tokens": 65587, + "fixed_tokens": 1104, + "processed_tokens": 1716355, + "tool_calls": 48, + "turns": 40 + }, + "render-046": { + "answer": "The cost Finding that rejected virtual shadow maps (3.1 ms, fnd-virtual-shadow-maps-cost-3-1-ms-on-meridian--r6x0sdfecms7j1yg) was a capture error; with cached pages VSM costs 1.4 ms, less than the 1.6 ms of four cascades, and cascade seams through saltglass remain unsolved under CSM. Reopened on 2026-01-12 against dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr; the record is dec-use-virtual-shadow-maps-for-the-sun--x8nqkn9q7eaf1ap4.", + "tokens": 29329, + "fixed_tokens": 1111, + "processed_tokens": 311926, + "tool_calls": 14, + "turns": 15 + }, + "render-047": { + "answer": "Because a release-build capture put the visibility buffer at 7.4 ms on Skiff against 8.1 ms for forward+ (6.3 ms with half-rate material resolve), so keeping a forward+ fallback was no longer worth it, and maintaining two opaque paths cost about an engineer-week a month. Record: dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 29560, + "fixed_tokens": 1107, + "processed_tokens": 404353, + "tool_calls": 17, + "turns": 18 + }, + "render-048": { + "answer": "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "tokens": 83065, + "fixed_tokens": 1111, + "processed_tokens": 5010661, + "tool_calls": 92, + "turns": 92 + }, + "render-049": { + "answer": "dec-use-virtual-shadow-maps-for-the-sun--x8nqkn9q7eaf1ap4", + "tokens": 107366, + "fixed_tokens": 1108, + "processed_tokens": 1634801, + "tool_calls": 30, + "turns": 28 + }, + "render-050": { + "answer": "con-frame-budget--fzdrbxns6dmrrmy3", + "tokens": 11861, + "fixed_tokens": 1110, + "processed_tokens": 90394, + "tool_calls": 7, + "turns": 8 + }, + "render-051": { + "answer": "iss-tern-ghosting-on-moving-saltglass--2yr3q6zf476wajqp", + "tokens": 18309, + "fixed_tokens": 1111, + "processed_tokens": 239854, + "tool_calls": 15, + "turns": 16 + }, + "render-052": { + "answer": "61 percent", + "tokens": 27726, + "fixed_tokens": 1100, + "processed_tokens": 387397, + "tool_calls": 17, + "turns": 18 + }, + "render-053": { + "answer": "every frame", + "tokens": 30738, + "fixed_tokens": 1105, + "processed_tokens": 494705, + "tool_calls": 20, + "turns": 20 + }, + "render-054": { + "answer": "debug-gpu-validation", + "tokens": 33400, + "fixed_tokens": 1105, + "processed_tokens": 700619, + "tool_calls": 25, + "turns": 26 + }, + "render-055": { + "answer": "fnd-tern-cost-at-4k-output-on-meridian-with-the-reac--81tcpzvhwzcy6wyn", + "tokens": 53695, + "fixed_tokens": 1104, + "processed_tokens": 776634, + "tool_calls": 22, + "turns": 22 + }, + "render-056": { + "answer": "208,000 meshlets", + "tokens": 28955, + "fixed_tokens": 1109, + "processed_tokens": 437011, + "tool_calls": 18, + "turns": 19 + }, + "render-057": { + "answer": "dec-visibility-buffer-on-all-platforms-skiff-resolve--k1ackk3c3275r5y1", + "tokens": 56644, + "fixed_tokens": 1105, + "processed_tokens": 2124924, + "tool_calls": 55, + "turns": 56 + }, + "render-058": { + "answer": "iss-which-upscaler-should-lantern-ship--g8c2y9tsr856tn2v", + "tokens": 46275, + "fixed_tokens": 1103, + "processed_tokens": 1355499, + "tool_calls": 41, + "turns": 42 + }, + "render-059": { + "answer": "fnd-sdf-gi-costs-4-7-ms-on-meridian--sm19rt2f274ajsvr", + "tokens": 38326, + "fixed_tokens": 1106, + "processed_tokens": 598689, + "tool_calls": 21, + "turns": 22 + }, + "render-060": { + "answer": "0.31", + "tokens": 32229, + "fixed_tokens": 1105, + "processed_tokens": 406144, + "tool_calls": 16, + "turns": 17 + }, + "render-061": { + "answer": "con-ui-composites-at-a-fixed-203-nits-in-hdr--de6aft7cvhdezn5s", + "tokens": 41561, + "fixed_tokens": 1106, + "processed_tokens": 517645, + "tool_calls": 20, + "turns": 17 + }, + "render-062": { + "answer": "con-frame-budget--j6vneva1g2ywj86m", + "tokens": 22930, + "fixed_tokens": 1100, + "processed_tokens": 133273, + "tool_calls": 8, + "turns": 9 + }, + "render-063": { + "answer": "dec-resolve-materials-with-one-uber-shader-over-bind--s9nbq2qn05mk5j6r", + "tokens": 31214, + "fixed_tokens": 1109, + "processed_tokens": 303168, + "tool_calls": 17, + "turns": 14 + }, + "render-064": { + "answer": "dec-use-a-cached-shadow-atlas-for-local-lights--8p6f2v774he38t5j", + "tokens": 33049, + "fixed_tokens": 1108, + "processed_tokens": 609710, + "tool_calls": 29, + "turns": 30 + }, + "render-065": { + "answer": "unknown", + "tokens": 36973, + "fixed_tokens": 1108, + "processed_tokens": 642184, + "tool_calls": 26, + "turns": 24 + }, + "render-066": { + "answer": "dec-use-cascaded-shadow-maps-with-four-cascades--s5c1y9jaf275afzr", + "tokens": 15189, + "fixed_tokens": 1106, + "processed_tokens": 260092, + "tool_calls": 19, + "turns": 19 + }, + "render-067": { + "answer": 4, + "tokens": 20757, + "fixed_tokens": 1110, + "processed_tokens": 141627, + "tool_calls": 8, + "turns": 9 + }, + "render-068": { + "answer": [ + "dec-exclude-caustic-luminance-from-tern-s-history-cl--xhxjn26ss343v2k4", + "dec-lock-caustic-photon-jitter-to-the-tide-phase--85x10k4h11t8nx7c", + "dec-use-the-probe-grid-alone-in-120-hz-performance-m--6dzm1nt40pjfpyae", + "dec-trace-sdf-gi-at-quarter-resolution-in-performanc--06cqt3mqn3fg5h6e", + "dec-read-paper-white-from-the-meridian-system-settin--7crbx73gqw1hn7xq", + "dec-add-an-in-game-paper-white-slider-from-80-to-400--1phfvxby983z8tjf" + ], + "tokens": 23312, + "fixed_tokens": 1103, + "processed_tokens": 124620, + "tool_calls": 7, + "turns": 8 + }, + "render-069": { + "answer": "iss-shadow-pages-run-out-at-spring-low-tide--f4fgy6b44jbkrsmh", + "tokens": 28144, + "fixed_tokens": 1104, + "processed_tokens": 206597, + "tool_calls": 10, + "turns": 11 + }, + "render-070": { + "answer": [], + "tokens": 57262, + "fixed_tokens": 1100, + "processed_tokens": 1260533, + "tool_calls": 31, + "turns": 32 + }, + "render-071": { + "answer": 3, + "tokens": 10733, + "fixed_tokens": 1101, + "processed_tokens": 72343, + "tool_calls": 8, + "turns": 7 + }, + "render-072": { + "answer": [ + "rsk-sdf-volumes-outgrow-the-render-target-memory-bud--xfd2pfztvskdcd2c", + "rsk-artists-cannot-match-concept-paintings-under-the--6cfh9vkamn0kbtcx" + ], + "tokens": 12961, + "fixed_tokens": 1109, + "processed_tokens": 106426, + "tool_calls": 10, + "turns": 9 + }, + "render-073": { + "answer": [ + "rsk-vsm-page-pool-overflows-when-tides-expose-the-fl--x1r161nrdngra4ty", + "rsk-gi-cannot-fit-the-120-hz-performance-mode--tqt3fj839dqfq5s7", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 17503, + "fixed_tokens": 1111, + "processed_tokens": 154434, + "tool_calls": 15, + "turns": 12 + }, + "render-074": { + "answer": 5, + "tokens": 7420, + "fixed_tokens": 1101, + "processed_tokens": 52830, + "tool_calls": 5, + "turns": 6 + }, + "render-075": { + "answer": "3", + "tokens": 37190, + "fixed_tokens": 1110, + "processed_tokens": 976826, + "tool_calls": 34, + "turns": 35 + }, + "render-076": { + "answer": false, + "tokens": 16684, + "fixed_tokens": 1104, + "processed_tokens": 224167, + "tool_calls": 16, + "turns": 17 + }, + "render-077": { + "answer": true, + "tokens": 41353, + "fixed_tokens": 1102, + "processed_tokens": 451643, + "tool_calls": 30, + "turns": 17 + }, + "render-078": { + "answer": [ + "dec-brine-curve-with-a-separate-hdr-shoulder-at-1-00--be68yywg78dysm2e", + "rsk-hdr-certification-slips-the-release-candidate--49t4xjbzmkgc22a4" + ], + "tokens": 41863, + "fixed_tokens": 1102, + "processed_tokens": 313059, + "tool_calls": 32, + "turns": 13 + }, + "render-079": { + "answer": 5, + "tokens": 47952, + "fixed_tokens": 1101, + "processed_tokens": 438522, + "tool_calls": 15, + "turns": 16 + }, + "render-080": { + "answer": "6 versions", + "tokens": 24073, + "fixed_tokens": 1104, + "processed_tokens": 198734, + "tool_calls": 10, + "turns": 11 + }, + "render-081": { + "answer": "unknown", + "tokens": 61462, + "fixed_tokens": 1104, + "processed_tokens": 1144700, + "tool_calls": 27, + "turns": 27 + }, + "render-082": { + "answer": false, + "tokens": 49697, + "fixed_tokens": 1105, + "processed_tokens": 1467568, + "tool_calls": 40, + "turns": 41 + }, + "render-083": { + "answer": null, + "tokens": 97362, + "fixed_tokens": 1113, + "processed_tokens": 3484516, + "tool_calls": 55, + "turns": 55 + }, + "tools-001": { + "answer": "completed", + "tokens": 20091, + "fixed_tokens": 1104, + "processed_tokens": 135959, + "tool_calls": 8, + "turns": 9 + }, + "tools-002": { + "answer": "47 minutes", + "tokens": 18811, + "fixed_tokens": 1110, + "processed_tokens": 139947, + "tool_calls": 8, + "turns": 9 + }, + "tools-003": { + "answer": "6 min 40 s", + "tokens": 14412, + "fixed_tokens": 1109, + "processed_tokens": 227594, + "tool_calls": 17, + "turns": 18 + }, + "tools-004": { + "answer": "unknown", + "tokens": 41222, + "fixed_tokens": 1108, + "processed_tokens": 324012, + "tool_calls": 11, + "turns": 12 + }, + "tools-005": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 24465, + "fixed_tokens": 1106, + "processed_tokens": 282253, + "tool_calls": 13, + "turns": 14 + }, + "tools-006": { + "answer": "dec-build-a-content-addressed-shader-compile-cache-s--e65rq9439k7xz5zh", + "tokens": 21334, + "fixed_tokens": 1110, + "processed_tokens": 140456, + "tool_calls": 8, + "turns": 9 + }, + "tools-007": { + "answer": "dec-raise-the-shader-cache-fetch-timeout-from-5-seco--4sgdf42ajyct6gt6", + "tokens": 109513, + "fixed_tokens": 1109, + "processed_tokens": 3280224, + "tool_calls": 43, + "turns": 44 + }, + "tools-008": { + "answer": "dec-key-the-shader-cache-on-raw-source-text-and-comp--hp6awmx3gq5rn9nc", + "tokens": 25671, + "fixed_tokens": 1106, + "processed_tokens": 218849, + "tool_calls": 11, + "turns": 12 + }, + "tools-009": { + "answer": "92.4%", + "tokens": 30520, + "fixed_tokens": 1107, + "processed_tokens": 762449, + "tool_calls": 33, + "turns": 34 + }, + "tools-010": { + "answer": "dec-cap-each-farm-agent-at-6-parallel-shader-compile--q5w4sfjm73tsj8x8", + "tokens": 47512, + "fixed_tokens": 1109, + "processed_tokens": 959298, + "tool_calls": 27, + "turns": 28 + }, + "tools-011": { + "answer": "fnd-two-compiler-builds-with-the-same-version-string--gmgr4ha0ndxqmcn9", + "tokens": 22835, + "fixed_tokens": 1112, + "processed_tokens": 201288, + "tool_calls": 11, + "turns": 12 + }, + "tools-012": { + "answer": 3, + "tokens": 20620, + "fixed_tokens": 1114, + "processed_tokens": 155266, + "tool_calls": 9, + "turns": 10 + }, + "tools-013": { + "answer": 6, + "tokens": 41175, + "fixed_tokens": 1110, + "processed_tokens": 515625, + "tool_calls": 18, + "turns": 19 + }, + "tools-014": { + "answer": false, + "tokens": 35104, + "fixed_tokens": 1110, + "processed_tokens": 334883, + "tool_calls": 13, + "turns": 13 + }, + "tools-015": { + "answer": [ + "iss-whether-the-audio-bank-packer-should-load-on-a-w--sqzp2rwp6s01w3b0", + "iss-whether-the-level-editor-autosave-needs-its-own--rkvmnbqe7w644zkr", + "iss-whether-the-artist-sandbox-should-load-on-a-work--73j819anqct8etfn", + "iss-whether-the-review-bot-should-load-on-a-worker-t--rtsjwvsksjdq15v1", + "iss-whether-the-level-editor-autosave-should-be-cach--radcjs4hyvh7p55q", + "iss-whether-the-crash-symbol-server-should-load-on-a--qvyphhgnsnm2cvqb", + "iss-whether-the-license-scanner-should-be-cached-bet--nqhd89czbfat9rya", + "iss-whether-the-doc-site-should-load-on-a-worker-thr--qxswe0pg52ekky0z", + "iss-whether-the-asset-search-should-load-on-a-worker--f9gpv3z522x6p6h4", + "iss-whether-the-cache-key-should-move-its-settings-t--jgtdn2f02p8fx5nr", + "iss-whether-the-review-bot-needs-its-own-memory-pool--pg3m8g9gn3y6v3kq", + "iss-whether-the-changelog-bot-should-skip-work-when--p2vrp5d682p3jrc3", + "iss-whether-the-cache-key-should-log-less-0068--cysv4n56q1qn7t2k", + "iss-whether-the-fetch-timeout-needs-its-own-unit-tes--11tqzws72gms2h8x", + "iss-whether-the-audio-bank-packer-needs-its-own-memo--qjrqrseast8d5mhf", + "iss-whether-the-fetch-timeout-should-print-fewer-war--mw8zejz6hk5m60x0", + "iss-whether-the-crash-symbol-server-should-ship-in-t--bhq91ajavah18wsk", + "iss-whether-the-asset-search-should-load-on-a-worker--80b27kqh36rpafdt", + "iss-whether-the-doc-site-should-ship-in-the-first-pa--wcdkq5zab7dxe5yj", + "iss-whether-the-patch-differ-should-be-cached-betwee--28b5zqb1ketwzf0r", + "iss-whether-the-test-runner-should-load-on-a-worker--aygay1r0qj4mzf78", + "iss-whether-the-texture-viewer-should-be-split-into--1jhgyqb70h5x7m7b", + "iss-whether-the-shader-compile-cache-needs-its-own-u--fyysrcv6fvbsvcsy", + "iss-whether-the-level-editor-autosave-should-be-spli--ncnwh0csw671fs0q", + "iss-whether-the-asset-search-should-ship-in-the-firs--hsb2avhx613prxew", + "iss-whether-the-test-runner-needs-its-own-memory-poo--bs6arq0qv37d6wmx", + "iss-whether-the-artist-sandbox-should-be-split-into--ek47ca2vgmrjanp8", + "rsk-shader-cache-storage-could-outgrow-the-4-tb-farm--76y9649mwrx7aavd" + ], + "tokens": 71052, + "fixed_tokens": 1109, + "processed_tokens": 847117, + "tool_calls": 22, + "turns": 23 + }, + "tools-016": { + "answer": "dec-key-the-shader-cache-on-preprocessed-source-defi--qx2da8e4cbkx0drp", + "tokens": 42301, + "fixed_tokens": 1109, + "processed_tokens": 977905, + "tool_calls": 32, + "turns": 33 + }, + "world-001": { + "answer": "64 KB", + "tokens": 25585, + "fixed_tokens": 1105, + "processed_tokens": 257209, + "tool_calls": 12, + "turns": 13 + }, + "world-002": { + "answer": "5.1 ms", + "tokens": 34409, + "fixed_tokens": 1108, + "processed_tokens": 359435, + "tool_calls": 13, + "turns": 14 + }, + "world-003": { + "answer": "41%", + "tokens": 80126, + "fixed_tokens": 1107, + "processed_tokens": 4117267, + "tool_calls": 77, + "turns": 78 + }, + "world-004": { + "answer": "4,100 file opens", + "tokens": 36199, + "fixed_tokens": 1107, + "processed_tokens": 576745, + "tool_calls": 21, + "turns": 22 + }, + "world-005": { + "answer": "11 m", + "tokens": 34426, + "fixed_tokens": 1106, + "processed_tokens": 532868, + "tool_calls": 19, + "turns": 20 + }, + "world-006": { + "answer": "3.4 reversals per minute", + "tokens": 44546, + "fixed_tokens": 1102, + "processed_tokens": 867928, + "tool_calls": 28, + "turns": 27 + }, + "world-007": { + "answer": "unknown", + "tokens": 52480, + "fixed_tokens": 1103, + "processed_tokens": 1743898, + "tool_calls": 44, + "turns": 45 + }, + "world-008": { + "answer": "10 Hz", + "tokens": 35250, + "fixed_tokens": 1103, + "processed_tokens": 610301, + "tool_calls": 22, + "turns": 23 + }, + "world-009": { + "answer": "unknown", + "tokens": 33420, + "fixed_tokens": 1112, + "processed_tokens": 723330, + "tool_calls": 27, + "turns": 28 + }, + "world-010": { + "answer": "29 GB/s", + "tokens": 26436, + "fixed_tokens": 1105, + "processed_tokens": 199858, + "tool_calls": 10, + "turns": 11 + }, + "world-011": { + "answer": "fnd-shipping-entity-names-costs-57-mb-at-full-popula--6w7gjvy7wdmdx9qx", + "tokens": 46378, + "fixed_tokens": 1104, + "processed_tokens": 1093547, + "tool_calls": 32, + "turns": 33 + }, + "world-012": { + "answer": "dec-run-engine-jobs-and-gull-scripts-on-one-thread-p--07pbzvqg00pmnrr3", + "tokens": 22947, + "fixed_tokens": 1109, + "processed_tokens": 168719, + "tool_calls": 9, + "turns": 10 + }, + "world-013": { + "answer": "unknown", + "tokens": 39053, + "fixed_tokens": 1110, + "processed_tokens": 997472, + "tool_calls": 32, + "turns": 33 + }, + "world-014": { + "answer": "unknown", + "tokens": 28562, + "fixed_tokens": 1110, + "processed_tokens": 267390, + "tool_calls": 12, + "turns": 13 + }, + "world-015": { + "answer": "3.0 ms per frame", + "tokens": 42263, + "fixed_tokens": 1112, + "processed_tokens": 473818, + "tool_calls": 17, + "turns": 16 + }, + "world-016": { + "answer": "unknown", + "tokens": 20722, + "fixed_tokens": 1111, + "processed_tokens": 138160, + "tool_calls": 8, + "turns": 9 + }, + "world-017": { + "answer": "dec-partition-the-world-into-128-m-cells--372b6czbjf29ymaw", + "tokens": 30123, + "fixed_tokens": 1106, + "processed_tokens": 454506, + "tool_calls": 18, + "turns": 19 + }, + "world-018": { + "answer": "dec-run-engine-jobs-on-a-work-stealing-thread-pool--1pd72x1pxpgwy21x", + "tokens": 24014, + "fixed_tokens": 1104, + "processed_tokens": 145065, + "tool_calls": 10, + "turns": 9 + }, + "world-019": { + "answer": "fnd-tide-priority-reads-starve-player-proximity-load--by0d7b0bk39xz3rk", + "tokens": 43997, + "fixed_tokens": 1108, + "processed_tokens": 1018549, + "tool_calls": 32, + "turns": 33 + }, + "world-020": { + "answer": "dec-schedule-streaming-reads-by-deadline-instead-of--q2x5aknabgcssdj2", + "tokens": 13242, + "fixed_tokens": 1109, + "processed_tokens": 100405, + "tool_calls": 8, + "turns": 9 + }, + "world-021": { + "answer": "dec-cut-navigation-tiles-to-64-m-so-three-fit-along--snvepk8ygeb3qsjs", + "tokens": 26630, + "fixed_tokens": 1104, + "processed_tokens": 208857, + "tool_calls": 10, + "turns": 11 + }, + "world-022": { + "answer": "fnd-41-of-alpha-crash-reports-have-no-usable-stack--35a50b07ferhf13m", + "tokens": 16233, + "fixed_tokens": 1102, + "processed_tokens": 150291, + "tool_calls": 10, + "turns": 11 + }, + "world-023": { + "answer": "dec-compress-streamed-assets-with-kelp--ezwc5p1tgsgm7v8m", + "tokens": 33237, + "fixed_tokens": 1101, + "processed_tokens": 380671, + "tool_calls": 15, + "turns": 16 + }, + "world-024": { + "answer": "dec-shrink-kelp-s-dictionary-to-1-mb-for-streamed-as--yh8hktfqnyqhepew", + "tokens": 29658, + "fixed_tokens": 1109, + "processed_tokens": 388253, + "tool_calls": 16, + "turns": 17 + }, + "world-025": { + "answer": "con-streaming-pool-budget--kca88sgh2j6sxvd5", + "tokens": 24697, + "fixed_tokens": 1102, + "processed_tokens": 206590, + "tool_calls": 10, + "turns": 11 + }, + "world-026": { + "answer": "fnd-retail-skiff-units-sustain-only-190-mb-s-of-stre--485sdcmm3gweevvn", + "tokens": 100409, + "fixed_tokens": 1100, + "processed_tokens": 2794332, + "tool_calls": 49, + "turns": 45 + }, + "world-027": { + "answer": "160 MB/s", + "tokens": 27772, + "fixed_tokens": 1105, + "processed_tokens": 353623, + "tool_calls": 15, + "turns": 16 + }, + "world-028": { + "answer": "unknown", + "tokens": 76807, + "fixed_tokens": 1104, + "processed_tokens": 1947292, + "tool_calls": 38, + "turns": 39 + }, + "world-029": { + "answer": "dec-run-engine-jobs-and-gull-scripts-on-one-thread-p--07pbzvqg00pmnrr3", + "tokens": 28764, + "fixed_tokens": 1101, + "processed_tokens": 661513, + "tool_calls": 33, + "turns": 34 + }, + "world-030": { + "answer": "fnd-world-core-state-at-alpha--tkbsm4867h65jqq8", + "tokens": 49528, + "fixed_tokens": 1111, + "processed_tokens": 1593202, + "tool_calls": 50, + "turns": 51 + }, + "world-031": { + "answer": "unknown", + "tokens": 38048, + "fixed_tokens": 1104, + "processed_tokens": 736237, + "tool_calls": 26, + "turns": 26 + }, + "world-032": { + "answer": "fnd-float-tide-heights-drift-apart-between-pc-and-me--7jb0fg4b5xzr7q5e", + "tokens": 33806, + "fixed_tokens": 1103, + "processed_tokens": 584698, + "tool_calls": 22, + "turns": 23 + }, + "world-033": { + "answer": "dec-save-player-changes-as-deltas-against-the-tide-f--n38scy1yaarsr8cw", + "tokens": 35500, + "fixed_tokens": 1105, + "processed_tokens": 677556, + "tool_calls": 24, + "turns": 25 + }, + "world-034": { + "answer": "unknown", + "tokens": 40821, + "fixed_tokens": 1107, + "processed_tokens": 919028, + "tool_calls": 30, + "turns": 31 + }, + "world-035": { + "answer": false, + "tokens": 40589, + "fixed_tokens": 1108, + "processed_tokens": 448136, + "tool_calls": 16, + "turns": 17 + }, + "world-036": { + "answer": "Kestrel retired the fiber scheduler for engine jobs because 41% of alpha crash reports had no usable stack (fiber worker dumps showed only scheduler frames), which blocked crash triage for beta, while a continuation-based thread pool kept workers 93% busy during tide turns, close to the 95% fibers gave. See dec-return-engine-jobs-to-a-thread-pool-with-continu--1zkc1v2x97taf50h.", + "tokens": 33203, + "fixed_tokens": 1099, + "processed_tokens": 484777, + "tool_calls": 19, + "turns": 19 + }, + "world-037": { + "answer": "fnd-hot-reload-drops-suspended-gull-coroutines--va0fn0svryzwbzws", + "tokens": 45465, + "fixed_tokens": 1107, + "processed_tokens": 812569, + "tool_calls": 25, + "turns": 26 + }, + "world-038": { + "answer": "3 archetypes", + "tokens": 15512, + "fixed_tokens": 1106, + "processed_tokens": 173101, + "tool_calls": 14, + "turns": 13 + }, + "world-039": { + "answer": "unknown", + "tokens": 78571, + "fixed_tokens": 1105, + "processed_tokens": 967935, + "tool_calls": 19, + "turns": 20 + }, + "world-040": { + "answer": "unknown", + "tokens": 70114, + "fixed_tokens": 1103, + "processed_tokens": 3003468, + "tool_calls": 60, + "turns": 61 + }, + "world-041": { + "answer": "61%", + "tokens": 24844, + "fixed_tokens": 1105, + "processed_tokens": 200946, + "tool_calls": 10, + "turns": 11 + }, + "world-042": { + "answer": "5.4 s", + "tokens": 30238, + "fixed_tokens": 1111, + "processed_tokens": 346543, + "tool_calls": 14, + "turns": 15 + }, + "world-043": { + "answer": "5.8 minutes", + "tokens": 47691, + "fixed_tokens": 1104, + "processed_tokens": 759696, + "tool_calls": 21, + "turns": 22 + }, + "world-044": { + "answer": "dec-move-engine-jobs-to-fibers--qjkgnm8cex8ydzfq", + "tokens": 48934, + "fixed_tokens": 1110, + "processed_tokens": 756163, + "tool_calls": 27, + "turns": 24 + }, + "world-045": { + "answer": "dec-run-engine-jobs-on-fibers-and-pin-gull-scripts-t--rw28cbqfd07rd9te", + "tokens": 27199, + "fixed_tokens": 1102, + "processed_tokens": 371026, + "tool_calls": 16, + "turns": 17 + }, + "world-046": { + "answer": "con-streaming-pool-budget--632bkbs79bwj4cr5", + "tokens": 27179, + "fixed_tokens": 1108, + "processed_tokens": 277904, + "tool_calls": 12, + "turns": 13 + }, + "world-047": { + "answer": "fnd-world-core-state-at-alpha--tkbsm4867h65jqq8", + "tokens": 27138, + "fixed_tokens": 1112, + "processed_tokens": 204018, + "tool_calls": 11, + "turns": 12 + }, + "world-048": { + "answer": "dec-compile-gull-to-bytecode-for-a-register-vm--jrqje646fpevp5f3", + "tokens": 30186, + "fixed_tokens": 1108, + "processed_tokens": 350961, + "tool_calls": 15, + "turns": 16 + }, + "world-049": { + "answer": "22%", + "tokens": 56422, + "fixed_tokens": 1107, + "processed_tokens": 1446198, + "tool_calls": 40, + "turns": 38 + }, + "world-050": { + "answer": "fnd-alpha-content-needs-6-2-ms-of-gull-time-per-fram--4mekgdpsdv5nk085", + "tokens": 48926, + "fixed_tokens": 1108, + "processed_tokens": 687435, + "tool_calls": 20, + "turns": 21 + }, + "world-051": { + "answer": "unknown", + "tokens": 37312, + "fixed_tokens": 1104, + "processed_tokens": 838952, + "tool_calls": 28, + "turns": 29 + }, + "world-052": { + "answer": "Kenji Arakawa", + "tokens": 46527, + "fixed_tokens": 1112, + "processed_tokens": 832013, + "tool_calls": 26, + "turns": 27 + }, + "world-053": { + "answer": "unknown", + "tokens": 46830, + "fixed_tokens": 1101, + "processed_tokens": 1193729, + "tool_calls": 35, + "turns": 36 + }, + "world-054": { + "answer": "Marta Lindqvist", + "tokens": 29874, + "fixed_tokens": 1103, + "processed_tokens": 357998, + "tool_calls": 17, + "turns": 16 + }, + "world-055": { + "answer": "Kelp stays because the decision to shrink Kelp's dictionary to 1 MB for streamed assets (dec-shrink-kelp-s-dictionary-to-1-mb-for-streamed-as--yh8hktfqnyqhepew) ships it on every platform, and cold on Skiff it beats Zstandard by 1.7x. That decision also closed the reopened Zstandard option (dec-compress-streamed-assets-with-zstandard--bs1t15cygc51cpje).", + "tokens": 44094, + "fixed_tokens": 1103, + "processed_tokens": 821961, + "tool_calls": 28, + "turns": 27 + }, + "world-056": { + "answer": "dec-run-engine-jobs-on-a-work-stealing-thread-pool--1pd72x1pxpgwy21x", + "tokens": 22706, + "fixed_tokens": 1101, + "processed_tokens": 193360, + "tool_calls": 10, + "turns": 11 + }, + "world-057": { + "answer": "[212, 87]", + "tokens": 19715, + "fixed_tokens": 1108, + "processed_tokens": 276961, + "tool_calls": 16, + "turns": 17 + }, + "world-058": { + "answer": "0.8 GB/s", + "tokens": 36148, + "fixed_tokens": 1102, + "processed_tokens": 621723, + "tool_calls": 22, + "turns": 23 + }, + "world-059": { + "answer": "dec-derive-the-missing-harmonic-phase-from-each-save--gbqrvvgyq2gy2gnw", + "tokens": 27283, + "fixed_tokens": 1108, + "processed_tokens": 358592, + "tool_calls": 18, + "turns": 17 + }, + "world-060": { + "answer": "dec-partition-the-world-into-192-m-cells--bk9r1c5jwb6fh8tc", + "tokens": 29702, + "fixed_tokens": 1105, + "processed_tokens": 368839, + "tool_calls": 16, + "turns": 17 + }, + "world-061": { + "answer": "236 MB", + "tokens": 19266, + "fixed_tokens": 1105, + "processed_tokens": 245428, + "tool_calls": 14, + "turns": 15 + }, + "world-062": { + "answer": "310 MB", + "tokens": 27117, + "fixed_tokens": 1103, + "processed_tokens": 372762, + "tool_calls": 19, + "turns": 18 + }, + "world-063": { + "answer": "con-streaming-pool-budget--632bkbs79bwj4cr5", + "tokens": 32016, + "fixed_tokens": 1107, + "processed_tokens": 517992, + "tool_calls": 20, + "turns": 21 + }, + "world-064": { + "answer": [ + "dec-force-strict-ieee-float-mode-for-the-tide-simula--ecwmqt7ncjngysqw", + "dec-have-the-host-stream-tide-heights-to-every-peer--a3rb1seedn2bj7k8" + ], + "tokens": 39555, + "fixed_tokens": 1101, + "processed_tokens": 579920, + "tool_calls": 20, + "turns": 21 + }, + "world-065": { + "answer": "3.1 ms", + "tokens": 50866, + "fixed_tokens": 1101, + "processed_tokens": 1645519, + "tool_calls": 49, + "turns": 50 + }, + "world-066": { + "answer": "46", + "tokens": 47467, + "fixed_tokens": 1107, + "processed_tokens": 1080380, + "tool_calls": 31, + "turns": 32 + }, + "world-067": { + "answer": "1.2 GB/s", + "tokens": 25453, + "fixed_tokens": 1113, + "processed_tokens": 374117, + "tool_calls": 22, + "turns": 19 + }, + "world-068": { + "answer": true, + "tokens": 39935, + "fixed_tokens": 1103, + "processed_tokens": 385878, + "tool_calls": 15, + "turns": 15 + }, + "world-069": { + "answer": 2, + "tokens": 314898, + "fixed_tokens": 1104, + "processed_tokens": 14423843, + "tool_calls": 84, + "turns": 81 + }, + "world-070": { + "answer": 3, + "tokens": 19699, + "fixed_tokens": 1103, + "processed_tokens": 139371, + "tool_calls": 8, + "turns": 9 + }, + "world-071": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--yxsjd26wcaz6fgx9" + ], + "tokens": 73042, + "fixed_tokens": 1104, + "processed_tokens": 2026040, + "tool_calls": 41, + "turns": 42 + }, + "world-072": { + "answer": [ + "dec-compress-streamed-assets-with-zstandard--bs1t15cygc51cpje", + "dec-compress-streamed-assets-with-lz4--v1pfsgfp0b3yjv4s" + ], + "tokens": 67462, + "fixed_tokens": 1104, + "processed_tokens": 1027237, + "tool_calls": 23, + "turns": 24 + }, + "world-073": { + "answer": false, + "tokens": 28661, + "fixed_tokens": 1106, + "processed_tokens": 373609, + "tool_calls": 16, + "turns": 17 + }, + "world-074": { + "answer": 7, + "tokens": 9160, + "fixed_tokens": 1106, + "processed_tokens": 69361, + "tool_calls": 6, + "turns": 7 + }, + "world-075": { + "answer": [ + "dec-store-components-in-archetype-tables-and-volatil--8679sjn071ycsbpv", + "dec-partition-the-world-into-192-meter-cells--q62ddwdv2n8wxay6", + "dec-return-engine-jobs-to-the-thread-pool-with-conti--547qbweqvrp1j5g3" + ], + "tokens": 6171, + "fixed_tokens": 1102, + "processed_tokens": 49992, + "tool_calls": 5, + "turns": 6 + }, + "world-076": { + "answer": "unknown", + "tokens": 72154, + "fixed_tokens": 1110, + "processed_tokens": 2830874, + "tool_calls": 63, + "turns": 64 + }, + "world-077": { + "answer": 251, + "tokens": 44042, + "fixed_tokens": 1107, + "processed_tokens": 831712, + "tool_calls": 28, + "turns": 29 + }, + "world-078": { + "answer": [ + "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "rsk-reworking-the-read-scheduler-may-slip-the-coast--9jzvs7etvb5jxqtp" + ], + "tokens": 15470, + "fixed_tokens": 1102, + "processed_tokens": 124934, + "tool_calls": 11, + "turns": 10 + }, + "world-079": { + "answer": [ + "con-streaming-pool-budget--8czwb4fjr5p5vpy7", + "con-no-runtime-code-generation-on-consoles--6cf25zqmsvqkdd73", + "con-tide-state-must-be-bit-identical-on-every-peer--zrpnp8qjyxzkas7t", + "con-saves-from-beta-onward-must-load-in-every-later--v6z856sxtgc5sb09" + ], + "tokens": 12934, + "fixed_tokens": 1105, + "processed_tokens": 90583, + "tool_calls": 7, + "turns": 8 + }, + "world-080": { + "answer": [ + "iss-gull-hot-reload-rejects-struct-layout-changes--3v2w2f9hg2gwy6aj", + "iss-gull-hot-reload-drops-suspended-coroutines--z7j2gmyny6gdxd07", + "iss-cross-platform-cloud-saves-are-unplanned--5bk0k4wrgj42ztfy" + ], + "tokens": 43854, + "fixed_tokens": 1107, + "processed_tokens": 900859, + "tool_calls": 28, + "turns": 29 + }, + "world-081": { + "answer": "rsk-hot-reload-may-corrupt-live-script-state--ec2ht2r1sq6h56ew", + "tokens": 72378, + "fixed_tokens": 1107, + "processed_tokens": 636546, + "tool_calls": 42, + "turns": 15 + } + } +}