Skip to content

fix(sessions): skip Codex replay a complete frontier covers - #4380

Merged
ScriptedAlchemy merged 15 commits into
masterfrom
devin/1792524686-codex-replay-covered-consumer
Oct 22, 2026
Merged

ScriptedAlchemy merged 15 commits into
masterfrom
devin/1792524686-codex-replay-covered-consumer

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

A restarted daemon skips replaying Codex rollouts to a late consumer when that consumer's durable complete frontier covers the replay index's exact corpus epoch. Acknowledgement returns the consumer to shared discovery. Incomplete frontiers and changed corpora still receive replay; newly added rollouts remain discoverable.

The regression restarts discovery with project and profile consumers, requires no redundant delivery for the complete profile frontier, then adds a rollout and requires delivery.

Validation and fleet repair:

  • On bb7ac81, all 623 session-runtime unit tests passed in 39.49s, including the late-consumer complete-frontier regression. The fleet passed 166 of 169 targets. Global-DB and session suites timed out; the root suite passed 928 tests and failed ten production-harness cases on code-index resident-memory admission refusals.
  • Those refusal readings include shared worker cgroup pressure, not just the test process's allocations. The Linux worker had a 64 GiB hard cap, a 48 GiB throttle point, a roughly 50 GiB recorded peak, and substantial throttling while admitting up to 48 simultaneous actions. Individual large Rust compiler processes were observed around 7 GiB RSS.
  • The worker was drained to zero running actions, then restarted with max_inflight_tasks=4. No active action was killed. Its memory limits and the scheduler were preserved. The new worker registered with four slots, accepted work, and reported no initial memory-pressure or throttle events. This capacity correction still needs validation under sustained CI load.
  • The branch is updated with master through dfe8edc, including schema trigger validation batched into one query instead of one query per trigger. The two timeout causes are not conclusively isolated; fresh CI must verify both suites and the publication failures without changing their deadlines, memory limits, or assertions.

Current head: f2a66c4. Full hosted CI on this head is required before merge.

Refs #4374.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@changeset-bot

changeset-bot Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: f2a66c4

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cross-referenced #4374. The replay shortcut is appropriately conditional on a complete durable frontier equal to the rebuilt index frontier, and the regression also checks a newly added rollout. No new concrete code defect found in this pass.

I changed the PR body's Fixes #4374 to Refs #4374: the issue also tracks the measured project-scope re-offer cost (approximately 28 ms/file), which this PR and #4374 (comment) explicitly leave open. The narrower restart/replay fix should leave that instrumentation/performance work tracked. Runtime tests were reviewed but not independently rerun here.

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed 5e4184e after reviewing native job 119099357125 / run 39546581307 attempt 3. That run passed 167/169 targets; MCP had 667 passed and 6 failed, and runtime acceptance had 130 passed and 4 failed. The concurrent master merge at 8f3160a is preserved.

The branch now uses the canonical sized memory admission from #4393, which charges the actual text-build allocation atomically instead of adding the synthetic watermark margin. It includes the measured-headroom regression and the reviewed race-safe blocked-progress/recovery coverage: artifact refusals publish under their existing claim, and worker-entry refusals cannot overwrite a newer progress epoch or Ready.

The distinct admitted advisory-cycle timeout is still not localized by the available evidence. Independent source tracing found no demonstrated deadlock or lost owner to justify another production change. Its failure assertions now include at most 16 KiB of the existing daemon log captured before the invocation within the original phase deadline; the snapshot timing is labeled explicitly. Original timeout and retry rules remain, and admitted timeouts still fail.

Pinned rustfmt on all six changed Rust files, the whole-delta diff check and commitlint pass. Native compilation/execution is delegated to fresh Bazel CI. The memory-accounting defect is independently established, but these changes are not presented as proof that it caused every readiness or advisory failure.

ScriptedAlchemy and others added 2 commits October 21, 2026 04:42
…dex-replay-covered-consumer

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	crates/tracedecay-code-index-runtime/tests/code_index_scheduler/serving.rs
@ScriptedAlchemy
ScriptedAlchemy enabled auto-merge (squash) October 22, 2026 00:26
@ScriptedAlchemy
ScriptedAlchemy merged commit 27c61da into master Oct 22, 2026
28 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant