Repository navigation
fix(memory): size and charge text builds in one admission - #4393
Conversation
|
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Reviewed the five-file change and its real artifact/ledger regressions. Sizing and charging now use the same locked resident-memory ledger state; the caller releases an insufficient reservation before its single bounded shed/trim retry and registers the minimum working set only after final refusal. Existing cgroup working-set calculation, memory ceilings and reader charge transfer/release are preserved. No new blocker found. Pinned rustfmt and diff checks pass; this review does not claim the author-reported native runs were repeated locally. Current-head repository/benchmark gates passed, with native CI still pending.
This also closes a remaining convergence gap in the earlier #4379/#4346 watermark accounting repair. I am bringing those branches onto this same implementation and removing their now-unused reserve_below API, while preserving their unrelated fixes and regressions. Cross-referenced the repository issues; #3736 is broader runtime work, so this focused change should not imply that issue is fully closed.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Pushed 9b7550a with the reviewed progress follow-up from #4368, preserving the existing storage callback-receipt regression in 74fc14c.
A memory refusal before text-builder initialization previously left the sealed-source progress snapshot looking unblocked. Both worker-entry and artifact-admission refusals now use the existing generation/owner-epoch guarded publisher to report ResidentMemory. There is no new retry loop or progress authority.
Added the existing cgroup-headroom regression and a native same-owner recovery regression: real ledger reservation held → repeated typed refusal with no leaked charge → reservation released and canonical headroom notification → the same owner publishes a Ready artifact and clears the block. Readiness failure diagnostics include the existing fixture phase.
All four follow-up files match the independently reviewed #4368 tree. Pinned rustfmt, diff checks, and commitlint pass; native compilation and regression execution remain for CI.
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Pushed 1083367 to address the stale memory-block race raised in comment 4334535151.
Initial artifact admission and publication-tail refusals now publish while their existing projection claim is held. The outer duplicate classification is removed. Worker admission uses the progress epoch observed before admission; the epoch/Ready checks and update are atomic under the existing progress authority, with generation and owner checks retained. A missing observed snapshot cannot authorize a later update. Verified owner installation clears an earlier refusal when it publishes Ready.
Regressions cover a delayed refusal after newer pending or Ready progress, a genuine unchanged pending refusal, owner replacement, direct claimed admission/recovery, and a real finalized artifact reaching Ready without another pending boundary. The new fixture uses deterministic memory sampling, and waits use the existing failure ceiling. Pinned rustfmt, diff checks and commitlint pass; native regression execution awaits fresh Bazel CI.
Text artifact builds can be sized from one RSS reading and then refused by a later fixed reservation after another task grows the process. That late refusal bypasses retained-owner reclamation, so historical indexing waits despite having reclaimable state.
Size and charge through the canonical process resident-memory authority under its ledger lock. Keep the existing minimum working set and single bounded shed/trim attempt; remove the temporary watermark-padding charge.
Memory refusals now remain attached to the progress they actually blocked. Artifact admission publishes under its projection claim; worker admission checks the observed progress epoch. A delayed refusal cannot overwrite newer pending or Ready progress, and successful Ready publication clears the prior block. The existing storage callback-receipt regression is retained.
Follow-up verification through Bazel on 25b0472:
Earlier verification of the admission change:
The original failure was in https://github.com/ScriptedAlchemy/tracedecay/actions/runs/39546585657/job/119048801093. No memory limit, working-set minimum, test timeout, workload, or assertion was relaxed.
Full workspace CI passed on
104f6ad9410b8b931559c2a3fb9532ffbfdbf7e4: all 169 Linux fleet test targets, aggregate Linux, strict Clippy, feature gates, repository/dashboard checks, and the shipped CLI build. The branch is now updated from master atc27f4c963c8b3ad41c174646316c52a314a6f7a8; fresh full CI on that integration head must finish before merge. Remaining query/readiness failures are not assumed to share this root cause.