Skip to content

Bound storage-report sizing to a shared filesystem budget - #4353

Merged
ScriptedAlchemy merged 20 commits into
masterfrom
copilot/simplify-storage-report-sizing
Oct 22, 2026
Merged

ScriptedAlchemy merged 20 commits into
masterfrom
copilot/simplify-storage-report-sizing

Conversation

Copilot AI commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Bound storage-report filesystem work across all sampled projects and report partial sizes as lower bounds.
  • Share the budget with retention inventory scans and preserve explicit coverage and failure reasons.

Motivation

Project page limits did not bound work inside each project, and registered and unregistered sizing used separate walks with different failure accounting. Partial traversal could therefore appear complete, while unavailable retention protection still triggered generation scanning.

Changes

  • Sizing and coverage. Use the footprint walker for registered and unregistered storage, with one per-call entry budget, cancellation and deadline checks, and explicit complete, partial, or unavailable accounting.
  • Pagination and rendering. Keep outer cursor state separate from subtree coverage; propagate coverage through report aggregation and CLI rendering.
  • Retention and path safety. Skip retention planning when protection is unavailable; budget its directory inventory and reject symlinked ancestors before traversal or free-page sampling.
  • Regression coverage. Add isolated tests for shared-budget exhaustion, cancellation, ancestor symlinks, and serialized partial coverage.

Test plan

  • Affected Bazel targets passed on b0d45bb; current-head CI is pending
  • bazel build //... passed on b0d45bb before the latest master update
  • Tested manually (describe below if applicable)

Full CI on b0d45bb85e1b3dbe8b447cb0076ddec3ceddbfd7 passed bazel build --config=ci //..., all 169 fleet test targets, the Linux aggregate, Clippy, repository gates, feature gates, Dashboard, and the shipped Linux CLI build. CI after that master update on 5193a8b passed 168/169 targets. Only the composition-root unit target failed (937 passed, 1 failed): committed_save_remains_noncurrent_until_its_broker_handoff captured its baseline before initial graph-admission work finished, allowing a setup generation to race the save assertion. Clippy, shipped CLI build, Dashboard, feature gates, and repository gates passed.

Current head fe58c29d0397923b1f749bd759431a68a8288f3b includes master through 44d8265c13a5feb70e7ea834c1d11ed413f3f2af and reuses the exact fixture fix from #4351 (6c70575): wait for the initial graph owner admission to settle before pinning the save baseline. All generation, freshness, broker-debt, and receipt assertions retain their existing bounds. The identical patch already passed both focused handoff tests and strict Clippy in #4351; that shared test lane owns the behavioral validation. Pinned formatting, whitespace checks, and commit hooks passed here. No production behavior or deadline changed. Fresh full CI on this combined head must pass before merge.

Checklist

  • CHANGELOG.md updated (under [Unreleased] if no version bump)
  • No secrets, credentials, or .env files included
  • Breaking changes documented (if any)

@changeset-bot

changeset-bot Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: f9c6974

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copilot AI changed the title [WIP] Simplify storage-report sizing in bounded filesystem walk Bound storage-report sizing to a shared filesystem budget Oct 19, 2026
Copilot AI requested a review from ScriptedAlchemy October 19, 2026 04:49
Copilot AI and others added 4 commits October 20, 2026 15:41
Co-authored-by: ScriptedAlchemy <25274700+ScriptedAlchemy@users.noreply.github.com>
Co-authored-by: ScriptedAlchemy <25274700+ScriptedAlchemy@users.noreply.github.com>
@ScriptedAlchemy
ScriptedAlchemy force-pushed the copilot/simplify-storage-report-sizing branch from b3e2ad9 to 30f62b9 Compare October 20, 2026 22:42
@ScriptedAlchemy
ScriptedAlchemy marked this pull request as ready for review October 21, 2026 06:24
devin-ai-integration[bot]

This comment was marked as resolved.

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up repairs pushed: the artifact-mark phase now uses the same budget/cancellation path as scope enumeration; offline sizing and sampling share one counter; truncated unregistered counts are marked as lower bounds; empty pagination results no longer erase unavailable coverage. I also repaired the two stale segment_scope_roots call signatures that would fail compilation and moved the import into the normal import block. Added focused Rust regressions for budget exhaustion/cancellation, offline sharing, lower-bound rendering/JSON, and unavailable page merging. Pinned rustfmt, diff checks, and commitlint pass; native Bazel validation is pending CI.

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed f162dfb to fix the concrete compile failures from CI run 39559627882 (Clippy job 119100236502 and native job 119100236798).

The remaining retention tests now pass the shared entry budget and assert cancellation/exhaustion behavior. The removed dir_size_bytes re-export and its live CLI caller are migrated to the canonical bounded walker; list --all shares one existing entry ceiling across registered and orphan stores, and partial/unreadable measurements remain unknown instead of contributing an exact total. Only a confirmed missing root reports zero. A failed root inspection no longer becomes an empty store through Path::exists(). The stored footprint callback also has the explicit types required by its FnMut interface.

Added focused real-filesystem regressions for shared budgets, incomplete/non-directory measurements, and the exists() == false/metadata-error case, and strengthened the retention budget assertions. Pinned rustfmt, git diff --check, and commitlint pass. Native compilation and the new Rust regressions are left to fresh Bazel CI; these checks have not been run locally. No entry ceiling, deadline, or assertion was weakened.

@ScriptedAlchemy
ScriptedAlchemy enabled auto-merge (squash) October 22, 2026 00:36
@ScriptedAlchemy
ScriptedAlchemy enabled auto-merge (squash) October 22, 2026 02:26
@ScriptedAlchemy
ScriptedAlchemy merged commit d80eef0 into master Oct 22, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

simplify(maintenance): unify storage-report sizing on a bounded, coverage-aware filesystem walk

2 participants