[minatoaquaMK2]

Why

Issue #300 asks for a tenant's data disk that outlives a claim, has one writer at a time, and survives hibernate. This PR replaces #368 with our current version. A tenant can create a disk on a node, attach it to a warm claim, keep it through release and restarts, hibernate and wake with it mounted, and grow it while the VM runs.

What changed

• A registry under managed_volume_dir records each disk (owner, size, file identity) and which VM holds it (boot ID, PID, start time). A hold is freed only after the VM and its process are gone, so a crash or a slow vm rm cannot hand the disk to a second writer.
• New routes PUT/GET/DELETE /v1/volumes/{name} and POST /v1/volumes/{name}/grow, with Go and Python SDK methods. Hibernate of a claim with one mounted writable managed disk runs a guest hook to stop writers, unmounts, detaches and captures. Wake does the reverse.
• We re-checked every edge case feat(sandboxd): reconcile live read-only volume attachments #339 handled (busy, bind and stacked mounts, a release racing a change, restarts mid-change, error codes, Retry-After, journal rules) against each managed path. Each gap we found is fixed with a test that fails first.

Scope

• Covers create, claim-time attach, single writer, hibernate and wake, and online growth.
• Leaves out placement by volume, archive, and Kubernetes.
• Online growth needs cocoon vm disk grow and cocoon vm disk inspect --disk-id, which cocoon does not ship yet. On current cocoon, growth answers 503 volume_grow_unsupported. We can split growth into its own PR until cocoon has them.

Question for maintainers

After a restart, if one leftover VM (no claim, only catalog volumes attached) cannot be confirmed removed, startup marks managed storage uncertain. Every managed volume call then answers 503, and every claim that holds a volume is marked failed, until the next restart. Nothing retries that VM's removal, so its catalog volumes stay held even after it is gone.

• Option A: keep this. It is strictly fail-closed.
• Option B: queue the VM for retry like the pool: a stale VM's attached volumes stay held until it is removed #349 stale-VM sweep, keep its disks held until removal is confirmed, and fence managed storage only when the VM's process identity cannot be read.

We lean to B. Which do you prefer?

Tradeoffs

• Holds live in a second file, registry.json, beside claims.json, so a hibernated claim keeps its writer lock and a crash mid-attach cannot free a disk early. claims.json keeps its flat format. We can move holds into claims.json if you prefer that.
• The registry version goes from 1 to 3 on first hibernate or grow. We can collapse it to one version before merge.

Blast Radius

Nodes without managed_volume_dir keep their behavior, with two shared fixes. A guest mount conflict or a too-old cocoon on hibernate or wake now answers 409 or 503 instead of 500. The Python SDK now rejects a bare string or dict as a claim's volume list.

Verification

• go test -race for sandboxd and e2e pass, except two tests that also fail on main when run as root. golangci-lint passes on linux and darwin, and ruff, pytest and mypy pass.
• Each gap fix is a failing-test commit followed by its fix. We checked out each pair and saw red, then green.
• Real-VM runs for this revision are still pending.

[minatoaquaMK2]

Pushed follow-ups from a second pass over the cases #339 covered. Each behavior fix is a failing test commit followed by its fix.

• Release no longer pops a mount stacked on top of a writable volume. UnmountVolume now takes the volume name and refuses unless the volume's own device is the mount, so the dirty marker stays when the volume was not really unmounted. This also changes release for catalog writable volumes, which had the same bug on main.
• Hibernate and wake no longer fsync an unchanged claims.json.
• A managed directory renamed or missing for a moment is now a retryable storage error everywhere (list, GET, claim, DELETE), not a permanent recovery_required.
• The Python SDK rejects a bare dict in set_volumes and patch_volumes, like claims do.
• Fixed the test helper behind the hang in TestManagedGrowthAdmissionSerializesOrdinaryReleaseAndReap (80 runs in a row pass).

Three more questions, besides the one in the description:

1. If the final registry write of a wake fails, we roll the sleep record back and keep the guest paused, then the retry runs the resume hook again. Rolling back keeps guest access from opening before the disk agrees, and hooks are required to be idempotent. Is that acceptable?
2. If claims.json is renamed but its directory fsync fails during a managed release, we fence all managed volumes until restart. We could instead lift the fence once the reap tick has durably rewritten the affected claims. Do you want that?
3. An interrupted create (backing file present, no recorded inode) shows creating in GET until an operator removes the file. Should GET show it as needing recovery instead?