feat(sweep): ✨ SRC sweep over the GPUs of a node

[robertodr]

🤖 AI text below 🤖

Phase 2 of running SRC on stacks too large for one GPU: split the sketch index among the GPUs of a node, so that the Phase 1 reference problem finishes close to G times faster on G GPUs. The mathematics, the random draws and the result stay the same.

Stacked on #42. This draft holds the design and the implementation plan so far. The code will follow on this branch.

Design

Spec: docs/superpowers/specs/2026-09-25-src-multi-gpu-design.md

Question	Decision
Execution model	one process per GPU (SPMD), launched with srun or mpirun
Collectives	NCCL through cupy.cuda.nccl; mpi4py for bootstrap
Public interface	an mpi4py communicator in Resources(comm=...); nothing is collective without it
Output	rank 0 in memory by default; with Resources(output_dir=...), .npy files and memmaps on every rank
Large inputs	file-backed, shared among the ranks of a node through the page cache
Success criterion	strong scaling: at least 80% parallel efficiency at 8 GPUs on 8xA100 and 8xH100 at the reference size; at least 90% on 2xA100

• Communicator: a small interface in _comm.py with three implementations:
  • single process;
  • mpi4py on host arrays, so CI can test the distributed logic without GPUs;
  • NCCL.
• Driver: each rank owns a cyclic subset of the sketch columns. Per site of the right-to-left pass it does two all-gathers, one of the sketch columns and one of the rows of the projected environment. The QR is repeated on every rank.
• Errors: errors before the sweep starts are raised on every rank; an error during the sweep aborts the job.
• Future extensions: the spec records GPUDirect Storage and compression as options for later.

Status

• Design (spec)
• Implementation plan (docs/superpowers/plans/2026-09-25-src-multi-gpu.md)
• Implementation, with unit tests and MPI tests on the CPU
• Multi-GPU tests and cluster scaling runs