Skip to content

feat(api): stream a track's stems as one zip - #1060

Merged
dylanjeffers merged 1 commit into
mainfrom
feat/stems-zip-download
Oct 23, 2026
Merged

dylanjeffers merged 1 commit into
mainfrom
feat/stems-zip-download

Conversation

@dylanjeffers

@dylanjeffers dylanjeffers commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

What

GET /v1/tracks/{id}/stems/download streams every stem the requester may download, plus the parent with include_parent=true, as one zip. It is the replacement for the archiver service (BullMQ + Redis + disk). The clients move over in AudiusProject/apps#14703, which depends on this.

  • Entries are stored uncompressed, so the zip's length is computed before any audio is fetched. The response carries a real Content-Length and the browser's download manager shows progress.
  • Access uses the same rules as /tracks/{id}/download: gated stems need access, and an owner or manager signature gets their own non-downloadable files via ownerDownloadLink.
  • Files with no download link or no blob info are left out instead of failing the archive. 404 if nothing is left, 403 if everything was gated.
  • Each file streams from the signed content-node URL. If a node errors, drops the connection or sends nothing for 30s, the file resumes on the next mirror with Range: bytes=<offset>-.
  • At most 32 concurrent zips per pod and 2 per user (by user id, else IP). Past that it returns 429.
  • X-Accel-Buffering: no so ingress-nginx doesn't spool large zips to disk.
  • inspectCid is split out of inspectTrack and now has a 10s timeout. It had none.

Testing

  • New tests in v1_track_stems_download_test.go, against a fake content node: exact length vs. bytes written with CRCs verified by archive/zip, resuming on a mirror after the primary drops mid-file, all mirrors failing, zip64 offsets past 4 GiB, filename dedupe and extension fallback, the handler end to end (parent + stems, without parent, missing blob, not found), and an owner getting a non-downloadable stem that a stranger doesn't.
  • Smoke-tested writeStemsZip against prod content nodes with links signed by api.audius.co: 3 of Andrew Lux's "Alone" stems streamed into a 155 MB zip in 19s, length matched the precomputed value exactly, all CRCs ok. A Range request against a real node (through its 302) also worked.
  • Full ./api/... suite: TestFeedScheduledReleases also fails on main. TestSearch needs Elasticsearch locally and wasn't run.

Rollout

Additive, and no client calls it until the apps PR ships. Leave the archiver and the /archive/* proxy running until older mobile store builds and desktop versions stop using them.

🤖 Generated with Claude Code

GET /v1/tracks/{id}/stems/download streams every stem the requester may
download, plus the parent with include_parent=true, as an uncompressed zip.
The zip length is computed up front so the browser shows real progress, and
there is no job, queue or disk involved. Replaces the archiver service.

A node that drops or stalls mid-file is resumed on the next mirror with a
Range request. Files that can't be downloaded are left out. Downloads are
capped per pod and per user.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dylanjeffers
dylanjeffers merged commit 982cf2d into main Oct 23, 2026
2 checks passed
@dylanjeffers
dylanjeffers deleted the feat/stems-zip-download branch October 23, 2026 20:38
dylanjeffers added a commit that referenced this pull request Oct 23, 2026
Follow-up to #1060, found validating it in prod.

Stems indexed with no `orig_filename` often have the upload's filename
as their title (`" 🍕DRUMS.wav"`). The zip's fallback took
`createFilename`'s `title + ".mp3"`, stripped the `.mp3`, and added the
extension from the blob's content type. Andrew Lux's "Alone" zip came
out with `DRUMS.wav.wav`, `LEADS.wav.wav` and so on, 11 of 12 entries.
Now a trailing audio extension in the title is replaced instead of kept.
Tests cover the emoji/whitespace title, case, a different content type,
and a title with a dot but no extension.

Not changed here: the single-file `/tracks/{id}/download` names the same
files `DRUMS.wav.mp3` even though it serves the WAV, because
`createFilename` assumes the transcode whenever `orig_filename` is
empty.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant