Skip to content

feat(antd-py): manifest API (REST + gRPC) - #356

Merged
Nic-dorman merged 3 commits into
mainfrom
nic/v2-1580-manifests-py
Oct 19, 2026
Merged

Nic-dorman merged 3 commits into
mainfrom
nic/v2-1580-manifests-py

Conversation

@Nic-dorman

@Nic-dorman Nic-dorman commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

Part of the manifest API fan-out. Stacked on #346 (the manifest endpoints in antd and antd-go), which sits on the pointer API #326; review those first.

RestClient, AsyncRestClient, GrpcClient and AsyncGrpcClient gain antd's
ADR-0006 manifest surface with the same signatures, models and errors,
mirroring antd-go:

  • build_manifest(entries, *, name=None, torrent_info_hashes=None,
    embed_roots=True)
  • decode_manifest(manifest, resolve=False)
  • decode_manifest_link(link, resolve=False), for a manifest link, a file
    link or a bare address

The models are frozen dataclasses: ManifestEntryInput (keyword-only;
data_map or address, optional path and size), ManifestEntryInfo,
BuiltManifest and DecodedManifest. Build options are keyword arguments,
and embed_roots defaults to True, the daemon's own default; it is always
sent explicitly. kind and source stay plain strings, as in antd-go. The
shared mapping lives in a new _manifests module, so a REST-only install
still never imports grpc.

Bytes: the public API takes and returns raw .ant bytes. REST carries
them as standard base64 in JSON, encoded on decode requests and decoded
strictly (validate=True) from build responses, so a malformed manifest
field raises InternalError instead of yielding the wrong bytes. gRPC
carries them as raw bytes.

Sizes are Python ints, the type the SDK already uses for file sizes.
Unset reads as None, never 0, on both transports. REST omits the field,
and a value that is not a JSON integer in u64 range also reads as None.
gRPC reads the proto3 optional fields by presence (HasField), so a
recorded 0 stays 0. Unset strings (path, name, link, data_map, error,
torrent hashes) read as None too, matching the pointer models.

gRPC limits: a manifest can be 64 MiB, and both responses carry it.
Python gRPC has no per-call message limit, so each gRPC client opens a
second channel to the same target, with grpc.max_receive_message_length
and grpc.max_send_message_length at 256 MiB (antd-go's
maxManifestGrpcMessage), and puts only the ManifestService stub on it.
Every other service keeps the defaults, and the channel connects on
first use. close() closes both.

Errors: no new classes or codes. An entry with both or neither of
data_map and address, or a size that is not a u64, is refused before
sending, as antd-go's checkManifestEntries does. That raises
BadRequestError with status_code 0 and the daemon's own wording
("entries[i]: give data_map or address, not both"), so one except covers
it and the daemon's refusals. The daemon's 400 BAD_REQUEST and gRPC
INVALID_ARGUMENT map to BadRequestError with the server's message, which
names entries[i]. An empty manifest or link is refused the same way, and
a str passed as manifest bytes (or bytes as a link) raises TypeError. One
entry's resolve failure stays in that entry's error. Nothing is uploaded
or paid for, so every call is safe to retry; the README says so.

Stubs: manifests added to scripts/generate_proto.py and regenerated with
the pinned grpcio-tools 1.80.0 (protobuf gencode 6.31.1). The existing
stubs regenerate byte-identically.

Tests: 72 new unit tests over a mock REST daemon and an in-process gRPC
ManifestService, sync and async. They cover the request shapes, the
base64 round trip (standard alphabet, strict decoding), None versus 0
for size and exact_size both ways, the both-or-neither and size checks
before sending, the embed_roots default, and the 400 / INVALID_ARGUMENT
mapping. A 5 MiB manifest is received and sent through the manifest
channel, while a default channel refuses the same response with
RESOURCE_EXHAUSTED. The suite is 435 passed, 11 skipped.

tests/test_live_manifests.py is gated on ANTD_TEST_URL and
ANTD_TEST_MANIFEST_ADDRESSES (gRPC also on ANTD_TEST_GRPC). It makes no
paid call, and passed over all four clients against a 10-node devnet:

  • a build from the two public fixtures and the private DataMap: kinds,
    effective names, encoded order and a non-empty link
  • decodes of the bytes and of the link with resolve: exact sizes match
    the fixtures, and the DataMap entry is embedded with its data_map
  • the fixture manifest link, and a bare address as a file link
  • client-side and daemon-side BadRequestError (an invalid path, ../x)
  • identical bytes from REST and gRPC

examples/10_manifests.py (optional ANTD_BASE_URL) stores two notes,
builds a manifest named example from notes/one.txt and notes/two.txt,
prints its link and entries, then decodes the link with resolve and
prints each exact size (26 and 43 bytes) against the same devnet.

Overlap with this SDK's pointer PR #335. A trial merge conflicts only in: antd-py/scripts/generate_proto.py, antd-py/src/antd/_grpc.py. Resolve by keeping both sides when the two land.

Version bumps happen at release time; nothing here bumps a package version.

🤖 Generated with Claude Code

RestClient, AsyncRestClient, GrpcClient and AsyncGrpcClient gain antd's
ADR-0006 manifest surface with the same signatures, models and errors,
mirroring antd-go:
- build_manifest(entries, *, name=None, torrent_info_hashes=None,
  embed_roots=True)
- decode_manifest(manifest, resolve=False)
- decode_manifest_link(link, resolve=False), for a manifest link, a file
  link or a bare address

The models are frozen dataclasses: ManifestEntryInput (keyword-only;
data_map or address, optional path and size), ManifestEntryInfo,
BuiltManifest and DecodedManifest. Build options are keyword arguments,
and embed_roots defaults to True, the daemon's own default; it is always
sent explicitly. kind and source stay plain strings, as in antd-go. The
shared mapping lives in a new _manifests module, so a REST-only install
still never imports grpc.

Bytes: the public API takes and returns raw .ant bytes. REST carries
them as standard base64 in JSON, encoded on decode requests and decoded
strictly (validate=True) from build responses, so a malformed manifest
field raises InternalError instead of yielding the wrong bytes. gRPC
carries them as raw bytes.

Sizes are Python ints, the type the SDK already uses for file sizes.
Unset reads as None, never 0, on both transports. REST omits the field,
and a value that is not a JSON integer in u64 range also reads as None.
gRPC reads the proto3 optional fields by presence (HasField), so a
recorded 0 stays 0. Unset strings (path, name, link, data_map, error,
torrent hashes) read as None too, matching the pointer models.

gRPC limits: a manifest can be 64 MiB, and both responses carry it.
Python gRPC has no per-call message limit, so each gRPC client opens a
second channel to the same target, with grpc.max_receive_message_length
and grpc.max_send_message_length at 256 MiB (antd-go's
maxManifestGrpcMessage), and puts only the ManifestService stub on it.
Every other service keeps the defaults, and the channel connects on
first use. close() closes both.

Errors: no new classes or codes. An entry with both or neither of
data_map and address, or a size that is not a u64, is refused before
sending, as antd-go's checkManifestEntries does. That raises
BadRequestError with status_code 0 and the daemon's own wording
("entries[i]: give data_map or address, not both"), so one except covers
it and the daemon's refusals. The daemon's 400 BAD_REQUEST and gRPC
INVALID_ARGUMENT map to BadRequestError with the server's message, which
names entries[i]. An empty manifest or link is refused the same way, and
a str passed as manifest bytes (or bytes as a link) raises TypeError. One
entry's resolve failure stays in that entry's error. Nothing is uploaded
or paid for, so every call is safe to retry; the README says so.

Stubs: manifests added to scripts/generate_proto.py and regenerated with
the pinned grpcio-tools 1.80.0 (protobuf gencode 6.31.1). The existing
stubs regenerate byte-identically.

Tests: 72 new unit tests over a mock REST daemon and an in-process gRPC
ManifestService, sync and async. They cover the request shapes, the
base64 round trip (standard alphabet, strict decoding), None versus 0
for size and exact_size both ways, the both-or-neither and size checks
before sending, the embed_roots default, and the 400 / INVALID_ARGUMENT
mapping. A 5 MiB manifest is received and sent through the manifest
channel, while a default channel refuses the same response with
RESOURCE_EXHAUSTED. The suite is 435 passed, 11 skipped.

tests/test_live_manifests.py is gated on ANTD_TEST_URL and
ANTD_TEST_MANIFEST_ADDRESSES (gRPC also on ANTD_TEST_GRPC). It makes no
paid call, and passed over all four clients against a 10-node devnet:
- a build from the two public fixtures and the private DataMap: kinds,
  effective names, encoded order and a non-empty link
- decodes of the bytes and of the link with resolve: exact sizes match
  the fixtures, and the DataMap entry is embedded with its data_map
- the fixture manifest link, and a bare address as a file link
- client-side and daemon-side BadRequestError (an invalid path, ../x)
- identical bytes from REST and gRPC

examples/10_manifests.py (optional ANTD_BASE_URL) stores two notes,
builds a manifest named example from notes/one.txt and notes/two.txt,
prints its link and entries, then decodes the link with resolve and
prints each exact size (26 and 43 bytes) against the same devnet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A mistake the client catches before sending is now indistinguishable
from the same mistake caught by the daemon: BadRequestError, status_code
400, and the daemon's wording.

- build_manifest reports every bad entry together, as the daemon's
  parse_entries does: "N invalid entries: entries[i]: give data_map or
  address, not both; entries[j]: needs a data_map or an address". At most
  20 are shown, then "; and K more". An empty string counts as unset. A
  size outside u64 is reported the same way, as "entries[i]: size must
  be ...".
- An empty manifest or link to decode raises BadRequestError with "needs
  a manifest (base64 .ant bytes) or a link".
- status_code was 0 and is now 400. Response-parsing failures (bad base64
  from the daemon) are unchanged and still raise InternalError.

All four clients share the checks in _manifests, so they behave the same.

Tests: the unit tests assert the type, the status and the exact message,
with new cases for two bad entries and for more than 20. The suite is 438
passed, 11 skipped. The live test passed over all four clients, and now
sends a both-sources and a neither-source entry past the client check to
the daemon and compares its message with the client's: they are equal
after the daemon's "Bad request: " prefix.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@Nic-dorman
Nic-dorman marked this pull request as ready for review October 16, 2026 09:07
@Nic-dorman
Nic-dorman changed the base branch from nic/v2-1570-antd-manifest-endpoints to main October 16, 2026 14:34

@dirvine dirvine left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formal review: APPROVE

Repository: WithAutonomi/ant-sdk, PR #356.
Reviewed head: 15ed5d175c1e93339a642facb84e04919abc836b. Original merge base: 6c907eda10948c37cd92dd921d9dd5f68bd3e817.

No material blocker remains in the Python manifest API. The strongest objection assumed protojson encoding for a Serde REST DTO; the actual daemon emits numeric manifest sizes. The partial panel returned 8/12 substantive reviews (4 APPROVE, 4 UNCERTAIN). The complete-source and runtime checks below resolve the material uncertainty without pretending that the original seats voted unanimously.

Source and test coverage

  • Complete Python manifest module and REST/gRPC adapters, tests and example against the daemon's actual Serde DTOs and protobuf contract.
  • Parent reran PYTHONPATH=src python -m pytest tests -q at the frozen #356 checkout: 438 passed, 11 skipped, exit 0. The imported antd package resolved to this checkout, despite using an existing venv from #335. ANTD_TEST* flags were removed.

Findings and adjudication

  • ManifestEntryInfo.size and exact_size are Option in antd/src/types.rs:909-934 and are served through axum Json responses. They are REST JSON numbers, not decimal strings merely because protojson would encode uint64 that way. Python's _opt_u64 matches the actual contract (_manifests.py:163-166).
  • The full _manifests.py module exists at the reviewed head. REST and gRPC call check_entries before sending, so the both/neither-source validation is not bypassed by the later wire-builder branch.
  • Manifest byte conversion is strict base64 for REST; malformed data raises InternalError instead of silently accepting the wrong bytes (_manifests.py:173-183). Absent optional sizes remain None rather than zero.

Panel dissent and its disposition

  • The decimal-string size objection is refuted by the daemon's REST serialization layer, not by assuming the SDK is authoritative. The missing-file objections were evidence-scope artifacts; the other scope and the checkout contain the implementation.
  • The borrowed-venv concern was checked with an actual import-path probe and a fresh exact-checkout unit-suite run. It did not silently execute the #335 package.

Review team: actual participation

8/12 scope/role/model seats returned substantive, receipt-validated reviews; 3/3 distinct named models. Partial panel; provisional coverage limits apply to this formal verdict. A receipt is evidence of participation, not proof that its findings are correct. Counts below are per evidence scope, not additional independent models.

  • source-1: glm: correctness=APPROVE, adversarial=UNCERTAIN; qwen: correctness=APPROVE, adversarial=APPROVE; ds4: correctness=unavailable (Incomplete termination; unavailable seat), adversarial=unavailable (Incomplete termination; unavailable seat).
  • source-2: glm: correctness=unavailable (Incomplete termination; unavailable seat), adversarial=UNCERTAIN; qwen: correctness=UNCERTAIN, adversarial=APPROVE; ds4: correctness=UNCERTAIN, adversarial=unavailable (Incomplete termination; unavailable seat).
  • Models: GLM 5.3 / OpenRouter z-ai/glm-5.3; local Qwen 3.8 / qwen38-27b; local DS4 Flash / deepseek-v4-flash. Missing seats received the configured same-model retry; no replacement model is counted as an independent vote.

Publication gates and limits

  • Live head rechecked against the reviewed head. Live base ea389c79c2ac40601c574e93031a66a9791db1d2: binary diff matches the reviewed diff, merge base is unchanged, no intersecting base-drift paths, and trial merge exited 0.
  • GitHub check snapshot: SUCCESS=9.
  • Partial panel; all three named models contributed, but four scope/role/model seats remained unavailable after retry. No full-panel consensus is claimed.
  • A broader bare pytest -q invocation also collected six unchanged CLI-reporting helpers under scripts/ as tests and failed setup with fixture 'name' not found. That failed invocation is retained separately; it is not a passing run. The explicit tests/ suite then passed as stated above. These script files are unchanged by this PR.
  • The 11 skipped tests were not exercised. No live manifest network resolve or daemon integration run was performed.
  • No paid or live-network tests. No merge, deployment, source change, or Linear state change. This verdict applies only to the exact head above; it is not a fleet-wide or cross-SDK integration sign-off.

main now has the pointer fan-out. Conflicts resolved exactly as in the
integration preview #367 (bc339ab): antd-py/scripts/generate_proto.py antd-py/src/antd/_grpc.py
The result matches #367 for antd-py byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Nic-dorman
Nic-dorman merged commit cd0c9ee into main Oct 19, 2026
9 checks passed
Nic-dorman added a commit that referenced this pull request Oct 19, 2026
Now that the py manifest API (#356) is on main. Conflicts in python-ci.yml and antd-mcp/README.md resolved as on the #367 preview (bc339ab), with the MCP tool-count tests from there (31 tools, 21 error codes).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants