Repository navigation
feat(antd-py): manifest API (REST + gRPC) - #356
Conversation
RestClient, AsyncRestClient, GrpcClient and AsyncGrpcClient gain antd's
ADR-0006 manifest surface with the same signatures, models and errors,
mirroring antd-go:
- build_manifest(entries, *, name=None, torrent_info_hashes=None,
embed_roots=True)
- decode_manifest(manifest, resolve=False)
- decode_manifest_link(link, resolve=False), for a manifest link, a file
link or a bare address
The models are frozen dataclasses: ManifestEntryInput (keyword-only;
data_map or address, optional path and size), ManifestEntryInfo,
BuiltManifest and DecodedManifest. Build options are keyword arguments,
and embed_roots defaults to True, the daemon's own default; it is always
sent explicitly. kind and source stay plain strings, as in antd-go. The
shared mapping lives in a new _manifests module, so a REST-only install
still never imports grpc.
Bytes: the public API takes and returns raw .ant bytes. REST carries
them as standard base64 in JSON, encoded on decode requests and decoded
strictly (validate=True) from build responses, so a malformed manifest
field raises InternalError instead of yielding the wrong bytes. gRPC
carries them as raw bytes.
Sizes are Python ints, the type the SDK already uses for file sizes.
Unset reads as None, never 0, on both transports. REST omits the field,
and a value that is not a JSON integer in u64 range also reads as None.
gRPC reads the proto3 optional fields by presence (HasField), so a
recorded 0 stays 0. Unset strings (path, name, link, data_map, error,
torrent hashes) read as None too, matching the pointer models.
gRPC limits: a manifest can be 64 MiB, and both responses carry it.
Python gRPC has no per-call message limit, so each gRPC client opens a
second channel to the same target, with grpc.max_receive_message_length
and grpc.max_send_message_length at 256 MiB (antd-go's
maxManifestGrpcMessage), and puts only the ManifestService stub on it.
Every other service keeps the defaults, and the channel connects on
first use. close() closes both.
Errors: no new classes or codes. An entry with both or neither of
data_map and address, or a size that is not a u64, is refused before
sending, as antd-go's checkManifestEntries does. That raises
BadRequestError with status_code 0 and the daemon's own wording
("entries[i]: give data_map or address, not both"), so one except covers
it and the daemon's refusals. The daemon's 400 BAD_REQUEST and gRPC
INVALID_ARGUMENT map to BadRequestError with the server's message, which
names entries[i]. An empty manifest or link is refused the same way, and
a str passed as manifest bytes (or bytes as a link) raises TypeError. One
entry's resolve failure stays in that entry's error. Nothing is uploaded
or paid for, so every call is safe to retry; the README says so.
Stubs: manifests added to scripts/generate_proto.py and regenerated with
the pinned grpcio-tools 1.80.0 (protobuf gencode 6.31.1). The existing
stubs regenerate byte-identically.
Tests: 72 new unit tests over a mock REST daemon and an in-process gRPC
ManifestService, sync and async. They cover the request shapes, the
base64 round trip (standard alphabet, strict decoding), None versus 0
for size and exact_size both ways, the both-or-neither and size checks
before sending, the embed_roots default, and the 400 / INVALID_ARGUMENT
mapping. A 5 MiB manifest is received and sent through the manifest
channel, while a default channel refuses the same response with
RESOURCE_EXHAUSTED. The suite is 435 passed, 11 skipped.
tests/test_live_manifests.py is gated on ANTD_TEST_URL and
ANTD_TEST_MANIFEST_ADDRESSES (gRPC also on ANTD_TEST_GRPC). It makes no
paid call, and passed over all four clients against a 10-node devnet:
- a build from the two public fixtures and the private DataMap: kinds,
effective names, encoded order and a non-empty link
- decodes of the bytes and of the link with resolve: exact sizes match
the fixtures, and the DataMap entry is embedded with its data_map
- the fixture manifest link, and a bare address as a file link
- client-side and daemon-side BadRequestError (an invalid path, ../x)
- identical bytes from REST and gRPC
examples/10_manifests.py (optional ANTD_BASE_URL) stores two notes,
builds a manifest named example from notes/one.txt and notes/two.txt,
prints its link and entries, then decodes the link with resolve and
prints each exact size (26 and 43 bytes) against the same devnet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A mistake the client catches before sending is now indistinguishable from the same mistake caught by the daemon: BadRequestError, status_code 400, and the daemon's wording. - build_manifest reports every bad entry together, as the daemon's parse_entries does: "N invalid entries: entries[i]: give data_map or address, not both; entries[j]: needs a data_map or an address". At most 20 are shown, then "; and K more". An empty string counts as unset. A size outside u64 is reported the same way, as "entries[i]: size must be ...". - An empty manifest or link to decode raises BadRequestError with "needs a manifest (base64 .ant bytes) or a link". - status_code was 0 and is now 400. Response-parsing failures (bad base64 from the daemon) are unchanged and still raise InternalError. All four clients share the checks in _manifests, so they behave the same. Tests: the unit tests assert the type, the status and the exact message, with new cases for two bad entries and for more than 20. The suite is 438 passed, 11 skipped. The live test passed over all four clients, and now sends a both-sources and a neither-source entry past the client check to the daemon and compares its message with the client's: they are equal after the daemon's "Bad request: " prefix. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
dirvine
left a comment
There was a problem hiding this comment.
Formal review: APPROVE
Repository: WithAutonomi/ant-sdk, PR #356.
Reviewed head: 15ed5d175c1e93339a642facb84e04919abc836b. Original merge base: 6c907eda10948c37cd92dd921d9dd5f68bd3e817.
No material blocker remains in the Python manifest API. The strongest objection assumed protojson encoding for a Serde REST DTO; the actual daemon emits numeric manifest sizes. The partial panel returned 8/12 substantive reviews (4 APPROVE, 4 UNCERTAIN). The complete-source and runtime checks below resolve the material uncertainty without pretending that the original seats voted unanimously.
Source and test coverage
- Complete Python manifest module and REST/gRPC adapters, tests and example against the daemon's actual Serde DTOs and protobuf contract.
- Parent reran PYTHONPATH=src python -m pytest tests -q at the frozen #356 checkout: 438 passed, 11 skipped, exit 0. The imported antd package resolved to this checkout, despite using an existing venv from #335. ANTD_TEST* flags were removed.
Findings and adjudication
- ManifestEntryInfo.size and exact_size are Option in antd/src/types.rs:909-934 and are served through axum Json responses. They are REST JSON numbers, not decimal strings merely because protojson would encode uint64 that way. Python's _opt_u64 matches the actual contract (_manifests.py:163-166).
- The full _manifests.py module exists at the reviewed head. REST and gRPC call check_entries before sending, so the both/neither-source validation is not bypassed by the later wire-builder branch.
- Manifest byte conversion is strict base64 for REST; malformed data raises InternalError instead of silently accepting the wrong bytes (_manifests.py:173-183). Absent optional sizes remain None rather than zero.
Panel dissent and its disposition
- The decimal-string size objection is refuted by the daemon's REST serialization layer, not by assuming the SDK is authoritative. The missing-file objections were evidence-scope artifacts; the other scope and the checkout contain the implementation.
- The borrowed-venv concern was checked with an actual import-path probe and a fresh exact-checkout unit-suite run. It did not silently execute the #335 package.
Review team: actual participation
8/12 scope/role/model seats returned substantive, receipt-validated reviews; 3/3 distinct named models. Partial panel; provisional coverage limits apply to this formal verdict. A receipt is evidence of participation, not proof that its findings are correct. Counts below are per evidence scope, not additional independent models.
source-1: glm: correctness=APPROVE, adversarial=UNCERTAIN; qwen: correctness=APPROVE, adversarial=APPROVE; ds4: correctness=unavailable (Incomplete termination; unavailable seat), adversarial=unavailable (Incomplete termination; unavailable seat).source-2: glm: correctness=unavailable (Incomplete termination; unavailable seat), adversarial=UNCERTAIN; qwen: correctness=UNCERTAIN, adversarial=APPROVE; ds4: correctness=UNCERTAIN, adversarial=unavailable (Incomplete termination; unavailable seat).- Models: GLM 5.3 / OpenRouter z-ai/glm-5.3; local Qwen 3.8 / qwen38-27b; local DS4 Flash / deepseek-v4-flash. Missing seats received the configured same-model retry; no replacement model is counted as an independent vote.
Publication gates and limits
- Live head rechecked against the reviewed head. Live base
ea389c79c2ac40601c574e93031a66a9791db1d2: binary diff matches the reviewed diff, merge base is unchanged, no intersecting base-drift paths, and trial merge exited 0. - GitHub check snapshot: SUCCESS=9.
- Partial panel; all three named models contributed, but four scope/role/model seats remained unavailable after retry. No full-panel consensus is claimed.
- A broader bare pytest -q invocation also collected six unchanged CLI-reporting helpers under scripts/ as tests and failed setup with fixture 'name' not found. That failed invocation is retained separately; it is not a passing run. The explicit tests/ suite then passed as stated above. These script files are unchanged by this PR.
- The 11 skipped tests were not exercised. No live manifest network resolve or daemon integration run was performed.
- No paid or live-network tests. No merge, deployment, source change, or Linear state change. This verdict applies only to the exact head above; it is not a fleet-wide or cross-SDK integration sign-off.
Part of the manifest API fan-out. Stacked on #346 (the manifest endpoints in antd and antd-go), which sits on the pointer API #326; review those first.
RestClient, AsyncRestClient, GrpcClient and AsyncGrpcClient gain antd's
ADR-0006 manifest surface with the same signatures, models and errors,
mirroring antd-go:
embed_roots=True)
link or a bare address
The models are frozen dataclasses: ManifestEntryInput (keyword-only;
data_map or address, optional path and size), ManifestEntryInfo,
BuiltManifest and DecodedManifest. Build options are keyword arguments,
and embed_roots defaults to True, the daemon's own default; it is always
sent explicitly. kind and source stay plain strings, as in antd-go. The
shared mapping lives in a new _manifests module, so a REST-only install
still never imports grpc.
Bytes: the public API takes and returns raw .ant bytes. REST carries
them as standard base64 in JSON, encoded on decode requests and decoded
strictly (validate=True) from build responses, so a malformed manifest
field raises InternalError instead of yielding the wrong bytes. gRPC
carries them as raw bytes.
Sizes are Python ints, the type the SDK already uses for file sizes.
Unset reads as None, never 0, on both transports. REST omits the field,
and a value that is not a JSON integer in u64 range also reads as None.
gRPC reads the proto3 optional fields by presence (HasField), so a
recorded 0 stays 0. Unset strings (path, name, link, data_map, error,
torrent hashes) read as None too, matching the pointer models.
gRPC limits: a manifest can be 64 MiB, and both responses carry it.
Python gRPC has no per-call message limit, so each gRPC client opens a
second channel to the same target, with grpc.max_receive_message_length
and grpc.max_send_message_length at 256 MiB (antd-go's
maxManifestGrpcMessage), and puts only the ManifestService stub on it.
Every other service keeps the defaults, and the channel connects on
first use. close() closes both.
Errors: no new classes or codes. An entry with both or neither of
data_map and address, or a size that is not a u64, is refused before
sending, as antd-go's checkManifestEntries does. That raises
BadRequestError with status_code 0 and the daemon's own wording
("entries[i]: give data_map or address, not both"), so one except covers
it and the daemon's refusals. The daemon's 400 BAD_REQUEST and gRPC
INVALID_ARGUMENT map to BadRequestError with the server's message, which
names entries[i]. An empty manifest or link is refused the same way, and
a str passed as manifest bytes (or bytes as a link) raises TypeError. One
entry's resolve failure stays in that entry's error. Nothing is uploaded
or paid for, so every call is safe to retry; the README says so.
Stubs: manifests added to scripts/generate_proto.py and regenerated with
the pinned grpcio-tools 1.80.0 (protobuf gencode 6.31.1). The existing
stubs regenerate byte-identically.
Tests: 72 new unit tests over a mock REST daemon and an in-process gRPC
ManifestService, sync and async. They cover the request shapes, the
base64 round trip (standard alphabet, strict decoding), None versus 0
for size and exact_size both ways, the both-or-neither and size checks
before sending, the embed_roots default, and the 400 / INVALID_ARGUMENT
mapping. A 5 MiB manifest is received and sent through the manifest
channel, while a default channel refuses the same response with
RESOURCE_EXHAUSTED. The suite is 435 passed, 11 skipped.
tests/test_live_manifests.py is gated on ANTD_TEST_URL and
ANTD_TEST_MANIFEST_ADDRESSES (gRPC also on ANTD_TEST_GRPC). It makes no
paid call, and passed over all four clients against a 10-node devnet:
effective names, encoded order and a non-empty link
the fixtures, and the DataMap entry is embedded with its data_map
examples/10_manifests.py (optional ANTD_BASE_URL) stores two notes,
builds a manifest named example from notes/one.txt and notes/two.txt,
prints its link and entries, then decodes the link with resolve and
prints each exact size (26 and 43 bytes) against the same devnet.
Overlap with this SDK's pointer PR #335. A trial merge conflicts only in: antd-py/scripts/generate_proto.py, antd-py/src/antd/_grpc.py. Resolve by keeping both sides when the two land.
Version bumps happen at release time; nothing here bumps a package version.
🤖 Generated with Claude Code