Repository navigation
fix: pay and finalize multi-batch merkle uploads (antd >= 0.12.0) - #162
Conversation
Uploads larger than one merkle tree (256 fresh chunks ~= 1 GiB) arrive
from antd >= 0.12.0 as multiple MerkleBatches with the legacy mirror
fields empty, so the worker submitted an empty commitments list and the
upload failed opaquely. The merkle branch now:
- plans batches from MerkleBatches (legacy single-batch fields remain as
the pre-0.12 fallback; both-empty errors with a clear version-gap
message) and validates every batch before any money moves
- pre-approves the summed per-batch max payout in one approve tx
(Signer.EnsureMerkleAllowance), then pays one payForMerkleTree tx per
batch, recording each confirmed spend before the next batch (V2-426;
one transactions row per batch, tx_hash = that batch's winner hash)
- finalizes via FinalizeMerkleUploadMulti with the index-aligned winner
list (single-batch keeps the legacy FinalizeMerkleUpload call)
- on a definitive mid-plan payment failure, salvage-finalizes the paid
batches ("" for unpaid) so their chunks store and a retry pays only
the remainder, then preserves the source (errPaidNoRetry); a
confirmation timeout skips salvage so the manual full finalize stays
possible
- sums estimatedUploadCost across all batches for the max_gas_fee check
V2-1056
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Found live on the multi-batch devnet e2e: batch 1's payForMerkleTree reverted with ERC20InsufficientAllowance because the allowance bound still used the V1-era "sum over pools of max candidate" formula. PaymentVaultV2 charges median16(winner pool quotes) * 2^depth and picks the winner pool at execution time (PaymentVaultV2.sol:140), so the correct worst case is the highest pool median shifted by depth, summed across batches. The single-batch path had the same latent under-approval bug; both bounds now share the contract-accurate formula, and the worker validates batch depth (1-8) before paying. V2-1056 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dirvine
left a comment
There was a problem hiding this comment.
Reviewed at exact head 85228779cc6701945fa57388636ff053e8334a2c.
The multi-batch flow is consistent with antd-go v0.12.0 and PaymentVaultV2: the allowance bound uses the upper median quote multiplied by 2^depth, payments are nonce-serialised, and the timeout/salvage branches avoid automatically re-paying an unconfirmed transaction. The single-batch compatibility path and multi-finalize ordering also match the SDK contract.
Verified locally:
go test ./... -count=1go test -race ./... -count=1go vet ./...- frontend
npm ci && npm run build git diff --check
Required exact-head CI is green (Docker, frontend, lint, smoke, PostgreSQL and SQLite).
Non-blocking follow-up: validateMerkleBatches does not yet fully enforce its “reject malformed data before any money moves” comment: it could also validate pool-hash syntax, the contract-required pool count, and non-negative uint256 candidate amounts. The later signer/contract checks plus partial-finalize path keep this from being a merge blocker, but validating those fields up front would make the stated invariant complete.
* chore(release): v0.13.0 — bundle antd v0.17.0 Changelog and upgrading notes for the 0.13.0 cut. The antd pins already moved on master (#166, #167, #169); this is the docs half: - upload payments: multi-batch merkle (#162), no requeue or second payment of a live upload (#170), partial-finalize resume (#171), and all-already-stored finalize (#172); - the requeue-window admin API, the behaviour changes (paid_unfinalized instead of an automatic second payment; 404 for not-found downloads), migration 015, licence files in releases (#165), Go 1.26.10, web advisories (#168), and the V2-1778 known issue. Also adds the missing [0.12.0] release link. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(release): state the crash-before-record gap in the re-payment guard The 0.13.0 notes described automatic re-payment as prevented outright. #170 leaves one gap: a crash after a payment confirms but before it is recorded leaves no ledger row, so the restarted worker can pay again for chunks not yet stored. Say so in the CHANGELOG and UPGRADING, with the reconciliation step, and drop the unqualified headlines (Hermes review on 0538f24). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Fixes multi-batch merkle uploads against antd >= 0.12.0 (V2-1056), plus a latent allowance-formula bug the e2e exposed.
Linear: https://linear.app/autonominetwork/issue/V2-1056/multi-batch-merkle-uploads-fail-merklebatches
What
antd >= 0.12.0 splits uploads larger than one merkle tree (256 fresh chunks ≈ 1 GiB) into multiple
MerkleBatchesand leaves the legacy mirror fields empty, so the worker previously submitted an empty commitments list and failed opaquely.merkleBatchPlan):MerkleBatcheswhen present (antd >= 0.12.0 always populates it for merkle, len >= 1), legacy single-batch synthesis as the pre-0.12 fallback, and a clear "needs antd >= 0.12.0" error when both are empty. All batches are validated (depth 1-8, 16 candidates/pool, parseable amounts) before any money moves.payMerkleBatches): onepayForMerkleTreetx per batch (no ABI change —payForMerkleTree2()in antd docs is a naming slip for PaymentVaultV2's function, confirmed against evmlib + ADR-0003). Each confirmed batch is recorded before the next is paid (V2-426): onetransactionsrow per batch,tx_hash= that batch's winner pool hash.Signer.EnsureMerkleAllowance): pre-approves the whole plan's worst case, so the per-batch approval short-circuits (1 approve instead of N).FinalizeMerkleUploadMultiwith the index-aligned winner list when > 1 batch; the single-batch path keeps the legacyFinalizeMerkleUploadcall unchanged.""for unpaid batches — the paid batches' chunks store (surfaced asPARTIAL_UPLOAD), then the upload fails preserve-paid (errPaidNoRetry). A confirmation timeout never salvages: the tx may still mine, and skipping keeps the manual full finalize possible (the error carries the pending tx hash + batch index). If salvage unexpectedly fully succeeds (unpaid batches deduped), the upload completes normally.median16(winner pool quotes) * 2^depthand picks the winner at execution time (PaymentVaultV2.sol:140), so batch 1 of the e2e reverted withERC20InsufficientAllowance. Both the allowance bound and themax_gas_feeestimate now use the contract-accurate worst case (highest pool median << depth, summed across batches). The single-batch path had the same latent under-approval bug.E2E proof (local devnet, 2026-08-24)
24-node devnet (merkle pools need 16 candidate peers; the 10-node preset can't quote) + anvil + antd v0.12.0, this branch's indelible:
batches=2(depth 8 + depth 5) → one approve tx → batch 1/2 paid 3.0 ANT, batch 2/2 paid 0.75 ANT (distinct winner hashes) →FinalizeMerkleUploadMulti→ completed, 291 chunks, downloaded byte-identical (cmpclean). DB: twotransactionsrows with the exact per-batch winner hashes/amounts;actual_cost= 3.75 ANT = sum.chunks_stored=256, chunks_failed=35, total_chunks=291→ uploadfailed/paid_unfinalized, temp source preserved, exactly one transaction row.Unit:
go test ./...green (batch planning/fallback/version-gap error, validation, pad/salvage decision, contract-formula payout bounds, classifyFailure rows for the new wrapping shapes).Notes for review
merkleAllowancePayeris an optional interface seam so the uncommitted hosted-payment work (payerinterface) merges without changes — payers lacking the method fall back to per-batch self-approval insidePayForMerkleTree.RequeueStuck(60)headroom vs N×5-min confirmation waits on very large files — follow-up ticket suggested.🤖 Generated with Claude Code