Skip to content

fix: read at the head by default, add a global --finalized, add send --all - #169

Merged
illuzen merged 8 commits into
mainfrom
n13/fee-estimate-at-head-and-transfer-all
Oct 19, 2026
Merged

illuzen merged 8 commits into
mainfrom
n13/fee-estimate-at-head-and-transfer-all

Conversation

@n13

@n13 n13 commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Found live, minutes after mainnet enacted spec 153. Sending 1.99 QTC from an account holding exactly 2 QTC was refused:

Insufficient balance for send. Have: 2 QTC, Need: 2.000167025 QTC
(estimated fee: 0.010167025 QTC)

The transfer was affordable. Only the quote was stale.

Cause

partial_fee_estimate calls TransactionPaymentApi_query_info at latest_finalized_block_ref (subxt tx_client.rs:583). QPoW finality trails the head, so the quote came from the runtime the upgrade had just replaced — and 153 cut FEE_SCALE tenfold. Measured on mainnet at the time:

best: 97174   finalized: 97073   lag: 101 blocks
runtime @ head:      spec 153   ← fees 10x cheaper
runtime @ finalized: spec 152   ← what the estimate used

Real fee ~0.001 QTC; quoted 0.0102.

1. Estimate at the block everything else reads

QuantusClient::partial_fee runs the same runtime API at get_latest_block(). Replaces both partial_fee_estimate call sites (send.rs, cold_signing.rs).

2. One switch for finality

An audit of every finalized reference found those two sites were the only wrong ones — everything else is a deliberate --finalized-tx opt-in, or already fixed by #152 (collect-rewards proofs) and #154 (client metadata). So rather than add a parallel mechanism, this unifies what exists.

ExecutionMode::finalized already decided how long to wait for a transaction. It now also decides which block reads are taken at. Reads happen in ~80 places with no reason to carry an ExecutionMode, so execute_command enters ExecutionMode::scope — a tokio task-local — for the command it dispatches, and get_latest_block consults it: one switch governs waits and reads, the way wait_for_transaction already governs waiting. Task-scoped rather than process-wide, so a library call to the dispatcher reads at the block its mode names, and concurrent callers never overwrite each other's choice.

--finalized-tx becomes --finalized (kept as a clap alias) since it no longer governs only transactions. events has its own --finalized, which shadows the global one, so that flag carries the alias too. system --runtime and events read the header at the block they report, so the flag moves both.

Some reads deliberately don't follow the scoped mode:

  • The signing nonce is always read at the head. A nonce from the finalized block makes the extrinsic outdated whenever the account has anything unfinalized.
  • The cold-signing mortality anchor is the head too. A 256-block era anchored at the finalized block, ~100 behind, leaves the cold signer ~156 blocks to answer, and none once finality lags by 256.
  • Wormhole proof tips follow the ExecutionMode the flow is handed, via QuantusClient::head_or_finalized. The proof block must match the wait, and library callers reaching handle_wormhole_command directly never enter the scope.
  • The SDK helpers at_best_block and at_finalized_block fetch the block they name.
  • exercise refuses --finalized. Its multisig and treasury proposals expire 3 to 100 blocks past the block read, and each step waits on the last, so finality's lag puts those deadlines in the past.

Default is the head. --finalized opts back in for callers who want finality's guarantees and will wait ~20 minutes for them.

3. send --all

Sweeping by subtracting an estimate cannot be made reliable — too low strands dust, too high is refused, which is exactly what happened above. --all submits Balances::transfer_all and lets the chain deduct the exact fee. --keep-alive — the flag #181 added for transfer_keep_alive — is transfer_all's keep-alive here: leave the existential deposit rather than reap.

quantus send --from m1 --to <dest> --all

--amount and --all are mutually exclusive, enforced by clap. handle_send_command keeps its fixed-amount signature; --all dispatches to the new handle_send_all_command.

The verbose fee line reads MiningRewards::FeesCollected from the included transaction. A balance delta can't separate the fee from what --all moved, and TransactionFeePaid omits the refund a reaped account forfeits: on a dev chain it reported 0.008028025 for a sweep that cost 0.008203025. send needs the inclusion block for this, so manual-nonce submission gains submit_transaction_with_nonce_and_inclusion_block; submit_transaction_with_nonce still returns the hash.

Verification

  • 456 lib tests pass (6 new): concurrent scopes each see their own finality flag, the cold-signing context anchors at the head under --finalized, wormhole tips and the SDK block helpers ignore a scoped mode, exercise refuses a finalized scope before touching the node, --all encodes transfer_all with the keep-alive byte, and a fixed amount still builds transfer_allow_death. A bin test parses --finalized-tx on either side of events. The one remaining lib test, loads_private_and_public_batch_verifiers, needs the circuit build CI performs.
  • SKIP_CIRCUIT_BUILD=1 cargo clippy --all-targets --locked -- -D warnings clean
  • cargo +nightly fmt --all -- --check clean
  • Exercised against live mainnet on 153
  • Testnet: --finalized system --runtime reports the finalized block, ~100 behind the head
  • Local dev chain: the fee line matches the sender's balance change to the planck for a fixed amount, a manual nonce with a tip, --all --keep-alive and a reaping --all. --finalized send from an account with an unfinalized transaction now signs with the head nonce and is included; it was rejected as outdated before

Follow-up, not in this PR

On a dev node the pool never reports InFinalizedBlock for a watched transaction: a --finalized wait ends in Dropped("Finality timeout") after 480 s even though its block is finalized, so the command reports failure for a finalized transfer. Predates this PR — --finalized-tx shared the wait.

COMPATIBLE_RUNTIMES still tops out at 149, so every mainnet command prints specVersion=153 … newer than this CLI's tested list. Cosmetic — transaction_version never moved off 6 — but all three chains are on 153 now and it should be bumped.

…--all

Sending 1.99 QTC from an account holding exactly 2 QTC was refused right after
mainnet enacted spec 153:

  Insufficient balance for send. Have: 2 QTC, Need: 2.000167025 QTC
  (estimated fee: 0.010167025 QTC)

subxt's partial_fee_estimate calls TransactionPaymentApi_query_info at
latest_finalized_block_ref. QPoW finality trails the head by ~100 blocks, so the
estimate was computed by spec 152 -- the runtime the upgrade had just replaced --
and 153 cut FEE_SCALE tenfold. The real fee was ~0.001 QTC. The transfer was
affordable; only the quote was stale.

Estimate against the same block as every other read, via
QuantusClient::partial_fee.

Unify the finality choice. ExecutionMode::finalized already decided how long to
wait for a transaction; it now also decides which block reads are taken at.
Reads happen in ~80 places that have no reason to carry an ExecutionMode, so
main publishes the flag once with ExecutionMode::install and
QuantusClient::get_latest_block consults it -- one switch for waits and reads.
wormhole_tip_block's own head/finalized branch and at_finalized_block go away,
since get_latest_block already answers that question.

--finalized-tx becomes --finalized (kept as an alias) because it no longer only
governs transactions.

Add send --all, which submits Balances::transfer_all and lets the chain deduct
the exact fee. Sweeping an account by subtracting an estimate cannot be done
reliably -- too low strands dust, too high is refused -- and --keep-alive
chooses between reaping the account and leaving the existential deposit.
@n13 n13 added the bot-review Request automated review from review-bot label Sep 30, 2026

@n13 n13 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer model: GPT-6 Sol

Verdict (advisory): Request changes

  1. [P1] Keep proof reads aligned with the requested finality for library callers. src/cli/wormhole.rs:1213-1216 now selects its proof block through get_latest_block(), whose global flag is installed only by the binary in src/main.rs:92-98. The public handle_wormhole_command(..., ExecutionMode { finalized: true, ... }) and execute_command paths do not install that flag. A library caller therefore waits for finalized transactions while building wormhole proofs from the unfinalized head; the prior code used the supplied mode to select the finalized block. Make the block selection honor the supplied mode outside main as well, and cover that public call path in a test.

  2. [P2] Make system --runtime honor the new global --finalized read setting. The flag promises finalized reads, but src/cli/system.rs:184-220 still gets runtime versions with no block hash and calls chain_getHeader and state_getRuntimeVersion with empty parameters. quantus --finalized system --runtime consequently reports the best block's height and runtime, including during the post-upgrade gap that motivates this PR. Resolve one selected block hash and use it for those runtime and header queries.

  3. [P3] Preserve fee reporting for send --all. src/cli/send.rs:697-704 passes the entire starting balance as the transfer amount, so the fee calculation at src/cli/send.rs:744-747 always saturates to zero for a sweep and the verbose fee line disappears even though a fee was paid. Derive the transferred amount or fee from the included transaction before reporting it.

Validation: cargo +nightly fmt --all -- --check and git diff --check passed. The three new focused tests passed. With SKIP_CIRCUIT_BUILD=1, 364 library tests passed when the generated verifier artifact test was excluded; without that exclusion, that test failed because the skipped build did not create its required bins. The PR's full build/test and analysis CI jobs passed at head e87b9d7.

@n13 n13 removed the bot-review Request automated review from review-bot label Sep 30, 2026
- Wormhole proof tips follow the ExecutionMode the flow is handed
  (QuantusClient::head_or_finalized). Library callers never install the
  switch, and the proof block must match the wait.
- system --runtime reads the runtime version and header at one selected
  block, so --finalized moves both.
- send reports the fee from the included transaction's
  MiningRewards::FeesCollected. A balance delta cannot separate it from
  what --all moved, and TransactionFeePaid omits the refund a reaped
  account forfeits.
- The signing nonce is always read at the head. Under --finalized the
  finalized nonce made the extrinsic outdated whenever the account had
  anything unfinalized.
@n13 n13 added the bot-review Request automated review from review-bot label Sep 30, 2026

@n13 n13 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer model: GPT-6 Sol

Verdict (advisory): Request changes

  1. [P2] Keep the published finalized-block SDK helper. src/lib.rs:62-66 removes the at_finalized_block re-export and src/cli/wormhole.rs deletes the public function. It was explicitly exported for SDK use at the base commit, so existing quantus_cli::at_finalized_block(...) callers will stop compiling, although this PR does not change the crate's 2.3.0 version. Restore the helper and re-export it; it can use head_or_finalized(true) internally. Also keep the surviving at_best_block helper pinned to the head: its new get_latest_block() call at src/cli/wormhole.rs:1477 can return a finalized block after ExecutionMode::install(true).

  2. [P2] Preserve the public manual-nonce submission result. src/cli/common.rs:709-716 changes submit_transaction_with_nonce from Result<H256> to Result<(H256, Option<H256>)>. Existing library callers expecting a transaction hash will fail to compile. Keep the old signature as a wrapper and add an inclusion-block variant for send, as this PR already does for automatic-nonce submission.

The previous wormhole tip, system --runtime, and send fee-reporting findings are addressed at head e4f6a2b. Validation: cargo +nightly fmt --all -- --check, git diff --check, and cargo metadata --no-deps --locked passed. Local focused test compilation was stopped because this detached worktree has a cold target directory; the current-head Ubuntu/macOS build-and-test, Clippy, format, and security CI checks passed.

@n13 n13 removed the bot-review Request automated review from review-bot label Sep 30, 2026
- Restore at_finalized_block and its lib.rs re-export, and pin
  at_best_block to the head. Both fetch the block they name via
  head_or_finalized, whatever --finalized installed.
- submit_transaction_with_nonce returns the hash again. send uses the new
  submit_transaction_with_nonce_and_inclusion_block, mirroring
  submit_transaction and submit_transaction_with_inclusion_block.
@n13 n13 added the bot-review Request automated review from review-bot label Sep 30, 2026

@n13 n13 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer model: GPT-6 Sol

Verdict (advisory): Request changes

  1. [P2] Keep the cold-signing mortality anchor at the best block. src/cli/cold_signing.rs:174-189 now takes get_latest_block() for the block number and hash embedded in a 256-block mortal signature. Under --finalized, that is the lagging finalized block, although the nonce is deliberately read at the head. With the roughly 100-block lag described in this PR, an offline signer has only about 156 blocks left to scan and return the QR; if finality lags by 256 blocks, the signature is unusable immediately. Use head_or_finalized(false) for this signing context, as the nonce path does, and cover --finalized in a cold-signing test.

  2. [P2] Preserve the public send handler's existing signature. src/cli/send.rs:596-601 changes pub handle_send_command from an amount &str to Option<&str> and adds sweep_keep_alive. Since cli::send is public and the crate remains at 2.3.0, existing Rust callers of this handler stop compiling. Keep the old signature as a fixed-amount wrapper and add a separate sweep-aware entry point for the CLI.

  3. [P2] Apply the supplied mode when the public dispatcher is called from the library. src/cli/mod.rs:391-397 accepts ExecutionMode but does not install its read setting; only the binary does so in src/main.rs:97. For example, a library call to execute_command(Commands::System { runtime: true, ... }, ..., ExecutionMode { finalized: true, ... }) still reads the best block in system --runtime, while the same command through main reads the finalized block. Make the dispatcher honor its supplied mode, without allowing concurrent library calls to overwrite one another's read selection.

The previous wormhole-tip, system-runtime, fee-reporting, SDK block-helper, and manual-nonce API findings are addressed at 5bbdbec.

Validation: git diff --check, cargo +nightly fmt --all -- --check, and cargo metadata --no-deps --locked --format-version 1 passed. The exact-head GitHub build/test matrix, Clippy/doc, format, examples, and security checks passed. Local transaction tests were not run because this detached worktree has no built target or node; the mortality and API findings follow from the changed call paths.

@n13 n13 removed the bot-review Request automated review from review-bot label Sep 30, 2026
n13 added 2 commits October 19, 2026 17:35
#181 added --keep-alive to send for transfer_keep_alive; one flag now
serves both a fixed amount and --all, where it is transfer_all's
keep_alive.
…lity at the head

ExecutionMode::install, a process-wide flag only main set, becomes
ExecutionMode::scope: a task-local that execute_command enters, so a
library call to the dispatcher reads at the block its mode names and
concurrent callers never overwrite each other's choice.

capture_tx_context anchors the 256-block mortal era at the head even
under --finalized, like the nonce: anchored ~100 blocks back the cold
signer would have ~156 blocks to answer, none once finality lags by 256.

handle_send_command keeps its fixed-amount signature; send --all goes
through the new handle_send_all_command. Both take the one --keep-alive
flag from #181, which for a sweep is transfer_all's keep_alive.
@n13

n13 commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Merged main (a0ded0f) and addressed the third review at a89e0ea.

Merge. #181 had added --keep-alive to send for transfer_keep_alive, colliding with this PR's --all-only --keep-alive. One flag now serves both: with --amount it is transfer_keep_alive, with --all it is transfer_all's keep-alive.

1. Cold-signing mortality anchor. capture_tx_context takes the block for the 256-block mortal era via head_or_finalized(false), as the nonce path does. tx_context_anchors_mortality_at_the_head_under_finalized captures a context inside a finalized scope against the mock node and asserts the head.

2. Public send handler. handle_send_command keeps its fixed-amount signature (&str amount, plus #181's keep_alive). --all dispatches to the new handle_send_all_command; both are thin wrappers over a private send_from_wallet.

3. Dispatcher honours its mode, without cross-talk. The process-wide ExecutionMode::install is gone. ExecutionMode::scope wraps a future in a tokio task-local, execute_command enters it for the command it runs, and get_latest_block reads it; outside any scope reads take the head. So a library call to execute_command(.., ExecutionMode { finalized: true, .. }) now reads system --runtime at the finalized block, and two concurrent library calls with different modes each keep their own. reads_follow_the_scoped_finality_flag joins a finalized and a head scope with yields between them and asserts each sees its own value. The wormhole tip and SDK block-helper tests now run under the opposite scope to prove they ignore it.

Verification: 429 lib tests pass with SKIP_CIRCUIT_BUILD=1 (the remaining one, loads_private_and_public_batch_verifiers, needs the circuit build); clippy --all-targets -D warnings and cargo +nightly fmt --check clean.

@n13 n13 added the bot-review Request automated review from review-bot label Oct 19, 2026

@n13 n13 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer model: GPT-6.1 Sol

Verdict (advisory): Request changes

Reviewed head a89e0ea6afcdcac8b965f4868a63b058cdbb74a3 against base a74fd735fdb7cae2381b66dbaaa513630c0d58d9. The previous cold-mortality, public send-handler, and concurrent dispatcher-scope findings are addressed.

  1. [P2] Align exercise reads with its transaction waits. The new scope at src/cli/mod.rs:407-409 also covers Commands::Exercise, but ExerciseCtx::wait_mode() still hardcodes finalized: false (src/cli/exercise/runner.rs:46-47). On a node whose finality trails the head, quantus --finalized exercise ... funds the ephemeral accounts and returns from the best-block inclusion wait, then fund_ephemeral_accounts rereads their old finalized balances (src/cli/exercise/mod.rs:535-544). It reports a zero funding delta and aborts setup even though the funds were already transferred. The same stale reads also reach the exercise budget accounting. Thread the requested mode through the exercise waits, or explicitly scope its head-based workflow to its own mode, and keep budget checks current. Cover setup with distinct best/finalized state.

  2. [P3] Preserve the old flag alias after the events subcommand. The alias at src/main.rs:51 is shadowed by the existing local Commands::Events::finalized argument. With the locked Clap 4.6.1, quantus events --finalized-tx now fails with “unexpected argument”, while placing the flag before events works. Reuse the global argument or retain the alias on the local definition, and check both positions.

Validation: SKIP_CIRCUIT_BUILD=1 cargo test --lib --locked -j 4 -- --skip loads_private_and_public_batch_verifiers passed all 429 runnable tests; the excluded test requires generated circuit artifacts. Pinned-nightly rustfmt, git diff --check, and locked Cargo metadata passed. An isolated Clap 4.6.1 probe confirmed amount/all exclusivity and reproduced the alias failure. All exact-head GitHub CI checks passed, including Ubuntu/macOS build/test and Clippy/doc. The exercise finding follows the dispatcher, wait, and balance-read call paths; no live transfers were submitted.

@n13 n13 removed the bot-review Request automated review from review-bot label Oct 19, 2026
n13 added 2 commits October 19, 2026 18:41
main split the mock node into test_support with mock_node(stall_storage)
returning MockNode. The tests this branch added take that shape, and the
mock keeps this branch's chain_getHeader handler.
…vents

ExerciseCtx::wait_mode hardcoded best-block inclusion while its balance
checks follow the scoped read block. Under --finalized, setup returned
from the inclusion wait, read the old finalized balances and aborted on
a zero funding delta. Waits now go to the finality reads are taken at,
via ExecutionMode::wait_at_read_finality.

events has its own --finalized, which shadows the global flag and its
finalized-tx alias. The local flag carries the alias too, so
`events --finalized-tx` parses again.

events also read the header for the reported block number at the best
block while the hash followed the scoped mode. Finalized and latest now
share one branch that reads the header at the chosen hash.
@n13 n13 added the bot-review Request automated review from review-bot label Oct 19, 2026

@n13 n13 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer model: GPT-6.1 Sol

Verdict (advisory): Request changes

Reviewed head a350116980dafb3aa2b82d4de4919d79c1c1253d against base 365ef8c11b4abb567fe69b70a04c117a08f3ad7c. The previous exercise funding/read mismatch and events alias findings are addressed. One related exercise regression remains.

  1. [P2] Keep exercise proposal lifetimes valid under --finalized. ExerciseCtx::wait_mode() now waits for finalization between steps, while the dispatcher scope also moves the scenarios' deadline calculations to the finalized block. The multisig lifecycle still sets expiry to that height plus 100 (src/cli/exercise/scenarios/multisig.rs:72), and expired-proposal cleanup uses plus 3 (:158). For example, finalized height 1000 and best height 1100 produce lifecycle expiry 1100, so the next block rejects the proposal with ExpiryInPast; cleanup is already invalid with a lag of only three blocks. The treasury proposal flow in src/cli/exercise/scenarios/vesting.rs:426-465 has the same plus-100 calculation and sequential waits. Even when a proposal initially has time remaining, waiting for finalization between propose/approve/execute consumes its lifetime. These exercise phases now fail after setup has already transferred funds. Keep these short-lived workflows on consistent head reads and inclusion waits, or adjust both deadlines and sequencing to accommodate finality. Cover distinct best/finalized heights and block advancement during waits.

Validation:

  • SKIP_CIRCUIT_BUILD=1 cargo test --lib --bin quantus --locked -j 4 -- --skip loads_private_and_public_batch_verifiers passed both suites. The excluded test requires generated circuit artifacts.
  • cargo +nightly-2026-08-31 fmt --all -- --check, git diff --check, and locked Cargo metadata passed.
  • Current-head GitHub format, Clippy/doc, security and dependency-cooldown checks passed; Ubuntu/macOS build/test and examples jobs were still running at review time.

The deadline finding follows the changed call paths and the runtime's expiry validation; live transaction validation was not run.

@n13 n13 removed the bot-review Request automated review from review-bot label Oct 19, 2026
Following the finalized block broke exercise's short-lived proposals.
Multisig and treasury proposals expire 3 to 100 blocks past the block
read, and each step waits on the last, so finality's ~100-block lag put
those deadlines in the past after setup had already moved funds.

Exercise goes back to head reads and inclusion waits, and
handle_exercise_command refuses a finalized scope up front rather than
ignore the flag.
@n13 n13 added the bot-review Request automated review from review-bot label Oct 19, 2026

@n13 n13 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer model: GPT-6.1 Sol

Verdict (advisory): Approve

Reviewed head 220567d1ad6ce27cd1c9159242734b707e10b7e0 against base 365ef8c11b4abb567fe69b70a04c117a08f3ad7c.

No blocking findings in the full 13-file diff. The previous exercise deadline regression is addressed by rejecting --finalized before node access or funding, and the events flag alias works on either side of the subcommand. The fee query preserves Subxt's runtime API encoding while selecting the requested block; both estimate paths use it. Signing nonces and cold-signing mortality remain anchored at the head. The sweep uses transfer_all, fee reporting reads the included extrinsic's events, and the existing send-handler and manual-nonce APIs are preserved.

Validation:

  • SKIP_CIRCUIT_BUILD=1 cargo test --lib --bin quantus --locked -j 4 -- --skip loads_private_and_public_batch_verifiers: 456 library tests and 455 binary tests passed. The excluded test requires generated circuit artifacts.
  • cargo +nightly-2026-08-31 fmt --all -- --check, cargo metadata --no-deps --locked --format-version 1, and git diff --check passed.
  • Exact-head GitHub CI: Ubuntu build/test, Clippy/doc, format, security, and dependency-cooldown checks passed. macOS build/test and examples were still running at review time.

No live transactions were submitted during this review.

@n13 n13 removed the bot-review Request automated review from review-bot label Oct 19, 2026
@illuzen
illuzen merged commit ab31f9d into main Oct 19, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants