Skip to content

Fence incremental mobile backups with server revisions - #4

Draft
antuz123 wants to merge 13 commits into
codex/r3-backup-source-parityfrom
codex/r4-incremental-backup-cas-review
Draft

antuz123 wants to merge 13 commits into
codex/r3-backup-source-parityfrom
codex/r4-incremental-backup-cas-review

Conversation

@antuz123

@antuz123 antuz123 commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

R4 local metadata repair — 13 October 2026, current draft head

Relay current source is 838049e537e07eb646a2a0393eb969fba43d0739, draft/unmerged/undeployed. The previously pinned b600416e5ba7325927d7d2871cff7776f4e593fe remains the historical host proof and initial native failure baseline. The final mobile candidate is Keeper #7045 d25d5660ac72443952ae1f7899cf456b5991c805; its exact-head Local handoff supersedes the earlier app-in-progress state below.

The matching genuine native-Realm/transactional-HTTP matrix now passes 260/260, zero skipped, under Node 22.23.3 on app code 3cc9368ca2f3b2149db5743a5f9d230aa516d6a0 (final head adds documentation only) and a clean Relay b600 checkout installed with frozen Yarn. All seven operations, real crypto/production producers, stale queued payload, A→B→A, remote conflicts, protocol/acknowledgement failures, partial fanout, final verification races and complete clean Recovery Key restore are included. Seven added cases cover six actual Redux Persist consent/boot/durability paths and one actual status-source projection with native/HTTP proof; canonical opt-out wins every read-order prefix, errors deny legacy fallback, late fallback preserves continued persistence, and saved Verified cannot outlive the session proof. This is Linux host evidence, not iOS/Android UI/MMKV/relaunch or deployed fleet evidence. Current app CodeQL is green; hosted Tests and the genuine job are tracked in #7045 separately.

AUTOMATIC_BACKUP_SOURCE_READY permits only DEVELOPMENT loopback validation; hosted app automatic writes remain fenced by AUTOMATIC_BACKUP_RELAY_ROLLOUT_READY=false. Native IOS/ANDROID, human review, real operator/runtime/index/transaction/auth/request-limit evidence and old-instance/in-flight/legacy-full-writer drainage remain open. The approved contributor runtime lock remains ee935; this paired fixture does not replace it. No merge, Relay deployment, production mutation, release signing, upload or publication is authorized or performed.


13 October Local follow-up — current 838049e head

Native Android local opt-in on the initial b600 fixture reached real snapshot 200 → revisioned repair 200 → snapshot 200 but failed complete readback: createNewApp precreated an AppImage without subscription, and repair set subscription only on insert. Keeper correctly withheld Verified. Current commit 838049e537e07eb646a2a0393eb969fba43d0739 writes the supplied subscription on an existing AppImage in explicit revisioned repair after revision/conflict checks. Legacy behavior and Keeper's safety gate are unchanged.

The focused production-route test is RED 0/1 on b600, GREEN 1/1 here. Full transactional HTTP is PASS 58/58, backup suite 120/120 and TypeScript compilation PASS on current source. A loopback-only paired service rebuilt from this commit then produced native Android Verified after complete readback and after one additional label mutation. See exact-head local comment. This does not establish final exact-SHA app acceptance or deployed Relay readiness.


13 October Cloud continuation — historical b600 head

Historical head: b600416e5ba7325927d7d2871cff7776f4e593fe.

  • Fixed a concrete local-stack gate: LOCAL_DEV now allows /revisionedBackupMutation; it previously returned 503 before the v2 handler. A focused regression proves the route is reached while hosted provider routes remain unavailable. This is the only production src/ runtime change after 82ee3b1; CAS, schemas and generation logic are unchanged.
  • Node 22.23.3 compilation, 105/105 baseline/local-mode tests and 120/120 transactional backup tests passed.
  • KEEPER_APP_SOURCE=/path/to/bitcoin-keeper yarn test:automatic-http passed 57/57 locally against disposable MongoDB 7 replica-set transactions and real loopback Express. The fixture mounts the actual local-mode gate, exact production HTTP handler source, shared Relay service graph and Mongoose schemas; ciphertext uses the compatible app's real AES encryption/decryption exports.
  • All seven operations cover canonical updated:true / preconditionApplied:true / newRevision, real encrypted readback, stale 409, old-route 404, malformed acknowledgement, valid-format acknowledgement equal to the old revision, and lost reply after a real commit. Scoped operations preserve unrelated remote-only records; explicit whole-backup deletion remains the separate destructive exception.
  • Additional cases cover subscription/public-ID/version seed and response metadata, invalid snapshot references, unavailable records, malformed snapshot shape and raw invalid JSON, first-success/next-failure fanout and a peer transaction between snapshot read and reply. Explicit malformed acknowledgements can omit updated/precondition/newRevision or return null, always after a real commit with the canonical response retained in the local trace. Response faults do not mock CAS or cryptographic success.
  • The fixture now exposes the exact legacy getAppImage read route for the app's genuine Recovery Key restore test. prepareRestore(appId) seeds an actual disposable free-tier App record so production receipt validation takes its normal no-provider branch. The backend test confirms unchanged encrypted records/revision and required headers. No legacy write route or automatic endpoint fallback is mounted. This backend read test is not a claim that native Realm restore has passed.
  • Syntax and git diff --check passed. Exact-head hosted Source readiness 38342719226 and Build and test 38342719221 passed; the optional paired 57-case HTTP matrix is local evidence, not a hosted job.
  • The matching app's genuine Realm + Redux + production transport seven-operation, complete-producer/fanout and clean Recovery Key restore matrix remains in progress in #7045. Backend results alone do not establish committed local payload identity, rollback, A→B→A, consent/account binding, or atomic final verification.
  • Hosted authentication/rate limits, provider bootstrap, actual transaction/index topology, deployed route parity, drainage of every old instance and in-flight legacy write, independent approval, and separately authorized rollout remain open. Keep this PR draft. No merge, deployment, production data operation or app release is authorized by this update.

12 October runtime update — historical source head

Runtime source head: 82ee3b10708a56d220170505d1bb7e3bc8429e77. The earlier fc17d96 verification below is historical.

  • Revisioned full repair now activates the same legacy incremental fence. This closes delayed legacy update/delete writes after a repair.
  • Label mutations verify account ownership before updating or deleting shared global label records.
  • Every successful v2 mutation returns newRevision calculated in its Mongo transaction after the generation update; the route refuses a success response without it.
  • Snapshots expose dangling or historically shared label references as unavailableLabelIds and withhold ambiguous ciphertext. Normal restore and legacy full backup fail closed; v2 repair can fill a uniquely owned missing label. Shared IDs still need a separately reviewed operator migration. An AppImages labels index supports the ownership query and must be verified during backend rollout.
  • On the source runtime tree: 120/120 isolated loopback Mongo tests plus one focused projected ownership scan fixture and TypeScript compilation passed. The stacked review tree differs from that tested tree only in Restore revisioned mobile backup protocol in public Relay #3 README text; its own TypeScript compile, source secret scan and diff check passed.
  • The matching app client is still under review. A stale action can prepare a payload before a newer repair, so merely attaching a fresh revision at send time is unsafe. Keep this PR draft; no merge, deployment or app release.

Purpose

Prevent a late, already-sent incremental backup mutation from overwriting a newer encrypted backup. This is a draft stacked on Relay #3, whose current head is d4d59b080370637ac82e3a7cf6ef58564f9a22ac; the only difference from the app's pinned Relay runtime ee935359f22ff7932ea0a62e9084d4b49801f4aa is README text.

Contract and safety

  • Add POST /revisionedBackupMutation for seven active incremental operations. The client sends the current getBackupSnapshot revision and the server checks it inside a Mongo transaction. Successful responses explicitly say preconditionApplied: true; stale revisions return 409 BACKUP_CHANGED.
  • Advance the backup generation with the mutation. Once a revisioned mutation commits, legacy unversioned incremental requests for that account fail closed. The distinct route makes an old Relay instance return 404 instead of silently ignoring a revision.
  • Deterministic race tests pause both old revisioned and legacy unversioned transactions, commit a newer wallet, then resume the old writes and confirm neither can replace the new ciphertext. This addresses the confirmed lost-response overwrite reproduced during Keeper app #7045 validation.

Verification

Exact stacked head fc17d9687e7238dcb2fc0618b97c0441a77c727d: 111/111 isolated loopback Mongo backend tests passed on the source commit (same runtime tree; #3 adds only README), TypeScript tsc --noEmit -p . passed on this stacked head, source secret scan passed, and diff check passed.

Rollout and open work

The matching app client contract is being tested separately in #7045. Do not merge or deploy from this draft. Before enabling the app client, every old Relay instance and its in-flight writes must be drained and the new route verified on the deployed target; a mixed fleet or delayed old instance can still accept an unchecked old request. No backend deployment, production data mutation, app release, or publication was performed.

antuz123 commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor Author

Matching R4 app proof — 13 October 2026, final Cloud handoff

Relay source remains b600416e5ba7325927d7d2871cff7776f4e593fe, draft/unmerged/undeployed, with exact-head compile/test and secret checks green. The final mobile candidate is Keeper #7045 d25d5660ac72443952ae1f7899cf456b5991c805; its exact-head Local handoff supersedes the earlier app-in-progress state below.

The matching genuine native-Realm/transactional-HTTP matrix now passes 260/260, zero skipped, under Node 22.23.3 on app code 3cc9368ca2f3b2149db5743a5f9d230aa516d6a0 (final head adds documentation only) and a clean Relay b600 checkout installed with frozen Yarn. All seven operations, real crypto/production producers, stale queued payload, A→B→A, remote conflicts, protocol/acknowledgement failures, partial fanout, final verification races and complete clean Recovery Key restore are included. Seven added cases cover six actual Redux Persist consent/boot/durability paths and one actual status-source projection with native/HTTP proof; canonical opt-out wins every read-order prefix, errors deny legacy fallback, late fallback preserves continued persistence, and saved Verified cannot outlive the session proof. This is Linux host evidence, not iOS/Android UI/MMKV/relaunch or deployed fleet evidence. Current app CodeQL is green; hosted Tests and the genuine job are tracked in #7045 separately.

AUTOMATIC_BACKUP_SOURCE_READY permits only DEVELOPMENT loopback validation; hosted app automatic writes remain fenced by AUTOMATIC_BACKUP_RELAY_ROLLOUT_READY=false. Native IOS/ANDROID, human review, real operator/runtime/index/transaction/auth/request-limit evidence and old-instance/in-flight/legacy-full-writer drainage remain open. The approved contributor runtime lock remains ee935; this paired fixture does not replace it. No merge, Relay deployment, production mutation, release signing, upload or publication is authorized or performed.

Copy link
Copy Markdown
Contributor Author

R4 local loopback repair follow-up

Relay source head: 838049e537e07eb646a2a0393eb969fba43d0739 (draft #4). Initial pinned b600416e5ba7325927d7d2871cff7776f4e593fe was tested with a disposable Android Keeper account: snapshot 200 → revisioned repair 200 → snapshot 200, but Keeper correctly rejected the readback at repair-readback. The production createNewApp route had already inserted an AppImage without subscription; repair wrote that field only on insert.

The narrow fix sets the supplied subscription on an existing AppImage during explicit revisioned repair, after the existing revision/conflict check. No Keeper readiness gate, approved contributor manifest, legacy writer behavior, hosted service, or production data was changed.

Disposable tests: focused production-route regression RED 0/1 on b600 and GREEN 1/1 on this head; full transactional HTTP matrix 58/58, backup suite 120/120, and Relay TypeScript check PASS. The loopback paired service was rebuilt from this head without resetting its disposable Mongo/account. Native Android opt-in on the modified local app then displayed Backup verified after complete readback; a subsequent local label mutation produced snapshot 200 → revisioned mutation 200 → readback 200 and returned to Verified. This is a local source/fixture result, not deployed Relay readiness or final exact-SHA Keeper signoff.

Private logs and source identities: /Users/devaccount/Workspace/keeper-r4-local-d25-evidence/headless/relay-metadata-*.log, relay-metadata-fix-result.txt, and paired-relay-switch-838049e.txt. No seed, ciphertext or account identifiers are included here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants