Skip to content

Exercise activity retries, heartbeat renewal and deadline authority - #438

Merged
rmcdaniel merged 26 commits into
mainfrom
rust-325-activity-deadline-followthrough
Oct 14, 2026
Merged

rmcdaniel merged 26 commits into
mainfrom
rust-325-activity-deadline-followthrough

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Exercise retry attempts, live heartbeat renewal and completion authority around the original activity deadlines across Rust, published PHP and embedded Workflow. Four separate fixtures verify original tasks/attempts/runs, complete retry snapshots, global timeout with remaining attempt budget, genuine renewal across the first heartbeat boundary, matching committed controls before and after expiry, conflicting outcomes and exact int64/null/Unicode results. Closed terminal metadata keeps the agreed 409/recorded=false observations and does not imply payload equality or universal 200 reconciliation.

The installed SDK parent poll retries only a matching 503/backend_lock_pressure response with the same logical poll ID and advertised backoff inside its original observer budget. Unknown/malformed refusals fail. Original JSON is copied into saved cases without rounding its int64 literals; normal maintenance handles active activity expiry without manually repairing the parent.

Validation: candidate c99b113/tree6eda44f2 qualifies SQLite, PostgreSQL16/17, MySQL8.0/8.4 and MariaDB10.11 in run38492234181. The final MySQL artifact hash and recorded CI merge6604 independently match; local three-runtime comparison and all twelve actual corruption-reader invocations pass. Earlier upstream Composer500 and semantic/harness failures keep their original disposition. Scenario deadlines and whole-job limits are unchanged. Installed pressure controls and the shared-reader suite pass. Exact all-six candidate audit.

Current-main integration is 97e63c0/tree1065360b83f0dbf531a67a7af8edbece19b97fec; it includes reviewed PHP metadata and #442 with native subtreef8ca2ece unchanged. Fresh combined shared38506521977/source38506521987 are pending; both boundary gates pass. Promotion waits for that exact tree. Frozen actors/dependencies/capabilities and #435's delivered results are unchanged. No release, performance, migration, historical cause or full #325 completion claim. Refs #325.

@rmcdaniel rmcdaniel changed the title Exercise activity timeout retries and terminal completion submissions Exercise activity retries, global deadlines and terminal submissions Oct 13, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

The three native SQLite HTTP originals pass at 297b3c1, with 63 corruption controls and stopped read-only relationship/deadline inspection. The successful retry cases retain one completion each; the global case retains only its actual parent timeout delivery. Native identical receipts are 200/recorded=false and conflicting payloads remain 409 under the settled receipt boundary. I am adding a bounded live heartbeat control next: renew before the original heartbeat bound, keep the same attempt alive past that bound, prove status reads do not renew, complete within the unchanged global/start-to-close bounds, and refuse post-terminal progress. This exercises existing authority without a new wire capability or runtime change. Other-database/default integration and full release qualification remain pending.

@rmcdaniel rmcdaniel changed the title Exercise activity retries, global deadlines and terminal submissions Exercise activity retries, heartbeat renewal and deadline authority Oct 13, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

Current head e8f1f09 wires the four locally qualified activity cases and their actual-record readers into the corrected shared profile. Native source, frozen package manifests and individual resource/deadline budgets are unchanged. The b675756 boundary run remains failed because test-phase container removal violated the repository isolation policy. The corrected phase uses a separate maintenance name, leaves removal to unconditional cleanup, and includes SQLite WAL/SHM companions in its stopped audit copy. Local isolation policy and all 72 policy tests pass; shell/PHP syntax and diff checks pass. Final hosted matrix/source gates remain pending. #435871eec1/run38477701278 remains independently in progress, with no promotion or release claim. Local compiler/runtime/dependency/database/raw-record resources are retired.

@rmcdaniel

Copy link
Copy Markdown
Member Author

The recorder precision correction at 6a79f7b9c2822c122ba7616db0e86a0b8fd0466f preserves original fixture/probe JSON when writing each case. It copies those validated bytes in bounded chunks, leaving canonical typed assertions and fixture semantics unchanged. Earlier rounded JavaScript diagnostic copies stay recorded as such.

Validation passes:

  • Nine focused capture/writer controls, including beyond-JavaScript-precision int64 literals and the bounded-heap writer. The original writer fails the new integer-literal control.
  • Shared Node reader suite: 4,821 passing checks, zero failures and one explicit skip. These are reader checks, not workflow counts.
  • Fresh real workflow 01m4vt3gtwbqavz55zethqa30h with normally installed PHP SDK 2.2.6 and unchanged published Server 2.5.14/Core 2.5.5. The frozen amd64 image is sha256:28fa0f3968ea253c727aad7fe583bb61c788553f1e4b96ece6ea4570d20faf42. The source fixture and saved fixture are equal under integer-aware readback; input, decoded final history and result retain 9007199254740993. Canonical typed output also matches. Three actual history events complete the same original.

Temporary probe/source directories are absent after recording. Both task containers, their isolated network, dependencies and generated recordings are removed. No compiler build or image build was required. Native subtree remains f8ca2ece95360786f404ec1f77512149ecd8c9cf. No frozen tuple, native code or result contract changes. #435/#438 current database runs remain independently pending; this local baseline case does not qualify their final head or a release.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Run 38484029903 at superseded head 9daf1bae3 has been explicitly cancelled after final recorder head 351e95491 was pushed. The new source preserves original JSON integer literals and therefore needs its own matrix. This retires duplicate runtime qualification and frees hosted runners for the final revision; it is not a same-source retry or a response to an observation timeout.

The cancelled original is not accepted as a pass. Final run 38486080308 remains the active #438 qualification. #435 5a9ab7d36 / run 38483816953 continues unchanged. Frozen tuples, individual deadlines and resource allocations remain unchanged.

@rmcdaniel
rmcdaniel changed the base branch from rust-325-corrected-marker-cohort to main October 14, 2026 00:16
@rmcdaniel

Copy link
Copy Markdown
Member Author

Final-head run 38486080308, SQLite job 115602724203, is failed and remains so. It stopped in the existing corrected activity-deadline family before the new retries/renewal family. The published PHP 2.6.2 parent workflow poll returned HTTP 503, poll_status=reason=backend_lock_pressure, task=null, matching default namespace/queue/kind, and Retry-After: 1. The fixture currently aborts this documented transient response.

Independent read-only SQL audit confirms the same schedule-to-start original retained its four events ending in one ActivityTimedOut, exact frozen retry snapshot and original run deadlines. Its activity was never claimed (attempt 0), and its canonical parent wakeup remained ready, unleased, attempt 0/repair 0. This supports a fixture retry correction, not a PHP defect or an explanation of older SQLite/namespace failures.

Next revision will retry only a fully matching pressure refusal through the normally installed Client response API, preserve the same logical poll request identity, respect the advertised backoff inside the original 35-second observer budget, and refuse unrelated/malformed failures. No outcome/admission retry, deadline extension, frozen tuple change or runtime source edit. Raw transport/database copies have been deleted after the immediate audit. The other current jobs keep their original records. PostgreSQL 17 also reports a failure whose logs are not available yet, so no all-six acceptance is claimed.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Revision c99b113 is pushed. It merges reviewed PHP #440/main 11261ef before independent combined-tree qualification; native subtree remains f8ca2ece, dependencies/versions/frozen published actors unchanged.

The deadline fixture now retries only fully matching documented poll pressure, preserves the same logical poll identity and advertised delay inside its original 35-second budget, and refuses unknown admission or malformed scope. Wire timing and strict readers prove the actual request/backoff/deadline. Successful empty polls still end their logical request.

Normally installed PHP SDK 2.3.9 controls pass 15 cases, including the old-helper refusal, actual one-second retry, independent next empty-poll identity, budget exhaustion and scope/error refusals. Seventeen reader controls pass. Shared Node checks pass 4,838 / zero failures / one explicit skip.

Committed-source real HTTP check 00:47:39.987–00:47:56.004 UTC passes all four original expiry cases on immutable PHP 2.6.2 amd64 944c384c/Core 2.5.14. One synthetic pre-forwarding parent-poll pressure response causes exactly one 1.000306-second wait and same-ID retry, without new admission or budget. Actual-record reader passes 85 corruption controls. After stopping both write-capable roles, read-only SQL confirms each original's exact event prefix and one leased parent, attempt 1/repair 0. The initial local observer setup error was preadmission and is not counted as a workload.

Original 38486080308 now has four passing DB jobs and two failed originals. SQLite's documented pressure observation and PostgreSQL 17's GitHub dist HTTP 500 remain failed. PG17 had passed both PHP/native deadline and new retry/renewal HTTP families before embedded installation failed. No shared historical cause, PHP correction or complete matrix is inferred. All immediate audit archives/DB copies, local application/dependencies/network and newly pulled image are deleted. Fresh c99 final-head and fresh-main gates remain required.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Final-head c99b1130090b8e29db0bcab7d1bdfa6fd34a0dbf run 38492234181 retains four passing database jobs and two failed originals. Both PostgreSQL 16 and MySQL 8.4 passed the portable marker/clock/activity-expiry slices and all four new retry/renewal HTTP cases before dependency installation stopped later qualification.

Independent artifact audits identify GitHub HTTP/2 500 dist downloads: PostgreSQL 16 failed on psy/psysh and later worker controls also failed on other packages; MySQL 8.4 failed on guzzle/guzzle in the corrected embedded original install and worker-fencing install. Artifact 11863986390 SHA-256 7c2156ccd48173a1a712004e4d974b88be4fd37b9f4a9dfa94ea23a9e7d59540 and 11864690956 SHA-256 8a0392a5bac95d0795b706f1223e4b0bfadcdf49d1ea89b822d2853ac3be9bfb were verified, inspected, and local downloads deleted.

One identical-head/limits PostgreSQL 16 job retry is now accepted. GitHub refuses the subsequent MySQL 8.4 retry while that run is active; it remains pending rather than being silently counted as rerun or passed. Original failures stay failed. No source, frozen tuple, request budget, native behavior or historical cause changes. Current-main PHP 2.7 metadata is later source awareness and still requires combined-tree qualification before promotion.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Exact-source PostgreSQL16 retry115636376944 PASS, including all new retry/renewal cases on PHP/native/embedded, ordinary recovery, worker fencing, pressure and TLS. Earlier PG16 dependency-install failure remains failed. Current c99 source now has five passing database configurations; MySQL84 remains pending.

The original failed MySQL84 job was carried into attempt2 with a new job ID. GitHub refused the stale original ID; the one same-head/limits retry is now submitted against current115636378295 after the PG16 job finished. No source, resource, request budget, dependency or frozen tuple changes. Original MySQL84 GitHub HTTP500 install failure remains failed. Final all-six qualification, current-main combined-tree check and promotion remain ahead.

@rmcdaniel

Copy link
Copy Markdown
Member Author

#438 candidate c99b1130090b8e29db0bcab7d1bdfa6fd34a0dbf now qualifies all six database configurations. Run 38492234181 attempt 3 is successful: the final unchanged-head/limits MySQL 8.4 retry 115649591578 passed, following the PostgreSQL 16 retry and four first-attempt passes. Earlier Composer HTTP 500 failures remain failed originals; no scenario budget or source was changed for either single-job retry.

The final MySQL artifact 11868135011 independently matches SHA-256 608652f48d870d0789a7e413119d66e06c2dc39adcbfa434ac473fba0b0faaa2. Actual recorded CI merge 6604a3b211121fdf85c7459290b6e919f88709ab and branch c99b1130 have the identical complete tree 6eda44f2aadffb046e0c0f018b126ccd1a4be0e3. Extracted only the three retry-family records and comparison outcome; independent local readers pass all four actual fixtures across PHP, Rust and embedded modes, all twelve corruption-reader invocations and three-runtime comparison. Original prefixes/tasks/attempts, entire retry snapshots and int64/null/Unicode values remain exact. Pre-expiry identical committed controls and before/after-expiry duplicate/conflicting outcomes follow the settled terminal-response boundary, without a universal HTTP 200 or content-equality inference from terminal metadata.

The completed job also passes ordinary fixture/process recovery, roles, corrected portable/range/clock/expiry profiles, source worker fencing, declared temporary database pressure and trusted/untrusted/wrong-host TLS controls. The original 75-minute whole-job ceiling and scenario deadlines are unchanged. Native source and all frozen actors remain unchanged.

The transport ZIP and extracted records are deleted immediately after this audit. Next is current-main 59b58d28 integration (including reviewed PHP metadata and #442) and a fresh combined-tree qualification before promotion. This is candidate qualification, not merge/main delivery, published native acceptance, performance, historical failure-cause attribution or full #325 completion.

@rmcdaniel

rmcdaniel commented Oct 14, 2026 •

Copy link
Copy Markdown
Member Author

Current-main 59b58d2841edca8a0cb43c2dd4d5f8df77832949 is integrated into the #438 candidate as 97e63c01aac160cd3a904f08628d63c700acb250, complete tree 1065360b83f0dbf531a67a7af8edbece19b97fec. Only the documentation conflict needed manual resolution; it preserves delivered #435 and merged #442 scope and #438's original failures/current candidate qualification. Native subtree f8ca2ece95360786f404ec1f77512149ecd8c9cf, frozen SDK/image/dependency references and receipt boundary are unchanged. Trusted reviewed PHP main metadata is included without retupling the separate conformance cohorts.

Fresh combined shared run 38506521977 is live on all six databases. Exact source/PHP 38506521987 and both boundary 38506521981/38506517415 pass. PR remains draft/unmerged; promotion requires the fresh full matrix and final review of this tree. Earlier c99/CI6604 all-six candidate qualification remains valid as its own record, not a substitute for these combined checks. No release/cutover/performance or full #325 claim. All local runtime/raw resources are retired; clean source only.

@rmcdaniel rmcdaniel left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed final combined head 97e63c0 / complete tree 1065360b. Native f8ca2ece and all frozen tuples are unchanged; reviewed poll-pressure identity/deadline handling and byte-preserving fixture/observation recording. All six original cells in 38506521977 attempt 1 and all applicable normal source/boundary checks pass. Conditional cancellation/prepared-local remains explicitly skipped. Prior failed records retain their dispositions. This qualifies the selected retry/renewal/receipt fixture expansion; fresh-main qualification, performance, takeover and rewrite delivery remain separate.

@rmcdaniel
rmcdaniel marked this pull request as ready for review October 14, 2026 04:23
@rmcdaniel
rmcdaniel merged commit 50ce8f4 into main Oct 14, 2026
14 checks passed
@rmcdaniel

Copy link
Copy Markdown
Member Author

#438 is merged at 50ce8f42967bc14c0169fc31b16fbbbd5827528f, complete tree 1065360b83f0dbf531a67a7af8edbece19b97fec, identical to reviewed 97e63c01 and the recorded CI merge e8f4fc49. Final combined 38506521977, attempt 1, passes all six original database cells without reruns. Every applicable normal source/repository/boundary gate passes; conditional cancellation/prepared-local qualification remains explicitly skipped. Final review is 5513035907.

The four new activity families cover active-parent before/after expiry and committed duplicate control, closed-parent terminal receipts, original global-deadline retries and live heartbeat renewal. Complete original retry snapshots, canonical typed values, original identities/deadlines and unchanged-history refusal controls remain strict. The documented closed-task PHP 409 boundary is preserved; no universal content-verified HTTP-200 reconciliation is implied. Native subtree f8ca2ece, frozen actors/tuples and default runtime are unchanged.

The earlier failed/cancelled originals stay unsuccessful with their documented causes/limits. Shared PHP/runtime causes are not inferred from the successful candidate. Automatic exact fresh-main qualification is now pending before final delivery. #443's later combined-tree gate, memo work in draft #444, performance, takeover and rewrite release remain separate. No local raw archives or database stacks are retained.

@rmcdaniel

Copy link
Copy Markdown
Member Author

#438 is delivered at merge 50ce8f42967bc14c0169fc31b16fbbbd5827528f, complete tree 1065360b83f0dbf531a67a7af8edbece19b97fec, identical to reviewed 97e63c01 and CI merge e8f4fc49.

The original final combined candidate 38506521977 and exact fresh-main 38511209412 pass all six database configurations on their first attempts: SQLite, PostgreSQL 16/17, MariaDB 10.11 and MySQL 8.0/8.4. Independent readback of each completed job confirms actual PHP, Rust and embedded passes for active-parent receipts, closed-parent receipts, global-deadline retry and heartbeat renewal. Existing corpus, actual process recovery, worker fencing, pressure and applicable TLS gates pass. Fresh main source and boundary checks pass; conditional cancellation/prepared-local qualification is explicitly skipped.

This qualifies the matched response boundaries and preserves the original retry/heartbeat snapshots and deadlines. It does not establish a stronger universal content-verified receipt contract, historical failure causes, performance, database takeover or rewrite release readiness. Native source and frozen packages remain unchanged. Earlier failed/cancelled originals keep their recorded dispositions. The next separate gates are #443's combined CPU deadline matrix and #444's initial-memo comparisons; #325 remains open.

@rmcdaniel
rmcdaniel deleted the rust-325-activity-deadline-followthrough branch October 14, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants