Skip to content

Keep runtime GPU context handles out of retained callable history - #1

Merged
trissim merged 1 commit into
mainfrom
fix/durable-decorator-context-20260929
Sep 30, 2026
Merged

trissim merged 1 commit into
mainfrom
fix/durable-decorator-context-20260929

Conversation

@trissim

@trissim trissim commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fix the cannot pickle '_thread._local' failure reproduced from a retained,
unpublished memory-decorated callable in OpenHCS history (OpenHCSDev/openhcs#169).

Keep GPU context storage on the existing importable ThreadGPUContext runtime
owner, not in decorator function globals that Dill persists by value. Delete
the old global handle/forwarder and call ThreadGPUContext.current() directly.
No filtering of history, permissive pickle handling, alternate registries,
GPU-framework import or change to the durable history format.

Evidence

  • Before: fresh unpublished @numpy callable and real ObjectState history
    export both raise TypeError: cannot pickle '_thread._local' object.
  • After: restored callable executes correctly; the runtime thread-local handle
    remains identical and outside the serialized declaration.
  • Focused source test run: python -m pytest -q -o addopts= tests/test_durable_decorator_context.py: 3 passed (0.15s), using OpenHCS's
    existing environment with source imports from the isolated ArrayBridge checkout.
  • Changed-source Ruff and diff checks pass.
  • Paired OpenHCS continuous native declaration/history save, fresh-registry
    restore, historical undo and return to head also passes with an unpickleable
    runtime handle retained. This uses the real document/ObjectState owners.

Integration/remaining scope

Paired OpenHCS gitlink/regression/diagnostic draft:
OpenHCSDev/openhcs#208 (references #169 and #131).
This draft is a source fix, not installed/live desktop acceptance. Coordinator
owns merge/install and serialized continuous real capture/restore validation.
No old H001 export or history is modified or reconstructed. No Closes claim.
Both PRs remain draft pending source/live journey and catalog-pattern review.
The paired validation note documents the concrete BOUND-2/BOUND-1 and MEMB-2
diagnostic revision plus its new-case ownership test, not just skill reads.
New heavy validation is held for Euler's serialized H003 run; existing source
checks have finished and released the shared lock. No services are restarted.

@trissim

trissim commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

Retained callable serialization: parent source review

Reviewed ArrayBridge PR1 head
06837ec267e3ca0b1734461d09919ff65cb1b1fe against remote main
fddd9857bcd21468d20bc8547a769e0c6c9f1906. Paired OpenHCS PR208
is d69d65adbd56633e1d1fd2a3b65678728cd6ada8. Linnaeus retains
the implementation tree; parent owns integration, installation and live checks.
Neither implementation tree was edited during this review.

Ownership and durable state

The existing importable ThreadGPUContext owns its thread-local runtime handle
and current-context lookup. The global handle and forwarding function are
deleted. Decorated unpublished callables no longer pull the thread-local
handle through function globals into durable history. No history filtering,
permissive serialization, alternate registry, compatibility reader or stored
history format change is introduced. The GPU stream/device implementation is
unchanged. New cases use the existing memory-type owner and context mechanism;
there is no second roster or copied framework decision in this patch.

Evidence and its limits

  • Parent independent run: four tests PASS in 3.030 seconds, zero failures,
    errors or skips; a 15-second process bound. XML:
    retained-callable-parent-source-tests-20260930.xml.
  • Three ArrayBridge cases exercise per-thread context identity and Dill
    round-trips of unpublished NumPy/CuPy decorated callables. Restored NumPy
    invocation executes and returns [4, 8]; CuPy execution is NOT tested.
  • The fourth test uses real ObjectStateRegistry, PipelineObjectStateBinding
    and PipelineDocumentAuthority: save current declarations and full history,
    clear/load the registry, navigate to a prior unpublished callable, execute
    [7, 14], return to head, and preserve the unpickleable runtime handle.
    This is a same-process fresh-registry journey, NOT a fresh application
    process or actual live GUI restart.
  • Original packaged structural ratchet, remote ArrayBridge main to reviewed
    head: PASS, 38 projected metrics, all deltas zero, one production source
    path. This is a structural screen, not a complete NRA proof.
  • Production diff reviewed in full: decorators.py, plus a development-only
    Dill dependency and the tests. Both source working trees are tracked clean.
  • Remote main protection disabled and rulesets empty at review. Optional
    hosted CI is deferred per owner, without an administrative override.

Ship the reviewed ArrayBridge source checkpoint. OpenHCS PR208, issues169/131,
current-main integration, installed capture/restore and MCP retention/slope
measurements remain unfinished. No merge/install/live or biological acceptance
is implied by this source review. Current resource assertion exits2: swap14.6
GiB, RAM18.5 GiB available; no new heavy/native/parallel job is launched.

Merge evidence will be appended after verifying remote authoritative state.

@trissim
trissim marked this pull request as ready for review September 30, 2026 11:54
@trissim
trissim merged commit 409b1e0 into main Sep 30, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant