Skip to content

sdk: compiled messages deeper than a list in a variable keep the variables of each turn #113

Closed as not planned

Description

@ephraimduncan

In the fix of #97 (#112), compile() looks for compiled messages in its variables: a message as a variable value, or as an item of a list (Python: a list or tuple) in a variable. When it finds one, the compiles two steps back remove their variables. Thus a chat history of compiled messages keeps the variables of its last two turns. In a test of 2000 turns, 2 turns kept their variables.

compile() does not look deeper in a variable. Examples: Python prompt.compile(context={"history": history}), or a TypeScript array of objects that hold compiled messages. In these cases each compile keeps its nested history while its messages live, and memory increases with the square of the turns.

Options:

  1. Walk lists, tuples, and plain dicts or objects (exact types only, so no app code runs) to a fixed depth, with a cap on the count of nodes.
  2. Put a hard cap on the count of compiles that keep variables. Risk: on a busy server, a long agent loop that compiles one time loses its variables.

Acceptance: a gc test in TypeScript (v8.setFlagsFromString + WeakRef) and in Python (gc + weakref) with the history in a nested dict or object shows the same count of kept turn variables after 150 and after 300 turns. The existing td.prompt.variables tests continue to pass.

Activity

  1. ephraimduncan commented on Oct 30, 2026

    @ephraimduncan
    ContributorAuthor

    #112 now removes the variables of each compile 5 minutes after the compile, or after the last call that names it. The depth of the messages in a variable has no effect on this time. Thus a nested history does not keep the variables of each turn, and memory does not increase with the square of the turns. The structural rule that this issue was about is not in #112.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions