You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
sdk: compiled messages deeper than a list in a variable keep the variables of each turn #113
In the fix of #97 (#112), compile() looks for compiled messages in its variables: a message as a variable value, or as an item of a list (Python: a list or tuple) in a variable. When it finds one, the compiles two steps back remove their variables. Thus a chat history of compiled messages keeps the variables of its last two turns. In a test of 2000 turns, 2 turns kept their variables.
compile() does not look deeper in a variable. Examples: Python prompt.compile(context={"history": history}), or a TypeScript array of objects that hold compiled messages. In these cases each compile keeps its nested history while its messages live, and memory increases with the square of the turns.
Options:
Walk lists, tuples, and plain dicts or objects (exact types only, so no app code runs) to a fixed depth, with a cap on the count of nodes.
Put a hard cap on the count of compiles that keep variables. Risk: on a busy server, a long agent loop that compiles one time loses its variables.
Acceptance: a gc test in TypeScript (v8.setFlagsFromString + WeakRef) and in Python (gc + weakref) with the history in a nested dict or object shows the same count of kept turn variables after 150 and after 300 turns. The existing td.prompt.variables tests continue to pass.
#112 now removes the variables of each compile 5 minutes after the compile, or after the last call that names it. The depth of the messages in a variable has no effect on this time. Thus a nested history does not keep the variables of each turn, and memory does not increase with the square of the turns. The structural rule that this issue was about is not in #112.
Activity
ephraimduncan commentedon Oct 30, 2026
#112 now removes the variables of each compile 5 minutes after the compile, or after the last call that names it. The depth of the messages in a variable has no effect on this time. Thus a nested history does not keep the variables of each turn, and memory does not increase with the square of the turns. The structural rule that this issue was about is not in #112.