Skip to content

fix(recording): preserve explicit macOS session surfaces - #3217

Merged
thymikee merged 3 commits into
callstack:mainfrom
janicduplessis:codex/record-macos-session-surface
Oct 7, 2026
Merged

thymikee merged 3 commits into
callstack:mainfrom
janicduplessis:codex/record-macos-session-surface

Conversation

@janicduplessis

@janicduplessis janicduplessis commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

open --platform macos --surface desktop loses its surface in recorded actions and saved replay scripts, so replay can target a different surface. Preserve explicitly supplied values in the existing recorder, journal projection, and .ad open round trip; keep omission unchanged. Closes #3216.

The recording-policy migration preserved the old allowlist's omission; macOS surface selection needs to survive recording.

Validation

Tested five-file commit f90a483 with an actual SessionStore event/script regression for all four surfaces and omission, plus malformed replay arguments. Without the recording fix, explicit journal values disappear; without the serializer fix, replay loses them. Both regressions fail before their fix and the complete focused family passes (42 tests).

pnpm check:affected --run passed on this exact commit: formatting, lint, types, layering, Fallow, build, and 5,170 related tests across 660 files. Live macOS replay and GitHub CI are pending; the shared installed daemon remains unchanged.

Review in cubic

@janicduplessis

Copy link
Copy Markdown
Contributor Author

Fresh review of f90a483 against 2212eac: no confirmed blocking issues in the five-file diff. The canonical recorder now preserves explicitly supplied surface values, and the open script serializer/parser round-trip all four values while retaining omission. Tests exercise the actual SessionStore event/script boundary and reject malformed surface arguments. Published head/base and validation claims match the reviewed source and terminal check evidence. Live macOS replay remains unverified; this review does not count it as a pass. Named Claude and codex-rescue review tools are unavailable in this session; this is an independent Codex review.

@thymikee

thymikee commented Oct 5, 2026

Copy link
Copy Markdown
Member

The code looks right at f90a483, but the Coverage check fails and live macOS evidence is still missing. The failure is caused by this diff: the new import of parseSessionSurface at open-script.ts:2 adds two modules (contracts/src/facades/session.ts and contracts/src/session-surface.ts) to the eager closure. That moves ad-script from 41 to 43, ad-replay from 62 to 64, and command-registry batch.ts from 80 to 82, so all 30 tests in eager-closure-budgets.test.ts fail. I did not run the gate locally. This is from the CI log chains and a read of the head sources.

Please keep the surface check but stop adding modules to the closure. One way is import type { SessionSurface } plus a Record<SessionSurface, true> table, so a new surface becomes a compile error instead of silent drift. In that case, throw AppError('INVALID_ARGS', ...) from kernel/errors, and please check that this module is already in the closure. Another way is to parse the value leniently and let replay dispatch refuse bad values through parseSessionSurface in src/platform-runtime-open-target.ts. That needs a typed boundary, not a cast, and it moves the refusal from parse time to replay time. A new @agent-device/contracts/session-surface subpath would still add one module and still fail.

The PR also changes replay on a device route: open ... --surface desktop|menubar|frontmost-app scripts now replay with that surface, where before they replayed as an app session. The tests stop at the SessionStore and parser boundary, and I traced the replay path only by reading code, not by running it. Please run this on a macOS host: open <app> --platform macos --surface desktop --save-script, close the session, then replay the saved .ad file. The output should show that the replayed session's surface is desktop and that the saved script contains --surface desktop. Please repeat it once for menubar. Then run it once with the surface omitted: the replayed surface must stay app, and --surface must not appear in the saved script.

Not blocking, take or leave: --surface=desktop (equals form) is not parsed and falls through as a positional, like the other open flags, but it is still a silent mis-parse. The new sentence in website/docs/docs/sessions.md sits mid-paragraph inside the events.ndjson privacy text, so it reads as part of that text. It would fit better in the open --surface or script-format docs.

No conflicts. Before merge, the new import in open-script.ts must go so the eager-closure gate passes, and the live macOS record, save, and replay evidence above must be posted.

@thymikee

thymikee commented Oct 6, 2026

Copy link
Copy Markdown
Member

Live macOS replay check

Head: f90a48311 (built from a clean worktree). Host: macOS 26.6.2 (25G83). App: Calculator. CLI: node bin/agent-device.mjs from this checkout, with an isolated state dir.

Each leg used a fresh session name with no active sessions. I ran open ... --save-script, snapshot -i, and close. Then I ran replay <file>.ad --keep-session --json and read the surface with appstate --json. --keep-session only suppresses the script's close, so the replayed session stays open for the read.

Result: all three legs pass.

Leg Saved open line Replayed appstate.surface
desktop open --surface desktop desktop
menubar open "Calculator" --surface menubar menubar
omitted open "Calculator" (no --surface) app

desktop

open Calculator --surface desktop fails with INVALID_ARGS: open --surface desktop does not accept an app target. That is existing behavior. So this leg uses open --platform macos --surface desktop --save-script desktop.ad.

context platform=macos device="<host>" kind=device theme=unknown
open --surface desktop
close

Replay: replayed: 1. appstate after replay: "surface": "desktop". The replayed open action in events.ndjson has flags: {"platform":"macos","surface":"desktop"}.

menubar

open Calculator --platform macos --surface menubar --save-script menubar.ad

context platform=macos device="<host>" kind=device theme=unknown
open "Calculator" --surface menubar
close

Replay: replayed: 1. appstate after replay: "surface": "menubar", appName: "Calculator". The replayed open action has flags.surface: "menubar".

surface omitted

open Calculator --platform macos --save-script app.ad

context platform=macos device="<host>" kind=device theme=unknown
open "Calculator"
close

The script does not contain --surface. Replay: replayed: 1. appstate after replay: "surface": "app". The replayed open action has flags: {"platform":"macos"} and no surface key.

Side note: surface from a config file

I also set AGENT_DEVICE_CONFIG to a file with {"surface":"desktop"} and ran open --platform macos --save-script with no flag. The saved script contains open --surface desktop. So a surface from config is recorded the same as a CLI flag. This keeps replay faithful. But the docs line says "explicitly supplied", which can read as "CLI flag only". Please confirm this is the intended behavior.

@thymikee

thymikee commented Oct 6, 2026

Copy link
Copy Markdown
Member

I pushed two commits to finish this PR:

  • 87a4b1a fixes the eager-closure failure. open-script.ts now imports SessionSurface as a type only and checks the value against a Record<SessionSurface, true> table. A bad value throws AppError('INVALID_ARGS') from kernel/errors, which ad-script already loads. All 30 eager-closure-budgets tests failed without it and pass with it.
  • c98897a moves the docs sentence out of the events.ndjson privacy paragraph in sessions.md. It is now beside the open --surface bullet in commands.md. It also says that a surface from a config file is saved the same way as a flag, which is what the live run showed.

I left the --surface=desktop equals form as it is. The other open replay flags do not parse that form either.

CI passes. It is still a draft, so please mark it ready when you agree.

@thymikee

thymikee commented Oct 6, 2026

Copy link
Copy Markdown
Member

This PR is ready at c98897a. The earlier evidence gap is closed, and the checks are green: 15 checks, none failing, including the Coverage job that runs the eager-closure budgets. Commit 87a4b1a removes the runtime contracts import that overlapped that job.

Not blocking: the replay surface parser in packages/ad-script/src/internal/open-script.ts (https://github.com/callstack/agent-device/blob/c98897a/packages/ad-script/src/internal/open-script.ts#L97) repeats the trim, lowercase and "Invalid surface" message logic from parseSessionSurface in packages/contracts/src/session-surface.ts. The Record<SessionSurface, true> typing keeps the value list in sync, but the normalization and message text can still drift. If the closure budget is raised, or the contracts parser gets a lighter entry, you could delete the local copy and import parseSessionSurface. Take it or leave it.

Evidence limits: I did not run the eager-closure-budgets tests locally. That pass rests on the CI summary and a read of the import graph. The live macOS run was on f90a483, not c98897a. I judged the delta not to touch the device route, since it only swaps the parser and edits docs, but the new parser was not live-run. I did not reproduce the "all 30 tests failed without it" claim. There are no review threads on this head.

No conflicts. The PR is still a draft, so you or a maintainer can mark it ready for merge.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 6, 2026
@janicduplessis
janicduplessis marked this pull request as ready for review October 7, 2026 03:46
Copilot AI balanced review requested due to automatic review settings October 7, 2026 03:46

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/ad-script/src/internal/open-script.ts">

<violation number="1" location="packages/ad-script/src/internal/open-script.ts:113">
P3: A script line ending in `--surface` (or `--surface` followed directly by another flag) passes `undefined` to `parseReplaySurface`, so the error reads `Invalid surface: undefined. Use app|frontmost-app|desktop|menubar.` — the message leaks the word `undefined` instead of saying the value is missing. Special-case the missing-value case with its own message before validating the value.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic

if (normalized !== undefined && isReplaySurface(normalized)) return normalized;
throw new AppError(
'INVALID_ARGS',
`Invalid surface: ${value}. Use ${Object.keys(REPLAY_SURFACES).join('|')}.`,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: A script line ending in --surface (or --surface followed directly by another flag) passes undefined to parseReplaySurface, so the error reads Invalid surface: undefined. Use app|frontmost-app|desktop|menubar. — the message leaks the word undefined instead of saying the value is missing. Special-case the missing-value case with its own message before validating the value.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At packages/ad-script/src/internal/open-script.ts, line 113:

<comment>A script line ending in `--surface` (or `--surface` followed directly by another flag) passes `undefined` to `parseReplaySurface`, so the error reads `Invalid surface: undefined. Use app|frontmost-app|desktop|menubar.` — the message leaks the word `undefined` instead of saying the value is missing. Special-case the missing-value case with its own message before validating the value.</comment>

<file context>
@@ -84,6 +94,26 @@ export function parseReplayOpenFlags(args: string[]): {
+  if (normalized !== undefined && isReplaySurface(normalized)) return normalized;
+  throw new AppError(
+    'INVALID_ARGS',
+    `Invalid surface: ${value}. Use ${Object.keys(REPLAY_SURFACES).join('|')}.`,
+  );
+}
</file context>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Surface parsing duplicates the canonical contract, and public documentation and installed help need correction.

Review effort: Balanced
Findings: 1 Medium severity · 2 Low severity

Open (3)
What changed in this PR

Preserves explicit macOS surfaces across action recording, journal events, and saved replay scripts while retaining omission semantics.

Changes:

  • Records and serializes explicit open --surface values.
  • Validates replayed surface arguments.
  • Adds regression coverage and documentation.
File Description
website/​docs/​docs/​commands.md Documents saved-script surface behavior.
src/​daemon/​__tests__/​session-store.test.ts Tests recording and round trips for all surfaces.
packages/​command-registry/​src/​flag-definitions-target.ts Enables surface recording.
packages/​ad-script/​src/​internal/​open-script.ts Serializes and parses replay surfaces.
packages/​ad-script/​src/​internal/​__tests__/​open-script.test.ts Tests malformed surface rejection.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +97 to +102
const REPLAY_SURFACES: Record<SessionSurface, true> = {
app: true,
'frontmost-app': true,
desktop: true,
menubar: true,
};
usageDescription: 'macOS session surface for open (defaults to app)',
projectConfig: true,
recorded: false,
recorded: true,
- `open <app> --launch-console <path>` captures launch-time stdout/stderr for direct iOS simulator app launches. It is not valid for URL opens or
non-simulator targets.
- `open --platform macos --surface app|frontmost-app|desktop|menubar` selects the macOS session surface explicitly. `app` is the default when an app argument is provided.
- A surface given to `open`, as a flag or from a config file, is saved in `--save-script` scripts, so `replay` reopens the same surface. When no surface is given, the script has no `--surface`, and replay opens an app session.
@thymikee
thymikee merged commit f01915d into callstack:main Oct 7, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Explicit macOS surface is lost from recorded actions and replay scripts

3 participants