Skip to content

feat(chat): inline previews for agent-written images, with copy to the user's clipboard - #1993

Open
aarontrowbridge wants to merge 2 commits into
mainfrom
image-preview-copy
Open

aarontrowbridge wants to merge 2 commits into
mainfrom
image-preview-copy

Conversation

@aarontrowbridge

Copy link
Copy Markdown
Member

Important

Problem: Images the agent writes to the server's disk (slide equations, plots) can't be seen in the chat panel, and an image can't be copied out of chat. The chat iframe can't reach the server's disk, a bare <img> can't send the server credential, file:// is stripped by the sanitizer and blocked by the page's security policy, and vscode.env.clipboard only handles text.
Approach:

  • Markdown ![alt](/abs/path.png) images load through the existing /file/content route, using a loader the app registers (the same pattern as the file-ref resolver).
  • Every chat image is framed: click opens a full-size preview with Copy, and hovering shows a copy button.
  • Copy sends a new clipboard-image-write message over the bridge. The extension host writes the PNG natively, so the paste lands on the machine the user is sitting at, which can differ from the server that produced the image.

Changes

  • session-ui
    • New markdown-local-images: path detection, loader registry and cache, and the image-framing and loading passes.
    • markdown.tsx wires those passes in and adds click-to-preview.
    • image-lightbox.tsx: a full-size preview with Copy and Close buttons.
    • copyImageToClipboard in util/clipboard: uses the bridge inside the frame, ClipboardItem outside it, and converts non-PNG images to PNG first.
    • A paper (white) backing behind framed images, so transparent black-ink figures stay visible in dark theme.
  • app: AmicodeFileRefBridge also registers the local-image loader. It turns the base64 from /file/content into a data: URL, and handles SVG returned as text.
  • extension:
    • clipboard-image-write handler in chat_bridge.ts. It accepts PNG only, is capped at 24 MB, requires the panel to be visible, and replies with a clipboard-image-written ack.
    • Platform writers: macOS osascript (the file path is passed as an argument, never inserted into the script), Linux xclip, or wl-copy under Wayland. Windows is served through WSL.
    • The writer waits for the launched process to exit rather than for its output streams to close (xclip and wl-copy fork a background process that keeps them open).
    • The new message kind is added to both chat_panel.ts allowlists.
  • manifest.json refreshed; the drift gate passes.

Testing

  • session-ui: new markdown-local-images.test.ts covers path classification, loader cache and dedupe, retrying failed loads, stale-unregister safety, one frame per image, that a local path never fires a network request, and that a cached image is set synchronously when a block is rebuilt. The full suite gives 457 passes and 1 failure, in the CodeMirror selection-highlight test, which this PR doesn't touch.
  • extension: three new bridge tests (ack shape; rejected payloads; per-platform command selection with no path in the script). Full vitest gives 5600 passes. tsc is clean.
  • app and session-ui typechecks (tsgo) are clean on the materialized tree.
  • Not yet verified in a live panel, since this needs an engine rebuild and an extension install.

Notes

  • This covers the clipboard part of the fleet-location gap (the agent didn't know which machine the user was at). The rest of that gap is drafted as a separate issue.

…e user's clipboard

Markdown images with an absolute server path load through /file/content and
render framed in chat; click opens a full-size preview, and the hover/preview
Copy action writes a PNG to the OS clipboard via a new clipboard-image-write
bridge kind handled by the extension host (osascript / xclip / wl-copy) — on
the machine the user is at, not the server.
@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: f59f5b4a-14ed-4ca2-b598-6e3ddf126703
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Chat image frames and the full-size preview gain a Download button next to
Copy. Inside the chat iframe the PNG rides the existing save-file bridge kind,
so the extension host opens a native Save dialog (default ~/Downloads) on the
machine the user is at; unframed, an anchor download stands. The file keeps
the authored basename (or the alt text), saved as .png since both save and
copy rasterize to PNG. Adds the outline-download v2 icon and the
ui.message.download string.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant