Skip to content

Feature: Play media and render HTML in the preview pane on Wayland - #198

Merged
MemerGamer merged 4 commits into
mainfrom
linux/wayland-previews
Oct 15, 2026
Merged

MemerGamer merged 4 commits into
mainfrom
linux/wayland-previews

Conversation

@MemerGamer

Copy link
Copy Markdown
Owner

Resolved / Related Issues

What changed

Audio/video and HTML previews now render in the preview pane on the native Wayland host, and on X11 as well. Neither depends on native window embedding. Note that X11 never had these previews: LinuxFiles doesn't reference Uno's MediaPlayer or WebView X11 add-ins, so on X11 media showed only metadata and HTML showed its source.

  • Media (libmpv, optional): frames come from the libmpv software render API and are drawn into a XAML WriteableBitmap, with play/pause, seek, and volume controls (volume is persisted). Ctrl+Space also toggles playback. Nothing is decoded until the user presses Play. The file is opened through PreviewFile (regular files only, inode pinned) and passed to mpv as /proc/self/fd/N. mpv runs with config, scripts, ytdl, and auto-loading disabled, the lavf demuxer only, protocol_whitelist=[file], and a container whitelist. Playlist expansion and path changes are rejected. Playback stops as soon as the selection changes.
  • HTML (WebKitGTK 4.1 + bubblewrap, optional): the app starts a copy of itself as a short-lived helper (--linuxfiles-html-snapshot). The helper renders the file in an offscreen WebKitGTK view and returns a PNG, which is shown in a scroll viewer. The web process runs in WebKit's bwrap sandbox. If bwrap or xdg-dbus-proxy is missing, or in Snap or Flatpak environments where WebKit would skip the sandbox, it refuses to render. JavaScript is off, and navigation, downloads, and network access are blocked: a CSP is injected, and the preconnect and prefetch features, which bypass CSP, are disabled. The snapshot covers only the bounded visible region. When rendering is unavailable, the existing source-text fallback is shown.
  • Release notes: not changed. Wayland still opens the page in the browser. The reasons and options are in the design note.
  • Design note: docs/linux-port/wayland-previews.md covers the media and HTML design, why I didn't use wl_subsurface native hosting, and what interactive HTML would cost (WPE WebKit is the recommended follow-up).
  • Packaging: AUR optdepends gain mpv and webkit2gtk-4.1. The Nix flake doesn't include either library yet, so both previews fall back there (see the note).

Steps used to test these changes

  1. dotnet build src/Files.App -f net10.0-desktop and dotnet build tests/Files.Platform.Tests -warnaserror both build clean.
  2. Files.Platform.Tests on a private virtual KWin: 1450/1450 passed. The new tests cover mpv rendering, resize, dispose races, disguised playlists, FIFO rejection, the helper's argument and process handling, JS disabled, zero connections to a loopback listener (including preconnect), the height crop, background trimming, sandbox refusal, and cancellation.
  3. Native Wayland host on a private kwin_wayland --virtual: mp4, webm, and mp3 play; a disguised .mp4 playlist shows "could not be played"; a selection change stops the player threads; the HTML preview renders; the Markdown preview is unchanged. During rendering I saw WebKitWebProcess running under /usr/bin/bwrap.
  4. X11 host on private Xvfb (scripts/linux/headless-run.sh): mp4 plays and the HTML preview renders.
  5. Codex reviewed the change twice. It found no blockers, and all of its should-fix findings are addressed (bounded snapshot, playback stop, pinned media fd, sandbox environment gaps, footer trimming, debounce).

Not verified: NativeAOT publish, a real desktop session with audio output, HiDPI in the full app (the helper's 2x output is covered by tests).

🤖 Generated with Claude Code

https://claude.ai/code/session_01BTZhMsNyJ6k4VC68CNb5WU

MemerGamer and others added 4 commits October 15, 2026 19:43
Renders frames with the libmpv software render API into a XAML image, so
playback works on the native Wayland host as well as X11 without native
window embedding. libmpv is optional; without it the metadata preview stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: GPT-6.1-sol (OpenAI Codex) <noreply@openai.com>
A short-lived helper process (the app started with a hidden switch)
renders the document with an offscreen WebKitGTK 4.1 view and returns a
PNG, so HTML previews work on the native Wayland host and on X11 without
embedding a native window. Scripts, navigation and network access are
blocked; without WebKitGTK the source text fallback stays.

Also documents the preview design and lists mpv and webkit2gtk-4.1 as
optional AUR dependencies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: GPT-6.1-sol (OpenAI Codex) <noreply@openai.com>
- Run the WebKit web process in its bubblewrap sandbox and refuse to render
  when bwrap or xdg-dbus-proxy is missing.
- Snapshot only the bounded visible region instead of the full document,
  and trim trailing background rows.
- Open media through PreviewFile and hand mpv the pinned descriptor.
- Stop playback as soon as the preview pane moves to another selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Refuse HTML rendering where WebKit skips bubblewrap (Snap, Flatpak,
  WEBKIT_DISABLE_SANDBOX_THIS_IS_DANGEROUS).
- Trim only trailing rows in the page background colour, so solid footers
  are kept.
- Cancel the current preview as soon as the selection changes instead of
  after the debounce, so playback stops while navigating.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MemerGamer
MemerGamer merged commit 8b71aa7 into main Oct 15, 2026
3 checks passed
@MemerGamer
MemerGamer deleted the linux/wayland-previews branch October 15, 2026 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant