Skip to content

fs: let glob walk a mounted virtual file system - #66719

Open
samuelmbabhazi wants to merge 1 commit into
nodejs:mainfrom
samuelmbabhazi:vfs/glob-reader
Open

samuelmbabhazi wants to merge 1 commit into
nodejs:mainfrom
samuelmbabhazi:vfs/glob-reader

Conversation

@samuelmbabhazi

Copy link
Copy Markdown

Since the native glob (#65392), fs.glob(), fs.globSync() and fsPromises.glob() find nothing in a mounted virtual file system: the walk reads the file system with libuv, which cannot see a mount point, where the JavaScript walk it replaced read through the fs hooks. On v26.11.1 the walk below finds the files; on main it returns [] for every API:

const myVfs = vfs.create();
myVfs.mkdirSync('/data/sub', { recursive: true });
myVfs.writeFileSync('/data/a.txt', '');
myVfs.writeFileSync('/data/sub/b.txt', '');
const cwd = `${myVfs.mount()}/data`;
fs.readdirSync(cwd, { recursive: true });   // [ 'sub', 'sub/b.txt', 'a.txt' ]
fs.globSync('**/*.txt', { cwd });           // v26.11.1: [ 'a.txt', 'sub/b.txt' ], main: []

This gives the native walk a reader. When the walk's cwd is under a mount, or one of its patterns names a mount absolutely, the hooked readdirSync(), lstatSync(), statSync() and realpathSync.native() serve its scandir, lstat, stat and realpath reads, so a mount and the real file system are walked by the same engine, with the same patterns, exclude (patterns and callback, with and without withFileTypes), maxDepth and followSymlinks semantics. A walk that reads nothing under a mount is unchanged.

Details:

  • src/glob/glob_walker.{h,cc}: an FsReader interface on WalkOptions, consulted by the guarded walk in place of libuv. A walk with a reader never runs free (the free walk and the listing prefetcher scan with raw libuv calls from pool threads) and tells two spellings of a directory apart by realpath, as on Windows: a virtual file system need not have inodes (the in-memory one numbers every stat() call afresh).
  • src/glob/node_glob.{h,cc}: JsFsReader calls a JavaScript function (op, path), like JsExcludeFilter calls the exclude callback. For the asynchronous APIs, a request with a reader runs each slice on the main thread from a SetImmediate, one batch per pull, so a walk still settles from a later loop turn and an iterator left early still cancels it. The pull's settlement is factored into Resolve()/Reject(), shared with the thread-pool path.
  • lib/internal/vfs/setup.js: the globReader(cwd, patterns) handler, which gives the walk its reader when the walk can reach a mount. A read that fails answers as a missing path, as the walk's own reads do. The reserved root sits under os.devNull, which is a character device, and the walk descends an absolute pattern from the root one segment at a time, so to the walk that one path is a directory holding the reserved root.
  • lib/internal/fs/glob.js: passes the reader to the binding.

The test covers the three APIs: relative patterns from a cwd under the mount (**, dot directories, symbolic links with and without followSymlinks, maxDepth), exclude as patterns and as a callback with and without withFileTypes, absolute patterns into the mount and into the reserved root from a real cwd, a .. pattern within the mount, the reserved root as cwd, the real file system while a VFS is mounted, a throwing exclude callback, early exit from the iterator, a tree larger than one batch, and the unmounted path.

Verified locally on Linux: the new test, every test/parallel/test-vfs-* test, test-fs-glob* and test-path-glob pass.

The native glob walks the file system with libuv, which cannot see a
mounted virtual file system: fs.glob(), fs.globSync() and
fsPromises.glob() find nothing under a mount point, where the
JavaScript walk they replaced read through the fs hooks.

Give the walk a reader. When its cwd is under a mount, or a pattern
names one absolutely, the hooked readdirSync(), lstatSync(), statSync()
and realpathSync.native() serve its reads, so a mount and the real file
system are walked alike, with the same patterns, exclusions, depth
limit and symbolic link rules. A walk with a reader runs on the main
thread; the asynchronous APIs advance it one batch per pull, from a
later loop turn as a thread-pool slice would settle in. A virtual file
system need not have inodes, so such a walk tells two spellings of a
directory apart by realpath, as on Windows. The reserved root sits
under os.devNull, a character device, and the walk descends an absolute
pattern from the root one segment at a time, so to the walk that one
path is a directory holding the reserved root.

Refs: nodejs#65392
Signed-off-by: Samuel Mbabhazi <samuelmbabhazi@gmail.com>
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. vfs Issues and PRs related to the virtual filesystem subsystem. labels Oct 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Welcome to Node.js, and thank you for your first contribution!

Before review, please take a moment to read:

Please make sure every commit is signed off. For a first pull request, GitHub Actions require collaborator approval and Jenkins CI must be started by a collaborator or triager, so an initial wait is normal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. fs Issues and PRs related to file-system APIs and the fs module. needs-ci PRs that need a full CI run. vfs Issues and PRs related to the virtual filesystem subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants