Repository navigation
feat: Add support for file-based overrides. - #2105
Merged
Merged
Conversation
…rm filesystem (#2062) ## Summary Adds two small capabilities to the platform filesystem abstraction, with no callers. They are the platform prerequisites for the shared file data engine in the next PR, split out so that PR contains only the engine and the file data source changes. - `Filesystem.getFileStats` (optional): returns a file's modification time and size, or `undefined` when the file does not exist, so a component can detect that a file changed, appeared, or disappeared. The Node implementation maps both a missing path and a path that uses a regular file as a directory to `undefined`, because Windows reports the second case as a missing path and the behavior should not differ by platform. Any other failure rejects. - Watch error reporting: `NodeFilesystem.watch` attaches an `error` listener to the `FSWatcher` and reports the failure through the watch callback with the event type `error`. Without a listener, an `error` event on an `FSWatcher` is an uncaught exception. The `Filesystem.watch` documentation states the convention so other implementations can follow it. Tests cover the stats for an existing file, a missing file, a file used as a directory, a permission failure (skipped on Windows and when running as root), and the error routing. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Extends the platform **Filesystem** abstraction with optional **`getFileStats`** (mtime + size, or `undefined` when the path is absent) and documents that **`watch`** should surface watcher failures via an **`error`** callback event instead of throwing. > > **`NodeFilesystem`** implements **`getFileStats`** with cross-platform semantics (`ENOENT` / `ENOTDIR` → `undefined`, other errors reject) and wires **`FSWatcher`** **`error`** events into the watch callback. New unit tests cover stats edge cases and watch error routing. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit b0ab055. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…ck (#2068) ## Summary Adds one capability to the platform filesystem abstraction, with no callers: the watch callback gains an optional third argument, the name of the entry within the watched path that changed, when the platform reports it. `NodeFilesystem.watch` passes the name that `fs.watch` supplies, or `undefined` when there is none. The existing two arguments keep their meaning, so existing callers are unaffected. A component that watches a directory rather than a file can use the name to ignore events for entries it does not own. The shared file data engine in the next PR does this, so that a change to an unrelated file in the same directory does not cause a reload. Follows #2062, which added the other platform prerequisites and has merged. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Extends the platform **`Filesystem.watch`** callback with an optional third argument, **`changedName`**, so directory watchers can tell which entry changed when the OS reports it. > > **`NodeFilesystem`** forwards Node’s `fs.watch` filename (or `undefined` when the platform passes a non-string). The first two callback arguments are unchanged, so existing two-argument handlers remain valid. Docs on the shared interface describe the new parameter; a unit test covers both named and unnamed change events. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit f71f31d. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
## Summary This is the first of the preliminary changes for the flag overrides feature described by the OVERRIDE specification. It adds a shared file data engine to the server SDK common package, so that the existing file data source, the FDv2 file data initializer, and the file-based override source that a later change adds share one reading, reloading, and merging implementation. How a source turns document data into flags stays with that source, in an explicit policy that the engine takes as input. This change adds the engine and nothing else. No source uses it yet; the existing sources keep their current loader until #2049 moves them onto the engine. The split keeps each change reviewable on its own: this one is the engine and its tests, and #2049 is the two sources' policies and the behavior they gain. ### The shared engine `packages/shared/sdk-server/src/data_sources/filedata/` holds what every file-based source needs: - `FileReloader`: runs one load at a time; waits for a burst of change notifications to settle before reloading, and reloads no later than one second after the first notification of a burst that does not settle; retries a failed load after a short delay; keeps the last good data by not applying a failed load and leaves its internal state intact; skips applying content identical to the last applied content; and reports every read, parse, and merge failure through an `onFailure` callback as a typed `LoadFailure` (kind `read`, `parse`, or `merge`, the original error, the path, and whether the failure repeats an earlier one). Its load never rejects, so no filesystem error can become an unhandled rejection. - `FileDirectoryWatcher`: watches the directories that contain the files, so a file replaced by a rename, deleted and recreated, or created after start is detected. An event that names an entry which is not one of the configured files causes a reload only when the metadata of a configured file in that directory changed, read through any symbolic link, so a mounted ConfigMap or Secret update (a link swap that never names the file) is detected while a busy sibling file costs no reload; when the platform does not report which entry changed, every event in the directory counts. Each configured file is also watched directly, so a path that is a symbolic link to a file in another directory is followed; that watch is set up again after each event from it, after a directory event that names the file, and after a change to the file's metadata seen through another entry, so it survives the file being replaced and follows a link that is retargeted. A directory that cannot be watched is retried. A watched directory that is deleted has its watch closed and set up again once the directory exists; a watch that reports an error is replaced the same way. - `mergeDocuments`: combines the documents of the configured files in order and lets the policy resolve duplicate keys. - One document parser, `parseDocumentByExtension`: the format is chosen by file extension only, which is the rule the file data sources already used. ### The injected policy The engine takes a `FileDataPolicy`, one per source, with five members: `parseDocument(path, data)`, `makeFlagWithValue(key, value, previous)`, `entryKey(mapKey, entry)` (the key under which a `flags` or `segments` entry is stored and checked for duplicates), `resolveDuplicateKey(category, key)` (returns `keepFirst` or `keepLast`, or throws to fail the load), and `missingFile` (`fail` or `skip`). The policies of the file data source and the FDv2 initializer are defined in #2049, each in its source's file, so the differences between the sources are visible in one place per source. The override source supplies its own policy in a later change. ### Tests The engine's tests cover the parsers, the merge, the reloader, and the watcher, including a deleted and recreated directory, a flood of events for a deleted directory, a watch error, the maximum debounce delay, and the repeat key. `MockFilesystem` and `testPolicy` are test helpers that the tests of #2049 also use. The platform additions the engine relies on, `Filesystem.getFileStats` and watch error reporting (#2062) and the changed entry name in the watch callback (#2068), have merged. [SDK-3251](https://launchdarkly.atlassian.net/browse/SDK-3251) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Adds a new **shared file data engine** under `packages/shared/sdk-server/src/data_sources/filedata/` so file-based flag sources can share one read/reload/merge path. Nothing in the SDK wires it up yet; existing sources keep their loaders until a follow-up. > > **`FileReloader`** loads configured paths in order, runs one reload at a time, debounces change triggers (with a max delay cap), retries failures, optionally skips unchanged content, and reports typed `LoadFailure`s via `onFailure` while keeping the last good state when a load fails. > > **`FileDirectoryWatcher`** watches each file’s parent directory (deduped) plus a direct watch per path. It filters noisy directory events using file metadata (for ConfigMap-style link swaps), renews watches after errors/replaces, retries missing directories, and exposes `verify()` after failed loads. > > **`mergeDocuments`** merges parsed documents using an injected **`FileDataPolicy`** (parse, `flagValues` expansion, duplicate keys, missing-file behavior). **`parseDocumentByExtension`** picks JSON vs YAML by extension. > > The PR adds broad unit tests (`MockFilesystem`, `testPolicy`) for parsing, merge, reloader, and watcher edge cases (deleted dirs, watch floods, in-flight metadata checks, strict ENOENT on file watches). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 1784d77. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> [SDK-3251]: https://launchdarkly.atlassian.net/browse/SDK-3251?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
) ## Summary This change moves the existing file data source and the FDv2 file data initializer onto the shared file data engine added in #2082, so that they and the file-based override source that a later change adds share one reading, reloading, and merging implementation. How a source turns document data into flags stays with that source, in an explicit policy. The previous loader, `FileLoader`, is removed. ### The injected translation The engine takes a `FileDataPolicy`, one per source; #2082 describes its members. Each policy is defined in its source's file, so the differences are visible in one place per source. `fileDataSourcePolicy` in `FileDataSource.ts`: - parser chosen by file extension; a YAML file without a configured parser fails with the existing message - a `flagValues` entry becomes a flag that is on and serves the value by fallthrough; its version starts at 1 and increments only when the value changes between loads, using the flag the last successful load produced for the same key - a flag or segment entry is keyed by its own `key` property, as the source has always stored it, so two entries with the same `key` property are duplicates whatever their map keys - a key that appears more than once fails the load with `found duplicate key: "<key>"` - a configured file that does not exist fails the load `fileDataInitializerPolicy` in `fileDataInitilizerFDv2.ts`: - parser chosen by file extension - every `flagValues` entry gets version 1; the initializer runs once and has no previous load - a flag or segment entry is keyed by its map key, as the initializer has always merged it - the last definition of a duplicate key wins, across files and between `flags` and `flagValues` in one file - a configured file that does not exist fails the load The later change that adds the override source supplies `fileOverrideSourcePolicy`: extension-based parsing with validation of the document shape, a `flagValues` entry expanded to an off flag, the configured `fail` or `ignore` duplicate handling, and a missing file that contributes nothing. ### Robustness the existing file data source gains The observable data semantics of the file data source and the FDv2 initializer are unchanged: the same documents produce the same flags, versions, and duplicate handling; the same errors reach the error handler; and the same messages are logged where they still apply. The unmodified baseline tests pass, and new tests pin the extension-based detection, the no-parser error text, and the documents the sources accepted before. With automatic updates on, the file data source now behaves differently in these ways: - A filesystem error during a reload (for example a file removed while it is being read) cannot become an unhandled promise rejection. Every failure is reported through the error handler. - The directories that contain the files are watched instead of the files themselves, so a rename-based write, as editors, deploy tools, and mounted ConfigMaps perform, is detected, and so is a file deleted and created again. - A watched directory that is deleted does not leave a dead watch behind. After each event from a watch, and after each failed load, the watcher checks that the directory still exists. A directory that no longer exists has its watch closed at once, which also ends the stream of events that Windows delivers for a deleted directory, and the watch is set up again through the existing one-second retry once the directory exists. When it is, one reload runs so that changes made in the meantime are picked up. An `error` event from the Node `FSWatcher` is routed to the same replacement instead of surfacing as an uncaught exception. - Change notifications settle for 100 ms before a reload, coalescing the burst of events one edit produces. A stream of notifications that never settles cannot postpone a reload without end: a reload runs no later than one second after the first notification of the burst. The previous per-file timestamp check is replaced by a comparison of the loaded content, so an unchanged file still does not reinitialize the store. - A failed load, for example a file read while it was being written, is retried after one second. The last good data stays in effect and the internal version baseline is intact, so the next successful load continues the version sequence. - A file that is missing at start is a load failure reported through the error handler, as before, and the retry loads it when it appears. - A read failure is now logged at error level as `Error loading files: <error>`; before it reached the error handler only. Processing failures keep `Error processing files: <error>`. Only an automatic retry can be a repeat: a failure that repeats on retry is logged at debug level and not reported to the error handler again, while a failure after a change notification is always reported, even when it matches the previous one. Consecutive retry failures of the same kind on the same path count as repeats regardless of their message, so a file that is written slowly, and fails to parse at a different position on each retry, is reported once. - An exception while the merged result is translated into store data, for example a flag entry that is not an object, is reported through the error handler as `Error processing files: <error>` and retried, as the old `_processFileData` catch did. Nothing from that load is remembered as the last good result, and the reload promise never rejects, so a bad entry can neither raise an unhandled rejection at startup nor stop later reloads. Without automatic updates the source loads once, as before. The FDv2 initializer loads once, with no watching, debouncing, or retry. ### Tests The baseline test files pass with one implementation-detail change in `FileDataSource.test.ts`: the tests that registered or fired a mocked `Filesystem.watch` per file path now do so on the files' directory, because the directory is what is watched. New tests cover each robustness change on the file data source, the two legacy policies, and the format-detection pins. The Node package gains a test over a real directory that is deleted and created again, and tests for the `FSWatcher` error routing and the `getFileStats` failure mapping. [SDK-3251](https://launchdarkly.atlassian.net/browse/SDK-3251) [SDK-3251]: https://launchdarkly.atlassian.net/browse/SDK-3251?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **Moves the file data source and FDv2 file data initializer onto the shared `filedata` engine** (`FileReloader`, `FileDirectoryWatcher`), removes `FileLoader`, and keeps per-source semantics in explicit `FileDataPolicy` implementations (`fileDataSourcePolicy`, `fileDataInitializerPolicy`). > > For **`FileDataSource` with `autoUpdate`**, watching and reloading change materially: parent **directories** are watched (plus direct watches on configured paths for symlinks), changes are **debounced** and **retried** on failure while last-good data and flag versions stay intact, deleted directories get watches torn down and re-established, and failures route through the error handler without unhandled rejections. **`FileDataInitializerFDv2`** uses the same loader path for a single startup read (no watch/retry). **`autoUpdate` docs** in `FileDataSourceOptions` describe the new behavior. > > **Tests** add policy/format/robustness coverage in `sdk-server` and integration-style Node tests (real dirs, symlinks, ConfigMap-style link swaps); existing `FileDataSource.test.ts` expectations shift to directory-level watch callbacks. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 8dda675. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
## Summary Adds `FilePoller` to the shared file data engine. It observes the configured files on a fixed interval through `Filesystem.getFileStats` and reports a change when any file's modification time or size changes, or when a file appears or disappears. Polls never overlap, and a callback that arrives after `close` is dropped. Nothing in this PR uses the poller. The file-based override source added later uses it as its default change detection, because polling behaves the same on every platform and filesystem where directory watching does not. Split out of the engine PR so that PR contains only what the existing file data sources use. [SDK-3251](https://launchdarkly.atlassian.net/browse/SDK-3251) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Introduces **`FilePoller`**, an internal interval-based change detector for the shared file data engine when directory watching is unavailable or unreliable. On each tick it compares **modification time and size** (via `Filesystem.getFileStats`, or **`getFileTimestamp` only** when stats are unsupported) across one or more paths and fires a single **`onChange`** when anything appears, disappears, or its observed metadata shifts. > > **Lifecycle behavior** is explicit: `start()` baselines state before scheduling polls; polls **do not overlap**; `close()` clears timers and **suppresses callbacks** from examinations that finish after shutdown. The class is **re-exported** from `filedata/index.ts`; **no existing callers** are added in this PR. > > Coverage is added in **`FilePoller.test.ts`** (mock filesystem, fake timers) for no-op intervals, multi-file batching, metadata fallback, and close/race cases. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 6d3ca83. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> [SDK-3251]: https://launchdarkly.atlassian.net/browse/SDK-3251?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
#2050) ## Summary This PR is based on the shared file data branch because it builds on the same feature branch history and should merge after it. This change adds the model and evaluator work that the flag overrides feature of the OVERRIDE specification builds on. Nothing in the SDK sets the marker yet, so the SDK's behavior is unchanged until the override store lands. Flag and segment definitions can carry an override marker. The Node SDK stores definitions in their JSON data model form, so the marker lives under a reserved, namespaced key. Only the new `overrideMarker` module names that key. Other code reads or strips the marker through its functions. The serializer omits the marker at the root of a flag or segment, so it never leaves the process, and leaves nested values untouched. Definitions read from LaunchDarkly or from a store lose the key as they are ingested, so only the override store can set it. Only the SDK's override store will set it, on the entries it holds. The evaluator marks an evaluation as override-affected when any definition it read carries the marker: the evaluated flag, a prerequisite flag at any depth, or a segment consulted during matching, whether or not the segment matched. The marking propagates upward only. A prerequisite evaluation starts its own marker from its own definition and merges it into its parent's marker when it completes, so the record of a prerequisite reflects only the definitions that its own evaluation read. Error results the evaluator produces are marked too, so a malformed override definition yields the caller's default with a marked error reason. The client's own replacement of a result on a requested-type mismatch keeps the marking in the override layer change that follows, where the marker is first set. The evaluation reason gains an optional `overrideAffected` indicator that is set only when true, following the `inExperiment` precedent. The evaluation result carries the same marking as a scalar so that event generation can key on it without reading the reason. Migration operation events keep the indicator on their evaluation reason when it is true. The OpenFeature provider reports it in flag metadata, following the `inExperiment` precedent. The tests mirror the Go evaluation library's marking tests: direct marking for off, fallthrough, rule match, and error results, prerequisite records at depth one and two, an unaffected sibling prerequisite, segments matched, not matched, negated, referenced through another segment, and resolved through big segment membership, unresolvable definitions, and the agreement between the result scalar and the reason indicator. [SDK-3247](https://launchdarkly.atlassian.net/browse/SDK-3247) [SDK-3247]: https://launchdarkly.atlassian.net/browse/SDK-3247?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Introduces experimental **flag override** plumbing: flag/segment definitions can carry an internal `_sdk_override` marker (set only by the future override store), and evaluations expose **`overrideAffected`** on the reason and result when any read definition was override-sourced—directly or via prerequisites/segments at any depth. > > The **evaluator** tracks marking per evaluation (including prerequisite records and segment reads, even on non-match/negated clauses), merges it up the prerequisite tree without mutating shared reason singletons, and omits `overrideAffected` from JSON unless it is `true` (same pattern as `inExperiment`). **Ingest/serialization** strips root `_sdk_override` from LD payloads and persistent-store round-trips while preserving nested customer data that happens to use the same key name. > > **OpenFeature** flag metadata and **migration op** event conversion forward `overrideAffected` when true. Until the override store lands, nothing sets the marker, so runtime behavior for normal LD data should stay the same aside from the new optional fields and strip-on-ingest safety. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit c97929f. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
## Summary This PR is based on the evaluator marking branch because it uses the marker and the marking that branch adds. This change adds the override layer described by the OVERRIDE specification to the FDv2 data system of the Node.js server SDK. An override source is configured with the new `overrides` property of the data system options, which accepts a source object or a factory function. A source is not a data source: it takes no part in the initializer and synchronizer pipeline and has no effect on initialization status, data availability, or data source status. The SDK starts it when the client is created, before the data source, passes it a sink, and closes it with the client. In offline mode the source is not started. The public `LDOverrideSource` and `LDOverrideSink` interfaces let an application supply a custom source. A configuration that is not a source fails client construction, the way an unsupported data source configuration does. The source supplies complete snapshots to the override store. Each snapshot replaces the whole store in one assignment, so the store holds exactly one snapshot at any instant. The store holds prepared, marked copies of the definitions and never modifies the objects the source supplied. An overlay at the store read boundary returns the override entry for a key in preference to LaunchDarkly data, and an enumeration includes every flag the store holds. Evaluation, prerequisite and segment resolution, and the all-flags read all go through the overlay. The client consults the override store before the not-initialized short circuit, and before it asks the data store anything: with a persistent store the initialization check is I/O, and an overridden flag must not wait on a store that is slow or unreachable. An overridden flag is served before the client has LaunchDarkly data, while a flag the store does not hold still gets the not-ready default. Evaluation never waits for the source: an evaluation reads the override store as it is at that moment. The source's initial load is part of starting the client, so the promise that `waitForInitialization` returns settles after the load as well, whether initialization succeeded or failed, bounded by the caller's timeout; for the file-based source that is a local read that completes within milliseconds of construction. A failed start of the source is logged and the client runs without overrides. The source is created before the data store, so an invalid override configuration fails construction before a persistent store has connected. The override layer checks the shape of every definition before it stores it. The SDK keeps definitions in their JSON form, so a hand-written definition with a field of the wrong type (a flag without `variations`, a target or clause without `values`, `rules` that are not an array, an `on` switch that is a string) would otherwise make evaluation throw instead of returning a malformed-flag result. The fields the SDK itself sets when it prepares a definition, such as the sets it generates for large target lists, cannot be supplied. A definition that fails the check is rejected with an error that names the kind, the key, and the field, and no part of that snapshot is applied: the layer keeps its previous contents. Only the key, `variations`, and `values` are required; every other field may be absent, as in LaunchDarkly data, so LaunchDarkly definitions, value-only overrides, and TestData flags all pass. The store's maps have no prototype, so a key such as `constructor` or `__proto__` is an ordinary key. The all-flags state keeps override-affected flags with their values, versions, and marked reasons, and presents them with `trackEvents` and `trackReason` false and no `debugEventsUntilDate`. While the client is not initialized and the store holds entries, the state is valid and holds only those flags. A type mismatch keeps the marking of the evaluation it replaces. Adding, changing, or removing an override fires the normal flag change notifications, including for the flags that depend on a changed flag or segment. The OVERRIDE specification's test vectors are vendored and run as a unit test through the full client stack, asserting the value, the variation index, and the reason. [SDK-3247](https://launchdarkly.atlassian.net/browse/SDK-3247) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Adds an **experimental** flag/segment override layer to the FDv2 server SDK via `dataSystem.overrides`, wired through public `LDOverrideSource` / `LDOverrideSink` APIs (source or factory). > > Override snapshots replace an in-memory `OverrideLayer` and are read through a `ReadStoreOverlay` so per-key overrides win over LaunchDarkly data without changing init or data-source status. `LDClientImpl` serves overridden flags before the not-ready short-circuit (including when the persistent store is slow), marks evaluations with `overrideAffected`, adjusts `allFlagsState` (override-only when uninitialized, strips tracking for affected flags), fans change notifications through `OverrideSink`, and makes `waitForInitialization` wait for the source’s initial load while evaluations do not. > > Definitions are validated and prepared atomically per snapshot; malformed snapshots are rejected and the previous layer is kept. OVERRIDE spec vectors and broad unit/integration tests cover precedence, dependencies, lifecycle, and configuration edge cases. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 27aa6a8. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> [SDK-3247]: https://launchdarkly.atlassian.net/browse/SDK-3247?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
) ## Summary This PR is based on the override store branch because it consumes the override marking that branch produces on evaluation results. This change adds the analytics event behavior of the OVERRIDE specification to the shared event code and the Node.js server SDK. An evaluation marked as override-affected produces no individual feature event and no debug event, even when the flag requests them. It is still counted in summary events like any other evaluation, in a separate counter that carries the `overrideAffected` marker. The marker is part of the summary aggregation key, so override-affected and other evaluations of the same flag, variation, and version accumulate into separate counters rather than collapsing together. The marker is present on a counter only when true, like the existing `unknown` marker. The client hands the marking to the event processor as a scalar on the input event, taken from the evaluation result. Event generation does not read the evaluation reason. The record of a prerequisite evaluation carries the prerequisite's own marking, so an unaffected prerequisite inside a marked top-level evaluation still produces its individual event, while a marked prerequisite record produces none and is counted under the marked counter for the prerequisite flag. The shared event summarizer and processor are also used by the client-side SDKs. The new field is optional and those SDKs never set it, so their output is unchanged. The OVERRIDE test vector runner now also asserts the marking of the summary counter that each evaluation contributes to, by flushing the events and reading the summary payload. [SDK-3247](https://launchdarkly.atlassian.net/browse/SDK-3247) [SDK-3247]: https://launchdarkly.atlassian.net/browse/SDK-3247?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Implements OVERRIDE-spec analytics behavior: evaluations marked **override-affected** are counted only in **summary** payloads, not as individual **feature** or **debug** events—even when `trackEvents` or debug windows would normally emit them. > > The SDK passes `overrideAffected` from the evaluation result into `InputEvalEvent` (not from the reason). **EventSummarizer** treats the flag as part of the aggregation key so marked and unmarked evaluations of the same flag/variation/version land in **separate counters**; counters include `overrideAffected: true` only when applicable. Prerequisite eval records use **each flag’s own** marking, so unaffected siblings in a prereq tree can still emit individual events while override-touched flags do not. > > Node server SDK wires this through `EventFactory.evalEventServer`, `Evaluator` prereq events, and `_sendEvalEvent`. Shared changes are backward-compatible (optional field; client SDKs do not set it). Tests cover factory/summarizer/processor behavior, end-to-end override flows, and OVERRIDE vector assertions on summary counter marking. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 3593897. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
## Summary
This PR is based on the events branch because it completes the feature
that the earlier branches build up, and it should merge after them.
This change adds the file-based override source described by the
OVERRIDE specification to the Node.js server SDK. It is configured as `{
type: 'file', paths: [...] }` in the `overrides` property of the data
system options, alongside the source object and factory forms that the
earlier change added.
The source reads one or more JSON or YAML files in the file data source
document format, with optional `flags`, `flagValues`, and `segments`
members, and combines their entries in the configured order. Duplicate
keys across files fail the reload by default, or keep the first file's
entry with the `ignore` handling. Change detection is one of two modes.
Polling, the default, examines the files once per second, with a one
second minimum, and works on every filesystem. Watching reacts to change
notifications for the directories that contain the files. Change
detection is in place before the initial load, so a change made between
the two is not missed.
A configured file that does not exist contributes no overrides, so a
file can be created later and deleting a file removes its overrides. A
file that exists but cannot be read or parsed fails that whole reload:
the last good overrides stay in effect, the failure is logged once, and
the source retries after one second. Every applied change is logged at
Info level with the overrides in effect and what each file supplied, and
an unchanged reload logs nothing. The initial load is part of starting
the client: the promise that `waitForInitialization` returns settles
after it, and an evaluation made before that reads the layer as it is at
that moment.
An invalid configuration fails client construction, the way an invalid
file initializer does: no file paths, an unrecognized change detection
mode, or a platform without filesystem support. An unrecognized
duplicate keys handling is logged and replaced by its default. A path
listed more than once is logged and the repeats are ignored. A polling
interval below the minimum is logged and raised to one second, and one
that is not a finite number or exceeds 2147483 seconds is logged and
replaced by the default.
The SDK does not include a YAML parser, so that it does not carry a
dependency that could conflict with the application's. YAML files are
read with the parser the application configures through the `yamlParser`
option, for example the `parse` function of the `yaml` package, as the
file data source already allows. A YAML file configured without a parser
can never load, so it fails client construction with a message that says
so, rather than failing on every reload. Neither the Node.js package nor
the shared server package gains a dependency; the `yaml` package is a
test dependency only.
The existing file data sources keep their current behavior. The override
source has its own document handling (format detection by file
extension, validation of the document shape, entries keyed by their map
key with a `key` field inside an entry set to that map key, so a
definition pasted under a new key overrides that key, value expansion to
a flag that is on and serves the value by fallthrough, missing-file
handling, retry) and none of it applies to the FDv1 `FileDataSource` or
the FDv2 file data initializer.
The tests mirror the Go file source tests over a mock filesystem:
initial load, YAML, ordered merge, duplicate handling, missing files
that appear and disappear, the Info log on each change, quiet watching
of an absent file, reloads in both modes, retention across a malformed
edit with recovery through the retry, and close. A server-node test
drives the source through the client on real files, including the
built-in YAML parser.
[SDK-3247](https://launchdarkly.atlassian.net/browse/SDK-3247)
[SDK-3247]:
https://launchdarkly.atlassian.net/browse/SDK-3247?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Overview**
> Adds an experimental **file-based override source** for
`dataSystem.overrides`, configured as `{ type: 'file', paths: [...] }`,
so apps can load flag/segment overrides from local JSON or YAML using
the same document shape as the file data source (`flags`, `flagValues`,
`segments`).
>
> The implementation wires **`FileOverrideSource`** through existing
file-data machinery (`FileReloader`, polling vs directory watching),
with override-specific policy: `flagValues` expand to on/fallthrough
flags, entries are keyed by map key, missing files contribute nothing,
failed reloads keep the last good snapshot with retry/logging, and
duplicate keys across files default to **fail** (or **ignore** with
precedence). **`createOverrideSource`** validates options at client
construction (paths, YAML without `yamlParser`, filesystem platform,
poll interval bounds). Public options are documented on
**`FileOverrideSourceOptions`**.
>
> Coverage includes unit tests on a mock filesystem, client-level Node
tests on real temp files, and a test-only **`yaml`** dependency for YAML
scenarios; production SDKs still require the app to supply a YAML
parser.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
889910b. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
## Summary This PR is based on the file-based override source branch because the test service configures that source. This change adds the `overrides` object of the SDK configuration to the Node.js server SDK contract test service, as the harness service specification describes, and advertises the `flag-overrides` capability. The object maps to the file-based override source of the FDv2 data system options: `filePaths` becomes `paths`, `duplicateKeysHandling` and `changeDetection` pass through when set, and `pollIntervalMs` becomes `pollInterval` in seconds. The harness sends `null` for an optional parameter it did not set, and the service treats null as unset, the way the Go service's pointer fields do. Overrides without a data system configuration are an error, because the override source is an option of the FDv2 data system. The shared harness parameter types in `contract-test-utils` gain the `overrides` object. The service reads its port from the `PORT` environment variable and still defaults to 8000, so several services can run side by side. Against the harness build that carries the flag overrides tests, the `flag overrides` suite runs 38 tests with none failing, and the full v3 suite with the existing FDv2 suppressions runs 7010 tests with none failing. [SDK-3247](https://launchdarkly.atlassian.net/browse/SDK-3247) [SDK-3247]: https://launchdarkly.atlassian.net/browse/SDK-3247?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Adds **file-based flag overrides** to the Node server SDK contract test service so the harness can run the `flag-overrides` suite. > > The service now advertises the `flag-overrides` capability and maps harness `overrides` config onto FDv2 `dataSystem.overrides` (`filePaths` → `paths`, optional duplicate-key handling, change detection, and `pollIntervalMs` converted to seconds). It wires in a **YAML parser** via the new `yaml` dependency because the SDK does not ship one. Overrides without `dataSystem` fail fast with an error; harness `null` optional fields are treated as unset. > > Shared **`SDKConfigOverridesParams`** is added to `contract-test-utils` on `ServerSDKConfigParams` so harness types match the service mapping. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit b146336. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…data tests (#2096) ## Summary Test-only cleanup of the file data engine tests that merged with #2082 and #2049. Several test files carried their own copies of a polling `waitFor` helper, a `sleep` helper, and an in-memory filesystem that duplicated the shared `MockFilesystem` the other file data tests already use. - One `waitFor` helper per package (`packages/shared/sdk-server/__tests__/waitFor.ts` and `packages/sdk/server-node/__tests__/waitFor.ts`). The server-node package does not depend on `@launchdarkly/js-sdk-common`, so its helper also exports `sleep`; the sdk-server tests use the `sleep` from js-sdk-common. - The two format detection tests use the shared `MockFilesystem` instead of an inline class. Neither test exercises the paths where the two mocks differed (missing files, timestamps, watches). - Every explicit timeout the tests passed is kept. No test changes what it checks. The open override stack (#2053) adds the same two helper files byte for byte, so the branches merge without conflict in either order. ## Tests sdk-server: 1347 passed, 7 skipped. server-node: 79 passed. Lint and format checks pass; the remaining lint warnings are pre-existing. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **Test-only refactor** that deduplicates helpers across file data source tests in `sdk-server` and `server-node`. > > Several tests dropped local copies of a polling **`waitFor`** helper and a **`settle`** sleep wrapper. They now import **`waitFor`** from package-level test modules (`__tests__/waitFor.ts` in each package). **`sdk-server`** uses **`sleep`** from `@launchdarkly/js-sdk-common`; **`server-node`** uses **`sleep`** from `@launchdarkly/js-server-sdk-common` where delays are needed. > > The two **format detection** tests no longer define an inline **`MockFilesystem`**; they use the shared **`./filedata/MockFilesystem`** and **`filesystem.set()`** instead of assigning **`fileData`** directly. > > **No production code or test assertions change**—only imports and shared utilities. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 06caa69. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
Contributor
|
@launchdarkly/js-sdk-common size report |
Contributor
|
@launchdarkly/js-client-sdk size report |
Contributor
|
@launchdarkly/js-client-sdk-common size report |
kinyoklion
marked this pull request as ready for review
October 15, 2026 20:52
joker23
approved these changes
Oct 15, 2026
## Summary Marks the public flag override API with the `@experimental` TSDoc tag: the `overrides` property of the data system options, `FileOverrideSourceOptions`, `LDOverrideSourceOptions`, `LDOverrideSource`, `LDOverrideSink`, and `overrideAffected` on `LDEvaluationReason`. Each already said "Flag overrides are currently experimental and subject to change." in prose; that sentence is kept and now follows the tag, which is the form the client-side debug override API uses. TypeDoc renders the tag as an `Experimental` badge on the declaration, and editors show it as a separate tag line in hover, so the status is visible without reading the comment body. ## Verification `typedoc` for `@launchdarkly/js-server-sdk-common` and `@launchdarkly/js-sdk-common` renders the badge on all six pages (`LDDataSystemOptions`, `FileOverrideSourceOptions`, `LDOverrideSourceOptions`, `LDOverrideSource`, `LDOverrideSink`, `LDEvaluationReason`) with no new warnings. Comments only; no code changes. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Adds the **`@experimental`** TSDoc tag to the public flag override surface so TypeDoc and IDE hovers show an **Experimental** badge, matching how other experimental APIs are documented. > > The tag is applied to **`overrides`** on data system options, **`FileOverrideSourceOptions`**, **`LDOverrideSourceOptions`**, **`LDOverrideSource`**, **`LDOverrideSink`**, and **`overrideAffected`** on **`LDEvaluationReason`**. Existing prose (“Flag overrides are currently experimental…”) is unchanged and now follows the tag instead of sitting inline in the main comment text. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 348c292. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
# Conflicts: # packages/sdk/server-node/contract-tests/src/index.ts # packages/shared/sdk-server/src/evaluation/Evaluator.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BEGIN_COMMIT_OVERRIDE
feat: Add experimental support for file-based overrides.
feat: Add optional file stats, watch error reporting, and the changed entry name to the platform filesystem interface
fix: Detect rename-based writes and recreated files in the file data source by watching the containing directories
fix: Re-establish the file data watch when a watched directory is deleted and created again
fix: Coalesce bursts of file change notifications into one file data reload
fix: Retry a failed file data reload and keep the last good data in effect
fix: Report file data read and processing failures through the error handler instead of an unhandled promise rejection
END_COMMIT_OVERRIDE
Every change in this PR has already been reviewed and merged through its own PR against
feat/overrides: #2062, #2068, #2082, #2049, #2067, #2050, #2051, #2052, #2053, #2054, #2096. This PR merges the branch intomainand adds nothing of its own.Note
Overview
Adds experimental file-based flag overrides on the FDv2 data system (
dataSystem.overrides), including a file source with JSON/YAML (injectable parser), polling or filesystem watching, and contract-test wiring for theflag-overridescapability.Override-affected evaluations are surfaced via
overrideAffectedon evaluation reasons (and OpenFeature flag metadata). Analytics omit individual feature/debug events for those evaluations but bucket them in separate summary counters;allFlagsStatecan strip event-tracking fields for overridden flags.File data loading is reworked around shared
FileReloader/FileDirectoryWatcher/FilePollermachinery: directory plus per-file watches (symlinks, atomic renames, Kubernetes ConfigMap-style..dataswaps), debounced reloads, automatic retry after failures while keeping last-good data, and safer error reporting. The platformFilesystemAPI gains optionalgetFileStats, watch error callbacks, and an optional changed entry name; Node implements these behaviors.Reviewed by Cursor Bugbot for commit ba50d5f. Bugbot is set up for automated code reviews on this repo. Configure here.