Skip to content

Resolve an empty store name to the default store - #1575

Open
dimitri-yatsenko wants to merge 4 commits into
masterfrom
fix/default-store-empty-name
Open

dimitri-yatsenko wants to merge 4 commits into
masterfrom
fix/default-store-empty-name

Conversation

@dimitri-yatsenko

@dimitri-yatsenko dimitri-yatsenko commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

Fixes #1574.

Why

The bare-@ store forms (<npy@>, <object@>, and the other store codecs) are documented to use the default store, but every insert, fetch, or garbage collection on such a column raised Storage '' is requested but not configured in stores. The declaration keeps the store name as "" so a codec can tell "store requested" from "no store"; Config.get_store_spec resolved the default only for None.

Once such columns can be written, a second problem appears: GarbageCollector compared each reference's recorded store name with the store it collects. Bare-@ references record "", so a collector bound to the default store by name would have counted their files as orphans, and collect(dry_run=False) would have deleted files that existing rows still reference.

What changes

  • Config.get_store_spec: an empty store name resolves like None, to stores.default, or to stores.filepath_default for filepath references.
  • GarbageCollector._references: a reference whose recorded store name is empty or missing matches the default store.

Tests

  • Unit: get_store_spec("") resolves to the default store and, for filepath references, to the filepath default; with no stores.default configured it raises stores.default is not configured.
  • Integration (TestBareAtUsesDefaultStore in test_gc.py): an insert into a <npy@> column raises when no default store is configured; a <npy@> column round-trips through the default store; and a destructive collection of the default store by name keeps the row's file and reports it as referenced. Without the gc.py change this test fails because the file is deleted; without either change the four tests of the default-store path fail.
  • Full suite: 1145 passed, 14 skipped.

The bare-@ forms (<npy@>, <object@>, and the other store codecs) keep the store
name as "" so a codec can tell "store requested" from "no store".
Config.get_store_spec treated only None as the default, so inserting, fetching,
or collecting such a column raised "Storage '' is requested but not configured
in stores". An empty name now resolves to stores.default, or to
stores.filepath_default for filepath references.

The garbage collector compared each reference's recorded store name with the
store it collects. Bare-@ references record "", so a collector bound to the
default store by name counted their files as orphans and, with dry_run=False,
deleted files that existing rows still reference. A reference with an empty or
missing store name now matches the default store.
@dimitri-yatsenko dimitri-yatsenko added this to the v2.3.5 milestone Oct 9, 2026
@dimitri-yatsenko dimitri-yatsenko added the bug Indicates an unexpected problem or unintended behavior label Oct 9, 2026
An empty store name means the default store only when stores.default is set.
Without it, get_store_spec and an insert into a <npy@> column must raise
"stores.default is not configured" rather than pick some store.
@lum-agilyti

Copy link
Copy Markdown

The default-store resolution and the named-collector fix look right to me, and the destructive GC test pins the second one nicely.

One thing I noticed while reading, which I may be missing context on: before this change, GarbageCollector(schema, store="") failed in the constructor because get_store_spec("") raised. Now it builds (the backend resolves to the default store), but self.store stays "", so in _references neither self.store is None nor (ref_store or self._default_store) == "" matches any reference. If I'm reading it right, collect(dry_run=False) would then treat every stored file in those schemas as an orphan. "" is also what the heading reports as attr.store for a bare-@ column, so it seems like a value a caller could plausibly pass. I haven't run this, it's from reading gc.py and settings.py, so please correct me if something upstream normalizes it.

A smaller question on semantics: bare-@ references record store "" rather than the store they were written to, so they follow whatever stores.default is at read or collect time. Is that intended if stores.default is later re-pointed?

… collector

GarbageCollector(schema, store="") built a collector whose store stayed "",
so _references matched no reference and collect(dry_run=False) would have
deleted every file in the default store. An empty name now builds the default
collector, as None does.

Codecs recorded the store name as declared, so a bare-@ reference stored ""
and followed stores.default: re-pointing the default would break reads, and a
collector bound to the old store by name would delete live files. Codecs now
record the resolved name (new Config.resolve_store_name, which get_store_spec
also uses), so each object stays with the store it was written to. Bare @
could not be written before this change, so no stored metadata has an empty
name; metadata that records none still means the current default.
@dimitri-yatsenko

Copy link
Copy Markdown
Member Author

Thanks, both points were right. Fixed in 52eff8f.

Empty store name in the collector. GarbageCollector(schema, store="") built a collector that matched no references, so a destructive collect would have treated every file in the default store as an orphan. __init__ now treats "" as None, so it is the default collector. A new test runs store="" against live rows and expects no orphans.

Bare-@ references following stores.default. Recording "" would let these references follow the default if it were re-pointed, breaking reads and letting a collector bound to the old store by name delete live files. Codecs now record the resolved store name (new Config.resolve_store_name, which get_store_spec also uses), so each object stays with the store it was written to. Bare @ could not be written before this PR, so no stored metadata has an empty name. Metadata that records no store still means the current default.

New tests check that the metadata records the resolved name, and that after re-pointing stores.default the existing objects still read and a collector bound to the old store keeps them. All three fail without the fix.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Indicates an unexpected problem or unintended behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: <npy@> (default store) fails with "Storage '' is requested but not configured in stores"

2 participants