Backpack is an experimental programmable local store for typed records. The SQLite API stores finite JSON with explicit codecs, local identity and provenance. Records recover their Python types after closing and reopening the database.
- Product: Backpack
- Distribution:
backpack-store - Import:
backpack_store - Python: 3.11 or newer
- Runtime dependencies: none
python -m pip install backpack-store==0.1.0Optional extras are backpack-store[pydantic], backpack-store[moves] and
backpack-store[host]. The backpack_store import stays the same for every extra.
from dataclasses import dataclass
from backpack_store import CodecRegistry, DataclassCodec
@dataclass(frozen=True)
class Note:
title: str
body: str
registry = CodecRegistry()
registry.register(DataclassCodec(Note, type_key="notes.note", schema_version=1))
encoded = registry.encode(Note("Example", "Local text"))
restored = registry.decode(encoded.type_key, encoded.schema_version, encoded.payload)
assert isinstance(restored, Note)A stable dotted type_key and positive schema_version identify each registered
codec. Registrations are explicit and reject duplicate keys or Python types.
Dataclass codecs support concrete non-recursive dataclasses, nested dataclasses,
JSON scalar fields, literals, optional/unions, typed lists, tuples and string-keyed
dictionaries. They validate every field; unknown or missing fields fail. A custom
codec can support other trusted model types. Stored names never cause imports.
Record envelopes contain a separate local UUID, payload schema version, revision,
UTC creation/update timestamps, normalized exact tags and optional Provenance.
Timestamps use YYYY-MM-DDTHH:MM:SS.ffffffZ. Provenance names the source URL,
system, object ID and capture time; a source ID is not the local record UUID.
JSON payloads must be finite objects, at most 1 MiB and 64 nesting levels. Tags
are case-sensitive NFC strings, trimmed, deduplicated and sorted (up to 64 supplied
values of 128 characters). Unknown types/versions and invalid payloads raise typed
errors. Importing the package opens no files or database and performs no network I/O.
from backpack_store import Backpack
with Backpack.open("notes.sqlite") as store:
store.register(DataclassCodec(Note, type_key="notes.note"))
written = store.put(Note("Example", "Local text"), tags=["work"])
with Backpack.open("notes.sqlite") as store:
store.register(DataclassCodec(Note, type_key="notes.note"))
note = store.get(written.id)
record = store.get_record(written.id) # envelope plus .value
page = store.find(Note, tags=["work"], limit=20)
updated = store.update(written.id, Note("Revised", "Local text"),
expected_revision=record.revision, tags=["work"])
assert updated.revision == 2
assert store.delete(written.id)Each put creates a new UUID and returns WriteResult(id, revision). update
replaces payload, tags and provenance together, preserves the local ID/type key
and creation time, and requires the expected revision. A stale revision raises
RevisionConflictError without changing the record. Missing reads/updates raise
NotFoundError; deletion returns whether a row existed. Payload, metadata and
tag writes share one transaction. Invalid input and database failures leave no
partial record.
find requires an explicitly registered Python type and matches all requested
tags. It returns at most 1,000 records (default 50) in ascending local UUID order.
Pass .next_cursor with the same type/tags to continue. Cursors are query-scoped;
concurrent inserts, updates or deletions can change later pages, so pagination
does not promise a frozen snapshot. get_raw returns recoverable JSON/metadata
without decoding an unknown registered type or payload version.
The database schema has its own version, separate from payload versions. Unknown
or nonempty unversioned databases fail safely. Use the connection on its opening
thread; a context manager closes it even after caller failure. Lock waiting is
bounded by timeout (default 5 seconds, allowed 0–60). No path is chosen by
default. Parent directories must already exist. Importing never opens a database.
Run python examples/persistence.py for a complete offline file-backed Note and
synthetic ProjectItem round trip, including filtering and deletion.
Register the current codec, then explicit trusted functions for every one-version step from an older payload to the current version:
store.register(DataclassCodec(NoteV2, type_key="notes.note", schema_version=2))
store.register_migration("notes.note", from_version=1, to_version=2,
migrate=lambda old: {"title": old["title"], "text": old["body"]})NoteV2 is an application-defined dataclass with title and text fields. Old
reads upgrade in memory; .payload and .schema_version on the returned envelope
still describe the original stored representation. Only an explicit
revision-checked update writes the current representation. Missing migration
steps and newer unknown versions raise UnknownVersionError; failed/invalid
migration output raises MigrationError. Migration functions never come from a
stored module name. Database schema version 1 is independent; no destructive
migration or automatic database replacement is provided.
page = store.export_jsonl("records-1.jsonl", limit=100, max_bytes=4_194_304)
# If page.next_cursor is set, export to another new explicit destination:
# store.export_jsonl("records-2.jsonl", cursor=page.next_cursor, limit=100)Export writes UTF-8 JSONL containing raw record envelopes with format_version: 1,
type/payload versions and provenance. No codec or migration runs during export.
It stops at the record or byte budget (default 1,000 records / 4 MiB; maximum
1,000 / 64 MiB). If a single record exceeds the byte budget, it fails before
creating a destination. Pages use ascending UUIDs and the same concurrent-write
limitations as find. A complete temporary file is published without overwriting
an existing path; the destination directory must exist and support hard links.
Export never rewrites the database.
The direct store remains independent of Hidden Moves and MCP. Optional moves
and host extras declare the capability-family dependencies. CI uses a reviewed
pinned family revision, runs standalone storage first, then installs the optional
wheels.
Installed entry-point discovery advertises backpack; load it explicitly and bind
an already-configured Backpack instance. The factory chooses no path and opens no
database. The application registers trusted models before selecting tools:
from backpack_store.host import build_catalog
from backpack_store.integration import READ_CAPABILITIES, WRITE_CAPABILITIES
read = build_catalog(store, READ_CAPABILITIES)
result = read.invoke("backpack.records.find", {"type_key": "notes.note", "limit": 10})
write = build_catalog(store, WRITE_CAPABILITIES, allow_writes=True)Read tools are backpack.records.get and .find; local write tools are .put and
.delete. A write profile requires explicit host intent and explicit selection.
Inputs contain local IDs, registered stable type keys and finite JSON payloads,
never Python classes, codec imports, SQL or database paths. put validates the
current registered model; find enforces the storage query limits. Concrete JSON
views preserve record identity/version/provenance without serializing arbitrary
Python objects. Effect annotations describe local reads/insertion/deletion;
application permissions and caller approval policy remain responsible for access.
serve_store(store, names, allow_writes=False) serves the same catalog over local
MCP stdio. The application owns the connection and cleanup. For a complete notes
application with a fixed trusted model and explicit path/profile:
python examples/notes_stdio.py --database notes.sqlite --profile read
# Select a write-capable host only when local writes are intended:
python examples/notes_stdio.py --database notes.sqlite --profile writeThese are local tools. No public endpoint or remote access to the SQLite file is
provided. Run python -m unittest discover -s integration_tests -v in the optional
integration environment to check real stdio, selection and consumer equivalence.
Install the pydantic extra to use the separate Pydantic 2 extension (supported
range >=2.13,<3; tested with 2.13.5). Importing the core never imports Pydantic.
The extension uses the same store, explicit identity and migration registry:
from pydantic import BaseModel
from backpack_store.pydantic_codec import PydanticCodec
class ValidatedNote(BaseModel):
title: str
priority: int
store.register(PydanticCodec(ValidatedNote, type_key="notes.validated", schema_version=1))
written = store.put(ValidatedNote(title="Example", priority=2))Register the same trusted model after reopening. The codec accepts the exact
registered class, serializes aliases and JSON representations, then revalidates
strictly before writing. Decode also uses strict JSON validation with extra fields
forbidden. This catches mutated instances and model_construct bypasses instead
of trusting an already-created instance. Payloads still obey the core finite JSON
object limits. Object-shaped nested models and datetime JSON round trips are
covered; scalar/list RootModel classes are rejected. Private/computed fields are
not part of the durable representation.
This codec does not add native Pydantic type support to tool signatures. The
optional provider continues to use its concrete JSON views. Run
python -m unittest discover -s pydantic_tests -v in the extension environment.
uv build
python -m pip install dist/backpack_store-0.1.0-py3-none-any.whl
python -m unittest discover -s tests -vCI installs the wheel built from its source distribution and runs the behavior suite on Python 3.11/3.13 across Linux, macOS and Windows. The original generated scaffold is preserved in the repository's initial commit.
This 0.1.x project is experimental. Published GitHub releases with matching v
tags run .github/workflows/release.yaml, verify the installed wheel and source
distribution, and upload through PyPI trusted publishing in environment pypi.
The initial release tag is v0.1.0.