Repository navigation
feat(storage): opt-in auto-GC after storage-tail compaction - #40
Open
tiensonqin wants to merge 1 commit into
Open
tiensonqin wants to merge 1 commit into
tiensonqin wants to merge 1 commit into
Conversation
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
collect_garbage was manual-only, so storage files only grew. The storage
record gains storage_auto_gc : auto_gc option — set it via ?auto_gc on
memory_storage, file_storage, the SQLite storage constructors, or
logseq_sqlite_storage.ml's storage, with no signature changes on
create_conn/init_db/restore_conn.
After every tail compaction (conn transact/apply_report and
persist_transact_tail), maybe_collect_garbage re-evaluates the addresses
unreachable from the new root and deletes them when they are both
>= min_garbage and >= garbage_fraction of all stored addresses.
default_auto_gc = { min_garbage = 1024; garbage_fraction = 0.5 }.
Default None keeps upstream's manual-only behaviour.
New exports: auto_gc/default_auto_gc/auto_collect_garbage/
maybe_collect_garbage. Documented in impl/datascript.mli + docs/design.md.
test_storage__test_auto_gc covers trip/floor/no-policy behaviour.
devin-ai-integration
Bot
force-pushed
the
devin/storage-auto-gc
branch
from
October 5, 2026 07:36
de0be8b to
b724e1a
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Storage.collect_garbagewas manual-only, so storage files only grew. This adds an opt-in auto-GC policy evaluated after every storage-tail compaction, with the default left at upstream's manual-only behavior.API surface (smallest that covers all conn/db entry points): a
storage_auto_gc : auto_gc optionfield on thestoragerecord — set it via?auto_gconmemory_storage,file_storage,Datascript_sqlite.storage, or thelogseq_sqlite_storage.mlexample storage. No signature changes oncreate_conn/init_db/conn_from_*/restore_conn/transact.Trigger points: after a compaction stores a fresh index snapshot — in
Conn.transact/Conn.apply_report(via a newmaybe_collect_garbagefield ontransact_context, keeping conn.ml policy-agnostic) and in the db-levelpersist_transact_tail—maybe_collect_garbage storagerecomputes addresses unreachable from the new root andstorage_deletes them when they are both>= min_garbageand>= garbage_fractionof all stored addresses. Withstorage_auto_gc = None(the default) it is a no-op;collect_garbageremains the manual path.Also exported for explicit use:
Storage.default_auto_gc,Storage.auto_collect_garbage(evaluate a policy once),Storage.maybe_collect_garbage(honorstorage_auto_gc). Refactoredcollect_garbageonto sharedlive_address_set/unreachable_addresseshelpers. Documented inimpl/datascript.mlianddocs/design.md.Split out of #38 — independent of the
storage_tailrepresentation change there; both land cleanly in either order.Tests
All 34 native suites pass.
test_storage__test_auto_gccovers: always-tripping policy leaves only reachable addresses while default storage accumulates garbage across repeated compactions; a highmin_garbagefloor keeps storage manual;maybe_collect_garbageis a no-op withoutstorage_auto_gcand honors it when set;default_auto_gcvalues.Link to Devin session: https://app.devin.ai/sessions/eb0e07b23c794857b6836d8a143d1688
Open in Devin Desktop: https://app.devin.ai/desktop/session/eb0e07b23c794857b6836d8a143d1688?variant=devin
Requested by: @tiensonqin