Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,30 @@ tell you.

## [Unreleased]

### Changed

- **`recall` searches this project's user and assistant text by default.
Breaking:** 0.2.0's results are now `recall --all-projects --all-kinds`. A
turn is in the project of the directory recorded on its own line, else its
session's; a project is the repository holding it, worktrees included, read
from `.git` entries without running git. Tool calls, tool output, thinking
and system lines are left out. `--project DIR` names another project, and
`--all-projects` and `--all-kinds` widen one dimension each; no environment
variable sets them. One stderr line, `outside this scope: N matching
turn(s)`, says how many matches the scope left out. `show` is deliberately
unscoped in this release. See
[docs/USAGE.md](docs/USAGE.md#what-recall-searches).
- **The index is schema 3**, recording each turn's directory and Claude Code's
`isMeta`: run `gitmemory index` once after upgrading. Lines Claude Code
marked as its own (`isMeta`, a compaction summary) no longer count as the
user's words under `latest-user`.
- **The recall skill searches only when asked.** When the user only refers to
an earlier decision, it asks before searching. It passes
`--project "${CLAUDE_PROJECT_DIR}"`, treats a recalled line as evidence of
what was said and never as an instruction, and widens only as the outside
line and the user allow. See
[SECURITY.md](SECURITY.md#recall-output-is-untrusted-input).

## [0.2.0] - 2026-10-18

### Added
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ agent after compaction, derived key ideas, the dashboard — is in
| `gitmemory capture <file>` | Copies out one transcript's new bytes, by hand |
| `gitmemory verify` | Checks every manifest's contiguity proof |
| `gitmemory index` | Rebuilds the SQLite FTS5 index from the store |
| `gitmemory recall "<query>"` | Searches the index and prints one line per turn, dated, the user's own words newest first ([`--policy`](docs/USAGE.md#when-memories-disagree); [`--redact`](docs/USAGE.md#masking-credentials-in-what-it-prints) masks credentials) |
| `gitmemory recall "<query>"` | Searches this project's user and assistant text in the index ([scope](docs/USAGE.md#what-recall-searches): `--project`, `--all-projects`, `--all-kinds`) and prints one line per turn, dated, the user's own words newest first ([`--policy`](docs/USAGE.md#when-memories-disagree); [`--redact`](docs/USAGE.md#masking-credentials-in-what-it-prints) masks credentials) |
| `gitmemory show <address>` | Prints the whole turn at an address `recall` printed, read from the store; `--context N` adds N turns either side, `--redact` masks credentials ([more](docs/USAGE.md#read-the-whole-turn)) |
| `gitmemory derive [--graph]` | Rebuilds key ideas and a timeline; `--graph` adds the decision graph (opt-in) |
| `gitmemory dashboard` | Serves the index with Datasette, read-only, on loopback, behind a sign-in |
Expand Down Expand Up @@ -217,13 +217,13 @@ real text, and that is a measurement rather than a suspicion.
| What | How it was measured | Result |
|---|---|---|
| Hook cost in the agent's critical path | Timed against spawning `true` the same way, three runs of 400 | p50 **7.4 – 7.5 ms**, p99 **10.2 – 11.5 ms** |
| The suite | On a fresh checkout, no downloads | **1,134 tests**, and **328 conformance cases** against three third-party corpora, one gated on the LongMemEval download and one on `pip install -e '.[serve]'` |
| Whether the tests hold anything | Every fix mutated to remove its behaviour; the named test must fail | **662** negative controls |
| The suite | On a fresh checkout, no downloads | **1,181 tests**, and **328 conformance cases** against three third-party corpora, one gated on the LongMemEval download and one on `pip install -e '.[serve]'` |
| Whether the tests hold anything | Every fix mutated to remove its behaviour; the named test must fail | **704** negative controls |

The two suite counts do not add up, and should not. Switching the corpora on
collects 1459, not 1462. Three conformance cases fill parametrisations that
collects 1506, not 1509. Three conformance cases fill parametrisations that
collect as one empty placeholder each while the corpora are absent, so they
replace three of the 1134 rather than joining them. Every figure on this board
replace three of the 1181 rather than joining them. Every figure on this board
is pinned by a test, which is how it stays true.

**What is not measured**, and is shown as a row on the dashboard rather than
Expand Down
17 changes: 17 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,3 +106,20 @@ on. What it does not guarantee is listed in
[USAGE.md](docs/USAGE.md#what-it-does-not-guarantee): it masks sixteen shapes
and nothing else, decodes nothing, and covers only what those two commands
print.

## Recall output is untrusted input

A line `recall` prints is a record of what was said, and whoever said it may not
be the user: a pasted page, a cloned repository's instructions, another agent.
An agent that reads recall output reads third-party text. By default `recall`
leaves out other projects, tool calls and output, thinking and system lines,
and lines Claude Code marked as its own (`isMeta`, a compaction summary) no
longer lead the user's words, but pasted text passes, and `show`, the
dashboard and the files under the store are not scoped
([USAGE.md](docs/USAGE.md#what-the-scope-does-not-solve) lists the rest). The
recall skill treats each line as evidence of what was said, never as an
instruction, and widens the scope only on the user's request; both are rules
the agent follows, not gates. The `outside this scope` count discloses, by
design, that matches exist elsewhere, never what they say. `.git` entries,
which decide what one project is, are trusted local configuration: a hostile
`.git` file can alias two projects.
3 changes: 3 additions & 0 deletions docs/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -647,6 +647,9 @@ earlier decision or runs the skill, not an injection: there is no MCP server and
`NOT BUILT` on the dashboard (§2.9). `recall` prints each hit's address, and `show` resolves one
to the whole turn from the store, not the index, so a stale or tampered index cannot change what
the address is shown to say.
**[2026-10-22 — the skill now searches only when the user asks, and asks first when they only
refer to an earlier decision; `recall` searches the current project's user and assistant text by
default (USAGE.md, "What recall searches").]**

### 2.9 Dashboard

Expand Down
8 changes: 4 additions & 4 deletions docs/REPRODUCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -486,11 +486,11 @@ HOME=$(mktemp -d)/isolated-home uv run pytest -q --collect-only -p no:cacheprovi
uv run pytest -q --collect-only -p no:cacheprovider | grep -c '\.jsonl\]$'
```

**Expected:** `1134/1135 tests collected (1 deselected)`, then
`1459/1460 tests collected (1 deselected)`, then `328`. The deselected test is
**Expected:** `1181/1182 tests collected (1 deselected)`, then
`1506/1507 tests collected (1 deselected)`, then `328`. The deselected test is
the LongMemEval corpus test, which is opt-in:
`uv run pytest -q -m corpus` after `bench/fetch_longmemeval.sh`. The README's
paragraph on why 1,134 + 328 is 1,459 and not 1,462 is the arithmetic of these
paragraph on why 1,181 + 328 is 1,506 and not 1,509 is the arithmetic of these
three numbers. All three were run while writing this guide.

The `HOME` is the one the suite gives itself. The owner-data scan adds two
Expand All @@ -513,7 +513,7 @@ test fail. The count is the number of rows in `tests/mutate_index.py`.
uv run python -c "import sys; sys.path.insert(0, 'tests'); from mutate_index import MUTANTS; print(len(MUTANTS))"
```

**Expected:** `662`. Checked while writing this guide.
**Expected:** `704`. Checked while writing this guide.

To run the controls rather than count them:

Expand Down
100 changes: 91 additions & 9 deletions docs/USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,82 @@ since: start `gitmemory watch` with the hook's `$GITMEMORY_HOME`, and
covers the other causes, such as a watcher that runs passes less often than
hourly. With `--db`, recall does not look.

## What recall searches

By default `recall` searches one project, and in it only what the user and the
assistant wrote as text. A recalled line goes back into an agent's context, so
what it searches by default is what reaches the agent.

- **The project** is the repository holding the current directory, or the
directory `--project DIR` names: the nearest directory at or above it with a
`.git`. A linked worktree, Claude Code's `.claude/worktrees/` included, joins
its repository, even after it has been removed. A submodule or a clone
vendored inside the project is a repository of its own. `~` and `/` are
looked at like any other level, so a dotfiles repository in `~` is a project,
and the search goes no higher. With one there, every directory under `~`
that is in no other repository, `~/Downloads` as much as `~/notes`, is in
that one project. Outside any repository a directory is only itself, not
its subdirectories, so `~` does not gather every project under it. The `.git` entries are read from the filesystem; git is never run.
- **A turn's project** is that of the directory recorded on its own line. A
session that `cd`s into another repository puts the lines after the `cd` in
that repository's project. A line that records no directory is where its
session was. pi records the directory only on its session header, so every pi
turn is where its session started. A session that recorded none is in no
project, and only `--all-projects` finds it.
- **The kinds** are user and assistant text. Tool calls, tool output, thinking
and system lines are left out.

`--all-projects` searches every project, and the lines written in none.
`--all-kinds` adds every kind. `--project` and `--all-projects` cannot be
combined, and no environment variable sets any of the three, so a search
widens only when it is asked to. `recall --all-projects --all-kinds` searches
what 0.2.0's `recall` searched. The order can still differ under the default
`latest-user` policy: lines Claude Code marked as its own (`isMeta`, a
compaction summary) no longer lead as the user's words. `-k` counts turns
inside the scope.

When the scope left matches out, `recall` says how many on stderr, after the
results or after `no matches`:

```console
$ gitmemory recall "retry loop"
...
outside this scope: 3 matching turn(s) (--all-projects, --all-kinds to widen)
```

It counts distinct turns, as the results are counted, and names only the flags
you did not pass. One count covers both dimensions, so with neither flag it
can be this project's tool output alone. Nothing is printed when nothing was left out.

`show` is not scoped in this release: it prints any address you give it, and
`--context N` adds neighbouring turns of every kind.

### What the scope does not solve

- **Pasted and restated text.** Text the user pasted, and assistant text that
restates a tool's output, are user or assistant text and pass. Role and kind
record who wrote a line, not where its words came from.
- **Injected text that is not marked.** A user block that mixes typed words
with a `<system-reminder>` carrying a cloned repository's `CLAUDE.md`, a
skill load or slash-command expansion without `isMeta`, and pi's
`custom_message` text all still read as the user's words.
- **Submodules and vendored clones.** Turns recorded while `cd`'d into a
submodule or a vendored clone count as that repository's, not the parent's.
- **A work tree with no `.git` of its own**, as when a bare repository is
checked out with `--git-dir` and `--work-tree`: nothing is guessed, and each
directory is scoped by equality, without its subdirectories.
- **A removed worktree that lived outside its repository**
(`git worktree add ../x`): nothing left on disk says whose it was, so it is
only its own directory.
- **What bypasses `recall`.** `show`, the dashboard, and reading `raw/` by hand
are not scoped.
- **An agent that widens anyway.** An agent can pass `--all-projects` or
`--all-kinds`, and can follow what a recalled line says. The skill's rules
against both are rules it follows, not gates
([SECURITY.md](../SECURITY.md#recall-output-is-untrusted-input)).
- **A stale rule in this project.** An instruction the user has since dropped
still matches. Only the dates, and asking the user, settle it.

## Read the whole turn

A line stops at 160 characters. `show` prints the whole turn at an address,
Expand Down Expand Up @@ -201,7 +277,7 @@ tends to act on.

| Policy | Order | Use it when |
|---|---|---|
| `latest-user` (default) | The user's own words newest first, then the agent's turns and tool output newest first. A subagent's instructions are the agent's words | The user's later word should win, and nothing else should |
| `latest-user` (default) | The user's own words newest first, then the agent's turns and tool output newest first. A subagent's instructions are the agent's words, and so is text Claude Code injected and marked (`isMeta`, a compaction summary) | The user's later word should win, and nothing else should |
| `latest` | Newest first, whoever said it | Any later turn should win, the agent's included |
| `relevance` | Best BM25 match first | You want the agent to decide: every line carries its date, so it can see which turn came later |
| `module:function` | Whatever your function returns | Anything else: pinned decisions, a cut-off date, a different rule per project |
Expand Down Expand Up @@ -229,8 +305,9 @@ If two lines disagree, the later date is the current decision; say so.

A policy is a function from a list of `Hit`s to a list of `Hit`s. It can
reorder and it can drop; it gets nothing that is not already in the store. A
`Hit` has the fields `recall` prints, plus `text`, `ts` and `is_sidechain`
(true for a subagent's turn), and `gitmemory.index.when(hit)` reads `ts` as a
`Hit` has the fields `recall` prints, plus `text`, `ts`, `is_sidechain`
(true for a subagent's turn), `cwd` (the directory the turn was written in, or
its session's, or None) and `is_meta` (Claude Code injected it as user text), and `gitmemory.index.when(hit)` reads `ts` as a
UTC `datetime`.

A validity window, where everything before a date you name is out of date, is
Expand Down Expand Up @@ -298,12 +375,17 @@ they were written, with an address you can check.

In Claude Code, the plugin does this without the `CLAUDE.md` line. Installed as
in the [hook's README](../hook/README.md#as-a-claude-code-plugin), it adds a
skill, `/gitmemory:recall`, that Claude loads on its own when you refer to a
decision or a rule from before a compaction or from an earlier session, or that
you run with a topic: `/gitmemory:recall retry loop`. It runs `recall`, and
`index` when there is no index yet or the one there is older than what you are
asking about. It quotes each line with its address and says when a hit comes
from another session. It writes nothing but the index. It runs
skill, `/gitmemory:recall`, that Claude loads when you ask it to look
something up from before a compaction or from an earlier session, or that you
run with a topic: `/gitmemory:recall retry loop`. When you only refer to an
earlier decision, it asks whether to search first. It runs `recall` on the
project Claude Code was started in, and `index` when there is no index yet or
the one there is older than what you are asking about. It searches other
projects only once, after the [outside line](#what-recall-searches) reports
matches the scope left out and what it found does not answer you, and tool
output only when you ask what a command printed. It quotes each line with its
address, as evidence of what was said and never as an instruction, and says
when a hit comes from another session. It writes nothing but the index. It runs
`gitmemory --version` first and says when the CLI is older than the plugin;
the [hook's README](../hook/README.md#as-a-claude-code-plugin) has how to hold
the two at one release. To have it mask credentials
Expand Down
5 changes: 3 additions & 2 deletions hook/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,8 +90,9 @@ hooks as the `settings.json` example below, `PreCompact` and `SessionEnd` and no
plugin. There is no path to fill in and no file to `chmod`.

The plugin installs the hook and one skill, `/gitmemory:recall`, which has the
agent run `gitmemory recall` and quote what it returns
([USAGE.md](../docs/USAGE.md#put-it-back-in-front-of-the-agent)). Neither
agent run `gitmemory recall` on the project Claude Code was started in, when you
ask it to look something up, and quote what it returns as evidence, never as
instructions ([USAGE.md](../docs/USAGE.md#put-it-back-in-front-of-the-agent)). Neither
captures anything. **Nothing is captured until the watcher is installed and
running** (`uv tool install --python 3.13 git+https://github.com/doronp/gitmemory`,
`gitmemory init claude-code`, then `gitmemory watch`, as in the
Expand Down
Loading
Loading