Skip to content

backup: allow Pebble's sstable reader through the import boundary - #1281

Open
bootjp wants to merge 2 commits into
mainfrom
feature/stoic-carson-i8u16m
Open

bootjp wants to merge 2 commits into
mainfrom
feature/stoic-carson-i8u16m

Conversation

@bootjp

@bootjp bootjp commented Oct 17, 2026 •

Copy link
Copy Markdown
Owner

Design: docs/design/2026_10_16_implemented_production_commonization.md
Design: docs/design/2026_04_29_implemented_snapshot_logical_decoder.md

Summary

The Test workflow (test (ubuntu-latest)) has been red on main since 927e4bf (run 38914301223). Its only failure:

--- FAIL: TestBackupLinksNoLiveClusterCode
    import_boundary_test.go:51: internal/backup links github.com/cockroachdb/pebble: github.com/bootjp/elastickv/internal/backup -> github.com/cockroachdb/pebble/v2/sstable

Two PRs merged on the same day with rules that conflict. Each was green on its own branch.

Which design is intended

Allow the sstable reader and keep the Pebble engine out (option 1).

  • The decoder design chose this package on purpose. Its backup: decode the .fsm files a running node persists #1276 addendum says: "This is the one place the package links a storage library, because the files are in Pebble's format."
  • The boundary is meant to keep live-cluster code out. That is the purpose stated in the test's comment, in the encoder design ("internal/backup links no live-cluster machinery") and in M4. Pebble was on the list because store brought in the Pebble engine and internal/encryption.
  • The sstable reader is not the engine. github.com/cockroachdb/pebble/v2 imports sstable, so the reverse import would be a cycle. go list -deps confirms that elastickv-backup, -snapshot-decode, -snapshot-encode and -snapshot-archive link no engine root, no store and no internal/encryption.
  • Option 2 (moving the reader out of internal/backup) would need a new injection seam. The decode CLI and the encoder's self-test both reach the reader through ReadSnapshotFile / DecodeSnapshot, so those would need an injection point. That is an API change made only to shrink what gets linked, and the decode CLI would still link the same reader.

Cost. I built each CLI with and without the import (go build -overlay, go1.26.6, linux/amd64). The import adds 5.1 to 5.2 MiB per binary, about 1,900 Pebble symbols. elastickv-backup and elastickv-snapshot-archive carry it without ever calling it. If that size matters, putting the reader behind such a seam is a possible follow-up. This PR does not do it.

Changes

  1. docs: records the decision.
    • Decoder addendum: the boundary test allows exactly this package.
    • Commonization doc, M4 "As built": corrects the claim that the four CLIs link no Pebble. They link no Pebble engine, and they do link the sstable reader.
    • Commonization doc, ## Decisions: adds the exception.
  2. backup: changes import_boundary_test.go.
    • backupAllowedDeps exempts exactly github.com/cockroachdb/pebble/v2/sstable. It is matched exactly, not by prefix.
    • The walk fails if an exemption goes unused, so the exemption cannot outlive the import.
    • The new table test TestForbiddenBackupDep checks that the exemption is exact. pebble/v2, pebble/v2/sstable/block and v1 pebble/sstable stay forbidden.

Behavior change

None. The changes are test and docs only. No production code changes, and binaries are the same as on main, which has linked sstable since #1276.

Risk

Low. The boundary is a little looser, by exactly one package. That package cannot bring in the engine, store or internal/encryption.

Test evidence

  • go test -race -count=1 ./internal/backup/... ./cmd/elastickv-backup/... ./cmd/elastickv-snapshot-decode/... ./cmd/elastickv-snapshot-encode/... ./cmd/elastickv-snapshot-archive/...: all pass.
  • golangci-lint v2.9.0 (the CI pin) with --config=.golangci.yaml ./...: 0 issues.
  • On unmodified main, TestBackupLinksNoLiveClusterCode fails with the error above.
  • Mutation checks. Each mutant is caught:
    • no allowlist (main's behaviour): both tests fail;
    • prefix-matched allowlist: sstable/block row fails;
    • engine root added to the allowlist: pebble/v2 row fails, and the unused-exemption check fails;
    • reader without the sstable import, run in a scratch worktree because go/build reads files from disk: the unused-exemption check fails.
  • Every other package in the failing main run already passed. This change touches no other package.
  • Jepsen: not applicable. No replication, MVCC, OCC or adapter path changes.

Self-review (five lenses)

  1. Data loss: no write, apply, snapshot or restore path changes. Test and docs only.
  2. Concurrency / distributed failures: none. The test is a static walk over the package graph, and it passes under -race.
  3. Performance: no runtime change. The link cost of sstable predates this PR, and is measured and recorded above and in the doc.
  4. Data consistency: no MVCC, OCC, HLC or routing change. The decoder still reads SST bundles with the Pebble version the store writes with.
  5. Test coverage: the exemption has a table test for its matching rules and a check that it is still used. Both were mutation-tested.

Generated by Claude Code

Summary by CodeRabbit

  • ドキュメント
    • バックアップ関連 CLI の依存範囲と、SST バンドルの読み込みに関する設計上の境界を明確化しました。
  • テスト
    • 依存関係の検証を更新し、許可された SST 読み込み依存のみが対象となることや、必要な CLI で実際に利用されることを確認するようにしました。

#1276 made the decoder read SST ingest bundles with Pebble's sstable
package, while #1277's boundary test forbids every Pebble package. Both
merged, and main has been red since. The decoder design already chose
the sstable reader for EKVSSTI1 bundles, whose files are Pebble
sstables. Record in both docs that the boundary keeps the Pebble engine
out but allows exactly that package, and correct the as-built claim
that the backup CLIs link no Pebble.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gz1MesaSCqfXkJa9zb8Ysx
TestBackupLinksNoLiveClusterCode has failed on main since 927e4bf:
snapshot_sst.go (#1276) imports github.com/cockroachdb/pebble/v2/sstable
and the boundary (#1277) forbids the whole Pebble module. Exempt exactly
that package. It cannot reach the Pebble engine, which imports it, and
importing the engine or any other Pebble package still fails the test.

The test also fails when an exemption goes unused, and
TestForbiddenBackupDep pins the matching rules: exact for allowed
paths, prefix for forbidden ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gz1MesaSCqfXkJa9zb8Ysx
@coderabbitai

coderabbitai Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 786a6379-80dc-4715-9e36-e8c4fdaff418
📥 Commits

Reviewing files that changed from the base of the PR and between 927e4bf and 9b1d187.

📒 Files selected for processing: 3
  • docs/design/2026_04_29_implemented_snapshot_logical_decoder.md
  • docs/design/2026_10_16_implemented_production_commonization.md
  • internal/backup/import_boundary_test.go

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

バックアップの依存境界テストで、Pebble の sstable パッケージを例外として許可し、その依存が実際に存在することを確認します。関連する設計文書に、CLI の依存関係と境界ルールを記載しました。

Changes: バックアップの依存境界

Layer / File(s) Summary
sstable インポート境界
internal/backup/import_boundary_test.go, docs/design/2026_04_29_implemented_snapshot_logical_decoder.md, docs/design/2026_10_16_implemented_production_commonization.md
依存境界テストで github.com/cockroachdb/pebble/v2/sstable の完全一致パスを許可し、その依存が走査結果に含まれることを確認します。親パスとサブパッケージは許可しません。設計文書に CLI の依存状況とこの境界ルールを記載しました。

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~15 minutes

Change: Other

Suggested reviewers: claude

Merge Risk: ⚪ Minimal · up to 9b1d1

This change updates the dependency-boundary checks and their documentation, with no identified production behavior risk. The summarized checks cover the intended sstable-only exception, so no merge-blocking issue remains.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files. (2 skipped: 2… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed タイトルは、Pebble の sstable リーダーをインポート境界で許可する主要な変更を正確かつ簡潔に示しています。
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 1 files. (2 skipped: 2 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

  • Fix all pre-merge checks with AI
✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@bootjp
bootjp marked this pull request as ready for review October 17, 2026 11:01
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T11:02:59.710549Z 9b1d187 Draft marked ready
🔒 Security Review ✅ Completed 2026-10-17T11:03:41.975353Z 9b1d187 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants