Skip to content

feat: Batch embeddings, upgrade model2vec - #298

Merged
Pringled merged 3 commits into
mainfrom
perf/batch-embeddings
Oct 16, 2026
Merged

Pringled merged 3 commits into
mainfrom
perf/batch-embeddings

Conversation

@Pringled

Copy link
Copy Markdown
Member

This PR upgrades model2vec to 0.10 and embeds new chunks in batches of 1024 across files instead of once per file. This leads to some nice speedups. Putting everything together since 0.6.2 we get these results:

27k files 0.6.2 main (model2vec 0.9) main + model2vec 0.10 main + model2vec 0.10 + batched embedding (this PR)
Cold build 91s 50s 42s 36s
Warm query 13.4s 1.9s - -

Cold builds are ~2.5x faster and warm queries ~7x faster on large repos.

I also reran all the benchmarks since this PR + the previous ones all affect the speed of semble.

@codecov

codecov Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
src/semble/index/create.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@greptile-apps

greptile-apps Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge; the remaining test-coverage suggestion is non-blocking.

Reviews (2) · Last reviewed commit: "Check every embedding row across batch b..." · Reviewed by Greptile

Comment thread tests/index/test_create.py Outdated
@Pringled
Pringled requested a review from stephantul October 16, 2026 07:26
@Pringled
Pringled merged commit b4aa7b1 into main Oct 16, 2026
17 checks passed
@Pringled
Pringled deleted the perf/batch-embeddings branch October 16, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants