Repository navigation
perf(registry): skip hourly AI catalog scans when no work is missing - #668
Open
harlan-github-agent[bot] wants to merge 1 commit into
Open
harlan-github-agent[bot] wants to merge 1 commit into
harlan-github-agent[bot] wants to merge 1 commit into
Conversation
…An idle hour sorted the whole eligible catalog per runtime kind before learning there was nothing to do. Count the missing work first and run the ordered scans only when a
Contributor
Author
🤖 READY · 93/100
|
Contributor
Author
There was a problem hiding this comment.
Harlan Agent Kit posted this automated review. AI open source policy.
Automated Review: READY.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔗 Linked issue
Closes #664
❓ Type of change
📚 Description
The d1-query-review routine measured 373 seconds and 19 million row reads per week on the hourly AI work scans, and the cost lands even when there is nothing to do: both work scans sort the entire eligible catalog by stars before
LIMIT, so an idle hour pays for a full sort and returns zero rows.The submit task now counts the missing work per kind first and only runs the ordered scan when the count is nonzero. The count keeps the CASE guard that skips trimming completed sources and has no sort step. Local fixture, 20k skills, fully idle:
On the same fixture the count agreed with the scan in both directions: zero and empty on an idle catalog, one row each after adding a single unprocessed skill, so a zero count is a safe skip signal. Work that appears between the count and the scan waits for the next hourly run.
One thing I could not verify locally: a scoped typecheck of the scheduled task file. An ad-hoc project pulls both workers'
Envdeclarations and reports unrelated binding errors, including on untouched lines, so that file is covered by CI'snuxt typecheckinstead.