Skip to content

perf(chat): remove quota and admission round trips - #1345

Merged
tannerlinsley merged 1 commit into
mainfrom
taren/chat-one-call-admission
Oct 5, 2026
Merged

tannerlinsley merged 1 commit into
mainfrom
taren/chat-one-call-admission

Conversation

@tannerlinsley

@tannerlinsley tannerlinsley commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

A live Assistant trace spent 916 ms reserving usage and 555 ms across two admission guards before reaching the model. Reservation now runs the existing lock, fresh role check, quota checks, and atomic accounting protocol inside one PostgreSQL call. Each ordinary admission guard reads current permissions and bot/thread lifecycle together, while retaining both pre- and post-preparation checks and the extra lifecycle check after retry readiness.

The migration uses a SECURITY INVOKER, VOLATILE function and the existing advisory lock keys, so old and new workers remain compatible during deployment. Deployment migrates and verifies the schema before releasing the worker.

Validation: full pnpm test, 278 real PostgreSQL runtime tests, including quota concurrency, revocation while waiting on a lock, admin role revocation, retries, and scope/lifecycle checks. Live phase timings are retained for the post-deployment comparison.

Summary by CodeRabbit

  • Performance
    • Reduced application overhead when reserving chat runs. Provider response times and response quality are not guaranteed to change.
  • Bug Fixes
    • Preserved access checks during chat-run reservations, including when permissions change while a reservation is waiting.
    • Continued enforcing user, shared spending, daily turn, and scheduled-turn limits, including for admins.

@tannerlinsley
tannerlinsley requested a review from a team October 5, 2026 07:23
@changeset-bot

changeset-bot Bot commented Oct 5, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 60a0942

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Oct 5, 2026

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c9e38437-a8f9-4686-af2a-0aaa222068e4
📥 Commits

Reviewing files that changed from the base of the PR and between 01bcb35 and 60a0942.

📒 Files selected for processing (9)
  • docs/tanchat-send-latency.md
  • drizzle/migrations/0041_tanchat_usage_reservation.sql
  • drizzle/migrations/meta/_journal.json
  • harness-tests/core/run-usage-admin.test.ts
  • harness-tests/pending-runtime/run-usage-admin.test.ts
  • scripts/verify-tanchat-schema.ts
  • src/chat/conversation-identity.server.ts
  • src/chat/server/conversation.ts
  • src/chat/server/run-usage.ts
 _______________________________________________________________________________________________________________________________
< Always code as if the person who ends up maintaining your code is a violent psychopath who knows where you live. - John Woods >
 -------------------------------------------------------------------------------------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
tanstack-com 60a0942 Oct 05 2026, 07:25 AM

@tannerlinsley
tannerlinsley merged commit aefe4a1 into main Oct 5, 2026
7 of 8 checks passed
@tannerlinsley
tannerlinsley deleted the taren/chat-one-call-admission branch October 5, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant