Skip to content

Add SWITCH kernels write-up: performance vs vLLM's Punica LoRA path - #148

Merged
yairallouche merged 1 commit into
generative-computing:mainfrom
lastras:docs/switch-kernels-writeup
Oct 15, 2026
Merged

yairallouche merged 1 commit into
generative-computing:mainfrom
lastras:docs/switch-kernels-writeup

Conversation

@lastras

@lastras lastras commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds docs/SWITCH_KERNELS.md, documenting the SWITCH fused-LoRA kernel backend (one fused base+shrink GEMM plus a Triton in-place expand kernel) against vLLM's three-launch Punica LoRA path.
  • Covers balanced-rank and heterogeneous-rank decode speedups (1.2-1.4x and up to 5.9x), absolute throughput, the idle-adapter dispatch gap (and why it's a dispatch-policy difference rather than a kernel one), prefill, 8B results, and methodology.
  • Adds the five supporting figures under docs/figures/.

Test plan

  • Ran the repo's link validator (.pre-commit/validate_links.py) against the new doc — clean.
  • Cross-checked every quantitative claim against the source benchmark tables.
  • Verified the CLI example (compose_granite_switch.py --adapters/--output, vllm.entrypoints.openai.api_server) still matches current flags on main.

Documents the SWITCH fused-LoRA kernel backend (fused base+shrink GEMM
plus a Triton in-place expand) against vLLM's three-launch Punica
path: 1.2-1.4x faster decode with balanced ranks, up to 5.9x faster
with one outsized adapter in the mix, and an explanation of the
single-graph idle gap (a dispatch-policy difference, not a kernel
one).

Signed-off-by: Luis A Lastras <lastrasl@us.ibm.com>
@yairallouche
yairallouche merged commit 7e72456 into generative-computing:main Oct 15, 2026
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants