Code for https://arxiv.org/pdf/2608.16157
-
Updated
Sep 7, 2026 - Python
#
gpu-offloading
Here are 5 public repositories matching this topic...
Train and serve MoE models that do not fit in VRAM: fused 4-bit experts, QLoRA, CPU/NVMe offload, and fast inference on consumer NVIDIA GPUs.
cuda transformers pytorch triton moe lora quantization nvme fine-tuning mixture-of-experts int4 fp8 llm-training llm-inference qlora bitsandbytes nf4 consumer-gpu mxfp4 gpu-offloading
-
Updated
Oct 5, 2026 - Python
Triton kernels for 4-bit MoE inference: grouped NF4/MXFP4 GEMM, INT4 GEMV, FP8 paged attention, and CPU/NVMe expert streaming.
cuda pytorch triton moe quantization nvme nvidia-gpu model-serving gpu-kernels mixture-of-experts int4 fp8 llm-inference bitsandbytes paged-attention nf4 mxfp4 grouped-gemm w4a16 gpu-offloading
-
Updated
Oct 5, 2026 - Python
Trace which MoE experts actually fire and pin the hot ones to GPU - ran Qwen3-30B-A3B at 5.5-7.2 tok/s on a 4GB GTX 1050 Ti
-
Updated
Sep 20, 2026 - Shell
gpu-offload: transparent CPU-to-GPU offloading. Runs the BLAS/LAPACK calls of unmodified programs (NumPy, R, Octave, C, Fortran) on an NVIDIA GPU when a per-machine calibration says it is faster, with CPU fallback. LD_PRELOAD + cuBLAS/cuSOLVER.
performance hpc gpu numpy linear-algebra cuda cublas nvidia scientific-computing gpu-acceleration blas lapack openblas drop-in ld-preload auto-tuning cusolver heterogeneous-computing wsl2 gpu-offloading
-
Updated
Oct 5, 2026 - C
Add this topic to your repo
To associate your repository with the gpu-offloading topic, visit your repo's landing page and select "manage topics."