SGLang is a high-performance serving framework for large language models and multimodal models.
Python 38.4k 10k
SGLang's TorchTPU Plugin
Python 71 23
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
Python 1.3k 381
High Performance Serving Runtime For Interactive Super Intelligence System
Python 1.7k 668
A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.
Python 5.5k 953
JAX backend for SGL
Python 369 160
FlashMLA: Efficient Multi-head Latent Attention Kernels
A workload for deploying LLM inference services on Kubernetes
SGLang Kernel Wheel Index
SGLang kernel library for Intel XPU
