I'm currently a second-year AI undergraduate at PSL University.
Lately I've been reading code in LLM serving, mostly around how agent conversations are stored and replayed.
Open source
- vllm-project/vllm#64937: fixed time-to-first-token in
vllm bench servefor servers that stream reasoning separately (e.g. SGLang). - vllm-project/agentic-api#519: regression tests for reasoning order on
previous_response_idcontinuations.
Elsewhere: than0316.github.io
