SGLang vs vLLM: The Winner Flips With Your Workload

TL;DR

Published benchmarks put SGLang about 29% ahead of a fully tuned vLLM on shared-context multi-turn work, 16,215 tokens per second against 12,553. On single-turn unique prompts the same comparison inverts and vLLM wins, 60 against 52.7. That is not two people disagreeing. It is one question with two correct answers depending on whether your prompts share a prefix. SGLang's RadixAttention automatically discovers and reuses shared prefixes; vLLM's continuous batching optimises raw throughput across independent requests. Agent loops repeat a system prompt and tool schema on every step, which lands them squarely in SGLang's case.

Key Takeaways

• "Which is faster" is unanswerable without the workload. SGLang leads by ~29% on shared context, vLLM leads on unique prompts.
• RadixAttention is the actual difference. SGLang keeps a radix tree of prefixes and reuses cached KV automatically, without you configuring anything.
• Agent workloads are shared-prefix by construction. The same system prompt and tool definitions go out on every one of 20 steps.
• vLLM is broader, not narrower. It runs on NVIDIA, AMD, Intel CPUs and GPUs, Google TPUs and AWS Trainium. SGLang's hardware surface is smaller.
• vLLM is 88,668 stars against SGLang's 31,619, and eleven months older. Both Apache-2.0, so the licence is not a differentiator.
• Neither replaces the other for local work. That is a different decision.

The question with two right answers

Most comparisons pick a winner. The published numbers do not support one, and the way they fail to is the most useful thing here.

Same engines, opposite results. The variable is not tuning or hardware, it is whether your requests share a prefix.

Note the second row is not a rounding error either. On genuinely independent prompts, vLLM's approach is simply better suited, and SGLang's prefix machinery has nothing to find and still costs something to maintain.

So the first question is not which engine. It is what your traffic actually looks like.

What each one is actually optimising

The names of the two techniques tell you the difference if you know what to listen for.

vLLM: PagedAttention plus continuous batching. KV cache memory is allocated in small pages on demand rather than one contiguous block per sequence, which cuts waste and lets far more requests sit in memory simultaneously. Continuous batching then pulls new requests into the running batch the moment compute frees up. The whole design targets many independent requests moving through the GPU efficiently.

SGLang: RadixAttention. SGLang maintains a radix tree of the prompts it has seen and their cached KV state. When a new request shares a prefix with something already cached, that prefix is not recomputed. It is discovered automatically, without a cache key you define or a config flag you set. The design targets many requests that overlap.

These are not competing implementations of the same idea. They optimise different bottlenecks, which is exactly why the benchmark inverts.

Related, and often confused with this: prompt caching at the API-provider level does something conceptually similar for hosted models, with the difference that you often have to structure requests deliberately to hit it. RadixAttention finds the overlap for you.

Why agent workloads land in SGLang's case

Here is where this stops being an infrastructure trivia question.

An agent step is not an independent request. Every call in the loop carries the same system prompt, the same tool definitions, and a conversation history that grows by an increment. Step 12 of an agent run shares almost everything with step 11.

That is the textbook shared-prefix workload, and it is not occasional. It is the default shape of anything built on tool-calling loops, which is most of what people are deploying now.

The corollary matters for cost as well as speed. Recomputing an identical 4,000-token system prompt and tool schema on every one of 20 steps is 80,000 tokens of prefill you did not need to pay for. That compounds the same way the latency does in AI agent costs, and it is the same underlying discipline as context engineering: the cheapest tokens are the ones you do not send twice.

The practical read: if you serve a chat product, an agent platform, or anything with a long fixed preamble, your traffic is shared-prefix and SGLang's advantage is real rather than theoretical. If you serve one-shot classification, embedding-style calls, or genuinely unrelated user prompts, it is not, and vLLM is the better fit.

Where vLLM still wins

Three things, and the first is the one that should weigh most in a real decision.

Hardware portability. vLLM runs on NVIDIA GPUs, AMD GPUs, Intel CPUs and GPUs, Google TPUs and AWS Trainium. SGLang's supported surface is narrower. If you are multi-cloud, or think you might move providers to chase capacity or price, that is the difference between a config change and a migration.

Ecosystem size, verified rather than asserted:

Both are Apache-2.0, so licensing is not a differentiator here. vLLM is roughly 2.8x the stars and eleven months older, which mostly shows up as more integrations, more deployment guides, and more people having already hit your bug.

A caveat on a claim you will see repeated: several comparisons state vLLM has roughly three times SGLang's contributor base. I could not verify that. GitHub's contributors endpoint caps at 500 and returns numbers near that cap for both projects, so it cannot support the comparison either way. Stars and forks are checkable; treat the contributor multiple as unsourced.

Single-turn throughput, per the benchmark above. If nothing in your traffic repeats, vLLM is the faster engine, not the safer-but-slower one.

Where TensorRT-LLM and TGI fit

Framing this as a two-horse race is convenient and slightly wrong. Two other engines show up in the same evaluations, and knowing why they are not the answer here is quicker than evaluating them.

TensorRT-LLM is NVIDIA's engine. It compiles a model into an optimised engine artifact targeted at a specific GPU, which is where its speed comes from and also where its cost comes from: a build step per model and per GPU generation, and no path off NVIDIA hardware. It is the right call when you have settled on one NVIDIA deployment target, throughput is the dominant business constraint, and you can absorb the operational weight of a compile pipeline.

TGI, HuggingFace's Text Generation Inference, is the conservative option. It integrates cleanly with the HuggingFace ecosystem and is straightforward to stand up. It has largely been overtaken on raw throughput by vLLM and SGLang, and tends to appear where an existing HuggingFace-centric stack makes it the path of least resistance.

For most teams the real shortlist is SGLang or vLLM, and the tiebreaker is the workload question above rather than a feature grid.

How to benchmark this on your own traffic

Given that the published result inverts, the only number that settles your case is one you produce. This is less work than it sounds, and there is exactly one thing you must get right.

Use your own prompts, not synthetic ones. This is the whole exercise. Synthetic benchmarks usually generate independent random prompts, which is the workload where vLLM wins, so a synthetic test will quietly recommend vLLM regardless of what your production traffic looks like. If 80% of your tokens are a repeated system prompt, your benchmark has to reproduce that ratio or it is measuring somebody else's problem.

A workable method:

1. Sample real traffic. A few hundred actual prompt and response pairs, with their real lengths. Keep the duplication that exists in them.
2. Measure your prefix-sharing ratio. What proportion of a typical request is identical to the previous one? That single number predicts which engine wins before you run anything.
3. Fix the variables. Same GPU, same model weights, same quantization, same max sequence length. Changing two things at once is how the published numbers ended up incomparable.
4. Sweep concurrency, rather than testing one level. The engines diverge under load, not at concurrency one.
5. Record TTFT and inter-token latency separately, not just total throughput. As the vLLM vs Ollama comparison shows, aggregate throughput can hide where the time actually goes.

GuideLLM is a reasonable tool for this and is what Red Hat used for its published comparison, and both engines ship their own benchmark scripts.

The result you want is not "engine X is faster". It is "engine X is faster on our prefix-sharing ratio at our concurrency", which is a sentence you can act on and defend.

Which to pick

SGLang if your traffic shares prefixes: agent loops, chat with a long system prompt, anything with a fixed preamble and many turns. The 29% is real and it is free, in the sense that you do not configure it.

vLLM if your prompts are genuinely independent, if you need hardware portability, or if you want the larger ecosystem and the shorter path to someone else having solved your problem.

Benchmark your own traffic before committing. That sounds like a cop-out and it is the actual lesson of the inverted result. A published number that does not state its workload shape tells you nothing about yours, and the two engines swap places on the variable that most benchmarks leave out.

If you have not yet decided whether to run production serving at all, that is the earlier question and it is covered in vLLM vs Ollama and self-hosting vs API.

Frequently asked questions

Is SGLang faster than vLLM?

On shared-context multi-turn workloads, yes, by roughly 29% in published comparisons, 16,215 tokens per second against 12,553. On single-turn unique prompts the result inverts and vLLM wins, 60 against 52.7. Neither is universally faster, and the deciding variable is whether your requests share a prefix.

What is RadixAttention?

SGLang's prefix-reuse mechanism. It maintains a radix tree of previously seen prompts and their cached KV state, so when a new request shares a prefix with something cached, that portion is not recomputed. The match is discovered automatically rather than configured.

How is that different from vLLM's PagedAttention?

They solve different problems. PagedAttention allocates KV cache in small pages on demand so more requests fit in memory, and continuous batching keeps the GPU fed with independent requests. RadixAttention targets overlap between requests. One optimises many separate requests, the other optimises repetition.

Which is better for AI agents?

SGLang, in most cases. Agent loops resend the same system prompt and tool definitions on every step, so the traffic is shared-prefix by construction. That is precisely the workload where prefix reuse pays, and it saves prefill cost as well as latency.

Does vLLM support prefix caching too?

Yes, vLLM has prefix caching. The practical difference reported in comparisons is that SGLang's radix tree discovers reuse opportunities automatically across varied conversation flows, whereas getting consistent benefit from other implementations more often depends on how you structure requests.

Which has the bigger community?

vLLM, clearly: 88,668 stars against 31,619, and 20,501 forks against 7,780, having started eleven months earlier. Both are Apache-2.0. Be sceptical of the frequently repeated claim about a 3x contributor gap, which the GitHub API cannot substantiate because it caps contributor listings.

What about TensorRT-LLM or TGI?

TensorRT-LLM is NVIDIA's engine and compiles a per-model, per-GPU artifact, which buys peak throughput on NVIDIA at the cost of a build pipeline and hardware lock-in. TGI is HuggingFace's server, easiest to adopt inside a HuggingFace stack but generally behind vLLM and SGLang on throughput. For most teams the shortlist is still SGLang or vLLM.

How do I benchmark SGLang against vLLM myself?

Use real sampled traffic rather than synthetic prompts, because synthetic tests generate independent prompts and therefore favour vLLM regardless of your actual workload. Measure your prefix-sharing ratio first, hold GPU, model, quantization and sequence length constant, sweep concurrency rather than testing one level, and record time-to-first-token separately from throughput.

Can I switch between them later?

Both expose OpenAI-compatible APIs, so client code generally moves with an endpoint change. The heavier lift is operational: deployment configuration, quantization format, and hardware assumptions, which is where vLLM's broader hardware support becomes a real consideration.

Sources

• SGLang and vLLM: repository metrics read from the GitHub API on 2026-08-10
• LMSYS, SGLang and RadixAttention: the original description of the prefix-reuse design
• SGLang documentation and vLLM documentation
• RunPod, when to choose SGLang over vLLM: multi-turn and KV cache reuse comparison

Throughput figures are quoted from published third-party comparisons on their own hardware and versions, and do not transfer to a different rig. Star and fork counts were read on 2026-08-10 and move daily.