TL;DR
Every comparison of these two frames it as control versus convenience, and treats self-hosting as the cautious option. That framing hides what you are actually choosing. A self-hosted gateway concentrates every provider key you have into one database, and makes you the party responsible for patching the thing in front of it. OpenRouter moves that concentration to somebody else's infrastructure along with your request data. Neither is safe in the abstract. You are picking which failure you would rather own.
This post is about the gateway decision itself. Reducing the bill once you have one is a separate exercise.
Standing up a gateway for more than one team? We design the AI systems around who carries the operational load, because that is the part that decides whether a self-hosted proxy is an asset or a liability in month eight.
Key Takeaways
- The real split is not control versus convenience. It is which risk you absorb: a credential store and a patch obligation, or a third party in your request path.
- In April 2026, LiteLLM disclosed CVE-2026-42208, a SQL injection in the proxy's API key verification path affecting v1.81.16 through v1.83.6. It was found through their own bug bounty and patched before disclosure, which is the process working.
- What followed is the part worth planning for. Sysdig observed exploitation attempts around 36 hours after disclosure, and CISA added it to the Known Exploited Vulnerabilities catalog on 8 May. Self-hosting means that clock is yours.
- LiteLLM's Rust gateway is real and dramatic, cutting p99 added latency from 257.7ms to 0.7ms in their own benchmarks. It is also an early beta behind a signup form with streaming still landing, so it is not something to plan a migration around yet.
- Per-team cost attribution is the most oversold feature in this category. Practitioners running LiteLLM at scale report tracking that works as one pool rather than per team or per app.
What a gateway actually centralises
Both products put one endpoint in front of many model providers. The difference is where the keys live.
With LiteLLM, you run the proxy and a PostgreSQL database. That database holds your virtual keys, your budgets, your usage records, and the provider credentials the proxy uses to reach OpenAI, Anthropic, Bedrock and everything else. That is the point of it. It is also, in security terms, a concentration of every credential you own behind one process.
With OpenRouter, there is nothing to deploy. You hold one key, they hold the provider relationships, and your requests transit their platform on the way out.
The part the comparisons leave out
On 29 April 2026, LiteLLM disclosed CVE-2026-42208, a SQL injection in the proxy's API key verification path. A crafted Authorization header on a normal endpoint could reach the Postgres database behind the proxy without authenticating first. Versions v1.81.16 through v1.83.6 were affected, with the fix in v1.83.7 and v1.83.10-stable recommended.
Two things deserve saying plainly, because the story is often told badly in both directions.
LiteLLM handled it well. The flaw came in through their own bug bounty programme and was patched before public disclosure. They published the affected range, a recommended version, and a Postgres query to help operators audit their own history. That is what good disclosure looks like, and it is not an argument against the project.
The window afterwards is the operational reality. Sysdig reported targeted exploitation attempts roughly 36 hours after disclosure, and CISA added the CVE to its Known Exploited Vulnerabilities catalog on 8 May with a federal patch deadline three days later.
If you run LiteLLM, this is a normal and manageable obligation, the same one you carry for any internet-facing service. If you chose it because a blog post told you self-hosting was the secure option and nobody mentioned patch duty, it is a surprise arriving at the worst moment.
The overhead nobody quotes
The other thing missing from these comparisons is what the Python proxy costs you in latency. LiteLLM's own published benchmarks put the Python gateway at 257.7ms of p99 added latency, against 2.3ms for Portkey and 4.5ms for Bifrost.
Their answer is a Rust rewrite, and the numbers are striking:
| LiteLLM Rust | LiteLLM Python | Portkey | Bifrost | |
|---|---|---|---|---|
| p99 added latency | 0.7ms | 257.7ms | 2.3ms | 4.5ms |
| Peak memory | 21.8MB | 329.5MB | 90.4MB | 199.1MB |
| Cost per million requests | $0.000175 | $0.015354 | $0.001042 | $0.001008 |
On a 30-turn Claude Code session, they measure the Rust path adding 0.03 seconds against the Python path's 0.97 seconds.
Read the status before you plan around it. LiteLLM describes the Rust gateway as an early beta behind a signup form, with streaming and the full feature surface still landing. Coverage elsewhere has reported it as though it shipped. It has not, and these are vendor benchmarks on vendor hardware, so treat the ratios as directional and the absolute numbers as a starting hypothesis.
Two claims worth deflating
A gateway does not raise your provider rate limits. If you bring your own Anthropic key, you are still inside Anthropic's limits for that key. Routing through a proxy changes where retries and fallbacks are configured, not what the upstream will accept.
Per-team cost attribution is harder than the feature list implies. Teams running LiteLLM at scale describe usage tracking that behaves as one aggregate pool, making it difficult to say which team or which application caused a spike, and end up building tagging and dashboards on top. Budget enforcement per virtual key is real. Clean attribution after the fact is the part that takes work.
Side by side
| OpenRouter | LiteLLM | |
|---|---|---|
| Shape | Managed service, nothing to deploy | Proxy plus Postgres that you run |
| Where provider keys sit | With OpenRouter | In your database |
| Who patches the gateway | OpenRouter | You, on their disclosure clock |
| Request data path | Transits their platform | Stays in your network until the provider call |
| Billing | Credits, one invoice | You pay each provider directly |
| Added latency | Managed, consistent | 257.7ms p99 on Python, 0.7ms on the Rust beta |
| Fails when | They have an outage or change limits | You have an outage, or miss a patch |
Which to pick
Pick OpenRouter if you do not have someone whose job includes patching a tier-zero service inside a week. That is a real staffing question, not a philosophical one, and answering it honestly is more useful than any feature table. You also get breadth of model access immediately and one invoice.
Pick LiteLLM if data residency or compliance requires the request path to stay inside your network, and you have the operational capacity to treat it as production infrastructure: patch alerts, an upgrade path, monitoring, and someone accountable for all three.
Run both if you want the self-hosted control plane and the managed breadth. LiteLLM in front as your single internal endpoint, OpenRouter configured as one of the providers behind it. You keep virtual keys and budgets in your own hands, and you inherit model coverage you did not have to negotiate. You also keep the patch obligation, so this is not a way to avoid the choice above.
Frequently asked questions
Is self-hosting LiteLLM more secure than using OpenRouter?
It is differently secure. Self-hosting keeps request data inside your network, and concentrates every provider credential you hold into one database behind one process that you are responsible for patching. CVE-2026-42208 in April 2026 is the concrete illustration: disclosed and patched responsibly, exploited in the wild within roughly 36 hours, added to CISA's KEV catalog nine days later. Whether that is safer than a managed platform depends entirely on your patch response time.
Can I use OpenRouter as a provider inside LiteLLM?
Yes, and it is a common arrangement. Applications call your LiteLLM endpoint and get your virtual keys and budgets, while OpenRouter sits behind it supplying model breadth. It does not reduce your operational responsibility for the proxy.
Will a gateway get around provider rate limits?
No. With your own provider key you remain subject to that provider's limits. A gateway changes where fallback and retry logic lives, which can make limits less disruptive, but it cannot raise a ceiling the provider sets.
Is the LiteLLM Rust gateway ready to use?
Not for production planning as of August 2026. LiteLLM describes it as an early beta requiring signup, with streaming and full feature parity still in progress. The published benchmarks are dramatic, and the direction is clear, but a beta is not a migration target.
Which is cheaper?
At low and bursty volume, OpenRouter usually wins because there is no infrastructure to pay for. At steady high volume, self-hosting can win on paper, provided you count the engineering time to run it. That second term is the one teams underestimate, and it is the most common reason people migrate off a self-hosted proxy after several months.