LLM Gateway

LLM Gateway is an open-source API gateway for large language models. It sits between your applications and multiple providers such as OpenAI, Anthropic and Google Vertex AI, so your code talks to one interface while the gateway decides where each request goes. Its API follows the OpenAI format, which keeps migration from existing code straightforward.

Beyond routing, it centralizes the management of provider API keys and tracks requests, token usage, response times and costs so teams can see what their LLM usage is doing. Performance monitoring lets you compare models on speed and cost-effectiveness. Topics on the repository also mention guardrails, rate limiting, observability and enterprise use.

You can use LLM Gateway in two ways: a hosted version at llmgateway.io where you create an account and get an API key, or a self-hosted deployment on your own infrastructure using a unified Docker image with PostgreSQL. The code is written in TypeScript and its licence is listed as other, so check the repository terms. It suits developers and platform teams who juggle several model vendors.