Home > Browse > AI Infrastructure > LiteLLM AI

LiteLLM AI

LiteLLM AI is an AI-powered AI Infrastructure tool — Open-source AI LLM API proxy and router. Best for: AI Infrastructure. Pricing: Free (GateOnAI Score: 73/100).

Open-source AI LLM API proxy and router

FreeLiveCode GenerationAPI DevelopmentMachine Learning

LiteLLM is an open‑source AI gateway that lets organizations expose any LLM, agent, or model‑control‑plane (MCP) behind a single OpenAI‑compatible API key. It supports over 140 providers and 1,800 models, allowing teams to swap models, add new endpoints, or bring internal fine‑tuned models into production without code changes. The platform offers visibility and control features such as per‑key, user, team, and org usage tracking, spend caps, rate limits, and audit logs, enabling chargeback and budget enforcement. Cost‑optimization tools include auto‑routing to the cheapest suitable model, load balancing across providers and regions, response caching, and prompt compression. Deployment is designed for quick self‑hosting: official Docker images, Helm chart, Terraform module, and one‑click installs for AWS, GCP, or Azure, with support for fully air‑gapped environments and Kubernetes autoscaling. Enterprise features such as SSO, JWT auth, virtual keys, and custom SLAs are available, and the project maintains transparent open‑source releases and security disclosures.

Summary written from LiteLLM AI's own website on 10 Oct 2026. Check the vendor's website for current details.

Visit LiteLLM AI

How to evaluate LiteLLM AI before you buy

What GateOnAI has on record

Questions to ask the vendor

  1. Will you sign a GDPR data processing agreement (Article 28) and share your list of sub-processors?
  2. Where is our data stored and processed, and is it used to train your models?
  3. Are there usage limits on the free version, and could it become paid?
  4. Can we export all our data, and how is it deleted when we cancel?
  5. (AI Infrastructure) Are our prompts, files or fine-tuning data used to train your models, and can we opt out in writing?
  6. (AI Infrastructure) In which regions are our data and model calls processed, and what uptime and rate limits apply to our plan?

Full checklist: 43 questions to ask an AI vendor · Alternatives to LiteLLM AI · Compare LiteLLM AI with Groq

More AI Infrastructure Tools

May Connect With

Tools whose input/output tags match those of LiteLLM AI (the tags are assigned automatically and the connections are not tested):

AI Prompts for LiteLLM AI

Ready-to-use prompt templates for LiteLLM AI, organized by profession:

Find similar tools | Browse all AI prompts