litellm
Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]
Effortlessly migrate from OpenAI or Anthropic SDKs to LiteLLM. Achieve zero code changes with the LiteLLM proxy or easily swap SDK calls for LiteLLM's completion function, saving time and resources.
LiteLLM Proxy Health Check and Monitoring Endpoints: A Complete Implementation GuideImplement LiteLLM proxy health check and monitoring endpoints. Validate proxy availability, LLM deployments, and integrations like Datadog and Slack. Get the complete guide.
LiteLLM Model-Specific Parameters and Provider Transformations: A Complete GuideMaster LiteLLM model-specific parameters & provider transformations. This guide explains how LiteLLM unifies LLM providers with a two-layer architecture for seamless API integration.
LiteLLM Embedding, Image Generation, and Audio Endpoints: Architecture and Usage DifferencesExplore LiteLLM embedding, image generation, and audio endpoint differences. Understand how LiteLLM unifies these distinct call types within its architecture for seamless API integration.
High Availability and Load Balancing LiteLLM Proxy: A Production Deployment GuideAchieve high availability and load balancing for your LiteLLM proxy. Learn how to deploy stateless, horizontally scaled, and automatically load-balanced model endpoints for production readiness.
Production Security Best Practices for LiteLLM Proxy: 10 Critical MeasuresImplement production security best practices for your LiteLLM proxy. Learn 10 critical measures to secure your API with env vars, encryption, and read-only filesystems.
LiteLLM Model Context Protocol (MCP) Servers and Tools: A Complete Technical GuideMaster LiteLLM Model Context Protocol (MCP) servers and tools. This guide explains how MCP enables OpenAI compatible clients to discover and execute external tools via a unified gateway.
LiteLLM Proxy Custom Prompt Management: A Complete Guide to Dynamic Prompt TemplatesMaster LiteLLM proxy custom prompt management. Inject, version, and retrieve dynamic prompt templates at runtime via API. Enhance your LLM applications effortlessly.
How to Configure Budget Limits and Spend Alerts in LiteLLMConfigure budget limits and spend alerts in LiteLLM with its three-layer system. Set caps, get Slack/email alerts, and enforce limits per model, team, or user. Optimize your LLM spending effectively.
LiteLLM Token Usage and Cost Tracking Across Providers: A Complete Technical GuideMaster LiteLLM token usage and cost tracking across providers. Our guide shows how LiteLLM normalizes counts and calculates costs for seamless LLM management.
How to Invoke A2A Agents with LiteLLM: A2A Protocol Implementation GuideLearn to invoke A2A agents using LiteLLM's A2A protocol support. This guide details direct agent communication and LLM-as-agent patterns through a unified proxy.
How LiteLLM Virtual Keys Work for API Key Management: A Complete Technical GuideLearn how LiteLLM virtual keys manage API keys, authenticate requests, enforce limits, and track usage for LLM providers. A complete technical guide.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" 
Maintain an open-source project? Get it listed too →