LLM & Providers

Model format

Models use the provider/model syntax:

anthropic/claude-sonnet-4-5
openai/gpt-4o
openrouter/google/gemini-2.5-flash

Built-in providers

Provider	Slug	Env variable	Model example
Anthropic	`anthropic`	`ANTHROPIC_API_KEY`	`anthropic/claude-sonnet-4-5`
OpenAI	`openai`	`OPENAI_API_KEY`	`openai/gpt-4o`
Google	`google`	`GEMINI_API_KEY`	`google/gemini-2.5-pro`
xAI	`xai`	`XAI_API_KEY`	`xai/grok-3-mini`
OpenRouter	`openrouter`	`OPENROUTER_API_KEY`	`openrouter/anthropic/claude-sonnet-4-5`
Groq	`groq`	`GROQ_API_KEY`	`groq/llama-3.3-70b`
Mistral	`mistral`	`MISTRAL_API_KEY`	`mistral/mistral-large`
Cerebras	`cerebras`	`CEREBRAS_API_KEY`	`cerebras/llama3.1-70b`
MiniMax	`minimax`	`MINIMAX_API_KEY`	`minimax/minimax-m2.5`
MiniMax (China)	`minimax-cn`	`MINIMAX_CN_API_KEY`	`minimax-cn/minimax-m2.5`
Amazon Bedrock	`amazon-bedrock`	`AWS_ACCESS_KEY_ID`	`amazon-bedrock/anthropic.claude-v2`
Azure OpenAI	`azure-openai-responses`	`AZURE_OPENAI_API_KEY`	`azure-openai-responses/gpt-4o`
Google Vertex	`google-vertex`	`GOOGLE_CLOUD_PROJECT`	`google-vertex/gemini-pro`
Hugging Face	`huggingface`	`HF_TOKEN`	`huggingface/meta-llama/Llama-3-70b`
Vercel AI Gateway	`vercel-ai-gateway`	`AI_GATEWAY_API_KEY`	`vercel-ai-gateway/anthropic/claude-sonnet-4-5`
z.ai	`zai`	`ZAI_API_KEY`	`zai/zai-default`
Kimi (Moonshot)	`kimi-coding`	`KIMI_API_KEY`	`kimi-coding/kimi-k2.5`
GitHub Copilot	`github-copilot`	`COPILOT_GITHUB_TOKEN`	`github-copilot/gpt-4o`
OpenAI Codex	`openai-codex`	`OPENAI_API_KEY`	`openai-codex/codex-mini`
Google Gemini CLI	`google-gemini-cli`	`GEMINI_API_KEY`	`google-gemini-cli/gemini-2.5-pro`
Google Antigravity	`google-antigravity`	`GEMINI_API_KEY`	`google-antigravity/gemini-2.5-pro`
OpenCode	`opencode`	`OPENCODE_API_KEY`	`opencode/opencode-default`
OpenCode (Go)	`opencode-go`	`OPENCODE_API_KEY`	`opencode-go/opencode-default`

Auto-inference

If you don’t specify a provider, Polpo infers it from the model name:

Prefix	Provider
`claude-`	anthropic
`gpt-`, `chatgpt-`, `codex-`	openai
`o1-`, `o3-`, `o4-`	openai
`gemini-`	google
`grok-`	xai
`mistral-`, `codestral-`, `devstral-`	mistral
`llama-`, `llama3`	groq

"model": "claude-sonnet-4-5"

is equivalent to:

"model": "anthropic/claude-sonnet-4-5"

Custom providers

Point to any OpenAI-compatible endpoint by adding a provider in .polpo/polpo.json:

{
  "providers": {
    "my-provider": {
      "baseUrl": "https://my-endpoint.com/v1",
      "api": "openai-completions"
    }
  }
}

Then use: "model": "my-provider/model-name"

API modes

Mode	Description
`openai-completions`	OpenAI Chat Completions API (default)
`openai-responses`	OpenAI Responses API
`anthropic-messages`	Anthropic Messages API

Custom model definitions

Define model metadata for accurate context window and cost tracking:

{
  "providers": {
    "my-provider": {
      "baseUrl": "https://my-endpoint.com/v1",
      "api": "openai-completions",
      "models": [
        {
          "id": "my-model",
          "name": "My Custom Model",
          "contextWindow": 128000,
          "maxTokens": 16384,
          "reasoning": true
        }
      ]
    }
  }
}

LLM Gateways

Polpo can route all LLM calls through a gateway configured in settings.gateway (see LLM Configuration). You can also register gateways as custom providers for model-level routing:

LiteLLM

{
  "providers": {
    "litellm": {
      "baseUrl": "https://my-litellm-proxy.com/v1",
      "api": "openai-completions"
    }
  }
}

"model": "litellm/gpt-4o"
"model": "litellm/claude-sonnet-4-5"

LiteLLM routes to 100+ providers through a single proxy. The model ID follows LiteLLM’s naming convention.

Vercel AI Gateway

{
  "providers": {
    "vercel": {
      "baseUrl": "https://gateway.ai.vercel.app/v1",
      "api": "openai-completions"
    }
  }
}

"model": "vercel/anthropic/claude-sonnet-4-5"
"model": "vercel/google/gemini-2.5-pro"

The Vercel AI Gateway provides automatic failover, cost tracking, and caching across providers.

Ollama (local)

{
  "providers": {
    "ollama": {
      "baseUrl": "http://localhost:11434/v1",
      "api": "openai-completions"
    }
  }
}

"model": "ollama/llama3.1:70b"

vLLM

{
  "providers": {
    "vllm": {
      "baseUrl": "https://my-vllm-server.com/v1",
      "api": "openai-completions"
    }
  }
}

The model ID is passed as-is to the gateway or server — no need to pre-define every model in the config.

​Model format

​Built-in providers

​Auto-inference

​Custom providers

​API modes

​Custom model definitions

​LLM Gateways

​LiteLLM

​Vercel AI Gateway

​Ollama (local)

​vLLM