LLM config (LiteLLM)

KubeClaw deploys a LiteLLM proxy by default and injects OPENAI_API_BASE into the Gateway container so all LLM SDK calls route through the proxy. This gives you per-agent virtual keys, budget caps, model fallback routing, and semantic caching behind a single endpoint.

Core settings

Proxy config

The litellm.proxy_config value maps directly to LiteLLM's config.yaml. This is where you define models, routing, and general settings.

Default proxy_config:

litellm:
  proxy_config:
    model_list:
      - model_name: "gpt-4o"
        litellm_params:
          model: "gpt-4o"
          api_key: "os.environ/OPENAI_API_KEY"
    litellm_settings:
      drop_params: true
    router_settings:
      routing_strategy: "simple-shuffle"
      num_retries: 2
      timeout: 120
    general_settings:
      master_key: "os.environ/PROXY_MASTER_KEY"

Adding models

Add more models by extending model_list. Provider API keys from secret.data are forwarded to the proxy pod via environmentSecrets.

litellm:
  proxy_config:
    model_list:
      - model_name: "gpt-4o"
        litellm_params:
          model: "gpt-4o"
          api_key: "os.environ/OPENAI_API_KEY"
      - model_name: "claude-sonnet"
        litellm_params:
          model: "anthropic/claude-sonnet-4-20250514"
          api_key: "os.environ/ANTHROPIC_API_KEY"

Then add the key to your secret:

secret:
  data:
    ANTHROPIC_API_KEY: "sk-ant-..."

Routing strategies

LiteLLM supports several routing strategies via router_settings.routing_strategy:

• simple-shuffle (default): random selection across healthy endpoints
• least-busy: routes to the model instance with the fewest in-flight requests
• latency-based-routing: picks the fastest responding endpoint
• cost-based-routing: picks the cheapest available model

Subcharts

Enable PostgreSQL when you need virtual keys, spend tracking, or team-based access controls. Redis is enabled by default for response caching.