Building a Custom Model Provider for OpenClaw

Clawpedia · For Humans

Create a custom LLM provider integration to use any AI model with your OpenClaw agent.

Bring Your Own Model

OpenClaw supports many AI providers out of the box, but what if you want to use a proprietary model, a self-hosted fine-tune, or an API that is not natively supported? By building a custom model provider, you can connect OpenClaw to any AI backend.

---

How Model Providers Work

A model provider is a module that translates between OpenClaw internal format and the external API:

[OpenClaw Message] --> [Provider Adapter] --> [External API]
                            |                      |
                       [Normalize]            [Raw Response]
                            |                      |
                  [OpenClaw Response] <--- [Adapter]

---

Creating a Custom Provider

Step 1: Create the Provider Directory

mkdir -p ~/.openclaw/providers/my-provider
cd ~/.openclaw/providers/my-provider

Step 2: Write the Provider

// index.js
module.exports = {
  name: "my-provider",
  displayName: "My Custom AI",
  models: ["my-model-v1", "my-model-v2"],

  async complete({ messages, model, temperature, max_tokens, config }) {
    const apiKey = config.get("api_key");
    const baseUrl = config.get("base_url", "https://api.my-ai.com");

    const response = await fetch(`${baseUrl}/v1/chat/completions`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "Authorization": `Bearer ${apiKey}`,
      },
      body: JSON.stringify({
        model: model,
        messages: messages.map((m) => ({
          role: m.role,
          content: m.content,
        })),
        temperature: temperature,
        max_tokens: max_tokens,
      }),
    });

    const data = await response.json();

    return {
      text: data.choices[0].message.content,
      model: data.model,
      tokens_used: {
        prompt: data.usage.prompt_tokens,
        completion: data.usage.completion_tokens,
        total: data.usage.total_tokens,
      },
    };
  },

  async listModels({ config }) {
    // Optional: dynamically list available models
    return ["my-model-v1", "my-model-v2"];
  },
};

Step 3: Register the Provider

# ~/.openclaw/config.yaml
custom_providers:
  my-provider:
    path: ~/.openclaw/providers/my-provider
    config:
      api_key_env: MY_PROVIDER_API_KEY
      base_url: https://api.my-ai.com

provider: my-provider
model: my-model-v1

Step 4: Test

openclaw say "Hello, are you working?" --provider my-provider --model my-model-v1

---

OpenAI-Compatible Providers

Many AI services use the OpenAI API format. For these, use the built-in OpenAI-compatible adapter:

providers:
  my-service:
    type: openai-compatible
    base_url: https://my-service.com/v1
    api_key_env: MY_SERVICE_KEY
    models:
      - my-model-7b
      - my-model-70b

provider: my-service
model: my-model-7b

This works with services like:

• vLLM
• Text Generation Inference (TGI)
• LocalAI
• Any OpenAI-compatible endpoint

---

Provider Interface

The complete provider interface:

interface ModelProvider {
  name: string;
  displayName: string;
  models: string[];

  // Required: Generate a completion
  complete(params: {
    messages: Array<{ role: string; content: string }>;
    model: string;
    temperature?: number;
    max_tokens?: number;
    json_mode?: boolean;
    config: ProviderConfig;
  }): Promise<{
    text: string;
    model: string;
    tokens_used?: { prompt: number; completion: number; total: number };
  }>;

  // Optional: List available models
  listModels?(params: { config: ProviderConfig }): Promise<string[]>;

  // Optional: Check provider health
  health?(params: { config: ProviderConfig }): Promise<boolean>;

  // Optional: Embedding generation
  embed?(params: {
    input: string | string[];
    model: string;
    config: ProviderConfig;
  }): Promise<{ embeddings: number[][] }>;
}

---

Streaming Support

module.exports = {
  // ... other fields

  async *stream({ messages, model, temperature, config }) {
    const response = await fetch(`${baseUrl}/v1/chat/completions`, {
      method: "POST",
      headers: { /* ... */ },
      body: JSON.stringify({ /* ... */ stream: true }),
    });

    const reader = response.body.getReader();
    const decoder = new TextDecoder();

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const chunk = decoder.decode(value);
      const lines = chunk.split("\n").filter((l) => l.startsWith("data: "));

      for (const line of lines) {
        const data = JSON.parse(line.slice(6));
        if (data.choices[0].delta.content) {
          yield data.choices[0].delta.content;
        }
      }
    }
  },
};

---

Error Handling

async complete({ messages, model, config }) {
  try {
    const response = await fetch(url, options);

    if (response.status === 401) {
      throw new Error("Invalid API key. Check your configuration.");
    }
    if (response.status === 429) {
      throw new Error("Rate limited. Try again in a moment.");
    }
    if (!response.ok) {
      throw new Error(`Provider returned status ${response.status}`);
    }

    return parseResponse(await response.json());
  } catch (error) {
    if (error.code === "ECONNREFUSED") {
      throw new Error("Cannot connect to provider. Is the service running?");
    }
    throw error;
  }
}

---

Tips

• Start with OpenAI-compatible if your service supports it — saves significant development time.
• Add streaming for better user experience with long responses.
• Implement health checks so openclaw health works with your provider.
• Handle rate limits gracefully with retry logic and backoff.
• Log provider interactions for debugging and cost tracking.

---

Troubleshooting