ai-provider-kit

AI Provider Kit

A comprehensive Go library for creating and managing AI provider instances with support for multiple AI services.

Features

• Multi-Provider Support: OpenAI, Anthropic, Gemini, Cerebras, OpenRouter, Qwen, and more
• Interface Segregation: Focused interfaces for specific capabilities
• Standardized Core API: Consistent request/response patterns across providers
• Provider Extensions: Preserve provider-specific capabilities while using standardized API
• Factory Pattern: Dynamic provider creation and configuration
• Authentication Support: API keys, OAuth 2.0, and custom authentication methods
• Health Monitoring: Automatic health checks and metrics collection
• Load Balancing: Multiple API keys with round-robin distribution
• Extensible: Easy to add new providers through the factory pattern
• Local Models: Support for LM Studio, Ollama, and Llama.cpp
• Streaming: Real-time response streaming
• Tool Calling: Built-in support for function calling
• Comprehensive Testing: 100% test coverage with unit and integration tests

Architecture

The AI Provider Kit features a modern, flexible architecture with interface segregation and standardized patterns:

• Interface Segregation: 9 focused interfaces instead of a monolithic provider interface
• Standardized Core API: Universal request/response types with provider-specific extensions
• Extensible Design: Easy to add new providers and capabilities

For complete architecture documentation, see the Architecture Guide.

Installation

go get github.com/cecil-the-coder/ai-provider-kit@latest

Quick Start

package main

import (
    "context"
    "log"

    "github.com/cecil-the-coder/ai-provider-kit/pkg/factory"
    "github.com/cecil-the-coder/ai-provider-kit/pkg/types"
)

func main() {
    // Create a new provider factory
    f := factory.NewProviderFactory()

    // Initialize default providers (with real implementations)
    factory.RegisterDefaultProviders(f)

    // Configure an OpenAI provider
    config := types.ProviderConfig{
        Type:         types.ProviderTypeOpenAI,
        Name:         "openai-primary",
        APIKey:       "your-api-key",
        DefaultModel: "gpt-4",
    }

    // Create provider instance
    provider, err := f.CreateProvider(types.ProviderTypeOpenAI, config)
    if err != nil {
        log.Fatal(err)
    }

    // Generate completion
    stream, err := provider.GenerateChatCompletion(context.Background(), types.GenerateOptions{
        Prompt: "Hello, world!",
        Stream: false,
    })
    if err != nil {
        log.Fatal(err)
    }
    defer stream.Close()
}

Supported Providers

Cloud Providers

• OpenAI: GPT models, embeddings, and more
• Anthropic: Claude models
• Google Gemini: Gemini models
• Cerebras: High-performance inference
• OpenRouter: Multi-provider routing
• Qwen: Alibaba Cloud models

Local Providers

• LM Studio: Local model serving
• Ollama: Local model management
• Llama.cpp: Efficient local inference

Configuration

Each provider can be configured with the following options:

type ProviderConfig struct {
    Type              ProviderType
    Name              string
    APIKey            string
    BaseURL           string
    DefaultModel      string
    Timeout           time.Duration
    MaxRetries        int
    RateLimit         RateLimitConfig
    OAuth             *OAuthConfig
    Headers           map[string]string
}

Authentication

API Key Authentication

config := types.ProviderConfig{
    Type:   types.ProviderTypeOpenAI,
    APIKey: "your-api-key-here",
}

OAuth 2.0 Authentication

config := types.ProviderConfig{
    Type: types.ProviderTypeAnthropic,
    OAuth: &types.OAuthConfig{
        ClientID:     "your-client-id",
        ClientSecret: "your-client-secret",
        TokenURL:     "https://api.anthropic.com/oauth/token",
    },
}

Load Balancing

Configure multiple API keys for load balancing:

provider, _ := f.CreateProvider(types.ProviderTypeOpenAI, types.ProviderConfig{
    Type:   types.ProviderTypeOpenAI,
    APIKeys: []string{
        "key1",
        "key2",
        "key3",
    },
    LoadBalancer: &types.LoadBalancerConfig{
        Strategy: types.LoadBalancerStrategyRoundRobin,
    },
})

Streaming

Enable streaming for real-time responses:

stream, err := provider.GenerateChatCompletion(context.Background(), types.GenerateOptions{
    Prompt: "Explain quantum computing",
    Stream: true,
})

for {
    chunk, err := stream.Next()
    if err != nil {
        break
    }

    if chunk.Done {
        break
    }

    fmt.Print(chunk.Content)
}

Tool Calling

ai-provider-kit provides comprehensive tool calling support across all providers with format translation, validation, and advanced control features.

Quick Example

tools := []types.Tool{
    {
        Name:        "get_weather",
        Description: "Get current weather for a location",
        InputSchema: map[string]interface{}{
            "type": "object",
            "properties": map[string]interface{}{
                "location": map[string]interface{}{
                    "type":        "string",
                    "description": "The city and state, e.g. San Francisco, CA",
                },
                "unit": map[string]interface{}{
                    "type": "string",
                    "enum": []string{"celsius", "fahrenheit"},
                },
            },
            "required": []string{"location"},
        },
    },
}

options := types.GenerateOptions{
    Prompt: "What's the weather in San Francisco?",
    Tools:  tools,
}

stream, err := provider.GenerateChatCompletion(context.Background(), options)
if err != nil {
    log.Fatal(err)
}
defer stream.Close()

chunk, err := stream.Next()
if err != nil {
    log.Fatal(err)
}

// Handle tool calls
if len(chunk.Choices) > 0 && len(chunk.Choices[0].Message.ToolCalls) > 0 {
    for _, toolCall := range chunk.Choices[0].Message.ToolCalls {
        // Execute tool locally
        result := executeToolCall(toolCall)

        // Send result back
        // (see TOOL_CALLING.md for complete multi-turn example)
    }
}

For a comprehensive guide, see TOOL_CALLING.md

Features

• Universal Format: Define tools once, use across all providers
• ToolChoice Control: Fine-grained control over tool selection
  • auto - Model decides whether to use tools
  • required - Must use at least one tool
  • none - Don't use tools
  • specific - Force a specific tool
• Parallel Tool Calling: Handle multiple tool calls in a single response
• Tool Validation: Optional validation via pkg/toolvalidator
• Streaming Support: Tool calls work with streaming responses
• Format Translation: Automatic conversion between provider formats

Supported Providers

All 6 providers support tool calling:

• OpenAI (native format)
• Anthropic (content blocks format)
• Gemini (function declarations format)
• Cerebras (OpenAI-compatible)
• Qwen (OpenAI-compatible)
• OpenRouter (OpenAI-compatible)

Health Monitoring

Monitor provider health and performance:

// Health check
err := provider.HealthCheck(context.Background())
if err != nil {
    log.Printf("Provider unhealthy: %v", err)
}

// Get metrics
metrics := provider.GetMetrics()
log.Printf("Requests: %d, Success Rate: %.2f%%",
    metrics.RequestCount,
    float64(metrics.SuccessCount)/float64(metrics.RequestCount)*100)

Custom Providers

Add your own provider implementations:

// Implement the Provider interface
type CustomProvider struct {
    // your fields
}

func (p *CustomProvider) GenerateChatCompletion(ctx context.Context, options types.GenerateOptions) (types.ChatCompletionStream, error) {
    // your implementation
}

// Register your provider
factory.RegisterProvider(types.ProviderTypeCustom, func(config types.ProviderConfig) types.Provider {
    return &CustomProvider{config: config}
})

Development

Prerequisites

• Go 1.23 or later
• Docker (optional)

Running Tests

# Run all tests
go test ./...

# Run with coverage
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out

# Run benchmarks
go test -bench=. ./...

Building

# Build library
go build ./...

# Build CLI tool (if available)
go build -o ai-provider-kit ./cmd/ai-provider-kit

Docker

# Build Docker image
docker build -t ai-provider-kit .

# Run with Docker
docker run -p 8080:8080 ai-provider-kit

Contributing

Contributions are welcome! Please see our Contributing Guidelines for details.

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the MIT License.

Summary: You are free to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the software, and to permit persons to whom the software is furnished to do so, subject to the following conditions:

• The copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
• The software is provided "AS IS", without warranty of any kind.

For the full license text, see the LICENSE file.

Changelog

See CHANGELOG.md for a list of changes and version history.

Support

• 📖 Documentation
• 🐛 Issue Tracker
• 💬 Discussions