README
¶
Provider Package Architecture
This package contains the provider-specific integrations for the LLM proxy. Each provider is implemented in its own subfolder, following a consistent structure to ensure maintainability and extensibility.
File Structure
Within each provider folder, following files typically exist:
provider.go: Implements theProviderinterface for the specific provider, handling all communication and logic required to interact with the provider's API. May use a providers SDK, orhttpclientto send requests.provider_test.go: Contains integration tests for the provider implementation, redirecting them tosuite.go.mapping.go: Contains mapping logic to convert between the core LLM proxy types (from thecorepackage) and the provider's domain types.types.go: Defines provider-specific domain types, such as request/response structs, enums, or helper types that are unique to the provider's API. Not necessary if the provider comes with a maintained go SDK.
Adding a New Provider
When implementing a new provider, follow these steps:
-
Check for a Go SDK:
- If the provider offers an official or community-maintained Go SDK, prefer using it directly in
provider.goand for type definitions.
- If the provider offers an official or community-maintained Go SDK, prefer using it directly in
-
Check for OpenAPI Specification:
- If no Go SDK is available, check if the provider publishes an OpenAPI (Swagger) YAML file. You can use this to generate Go domain types automatically with oapi-codegen into a
types.gen.gofile, reducing manual work and ensuring type safety.
- If no Go SDK is available, check if the provider publishes an OpenAPI (Swagger) YAML file. You can use this to generate Go domain types automatically with oapi-codegen into a
-
Manual Implementation:
- If neither a Go SDK nor an OpenAPI spec is available, manually implement the provider's types in a
types.gofile.
- If neither a Go SDK nor an OpenAPI spec is available, manually implement the provider's types in a
The testsuite Package
The testsuite subfolder provides a reusable set of tests and test utilities for provider implementations. It defines a standard interface and a collection of test cases that all providers should pass to ensure consistent behavior and compatibility with the LLM proxy's expectations.
Documentation
¶
Overview ¶
Package provider contains common constants for LLM providers
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrIncompleteResponse indicates that the response from the LLM is incomplete ErrIncompleteResponse = errors.New("response is incomplete") // ErrNotSupported is used to indicate non-supported features ErrNotSupported = errors.New("not supported") )
Functions ¶
This section is empty.
Types ¶
This section is empty.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package anthropic implements the Anthropic LLM provider using the official anthropic-sdk-go.
|
Package anthropic implements the Anthropic LLM provider using the official anthropic-sdk-go. |
|
Package gemini implements the Gemini LLM provider using the Google genai SDK.
|
Package gemini implements the Gemini LLM provider using the Google genai SDK. |
|
Package mistral implements the Mistral LLM provider.
|
Package mistral implements the Mistral LLM provider. |
|
Package openai implements the OpenAI LLM provider.
|
Package openai implements the OpenAI LLM provider. |
|
Package selfhosted implements the SelfHosted LLM provider for routing chat requests to operator-run inference servers (llama-server, vLLM, Ollama, etc.) via the OpenAI-compatible Chat Completions API.
|
Package selfhosted implements the SelfHosted LLM provider for routing chat requests to operator-run inference servers (llama-server, vLLM, Ollama, etc.) via the OpenAI-compatible Chat Completions API. |