Documentation
¶
Overview ¶
Package llmd provides integration with the llm-d Router's OpenAI-compatible API.
Index ¶
- Variables
- func New(cfg providers.ProviderConfig, opts providers.ProviderOptions) core.Provider
- type ControlConfig
- type Provider
- func (p *Provider) ChatCompletion(ctx context.Context, req *core.ChatRequest) (*core.ChatResponse, error)
- func (p *Provider) Embeddings(ctx context.Context, req *core.EmbeddingRequest) (*core.EmbeddingResponse, error)
- func (p *Provider) ListModels(ctx context.Context) (*core.ModelsResponse, error)
- func (p *Provider) Passthrough(ctx context.Context, req *core.PassthroughRequest) (*core.PassthroughResponse, error)
- func (p *Provider) Responses(ctx context.Context, req *core.ResponsesRequest) (*core.ResponsesResponse, error)
- func (p *Provider) SetBaseURL(baseURL string)
- func (p *Provider) StreamChatCompletion(ctx context.Context, req *core.ChatRequest) (io.ReadCloser, error)
- func (p *Provider) StreamResponses(ctx context.Context, req *core.ResponsesRequest) (io.ReadCloser, error)
Constants ¶
This section is empty.
Variables ¶
var Registration = providers.Registration{ Type: "llmd", New: New, PassthroughSemanticEnricher: passthroughSemanticEnricher, Discovery: providers.DiscoveryConfig{ RequireBaseURL: true, AllowAPIKeyless: true, }, }
Registration provides factory registration for the llm-d provider. An llm-d deployment has no universal endpoint, so base_url is required. Router authentication is optional and depends on the Gateway in front of llm-d.
Functions ¶
func New ¶
func New(cfg providers.ProviderConfig, opts providers.ProviderOptions) core.Provider
New creates an llm-d provider.
Types ¶
type ControlConfig ¶
ControlConfig contains trusted llm-d request-classification settings.
type Provider ¶
type Provider struct {
// contains filtered or unexported fields
}
Provider implements the OpenAI-compatible inference subset understood by llm-d's openai-parser. It intentionally does not advertise native files, batches, audio, or Responses lifecycle interfaces.
func (*Provider) ChatCompletion ¶
func (p *Provider) ChatCompletion(ctx context.Context, req *core.ChatRequest) (*core.ChatResponse, error)
ChatCompletion sends a chat completion request to llm-d.
func (*Provider) Embeddings ¶
func (p *Provider) Embeddings(ctx context.Context, req *core.EmbeddingRequest) (*core.EmbeddingResponse, error)
Embeddings sends an embeddings request to llm-d.
func (*Provider) ListModels ¶
ListModels asks the llm-d route for its OpenAI-compatible model inventory. Operators should configure providers.<name>.models when their HTTPRoute does not forward GET /v1/models to a model server.
func (*Provider) Passthrough ¶
func (p *Provider) Passthrough(ctx context.Context, req *core.PassthroughRequest) (*core.PassthroughResponse, error)
Passthrough routes an opaque request through llm-d after removing all client-supplied credentials and llm-d control headers.
func (*Provider) Responses ¶
func (p *Provider) Responses(ctx context.Context, req *core.ResponsesRequest) (*core.ResponsesResponse, error)
Responses sends a Responses API request to llm-d.
func (*Provider) SetBaseURL ¶
SetBaseURL changes the llm-d Router endpoint.
func (*Provider) StreamChatCompletion ¶
func (p *Provider) StreamChatCompletion(ctx context.Context, req *core.ChatRequest) (io.ReadCloser, error)
StreamChatCompletion streams a chat completion request through llm-d.
func (*Provider) StreamResponses ¶
func (p *Provider) StreamResponses(ctx context.Context, req *core.ResponsesRequest) (io.ReadCloser, error)
StreamResponses streams a Responses API request through llm-d.
Source Files