Documentation
¶
Overview ¶
Package llm defines a provider-agnostic interface for streaming chat completions against OpenAI, Anthropic, and Azure AI Foundry.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type CallOptions ¶
type CallOptions struct {
Model string
MaxTokens int
APIKey string
// Endpoint is used by Azure AI Foundry only; OpenAI and Anthropic ignore it.
Endpoint string
// Deployment (Azure AI Foundry): the model-deployment name.
Deployment string
}
CallOptions controls a single provider invocation.
type Message ¶
type Message struct {
Role Role
Content string
ToolUses []ToolUse // RoleAssistant: tool_use blocks the model emitted
ToolUseID string // RoleTool: id of the call this message answers
}
Message is a single chat message.
type Provider ¶
type Provider interface {
Chat(ctx context.Context, messages []Message, opts CallOptions, onChunk func(StreamChunk)) (Usage, error)
}
Provider executes a single chat completion with streaming output.
Implementations MUST:
- stream chunks to onChunk as they arrive (empty deltas are skipped),
- return Usage with final input/output token counts when reported by the upstream API; zero values are returned if the provider does not emit a usage record,
- honor ctx cancellation / deadline,
- return an error wrapping the underlying SDK error, prefixed with the provider name (e.g. "openai chat: ..."), when the streamed call ultimately fails.
func NewAnthropic ¶
NewAnthropic returns a Provider backed by github.com/anthropics/anthropic-sdk-go. When baseURL is empty, the SDK's default (api.anthropic.com) is used.
func NewAzureFoundry ¶
func NewAzureFoundry() Provider
NewAzureFoundry returns a Provider backed by Azure AI Foundry (Azure OpenAI). It uses the openai-go/v3 client together with its `azure` subpackage, which handles Azure's URL shape (/openai/deployments/{deployment}/chat/completions) and `api-key` header authentication.
Endpoint and Deployment must be supplied via CallOptions.
func NewOpenAI ¶
NewOpenAI returns a Provider backed by github.com/openai/openai-go. When baseURL is empty, the SDK's default (api.openai.com) is used.
func NewProvider ¶
NewProvider returns a default-configured provider by its config name. For testing with mock endpoints, call NewOpenAI/NewAnthropic directly. CRONFOUNDRY_OPENAI_BASE_URL, CRONFOUNDRY_ANTHROPIC_BASE_URL, and CRONFOUNDRY_OPENROUTER_BASE_URL override the respective API endpoints (useful in tests and local development).
type Role ¶
type Role string
Role is the conversational role of a Message.
const ( RoleSystem Role = "system" RoleUser Role = "user" RoleAssistant Role = "assistant" RoleTool Role = "tool" )
Conversational roles supported by the Provider interface. The chat protocol currently distinguishes only system (instructions) and user (prompt) messages; assistant replies are streamed back as StreamChunks.
type StreamChunk ¶
type StreamChunk struct {
Delta string
}
StreamChunk is an incremental portion of the assistant's response.
type ToolCapableProvider ¶
type ToolCapableProvider interface {
Provider
ChatTurn(
ctx context.Context,
messages []Message,
tools []ToolDef,
opts CallOptions,
onChunk func(StreamChunk),
) (TurnResult, error)
}
ToolCapableProvider supports tool-aware completion turns.
func NewCopilotEnterprise ¶
func NewCopilotEnterprise() ToolCapableProvider
NewCopilotEnterprise returns a ToolCapableProvider backed by the GitHub Copilot Enterprise chat completions API (OpenAI-compatible).
The caller must supply a Copilot **IDE token** (NOT the OAuth access token) as CallOptions.APIKey. IDE tokens are minted by webapi.ResolveCopilotToken via api.github.com/copilot_internal/v2/token and live ~30 min; the runner fetches one per run from /internal/runs/{id}/copilot-token.
Copilot-Integration-Id must be one of GitHub's accepted values ("vscode-chat", "jetbrains", "vim", etc.) — the endpoint rejects arbitrary identifiers. We use "vscode-chat" because it's the canonical chat-completion path; arbitrary user-agents like "cronfoundry" produce 401s from the model picker.
type ToolDef ¶
type ToolDef struct {
Name string
Description string
InputSchema json.RawMessage
}
ToolDef describes a tool available to the model.
type ToolUse ¶
type ToolUse struct {
ID string
Name string
Input json.RawMessage
}
ToolUse represents a tool invocation emitted by the model.
Source Files