Documentation
¶
Overview ¶
Package llm is the only way apps run LLM inference (docs/adr/0009-copilot-sdk-llm-gateway.md): a thin, provider-neutral client for platformd's gateway. Apps never talk to a model provider directly.
Index ¶
- Variables
- func Cosine(a, b []float32) float64
- type Client
- func (c *Client) Classify(ctx context.Context, text string, categories []string) (string, error)
- func (c *Client) Complete(ctx context.Context, prompt string, opts ...Option) (string, error)
- func (c *Client) CompleteJSON(ctx context.Context, prompt string, out any, opts ...Option) error
- func (c *Client) Embed(ctx context.Context, texts []string) (*Embedded, error)
- func (c *Client) EmbedQuery(ctx context.Context, q string) (*Embedded, error)
- func (c *Client) Healthy(ctx context.Context) error
- type Embedded
- type Option
- type Tool
Constants ¶
This section is empty.
Variables ¶
ErrEmbedUnavailable reports that the gateway has no embeddings backend (BESPOKE_LEMONADE_URL unset on platformd). Callers MUST degrade — switch semantic features off and keep lexical paths working — rather than fail the surrounding feature (ADR-0029).
Functions ¶
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
func New ¶
New returns the gateway client for an app. The gateway address comes from BESPOKE_LLM_URL (set in the production env file); the default matches platformd's local internal listener.
func (*Client) Classify ¶
Classify returns the category from categories that best fits text — the canonical tier-1 capability helper (docs/adr/0012-internal-services-two-tier.md): pure composition over CompleteJSON, with the model's answer validated against the caller's list so apps never receive an invented category.
func (*Client) CompleteJSON ¶
CompleteJSON asks for JSON only and unmarshals the response into out (pass a pointer). Markdown fences are stripped if the model adds them.
func (*Client) Embed ¶ added in v0.3.0
Embed embeds documents for storage and later ranking with llm.Cosine (ADR-0029). Mechanical like Classify — no options, never user-brief-tagged. At most 64 texts of 8KB each per call; embedding is best-effort by design, so a failed Embed should never fail the write that triggered it. Returns ErrEmbedUnavailable (wrapped) when the gateway has no backend.
func (*Client) EmbedQuery ¶ added in v0.3.0
EmbedQuery embeds a search query for ranking against stored document vectors (retrieval models treat queries and documents asymmetrically; the gateway applies the model's task prefixes). One text in, one vector out at Vectors[0].
type Embedded ¶ added in v0.3.0
Embedded is the result of an embedding call: one vector per input text, in input order, plus the backend model that produced them. Store Model alongside vectors — a model change invalidates them (ADR-0029).
type Option ¶
type Option func(*request)
func WithBuiltins ¶
WithBuiltins re-enables curated runtime builtins for assistant surfaces (ADR-0024) — e.g. "web_search", "web_fetch", and the read-only GitHub tools. The gateway rejects names outside its allowlist, so this is a request, not a grant. Requires WithUser, like tools.
func WithSystem ¶
WithSystem adds system instructions to a completion.
type Tool ¶
type Tool struct {
Name string `json:"name"`
Description string `json:"description"`
Schema map[string]any `json:"schema"`
URL string `json:"url"`
}
Tool is an action the model may call during a completion (ADR-0021). The gateway invokes URL with the tagged user's identity when the model requests it.
Source Files