llm

package
v0.3.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 4, 2026 License: GPL-3.0 Imports: 23 Imported by: 0

Documentation

Overview

Package llm is an optional multi-provider chat client (OpenAI, Anthropic, Gemini, OpenRouter, Ollama, LM Studio, any OpenAI-compatible server, …) used for natural-language recommendation rows: "more like this" for a title and themed rows from a user's history ("Slow-burn sci-fi with a twist"). The model only ever returns titles+years; they are resolved to catalogue items via core.Catalog, so hallucinated titles simply drop out. Everything is off unless config.LLM.Enabled, results are cached, and Home rows are served stale-or-omitted while a refresh runs in the background — the LLM never blocks a Home request.

Index

Constants

View Source 
const StreamIdleTimeout = 2 * time.Minute

StreamIdleTimeout bounds silence on a model stream. A stream that keeps sending runs until the caller's own deadline; the client's whole-request timeout would cut a long answer off mid-sentence.

Variables

View Source 
var ErrStreamIdle = errors.New("the language model sent nothing for 2 minutes")

ErrStreamIdle reports a model stream that sent nothing for StreamIdleTimeout.

View Source 
var ErrToolsUnsupported = errors.New("this model can't use tools")

ErrToolsUnsupported reports that the configured provider or model cannot use tools. Callers show a clean "choose another model" message.

Functions

func StripReasoning 

func StripReasoning(content string) string

StripReasoning removes <think>…</think>-style blocks that reasoning models (DeepSeek-R1, Qwen3, …) emit before their answer, plus an unterminated leading block when the model ran out of tokens mid-thought.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client talks to the owner's chosen provider (config.LLMProviders).

func NewClient 

func NewClient(opts ClientOptions) *Client

NewClient builds a Client.

func (*Client) Chat 

func (c *Client) Chat(ctx context.Context, msgs []Message, jsonMode bool) (string, error)

Chat sends messages and returns the assistant content. Anthropic is spoken natively; Gemini and every other provider through the OpenAI chat-completions shape.

func (*Client) Enabled 

func (c *Client) Enabled() bool

Enabled reports whether the LLM is configured on.

func (*Client) ModelName added in v0.3.3 

func (c *Client) ModelName() string

ModelName returns the configured model.

func (*Client) Models added in v0.2.2 

func (c *Client) Models(ctx context.Context) ([]Model, error)

Models lists the chat models the configured provider offers, sorted by ID. It works whether or not the LLM is enabled, so the owner can pick a model before switching it on. Discovery follows appkit-ai's listModels: OpenAI keeps only GPT/o-series chat models, Gemini only models that support generateContent, and OpenAI-compatible servers drop embedding, speech and image models.

func (*Client) ProviderID added in v0.3.3 

func (c *Client) ProviderID() string

ProviderID returns the configured provider id.

func (*Client) StreamTools added in v0.3.3 

func (c *Client) StreamTools(ctx context.Context, req ToolRequest) (<-chan Event, error)

StreamTools streams a tool-calling chat. The channel closes after a done or error event.

func (*Client) ToolsSupported added in v0.3.3 

func (c *Client) ToolsSupported() (bool, string)

ToolsSupported reports whether tool calling can run, with a plain reason when it cannot.

func (*Client) UpdateConfig 

func (c *Client) UpdateConfig(cfg config.LLM)

UpdateConfig swaps the LLM settings.

type ClientOptions 

type ClientOptions struct {
	Config     config.LLM
	HTTPClient *http.Client
	Logger     *slog.Logger
}

ClientOptions build a Client from a config snapshot.

type Event added in v0.3.3 

type Event struct {
	Kind         EventKind
	Text         string
	ToolCallID   string
	ToolName     string
	InputDelta   string
	Input        json.RawMessage
	InputTokens  int
	OutputTokens int
	FinishReason string
	Err          error
}

Event is one streamed model output unit.

type EventKind added in v0.3.3 

type EventKind string

EventKind classifies stream events. 

const (
	EventTextDelta     EventKind = "text-delta"
	EventToolCallStart EventKind = "tool-call-start"
	EventToolCallDelta EventKind = "tool-call-delta"
	EventToolCall      EventKind = "tool-call"
	EventUsage         EventKind = "usage"
	EventDone          EventKind = "done"
	EventError         EventKind = "error"
)

type Message 

type Message struct {
	Role    string `json:"role"`
	Content string `json:"content"`
}

Message is one chat message.

type Model added in v0.2.2 

type Model struct {
	ID string `json:"id"`
	// Label is the provider's display name when it differs from the ID.
	Label string `json:"label,omitempty"`
}

Model is one model a provider offers.

type Service 

type Service struct {
	// contains filtered or unexported fields
}

Service turns model suggestions into catalogue titles and Home hubs.

Every model call runs in a background goroutine (single-flighted per cache key, detached from the request context, bounded by ServiceOptions.CallTimeout). Callers only ever wait as long as their own context allows:

  • ThemedRows never waits: it serves the cached rows (fresh or stale, or nothing yet) and starts a refresh when needed.
  • MoreLikeThis and ThemedRowsSync wait for the in-flight call until ctx is done, then fall back to stale results. Bound ctx with a short timeout on request paths; the result still lands in the cache.

Failures back off (ErrorBackoff) so a dead endpoint is not hammered from every page view.

func New 

func New(opts ServiceOptions) *Service

New builds a Service over a Client (share the Client if you also need raw Chat).

func (*Service) Chat added in v0.3.0 

func (s *Service) Chat(ctx context.Context, system, user string) (string, error)

Chat is one completion. Language features use it for translations. Home recommendations keep using the cached Service methods.

func (*Service) Client added in v0.3.3 

func (s *Service) Client() *Client

Enabled reports whether the underlying client is on and a catalog is wired. Client returns the underlying chat client (shared with Ask Ether so config updates propagate to both).

func (*Service) Enabled 

func (s *Service) Enabled() bool

func (*Service) Invalidate 

func (s *Service) Invalidate(userID int64)

Invalidate drops cached rows for a user (e.g. on an admin "refresh").

func (*Service) MoreLikeThis 

func (s *Service) MoreLikeThis(ctx context.Context, title *core.Title, limit int) ([]core.Title, error)

MoreLikeThis asks the model for titles similar to one title and resolves them. It waits for the model only as long as ctx allows, then returns the stale result (or nil) while the call finishes into the cache.

func (*Service) NaturalSearch 

func (s *Service) NaturalSearch(ctx context.Context, u *core.User, query string, limit int) ([]core.Title, error)

NaturalSearch resolves model suggestions through the real catalogue and falls back to ordinary search on every disabled/failed/unresolvable path. Plex still applies its mandatory visibility guard to the returned titles.

func (*Service) RefineVibe 

func (s *Service) RefineVibe(ctx context.Context, u *core.User, query string) (core.VibeQuery, bool, error)

RefineVibe implements core.VibeRefiner: it asks the model to map a query the deterministic parser could not read ("rainy sunday movies that feel like a warm hug") to the same structured filter. The reply is strictly validated against the catalogue's vocabulary, results (including "not a vibe") are cached, and the call runs under the caller's context — the search path bounds it to about a second and falls back to plain search on any failure.

func (*Service) SeasonRecap 

func (s *Service) SeasonRecap(ctx context.Context, u *core.User, key core.RatingKey, st core.State) (string, error)

SeasonRecap sends only completed episode overviews from this viewer's history. It never sends future episode names/plots to the optional model. Disabled/missing LLMs return an empty recap instead of fabricated prose.

func (*Service) ThemedRows 

func (s *Service) ThemedRows(ctx context.Context, user *core.User, seeds []core.Title, limit int) ([]core.Hub, error)

ThemedRows never blocks: it returns the user's cached rows (fresh or stale) and kicks off a background refresh when they are missing, expired, or were computed from different seeds more than MinRefresh ago. This is the shape the Home screen wants.

func (*Service) ThemedRowsContrastiveSync added in v0.2.7 

func (s *Service) ThemedRowsContrastiveSync(ctx context.Context, user *core.User, seeds, dislikes []core.Title, vetoGenres []string, limit int) ([]core.Hub, error)

ThemedRowsContrastiveSync is ThemedRowsSync with negative evidence (abandoned/ignored titles and disliked genres) so the model steers away from what the viewer bounced off.

func (*Service) ThemedRowsSync 

func (s *Service) ThemedRowsSync(ctx context.Context, user *core.User, seeds []core.Title, limit int) ([]core.Hub, error)

ThemedRowsSync is ThemedRows that waits (until ctx is done) for a needed refresh — for admin previews and warm-up jobs, not request paths.

func (*Service) UpdateConfig 

func (s *Service) UpdateConfig(cfg config.LLM)

UpdateConfig applies new endpoint/model credentials and invalidates results computed with previous settings. An old in-flight call cannot refill them.

type ServiceOptions 

type ServiceOptions struct {
	Client  *Client
	Catalog core.Catalog
	Logger  *slog.Logger
	// TTL is how long results are fresh (default 6h).
	TTL time.Duration
	// MinRefresh is the minimum age before themed rows are recomputed
	// because the user's history changed (default 15m).
	MinRefresh time.Duration
	// ErrorBackoff suppresses retries after a failed call (default 5m).
	ErrorBackoff time.Duration
	// CallTimeout bounds one background model call + resolution (default 3m).
	CallTimeout time.Duration
	// MaxEntries bounds the cache (default 1024); the least recently
	// computed entries are evicted.
	MaxEntries int
	Now        func() time.Time
}

ServiceOptions build a Service.

type StreamMessage added in v0.3.3 

type StreamMessage struct {
	Role        string
	Content     string
	ToolCalls   []ToolCall
	ToolResults []ToolResult
}

StreamMessage is a tool-capable chat message. Role is system, user, assistant or tool. Assistant messages may carry ToolCalls; tool messages carry ToolResults.

type ThemedRow 

type ThemedRow struct {
	Title  string     `json:"title"`
	Reason string     `json:"reason,omitempty"`
	Titles []TitleRef `json:"titles"`
}

ThemedRow is one named row of suggestions.

func ParseRows 

func ParseRows(content string) []ThemedRow

ParseRows extracts themed rows from model output.

type TitleRef 

type TitleRef struct {
	Title  string `json:"title"`
	Year   int    `json:"year,omitempty"`
	Type   string `json:"type,omitempty"` // movie|show|tv
	Reason string `json:"reason,omitempty"`
}

TitleRef is a title the model suggested.

func ParseTitles 

func ParseTitles(content string) []TitleRef

ParseTitles extracts title refs from model output in whatever shape it arrived: {"titles":[...]}, {"rows":[...]}, a bare array, or a plain list of "Title (Year)" lines.

func ParseTitlesLinesOnly 

func ParseTitlesLinesOnly(content string) []TitleRef

ParseTitlesLinesOnly is the plain-text fallback used by ParseRows.

type ToolCall added in v0.3.3 

type ToolCall struct {
	ID    string
	Name  string
	Input json.RawMessage
}

ToolCall is one model-requested tool call.

type ToolDef added in v0.3.3 

type ToolDef struct {
	Name        string
	Description string
	InputSchema json.RawMessage // JSON Schema object
}

ToolDef is one tool offered to a model.

type ToolRequest added in v0.3.3 

type ToolRequest struct {
	System      string
	Messages    []StreamMessage
	Tools       []ToolDef
	Temperature float64
	MaxTokens   int
}

ToolRequest is one streaming tool-calling request.

type ToolResult added in v0.3.3 

type ToolResult struct {
	ToolCallID string
	Name       string
	Content    string
	IsError    bool
}

ToolResult is one tool outcome fed back to the model.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.