Help us shape the future of Go! Take the 2026 Go Developer Survey

llm

package
v0.2.53 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: GPL-3.0 Imports: 20 Imported by: 0

Documentation

Overview

Package llm is an optional multi-provider chat client (OpenAI, Anthropic, Gemini, OpenRouter, Ollama, LM Studio, any OpenAI-compatible server, …) used for natural-language recommendation rows: "more like this" for a title and themed rows from a user's history ("Slow-burn sci-fi with a twist"). The model only ever returns titles+years; they are resolved to catalogue items via core.Catalog, so hallucinated titles simply drop out. Everything is off unless config.LLM.Enabled, results are cached, and Home rows are served stale-or-omitted while a refresh runs in the background — the LLM never blocks a Home request.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func StripReasoning

func StripReasoning(content string) string

StripReasoning removes <think>…</think>-style blocks that reasoning models (DeepSeek-R1, Qwen3, …) emit before their answer, plus an unterminated leading block when the model ran out of tokens mid-thought.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client talks to the owner's chosen provider (config.LLMProviders).

func NewClient

func NewClient(opts ClientOptions) *Client

NewClient builds a Client.

func (*Client) Chat

func (c *Client) Chat(ctx context.Context, msgs []Message, jsonMode bool) (string, error)

Chat sends messages and returns the assistant content. Anthropic is spoken natively; Gemini and every other provider through the OpenAI chat-completions shape.

func (*Client) Enabled

func (c *Client) Enabled() bool

Enabled reports whether the LLM is configured on.

func (*Client) Models added in v0.2.2

func (c *Client) Models(ctx context.Context) ([]Model, error)

Models lists the chat models the configured provider offers, sorted by ID. It works whether or not the LLM is enabled, so the owner can pick a model before switching it on. Discovery follows appkit-ai's listModels: OpenAI keeps only GPT/o-series chat models, Gemini only models that support generateContent, and OpenAI-compatible servers drop embedding, speech and image models.

func (*Client) UpdateConfig

func (c *Client) UpdateConfig(cfg config.LLM)

UpdateConfig swaps the LLM settings.

type ClientOptions

type ClientOptions struct {
	Config     config.LLM
	HTTPClient *http.Client
	Logger     *slog.Logger
}

ClientOptions build a Client from a config snapshot.

type Message

type Message struct {
	Role    string `json:"role"`
	Content string `json:"content"`
}

Message is one chat message.

type Model added in v0.2.2

type Model struct {
	ID string `json:"id"`
	// Label is the provider's display name when it differs from the ID.
	Label string `json:"label,omitempty"`
}

Model is one model a provider offers.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service turns model suggestions into catalogue titles and Home hubs.

Every model call runs in a background goroutine (single-flighted per cache key, detached from the request context, bounded by ServiceOptions.CallTimeout). Callers only ever wait as long as their own context allows:

  • ThemedRows never waits: it serves the cached rows (fresh or stale, or nothing yet) and starts a refresh when needed.
  • MoreLikeThis and ThemedRowsSync wait for the in-flight call until ctx is done, then fall back to stale results. Bound ctx with a short timeout on request paths; the result still lands in the cache.

Failures back off (ErrorBackoff) so a dead endpoint is not hammered from every page view.

func New

func New(opts ServiceOptions) *Service

New builds a Service over a Client (share the Client if you also need raw Chat).

func (*Service) Enabled

func (s *Service) Enabled() bool

Enabled reports whether the underlying client is on and a catalog is wired.

func (*Service) Invalidate

func (s *Service) Invalidate(userID int64)

Invalidate drops cached rows for a user (e.g. on an admin "refresh").

func (*Service) MoreLikeThis

func (s *Service) MoreLikeThis(ctx context.Context, title *core.Title, limit int) ([]core.Title, error)

MoreLikeThis asks the model for titles similar to one title and resolves them. It waits for the model only as long as ctx allows, then returns the stale result (or nil) while the call finishes into the cache.

func (*Service) NaturalSearch

func (s *Service) NaturalSearch(ctx context.Context, u *core.User, query string, limit int) ([]core.Title, error)

NaturalSearch resolves model suggestions through the real catalogue and falls back to ordinary search on every disabled/failed/unresolvable path. Plex still applies its mandatory visibility guard to the returned titles.

func (*Service) RefineVibe

func (s *Service) RefineVibe(ctx context.Context, u *core.User, query string) (core.VibeQuery, bool, error)

RefineVibe implements core.VibeRefiner: it asks the model to map a query the deterministic parser could not read ("rainy sunday movies that feel like a warm hug") to the same structured filter. The reply is strictly validated against the catalogue's vocabulary, results (including "not a vibe") are cached, and the call runs under the caller's context — the search path bounds it to about a second and falls back to plain search on any failure.

func (*Service) SeasonRecap

func (s *Service) SeasonRecap(ctx context.Context, u *core.User, key core.RatingKey, st core.State) (string, error)

SeasonRecap sends only completed episode overviews from this viewer's history. It never sends future episode names/plots to the optional model. Disabled/missing LLMs return an empty recap instead of fabricated prose.

func (*Service) ThemedRows

func (s *Service) ThemedRows(ctx context.Context, user *core.User, seeds []core.Title, limit int) ([]core.Hub, error)

ThemedRows never blocks: it returns the user's cached rows (fresh or stale) and kicks off a background refresh when they are missing, expired, or were computed from different seeds more than MinRefresh ago. This is the shape the Home screen wants.

func (*Service) ThemedRowsContrastiveSync added in v0.2.7

func (s *Service) ThemedRowsContrastiveSync(ctx context.Context, user *core.User, seeds, dislikes []core.Title, vetoGenres []string, limit int) ([]core.Hub, error)

ThemedRowsContrastiveSync is ThemedRowsSync with negative evidence (abandoned/ignored titles and disliked genres) so the model steers away from what the viewer bounced off.

func (*Service) ThemedRowsSync

func (s *Service) ThemedRowsSync(ctx context.Context, user *core.User, seeds []core.Title, limit int) ([]core.Hub, error)

ThemedRowsSync is ThemedRows that waits (until ctx is done) for a needed refresh — for admin previews and warm-up jobs, not request paths.

func (*Service) UpdateConfig

func (s *Service) UpdateConfig(cfg config.LLM)

UpdateConfig applies new endpoint/model credentials and invalidates results computed with previous settings. An old in-flight call cannot refill them.

type ServiceOptions

type ServiceOptions struct {
	Client  *Client
	Catalog core.Catalog
	Logger  *slog.Logger
	// TTL is how long results are fresh (default 6h).
	TTL time.Duration
	// MinRefresh is the minimum age before themed rows are recomputed
	// because the user's history changed (default 15m).
	MinRefresh time.Duration
	// ErrorBackoff suppresses retries after a failed call (default 5m).
	ErrorBackoff time.Duration
	// CallTimeout bounds one background model call + resolution (default 3m).
	CallTimeout time.Duration
	// MaxEntries bounds the cache (default 1024); the least recently
	// computed entries are evicted.
	MaxEntries int
	Now        func() time.Time
}

ServiceOptions build a Service.

type ThemedRow

type ThemedRow struct {
	Title  string     `json:"title"`
	Reason string     `json:"reason,omitempty"`
	Titles []TitleRef `json:"titles"`
}

ThemedRow is one named row of suggestions.

func ParseRows

func ParseRows(content string) []ThemedRow

ParseRows extracts themed rows from model output.

type TitleRef

type TitleRef struct {
	Title  string `json:"title"`
	Year   int    `json:"year,omitempty"`
	Type   string `json:"type,omitempty"` // movie|show|tv
	Reason string `json:"reason,omitempty"`
}

TitleRef is a title the model suggested.

func ParseTitles

func ParseTitles(content string) []TitleRef

ParseTitles extracts title refs from model output in whatever shape it arrived: {"titles":[...]}, {"rows":[...]}, a bare array, or a plain list of "Title (Year)" lines.

func ParseTitlesLinesOnly

func ParseTitlesLinesOnly(content string) []TitleRef

ParseTitlesLinesOnly is the plain-text fallback used by ParseRows.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL