Documentation
¶
Overview ¶
Package modeldconn is the runtime's client seam to the modeld daemon: it resolves the current lease leader (via modeldprobe), dials it over the gRPC transport, and opens sessions. Local backend providers (llama, openvino) call OpenSession instead of constructing an in-process CGO session, so the runtime stays pure Go and talks to modeld only through runtime/transport.
Index ¶
- func Available() bool
- func Backend() string
- func Describe(ctx context.Context, ref ModelRef, cfg transport.Config) (transport.ModelInfo, error)
- func Embed(ctx context.Context, ref ModelRef, cfg transport.Config, text string) (transport.EmbedResult, error)
- func LoadModel(ctx context.Context, ref ModelRef, cfg transport.Config) (transport.ActiveModel, error)
- func OpenSession(ctx context.Context, ref ModelRef, cfg transport.Config) (transport.Session, error)
- func SetDataRoot(root string)
- func Status(ctx context.Context) (transport.DaemonStatus, error)
- func UnloadModel(ctx context.Context, expectedGeneration uint64) error
- type ModelRef
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Available ¶
func Available() bool
Available is the cheap, offline check (lease inspection, no network): is a modeld owner currently holding a fresh lease? Providers use it to gate capability advertisement without a round-trip per call.
func Backend ¶
func Backend() string
Backend is the cheap, offline check for the inference backend the running modeld owner serves ("llama"/"openvino"/"none"), read from the lease. Empty when no owner holds a fresh lease. Providers gate on this so the backend modeld is NOT in advertises no local capability instead of failing deep in the engine.
func Describe ¶
Describe asks the running modeld owner for a model's capabilities, read from the model metadata by the backend that serves it. This is the modeld→runtime info-flow for model facts (e.g. the trained context window): the runtime is the consumer and never parses model files itself. A not-running/unreachable owner surfaces the probe's typed error.
func Embed ¶ added in v0.32.5
func Embed(ctx context.Context, ref ModelRef, cfg transport.Config, text string) (transport.EmbedResult, error)
Embed asks the running modeld owner to compute a one-shot embedding for text. It uses the same typed model handle and owner fence as Describe, but does not open a persistent session.
func LoadModel ¶ added in v0.32.5
func LoadModel(ctx context.Context, ref ModelRef, cfg transport.Config) (transport.ActiveModel, error)
LoadModel explicitly activates the single modeld slot, switching away from an idle active model when needed.
func OpenSession ¶
func OpenSession(ctx context.Context, ref ModelRef, cfg transport.Config) (transport.Session, error)
OpenSession resolves the modeld leader, confirms it actually answers a health probe, and opens a session on it. The returned session is resident in modeld; the caller drives EnsurePrefix/PrefillSuffix/Decode over the wire. A not-running/unreachable owner surfaces the probe's typed error; a model typed for a backend the daemon does not serve surfaces transport.ErrBackendMismatch.
func SetDataRoot ¶
func SetDataRoot(root string)
SetDataRoot overrides the data root used to locate the owner lease.
Types ¶
type ModelRef ¶
ModelRef is the typed model handle the runtime passes to modeld: a logical Name + backend Type + content Digest form the cache identity, and Path is the runtime-resolved on-disk location (GGUF file or IR directory) the daemon loads from. Type lets the daemon reject a model it does not serve.
Source Files