Documentation
¶
Overview ¶
Package dmrmodels handles Docker Model Runner endpoint discovery and model listing. It is deliberately free of the OpenAI SDK (and any other model client dependency) so that pkg/runtime — which only needs DMR model discovery — does not link the full DMR provider client into embedders' binaries. The full client lives in the parent dmr package.
Index ¶
- Variables
- func DefaultHostURL() string
- func DockerModelEndpointAndEngine(ctx context.Context) (endpoint, engine string, err error)
- func IsNotInstalledError(err error) bool
- func ListModels(ctx context.Context) ([]string, error)
- func ListModelsAt(ctx context.Context, httpClient *http.Client, baseURL string) ([]string, error)
- func ResolveBaseURL(ctx context.Context, cfg *latest.ModelConfig, endpoint string) (string, *http.Client)
- type Metadata
- type Model
Constants ¶
This section is empty.
Variables ¶
var ErrNotInstalled = errors.New("docker model runner is not available\nplease install it and try again (https://docs.docker.com/ai/model-runner/get-started/)")
ErrNotInstalled is returned when Docker Model Runner is not installed.
Functions ¶
func DefaultHostURL ¶ added in v1.121.0
func DefaultHostURL() string
DefaultHostURL is the default DMR URL when running on the host with no explicit endpoint. It targets the standard local model-runner port.
func DockerModelEndpointAndEngine ¶
DockerModelEndpointAndEngine shells out to `docker model status --json` and returns the resolved endpoint URL and the active inference engine name.
func IsNotInstalledError ¶
IsNotInstalledError reports whether a `docker model` invocation failed because the Docker installation predates Model Runner (the CLI rejects the --json flag). Matching on content rather than the exact message keeps the detection stable across docker CLI usage-text changes.
func ListModels ¶
ListModels returns sorted, unique IDs of locally available models.
func ListModelsAt ¶
ListModelsAt lists model IDs at an already resolved endpoint.
func ResolveBaseURL ¶
func ResolveBaseURL(ctx context.Context, cfg *latest.ModelConfig, endpoint string) (string, *http.Client)
ResolveBaseURL determines the correct base URL to talk to Docker Model Runner, mirroring the behavior of the `docker model` CLI as closely as possible.
High‑level rules:
- If the user explicitly configured a BaseURL or MODEL_RUNNER_HOST, use that (no fallbacks).
- For Desktop endpoints (model-runner.docker.internal) on the host, route through the Docker Engine experimental endpoints prefix over the Unix socket.
- For standalone / offload endpoints like http://172.17.0.1:12435/engines/v1/, use localhost:<port>/engines/v1/ on the host, and the gateway IP:port inside containers.
- Keep a small compatibility workaround for the legacy http://:0/engines/v1/ endpoint.
- Test connectivity and try fallback URLs if the primary endpoint is unreachable.
It also returns an *http.Client when a custom transport (e.g., Docker Unix socket) is needed.
Types ¶
type Metadata ¶ added in v1.142.0
type Metadata struct {
ContextWindow int32 `json:"context_window,omitempty"`
Architecture string `json:"architecture,omitempty"`
Parameters string `json:"parameters,omitempty"`
Quantization string `json:"quantization,omitempty"`
Size string `json:"size,omitempty"`
}
Metadata describes the packaged model, not necessarily its running configuration.
type Model ¶ added in v1.142.0
Model is an entry from the OpenAI-compatible models endpoint.
func GetModel ¶ added in v1.142.0
GetModel retrieves metadata using the runner's own model-reference resolution.
func ListModelsWithMetadata ¶ added in v1.142.0
ListModelsWithMetadata discovers the runner and lists its locally available models.
func ListModelsWithMetadataAt ¶ added in v1.142.0
func ListModelsWithMetadataAt(ctx context.Context, httpClient *http.Client, baseURL string) ([]Model, error)
ListModelsWithMetadataAt retains optional DMR metadata, including on older runners that return only IDs. The result is sorted and deduplicated by ID.
Source Files