Documentation
¶
Index ¶
- func NormalizeModelName(model string) string
- type ClientConfig
- type HTTPHandler
- type Manager
- func (m *Manager) BearerTokenForModel(ctx context.Context, ref string) (string, error)
- func (m *Manager) Delete(reference string, force bool) (*distribution.DeleteModelResponse, error)
- func (m *Manager) GetBundle(ref string) (types.ModelBundle, error)
- func (m *Manager) GetDiskUsage() (int64, error)
- func (m *Manager) GetLocal(ref string) (types.Model, error)
- func (m *Manager) GetRemote(ctx context.Context, ref string) (types.ModelArtifact, error)
- func (m *Manager) GetRemoteBlobURL(ref string, digest v1.Hash) (string, error)
- func (m *Manager) InStore(ref string) (bool, error)
- func (m *Manager) List() ([]*Model, error)
- func (m *Manager) Load(r io.Reader, progressWriter io.Writer) error
- func (m *Manager) Package(ref string, tag string, contextSize uint64) error
- func (m *Manager) Pull(model string, bearerToken string, r *http.Request, w http.ResponseWriter) error
- func (m *Manager) Purge() error
- func (m *Manager) Push(model string, r *http.Request, w http.ResponseWriter) error
- func (m *Manager) RawList() ([]types.Model, error)
- func (m *Manager) ResolveID(modelRef string) string
- func (m *Manager) Tag(ref, target string) error
- type Model
- type ModelCreateRequest
- type ModelPackageRequest
- type OpenAIModel
- type OpenAIModelList
- type SimpleModel
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NormalizeModelName ¶
NormalizeModelName adds the default organization prefix (ai/) and tag (:latest) if missing. It also converts Hugging Face model names to lowercase. Examples:
- "gemma3" -> "ai/gemma3:latest"
- "gemma3:v1" -> "ai/gemma3:v1"
- "myorg/gemma3" -> "myorg/gemma3:latest"
- "ai/gemma3:latest" -> "ai/gemma3:latest" (unchanged)
- "hf.co/model" -> "hf.co/model:latest" (unchanged - has registry)
- "hf.co/Model" -> "hf.co/model:latest" (converted to lowercase)
Types ¶
type ClientConfig ¶
type HTTPHandler ¶ added in v1.0.7
type HTTPHandler struct {
// contains filtered or unexported fields
}
HTTPHandler manages inference model pulls and storage.
func NewHTTPHandler ¶ added in v1.0.7
func NewHTTPHandler(log logging.Logger, c ClientConfig, allowedOrigins []string, memoryEstimator memory.MemoryEstimator) *HTTPHandler
NewHTTPHandler creates a new model's handler.
func (*HTTPHandler) RebuildRoutes ¶ added in v1.0.7
func (h *HTTPHandler) RebuildRoutes(allowedOrigins []string)
func (*HTTPHandler) ServeHTTP ¶ added in v1.0.7
func (h *HTTPHandler) ServeHTTP(w http.ResponseWriter, r *http.Request)
ServeHTTP implement net/http.HTTPHandler.ServeHTTP.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager handles the business logic for model management operations.
func NewManager ¶
func NewManager(log logging.Logger, c ClientConfig) *Manager
NewManager creates a new model models with the provided clients.
func (*Manager) BearerTokenForModel ¶
BearerTokenForModel returns the bearer token needed to pull a given model.
func (*Manager) Delete ¶ added in v1.0.7
func (m *Manager) Delete(reference string, force bool) (*distribution.DeleteModelResponse, error)
Delete deletes a model from storage and returns the delete response
func (*Manager) GetBundle ¶
func (m *Manager) GetBundle(ref string) (types.ModelBundle, error)
GetBundle returns model bundle.
func (*Manager) GetDiskUsage ¶
func (*Manager) GetLocal ¶ added in v1.0.7
GetLocal returns a single model by reference. This is the core business logic for retrieving a model from the distribution client.
func (*Manager) GetRemoteBlobURL ¶ added in v1.0.7
GetRemoteBlobURL returns the URL of a given model blob.
func (*Manager) Pull ¶ added in v1.0.7
func (m *Manager) Pull(model string, bearerToken string, r *http.Request, w http.ResponseWriter) error
Pull pulls a model to local storage. Any error it returns is suitable for writing back to the client.
type Model ¶
type Model struct {
// ID is the globally unique model identifier.
ID string `json:"id"`
// Tags are the list of tags associated with the model.
Tags []string `json:"tags,omitempty"`
// Created is the Unix epoch timestamp corresponding to the model creation.
Created int64 `json:"created"`
// Config describes the model.
Config types.Config `json:"config"`
}
func ToModelFromArtifact ¶ added in v1.0.7
func ToModelFromArtifact(artifact types.ModelArtifact) (*Model, error)
ToModelFromArtifact converts a types.ModelArtifact (typically from remote registry) to the API Model representation. Remote models don't have tags.
type ModelCreateRequest ¶
type ModelCreateRequest struct {
// From is the name of the model to pull.
From string `json:"from"`
// IgnoreRuntimeMemoryCheck indicates whether the server should check if it has sufficient
// memory to run the given model (assuming default configuration).
IgnoreRuntimeMemoryCheck bool `json:"ignore-runtime-memory-check,omitempty"`
// BearerToken is an optional bearer token for authentication.
BearerToken string `json:"bearer-token,omitempty"`
}
ModelCreateRequest represents a model create request. It is designed to follow Docker Engine API conventions, most closely following the request associated with POST /images/create. At the moment is only designed to facilitate pulls, though in the future it may facilitate model building and refinement (such as fine tuning, quantization, or distillation).
type ModelPackageRequest ¶
type ModelPackageRequest struct {
// From is the name of the source model to package from.
From string `json:"from"`
// Tag is the name to give the new packaged model.
Tag string `json:"tag"`
// ContextSize specifies the context size to set for the new model.
ContextSize uint64 `json:"context-size,omitempty"`
}
ModelPackageRequest represents a model package request, which creates a new model from an existing one with modified properties (e.g., context size).
type OpenAIModel ¶
type OpenAIModel struct {
// ID is the model tag.
ID string `json:"id"`
// Object is the object type. For OpenAIModel, it is always "model".
Object string `json:"object"`
// Created is the Unix epoch timestamp corresponding to the model creation.
Created int64 `json:"created"`
// OwnedBy is the model owner. At the moment, it is always "docker".
OwnedBy string `json:"owned_by"`
}
OpenAIModel represents a locally stored model using OpenAI conventions.
type OpenAIModelList ¶
type OpenAIModelList struct {
// Object is the object type. For OpenAIModelList, it is always "list".
Object string `json:"object"`
// Data is the list of models.
Data []*OpenAIModel `json:"data"`
}
OpenAIModelList represents a list of models using OpenAI conventions.
func ToOpenAIList ¶
func ToOpenAIList(l []types.Model) (*OpenAIModelList, error)
ToOpenAIList converts the model list to its OpenAI API representation. This function never returns a nil slice (though it may return an empty slice).
type SimpleModel ¶
SimpleModel is a wrapper that allows creating a model with modified configuration
func (*SimpleModel) Descriptor ¶
func (s *SimpleModel) Descriptor() (types.Descriptor, error)
Source Files