mlx

package

v1.0.10 Latest Latest Go to latest Published: Dec 17, 2025 License: Apache-2.0 Imports: 14 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/docker/model-runner

Links

Open Source Insights

Documentation ¶

Index ¶

Constants
Variables
func GetMaxTokens(modelCfg types.Config, backendCfg *inference.BackendConfiguration) *uint64
func New(log logging.Logger, modelManager *models.Manager, serverLog logging.Logger, ...) (inference.Backend, error)
type Config
- func NewDefaultMLXConfig() *Config
- func (c *Config) GetArgs(bundle types.ModelBundle, socket string, mode inference.BackendMode, ...) ([]string, error)

Constants ¶

View Source

const (
	// Name is the backend name.
	Name = "mlx"
)

Variables ¶

View Source

var ErrStatusNotFound = errors.New("Python or mlx-lm not found")

Functions ¶

func GetMaxTokens ¶ added in v1.0.5

func GetMaxTokens(modelCfg types.Config, backendCfg *inference.BackendConfiguration) *uint64

GetMaxTokens returns the max tokens (context size) from model config or backend config. Model config takes precedence over backend config. Returns nil if neither is specified (MLX will use model defaults).

func New ¶

func New(log logging.Logger, modelManager *models.Manager, serverLog logging.Logger, conf *Config) (inference.Backend, error)

New creates a new MLX-based backend.

Types ¶

type Config ¶ added in v1.0.5

type Config struct {
	// Args are the base arguments that are always included.
	Args []string
}

Config is the configuration for the MLX backend.

func NewDefaultMLXConfig ¶ added in v1.0.5

func NewDefaultMLXConfig() *Config

NewDefaultMLXConfig creates a new MLXConfig with default values.

func (*Config) GetArgs ¶ added in v1.0.5

func (c *Config) GetArgs(bundle types.ModelBundle, socket string, mode inference.BackendMode, config *inference.BackendConfiguration) ([]string, error)

GetArgs implements BackendConfig.GetArgs.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL