config

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 12, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Overview

Package config provides configuration management for the xw application.

This package handles all configuration-related functionality including:

  • Server configuration (host, port, address)
  • Storage paths (config directory, models directory)
  • Default values and environment-specific settings

The configuration is designed to be flexible and can be customized for different deployment scenarios (development, production, systemd service).

Package config provides configuration version management functionality.

This file implements configuration package management including:

  • Fetching and parsing package registry (packages.json)
  • Downloading configuration packages
  • Version compatibility checking
  • Package extraction and verification

Package config - device_adapter.go provides adapters for device configuration.

This module converts device configuration into formats used by the device manager, avoiding circular dependencies between config and device packages.

Package config - device_config.go implements device configuration loading and management.

This module provides a flexible configuration system for AI chip detection and management. Device configurations are loaded from YAML files, allowing easy addition of new chip types without code changes.

The configuration system supports multiple bus types (PCIe, CXL, virtual) and provides a clean abstraction for chip vendor and model definitions.

Package config - model_config.go implements model configuration loading and management.

This module provides a flexible configuration system for AI model definitions. Model configurations are loaded from YAML files, allowing easy addition of new models without code changes.

The configuration system supports model metadata, hardware compatibility, runtime backends, and quantization strategies.

Package config provides configuration management for the xw CLI tool.

This file defines extended sandbox configuration structures, which allow users to define device sandbox configurations for additional chip types without modifying the source code. This is particularly useful for supporting niche accelerator chips in a flexible and maintainable way.

Extended sandbox configurations are defined within devices.yaml under each chip model's ext_sandboxes field.

Index

Constants

View Source
const (
	// DefaultServerHost is the default server host address.
	// The server listens on localhost by default for security.
	DefaultServerHost = "localhost"

	// DefaultServerPort is the default server port.
	// Port 11581 is used as it doesn't require root privileges.
	DefaultServerPort = 11581

	// DefaultRegistry is the default configuration package registry URL.
	DefaultRegistry = "https://xw.tsingmao.com/packages.json"

	// DefaultConfigDirName is the default configuration directory name.
	// This directory is created in the user's home directory.
	DefaultConfigDirName = ".xw"

	// DefaultDataDirName is the default data directory name.
	// This subdirectory under config dir contains all runtime data.
	DefaultDataDirName = "data"

	// DefaultModelsDir is the default models directory name.
	// Model files are stored in this subdirectory within the data directory.
	DefaultModelsDir = "models"
)
View Source
const (
	// ServerConfFileName is the name of the server configuration file
	ServerConfFileName = "server.conf"

	// ServerNameLength is the length of the server name
	ServerNameLength = 6
)

Variables

This section is empty.

Functions

func ClearModelsConfigCache

func ClearModelsConfigCache()

ClearModelsConfigCache clears the model configuration cache.

This function invalidates the cached model configuration, forcing the next LoadModelsConfig call to read from disk.

func FindChipModelByIdentifier

func FindChipModelByIdentifier(config *DevicesConfig, vendorID, deviceID, subsystemDeviceID string) (*ChipVendorConfig, *ChipModelConfig, *ChipVariant)

FindChipModelByIdentifier searches for a chip model by PCIe hardware identifier.

This method is used during device detection to match discovered hardware against known chip models. It supports variant matching for chip sub-versions.

Three-phase matching logic (优雅降级):

Phase 1 (Variant match): If model has variants, try to match subsystem_device_id
  - Returns base model + matched variant
Phase 2 (Legacy exact match): Try configs WITH subsystem_device_id (deprecated)
  - For backward compatibility with old config format
Phase 3 (Fallback): Match configs WITHOUT subsystem_device_id or variants
  - Acts as generic fallback for unknown variants

Configuration example (new format with variants):

  • config_key: ascend-910b device_id: "0xd802" variants:
  • subsystem_device_id: "0x0001" variant_key: "ascend-910b1" variant_name: "910B1"
  • subsystem_device_id: "0x0002" variant_key: "ascend-910b2" variant_name: "910B2" # Base config acts as fallback if no variant matches

Parameters:

  • config: DevicesConfig to search
  • vendorID: PCIe vendor identifier (e.g., "0x19e5")
  • deviceID: PCIe device identifier (e.g., "0xd802")
  • subsystemDeviceID: PCIe subsystem device identifier (e.g., "0x0001"), can be empty

Returns:

  • Pointer to ChipVendorConfig for the vendor if found
  • Pointer to ChipModelConfig (base model) if found
  • Pointer to ChipVariant if a variant matched (nil if using base config)
  • All nil if not found

func GenerateServerName

func GenerateServerName() string

GenerateServerName generates a random 6-character server name consisting of uppercase, lowercase letters and numbers

func GetAllConfigKeys

func GetAllConfigKeys(config *DevicesConfig) []string

GetAllConfigKeys returns a list of all config keys defined in the configuration.

Useful for validation and displaying available chip types.

Parameters:

  • config: DevicesConfig to extract keys from

Returns:

  • Slice of config key strings

func GetAllModelIDs

func GetAllModelIDs(config *ModelsConfig) []string

GetAllModelIDs returns a list of all model IDs defined in the configuration.

Useful for validation and displaying available models.

Parameters:

  • config: ModelsConfig to extract IDs from

Returns:

  • Slice of model ID strings

func GetConfigKeyByModelName

func GetConfigKeyByModelName(modelName string) (string, error)

GetConfigKeyByModelName returns the configuration key for a chip model name.

This function looks up the ConfigKey for a given ModelName from the device configuration. The ConfigKey is used in runtime configuration files (e.g., runtime_images.yaml).

Parameters:

  • modelName: Human-readable chip model name (e.g., "Ascend 910B", "Ascend 310P")

Returns:

  • Configuration key (e.g., "ascend-910b", "ascend-310p")
  • Error if the model name is not found or configuration loading fails

Example:

configKey, err := GetConfigKeyByModelName("Ascend 310P")
// Returns: "ascend-310p", nil

func GetImageForChipAndEngine

func GetImageForChipAndEngine(config RuntimeImagesConfig, chipModel, engine, arch string) (string, error)

GetImageForChipAndEngine returns the Docker image for a specific chip model, engine, and architecture.

This function looks up the appropriate Docker image based on the chip model (e.g., "ascend-910b", "ascend-310p"), inference engine (e.g., "vllm", "mindie"), and CPU architecture (e.g., "arm64", "amd64").

Parameters:

  • config: RuntimeImagesConfig to query
  • chipModel: Chip model identifier (e.g., "ascend-910b", "ascend-310p")
  • engine: Inference engine name (e.g., "vllm", "mindie")
  • arch: CPU architecture (e.g., "arm64", "amd64")

Returns:

  • Docker image URL if found
  • Error if chip model, engine, or architecture is not configured

Example:

image, err := GetImageForChipAndEngine(config, "ascend-910b", "vllm", "arm64")
// Returns: "quay.io/ascend/vllm-ascend:v0.11.0rc0-arm64"

func GetImageForChipAndEngineAuto

func GetImageForChipAndEngineAuto(config RuntimeImagesConfig, chipModel, engine string) (string, error)

GetImageForChipAndEngineAuto automatically detects the system architecture and returns the appropriate Docker image.

This is a convenience function that calls GetImageForChipAndEngine with the current system's architecture.

Parameters:

  • config: RuntimeImagesConfig to query
  • chipModel: Chip model identifier
  • engine: Inference engine name

Returns:

  • Docker image URL for the current architecture
  • Error if lookup fails or architecture is not supported

Example:

image, err := GetImageForChipAndEngineAuto(config, "ascend-910b", "vllm")
// Automatically uses "arm64" or "amd64" based on system

func GetSupportedChipModels

func GetSupportedChipModels(config RuntimeImagesConfig) []string

GetSupportedChipModels returns a list of all configured chip models.

Parameters:

  • config: RuntimeImagesConfig to query

Returns:

  • Slice of chip model identifiers

func GetSupportedDeviceTypes

func GetSupportedDeviceTypes() ([]api.DeviceType, error)

GetSupportedDeviceTypes returns all device types from device configuration.

This function reads the device configuration and extracts all config_key values, which represent the supported device types.

Returns:

  • Slice of DeviceType values from configuration
  • Error if configuration cannot be loaded

Example:

types, err := GetSupportedDeviceTypes()
if err != nil {
    log.Fatalf("Failed to get device types: %v", err)
}
fmt.Printf("Supported device types: %v\n", types)

func GetSupportedEnginesForChip

func GetSupportedEnginesForChip(config RuntimeImagesConfig, chipModel string) ([]string, error)

GetSupportedEnginesForChip returns a list of configured engines for a chip model.

Parameters:

  • config: RuntimeImagesConfig to query
  • chipModel: Chip model identifier

Returns:

  • Slice of engine names
  • Error if chip model is not configured

func GetSystemArchitecture

func GetSystemArchitecture() (string, error)

GetSystemArchitecture returns the current system's CPU architecture in a format compatible with Docker image tags.

This function uses Go's runtime.GOARCH to determine the architecture and maps it to the standard Docker platform naming:

  • "arm64": ARM 64-bit (aarch64)
  • "amd64": x86 64-bit (x86_64)

Returns:

  • Architecture string ("arm64" or "amd64")
  • Error if the architecture is not supported

Example:

arch, err := GetSystemArchitecture()
// On ARM64 system: returns "arm64", nil
// On x86_64 system: returns "amd64", nil

func GetTemplateParams

func GetTemplateParams(cfg *RuntimeParamsConfig, chipConfigKey, modelID, backendName string) []string

GetTemplateParams retrieves parameters for a specific template.

Parameters:

  • cfg: Runtime parameters configuration
  • chipConfigKey: Chip config key (e.g., "ascend-910b")
  • modelID: Model identifier (e.g., "qwen2-72b")
  • backendName: Backend name without mode (e.g., "mlguider", not "mlguider:docker")

Returns:

  • List of parameters in key=value format

func LoadExtSandboxesFromDevices

func LoadExtSandboxesFromDevices(engineName string) map[string]*ExtSandboxConfig

LoadExtSandboxesFromDevices extracts extended sandbox configurations from device config.

This function scans all loaded device configurations and collects extended sandbox definitions for the specified engine. It returns a map where keys are device config_keys and values are the corresponding sandbox configurations.

The function expects device configuration to already be loaded via LoadDevicesConfig() or LoadDevicesConfigFrom().

Parameters:

  • engineName: Engine name to extract configs for ("vllm", "mindie", "mlguider")

Returns:

  • Map of device config_key -> ExtSandboxConfig for the specified engine
  • Empty map if device config is not loaded or no extended sandboxes are defined

Example usage:

// After loading device config
extConfigs := config.LoadExtSandboxesFromDevices("vllm")
for deviceType, cfg := range extConfigs {
    sandbox := runtime.NewExtSandbox(deviceType, cfg)
    // Register sandbox...
}

func SaveDevicesConfig

func SaveDevicesConfig(config *DevicesConfig, path string) error

SaveDevicesConfig writes a DevicesConfig to a YAML file.

This method is primarily used for:

  • Generating template configuration files
  • Exporting modified configurations
  • Testing and validation

Parameters:

  • config: Configuration to save
  • path: File path to write to

Returns:

  • Error if file cannot be written

func SaveModelsConfig

func SaveModelsConfig(config *ModelsConfig, path string) error

SaveModelsConfig writes a ModelsConfig to a YAML file.

This method is primarily used for:

  • Generating template configuration files
  • Exporting modified configurations
  • Testing and validation

Parameters:

  • config: Configuration to save
  • path: File path to write to

Returns:

  • Error if file cannot be written

func SetImageForChipAndEngine

func SetImageForChipAndEngine(config RuntimeImagesConfig, chipModel, engine, arch, image string)

SetImageForChipAndEngine sets the Docker image for a specific chip model, engine, and architecture.

This function allows runtime modification of the image configuration. Note: This only modifies the in-memory configuration; call writeRuntimeImagesConfig to persist changes to disk.

Parameters:

  • config: RuntimeImagesConfig to modify
  • chipModel: Chip model identifier
  • engine: Inference engine name
  • arch: CPU architecture
  • image: Docker image URL to set

Types

type ChipModelConfig

type ChipModelConfig struct {
	// ConfigKey is the unique identifier used in runtime configuration
	// This key maps to runtime images and deployment settings
	// Example: "ascend-910b", "ascend-310p"
	ConfigKey string `yaml:"config_key"`

	// ModelName is the human-readable chip model name
	// Example: "Ascend 910B", "Ascend 310P"
	ModelName string `yaml:"model_name"`

	// DeviceID is the PCIe device identifier (16-bit hex value)
	// Example: "0xd802" for Ascend 910B
	DeviceID string `yaml:"device_id"`

	// SubsystemDeviceID is the PCIe subsystem device identifier (16-bit hex value, optional)
	// DEPRECATED: Use Variants instead for chip sub-versions
	// When empty, matching is based on VendorID and DeviceID only
	SubsystemDeviceID string `yaml:"subsystem_device_id,omitempty"`

	// Variants defines a list of chip sub-versions with different subsystem_device_id
	// Used to distinguish between models like 910B1, 910B2, etc.
	// If hardware matches a variant, the variant's config_key is used
	// All variants share the same runtime_images from base model config
	// If no variant matches, the base model config is used as fallback
	Variants []ChipVariant `yaml:"variants,omitempty"`

	// Generation groups related chip models (optional)
	// Example: "Ascend 9xx", "Ascend 3xx"
	Generation string `yaml:"generation,omitempty"`

	// Capabilities lists supported features and precision modes
	// Example: ["int8", "fp16", "inference"]
	Capabilities []string `yaml:"capabilities,omitempty"`

	// ChipsPerDevice specifies how many AI chips are on each physical PCI device
	// Default: 1 (single-chip card)
	// Example: 2 for dual-chip cards like Ascend 910B Duo
	// This allows proper device enumeration where one PCI device contains multiple inference cores
	ChipsPerDevice int `yaml:"chips_per_device,omitempty"`

	// Topology defines the physical topology for this chip model
	// Used for topology-aware allocation specific to this chip type
	Topology *TopologyConfig `yaml:"topology,omitempty"`

	// RuntimeImages maps inference engines to their Docker images by architecture
	// Structure: engine_name -> architecture -> image_url
	// Example: {"vllm": {"arm64": "quay.io/...", "amd64": "..."}}
	// All variants of this chip model share the same runtime images
	RuntimeImages map[string]map[string]string `yaml:"runtime_images,omitempty"`

	// ExtSandboxes defines configuration-based sandboxes for this chip model (optional)
	// Contains both common configuration and engine-specific configs
	// Common fields (devices, volumes, runtime) are shared by all engines
	// Engine configs are merged with common settings
	ExtSandboxes *ExtSandboxesConfig `yaml:"ext_sandboxes,omitempty"`
}

ChipModelConfig defines configuration for a specific chip model.

Each chip model has unique characteristics that affect how xw manages it:

  • Hardware identification (for device detection)
  • Human-readable names (for display and logging)
  • Capabilities (for compatibility checks)
  • Generation info (for version-specific handling)
  • Runtime images (Docker images for inference engines)
  • Variants (optional sub-versions with different subsystem_device_id)

func FindChipModelByConfigKey

func FindChipModelByConfigKey(config *DevicesConfig, configKey string) *ChipModelConfig

FindChipModelByConfigKey searches for a chip model by its config key.

This is a convenience method for looking up chip configuration by the base model's config_key identifier. It does NOT search variant keys.

Parameters:

  • config: DevicesConfig to search
  • configKey: The base model config_key to find (e.g., "ascend-910b")

Returns:

  • Pointer to ChipModelConfig if found (base model)
  • nil if not found

type ChipModelInfo

type ChipModelInfo struct {
	VendorID     string
	DeviceID     string
	ModelName    string
	ConfigKey    string
	DeviceType   api.DeviceType
	Generation   string
	Capabilities []string
}

ChipModelInfo represents chip model information in a format usable by device package.

This structure avoids direct dependency on device package types, breaking the circular dependency between config and device packages.

func LoadChipModels

func LoadChipModels() ([]ChipModelInfo, error)

LoadChipModels loads chip models from the device configuration.

This function reads the device configuration and converts it into a simple structure that can be easily consumed by the device detection code.

Returns:

  • Slice of chip model information
  • Error if configuration cannot be loaded or is invalid

Example:

models, err := LoadChipModels()
if err != nil {
    log.Fatalf("Failed to load chip models: %v", err)
}
for _, model := range models {
    fmt.Printf("Loaded: %s (%s)\n", model.ModelName, model.ConfigKey)
}

func LookupChipModelByPCIID

func LookupChipModelByPCIID(vendorID, deviceID string) (*ChipModelInfo, error)

LookupChipModelByPCIID looks up a chip model by PCIe vendor and device IDs.

This function searches the device configuration for a chip model matching the specified PCIe identifiers. It's used during hardware detection to identify discovered devices.

Parameters:

  • vendorID: PCIe vendor ID (e.g., "0x19e5")
  • deviceID: PCIe device ID (e.g., "0xd802")

Returns:

  • Pointer to ChipModelInfo if found
  • nil if not found
  • Error if configuration cannot be loaded

Example:

model, err := LookupChipModelByPCIID("0x19e5", "0xd802")
if err != nil {
    log.Fatalf("Lookup failed: %v", err)
}
if model != nil {
    fmt.Printf("Found: %s\n", model.ModelName)
}

type ChipVariant

type ChipVariant struct {
	// SubsystemDeviceID is the PCIe subsystem device identifier (16-bit hex value)
	// Example: "0x0001" for 910B1, "0x0002" for 910B2
	SubsystemDeviceID string `yaml:"subsystem_device_id"`

	// VariantKey is the unique identifier for this variant
	// Example: "ascend-910b1", "ascend-910b2"
	VariantKey string `yaml:"variant_key"`

	// VariantName is the human-readable variant name
	// Example: "910B1", "910B2"
	VariantName string `yaml:"variant_name,omitempty"`
}

ChipVariant defines a specific variant of a chip model.

Variants are used to distinguish between different sub-versions of the same chip that share the same device_id but have different subsystem_device_id. Example: 910B1, 910B2, 910B3 are variants of 910B Note: All variants of the same chip model use the same runtime images from base config

type ChipVendorConfig

type ChipVendorConfig struct {
	// VendorName is the company/organization name
	// Example: "Huawei", "Baidu", "Cambricon"
	VendorName string `yaml:"vendor_name"`

	// VendorID is the PCIe vendor identifier (16-bit hex value)
	// Example: "0x19e5" for Huawei
	VendorID string `yaml:"vendor_id"`

	// ChipModels lists all chip models from this vendor
	ChipModels []ChipModelConfig `yaml:"chip_models"`
}

ChipVendorConfig defines configuration for a chip vendor.

Vendors are manufacturers of AI accelerator chips. Each vendor may produce multiple chip families and models.

type Config

type Config struct {
	// Server holds the HTTP server configuration including host and port.
	Server ServerConfig `json:"server"`

	// Storage holds the storage configuration including directories for
	// data and configuration files.
	Storage StorageConfig `json:"storage"`

	// RuntimeParams holds runtime parameter templates loaded at startup.
	RuntimeParams *RuntimeParamsConfig `json:"-"`

	// BinaryVersion is the version of the xw binary (e.g., "v0.0.1").
	// Set from main.Version during initialization.
	// Used as the default config_version if not specified in server.conf.
	BinaryVersion string `json:"-"`
}

Config represents the complete application configuration.

This is the root configuration struct that contains all settings required for running the xw application, including server and storage configurations. The struct can be serialized to/from JSON for persistence.

func NewConfigWithCustomDirs

func NewConfigWithCustomDirs(configDir, dataDir string) *Config

NewConfigWithCustomDirs creates a new configuration with custom directories.

This function allows specifying custom configuration and data directories instead of using the defaults. Useful for:

  • Testing with isolated environments
  • Running multiple instances
  • Custom deployment scenarios

Parameters:

  • configDir: Custom configuration files directory path (empty string uses default ~/.xw)
  • dataDir: Custom data directory path (empty string uses configDir/data)

Returns:

  • A pointer to a newly created Config with the specified directories

Example:

cfg := config.NewConfigWithCustomDirs("/opt/xw", "")
// Config files: /opt/xw/*.yaml
// Data directory: /opt/xw/data
// Models will be stored in /opt/xw/data/models/

func NewDefaultConfig

func NewDefaultConfig() *Config

NewDefaultConfig creates a new configuration instance with default values.

This function initializes a Config struct with sensible defaults suitable for user-level deployment. The configuration uses:

  • Server: localhost:11581 for local-only access
  • ConfigDir: ~/.xw for configuration files (devices.yaml, models.yaml)
  • DataDir: ~/.xw/data for runtime data and models

Returns:

  • A pointer to a newly created Config with default values.

Example:

cfg := config.NewDefaultConfig()
fmt.Printf("Server: %s\n", cfg.GetServerAddress())

func (*Config) EnsureDirectories

func (c *Config) EnsureDirectories() error

EnsureDirectories creates all required directories if they don't exist.

This method ensures that the directory structure needed by the application exists on the filesystem. It creates:

  • The main configuration directory (ConfigDir)
  • The models storage directory (ModelsDir)

Directories are created with 0755 permissions (rwxr-xr-x), allowing the owner full access and read/execute access for group and others.

Returns:

  • nil if all directories were created successfully or already exist
  • error if any directory creation fails

Example:

if err := cfg.EnsureDirectories(); err != nil {
    log.Fatalf("Failed to create directories: %v", err)
}

func (*Config) GetOrCreateServerIdentity

func (c *Config) GetOrCreateServerIdentity() (*ServerIdentity, error)

GetOrCreateServerIdentity retrieves the server identity from server.conf or creates a new one if it doesn't exist.

This method ensures that all required fields (name, registry, config_version) are present and populated with appropriate default values if missing. The config_version defaults to the binary version if not specified.

Returns the server identity or an error if reading/writing fails.

func (*Config) GetServerAddress

func (c *Config) GetServerAddress() string

GetServerAddress returns the complete HTTP server address.

This method constructs the full server URL from the host and port configuration. The returned address can be used by HTTP clients to connect to the server.

Returns:

Example:

addr := cfg.GetServerAddress()
// Returns: "http://localhost:11581"

func (*Config) LoadServerConfig

func (c *Config) LoadServerConfig() error

LoadServerConfig loads server configuration from server.conf

func (*Config) LoadVersionedConfigs

func (c *Config) LoadVersionedConfigs(configVersion string, loadModels func(string) error) error

LoadVersionedConfigs loads all configuration files at startup.

This method loads the three main configuration files from the versioned config directory at startup time:

  • runtime_params.yaml: Runtime parameter templates (stored in Config.RuntimeParams)
  • devices.yaml: Device and runtime images config (cached globally)
  • models.yaml: Model definitions (registered globally via loadModels callback)

After this call, all configurations are loaded and ready to use throughout the application lifecycle. No further path handling or file loading needed.

Parameters:

  • configVersion: Configuration version string (e.g., "v0.0.1")
  • loadModels: Callback function to load and register models from models.yaml

Returns:

  • error if any configuration file fails to load

Example:

err := cfg.LoadVersionedConfigs("v0.0.1", server.InitializeModels)

func (*Config) ReloadVersionedConfigs

func (c *Config) ReloadVersionedConfigs(configVersion string, loadModels func(string) error) error

ReloadVersionedConfigs reloads all configuration files without server restart.

This method implements transactional reload with validation phase: 1. All configurations are loaded and validated without modifying caches 2. Only if all validations succeed, caches are updated atomically 3. If any configuration fails, no changes are applied

Configuration files reloaded:

  • runtime_params.yaml: Runtime parameter templates (stored in Config.RuntimeParams)
  • devices.yaml: Device and runtime images config (cache cleared and reloaded)
  • models.yaml: Model definitions (re-registered, overwrites existing)

This method should be called when configuration needs to be updated without restarting the server (e.g., after running 'xw update').

Parameters:

  • configVersion: Configuration version string (e.g., "v0.0.1")
  • loadModels: Callback function to load and register models from models.yaml

Returns:

  • error if any configuration file fails to reload (no changes applied)

Example:

err := cfg.ReloadVersionedConfigs("v0.0.1", server.InitializeModels)

func (*Config) SaveServerConfig

func (c *Config) SaveServerConfig() error

SaveServerConfig saves current server configuration to server.conf

type DeviceConfigLoader

type DeviceConfigLoader struct {
	// contains filtered or unexported fields
}

DeviceConfigLoader handles loading and caching of device configurations.

The loader implements singleton pattern with lazy initialization. Configurations are loaded once and cached for the lifetime of the application.

Thread Safety: All methods are safe for concurrent use.

type DevicesConfig

type DevicesConfig struct {
	// Version specifies the configuration schema version
	// Used for compatibility checking and migration
	Version string `yaml:"version"`

	// Vendors contains all supported chip vendors and their models
	// Each vendor's chip models can define their own topology configuration
	Vendors []ChipVendorConfig `yaml:"vendors"`
}

DevicesConfig is the root configuration structure for device definitions.

This structure maps to the YAML configuration file and contains all vendor and chip model definitions.

func GetDevicesConfig

func GetDevicesConfig() (*DevicesConfig, error)

GetDevicesConfig returns the cached device configuration.

This method provides access to the previously loaded configuration without re-reading the file. Configuration must be loaded first via LoadDevicesConfig() or LoadDevicesConfigFrom() (typically done during server startup).

Returns:

  • Pointer to DevicesConfig
  • Error if configuration not loaded yet

func LoadDevicesConfig

func LoadDevicesConfig() (*DevicesConfig, error)

LoadDevicesConfig returns the cached device configuration.

This function is a convenience wrapper around GetDevicesConfig(). Configuration must be loaded first via LoadDevicesConfigFrom() during server initialization (typically called from LoadVersionedConfigs).

Returns:

  • Pointer to loaded DevicesConfig
  • Error if configuration not loaded yet

Example:

config, err := LoadDevicesConfig()
if err != nil {
    log.Fatalf("Device config not loaded: %v", err)
}
for _, vendor := range config.Vendors {
    fmt.Printf("Loaded vendor: %s\n", vendor.VendorName)
}

func LoadDevicesConfigFrom

func LoadDevicesConfigFrom(configPath string) (*DevicesConfig, error)

LoadDevicesConfigFrom loads device configuration from a specified path.

This function is used when you need to load configuration from a specific file path instead of the default location. It implements a singleton pattern with caching.

Parameters:

  • configPath: Path to configuration file (empty string for default)

Returns:

  • Pointer to loaded DevicesConfig
  • Error if file cannot be read, parsed, or validated

Example:

config, err := LoadDevicesConfigFrom("/custom/path/devices.yaml")
if err != nil {
    log.Fatalf("Failed to load device config: %v", err)
}

func ReloadDevicesConfig

func ReloadDevicesConfig(configPath string) (*DevicesConfig, error)

ReloadDevicesConfig forces a reload of the device configuration.

This method clears the cache and re-reads the configuration file. Useful for applying configuration changes without restarting the application.

Parameters:

  • configPath: Optional path to configuration file (empty for default)

Returns:

  • Pointer to reloaded DevicesConfig
  • Error if reload fails

type ExtSandboxConfig

type ExtSandboxConfig struct {
	// DeviceEnv specifies the environment variable name for device visibility.
	// This variable will be automatically set to a comma-separated list of
	// device indices (e.g., "0,1,2") when the container is created.
	//
	// Common examples:
	//   - Ascend NPU: ASCEND_RT_VISIBLE_DEVICES
	//   - NVIDIA GPU: CUDA_VISIBLE_DEVICES
	//   - Kunlun XPU: XPU_VISIBLE_DEVICES
	//   - Cambricon MLU: MLU_VISIBLE_DEVICES
	DeviceEnv string `yaml:"device_env"`

	// Environment contains additional static environment variables to set
	// in the container. These are set as-is without any template processing.
	//
	// Example:
	//   environment:
	//     LOG_LEVEL: "3"
	//     WORKER_METHOD: spawn
	Environment map[string]string `yaml:"environment"`

	// Devices lists all device node paths that may need to be mounted.
	//
	// Device mounting behavior:
	//   - Paths ending with a digit (e.g., /dev/xpu0, /dev/xpu5):
	//     Mounted only when the corresponding device index is allocated.
	//   - Paths not ending with a digit (e.g., /dev/xpu_ctl):
	//     Mounted as shared devices for all containers.
	//
	// Example:
	//   devices:
	//     - /dev/xpu0          # Mounted only when device 0 is allocated
	//     - /dev/xpu1          # Mounted only when device 1 is allocated
	//     - /dev/xpu_ctl       # Always mounted (shared control device)
	Devices []string `yaml:"devices"`

	// Volumes defines host-to-container volume mounts in "host:container" format.
	// If no container path is specified (no colon), the same path is used for both.
	//
	// Example:
	//   volumes:
	//     - /usr/local/xpu:/usr/local/xpu
	//     - /root/.cache:/root/.cache
	Volumes []string `yaml:"volumes"`

	// Privileged indicates whether the container requires privileged mode.
	//
	// Privileged mode grants the container extended permissions, including
	// access to all devices and the ability to modify system settings.
	// While less secure, some accelerators require it for proper operation.
	Privileged bool `yaml:"privileged"`

	// Runtime specifies the Docker runtime to use (e.g., "runc", "nvidia").
	// Defaults to "runc" if not specified.
	Runtime string `yaml:"runtime"`

	// ShmSizeGB specifies the shared memory size in gigabytes.
	// If not specified or zero, defaults to 16GB.
	//
	// Shared memory is required for:
	//   - Inter-process communication in distributed inference
	//   - PyTorch DataLoader workers
	//   - Model tensor sharing
	ShmSizeGB int `yaml:"shm_size_gb,omitempty"`

	// Capabilities lists Linux capabilities required by the container.
	//
	// Common capabilities for AI accelerators:
	//   - SYS_ADMIN: System administration operations
	//   - SYS_RAWIO: Direct device I/O access
	//   - IPC_LOCK: Memory locking for device buffers
	//   - SYS_RESOURCE: Resource limit adjustments
	Capabilities []string `yaml:"capabilities"`
}

ExtSandboxConfig defines the configuration for an extended device sandbox.

This configuration maps directly to the DeviceSandbox interface methods, providing a declarative way to define device-specific container requirements for AI accelerators.

The configuration is designed to support common device patterns while maintaining simplicity. Complex scenarios (e.g., custom device initialization, dynamic configuration) should be implemented in code-based sandboxes.

Configuration Structure:

Extended sandboxes support chip-level common configuration to reduce duplication. Common settings (devices, volumes, runtime) are defined at chip model level and automatically merged with engine-specific configurations.

Example configuration:

chip_models:
  - config_key: kunlun-r200
    ext_sandboxes:
      # Common configuration (shared by all engines)
      devices:
        - /dev/xpu0
        - /dev/xpu1
        - /dev/xpu_ctl
      volumes:
        - /usr/local/xpu:/usr/local/xpu
      runtime: runc
      # Engine-specific configurations (device_env, privileged, etc.)
      vllm:
        device_env: XPU_VISIBLE_DEVICES
        environment:
          XPU_WORKER_MULTIPROC_METHOD: spawn
        privileged: true
        shm_size_gb: 50
        capabilities:
          - SYS_ADMIN
        # Optional: engines can add extra volumes
        volumes:
          - /root/.cache:/root/.cache
      mindie:
        device_env: XPU_DEVICE_IDS
        privileged: true
        shm_size_gb: 100

Merge Strategy:

  • Devices: common.devices + engine.devices (appended)
  • Volumes: common.volumes + engine.volumes (appended)
  • Runtime: engine.runtime overrides common.runtime if specified
  • Other fields: managed by engine config only

type ExtSandboxesConfig

type ExtSandboxesConfig struct {
	// Common configuration shared by all engines
	Devices []string `yaml:"devices,omitempty"`
	Volumes []string `yaml:"volumes,omitempty"`
	Runtime string   `yaml:"runtime,omitempty"`

	// Engine-specific configurations
	// Populated during YAML unmarshaling with all non-common keys
	Engines map[string]*ExtSandboxConfig `yaml:"-"`
}

ExtSandboxesConfig contains both common configuration and engine-specific configs.

This structure allows defining shared settings (devices, volumes, runtime) that apply to all engines, along with engine-specific configurations that define unique behavior.

Example YAML:

ext_sandboxes:
  # Common configuration (shared by all engines)
  devices:
    - /dev/xpu0
    - /dev/xpu1
  volumes:
    - /usr/local/xpu:/usr/local/xpu
  runtime: runc
  # Engine-specific configurations
  vllm:
    device_env: XPU_VISIBLE_DEVICES
    privileged: true
  mindie:
    device_env: XPU_DEVICE_IDS
    privileged: true

func (*ExtSandboxesConfig) UnmarshalYAML

func (c *ExtSandboxesConfig) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML implements custom YAML unmarshaling for ExtSandboxesConfig. It separates common fields (devices, volumes, runtime) from engine-specific configs.

type ModelConfig

type ModelConfig struct {

	// ModelID is the unique identifier for this model
	// Convention: lowercase, hyphen-separated (e.g., "qwen2-7b")
	ModelID string `yaml:"model_id"`

	// SourceID is the model ID on the source platform
	// Examples: "qwen/Qwen2-7B" (ModelScope), "Qwen/Qwen2-7B" (HuggingFace)
	SourceID string `yaml:"source_id"`

	// Parameters is the model size in billions of parameters
	// Example: 7.0 for 7B model
	Parameters float64 `yaml:"parameters,omitempty"`

	// ContextLength is the maximum context window size in tokens
	ContextLength int `yaml:"context_length,omitempty"`

	// SupportedDevices maps device types to their supported engines
	// Key: device config_key (e.g., "ascend-910b")
	// Value: list of engines in priority order (e.g., ["vllm:docker", "mindie:docker"])
	// Example:
	//   ascend-910b:
	//     - vllm:docker
	//     - mindie:docker
	SupportedDevices map[string][]string `yaml:"supported_devices"`

	// Tag specifies the model variant (e.g., "main", "int8", "fp16")
	Tag string `yaml:"tag,omitempty"`

	// Capabilities lists the model's supported features
	// Common values: "completion", "vision", "tool_use", "function_calling"
	Capabilities []string `yaml:"capabilities,omitempty"`
}

ModelConfig defines configuration for an AI model.

This structure represents a model that can be deployed and served by xw.

func FindModelByID

func FindModelByID(config *ModelsConfig, modelID string) *ModelConfig

FindModelByID searches for a model by its ID.

This is the primary method for looking up model configuration.

Parameters:

  • config: ModelsConfig to search
  • modelID: The model_id to find

Returns:

  • Pointer to ModelConfig if found
  • nil if not found

func FindModelsByDeviceType

func FindModelsByDeviceType(config *ModelsConfig, deviceConfigKey string) []*ModelConfig

FindModelsByDeviceType returns all models compatible with a device type.

This method is used to filter models based on hardware availability.

Parameters:

  • config: ModelsConfig to search
  • deviceConfigKey: Device config key to filter by

Returns:

  • Slice of pointers to compatible ModelConfig objects

type ModelConfigLoader

type ModelConfigLoader struct {
	// contains filtered or unexported fields
}

ModelConfigLoader handles loading and caching of model configurations.

The loader implements singleton pattern with lazy initialization. Configurations are loaded once and cached for the lifetime of the application.

Thread Safety: All methods are safe for concurrent use.

type ModelsConfig

type ModelsConfig struct {
	// Version specifies the configuration schema version
	// Used for compatibility checking and migration
	Version string `yaml:"version"`

	// Models contains all available model configurations
	Models []ModelConfig `yaml:"models"`

	// ModelGroups provides logical grouping of related models (optional)
	// Example: {"qwen2": ["qwen2-0.5b", "qwen2-7b", "qwen2-72b"]}
	ModelGroups map[string][]string `yaml:"model_groups,omitempty"`
}

ModelsConfig is the root configuration structure for model definitions.

This structure maps to the YAML configuration file and contains all model definitions available in the system.

func GetModelsConfig

func GetModelsConfig() (*ModelsConfig, error)

GetModelsConfig returns the cached model configuration.

This method provides access to the previously loaded configuration without re-reading the file. Configuration must be loaded first via LoadModelsConfig() during server initialization.

Returns:

  • Pointer to ModelsConfig
  • Error if configuration not loaded yet

func LoadModelsConfig

func LoadModelsConfig(configPath string) (*ModelsConfig, error)

LoadModelsConfig loads model configuration from the specified file path.

This method reads and parses the YAML configuration file, validating the structure and content. Configuration path must be explicitly provided (typically during server initialization via LoadVersionedConfigs).

The configuration is cached after first load. Subsequent calls return the cached configuration without re-reading the file.

Parameters:

  • configPath: Path to configuration file (must not be empty)

Returns:

  • Pointer to loaded ModelsConfig
  • Error if file cannot be read, parsed, or validated

Example:

config, err := LoadModelsConfig("/root/.xw/0.0.1/models.yaml")
if err != nil {
    log.Fatalf("Failed to load model config: %v", err)
}
for _, model := range config.Models {
    fmt.Printf("Loaded model: %s (%s)\n", model.ModelName, model.ModelID)
}

func ReloadModelsConfig

func ReloadModelsConfig(configPath string) (*ModelsConfig, error)

ReloadModelsConfig forces a reload of the model configuration.

This method clears the cache and re-reads the configuration file. Useful for applying configuration changes without restarting the application.

Parameters:

  • configPath: Optional path to configuration file (empty for default)

Returns:

  • Pointer to reloaded ModelsConfig
  • Error if reload fails

type Package

type Package struct {
	// Version is the package version (e.g., "v0.0.1")
	Version string `json:"version"`

	// Name is the human-readable package name
	Name string `json:"name"`

	// Description provides a brief description of this version
	Description string `json:"description"`

	// ReleaseDate is when this version was released (YYYY-MM-DD)
	ReleaseDate string `json:"release_date"`

	// DownloadURL is the URL to download the package tarball
	DownloadURL string `json:"download_url"`

	// SHA256 is the expected SHA256 checksum of the package
	SHA256 string `json:"sha256"`

	// MinXwVersion is the minimum xw binary version required
	MinXwVersion string `json:"min_xw_version"`
}

Package represents a configuration package version from the registry.

type PackageRegistry

type PackageRegistry struct {
	// Schema is the JSON schema URL
	Schema string `json:"$schema,omitempty"`

	// Name is the registry name
	Name string `json:"name"`

	// Description describes the registry
	Description string `json:"description"`

	// UpdatedAt is when the registry was last updated
	UpdatedAt string `json:"updated_at"`

	// Packages is the list of available configuration packages
	Packages []Package `json:"packages"`
}

PackageRegistry represents the structure of packages.json

type RuntimeImagesConfig

type RuntimeImagesConfig map[string]map[string]map[string]string

RuntimeImagesConfig represents the configuration for Docker images used by different chip types, inference engines, and CPU architectures.

Structure:

  • Map of chip model to its engine configurations Key: Chip model name (e.g., "ascend-910b", "ascend-310p") Value: Map of engine name to architecture-specific images

Example YAML:

ascend-910b:
  vllm:
    arm64: quay.io/ascend/vllm-ascend:v0.11.0rc0-arm64
    amd64: quay.io/ascend/vllm-ascend:v0.11.0rc0-amd64
  mindie:
    arm64: harbor.tsingmao.com/xuanwu/mindie:2.2.RC1-800I-A2-arm64
    amd64: harbor.tsingmao.com/xuanwu/mindie:2.2.RC1-800I-A2-amd64

func LoadRuntimeImagesConfig

func LoadRuntimeImagesConfig() (RuntimeImagesConfig, error)

LoadRuntimeImagesConfig loads runtime images configuration from devices configuration.

This method extracts runtime images from the devices.yaml configuration file using the default configuration path.

Returns:

  • RuntimeImagesConfig instance extracted from devices config
  • Error if devices config cannot be loaded

Example:

config, err := LoadRuntimeImagesConfig()
if err != nil {
    log.Fatalf("Failed to load runtime images config: %v", err)
}

func LoadRuntimeImagesConfigFrom

func LoadRuntimeImagesConfigFrom(devicesConfigPath string) (RuntimeImagesConfig, error)

LoadRuntimeImagesConfigFrom loads runtime images configuration from a specific devices config file.

This method is used when you need to load from a non-default configuration path.

Parameters:

  • devicesConfigPath: Path to the devices configuration file

Returns:

  • RuntimeImagesConfig instance extracted from devices config
  • Error if devices config cannot be loaded

Example:

config, err := LoadRuntimeImagesConfigFrom("/custom/path/devices.yaml")
if err != nil {
    log.Fatalf("Failed to load runtime images config: %v", err)
}

type RuntimeParamsConfig

type RuntimeParamsConfig struct {
	// Version specifies the configuration schema version
	Version string `yaml:"version"`

	// Templates contains all parameter templates
	Templates []RuntimeParamsTemplate `yaml:"templates"`
}

RuntimeParamsConfig is the root configuration for runtime parameter templates.

func LoadRuntimeParamsConfig

func LoadRuntimeParamsConfig() (*RuntimeParamsConfig, error)

LoadRuntimeParamsConfig loads runtime parameter templates from the default location.

Returns:

  • Runtime parameters configuration
  • Error if file exists but cannot be parsed (returns empty config if file doesn't exist)

func LoadRuntimeParamsConfigFrom

func LoadRuntimeParamsConfigFrom(configPath string) (*RuntimeParamsConfig, error)

LoadRuntimeParamsConfigFrom loads runtime parameter templates from a specific file.

Parameters:

  • configPath: Path to the runtime_params.yaml file

Returns:

  • Runtime parameters configuration
  • Error if file exists but cannot be parsed (returns empty config if file doesn't exist)

type RuntimeParamsTemplate

type RuntimeParamsTemplate struct {
	// Name is the template identifier (chip_model_backend)
	Name string `yaml:"name"`

	// Params is a list of key=value parameters
	// Example: ["max_batch_size=128", "tensor_parallel=4"]
	Params []string `yaml:"params"`
}

RuntimeParamsTemplate represents a parameter template for a specific runtime configuration.

Template name format: {chip_config_key}_{model_id}_{backend_name} Example: ascend-910b_qwen2-72b_mlguider

type ServerConfig

type ServerConfig struct {
	// Name is the unique identifier for this server instance.
	Name string `json:"name"`

	// Registry is the configuration package registry URL.
	Registry string `json:"registry"`

	// Host is the server host address (e.g., "localhost", "0.0.0.0").
	// Using "localhost" restricts access to local clients only.
	// Using "0.0.0.0" allows access from any network interface.
	Host string `json:"host"`

	// Port is the TCP port number the server listens on.
	// Common values are 11581 (default) or other non-privileged ports.
	Port int `json:"port"`

	// Address is the computed full server address.
	// This field is not serialized and is computed from Host and Port.
	// Format: "http://host:port"
	Address string `json:"-"`
}

ServerConfig represents the HTTP server configuration.

This configuration controls how the xw server listens for incoming HTTP connections from CLI clients or other API consumers.

type ServerIdentity

type ServerIdentity struct {
	// Name is the unique identifier for this server instance.
	// Generated randomly on first start if not present.
	Name string `json:"name"`

	// Registry is the URL to the configuration package registry.
	// This registry maintains available configuration versions.
	Registry string `json:"registry"`

	// ConfigVersion is the currently active configuration version.
	// Defaults to the binary version (main.Version) if not specified.
	// Format: vX.Y.Z (e.g., "v0.0.1")
	ConfigVersion string `json:"config_version"`
}

ServerIdentity represents the server's unique identity and configuration state. This struct holds persistent server-specific settings that are stored in server.conf.

type StorageConfig

type StorageConfig struct {
	// ConfigDir is the absolute path to the configuration files directory.
	// Contains YAML configuration files like devices.yaml, models.yaml.
	// Example: "/home/user/.xw"
	ConfigDir string `json:"config_dir"`

	// DataDir is the absolute path to the main data directory.
	// Contains all runtime data including models and server state.
	// Example: "/home/user/.xw/data"
	DataDir string `json:"data_dir"`
}

StorageConfig represents the storage and persistence configuration.

This configuration defines where the application stores its data and configuration files.

func (*StorageConfig) GetModelsDir

func (s *StorageConfig) GetModelsDir() string

GetModelsDir returns the models storage directory path. Models are stored in a "models" subdirectory within the data directory. Example: ~/.xw/data/models

type TopologyBox

type TopologyBox struct {
	// Devices is a list of logical chip indices in this box
	// Example: [0, 1, 2, 3] means logical chips 0-3 have high-speed interconnect
	// Note: Use logical chip indices, NOT physical device indices
	Devices []int `yaml:"devices"`
}

TopologyBox represents a group of devices with high-speed interconnection.

Devices within the same box are considered to have zero distance for allocation optimization. Devices in different boxes have a distance equal to the absolute difference of their box indices.

type TopologyConfig

type TopologyConfig struct {
	// Boxes is a list of device groups with high-speed interconnection
	// Each box contains devices with zero intra-box distance
	// Inter-box distance equals |box_index_a - box_index_b|
	Boxes []TopologyBox `yaml:"boxes,omitempty"`
}

TopologyConfig defines the physical topology of devices.

This configuration enables topology-aware device allocation, ensuring that allocated devices are as close as possible in the physical topology.

type VersionManager

type VersionManager struct {
	// contains filtered or unexported fields
}

VersionManager handles configuration version operations.

func NewVersionManager

func NewVersionManager(cfg *Config) *VersionManager

NewVersionManager creates a new version manager.

func (*VersionManager) DownloadPackage

func (vm *VersionManager) DownloadPackage(pkg *Package) error

DownloadPackage downloads and extracts a configuration package.

Parameters:

  • pkg: The package to download

Returns:

  • nil on success, error on failure

func (*VersionManager) FetchRegistry

func (vm *VersionManager) FetchRegistry() (*PackageRegistry, error)

FetchRegistry fetches and parses the package registry from the configured URL.

Returns:

  • The parsed registry or an error if fetching/parsing fails

func (*VersionManager) FindPackage

func (vm *VersionManager) FindPackage(version string) (*Package, error)

FindPackage finds a specific package version in the registry.

Parameters:

  • version: The version to find (e.g., "v0.0.1")

Returns:

  • The package or an error if not found

func (*VersionManager) GetCompatibleVersions

func (vm *VersionManager) GetCompatibleVersions(binaryVersion string) ([]Package, error)

GetCompatibleVersions returns all versions compatible with the given binary version, sorted from newest to oldest.

Parameters:

  • binaryVersion: Current xw binary version

Returns:

  • Slice of compatible packages sorted by version (newest first)

func (*VersionManager) GetCurrentVersion

func (vm *VersionManager) GetCurrentVersion() (string, error)

GetCurrentVersion returns the currently active configuration version.

func (*VersionManager) GetLatestCompatibleVersion

func (vm *VersionManager) GetLatestCompatibleVersion(binaryVersion string) (*Package, error)

GetLatestCompatibleVersion returns the latest version compatible with the current binary.

Parameters:

  • binaryVersion: Current xw binary version (e.g., "v0.0.1")

Returns:

  • The latest compatible package or an error if none found

func (*VersionManager) IsVersionCompatible

func (vm *VersionManager) IsVersionCompatible(pkg *Package, binaryVersion string) (bool, error)

IsVersionCompatible checks if a package version is compatible with the binary version.

Parameters:

  • pkg: The package to check
  • binaryVersion: Current xw binary version

Returns:

  • true if compatible, false otherwise

func (*VersionManager) IsVersionInstalled

func (vm *VersionManager) IsVersionInstalled(version string) bool

IsVersionInstalled checks if a version is already installed locally.

func (*VersionManager) ListInstalledVersions

func (vm *VersionManager) ListInstalledVersions() ([]string, error)

ListInstalledVersions scans the local config directory and returns all installed versions.

This method discovers configuration versions by examining the config directory (typically ~/.xw/) for subdirectories that look like version numbers.

Returns:

  • Slice of installed version strings (without "v" prefix)
  • Error if the config directory cannot be read

func (*VersionManager) SwitchVersion

func (vm *VersionManager) SwitchVersion(version string) error

SwitchVersion switches to a different configuration version.

This updates the config_version in server.conf but does not restart the server.

Parameters:

  • version: The version to switch to

Returns:

  • nil on success, error if the version doesn't exist locally

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL