client

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 12, 2026 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package client provides an HTTP client for communicating with the xw server.

This package implements the client-side of the xw API, enabling CLI tools and other applications to interact with the xw server over HTTP. It provides:

  • High-level methods for all API operations
  • Automatic request/response serialization
  • Error handling and status code processing
  • Configurable timeouts and HTTP client settings

The client handles all HTTP communication details, allowing callers to work with native Go types rather than raw HTTP requests and responses.

Example usage: 

client := client.NewClient("http://localhost:11581")
models, err := client.ListModels(device.ConfigKeyAscend910B, false)
if err != nil {
    log.Fatalf("Failed to list models: %v", err)
}

Package client - config.go implements configuration management operations.

This file provides methods for querying and modifying server configuration settings such as server name and registry URL.

Package client - devices.go implements device management operations.

This file provides methods for querying and managing AI accelerator devices available on the server. These operations help discover device capabilities and validate hardware compatibility.

Package client - execution.go implements model execution operations.

This file provides methods for executing AI models with input data and retrieving inference results. These operations handle model invocation and result processing.

Package client - instances.go implements model instance management operations.

This file provides methods for managing runtime model instances including starting, stopping, listing, and monitoring model containers. These operations handle the lifecycle of running model services.

Package client - models.go implements model management operations.

This file provides methods for querying, listing, downloading, and managing AI models in the xw system. These operations interact with the server's model registry and local model storage.

Package client - request.go implements low-level HTTP request handling.

This file provides the core HTTP request/response logic used by all API methods in the client. It handles request serialization, response parsing, error handling, and status code validation.

Package client - sse.go implements SSE (Server-Sent Events) support.

This file provides SSE-based communication for streaming operations like model downloads, where the server needs to send real-time progress updates to the client.

Package client - system.go implements system information operations.

This file provides methods for querying server status, version information, and health checks. These operations help monitor and validate the server state.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is the HTTP client for communicating with the xw server.

The Client provides a high-level interface for all server API operations. It manages HTTP connections, request serialization, response parsing, and error handling. All methods are thread-safe and can be called concurrently.

The client uses a configurable HTTP client with sensible defaults for timeouts and connection pooling.

func NewClient 

func NewClient(baseURL string) *Client

NewClient creates a new client instance configured to communicate with a specific xw server.

The client is created with default HTTP settings including:

  • 30-second timeout for all requests
  • Automatic connection pooling and keep-alive
  • Automatic retry of idempotent requests

Parameters:

Returns:

  • A pointer to a configured Client ready for use.

Example: 

client := client.NewClient("http://localhost:11581")
health, err := client.Health()

func (*Client) CheckInstanceReady 

func (c *Client) CheckInstanceReady(alias string) (bool, error)

CheckInstanceReady checks if a model instance is ready to serve requests.

This method verifies that the instance's endpoint is accessible and responding.

Parameters:

  • alias: The instance alias to check

Returns:

  • true if the instance is ready, false otherwise
  • error if the request fails

func (*Client) GetBaseURL 

func (c *Client) GetBaseURL() string

GetBaseURL returns the server's base URL.

This returns the complete base URL of the xw server that the client is configured to communicate with. The URL can be used directly for API requests through the server's proxy.

Returns:

func (*Client) GetConfigInfo 

func (c *Client) GetConfigInfo() (*ConfigInfo, error)

GetConfigInfo retrieves all server configuration settings.

This method queries the server for comprehensive configuration information including server identity, network settings, and storage paths.

Returns:

  • A pointer to ConfigInfo with server configuration details
  • An error if the request fails

Example: 

info, err := client.GetConfigInfo()
if err != nil {
    log.Fatalf("Failed to get config: %v", err)
}
fmt.Printf("Server: %s\n", info.Name)

func (*Client) GetConfigValue 

func (c *Client) GetConfigValue(key string) (string, error)

GetConfigValue retrieves a specific configuration value.

This method queries the server for the value of a specific configuration key.

Parameters:

  • key: The configuration key to retrieve (e.g., "name", "registry")

Returns:

  • The configuration value as a string
  • An error if the request fails or the key is not found

Example: 

value, err := client.GetConfigValue("name")
if err != nil {
    log.Fatalf("Failed to get config value: %v", err)
}
fmt.Println(value)

func (*Client) GetCurrentVersion 

func (c *Client) GetCurrentVersion() (*CurrentVersionResponse, error)

GetCurrentVersion retrieves the currently active configuration version.

Returns:

  • CurrentVersionResponse with current version info
  • error if the request fails

func (*Client) GetModel 

func (c *Client) GetModel(modelID string) (map[string]interface{}, error)

GetModel retrieves detailed information about a specific model.

Parameters:

  • modelID: The unique model identifier

Returns:

  • Map containing model details
  • Error if the model doesn't exist or the request fails

func (*Client) GetSupportedDevices 

func (c *Client) GetSupportedDevices() ([]string, error)

GetSupportedDevices retrieves the list of device types supported by the server.

This method queries the server for device types that are configured and supported based on the server's configuration files. It's used to filter models and validate device compatibility.

Returns:

  • A slice of device type strings (e.g., ["ascend-910b", "ascend-310p"])
  • An error if the request fails or the server returns an error

Example: 

deviceTypes, err := client.GetSupportedDevices()
if err != nil {
    log.Fatalf("Failed to get supported devices: %v", err)
}
for _, dt := range deviceTypes {
    fmt.Printf("Supported: %s\n", dt)
}

func (*Client) Health 

func (c *Client) Health() (*api.HealthResponse, error)

Health checks the server's health and readiness status.

This method performs a health check to verify that the server is running and ready to handle requests. It's commonly used by monitoring systems, load balancers, and before attempting other operations.

Returns:

  • A pointer to HealthResponse with status information
  • An error if the request fails or the server is unhealthy

Example: 

health, err := client.Health()
if err != nil {
    log.Fatalf("Server is unhealthy: %v", err)
}
fmt.Printf("Server status: %s\n", health.Status)

func (*Client) ListDevices 

func (c *Client) ListDevices() ([]DeviceInfo, error)

ListDevices retrieves a list of devices detected on the server machine.

This method queries the server for all AI accelerator devices available on the server hardware. It's used by the CLI to display device information.

Returns:

  • A slice of DeviceInfo structs representing detected hardware
  • An error if the request fails or the server returns an error

Example: 

devices, err := client.ListDevices()
if err != nil {
    log.Fatalf("Failed to list devices: %v", err)
}
for _, device := range devices {
    fmt.Printf("%s: %s (available=%v)\n", device.Type, device.Name, device.Available)
}

func (*Client) ListDownloadedModels 

func (c *Client) ListDownloadedModels() ([]api.DownloadedModel, error)

ListDownloadedModels queries models that have been downloaded.

This method returns only models that are currently downloaded and available locally.

Returns:

  • Slice of downloaded model information
  • An error if the request fails

func (*Client) ListInstances 

func (c *Client) ListInstances(all bool) ([]interface{}, error)

ListInstances lists running model instances.

Parameters:

  • all: If true, includes stopped instances

Returns:

  • Slice of instance information maps
  • error if the request fails

func (*Client) ListModels 

func (c *Client) ListModels(deviceType api.DeviceType, showAll bool) ([]api.Model, error)

ListModels retrieves a list of available models from the server.

This method queries the server for models, optionally filtering by device compatibility. It's commonly used to display available models to users or validate model availability before operations.

Parameters:

  • deviceType: Filter models by device type (use DeviceTypeAll for no filter)
  • showAll: If true, returns all models regardless of device type

Returns:

  • A slice of Model structs matching the filter criteria
  • An error if the request fails or the server returns an error

Example: 

// List all models compatible with Ascend 910B devices
models, err := client.ListModels(device.ConfigKeyAscend910B, false)
if err != nil {
    log.Fatalf("Failed to list models: %v", err)
}
for _, model := range models {
    fmt.Printf("%s (v%s): %s\n", model.Name, model.Version, model.Description)
}

func (*Client) ListModelsWithStats 

func (c *Client) ListModelsWithStats(deviceType api.DeviceType, showAll bool) (*api.ListModelsResponse, error)

ListModelsWithStats queries available models with statistics.

This method is similar to ListModels but returns the full response including statistics about total models, available models, and detected devices.

Parameters:

  • deviceType: Filter models by device type (DeviceTypeAll for no filter)
  • showAll: If true, show all models regardless of device availability

Returns:

  • A pointer to ListModelsResponse containing models and statistics
  • An error if the request fails

func (*Client) ListVersions 

func (c *Client) ListVersions() (*ListVersionsResponse, error)

ListVersions retrieves all available configuration versions from the server.

Returns:

  • ListVersionsResponse with version information
  • error if the request fails

func (*Client) Pull 

func (c *Client) Pull(model, version string, progressCallback func(string)) (*api.PullResponse, error)

Pull downloads and installs a model with streaming progress updates.

This method downloads a model from ModelScope with real-time progress updates via Server-Sent Events (SSE). Progress messages are sent to the provided callback function as they arrive.

Parameters:

  • model: The ModelScope model ID (e.g., "Qwen/Qwen2-7B")
  • version: The specific version (empty string for latest)
  • progressCallback: Function called for each progress message

Returns:

  • A pointer to PullResponse with final status
  • An error if the request fails

Example: 

resp, err := client.Pull("Qwen/Qwen2-7B", "", func(msg string) {
    fmt.Println(msg)
})

func (*Client) ReloadConfig 

func (c *Client) ReloadConfig() error

ReloadConfig reloads all configuration files on the server.

This method triggers the server to reload devices.yaml, models.yaml, and runtime_params.yaml from the current configuration version without requiring a server restart.

Returns:

  • An error if the request fails

Example: 

err := client.ReloadConfig()
if err != nil {
    log.Fatalf("Failed to reload config: %v", err)
}

func (*Client) RemoveInstance 

func (c *Client) RemoveInstance(instanceID string, force bool) error

RemoveInstance removes a model instance.

This method sends a request to the server to remove the specified instance. The instance must be stopped first unless force is true.

Parameters:

  • instanceID: ID of the instance to remove
  • force: If true, removes the instance even if it's running

Returns:

  • Error if the request fails or the server returns an error

func (*Client) RemoveInstanceByAlias 

func (c *Client) RemoveInstanceByAlias(alias string, force bool) error

RemoveInstanceByAlias removes a model instance by its alias.

This method sends a request to the server to remove the specified instance using alias. The instance must be stopped first unless force is true.

Parameters:

  • alias: Alias of the instance to remove
  • force: If true, removes the instance even if it's running

Returns:

  • Error if the request fails or the server returns an error

func (*Client) Run 

func (c *Client) Run(model, input string, options map[string]interface{}) (*api.RunResponse, error)

Run executes a model on the server with the given input.

This method sends an inference request to the server, which loads the specified model (if not already loaded) and processes the input. The model must have been pulled before it can be run.

Parameters:

  • model: The name of the model to execute
  • input: The input data to process
  • options: Optional model-specific parameters (can be nil)

Returns:

  • A pointer to RunResponse containing output and metrics
  • An error if the request fails or the model is not found

Example: 

resp, err := client.Run("llama3-8b", "Hello, world!", nil)
if err != nil {
    log.Fatalf("Model execution failed: %v", err)
}
fmt.Printf("Output: %s\n", resp.Output)
fmt.Printf("Latency: %vms\n", resp.Metrics["latency_ms"])

func (*Client) RunModel 

func (c *Client) RunModel(opts interface{}) (map[string]interface{}, error)

RunModel starts a model instance.

This method sends a request to start a model instance with the specified options.

Parameters:

  • opts: Runtime options for the model

Returns:

  • Response map with instance information
  • error if the request fails

func (*Client) RunModelWithSSE 

func (c *Client) RunModelWithSSE(opts interface{}, progressCallback func(string)) (map[string]interface{}, error)

RunModelWithSSE starts a model instance with SSE streaming for progress updates.

This method sends a request to start a model with real-time progress via SSE.

Parameters:

  • opts: Runtime options for the model
  • progressCallback: Function called for each progress event

Returns:

  • error if the request fails

func (*Client) RunModelWithSSEContext 

func (c *Client) RunModelWithSSEContext(ctx context.Context, opts interface{}, progressCallback func(string)) (map[string]interface{}, error)

RunModelWithSSEContext starts a model instance with SSE streaming and context support.

This method sends a request to start a model with real-time progress via SSE, and supports cancellation via context.

Parameters:

  • ctx: Context for cancellation
  • opts: Runtime options for the model
  • progressCallback: Function called for each progress event

Returns:

  • error if the request fails

func (*Client) SetConfigValue 

func (c *Client) SetConfigValue(key, value string) error

SetConfigValue sets a specific configuration value.

This method updates a configuration value on the server. The change is immediately persisted to disk.

Parameters:

  • key: The configuration key to set (e.g., "name", "registry")
  • value: The new value for the configuration key

Returns:

  • An error if the request fails or validation fails

Example: 

err := client.SetConfigValue("name", "xw-prod-01")
if err != nil {
    log.Fatalf("Failed to set config: %v", err)
}

func (*Client) StopInstance 

func (c *Client) StopInstance(instanceID string, force bool) error

StopInstance stops a running model instance.

Parameters:

  • instanceID: The instance to stop
  • force: If true, force stop even if in use

Returns:

  • error if the request fails

func (*Client) StopInstanceByAlias 

func (c *Client) StopInstanceByAlias(alias string, force bool) error

StopInstanceByAlias stops a model instance by its alias.

This method sends a request to the server to stop the specified instance using alias.

Parameters:

  • alias: Alias of the instance to stop
  • force: If true, forces stop even if instance is processing requests

Returns:

  • Error if the request fails or the server returns an error

func (*Client) StreamInstanceLogs 

func (c *Client) StreamInstanceLogs(alias string, follow bool, logCallback func(string)) error

StreamInstanceLogs streams logs from a running instance.

This method connects to the server to stream container logs. The logCallback function is called for each log line received.

Parameters:

  • alias: Alias of the instance to stream logs from
  • follow: If true, stream logs in real-time; if false, return existing logs and exit
  • logCallback: Function called for each log line

Returns:

  • Error if the request fails or the stream is interrupted

func (*Client) Update 

func (c *Client) Update(version string) (*UpdateResponse, error)

Update updates the configuration to the specified version. If version is empty, updates to the latest compatible version.

Parameters:

  • version: Target version string (e.g., "v0.0.2"), or empty for latest

Returns:

  • UpdateResponse with update results
  • error if the update fails

func (*Client) Version 

func (c *Client) Version() (*api.VersionResponse, error)

Version retrieves version and build information from the server.

This method queries the server for its version, build time, and git commit. Useful for debugging, compatibility checking, and displaying in help text.

Returns:

  • A pointer to VersionResponse with server version details
  • An error if the request fails

Example: 

ver, err := client.Version()
if err != nil {
    log.Fatalf("Failed to get version: %v", err)
}
fmt.Printf("Server v%s (built %s, commit %s)\n",
    ver.Version, ver.BuildTime, ver.GitCommit)

type ConfigGetRequest 

type ConfigGetRequest struct {
	Key string `json:"key"`
}

ConfigGetRequest represents the request body for getting configuration.

type ConfigGetResponse 

type ConfigGetResponse struct {
	Key   string `json:"key"`
	Value string `json:"value"`
}

ConfigGetResponse represents the response for getting configuration.

type ConfigInfo 

type ConfigInfo struct {
	Name          string `json:"name"`
	Registry      string `json:"registry"`
	ConfigVersion string `json:"config_version"`
	Host          string `json:"host"`
	Port          int    `json:"port"`
	ConfigDir     string `json:"config_dir"`
	DataDir       string `json:"data_dir"`
}

ConfigInfo represents the server configuration information response.

type ConfigReloadResponse 

type ConfigReloadResponse struct {
	Message       string `json:"message"`
	ConfigVersion string `json:"config_version"`
}

ConfigReloadResponse represents the response for reloading configuration.

type ConfigSetRequest 

type ConfigSetRequest struct {
	Key   string `json:"key"`
	Value string `json:"value"`
}

ConfigSetRequest represents the request body for setting configuration.

type ConfigSetResponse 

type ConfigSetResponse struct {
	Message string `json:"message"`
}

ConfigSetResponse represents the response for setting configuration.

type CurrentVersionResponse 

type CurrentVersionResponse struct {
	ConfigVersion string `json:"config_version"`
	Installed     bool   `json:"installed"`
}

CurrentVersionResponse represents the current version information.

type DeviceInfo 

type DeviceInfo struct {
	VendorID            string   `json:"vendor_id"`
	DeviceID            string   `json:"device_id"`
	BusAddress          string   `json:"bus_address"`
	ModelName           string   `json:"model_name"`
	ConfigKey           string   `json:"config_key"`
	DeviceType          string   `json:"device_type"`
	Generation          string   `json:"generation"`
	Capabilities        []string `json:"capabilities"`
	PhysicalDeviceIndex int      `json:"physical_device_index"`
	ChipIndex           int      `json:"chip_index"`
	ChipsPerDevice      int      `json:"chips_per_device"`
}

DeviceInfo represents device information returned from the server.

type ListVersionsResponse 

type ListVersionsResponse struct {
	CurrentXwVersion     string           `json:"current_xw_version"`
	CurrentConfigVersion string           `json:"current_config_version"`
	CompatibleVersions   []config.Package `json:"compatible_versions"`
	IncompatibleVersions []config.Package `json:"incompatible_versions"`
	InstalledVersions    []string         `json:"installed_versions"`
}

ListVersionsResponse represents the response from the list versions endpoint.

type SSEMessage 

type SSEMessage struct {
	// Type indicates the message category
	Type string `json:"type"`

	// Message contains the human-readable message content
	Message string `json:"message"`

	// Status indicates the final status (used with "complete" type)
	Status string `json:"status,omitempty"`

	// Path contains the file or resource path (if applicable)
	Path string `json:"path,omitempty"`
}

SSEMessage represents a parsed Server-Sent Events message.

SSE messages are sent from the server during long-running operations to provide real-time status updates and progress information.

Message types:

  • "status": General status update
  • "progress": Download progress update
  • "heartbeat": Keep-alive signal
  • "error": Error occurred during operation
  • "complete": Operation completed successfully
  • "end": Stream termination signal

type UpdateRequest 

type UpdateRequest struct {
	Version string `json:"version"`
}

UpdateRequest represents the update request body.

type UpdateResponse 

type UpdateResponse struct {
	Success         bool   `json:"success"`
	PreviousVersion string `json:"previous_version"`
	NewVersion      string `json:"new_version"`
	Downloaded      bool   `json:"downloaded"`
	Message         string `json:"message"`
	RestartRequired bool   `json:"restart_required"`
}

UpdateResponse represents the update response.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.