manager

package
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 18, 2025 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NeedsQueuing 

func NeedsQueuing(location, serviceType string) bool

NeedsQueuing determines if request needs to enter queuing mechanism (exported function)

Types

type Cleaner 

type Cleaner struct {
	// contains filtered or unexported fields
}

Cleaner 模型自动清理服务 负责定期检查并卸载空闲超时的模型

func NewCleaner 

func NewCleaner(queueChecker QueueStatusChecker, modelLoader ModelLoader) *Cleaner

NewCleaner 创建新的清理器

func (*Cleaner) ForceCleanup 

func (c *Cleaner) ForceCleanup()

ForceCleanup 强制执行一次清理

func (*Cleaner) GetIdleTimeout 

func (c *Cleaner) GetIdleTimeout() time.Duration

GetIdleTimeout 获取空闲超时时间

func (*Cleaner) GetStats 

func (c *Cleaner) GetStats() map[string]interface{}

GetStats 获取清理器统计信息

func (*Cleaner) IsStarted 

func (c *Cleaner) IsStarted() bool

IsStarted 检查清理服务是否已启动

func (*Cleaner) SetIdleTimeout 

func (c *Cleaner) SetIdleTimeout(timeout time.Duration)

SetIdleTimeout 设置空闲超时时间

func (*Cleaner) Start 

func (c *Cleaner) Start(cleanupInterval time.Duration)

Start 启动清理服务

func (*Cleaner) Stop 

func (c *Cleaner) Stop()

Stop 停止清理服务

type Loader 

type Loader struct {
	// contains filtered or unexported fields
}

Loader 模型加载管理器

func NewLoader 

func NewLoader(stateManager ModelStateManager) *Loader

NewLoader 创建新的加载器

func (*Loader) EnsureModelLoaded 

func (l *Loader) EnsureModelLoaded(ctx context.Context, modelName string, providerInstance provider.ModelServiceProvider, providerName, providerType string) error

EnsureModelLoaded 确保模型已加载，如果未加载则自动加载

func (*Loader) ForceUnloadModel 

func (l *Loader) ForceUnloadModel(modelName string) error

ForceUnloadModel 强制卸载指定模型

func (*Loader) GetAllModelStates 

func (l *Loader) GetAllModelStates() map[string]*ModelState

GetAllModelStates 获取所有模型状态信息

func (*Loader) GetCurrentModel 

func (l *Loader) GetCurrentModel() string

GetCurrentModel 获取当前加载的模型

func (*Loader) GetIdleModels 

func (l *Loader) GetIdleModels(idleTimeout time.Duration) []*ModelState

GetIdleModels 获取空闲超时的模型列表

func (*Loader) GetModelState 

func (l *Loader) GetModelState(modelName string) (*ModelState, bool)

GetModelState 获取模型状态信息

func (*Loader) GetStats 

func (l *Loader) GetStats() map[string]interface{}

GetStats 获取加载器统计信息

func (*Loader) InitializeRunningModels 

func (l *Loader) InitializeRunningModels()

InitializeRunningModels 初始化时获取已运行的模型并清理它们

func (*Loader) LoadModel 

func (l *Loader) LoadModel(modelName string, provider provider.ModelServiceProvider) error

LoadModel 加载模型

func (*Loader) MarkModelIdle 

func (l *Loader) MarkModelIdle(modelName string) error

MarkModelIdle 标记模型为空闲

func (*Loader) MarkModelInUse 

func (l *Loader) MarkModelInUse(modelName string) error

MarkModelInUse 标记模型为使用中

func (*Loader) SwitchModel 

func (l *Loader) SwitchModel(newModel string, provider provider.ModelServiceProvider) error

SwitchModel 切换模型

func (*Loader) UnloadModel 

func (l *Loader) UnloadModel(modelName string, provider provider.ModelServiceProvider) error

UnloadModel 卸载模型

type Manager 

type Manager struct {
	// contains filtered or unexported fields
}

Manager model manager, implements ModelStateManager interface

func GetModelManager 

func GetModelManager() *Manager

GetModelManager 获取全局模型管理器实例

func GetModelMemoryManager 

func GetModelMemoryManager() *Manager

GetModelMemoryManager 获取全局模型管理器实例（保持向后兼容）

func NewManager 

func NewManager() *Manager

NewManager 创建新的模型管理器

func (*Manager) CompleteLocalModelRequest 

func (m *Manager) CompleteLocalModelRequest(taskID uint64)

CompleteLocalModelRequest 完成本地模型请求处理

func (*Manager) EnqueueLocalModelRequest 

func (m *Manager) EnqueueLocalModelRequest(ctx context.Context, modelName string, providerInstance provider.ModelServiceProvider, providerName, providerType string, taskID uint64) (chan struct{}, chan error, error)

EnqueueLocalModelRequest 将本地模型请求加入队列，返回准备完成通道和错误通道

func (*Manager) ForceUnloadModel 

func (m *Manager) ForceUnloadModel(modelName string) error

ForceUnloadModel 强制卸载指定模型

func (*Manager) GetAllModelStates 

func (m *Manager) GetAllModelStates() map[string]*ModelState

GetAllModelStates 获取所有模型状态信息

func (*Manager) GetCurrentModel 

func (m *Manager) GetCurrentModel() string

GetCurrentModel 获取当前加载的模型

func (*Manager) GetModelState 

func (m *Manager) GetModelState(modelName string) (*ModelState, bool)

GetModelState 获取模型状态信息

func (*Manager) GetStats 

func (m *Manager) GetStats() map[string]interface{}

GetStats 获取管理器统计信息

func (*Manager) MarkModelIdle 

func (m *Manager) MarkModelIdle(modelName string) error

MarkModelIdle 标记模型为空闲

func (*Manager) MarkModelInUse 

func (m *Manager) MarkModelInUse(modelName string) error

MarkModelInUse 标记模型为使用中

func (*Manager) SetCurrentModel 

func (m *Manager) SetCurrentModel(modelName string)

SetCurrentModel 设置当前模型

func (*Manager) SetIdleTimeout 

func (m *Manager) SetIdleTimeout(timeout time.Duration)

SetIdleTimeout 设置空闲超时时间

func (*Manager) Start 

func (m *Manager) Start(cleanupInterval time.Duration)

Start 启动模型管理器

func (*Manager) Stop 

func (m *Manager) Stop()

Stop 停止模型管理器

type ModelLoader 

type ModelLoader interface {
	SwitchModel(newModel string, provider provider.ModelServiceProvider) error
	LoadModel(modelName string, provider provider.ModelServiceProvider) error
	UnloadModel(modelName string, provider provider.ModelServiceProvider) error
	ForceUnloadModel(modelName string) error
}

ModelLoader 模型加载接口

type ModelState 

type ModelState struct {
	ModelName    string                        // 模型名称
	ProviderName string                        // 提供商名称
	ProviderType string                        // 提供商类型
	Status       ModelStatus                   // 当前状态
	LastUsedTime time.Time                     // 最后使用时间
	LoadedTime   time.Time                     // 加载时间
	RefCount     int                           // 引用计数（并发使用数）
	Provider     provider.ModelServiceProvider // 提供商实例
	// contains filtered or unexported fields
}

ModelState 模型状态信息

func (*ModelState) DecrementRef 

func (ms *ModelState) DecrementRef()

DecrementRef 减少引用计数，如果为0则设置为空闲

func (*ModelState) GetRefCount 

func (ms *ModelState) GetRefCount() int

GetRefCount 获取引用计数（线程安全）

func (*ModelState) GetStatus 

func (ms *ModelState) GetStatus() ModelStatus

GetStatus 获取当前状态（线程安全）

func (*ModelState) IncrementRef 

func (ms *ModelState) IncrementRef()

IncrementRef 增加引用计数并设置为使用中

func (*ModelState) IsIdle 

func (ms *ModelState) IsIdle(idleTimeout time.Duration) bool

IsIdle 检查是否空闲且超过指定时间

func (*ModelState) SetStatus 

func (ms *ModelState) SetStatus(status ModelStatus)

SetStatus 设置状态（线程安全）

func (*ModelState) UpdateLastUsedTime 

func (ms *ModelState) UpdateLastUsedTime()

UpdateLastUsedTime 更新最后使用时间

type ModelStateManager 

type ModelStateManager interface {
	GetCurrentModel() string
	MarkModelInUse(modelName string) error
	MarkModelIdle(modelName string) error
	SetCurrentModel(modelName string)
}

ModelStateManager 模型状态管理接口

type ModelStatus 

type ModelStatus int

ModelStatus 模型状态枚举 

const (
	ModelStatusUnloaded  ModelStatus = iota // 未加载
	ModelStatusLoading                      // 加载中
	ModelStatusIdle                         // 空闲
	ModelStatusInUse                        // 使用中
	ModelStatusUnloading                    // 卸载中
)

func (ModelStatus) String 

func (s ModelStatus) String() string

type Queue 

type Queue struct {
	// contains filtered or unexported fields
}

Queue model request queue

func NewQueue 

func NewQueue(stateManager ModelStateManager, loader ModelLoader) *Queue

NewQueue creates a new queue

func (*Queue) CompleteRequest 

func (q *Queue) CompleteRequest(taskID uint64)

CompleteRequest 完成请求处理

func (*Queue) EnqueueRequest 

func (q *Queue) EnqueueRequest(request *QueuedRequest) error

EnqueueRequest 将请求加入队列（非阻塞）

func (*Queue) GetCurrentRequest 

func (q *Queue) GetCurrentRequest() *QueuedRequest

GetCurrentRequest 获取当前处理的请求

func (*Queue) GetQueueLength 

func (q *Queue) GetQueueLength() int

GetQueueLength 获取当前队列长度

func (*Queue) GetStats 

func (q *Queue) GetStats() map[string]interface{}

GetStats 获取队列统计信息

func (*Queue) HasActiveRequests 

func (q *Queue) HasActiveRequests() bool

HasActiveRequests 检查是否有活跃的请求

func (*Queue) HasPendingRequests 

func (q *Queue) HasPendingRequests() bool

HasPendingRequests 检查是否有待处理的请求

func (*Queue) IsProcessing 

func (q *Queue) IsProcessing() bool

IsProcessing 检查是否正在处理请求

func (*Queue) Start 

func (q *Queue) Start()

Start starts queue processing

func (*Queue) Stop 

func (q *Queue) Stop()

Stop stops queue processing

type QueueStatusChecker 

type QueueStatusChecker interface {
	HasPendingRequests() bool
	HasActiveRequests() bool
	GetCurrentRequest() *QueuedRequest
}

QueueStatusChecker 队列状态检查接口

type QueuedRequest 

type QueuedRequest struct {
	TaskID       uint64                        // Task ID
	ModelName    string                        // Model name
	Provider     provider.ModelServiceProvider // Provider instance
	ProviderName string                        // Provider name
	ProviderType string                        // Provider type
	Context      context.Context               // Context
	StartTime    time.Time                     // Start time
	ReadyChan    chan struct{}                 // Model ready notification channel
	CompleteChan chan struct{}                 // Task execution completion notification channel
	ErrorChan    chan error                    // Error notification channel
}

QueuedRequest queued request

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.