Documentation
¶
Index ¶
Constants ¶
View Source
const (
// Name is the backend name.
Name = "sglang"
)
Variables ¶
Functions ¶
func GetContextLength ¶
func GetContextLength(modelCfg types.ModelConfig, backendCfg *inference.BackendConfiguration) *int32
GetContextLength returns the context length (context size) from model config or backend config. Model config takes precedence over backend config. Returns nil if neither is specified (SGLang will auto-derive from model).
func New ¶
func New(log logging.Logger, modelManager *models.Manager, serverLog logging.Logger, conf *Config, customPythonPath string) (inference.Backend, error)
New creates a new SGLang-based backend. customPythonPath is an optional path to a custom python3 binary; if empty, the default path is used.
Types ¶
type Config ¶
type Config struct {
// Args are the base arguments that are always included.
Args []string
}
Config is the configuration for the SGLang backend.
func NewDefaultSGLangConfig ¶
func NewDefaultSGLangConfig() *Config
NewDefaultSGLangConfig creates a new SGLangConfig with default values.
func (*Config) GetArgs ¶
func (c *Config) GetArgs(bundle types.ModelBundle, socket string, mode inference.BackendMode, config *inference.BackendConfiguration) ([]string, error)
GetArgs implements BackendConfig.GetArgs.
Source Files
Click to show internal directories.
Click to hide internal directories.