localllm

package

v0.40.2 Latest Latest Go to latest Published: May 26, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/kombifyio/SpeechKit

Links

Open Source Insights

Documentation ¶

Rendered for

Overview ¶

Package localllm manages the lifecycle of the local llama.cpp HTTP server SpeechKit ships alongside the desktop bundle for offline LLM inference. Owns process spawn, health-probing, graceful shutdown, and the OpenAI-compatible client wrapper the Assist + Voice Agent paths route to when Local is the selected provider.

Audit 2026-05-24 maintainability sweep.

Index ¶

func FindServerBinary() (string, error)
func ProbeEndpoint(ctx context.Context, baseURL string) error
func ValidateModelPath(path string) error
type InstallStatus
type Server
- func NewServer(port int, modelPath, gpu string) *Server

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func FindServerBinary ¶

func FindServerBinary() (string, error)

func ProbeEndpoint ¶

func ProbeEndpoint(ctx context.Context, baseURL string) error

func ValidateModelPath ¶

func ValidateModelPath(path string) error

ValidateModelPath verifies that path points at a safe GGUF model filename.

Types ¶

type InstallStatus ¶

type InstallStatus struct {
	BinaryFound bool     `json:"binaryFound"`
	BinaryPath  string   `json:"binaryPath"`
	ModelFound  bool     `json:"modelFound"`
	ModelPath   string   `json:"modelPath"`
	ModelBytes  int64    `json:"modelBytes"`
	ServerReady bool     `json:"serverReady"`
	Problems    []string `json:"problems,omitempty"`
}

type Server ¶

type Server struct {
	BaseURL     string
	Port        int
	ModelPath   string
	GPU         string
	ContextSize int
	Threads     int
	Validation  netsec.ValidationOptions
	// contains filtered or unexported fields
}

Server manages SpeechKit's bundled llama.cpp OpenAI-compatible server.

func NewServer ¶

func NewServer(port int, modelPath, gpu string) *Server

func (*Server) Health ¶

func (s *Server) Health(ctx context.Context) error

func (*Server) IsReady ¶

func (s *Server) IsReady() bool

func (*Server) RuntimeKey ¶

func (s *Server) RuntimeKey() string

func (*Server) StartServer ¶

func (s *Server) StartServer(ctx context.Context) error

func (*Server) StopServer ¶

func (s *Server) StopServer()

func (*Server) VerifyInstallation ¶

func (s *Server) VerifyInstallation() InstallStatus

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL