Documentation
¶
Overview ¶
Package worker provides subprocess worker management for llm-compiler. It handles spawning worker processes, communication protocol, and lifecycle management.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func IsWorkerProcess ¶
func IsWorkerProcess() bool
IsWorkerProcess returns true if the current process is running as a worker
func ShouldUseSubprocess ¶
func ShouldUseSubprocess() bool
ShouldUseSubprocess returns true if subprocess mode is enabled
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client manages communication with a worker subprocess
func NewClientWithFd ¶
NewClientWithFd creates a worker subprocess and passes the given file as fd3
type Handler ¶
type Handler interface {
// Generate processes a prompt and returns the generated text
Generate(prompt, modelSpec string, maxTokens int) (string, error)
}
Handler is the interface that must be implemented to handle worker requests
type Pool ¶
type Pool struct {
// contains filtered or unexported fields
}
Pool manages a pool of worker clients for parallel processing
type Request ¶
type Request struct {
ID string `json:"id"`
ModelSpec string `json:"model_spec"`
Prompt string `json:"prompt"`
MaxTokens int `json:"max_tokens"`
}
Request is sent from client to worker over stdin as JSON newline.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server runs inside a spawned worker process and handles incoming requests
func (*Server) Run ¶
func (s *Server) Run()
Run starts the server loop, reading requests from stdin and writing responses to stdout. This method blocks until stdin is closed or an error occurs.
func (*Server) WriteStatus ¶
WriteStatus writes a status message to the status output (fd3 or stderr)
Source Files