llmworker

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 30, 2026 License: BSD-2-Clause Imports: 10 Imported by: 0

Documentation

Overview

Package llmworker is a pipeline worker built around a language model, with the tool handling that makes one usable from the bus.

A worker's tools run while the pipeline that called them is still running, and what a tool does often needs to reach that pipeline: appending to the conversation, ending the session, handing over to another worker. Doing any of it while the call is still in flight puts it out of order with the result the model is waiting for. This package holds it back until the call is done.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type ActivationArgs

type ActivationArgs struct {
	workers.BaseActivationArgs
	// Messages are appended to the conversation when the worker activates.
	Messages []map[string]any
	// RunLLM says whether the model answers them. Nil answers, which is what a
	// caller sending messages almost always means.
	RunLLM *bool
}

ActivationArgs are what an LLM worker understands when it is activated: the messages to put into the conversation, and whether to answer them.

func ActivationArgsFrom

func ActivationArgsFrom(args map[string]any) ActivationArgs

ActivationArgsFrom reads what an LLM worker understands out of the arguments it was activated with, ignoring anything else in them.

func (ActivationArgs) ToMap

func (a ActivationArgs) ToMap() map[string]any

ToMap implements workers.ActivationArgs.

type Config

type Config struct {
	// Name is what other workers address this one by.
	Name string
	// LLM is the language model service the worker runs. The tools are
	// registered on it.
	LLM ToolRegistrar
	// Pipeline is the pipeline to run. Nil runs the model on its own, which is
	// what a worker doing nothing but answering needs.
	Pipeline processor.Processor
	// Tools are the tools the model may call. Each is registered on the service
	// with the call options it carries, wrapped so that what its handler queues
	// is held until the call finishes.
	Tools []frames.Tool
	// Active reports whether the worker starts active. Nil leaves it inactive,
	// because a worker with a model behind it is almost always started by
	// something else deciding it is time.
	Active *bool
	// Bridged names the bridges this worker's pipeline exchanges frames over.
	// Non-nil wraps the pipeline in bus edges, and turns off the RTVI processor
	// a standalone worker gets, since a bridged worker has no client of its own.
	Bridged []string
	// DeferToolFrames says whether what a tool handler queues is held until
	// every call in flight has finished. Nil holds it, which is what keeps a
	// tool's own frames behind the result the model is waiting for.
	DeferToolFrames *bool
	// WorkerConfig is the rest of the pipeline worker's configuration. Name,
	// Active, Bridged and the RTVI setting are taken from the fields above.
	WorkerConfig pipeline.WorkerConfig
}

Config configures an LLM worker.

type ContextConfig

type ContextConfig struct {
	// Name is what other workers address this one by.
	Name string
	// LLM is the language model service the worker runs.
	LLM ToolRegistrar
	// Tools are the tools the model may call.
	Tools []frames.Tool
	// Active reports whether the worker starts active; nil leaves it inactive.
	Active *bool
	// Bridged names the bridges this worker's pipeline exchanges frames over.
	Bridged []string
	// DeferToolFrames says whether what a tool handler queues is held until the
	// calls in flight finish; nil holds it.
	DeferToolFrames *bool
	// Context is the conversation to run. Nil starts an empty one.
	Context *frames.LLMContext
	// AggregatorOptions configure the aggregator pair the worker builds.
	AggregatorOptions []aggregators.Option
	// WorkerConfig is the rest of the pipeline worker's configuration.
	WorkerConfig pipeline.WorkerConfig
}

ContextConfig configures a context worker. It is Config without the pipeline, which the worker builds itself around the conversation it owns.

type ContextWorker

type ContextWorker struct {
	*Worker
	// contains filtered or unexported fields
}

ContextWorker is an LLM worker that owns its conversation.

It is for a worker that tracks a conversation of its own rather than sharing the one a transport pipeline carries: the pipeline is built as user aggregator, model, assistant aggregator, so the worker keeps its own history without the caller wiring it up.

func NewContext

func NewContext(cfg ContextConfig) *ContextWorker

NewContext builds a worker that owns its conversation.

func (*ContextWorker) AssistantAggregator

func (c *ContextWorker) AssistantAggregator() *aggregators.AssistantAggregator

AssistantAggregator is the half that gathers what the bot said, and is where the events reporting a finished turn are raised.

func (*ContextWorker) Context

func (c *ContextWorker) Context() *frames.LLMContext

Context is the conversation this worker owns.

func (*ContextWorker) UserAggregator

func (c *ContextWorker) UserAggregator() *aggregators.UserAggregator

UserAggregator is the half that gathers what the user said.

type ToolRegistrar

type ToolRegistrar interface {
	processor.Processor
	RegisterFunction(name string, h llm.FunctionCallHandler, opts ...llm.RegisterOption)
}

ToolRegistrar is what an LLM worker needs of the service it runs: somewhere to register the tools. The LLM service base satisfies it.

type Worker

type Worker struct {
	*pipeline.Worker
	// contains filtered or unexported fields
}

Worker runs a language model as a worker on the bus, and holds back what its tools do until the calls that asked for it have finished.

A tool handler runs while the model is still waiting for its result. Anything the handler does to the pipeline in the meantime, appending to the conversation, ending the session, handing over to another worker, would land ahead of that result and put the turn out of order. What the handler queues is therefore held, and released once the last call in flight is done; ending and handing over wait for the same moment.

func New

func New(cfg Config) *Worker

New builds an LLM worker.

func (*Worker) ActivateWorker

func (w *Worker) ActivateWorker(ctx context.Context, name string, opts workers.ActivateOptions)

ActivateWorker hands over to another worker once the tool call asking for it is done, for the reason End waits.

func (*Worker) End

func (w *Worker) End(ctx context.Context, reason string)

End brings the session to a close once the tool call asking for it is done.

A tool that ends the session is still running when it asks, and ending the worker underneath it would leave the rest of that call to nobody. Deliver the call's result first, so what the model says about it is spoken before the session goes.

func (*Worker) LLM

func (w *Worker) LLM() ToolRegistrar

LLM is the service this worker runs.

func (*Worker) OnActivated

func (w *Worker) OnActivated(ctx context.Context, args map[string]any)

OnActivated advertises the tools and puts the activation's messages into the conversation.

func (*Worker) ProcessDeferredFrames

func (w *Worker) ProcessDeferredFrames(_ context.Context, held []deferredFrame) []deferredFrame

ProcessDeferredFrames is called with the frames held during a run of tool calls, just before they are queued, and returns the ones to queue. The default returns them as they are; override it to inspect, reorder or drop them.

func (*Worker) QueueFrame

func (w *Worker) QueueFrame(ctx context.Context, f frames.Frame, dir ...processor.Direction)

QueueFrame queues a frame, holding it when one of this worker's own tool handlers is what queued it.

A frame queued from inside a handler, or from anything that handler passes its context to, is held and delivered once the last call finishes. Everything else is queued at once: the worker's own traffic, what arrives over the bus, and what a handler on a different worker queues here, which this worker would never release.

Reaching past this to the embedded worker's QueueFrame queues without holding, which is what a caller wants when it is deliberately acting inside a call.

func (*Worker) ToolCallActive

func (w *Worker) ToolCallActive() bool

ToolCallActive reports whether any of this worker's tools is running.

func (*Worker) Tools

func (w *Worker) Tools() []frames.Tool

Tools are the tools the worker registered, which is what it advertises when it activates.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL