Documentation
¶
Overview ¶
Package llmworker is a pipeline worker built around a language model, with the tool handling that makes one usable from the bus.
A worker's tools run while the pipeline that called them is still running, and what a tool does often needs to reach that pipeline: appending to the conversation, ending the session, handing over to another worker. Doing any of it while the call is still in flight puts it out of order with the result the model is waiting for. This package holds it back until the call is done.
Index ¶
- type ActivationArgs
- type Config
- type ContextConfig
- type ContextWorker
- type ToolRegistrar
- type Worker
- func (w *Worker) ActivateWorker(ctx context.Context, name string, opts workers.ActivateOptions)
- func (w *Worker) End(ctx context.Context, reason string)
- func (w *Worker) LLM() ToolRegistrar
- func (w *Worker) OnActivated(ctx context.Context, args map[string]any)
- func (w *Worker) ProcessDeferredFrames(_ context.Context, held []deferredFrame) []deferredFrame
- func (w *Worker) QueueFrame(ctx context.Context, f frames.Frame, dir ...processor.Direction)
- func (w *Worker) ToolCallActive() bool
- func (w *Worker) Tools() []frames.Tool
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ActivationArgs ¶
type ActivationArgs struct {
workers.BaseActivationArgs
// Messages are appended to the conversation when the worker activates.
Messages []map[string]any
// RunLLM says whether the model answers them. Nil answers, which is what a
// caller sending messages almost always means.
RunLLM *bool
}
ActivationArgs are what an LLM worker understands when it is activated: the messages to put into the conversation, and whether to answer them.
func ActivationArgsFrom ¶
func ActivationArgsFrom(args map[string]any) ActivationArgs
ActivationArgsFrom reads what an LLM worker understands out of the arguments it was activated with, ignoring anything else in them.
func (ActivationArgs) ToMap ¶
func (a ActivationArgs) ToMap() map[string]any
ToMap implements workers.ActivationArgs.
type Config ¶
type Config struct {
// Name is what other workers address this one by.
Name string
// LLM is the language model service the worker runs. The tools are
// registered on it.
LLM ToolRegistrar
// Pipeline is the pipeline to run. Nil runs the model on its own, which is
// what a worker doing nothing but answering needs.
Pipeline processor.Processor
// Tools are the tools the model may call. Each is registered on the service
// with the call options it carries, wrapped so that what its handler queues
// is held until the call finishes.
Tools []frames.Tool
// Active reports whether the worker starts active. Nil leaves it inactive,
// because a worker with a model behind it is almost always started by
// something else deciding it is time.
Active *bool
// Bridged names the bridges this worker's pipeline exchanges frames over.
// Non-nil wraps the pipeline in bus edges, and turns off the RTVI processor
// a standalone worker gets, since a bridged worker has no client of its own.
Bridged []string
// DeferToolFrames says whether what a tool handler queues is held until
// every call in flight has finished. Nil holds it, which is what keeps a
// tool's own frames behind the result the model is waiting for.
DeferToolFrames *bool
// WorkerConfig is the rest of the pipeline worker's configuration. Name,
// Active, Bridged and the RTVI setting are taken from the fields above.
WorkerConfig pipeline.WorkerConfig
}
Config configures an LLM worker.
type ContextConfig ¶
type ContextConfig struct {
// Name is what other workers address this one by.
Name string
// LLM is the language model service the worker runs.
LLM ToolRegistrar
// Tools are the tools the model may call.
Tools []frames.Tool
// Active reports whether the worker starts active; nil leaves it inactive.
Active *bool
// Bridged names the bridges this worker's pipeline exchanges frames over.
Bridged []string
// DeferToolFrames says whether what a tool handler queues is held until the
// calls in flight finish; nil holds it.
DeferToolFrames *bool
// Context is the conversation to run. Nil starts an empty one.
Context *frames.LLMContext
// AggregatorOptions configure the aggregator pair the worker builds.
AggregatorOptions []aggregators.Option
// WorkerConfig is the rest of the pipeline worker's configuration.
WorkerConfig pipeline.WorkerConfig
}
ContextConfig configures a context worker. It is Config without the pipeline, which the worker builds itself around the conversation it owns.
type ContextWorker ¶
type ContextWorker struct {
*Worker
// contains filtered or unexported fields
}
ContextWorker is an LLM worker that owns its conversation.
It is for a worker that tracks a conversation of its own rather than sharing the one a transport pipeline carries: the pipeline is built as user aggregator, model, assistant aggregator, so the worker keeps its own history without the caller wiring it up.
func NewContext ¶
func NewContext(cfg ContextConfig) *ContextWorker
NewContext builds a worker that owns its conversation.
func (*ContextWorker) AssistantAggregator ¶
func (c *ContextWorker) AssistantAggregator() *aggregators.AssistantAggregator
AssistantAggregator is the half that gathers what the bot said, and is where the events reporting a finished turn are raised.
func (*ContextWorker) Context ¶
func (c *ContextWorker) Context() *frames.LLMContext
Context is the conversation this worker owns.
func (*ContextWorker) UserAggregator ¶
func (c *ContextWorker) UserAggregator() *aggregators.UserAggregator
UserAggregator is the half that gathers what the user said.
type ToolRegistrar ¶
type ToolRegistrar interface {
processor.Processor
RegisterFunction(name string, h llm.FunctionCallHandler, opts ...llm.RegisterOption)
}
ToolRegistrar is what an LLM worker needs of the service it runs: somewhere to register the tools. The LLM service base satisfies it.
type Worker ¶
Worker runs a language model as a worker on the bus, and holds back what its tools do until the calls that asked for it have finished.
A tool handler runs while the model is still waiting for its result. Anything the handler does to the pipeline in the meantime, appending to the conversation, ending the session, handing over to another worker, would land ahead of that result and put the turn out of order. What the handler queues is therefore held, and released once the last call in flight is done; ending and handing over wait for the same moment.
func (*Worker) ActivateWorker ¶
ActivateWorker hands over to another worker once the tool call asking for it is done, for the reason End waits.
func (*Worker) End ¶
End brings the session to a close once the tool call asking for it is done.
A tool that ends the session is still running when it asks, and ending the worker underneath it would leave the rest of that call to nobody. Deliver the call's result first, so what the model says about it is spoken before the session goes.
func (*Worker) OnActivated ¶
OnActivated advertises the tools and puts the activation's messages into the conversation.
func (*Worker) ProcessDeferredFrames ¶
ProcessDeferredFrames is called with the frames held during a run of tool calls, just before they are queued, and returns the ones to queue. The default returns them as they are; override it to inspect, reorder or drop them.
func (*Worker) QueueFrame ¶
QueueFrame queues a frame, holding it when one of this worker's own tool handlers is what queued it.
A frame queued from inside a handler, or from anything that handler passes its context to, is held and delivered once the last call finishes. Everything else is queued at once: the worker's own traffic, what arrives over the bus, and what a handler on a different worker queues here, which this worker would never release.
Reaching past this to the embedded worker's QueueFrame queues without holding, which is what a caller wants when it is deliberately acting inside a call.
func (*Worker) ToolCallActive ¶
ToolCallActive reports whether any of this worker's tools is running.