Documentation
¶
Overview ¶
Package flows builds structured, multi-step conversations on top of jargo's LLM service.
A conversation is a graph of nodes. Each node sets the assistant's persona and objective, the tools it may call, and what runs on the way in and out. A tool whose handler returns the next node is an edge that moves the conversation on; one that returns no next node only gathers data. A caller outside the graph moves it with SetNode, for a transition the conversation never asked for.
A FlowManager sits beside the pipeline rather than in it. It steers the conversation by queueing frames as nodes are entered: the persona as the LLM service's system instruction, the node's objective as messages, and its toolset. Queueing rather than writing to the conversation directly is what orders each change against whatever else is in flight, and what lets a realtime service learn the toolset changed. The LLM service's tool loop then carries out each transition.
Transitions ¶
An edge function does not move the flow where it stands. It records the transition and reports its result with a context-updated callback, so the move happens once the result has reached the conversation and every other tool call of that turn has reported. A turn that called three tools therefore enters the next node once, against a finished turn.
Context strategies ¶
Entering a node appends its objective to the conversation by default, so everything said so far is kept. A node or the whole flow may instead reset the conversation, or reset it to a generated summary of what was said, which keeps the substance of a long call without its length.
Actions ¶
Actions are how a flow does something other than talk. Three are built in: "tts_say" speaks a fixed line, "end_conversation" ends the call, and "function" runs a handler at the point in the pipeline it reaches, once the speech queued ahead of it has been said. An application registers whatever else it needs with RegisterAction. A node's pre-actions run before its context is applied; its post-actions run after, or, on a node that waits for the user, once the assistant's first turn there is over.
Index ¶
- Variables
- type ActionConfig
- type ActionFinishedFrame
- type ActionHandler
- type Config
- type ContextStrategy
- type ContextStrategyConfig
- type Enqueuer
- type FlowManager
- func (fm *FlowManager) CurrentContext() ([]frames.Message, error)
- func (fm *FlowManager) CurrentNode() string
- func (fm *FlowManager) Initialize(ctx context.Context, node *NodeConfig) error
- func (fm *FlowManager) RegisterAction(actionType string, h ActionHandler) error
- func (fm *FlowManager) SetNode(ctx context.Context, node *NodeConfig) error
- func (fm *FlowManager) State() *State
- type FunctionActionFrame
- type Handler
- type Inferencer
- type NodeConfig
- type NodeFunction
- type State
- type Watcher
Constants ¶
This section is empty.
Variables ¶
var ( // ErrFlow is the error every flow failure wraps. Match it to catch them all. ErrFlow = errors.New("flows: flow error") // ErrFlowInitialization reports that the flow could not be entered: // its opening node failed to be set up. ErrFlowInitialization = errors.New("flows: initialization failed") // ErrFlowTransition reports that a transition could not be made, which is // almost always a manager that was asked to move before it was initialized. ErrFlowTransition = errors.New("flows: transition failed") // ErrInvalidFunction reports a function a node cannot offer: one with no // name, no handler, or a shape the manager cannot call. ErrInvalidFunction = errors.New("flows: invalid function") // ErrAction reports that an action failed to run. ErrAction = errors.New("flows: action failed") )
The errors a flow reports. They form a hierarchy: every one of them wraps ErrFlow, so a caller that only wants to know the flow failed matches on that, and one that wants to know how matches on the specific error.
var NoResponse = &NodeConfig{Name: "no-response"}
NoResponse is the sentinel a Handler returns in its next-node slot to finish the call without transitioning and without generation re-running.
It is for a tool that has already said its piece: one that plays audio of its own, or that ends the conversation. A result ordinarily goes back to the model for it to answer from, and that answer would be spoken over whatever the tool started. The result still reaches the conversation, so the model knows what happened when the user speaks next.
Functions ¶
This section is empty.
Types ¶
type ActionConfig ¶ added in v0.1.0
type ActionConfig struct {
// Type identifies the action. It is required.
Type string
// Handler runs the action. It is required for the "function" action, which is
// nothing but a handler to run in pipeline order, and optional otherwise: for
// any other type it registers the handler under Type when the action is first
// seen.
Handler ActionHandler
// Text is what "tts_say" speaks, or the optional goodbye "end_conversation"
// speaks before ending.
Text string
// AppendTextToContext controls whether the text the built-in speaking actions
// say is written to the conversation. Nil defaults to true.
AppendTextToContext *bool
// Params carries whatever else the action needs, for a custom handler to
// read. The built-in actions do not look at it.
Params map[string]any
}
ActionConfig configures one action to run on entering or leaving a node.
Type names the action, and is what decides which handler runs: one of the built-ins ("tts_say", "end_conversation", "function") or a type registered with FlowManager.RegisterAction. A config carrying a Handler registers it under Type the first time the action is seen, so an action can bring its own implementation rather than being registered up front.
type ActionFinishedFrame ¶ added in v0.1.0
type ActionFinishedFrame struct {
frames.BaseControlFrame
}
ActionFinishedFrame marks an action as complete. An action that queues frames queues one of these behind them, so the action counts as finished when its frames have been processed rather than when it returned.
func NewActionFinishedFrame ¶ added in v0.1.0
func NewActionFinishedFrame() *ActionFinishedFrame
NewActionFinishedFrame builds an ActionFinishedFrame.
type ActionHandler ¶ added in v0.1.0
type ActionHandler func(ctx context.Context, action ActionConfig, fm *FlowManager) error
ActionHandler runs one action. It is given the action that named it and the manager driving the flow, so an action can read the flow's state or move it.
type Config ¶
type Config struct {
// Enqueuer queues the frames a flow produces; usually the *pipeline.Worker.
Enqueuer Enqueuer
// Watcher reports frames reaching the end of the pipeline, which is how an
// action learns it has finished; usually the same *pipeline.Worker. Leaving it
// nil gives up the actions that wait on the pipeline.
Watcher Watcher
// Aggregators are the conversation aggregators the pipeline shares. The flow
// reads the conversation through them, and asks the assistant side whether a
// turn's tool calls have all reported before it transitions.
Aggregators *aggregators.Pair
// LLM answers the one-shot inference the reset-with-summary strategy needs. It
// is only required by that strategy; the concrete LLM services satisfy it.
LLM Inferencer
// ContextStrategy is what happens to the conversation when a node is entered.
// Nil appends, keeping everything said so far.
ContextStrategy *ContextStrategyConfig
// GlobalFunctions are offered at every node, ahead of the node's own.
GlobalFunctions []NodeFunction
}
Config configures a FlowManager. The references are the same instances wired into the pipeline.
type ContextStrategy ¶ added in v0.1.0
type ContextStrategy string
ContextStrategy says what happens to the conversation when a node is entered.
const ( // ContextStrategyAppend adds the node's task messages to the conversation, // keeping everything said so far. It is the default. ContextStrategyAppend ContextStrategy = "append" // ContextStrategyReset replaces the conversation with the node's task // messages, so the node starts from a clean slate. ContextStrategyReset ContextStrategy = "reset" // ContextStrategyResetWithSummary replaces the conversation with a summary of // what was said, followed by the node's task messages. It keeps the substance // of a long conversation without its length. ContextStrategyResetWithSummary ContextStrategy = "reset_with_summary" )
type ContextStrategyConfig ¶ added in v0.1.0
type ContextStrategyConfig struct {
// Strategy is what happens to the conversation.
Strategy ContextStrategy
// SummaryPrompt guides the summary, and is required by
// ContextStrategyResetWithSummary.
SummaryPrompt string
}
ContextStrategyConfig configures how the conversation is updated when a node is entered. Set it on the manager for every node, or on a node to override it for that one.
func (ContextStrategyConfig) Validate ¶ added in v0.1.0
func (c ContextStrategyConfig) Validate() error
Validate reports whether the configuration is usable.
type Enqueuer ¶
type Enqueuer interface {
QueueFrame(f frames.Frame, dir ...processor.Direction)
QueueFrames(fs []frames.Frame, dir ...processor.Direction)
}
Enqueuer injects frames into the running pipeline; *pipeline.Worker satisfies it. Everything a flow does to the conversation it does by queueing a frame, so the change is ordered against whatever else is in flight.
type FlowManager ¶
type FlowManager struct {
// contains filtered or unexported fields
}
FlowManager drives a conversation through a graph of nodes.
It is not a pipeline processor. It holds the conversation and the task and steers the conversation by queueing frames as nodes are entered: the persona, the node's objective and its toolset. The LLM service's tool loop then carries out each transition. Build one with New and enter the graph with Initialize.
func (*FlowManager) CurrentContext ¶ added in v0.1.0
func (fm *FlowManager) CurrentContext() ([]frames.Message, error)
CurrentContext returns the conversation as it stands.
func (*FlowManager) CurrentNode ¶
func (fm *FlowManager) CurrentNode() string
CurrentNode returns the name of the node the flow is on, or "" before Initialize.
func (*FlowManager) Initialize ¶
func (fm *FlowManager) Initialize(ctx context.Context, node *NodeConfig) error
Initialize enters the flow.
Passing a node starts the conversation there. Passing nil initializes the manager without entering a node, for a flow whose opening node is decided later and set with SetNode. Initializing twice is reported and otherwise ignored.
func (*FlowManager) RegisterAction ¶ added in v0.1.0
func (fm *FlowManager) RegisterAction(actionType string, h ActionHandler) error
RegisterAction records a handler for an action type, so a node may use it in its pre- or post-actions. The built-in types ("tts_say", "end_conversation" and "function") are registered already.
func (*FlowManager) SetNode ¶ added in v0.1.0
func (fm *FlowManager) SetNode(ctx context.Context, node *NodeConfig) error
SetNode transitions the flow to node. Enter the flow with Initialize first.
It is the manual transition, for a caller that is not part of the graph: a processor moving the conversation on something the model was never asked about. A node function transitions by returning the next node instead, which keeps the move on the tool loop that made it.
func (*FlowManager) State ¶ added in v0.1.0
func (fm *FlowManager) State() *State
State is the data a flow keeps across nodes: what the assistant has gathered so far, and anything else the handlers need to share. It is safe for concurrent use.
type FunctionActionFrame ¶ added in v0.1.0
type FunctionActionFrame struct {
frames.BaseControlFrame
// Action is the action that named the handler.
Action ActionConfig
// Function is the handler to run.
Function ActionHandler
}
FunctionActionFrame carries a function action to run once it reaches the end of the pipeline. Queueing it rather than running the handler on the spot is what makes the handler happen at the right moment: after the speech queued ahead of it has been said, rather than while it is still being synthesized.
func NewFunctionActionFrame ¶ added in v0.1.0
func NewFunctionActionFrame(action ActionConfig, fn ActionHandler) *FunctionActionFrame
NewFunctionActionFrame builds a FunctionActionFrame.
type Handler ¶
type Handler func(ctx context.Context, args json.RawMessage, fm *FlowManager) (string, *NodeConfig, error)
Handler runs when the model calls a node function.
args carries the raw JSON arguments the model produced, and fm is the manager driving the flow. It returns the result to feed back to the model as the tool result and, optionally, the next node to move to: a non-nil next transitions the flow, a nil next leaves it on the current node and answers from the result, and NoResponse leaves it there without the assistant being asked to say anything.
Returning an empty result marks the function as transition-only: the manager substitutes an acknowledgement, since the model called the function and is owed an answer whether or not the function had one of its own.
type Inferencer ¶ added in v0.1.0
type Inferencer interface {
RunInference(ctx context.Context, convo *frames.LLMContext, opts llm.InferenceOptions) (string, error)
}
Inferencer answers a conversation once, off to the side of the pipeline. It is what the reset-with-summary strategy summarizes through; the concrete LLM services satisfy it.
type NodeConfig ¶
type NodeConfig struct {
// Name labels the node in logs. It is optional; an unnamed node is given a
// generated name so two of them are told apart.
Name string
// RoleMessage is the assistant's persona. It is sent as the LLM service's
// system instruction on entry and, being sticky, persists across later
// transitions until another node sets its own. Leave it empty to keep the
// current persona.
RoleMessage string
// TaskMessages state the assistant's objective at this node. They are added to
// the conversation on entry, as the node's context strategy says.
//
// It is required: a nil slice is a node that never said what it is for, and is
// rejected. An empty non-nil slice is a node that deliberately adds nothing,
// which is allowed.
TaskMessages []frames.Message
// Functions are the tools available at this node. A function whose handler
// returns a next node is an edge that transitions the flow; one that returns
// no next node only gathers data.
Functions []NodeFunction
// PreActions run before the node's context is applied, so what they say is
// spoken ahead of anything the node generates.
PreActions []ActionConfig
// PostActions run after the node's context is applied. On a node that speaks
// on entry they run immediately; on one that waits for the user they are held
// back until the assistant's first turn at this node is over.
PostActions []ActionConfig
// ContextStrategy overrides the manager's strategy for this node. Nil uses the
// manager's.
ContextStrategy *ContextStrategyConfig
// RespondImmediately controls whether the assistant generates a response as
// soon as the node is entered. It defaults to true; point it at false for a
// node that should wait for the user to speak first, such as the opening node
// of a call the user initiates.
RespondImmediately *bool
}
NodeConfig defines one state of a conversation: the assistant's task at this point, the tools it may call, what runs on the way in and out, and whether it speaks on entry.
type NodeFunction ¶
type NodeFunction struct {
// Name is the tool name the model calls. It must be unique within a node and
// is required.
Name string
// Description tells the model when to call the tool.
Description string
// Properties describes the tool's arguments, as the properties of a JSON
// Schema object: a map of argument name to its schema.
Properties map[string]any
// Required names the arguments the model must supply.
Required []string
// Handler runs the call and may return the next node. It is required.
Handler Handler
// CancelOnInterruption sets whether a call to this tool is canceled when the
// user interrupts. Nil leaves the flow default, which does not cancel: a flow
// function is usually doing work the conversation still needs whether or not
// the user spoke over it.
CancelOnInterruption *bool
// TimeoutSecs bounds how long a call to this tool may take, overriding the
// service-wide bound. Nil leaves the service-wide one.
TimeoutSecs *float64
}
NodeFunction is a tool offered to the model at a node. It pairs the schema the model sees with the handler that runs when the model calls it, and with the options the call runs under.
type State ¶ added in v0.1.0
type State struct {
// contains filtered or unexported fields
}
State reads and writes a flow's shared data.
type Watcher ¶ added in v0.1.0
type Watcher interface {
// Events is where the handler for the frames reaching the end of the
// pipeline is attached.
Events() *events.Registry
SetReachedDownstreamFilter(f pipeline.FrameFilter)
}
Watcher reports frames that reach the end of the pipeline. *pipeline.Worker satisfies it. The action manager uses it to learn when its actions have finished and when the assistant's turn is over, and says which frames it wants to hear about.