Documentation
¶
Overview ¶
Package steerloop is a controllable agent loop for CloudWeGo Eino.
A Loop calls a tool-calling chat model, executes the tools it asks for, feeds the results back and repeats until a stop condition is met. While it runs, the caller can:
- read a typed stream of events (model turns, tool calls, tool results);
- stop it, inject a user message, or change its configuration;
- hand named tool calls to a pause function, for example to ask a human.
The loop also retries transient model failures, aborts a streamed turn that stops producing chunks, and emits an audit record of every model turn.
A minimal run:
loop, err := steerloop.New(steerloop.DefaultConfig(), chatModel, tools, executor, nil)
if err != nil {
return err
}
for evt := range loop.Start(ctx, messages) {
if done, ok := evt.Data.(steerloop.DoneData); ok {
fmt.Println(done.Reason, done.FinalContent)
}
}
The model is any Eino ToolCallingChatModel. Tools are described with Eino ToolInfo values and executed through the ToolExecutor the caller supplies.
Index ¶
- Constants
- Variables
- func IsTransientLLMError(err error) bool
- func NewToolInfo(name, desc string, params map[string]any) (*schema.ToolInfo, error)
- func SerializeAssistantToolCalls(msg *schema.Message) []byte
- func SerializeEinoMessagesForAudit(msgs []*schema.Message) (raw []byte, msgCount, charCount int32)
- func StopConditionLabel(sc StopCondition) string
- type ActiveLoop
- type AwaitingInputData
- type Control
- type ControlType
- type DoneData
- type DoneReason
- type ErrorData
- type Event
- type EventSink
- type EventType
- type InjectData
- type LLMTurnData
- type LLMUsageSummary
- type Loop
- func (l *Loop) Events() <-chan Event
- func (l *Loop) GetCurrentIteration() int
- func (l *Loop) GetRunningMessages() []*schema.Message
- func (l *Loop) GetState() LoopState
- func (l *Loop) Inject(message string) bool
- func (l *Loop) Result() (DoneData, bool)
- func (l *Loop) SendControlSignal(ctrl Control) bool
- func (l *Loop) Start(ctx context.Context, startingMessages []*schema.Message) <-chan Event
- func (l *Loop) Stop()
- func (l *Loop) Wait()
- type LoopConfig
- type LoopState
- type ModifyConfigData
- type PauseFunc
- type PauseIn
- type ProgressData
- type ReflectionData
- type Registry
- func (r *Registry[K]) Get(key K) (*ActiveLoop, bool)
- func (r *Registry[K]) Has(key K) bool
- func (r *Registry[K]) Inject(key K, message string) error
- func (r *Registry[K]) Register(key K, loop *Loop, cancel context.CancelFunc)
- func (r *Registry[K]) RunningMessages(key K) ([]*schema.Message, error)
- func (r *Registry[K]) Stop(key K) error
- func (r *Registry[K]) Unregister(key K)
- type ResponseData
- type StopCondition
- type ThinkingData
- type ToolCallData
- type ToolExecutor
- type ToolExecutorFunc
- type ToolResultData
Constants ¶
const FinishToolName = "finish"
FinishToolName is the reserved tool name that ends a StopOnFinishTool run. Its arguments become DoneData.FinalContent. It never reaches the ToolExecutor.
Variables ¶
var ErrControlQueueFull = errors.New("steerloop: control queue full")
ErrControlQueueFull is returned when a control could not be queued.
var ErrNoActiveLoop = errors.New("steerloop: no active loop")
ErrNoActiveLoop is returned by a Registry when the key has no running loop.
Functions ¶
func IsTransientLLMError ¶
IsTransientLLMError is the default retry classifier. It reports true for a step deadline, EOF, an empty reply, an idle stream and a reset connection.
func NewToolInfo ¶
NewToolInfo builds an Eino tool definition from a name, a description and a JSON Schema object for the parameters. A nil schema means no parameters.
func SerializeAssistantToolCalls ¶
SerializeAssistantToolCalls JSON-encodes the tool calls of an assistant message, or "[]" when there are none.
func SerializeEinoMessagesForAudit ¶
SerializeEinoMessagesForAudit JSON-encodes the messages sent to the model as {"messages":[...]}. When the result would exceed 256 KiB the oldest messages are dropped and "truncated" is set. It returns the JSON, the number of messages kept and the character count of all messages.
func StopConditionLabel ¶
func StopConditionLabel(sc StopCondition) string
StopConditionLabel returns the stable name used in llm_turn events.
Types ¶
type ActiveLoop ¶
type ActiveLoop struct {
Loop *Loop
Cancel context.CancelFunc
}
ActiveLoop is a running Loop with the function that cancels its context.
type AwaitingInputData ¶
type AwaitingInputData struct {
Question string `json:"question"`
Options []string `json:"options,omitempty"`
Payload any `json:"payload,omitempty"`
}
AwaitingInputData is the payload a pause function emits with EventAwaitingInput. Payload carries anything else the caller's UI needs.
type Control ¶
type Control struct {
Type ControlType
Data any
}
Control is a signal sent to a running loop with SendControlSignal.
func NewInjectControl ¶
NewInjectControl returns an inject signal carrying message.
func NewModifyConfigControl ¶
func NewModifyConfigControl(data ModifyConfigData) Control
NewModifyConfigControl returns a configuration change signal.
type ControlType ¶
type ControlType string
ControlType classifies a control signal.
const ( // ControlStop stops the loop at the next iteration boundary. ControlStop ControlType = "stop" // ControlInject appends a user message that the model sees on its next // call. Inside a pause function's Block, it is the reply being waited for. ControlInject ControlType = "inject" // ControlModifyConfig changes configuration during the run. ControlModifyConfig ControlType = "modify_config" )
type DoneData ¶
type DoneData struct {
Reason DoneReason `json:"reason"`
Iterations int `json:"iterations"`
FinalContent string `json:"final_content,omitempty"`
// Error is the fatal error message when Reason is DoneReasonError.
Error string `json:"error,omitempty"`
// Awaiting is set when Reason is DoneReasonAwaitingHuman.
Awaiting *AwaitingInputData `json:"awaiting,omitempty"`
// DroppedEvents counts intermediate events dropped because the event
// channel was full.
DroppedEvents int `json:"dropped_events,omitempty"`
LLMUsage LLMUsageSummary `json:"llm_usage"`
}
DoneData is the payload of EventDone, emitted exactly once per run.
type DoneReason ¶
type DoneReason string
DoneReason says why the loop ended.
const ( // DoneReasonFinished is normal completion. DoneReasonFinished DoneReason = "finished" // DoneReasonStopped means a stop signal or a cancelled wait ended the run. DoneReasonStopped DoneReason = "stopped" // DoneReasonMaxIterations means the iteration cap was reached. DoneReasonMaxIterations DoneReason = "max_iterations" // DoneReasonTimeout means TotalTimeout passed or the context was cancelled. DoneReasonTimeout DoneReason = "timeout" // DoneReasonError means a fatal error ended the run. DoneReasonError DoneReason = "error" // DoneReasonAwaitingHuman means a pause function ended the run to wait for // a person; the caller resumes later with a new loop. DoneReasonAwaitingHuman DoneReason = "awaiting_human" )
type Event ¶
type Event struct {
Type EventType `json:"type"`
Data any `json:"data"`
Iteration int `json:"iteration"`
Timestamp time.Time `json:"timestamp"`
}
Event is one update from a running loop.
type EventSink ¶
EventSink receives every event the loop emits, synchronously, before the event is offered to the event channel. It must not block for long.
type EventType ¶
type EventType string
EventType classifies an event.
const ( // EventThinking is emitted before each model call. EventThinking EventType = "thinking" // EventToolCall is emitted when the model asks for a tool, before it runs. EventToolCall EventType = "tool_call" // EventToolResult is emitted when a tool finishes, with its result or error. EventToolResult EventType = "tool_result" // EventResponse is emitted when the model replies with text and no tool call. EventResponse EventType = "response" // EventReflection is emitted for the optional self-review reply. EventReflection EventType = "reflection" // EventProgress is a human-readable status message. EventProgress EventType = "progress" // EventError reports a failure. ErrorData.Fatal says whether the loop stopped. EventError EventType = "error" // EventDone is the last event. DoneData.Reason says why the loop ended. EventDone EventType = "done" // EventLLMTurn is the audit record of one model call that led to tool calls. EventLLMTurn EventType = "llm_turn" // EventAwaitingInput is emitted by a pause function that is waiting for a // reply. The loop has not ended. EventAwaitingInput EventType = "awaiting_input" )
type InjectData ¶
type InjectData struct {
Message string
}
InjectData is the payload of ControlInject.
type LLMTurnData ¶
type LLMTurnData struct {
TurnIndex int `json:"turn_index"`
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
CostAmount float64 `json:"cost_amount"`
Provider string `json:"provider,omitempty"`
Model string `json:"model,omitempty"`
DurationMs int64 `json:"duration_ms"`
RequestMessagesJSON []byte `json:"request_messages_json,omitempty"`
ResponseKind string `json:"response_kind"`
AssistantContent string `json:"assistant_content,omitempty"`
AssistantToolCallsJSON []byte `json:"assistant_tool_calls_json,omitempty"`
MaxIterations int `json:"max_iterations,omitempty"`
StopCondition string `json:"stop_condition,omitempty"`
}
LLMTurnData is the payload of EventLLMTurn.
type LLMUsageSummary ¶
type LLMUsageSummary struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
TotalTokens int `json:"total_tokens"`
Model string `json:"model,omitempty"`
}
LLMUsageSummary totals token use over a run. TotalTokens is input plus output, plus any totals a provider reported without a split.
type Loop ¶
type Loop struct {
// contains filtered or unexported fields
}
Loop runs one agent conversation: call the model, execute the tools it asks for, feed the results back, repeat. A Loop is started once and is safe for concurrent use.
func New ¶
func New(cfg LoopConfig, llm model.ToolCallingChatModel, tools []*schema.ToolInfo, executor ToolExecutor, logger *slog.Logger) (*Loop, error)
New creates a Loop. llm is any Eino tool-calling chat model; when tools is not empty the model is bound to them with WithTools. executor runs the tools. A nil logger discards log output.
func (*Loop) GetCurrentIteration ¶
GetCurrentIteration returns the number of model calls started so far.
func (*Loop) GetRunningMessages ¶
GetRunningMessages returns a copy of the conversation as it stands now.
func (*Loop) Inject ¶
Inject queues a user message. It returns false when the control queue is full.
func (*Loop) Result ¶
Result returns the outcome of the run. ok is false until the loop has ended. It is the way to learn the outcome without reading the event channel.
func (*Loop) SendControlSignal ¶
SendControlSignal queues a control for the loop. It returns false when the control queue is full. Use Stop to stop a loop: it cannot be lost.
func (*Loop) Start ¶
Start begins the run with the given messages and returns the event channel, which is closed after the done event. The loop keeps its own copy of the slice, so the caller's slice is never appended to. A second call returns a channel holding one fatal error.
type LoopConfig ¶
type LoopConfig struct {
// MaxIterations caps the number of model calls. The loop never exceeds it.
MaxIterations int
// StopCondition decides when the loop ends on its own.
StopCondition StopCondition
// Temperature is passed to the model on every call.
Temperature float64
// StepTimeout bounds one model call. Zero means no per-call bound.
StepTimeout time.Duration
// TotalTimeout bounds the whole run. Zero means no bound.
TotalTimeout time.Duration
// EnableReflection adds one self-review call after the final reply in
// StopOnNoToolCalls mode.
EnableReflection bool
// ReflectionPrompt is the instruction for that call. The loop appends the
// reply under review. Empty means a built-in neutral prompt.
ReflectionPrompt string
// StreamEvents turns on intermediate events. When false only error and
// done events are emitted.
StreamEvents bool
// EventBuffer is the capacity of the event channel. Zero means 64.
// Intermediate events are dropped, and counted, when it is full.
EventBuffer int
// EventSink, when set, receives every emitted event.
EventSink EventSink
// DoneDeliveryGrace is how long the loop waits for a reader to take the
// done event when the event channel is full. Zero means 5 seconds.
DoneDeliveryGrace time.Duration
// Pauses maps a tool name to a pause function. A call to such a tool is
// handed to the function instead of the ToolExecutor.
Pauses map[string]PauseFunc
// AutoNudgeText is the user message appended when the model replies with
// text only in StopOnFinishTool mode. Empty means a built-in nudge.
AutoNudgeText string
// DisableAutoNudge turns that message off.
DisableAutoNudge bool
// RetryMaxAttempts is the number of extra attempts after a transient model
// failure. Zero means 1; -1 disables retries.
RetryMaxAttempts int
// IsTransient classifies a model error as worth retrying. Nil means the
// built-in classifier (deadline exceeded, EOF, empty reply, idle stream,
// connection reset).
IsTransient func(error) bool
// DisableStreaming makes the loop call Generate instead of Stream.
// Streaming is what lets the loop detect a hung turn early.
DisableStreaming bool
// IdleChunkTimeout is the longest gap allowed between streamed chunks
// after the first one. Zero means 30 seconds.
IdleChunkTimeout time.Duration
// DisableStepTimeoutDecoupling makes each model call inherit the remaining
// TotalTimeout. By default a call gets the full StepTimeout even when less
// of TotalTimeout remains; cancellation of the run still reaches it.
DisableStepTimeoutDecoupling bool
// HistoryCompactor, when set, is applied to the message history before
// each model call. Its result is what the model receives; the stored
// history is not changed.
HistoryCompactor func(msgs []*schema.Message) []*schema.Message
// CallOptions builds the per-call model options. Nil means
// model.WithTemperature(temperature).
CallOptions func(temperature float32) []model.Option
// ModelIdentity names the provider and model for llm_turn events. It is
// called once per turn, so it may change during a run.
ModelIdentity func() (provider, model string)
// CostEstimator prices one turn for llm_turn events. Nil means cost 0.
CostEstimator func(provider, model string, inputTokens, outputTokens int) float64
// TranscriptLogLevel is the level at which the loop logs the messages
// added on each iteration. These lines contain prompt and reply text. The
// zero value is slog.LevelInfo; DefaultConfig sets slog.LevelDebug to keep
// them out of normal logs.
TranscriptLogLevel slog.Level
}
LoopConfig configures one Loop. The zero value is usable only after MaxIterations is set; DefaultConfig returns sensible values.
func DefaultConfig ¶
func DefaultConfig() LoopConfig
DefaultConfig returns a configuration suited to an interactive assistant: up to 12 model calls, stop when the model answers without a tool call, and intermediate events on.
type LoopState ¶
type LoopState string
LoopState is the lifecycle state of a Loop.
const ( // StateIdle is the state before Start. StateIdle LoopState = "idle" // StateRunning is the state between Start and the done event. StateRunning LoopState = "running" // StateDone means the loop ended on its own: finished, max iterations, // timeout, or handed over to a human. StateDone LoopState = "done" // StateStopped means the loop ended because it was told to stop. StateStopped LoopState = "stopped" // StateError means the loop ended on a fatal error. StateError LoopState = "error" )
type ModifyConfigData ¶
ModifyConfigData is the payload of ControlModifyConfig. Nil fields are left unchanged.
type PauseFunc ¶
PauseFunc handles a call to a tool registered in LoopConfig.Pauses. The loop waits for it to return. reply becomes the tool's result. Returning cont false ends the run: with DoneReasonAwaitingHuman if the function emitted EventAwaitingInput, otherwise with DoneReasonStopped.
type PauseIn ¶
type PauseIn struct {
// CallID, Name and Arguments identify the tool call being handled.
CallID string
Name string
Arguments string
// Messages is a copy of the conversation so far.
Messages []*schema.Message
// Tools are the tools the loop was created with.
Tools []*schema.ToolInfo
// Execute runs a tool through the loop's ToolExecutor.
Execute func(ctx context.Context, name, args string) (string, error)
// Call makes a model call with the loop's retry and timeout handling. A
// non-empty tools list binds the model to those tools for this call.
Call func(ctx context.Context, tools []*schema.ToolInfo, msgs []*schema.Message) (*schema.Message, error)
// Emit sends an event on the loop's event channel.
Emit func(event EventType, data any)
// Block waits for the next Inject and returns its message. ok is false
// when the loop was stopped or its context ended while waiting.
Block func(ctx context.Context) (reply string, ok bool)
}
PauseIn is what a pause function can see and do.
type ProgressData ¶
ProgressData is the payload of EventProgress.
type ReflectionData ¶
type ReflectionData struct {
Content string `json:"content"`
}
ReflectionData is the payload of EventReflection.
type Registry ¶
type Registry[K comparable] struct { // contains filtered or unexported fields }
Registry tracks running loops by a key of the caller's choice, such as a user or session identifier, so another goroutine can stop or steer them. The zero value is not usable; call NewRegistry.
func NewRegistry ¶
func NewRegistry[K comparable]() *Registry[K]
NewRegistry returns an empty registry.
func (*Registry[K]) Get ¶
func (r *Registry[K]) Get(key K) (*ActiveLoop, bool)
Get returns the loop under key.
func (*Registry[K]) Inject ¶
Inject sends a user message to the loop under key. It returns ErrNoActiveLoop when there is none and ErrControlQueueFull when the loop's control queue is full.
func (*Registry[K]) Register ¶
func (r *Registry[K]) Register(key K, loop *Loop, cancel context.CancelFunc)
Register records loop under key, replacing any loop already there.
func (*Registry[K]) RunningMessages ¶
RunningMessages returns a copy of the conversation of the loop under key.
func (*Registry[K]) Stop ¶
Stop stops the loop under key. It returns ErrNoActiveLoop when there is none.
func (*Registry[K]) Unregister ¶
func (r *Registry[K]) Unregister(key K)
Unregister removes the loop under key.
type ResponseData ¶
type ResponseData struct {
Content string `json:"content"`
}
ResponseData is the payload of EventResponse.
type StopCondition ¶
type StopCondition int
StopCondition decides when the loop ends on its own.
const ( // StopOnNoToolCalls ends the loop when the model replies without calling // a tool. Suits chat and question answering. StopOnNoToolCalls StopCondition = iota // StopOnFinishTool runs until the model calls the tool named // FinishToolName; its arguments are the final output. Suits work that must // end in a structured result. StopOnFinishTool // StopOnExplicit runs until a stop signal, the iteration cap or a timeout. StopOnExplicit )
type ThinkingData ¶
type ThinkingData struct {
Message string `json:"message"`
}
ThinkingData is the payload of EventThinking.
type ToolCallData ¶
type ToolCallData struct {
ToolCallID string `json:"tool_call_id"`
Name string `json:"name"`
Arguments string `json:"arguments"`
}
ToolCallData is the payload of EventToolCall.
type ToolExecutor ¶
type ToolExecutor interface {
// ExecuteTool runs the named tool with its JSON arguments and returns the
// text result.
ExecuteTool(ctx context.Context, name string, arguments string) (string, error)
}
ToolExecutor runs the tools the model asks for. The loop calls ExecuteTool once per tool call and feeds the result back to the model. An error is not fatal: the model receives it as the tool's result and can react.
type ToolExecutorFunc ¶
ToolExecutorFunc adapts a function to ToolExecutor.
func (ToolExecutorFunc) ExecuteTool ¶
func (f ToolExecutorFunc) ExecuteTool(ctx context.Context, name string, arguments string) (string, error)
ExecuteTool calls f.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
steerloop
command
Command steerloop runs an agent loop from the terminal: a model, tools declared in a manifest, live steering, and a JSONL record of the run.
|
Command steerloop runs an agent loop from the terminal: a model, tools declared in a manifest, live steering, and a JSONL record of the run. |
|
examples
|
|
|
ask_user
command
Ask_user shows a pause tool: when the model calls ask_user, the loop hands the call to a function that waits for a person, and the answer becomes the tool's result.
|
Ask_user shows a pause tool: when the model calls ask_user, the loop hands the call to a function that waits for a person, and the answer becomes the tool's result. |
|
minimal
command
Minimal shows the smallest useful loop: a model, one tool, and the event stream.
|
Minimal shows the smallest useful loop: a model, one tool, and the event stream. |
|
steer
command
Steer shows the three controls on a running loop: inject a message, change the configuration, and stop.
|
Steer shows the three controls on a running loop: inject a message, change the configuration, and stop. |
|
internal
|
|
|
exectool
Package exectool runs command-line tools declared in a YAML manifest.
|
Package exectool runs command-line tools declared in a YAML manifest. |
|
provider
Package provider builds an Eino chat model from a "provider:model" string.
|
Package provider builds an Eino chat model from a "provider:model" string. |
|
Package steerlooptest provides a scripted chat model and a recording tool executor for testing code built on steerloop without calling a provider.
|
Package steerlooptest provides a scripted chat model and a recording tool executor for testing code built on steerloop without calling a provider. |