Documentation
¶
Overview ¶
Package agentcore runs AI agents: a model that calls tools, turn by turn, until its task is done. It is built on litellm, whose messages, blocks and events it uses as they are, and leaves policy — which model, which tools, when to stop, how to render — to the application.
Run is the loop: given a Config and a history, it calls the model and the tools it asks for, and returns the history it ended with. It holds no state; the caller owns the history. Every lifecycle signal — streamed content, tool calls, retries, compactions — reaches Config.Emit as an Event, in order, and every message entering the history passes it as a MessageEnd first, so an application stores messages there durably.
Agent keeps a history for an application and runs it: prompting, steering a run under way, queuing follow-ups, and subscribing to the events. A run, of either, ends when its context is cancelled.
The model is a litellm Client (Model); a tool is a Tool. A long history is compacted by a Compactor, such as the summarizing one in agentcore/compact. Packages tools, subagent and task hold coding tools, sub-agents and background tasks, built on the same few types.
Index ¶
- Constants
- Variables
- func Estimate(cfg Config, history []Message) int
- func ReportProgress(ctx context.Context, progress any)
- func SummaryText(m Message) string
- func WithProgress(ctx context.Context, fn func(progress any)) context.Context
- type Agent
- func (a *Agent) ClearQueues() (steering, followUp []Message)
- func (a *Agent) Compact(ctx context.Context) error
- func (a *Agent) Config() Config
- func (a *Agent) FollowUp(msgs ...Message)
- func (a *Agent) Messages() []Message
- func (a *Agent) Prompt(ctx context.Context, prompts ...Message) error
- func (a *Agent) SetConfig(cfg Config)
- func (a *Agent) SetMessages(history []Message) error
- func (a *Agent) Steer(msgs ...Message)
- func (a *Agent) Subscribe(fn func(Event) error) (unsubscribe func())
- type Call
- type Compaction
- type CompactionEnd
- type CompactionStart
- type Compactor
- type Config
- type EndReason
- type Event
- type Message
- func LastResponse(history []Message) Message
- func Run(ctx context.Context, cfg Config, history []Message, prompts ...Message) ([]Message, error)
- func SummaryMessage(summary string) Message
- func ToolResult(id string, result Result) Message
- func User(blocks ...litellm.Block) Message
- func UserText(text string) Message
- type MessageDelta
- type MessageEnd
- type MessageStart
- type Model
- type Result
- type Retry
- type RunEnd
- type StopInfo
- type StopReason
- type Tool
- type ToolCall
- type ToolEnd
- type ToolFunc
- type ToolMiddleware
- type ToolStart
- type ToolUpdate
- type TurnEnd
- type Usage
Constants ¶
const KindResume = "resume"
KindResume marks the prompt the loop adds after a response was cut off at the output token limit, to have the model resume it.
const KindSummary = "summary"
KindSummary marks the summary a compaction put in place of the history it replaced.
Variables ¶
var ErrBusy = errors.New("agent is running")
ErrBusy refuses to start a run while an Agent runs one.
var ErrMaxTurns = errors.New("max turns reached")
ErrMaxTurns ends a run that reached its MaxTurns.
var ErrNoCompactor = errors.New("agentcore: no compactor configured")
ErrNoCompactor refuses to compact with a Config that has no Compactor.
var ErrNothingToContinue = errors.New("nothing to continue: the history ends with a response")
ErrNothingToContinue refuses a run without prompts when the model has nothing to answer, as when the history ends with its response.
Functions ¶
func Estimate ¶ added in v1.8.5
Estimate approximates the input tokens of the next call cfg makes for history. It counts from the usage of the last response that measured the history as it now stands, adding estimates for the messages since, and estimates all of it, system prompt and tools included, when no response did, as right after a compaction.
func ReportProgress ¶ added in v1.8.5
ReportProgress reports the progress of the tool call ctx runs, such as its output so far; the run emits it as a ToolUpdate. What progress is, each tool documents.
func SummaryText ¶ added in v1.8.5
SummaryText returns the summary a SummaryMessage holds.
Types ¶
type Agent ¶
type Agent struct {
// contains filtered or unexported fields
}
Agent is a conversation an application drives: it keeps the history and runs it, one run at a time, taking the messages that arrive meanwhile. It replaces the Emit, Steering and FollowUp of its Config: events go to its subscribers. A run ends when the context it was started with is cancelled. An Agent is safe for concurrent use.
func NewAgent ¶
NewAgent returns an Agent that runs history with cfg. Its runs replace cfg's Emit, Steering and FollowUp with the Agent's: subscribers, Steer and FollowUp.
func (*Agent) ClearQueues ¶ added in v1.8.5
ClearQueues takes back the messages queued by Steer and FollowUp that no run delivered yet, such as to drop them when the user aborts, or to start a run for those that arrived as one ended.
func (*Agent) Compact ¶ added in v1.8.5
Compact has the Config's Compactor rewrite the history now, as a run does when the history outgrows its CompactAt. Subscribers receive its CompactionStart and CompactionEnd. It fails with ErrBusy while a run is under way, and with ErrNoCompactor when the Config has none.
func (*Agent) FollowUp ¶
FollowUp delivers msgs once the run under way, or else the next, would stop.
func (*Agent) Prompt ¶
Prompt runs the history with prompts added, and returns when the run ends, with its error. Without prompts, it answers the history as it is, such as after a run failed; see Run for when there is nothing to answer. It fails with ErrBusy while a run is under way; use Steer or FollowUp then.
func (*Agent) SetConfig ¶ added in v1.8.5
SetConfig replaces the Agent's Config, from the next run on.
func (*Agent) SetMessages ¶
SetMessages replaces the history, such as to start over; it fails with ErrBusy while a run is under way.
func (*Agent) Steer ¶
Steer delivers msgs before the next model call of the run under way, or of the next run.
func (*Agent) Subscribe ¶
Subscribe has fn receive the events of the Agent's runs, after those of earlier subscribers, until unsubscribe is called. An error fn returns stops the run, as one Emit returns does, and later subscribers do not receive the event: storing messages on MessageEnd keeps them out of the history when that fails. Every subscriber receives the RunEnd.
type Call ¶ added in v1.8.5
Call is a model call exactly as the loop makes it. A side call that extends its request, such as for a summary, reads the loop's prompt cache.
type Compaction ¶ added in v1.8.5
type Compaction struct {
// Messages is the history that replaces it.
Messages []Message
// Replaced is how many messages of the history the rewrite stands in for.
Replaced int
// Usage is what the model calls of the rewrite used, if any; a run
// prices it as it does responses.
Usage *Usage
}
Compaction is a rewrite of a history.
type CompactionEnd ¶ added in v1.8.5
type CompactionEnd struct {
Compaction *Compaction
Err error
}
CompactionEnd ends a compaction: with the Compaction that replaces the history, nil when there was nothing to compact, or with Err. Returning an error from Emit for a Compaction keeps the history as it was and stops the run, so an application stores compactions here durably.
type CompactionStart ¶ added in v1.8.5
type CompactionStart struct{}
CompactionStart begins a compaction of the history.
type Compactor ¶ added in v1.8.5
type Compactor interface {
// Compact rewrites history. call builds the call the loop would make for
// a history, as BuildCall does with the run's Config: a compactor that
// summarizes with the conversation's model can extend the call for the
// part of the history it replaces, which earlier calls sent, and read it
// from the prompt cache. A nil Compaction means there is nothing to
// compact.
Compact(ctx context.Context, history []Message, call func([]Message) Call) (*Compaction, error)
}
Compactor rewrites a history into a shorter one that carries the same work forward. The loop decides when to call it (Config.CompactAt, and a context overflow) and records the result; an application may also call it itself, such as for a manual compaction.
The deferred tools a history loaded stay offered only while a tool reference in it names them (see Tool.Deferred): once a compaction replaced the results holding the references, the model searches for them again.
type Config ¶ added in v1.8.5
type Config struct {
Model Model
// System is the system prompt: text blocks, which may carry cache
// breakpoints.
System []litellm.Block
Tools []Tool
// Emit receives the run's events, one at a time, in order. An error it
// returns stops the run: the tool calls under way are cancelled, no
// further event is delivered but the RunEnd carrying the error, and the
// event refused takes no effect; see MessageEnd and CompactionEnd. Nil
// drops the events.
Emit func(Event) error
// Steering returns the messages to deliver before the next model call,
// such as what a user typed while the run worked. It is called at the
// start, after each turn, and after each compaction in the run, which
// can take long.
Steering func() []Message
// FollowUp returns the messages to go on with when the run would stop.
FollowUp func() []Message
// OnStop is consulted when the run would stop and neither Steering nor
// FollowUp has messages: after a response that calls no tool, or a turn
// a tool Terminated. It returns messages to go on with, none to stop, or
// an error to end the run with.
//
// Messages Steering, FollowUp and OnStop return are recorded at once,
// even when the run then ends, cancelled or at MaxTurns, before the
// model answers them. None is consulted once the run is cancelled.
OnStop func(ctx context.Context, s StopInfo) ([]Message, error)
// Middleware wraps every tool call, the first outermost.
Middleware []ToolMiddleware
// MaxToolConcurrency bounds the parallel tool calls running at once;
// below 2, calls run one by one.
MaxToolConcurrency int
// MaxToolErrors disables a tool, for the rest of the run, once its calls
// failed that many turns in a row; 0 never does. A refused call, or one
// of an unknown tool, does not count.
MaxToolErrors int
// Compactor rewrites the history when it grows past CompactAt estimated
// tokens, before the call that would send it, and when the provider
// reports a context overflow, after which the call is made again. A
// compaction at CompactAt that fails is reported and the call made
// anyway; one on overflow must succeed. Zero CompactAt compacts on
// overflow only. CompactAt should sit well above what a compaction
// keeps, or every call compacts again.
Compactor Compactor
CompactAt int
// Cache, if set, places two cache breakpoints: after each call's last
// message, and after the last message of the call before, so a call
// reads the history that call sent from the prompt cache however many
// blocks the turn between them added. With the system prompt's, they
// must stay within the provider's limit, 4 on Anthropic; breakpoints of
// a longer TTL must come before those of a shorter one.
Cache *litellm.CacheControl
// MaxTurns bounds the responses of a run; 0 means 100.
MaxTurns int
// Retry paces the model calls made again after failing for a while,
// such as on a rate limit or a dropped connection: a response, or a
// compaction, makes at most Retry.MaxAttempts calls. The zero Policy
// makes none again.
Retry retry.Policy
}
Config configures a run.
func (Config) Validate ¶ added in v1.8.5
Validate reports a Config a run cannot use: a Pricing that does not fit, a Model.Request already holding Messages or Tools, which a run sets, or a tool without a name, with another's name, or with a schema that is not JSON. Run and the runs of an Agent validate it themselves.
type Event ¶
type Event interface {
// contains filtered or unexported methods
}
Event is something that happened in a run, delivered to Config.Emit. The events are the types below; switch on them by type.
A run streams each response as MessageStart, then MessageDelta events, and records it with MessageEnd, which every message entering the history passes through. Events hold no state that changes later: a MessageEnd's message is final, and the streamed content is in the deltas only.
type Message ¶
type Message struct {
Role litellm.Role `json:"role"`
Blocks []litellm.Block `json:"blocks"`
// Kind marks a message the loop or the application added for its own
// purposes, such as KindSummary. Models see it as any message of its
// role.
Kind string `json:"kind,omitempty"`
// Stop, Usage, Provider and Model describe an assistant message: why the
// response ended, what it used, and what produced it.
Stop StopReason `json:"stop,omitempty"`
Usage *Usage `json:"usage,omitempty"`
Provider string `json:"provider,omitempty"`
Model string `json:"model,omitempty"`
// Time is when the message entered the history.
Time time.Time `json:"time"`
}
Message is one entry of a conversation: a model message, in litellm's roles and blocks, with what the loop knows of it. A tool result is a message of role litellm.RoleTool holding one litellm.ToolResultBlock. Messages encode as JSON, so a history is stored as it is.
func LastResponse ¶ added in v1.8.5
LastResponse returns the last response in history, the zero Message when there is none.
func Run ¶ added in v1.8.5
Run runs cfg's agent on history: it records prompts, then calls the model and the tools it asks for, turn by turn, until the model stops, ctx is cancelled or an error ends the run. It returns the history the run ended with, compactions included, and the error that ended it: ctx's error when cancelled. Every message entering the history passes Emit as a MessageEnd first, and the last event is always a RunEnd.
Without prompts, Run answers the history as it stands; it fails with ErrNothingToContinue when the model has nothing to answer, as when the history ends with a response.
func SummaryMessage ¶ added in v1.8.5
SummaryMessage returns the message that stands in for history a compaction replaced with summary, telling the model to carry on from it. Models read it as a user message.
func ToolResult ¶
ToolResult returns the message that answers the tool call id with result.
func (Message) Text ¶ added in v1.8.5
Text returns the text of m's text blocks, or of the tool result it holds.
func (Message) ToolCalls ¶
func (m Message) ToolCalls() []litellm.ToolUseBlock
ToolCalls returns the tool calls m makes.
func (Message) ToolResult ¶ added in v1.8.5
func (m Message) ToolResult() (litellm.ToolResultBlock, bool)
ToolResult returns the tool result m holds.
func (*Message) UnmarshalJSON ¶ added in v1.8.5
type MessageDelta ¶ added in v1.8.5
MessageDelta is a streamed event of the response under way: a block started, text or arguments added, a block ended, usage. A Retry discards the response so far.
type MessageEnd ¶ added in v1.8.5
type MessageEnd struct{ Message Message }
MessageEnd records a message entering the history: a prompt, a response, a tool result. Returning an error from Emit for it keeps it out and stops the run, so an application stores messages here durably.
type MessageStart ¶ added in v1.8.5
type MessageStart struct{}
MessageStart begins a response; MessageDelta events stream its content.
type Model ¶ added in v1.8.5
type Model struct {
Client *litellm.Client
// Request is the template of every call: the model's name and settings,
// such as MaxTokens, Thinking and ProviderOptions. The loop sets its
// Messages and Tools.
Request litellm.Request
// Pricing, if set, prices each response's usage.
Pricing *catalog.Pricing
}
Model is the model a run calls.
type Result ¶ added in v1.8.5
type Result struct {
// Content is what the model reads: text, images and tool references.
Content []litellm.Block
IsError bool
// Terminate ends the run once the turn's results are recorded, as a tool
// that completes the task does. The run's OnStop may keep it going.
Terminate bool
}
Result is what a tool call returns to the model.
func ErrorResult ¶ added in v1.8.5
ErrorResult returns text as a failed result.
func JSONResult ¶ added in v1.8.5
JSONResult returns v, as JSON text, as a result. Characters such as < and & stay as they are, as the model reads them.
func TextResult ¶ added in v1.8.5
TextResult returns text as a result.
type Retry ¶ added in v1.8.5
Retry reports a model call that failed with Err, which the run makes again after Delay; one that overflowed the context, after compacting. Attempt numbers the call to come, the first made again being 2.
type RunEnd ¶ added in v1.8.5
type RunEnd struct {
Reason EndReason
Err error
// Turns, ToolCalls and FailedCalls count the run's responses, the tool
// calls they made and the calls whose result is an error, refused calls
// included, which Config.MaxToolErrors does not count.
Turns, ToolCalls, FailedCalls int
}
RunEnd ends a run, with the error that ended it. It is always the run's last event, delivered even after Emit failed.
type StopInfo ¶ added in v1.6.4
type StopInfo struct {
// Message is the last response.
Message Message
// Terminated reports that a tool of the last turn terminated the run.
Terminated bool
// Turns is how many responses the run had.
Turns int
}
StopInfo is what OnStop decides on.
type StopReason ¶
type StopReason string
StopReason is why a response ended.
const ( // StopEnd is a response the model ended. StopEnd StopReason = "end" // StopToolUse is a response that calls tools. StopToolUse StopReason = "tool_use" // StopLength is a response cut off at the output token limit. StopLength StopReason = "length" // StopSafety is a response the vendor's safety system ended or refused. StopSafety StopReason = "safety" // StopError is a response the vendor ended with an error. StopError StopReason = "error" // StopAborted is a response the run was cancelled during; it holds what // streamed before. StopAborted StopReason = "aborted" // StopOther is a reason the vendor gave that has no equivalent here. StopOther StopReason = "other" )
type Tool ¶
type Tool struct {
Name string
Description string
// Schema is the JSON Schema of the arguments. Calls are validated
// against it before they run, and the model sees what does not fit.
Schema map[string]any
// Label names the tool for people, such as "Edit File".
Label string
// Deferred tools are offered to the model only once a
// litellm.ToolReferenceBlock in the history names them, as a tool search
// returns; until then they cost no context. See litellm.Tool.
Deferred bool
// Parallel lets the tool's calls run alongside the other parallel calls
// of their turn; any other call runs alone.
Parallel bool
// Check, if set, vets a call before it is approved and run, such as that
// a file was read before it is edited, and may preview for people what
// the call will do, such as a diff to approve. An error fails the call.
Check func(ctx context.Context, args json.RawMessage) (preview string, err error)
// Run makes a call. An error fails it, and the model reads the error.
Run func(ctx context.Context, args json.RawMessage) (Result, error)
}
Tool is a function the model may call.
type ToolCall ¶
type ToolCall struct {
ID string
Name string
Args json.RawMessage
// Tool is the tool called; nil when the model called an unknown one.
Tool *Tool
// Preview is what the tool's Check previewed.
Preview string
}
ToolCall is a call of a tool, as the loop runs it.
type ToolMiddleware ¶ added in v1.5.1
ToolMiddleware wraps the running of each tool call, once its arguments are valid and its tool's Check passed: for approval, auditing or rewriting the arguments. It calls next to go on; returning without calling next, such as with an ErrorResult, refuses the call.
type ToolStart ¶ added in v1.8.5
type ToolStart struct{ Call ToolCall }
ToolStart begins a tool call, once its arguments were checked and before the middleware, such as an approval, runs it.
type ToolUpdate ¶ added in v1.8.5
ToolUpdate is the progress a running tool reported, see ReportProgress; it comes between the call's ToolStart and ToolEnd.
type TurnEnd ¶ added in v1.8.5
TurnEnd ends a turn: a response and the results of the tools it called, all recorded.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package compact holds Summarizer, the agentcore.Compactor that replaces older history with a structured checkpoint summary.
|
Package compact holds Summarizer, the agentcore.Compactor that replaces older history with a structured checkpoint summary. |
|
examples
|
|
|
multi
command
Multi runs an agent that delegates to sub-agents, a scout and a reviewer:
|
Multi runs an agent that delegates to sub-agents, a scout and a reviewer: |
|
single
command
Single runs one agent with the coding tools on a task:
|
Single runs one agent with the coding tools on a task: |
|
Package schema builds the JSON Schema of a tool's arguments, as the map[string]any agentcore's Tool.Schema holds.
|
Package schema builds the JSON Schema of a tool's arguments, as the map[string]any agentcore's Tool.Schema holds. |
|
Package subagent lets a model delegate tasks to sub-agents, which work on them in contexts of their own and report back.
|
Package subagent lets a model delegate tasks to sub-agents, which work on them in contexts of their own and report back. |
|
Package task runs work in the background as tasks the model can follow: each has an ID, a status and an output file, and a message announces it when it ends.
|
Package task runs work in the background as tasks the model can follow: each has an ID, a status and an output file, and a message announces it when it ends. |
|
Package tools holds the coding tools of an agent.
|
Package tools holds the coding tools of an agent. |