Documentation
¶
Overview ¶
Package state provides conversation checkpoint storage for agent runs. Checkpoints are written by the model-router sidecar after each LLM response and read on pod restart to enable seamless failure recovery.
Storage is tiered:
- Active runs: Redis (fast, TTL-based expiry)
- Completed/archived: object storage (S3/GCS/Azure Blob) if configured
All values are zstd-compressed before storage to reduce cost.
Index ¶
Constants ¶
const LocalCacheDirEnv = "AGENTORC_WARM_CACHE_DIR"
LocalCacheDirEnv is the env var the model-router reads to activate the warm-pod local disk cache. When non-empty, the operator has mounted a disk-backed emptyDir at this path into the model-router sidecar and the store is wrapped with a localCacheStore.
const LocalCacheFileExt = ".msg.zst"
LocalCacheFileExt is the extension appended to on-disk checkpoint files.
Variables ¶
This section is empty.
Functions ¶
func IsTerminalTraceEventJSON ¶
IsTerminalTraceEventJSON reports whether a trace-event JSON payload (the value of a Redis stream "ev" entry) represents a terminal event — i.e. the OpenAI turn loop has completed and TailTokens should close the UI SSE channel. Exported so the apiserver's external HTTP client and tests reuse the same rule.
func ParseRedisURL ¶ added in v0.8.0
ParseRedisURL wraps redis.ParseURL so the error never echoes the URL back: a *url.Error from net/url embeds the full input, password included.
Types ¶
type Config ¶
type Config struct {
// Backend selects the storage backend: "redis" (default).
// Future backends: "postgres", "s3", "gcs", "azureblob".
Backend string `json:"backend"`
// RedisURL is the Redis connection URL.
// Example: "redis://redis.infra.svc.cluster.local:6379"
RedisURL string `json:"redisUrl,omitempty"`
// RedisPasswordFile, when set, is a file holding the Redis password. It
// overrides any password in RedisURL. The operator mounts it from a Secret
// so the password never appears in the model-router ConfigMap.
RedisPasswordFile string `json:"redisPasswordFile,omitempty"`
// TTLSeconds is the Redis key TTL beyond the run timeout.
// Keys auto-expire, so no manual cleanup is needed.
TTLSeconds int `json:"ttlSeconds,omitempty"`
// CheckpointKey is the Redis key prefix for this run.
// Format: "agentorca/runs/<run-id>/state"
CheckpointKey string `json:"checkpointKey,omitempty"`
// MaxTokenStreamLen is the Redis XADD MAXLEN (APPROX) applied to each run's token
// and trace-event stream ("tokens:<ns>:<run>"). The stream holds one entry per
// streamed token delta + trace event, so long-running/Reasoning-heavy runs can
// exceed the historical 10k cap and silently lose the OLDEST tokens on replay.
// 0 (default) = 100000 (~10x the old floor), enough headroom for very long runs
// while still bounding per-run memory. Tools/agents reading the stream via
// TailTokens keep the newest entries; set higher per-deployment if you run
// extremely long sessions. Not related to output truncation mid-stream.
MaxTokenStreamLen int `json:"maxTokenStreamLen,omitempty"`
}
Config holds connection parameters for the state store, sourced from Helm values and injected into the model-router config file.
func (Config) WithPasswordFile ¶ added in v0.8.0
WithPasswordFile moves the Redis password out of the config. It returns a copy of c whose RedisURL has the password removed and whose RedisPasswordFile is set to passwordFile, together with the password itself. When RedisURL carries no password, c is returned unchanged with an empty password.
The operator uses this so the password reaches the model-router through a mounted Secret instead of the plain-text router ConfigMap.
type Store ¶
type Store interface {
// SaveMessages persists the full conversation history.
// The implementation may write incrementally (delta) or as a full snapshot.
SaveMessages(ctx context.Context, key string, messages []json.RawMessage, ttl time.Duration) error
// LoadMessages retrieves the full conversation history from a checkpoint key.
LoadMessages(ctx context.Context, key string) ([]json.RawMessage, error)
// SaveSpend persists the cumulative USD spend for a run.
// Called after every LLM response so the value survives pod crashes.
SaveSpend(ctx context.Context, key string, usd float64, ttl time.Duration) error
// LoadSpend retrieves the cumulative USD spend for a run.
// Returns 0 if no spend has been recorded.
LoadSpend(ctx context.Context, key string) (float64, error)
// SaveToken appends a streaming token to a Redis Stream for real-time UI
// streaming. Completion is signalled by a terminal trace event (see
// IsTerminalTraceEventJSON and TailTokens), not by an empty token, so every
// value saved here is real streaming content under the "t" field.
SaveToken(ctx context.Context, key string, token string) error
// SaveTraceEvent appends a structured trace event (JSON) to the Redis Stream.
// These are interleaved with tokens so the UI can display tool calls in order.
SaveTraceEvent(ctx context.Context, key string, eventJSON string) error
// TailTokens returns a channel that yields tokens and trace events in order.
// Regular tokens are plain strings. Trace events are prefixed with "\x00" followed
// by JSON. Replays all prior entries first (handles page refresh), then blocks for
// new ones. The channel closes when ctx is cancelled, when a terminal trace event
// (done/fail/finalOutput) is received, or after a 10-minute absolute deadline.
TailTokens(ctx context.Context, key string) (<-chan string, error)
// ReadTraceEvents returns all trace entries from the token-stream Redis key,
// ordered by stream entry ID. Tokens are converted to
// {"type":"token","content":"..."} events; structured trace events (ev field)
// are returned with their original JSON payload. Each entry receives a
// sequential id and an ISO timestamp parsed from the Redis stream ID.
// Returns nil if the stream does not exist or is empty. Used at archival
// time to snapshot a run's full execution trace into PostgreSQL.
ReadTraceEvents(ctx context.Context, key string) ([]TraceEntry, error)
// SaveAnswer stores a human's clarification answer for a run.
SaveAnswer(ctx context.Context, key string, answer string, ttl time.Duration) error
// LoadAnswer retrieves a pending clarification answer. Returns "" if none is set.
LoadAnswer(ctx context.Context, key string) (string, error)
// SaveHTTPOutput stores the output captured from an http-mode agent run.
SaveHTTPOutput(ctx context.Context, runName string, output string) error
// LoadHTTPOutput retrieves http-mode agent run output. Returns "" if not set.
LoadHTTPOutput(ctx context.Context, runName string) (string, error)
// DeleteKey removes a key from the store.
DeleteKey(ctx context.Context, key string) error
// SaveKV stores an arbitrary key-value pair within a scoped prefix.
// Values are zstd-compressed. Maximum value size is 1MB (enforced by callers).
SaveKV(ctx context.Context, scope, key string, value []byte, ttl time.Duration) error
// LoadKV retrieves a value by scope and key. Returns nil, nil if not found.
LoadKV(ctx context.Context, scope, key string) ([]byte, error)
// DeleteKV removes a key within a scope.
DeleteKV(ctx context.Context, scope, key string) error
// ListKV returns all keys within a scope.
ListKV(ctx context.Context, scope string) ([]string, error)
// ListMessageKeys returns conversation-checkpoint keys matching a glob pattern.
// The pattern uses Redis SCAN MATCH semantics: '*' matches any run of
// characters including '/'. Implementations without a native scan should
// return (nil, nil). Used to enumerate prior-run checkpoints for the
// warm-pool local cache + history retrieval.
ListMessageKeys(ctx context.Context, pattern string) ([]string, error)
// SignalCancel sets a cancellation flag for a run in Redis.
// The flag has a short TTL (5 minutes) and auto-expires.
// External systems can also write this key directly: SET cancel:<ns>:<run> 1 EX 300
SignalCancel(ctx context.Context, ns, runName string) error
// IsCancelled checks whether the cancel flag is set for a run.
IsCancelled(ctx context.Context, ns, runName string) (bool, error)
// Ping verifies connectivity to the backing store (e.g. Redis).
// Used by /readyz. Implementations must return nil if reachable.
Ping(ctx context.Context) error
// Close releases any resources held by the store.
Close() error
}
Store is the interface for conversation state storage.
func NewLocalCacheStore ¶
NewLocalCacheStore wraps a durable backing Store (Redis in production) with a local-disk L1 cache rooted at dir. The cache is write-through (every SaveMessages persists to disk AND the backing store) and read-preferred (LoadMessages returns the local copy when present, falling back to the backing store on a local miss). This lets a warm pod resume a conversation without a Redis round-trip and keeps serving if Redis is transiently unavailable, while Redis remains the durable source of truth (the local cache is wiped when the pod is deleted).
On-disk checkpoint keys mirror the storage key as a path (e.g. the key "agentorca/runs/<run>/state" is stored at <dir>/agentorca/runs/<run>/state.msg.zst). Path traversal is rejected: keys are confined to dir via filepath.Clean + a prefix check, and only relative component paths are accepted.
func NewStoreFromConfig ¶
NewStoreFromConfig creates the appropriate Store implementation based on cfg.Backend. When Backend is "" (not configured), a no-op store is returned so the model-router starts successfully without any external state dependency.
type TraceEntry ¶
type TraceEntry struct {
// ID is a sequential counter assigned during archival for React key stability.
ID int `json:"id"`
// Event is the discriminated trace event JSON (e.g. {"type":"toolCall",...}).
Event json.RawMessage `json:"event"`
// TS is an RFC3339 timestamp derived from the Redis stream entry ID.
TS string `json:"ts"`
// ChildRunName is set when this entry originated from a child run's stream.
ChildRunName string `json:"childRunName,omitempty"`
}
TraceEntry is a single timestamped entry in a run's execution trace, mirroring the UI's TraceEntry TypeScript interface. Used to archive the full trace (tokens, tool calls, tool results, etc.) from the Redis stream so that historical runs can be rendered with the same TraceAccordion component as live runs.