Documentation
¶
Index ¶
- func SummaryMessage(summary string) sdk.Message
- type AnchorPolicy
- type Event
- type EventError
- type ListLogsResponse
- type Log
- type Pass
- type Reason
- type Service
- func (s *Service) ActiveSessions(botID string) []string
- func (s *Service) Compact(ctx context.Context, cfg TriggerConfig) ([]Pass, error)
- func (s *Service) Fold(ctx context.Context, ev Event) ([]Pass, error)
- func (s *Service) ListLogs(ctx context.Context, botID string, limit, offset int) ([]Log, int64, error)
- func (s *Service) SetEventPublisher(p messageevent.Publisher)
- func (s *Service) SetHookService(h *hooks.Service)
- type Source
- type Summarizer
- type TriggerConfig
- type TriggerModel
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func SummaryMessage ¶ added in v1.0.0
SummaryMessage is the message a summary renders as in a request.
Types ¶
type AnchorPolicy ¶ added in v1.0.0
type AnchorPolicy int
AnchorPolicy is how a fold keeps a request's anchors.
const ( // AnchorsCapped keeps the anchors after the newest call group, the input // and steers the next call answers, whatever they take, and the older // ones newest first within the verbatim budget. AnchorsCapped AnchorPolicy = iota // AnchorsFirst keeps the inputs a run answers, a Discuss batch, newest // first while they leave a tenth of the room for the summary; call groups // take what they leave of the verbatim budget. AnchorsFirst )
type Event ¶ added in v1.0.0
type Event struct {
BotID string
SessionID string
// ModelRecordID names the summarizer's model row in the compaction log.
ModelRecordID string
// Request is what the caller sends next and Sources what each of its
// messages renders. The event decides what of it stays verbatim (retain).
Request sdk.Request
Sources []Source
// Current is the input the caller's next request answers when the
// request does not hold it: the summarizer reads it to judge what the
// work needs.
Current []sdk.Message
// AnchorPolicy is how a fold keeps the request's anchors.
AnchorPolicy AnchorPolicy
// Window is the summarizer's context window and MaxOutputTokens its
// output cap; Allowance is what the caller's next request may take.
Window int
MaxOutputTokens int
Allowance int
// FixedOutput marks a summarizer that answers within all of
// MaxOutputTokens: the run's own model, whose output settings a fork of
// the run's request keeps, since the provider settings the fork shares
// with the run's requests (a thinking budget) hold for the output those
// ask for. A pass then reads only what the window holds beside all of it,
// as the run's own requests do. Otherwise a fork asks for less output
// when that lets a pass read more.
FixedOutput bool
// Pressure is the request size that called for the event, for hooks.
Pressure int
// ReplacedTurnID is the turn an edit or retry replaces: the event first
// moves the session to the checkpoint its replacement keeps.
ReplacedTurnID string
Summarizer Summarizer
// Apply splices every committed pass into the caller's request as it
// commits, and returns the request and sources a further pass reads.
Apply func(Pass) (sdk.Request, []Source)
// contains filtered or unexported fields
}
Event is one compaction event over a request its caller holds: the run's next request, or one built from rows the run cannot show.
type EventError ¶ added in v1.0.0
EventError is a pass that failed, with the reason the event reports for it. Passes committed before it stay committed.
func (*EventError) Error ¶ added in v1.0.0
func (e *EventError) Error() string
func (*EventError) Unwrap ¶ added in v1.0.0
func (e *EventError) Unwrap() error
type ListLogsResponse ¶
type ListLogsResponse struct {
Items []Log `json:"items"`
TotalCount int64 `json:"total_count"`
} // @name compaction.ListLogsResponse
ListLogsResponse is the API response for listing compaction logs.
type Log ¶
type Log struct {
ID string `json:"id"`
BotID string `json:"bot_id"`
SessionID string `json:"session_id,omitempty"`
Status string `json:"status"`
Summary string `json:"summary"`
MessageCount int `json:"message_count"`
ErrorMessage string `json:"error_message"`
Usage any `json:"usage,omitempty"`
ModelID string `json:"model_id,omitempty"`
StartedAt time.Time `json:"started_at"`
CompletedAt *time.Time `json:"completed_at,omitempty"`
} // @name compaction.Log
Log represents a compaction log entry.
type Pass ¶ added in v1.0.0
type Pass struct {
SummaryID string
Summary string
// Order is the request after the pass, by the messages of the request it
// read: the index of each that stays, or -1 for the summary, which takes
// the place turn.PlaceSummary gives it; cut is where the newest messages
// the event keeps start in it.
Order []int
// More reports candidates left that the pass could not read; the event
// folds them in a later pass.
More bool
// contains filtered or unexported fields
}
Pass is one committed pass of a compaction event: the summary it wrote and the request it leaves.
type Reason ¶ added in v1.0.0
type Reason string
Reason names why a compaction event failed.
const ( ReasonSummarizerFailed Reason = "summarizer_failed" ReasonTruncated Reason = "truncated" ReasonEmpty Reason = "empty" ReasonIneffective Reason = "ineffective" ReasonStaleSource Reason = "stale_source" ReasonInputOverflow Reason = "input_overflow" ReasonStillOver Reason = "still_over" ReasonProtectedOverflow Reason = "protected_overflow" )
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service manages context compaction for bot conversations.
func NewService ¶
NewService creates a new compaction Service.
func (*Service) ActiveSessions ¶ added in v0.20.0
ActiveSessions is a process-local activity snapshot, not an admission gate.
func (*Service) Compact ¶ added in v1.0.0
Compact runs one compaction event over a session's history read from its rows, for a caller whose request cannot render them: a load the byte budget truncated, a run that is not armed, input a Discuss composer left out. It folds the summaries of the session's active checkpoint and every row the caller does not keep into one summary, each pass forking a request built from the rows (see synthesize) through the compaction model. An edit or retry's event first rewinds the session to the checkpoint its replacement keeps, as the replacement's first step does. It returns the passes it committed; a failed pass ends the event with an *EventError and leaves the passes before it committed.
func (*Service) Fold ¶ added in v1.0.0
Fold runs one compaction event over a request its caller holds: it decides what of the request stays verbatim, forks the request before the newest messages it keeps and asks for the summary that replaces everything else, the summaries of the checkpoint the request renders included. A pass reads as much of the request as the summarizer's window holds; the event passes again only for what that left. It returns the passes it committed; a failed pass ends the event with an *EventError and leaves the passes before it.
func (*Service) ListLogs ¶
func (s *Service) ListLogs(ctx context.Context, botID string, limit, offset int) ([]Log, int64, error)
ListLogs returns paginated compaction logs for a bot.
func (*Service) SetEventPublisher ¶ added in v0.20.0
func (s *Service) SetEventPublisher(p messageevent.Publisher)
SetEventPublisher wires lightweight activity notifications at startup.
func (*Service) SetHookService ¶
type Source ¶ added in v1.0.0
type Source struct {
// Sources are the history rows, inbound messages or summary the message
// renders. A message with none is runtime context the caller rebuilds
// for every request: it stays where it is and no summary stands for it.
Sources []turn.ContextMessageSource
// Anchor marks the run's input or a steer, which a fold keeps verbatim
// where it is before anything else it keeps (retain).
Anchor bool
}
Source is what one message of a folded request stands for.
type Summarizer ¶ added in v1.0.0
Summarizer answers a fork request: the run's own model, or the compaction model for a request built from rows.
type TriggerConfig ¶
type TriggerConfig struct {
BotID string
SessionID string
ModelID string // runtime model slug sent to the provider
ModelRecordID string // models.id row UUID recorded as the log's provenance
ClientType string
APIKey string //nolint:gosec // runtime credential, not a hardcoded secret
CodexAccountID string
BaseURL string
ChatCompletionsCompat string
HTTPClient *http.Client
// TotalInputTokens is the history pressure that called for this
// compaction, reported to hooks.
TotalInputTokens int
// SummaryWindowTokens and MaxOutputTokens are the summarizer model's
// declared window and output cap; they size the summarizer call.
SummaryWindowTokens int
MaxOutputTokens int
// CandidatesBefore, when set, is where the caller's history ends: the
// cutoff an edit or retry replays history before, or the turn the request
// runs as. Rows from it on are not folded.
CandidatesBefore *messagepkg.HistoryPosition
// ReplacedTurnID is the turn an edit or retry replaces. The replacement
// rewinds the session to the checkpoint that does not depend on that
// turn when its first step commits; an event the request needs earlier
// rewinds first, so it folds onto the checkpoint the replacement keeps.
ReplacedTurnID string
// Anchors are kept verbatim where they are, as AnchorPolicy keeps them: a
// run's input and steers, or the inputs a Discuss run answers.
Anchors []turn.ContextMessageSource
AnchorPolicy AnchorPolicy
// CurrentRequest is the caller's current input, which the summarizer
// reads to judge what the work needs.
CurrentRequest string
// HistoryBudgetTokens is the room the caller's request has for what the
// history renders: the summary and the rows it keeps raw.
HistoryBudgetTokens int
}
TriggerConfig holds the parameters needed to trigger a compaction.
func NewTriggerConfig ¶
func NewTriggerConfig(model TriggerModel) TriggerConfig
NewTriggerConfig carries a resolved summarizer into the engine.
type TriggerModel ¶
type TriggerModel struct {
Slug string // runtime model slug sent to the provider
RecordID string // models.id row UUID recorded as the log's provenance
ClientType string
APIKey string //nolint:gosec // runtime credential, not a hardcoded secret
CodexAccountID string
BaseURL string
ChatCompletionsCompat string
WindowTokens int // summarizer model's declared context window
MaxOutputTokens int // summarizer model's declared output cap, zero when unknown
}
TriggerModel is the resolved summarizer identity handed to the engine: selection policy and credential resolution belong to the orchestration surfaces, the engine only receives this completed contract.