Documentation
¶
Overview ¶
Package aggregators assembles the conversation around an LLM. The user aggregator collects transcriptions into a user message and triggers the LLM; the assistant aggregator collects the streamed response into an assistant message. Both share one LLMContext, so the conversation accrues across turns.
Place the user aggregator before the LLM and the assistant aggregator at the end of the pipeline:
pipeline.New(input, stt, agg.User(), llm, tts, output, agg.Assistant())
By default the user turn ends when the STT service finalizes a transcription. With WithTurnTaking, the turn instead ends when a turntaking.Detector reports end-of-turn (a UserStoppedSpeakingFrame), gated on having a finalized transcript — so a Smart Turn model, not STT endpointing, decides when the bot responds. Add the turntaking.Detector right after the input transport:
pipeline.New(input, detector, stt, agg.User(), llm, tts, output, agg.Assistant())
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AssistantAggregator ¶
AssistantAggregator collects the LLM's streamed text into a single assistant message and appends it to the context when the response completes. If the response is interrupted (barge-in), the partial text gathered so far is committed so the context reflects what the bot actually said. The response fields are touched from both the process goroutine (text frames) and the input goroutine (the InterruptionFrame system frame), so they are mutex-guarded.
func (*AssistantAggregator) ProcessFrame ¶
func (a *AssistantAggregator) ProcessFrame(ctx context.Context, f frames.Frame, dir processor.Direction) error
ProcessFrame collects LLM text into an assistant message.
type Option ¶
type Option func(*options)
Option configures an aggregator Pair.
func WithSummarization ¶
func WithSummarization(cfg SummarizeConfig) Option
WithSummarization enables automatic context summarization on the assistant aggregator (see SummarizeConfig):
sum := llm.NewSummarizer(anthropic.NewLLM(anthropic.Config{}))
pair := aggregators.New(ctx, aggregators.WithSummarization(
aggregators.SummarizeConfig{Summarizer: sum}))
func WithTurnTaking ¶
func WithTurnTaking() Option
WithTurnTaking gates the user turn on end-of-turn detection: the LLM runs when a turntaking.Detector reports the turn complete (UserStoppedSpeakingFrame) and a finalized transcript is in hand, rather than on STT finalization alone.
type Pair ¶
type Pair struct {
// contains filtered or unexported fields
}
Pair is a user and assistant aggregator sharing one conversation context.
func New ¶
func New(ctx *frames.LLMContext, opts ...Option) *Pair
New builds a user/assistant aggregator pair around ctx.
func (*Pair) Context ¶
func (p *Pair) Context() *frames.LLMContext
Context returns the shared conversation context.
type SummarizeConfig ¶
type SummarizeConfig struct {
// Summarizer produces the summary. Required; a zero Summarizer disables
// summarization.
Summarizer Summarizer
// TriggerTokens is the estimated context size, in tokens, above which older
// turns are summarized. Zero uses a default tuned for voice sessions.
TriggerTokens int
// KeepRecentMessages is how many of the most recent messages stay out of the
// summary. Zero uses a default; values below the minimum are raised to it.
KeepRecentMessages int
// Timeout bounds a single summarization call. Zero uses a default.
Timeout time.Duration
}
SummarizeConfig enables automatic context summarization on the assistant aggregator. When the shared context grows past TriggerTokens (estimated), older turns are folded into a summary appended to the system prompt while the most recent KeepRecentMessages are kept verbatim. Summarization runs in the background, so it never adds latency to a turn.
type Summarizer ¶
type Summarizer interface {
Summarize(ctx context.Context, prior string, dropped []frames.Message) (string, error)
}
Summarizer condenses the oldest conversation messages into a compact running summary. It receives the prior summary (empty on the first compaction) and the messages being dropped, and returns the new summary that stands in for them. service/llm.Summarizer adapts any LLM Generator to this contract.
type UserAggregator ¶
UserAggregator collects transcriptions into a user message and, when the user's turn ends, appends it to the context and triggers the LLM with an LLMContextFrame. With turn taking enabled it also tracks speaking and end-of-turn frames; those are system frames handled on a different goroutine than transcriptions, so the aggregation state is mutex-guarded.
func (*UserAggregator) ProcessFrame ¶
func (u *UserAggregator) ProcessFrame(ctx context.Context, f frames.Frame, dir processor.Direction) error
ProcessFrame collects transcriptions and triggers the LLM.
Source Files