assist

package
v0.40.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 26, 2026 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package assist implements the Assist Mode pipeline: STT transcript → Codeword check → LLM → TTS → Result with both text and audio.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Decision added in v0.18.0

type Decision struct {
	Route   Route
	Intent  shortcuts.Intent
	Utility UtilityDefinition
	Payload string
	Locale  string
}

type InMemorySkillContextStore added in v0.37.8

type InMemorySkillContextStore struct {
	// contains filtered or unexported fields
}

InMemorySkillContextStore is the default implementation. It uses a mutex-guarded map and prunes expired entries lazily on every Get.

func NewInMemorySkillContextStore added in v0.37.8

func NewInMemorySkillContextStore(ttl time.Duration, now func() time.Time) *InMemorySkillContextStore

NewInMemorySkillContextStore returns a ready store. A zero or negative ttl falls back to 60 seconds. The clock override is for tests; nil uses time.Now.

func (*InMemorySkillContextStore) Clear added in v0.37.8

func (s *InMemorySkillContextStore) Clear(key string)

Clear removes any context stored under key. Safe to call when no entry exists.

func (*InMemorySkillContextStore) Get added in v0.37.8

Get returns the active context for key, after lazily pruning any expired entry.

func (*InMemorySkillContextStore) Len added in v0.37.8

func (s *InMemorySkillContextStore) Len() int

Len reports how many entries are currently tracked. Exported for tests + diagnostics; production code should not depend on it.

func (*InMemorySkillContextStore) Set added in v0.37.8

func (s *InMemorySkillContextStore) Set(key string, intent shortcuts.Intent, state map[string]string)

Set stores a context for key. ExpiresAt is `now + ttl`.

type Pipeline

type Pipeline struct {
	// contains filtered or unexported fields
}

Pipeline orchestrates the Assist Mode flow.

func NewPipeline

func NewPipeline(assistFlow *core.Flow[flows.AssistInput, flows.AssistOutput, struct{}], executor ToolExecutor, ttsRouter *tts.Router, ttsEnabled bool, opts ...PipelineOption) *Pipeline

NewPipeline creates an Assist Pipeline.

func (*Pipeline) CanHandleWithoutDirectReplyModel added in v0.22.4

func (p *Pipeline) CanHandleWithoutDirectReplyModel(transcript string, opts ProcessOpts) bool

CanHandleWithoutDirectReplyModel reports whether a transcript maps to a local utility path that does not require an Assist LLM.

func (*Pipeline) HasDirectReplyModel added in v0.22.4

func (p *Pipeline) HasDirectReplyModel() bool

HasDirectReplyModel reports whether this pipeline can answer non-utility Assist requests.

func (*Pipeline) Process

func (p *Pipeline) Process(ctx context.Context, transcript string, opts ProcessOpts) (*Result, error)

Process takes a transcript and produces a Result with text and optional audio.

type PipelineOption added in v0.18.0

type PipelineOption func(*Pipeline)

func WithRouter added in v0.18.0

func WithRouter(router *Router) PipelineOption

func WithSkillContextStore added in v0.37.8

func WithSkillContextStore(store SkillContextStore) PipelineOption

WithSkillContextStore enables v0.38.0 multi-turn skill follow-ups. When the host wires a store, callers must also pass ProcessOpts.SessionKey on each Process call so the pipeline knows which conversation to attach state to. Nil disables follow-ups — every utterance is treated as fresh.

type ProcessOpts

type ProcessOpts struct {
	Locale    string // "de", "en", etc.
	Selection string // Currently selected text
	Context   string // Additional context
	Target    any    // Host-specific target for insertion/execution

	// SessionKey identifies the user conversation for v0.38.0 multi-
	// turn follow-ups. Hosts derive this from the authenticated user
	// (e.g. Identity.UserID) or a per-wake-word session id. Empty
	// disables follow-ups for this call.
	SessionKey string
}

ProcessOpts configures a single Assist request.

type Result

type Result struct {
	Text      string // Full response text (always present)
	SpeakText string // TTS-optimized text
	Audio     []byte // TTS audio bytes (present when TTS enabled)
	Format    string // Audio format ("mp3", "wav", etc.)
	Action    string // "respond", "execute", "silent", "shortcut"
	Locale    string // Response language
	Shortcut  string // Matched shortcut intent, if any
	Surface   ResultSurface
	Kind      ResultKind
}

Result is the framework output for Assist Mode. Always contains Text. Contains Audio when TTS is enabled.

type ResultKind added in v0.22.1

type ResultKind string
const (
	ResultKindAnswer        ResultKind = "answer"
	ResultKindWorkProduct   ResultKind = "work_product"
	ResultKindUtilityAction ResultKind = "utility_action"
)

type ResultSurface added in v0.22.1

type ResultSurface string
const (
	ResultSurfacePanel     ResultSurface = "panel"
	ResultSurfaceActionAck ResultSurface = "action_ack"
	ResultSurfaceSilent    ResultSurface = "silent"
)

type Route added in v0.18.0

type Route string
const (
	RouteDirectReply Route = "direct_reply"
	RouteToolIntent  Route = "tool_intent"
	RouteClarify     Route = "clarify"
)

type Router added in v0.18.0

type Router struct {
	// contains filtered or unexported fields
}

func NewRouter added in v0.18.0

func NewRouter(opts ...RouterOption) *Router

func (*Router) Decide added in v0.18.0

func (r *Router) Decide(transcript string, opts ProcessOpts) Decision

func (*Router) UtilityFor added in v0.37.8

func (r *Router) UtilityFor(intent shortcuts.Intent) UtilityDefinition

UtilityFor returns the registered UtilityDefinition for an intent, or the zero value when none is registered. Used by the multi-turn follow-up code path in Process() so the pipeline can rebuild a Decision without re-running keyword resolution. v0.38.0.

type RouterOption added in v0.18.0

type RouterOption func(*Router)

func WithResolver added in v0.18.0

func WithResolver(resolver *shortcuts.Resolver) RouterOption

func WithUtilityRegistry added in v0.24.0

func WithUtilityRegistry(registry *UtilityRegistry) RouterOption

type SkillContext added in v0.37.8

type SkillContext struct {
	Intent    shortcuts.Intent
	State     map[string]string
	ExpiresAt time.Time
}

SkillContext is the v0.38.0 multi-turn state container. When a skill returns a ToolResult with FollowupNeeded=true the pipeline stores its Intent + state under a per-user key. On the user's next transcript the pipeline reuses the stored Intent rather than running fresh keyword routing — that lets a Timer skill ask "for how long?" and treat the next utterance as the duration answer.

Threading: every method is safe for concurrent use. The TTL prune runs inline on every Get to avoid a background goroutine the caller would otherwise have to lifecycle.

Scope: keyed by UserID (anonymous "session" for un-auth flows). Multi-tenant installations must include OrgID in the key. The constructor lets callers pick the key shape.

type SkillContextStore added in v0.37.8

type SkillContextStore interface {
	// Get returns the active context for a key. The ok return is
	// false when no context exists or it has expired.
	Get(key string) (SkillContext, bool)

	// Set stores or replaces the context. ExpiresAt is set by the
	// store using its configured TTL — callers do not set it.
	Set(key string, intent shortcuts.Intent, state map[string]string)

	// Clear removes any active context for the key. Idempotent.
	Clear(key string)
}

SkillContextStore is the storage contract. v0.38.0 ships only the in-memory implementation but the interface lets future deployments swap in a Redis or shared-store backing for satellite topologies.

type ToolCall added in v0.18.0

type ToolCall struct {
	Intent     shortcuts.Intent
	Payload    string
	Transcript string
	Locale     string
	Selection  string
	Context    string
	Target     any
}

type ToolExecutor added in v0.18.0

type ToolExecutor interface {
	Execute(context.Context, ToolCall) (ToolResult, error)
}

type ToolResult added in v0.18.0

type ToolResult struct {
	Text      string
	SpeakText string
	Action    string
	Locale    string
	Surface   ResultSurface
	Kind      ResultKind

	// FollowupNeeded signals a multi-turn skill: the pipeline stores
	// the current Intent + FollowupState under the caller's session
	// key (see ProcessOpts.SessionKey) and the next transcript will
	// re-route to the same skill. When false, any prior follow-up
	// state for the same session is cleared. v0.38.0 (Phase 2).
	FollowupNeeded bool

	// FollowupState carries skill-private data across turns. The
	// pipeline echoes it back via ToolCall.Context on the next
	// invocation. Keep entries small — this is in-memory state, not
	// long-term persistence.
	FollowupState map[string]string
}

type UtilityDefinition added in v0.24.0

type UtilityDefinition struct {
	ID             UtilityID
	Intent         shortcuts.Intent
	Label          string
	Input          UtilityInputRequirement
	DefaultSurface ResultSurface
	DefaultKind    ResultKind
	RequiresModel  bool
	Enabled        bool
}

type UtilityID added in v0.24.0

type UtilityID string
const (
	UtilityCopyLast   UtilityID = "copy_last"
	UtilityInsertLast UtilityID = "insert_last"
	UtilitySummarize  UtilityID = "summarize"
	UtilityQuickNote  UtilityID = "quick_note"

	// Voice-Companion utilities. Wired in Phase 0 with Enabled=false; the
	// per-skill executors land in internal/assist/skills/voice_companion/
	// during Phase 1 and flip Enabled=true through the host registration
	// path. See docs/voice-companion.md.
	UtilityTime          UtilityID = "time"
	UtilityDate          UtilityID = "date"
	UtilityWeather       UtilityID = "weather"
	UtilityTimer         UtilityID = "timer"
	UtilityReminder      UtilityID = "reminder"
	UtilityMath          UtilityID = "math"
	UtilityWikipedia     UtilityID = "wikipedia"
	UtilityHomeAssistant UtilityID = "home_assistant"
)

type UtilityInputRequirement added in v0.24.0

type UtilityInputRequirement string
const (
	UtilityInputNone              UtilityInputRequirement = "none"
	UtilityInputLastTranscript    UtilityInputRequirement = "last_transcript"
	UtilityInputSelectionOptional UtilityInputRequirement = "selection_optional"
	UtilityInputUtterance         UtilityInputRequirement = "utterance"
)

type UtilityRegistry added in v0.24.0

type UtilityRegistry struct {
	// contains filtered or unexported fields
}

func DefaultUtilityRegistry added in v0.24.0

func DefaultUtilityRegistry() *UtilityRegistry

func NewUtilityRegistry added in v0.24.0

func NewUtilityRegistry() *UtilityRegistry

func (*UtilityRegistry) Definition added in v0.24.0

func (r *UtilityRegistry) Definition(intent shortcuts.Intent) (UtilityDefinition, bool)

func (*UtilityRegistry) List added in v0.24.0

func (r *UtilityRegistry) List() []UtilityDefinition

func (*UtilityRegistry) Register added in v0.24.0

func (r *UtilityRegistry) Register(def UtilityDefinition)

func (*UtilityRegistry) Supports added in v0.24.0

func (r *UtilityRegistry) Supports(intent shortcuts.Intent) bool

Directories

Path Synopsis
skills
voice_companion
Package voice_companion provides ToolExecutor-compatible skill plugins for SpeechKit's Voice-Companion pattern.
Package voice_companion provides ToolExecutor-compatible skill plugins for SpeechKit's Voice-Companion pattern.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL