Documentation
¶
Overview ¶
Package live exposes the low-level Voice Agent realtime-protocol types.
This is the public-API surface for building custom realtime providers (Gemini Live, OpenAI Realtime, or a third-party WebSocket model) and for libraries that drive a SpeechKit session directly without going through the higher-level [Service] in the parent package.
Most embedders use the parent [voiceagent] package or github.com/kombifyio/SpeechKit/pkg/speechkit/agentkit instead; reach for this package only when you need to plug in your own provider or read the raw LiveMessage stream.
Index ¶
- Constants
- Variables
- func AppendContextPrompt(base, contextPrompt string) string
- func EventTypesContain(values []LiveEventType, want LiveEventType) bool
- func NormalizeProviderID(providerOrProfile string) string
- func RenderHostInstructionUpdate(cfg LiveConfig) string
- func ShouldTryFallback(primary, fallback string) bool
- func UpsampleMicPCM16Mono(src []byte) []byte
- type ActivityDetectionPolicy
- type ActivityHandling
- type Callbacks
- type CapabilityRequirement
- type ContextCompressionPolicy
- type EndSensitivity
- type HostPromptEvent
- type HostPromptEventType
- type HostPromptKind
- type IdleConfig
- type IdleTimer
- type LatencyProfile
- type LiveCapabilityFlag
- type LiveConfig
- type LiveEndpointReporter
- type LiveEventType
- type LiveInstructionUpdater
- type LiveMessage
- type LiveModelDescriptor
- type LivePolicies
- type LiveProvider
- type LiveReconnector
- type LiveSessionCapabilities
- type ModelLifecyclePolicy
- type ProviderDescriptor
- type ProviderFallback
- type ProviderFallbackKind
- type ProviderIntent
- type ProviderIntentError
- type ProviderRejection
- type ProviderSelectionPolicy
- type ResolvedLiveOptions
- type ResolvedProviderPlan
- type ResumeHandle
- type Session
- func (s *Session) AdvanceWorkflowStep(ctx context.Context, reason string) error
- func (s *Session) CurrentState() State
- func (s *Session) EndAudioStream() error
- func (s *Session) ProviderName() string
- func (s *Session) SendAgentProgress(text string) (bool, error)
- func (s *Session) SendAudio(chunk []byte) error
- func (s *Session) SendText(text string) error
- func (s *Session) SendToolResponse(response ToolResponse) error
- func (s *Session) Start(ctx context.Context, cfg LiveConfig, idleCfg IdleConfig) error
- func (s *Session) Stop()
- func (s *Session) StopWithTimeout(timeout time.Duration)
- type SessionCapabilities
- type StartSensitivity
- type State
- type ThinkingLevel
- type ThinkingPolicy
- type ToolBehavior
- type ToolCall
- type ToolDefinition
- type ToolResponse
- type ToolResponseScheduling
- type TurnCoverage
- type WorkflowConfig
- type WorkflowStep
Constants ¶
const ( // DefaultStopCloseTimeout bounds how long Stop waits for provider.Close. // Gemini/Deepgram/AssemblyAI Close has no timeout of its own; holding the // session mutex across an unbounded close wedged every caller, including // the server target and SDK hosts that do not wrap Stop themselves. DefaultStopCloseTimeout = 3 * time.Second )
Variables ¶
var ErrNoMatchingProvider = errors.New("speechkit live: no provider satisfies intent")
Functions ¶
func AppendContextPrompt ¶ added in v0.64.3
func EventTypesContain ¶ added in v0.64.3
func EventTypesContain(values []LiveEventType, want LiveEventType) bool
EventTypesContain reports whether want appears in values. Frames carry several normalized meanings when one provider event maps to more than one, so callers ask rather than compare a single field.
func NormalizeProviderID ¶ added in v0.47.0
NormalizeProviderID maps common provider aliases and public profile ids to the canonical provider id used by ProviderDescriptor.Provider.
func RenderHostInstructionUpdate ¶
func RenderHostInstructionUpdate(cfg LiveConfig) string
RenderHostInstructionUpdate turns a workflow step change into a provider text update. Providers that implement LiveInstructionUpdater receive the structured LiveConfig instead.
func ShouldTryFallback ¶ added in v0.64.3
ShouldTryFallback reports whether a configured fallback model is worth attempting after the primary failed: it must be set, and it must not be the model that just failed.
func UpsampleMicPCM16Mono ¶ added in v0.64.3
UpsampleMicPCM16Mono linearly interpolates a 16-bit signed little-endian PCM mono buffer from SpeechKit's mic rate to OpenAI's input rate. For the 16 kHz to 24 kHz path (ratio 2:3) the cost is negligible compared to the WS round-trip; CPU profiling at scale should still consider replacing this with a polyphase FIR resampler if it shows up as a hot path.
Types ¶
type ActivityDetectionPolicy ¶
type ActivityDetectionPolicy struct {
Automatic bool
StartSensitivity StartSensitivity
EndSensitivity EndSensitivity
PrefixPaddingMs int32
SilenceDurationMs int32
ActivityHandling ActivityHandling
TurnCoverage TurnCoverage
}
ActivityDetectionPolicy defines server-side VAD/session turn behavior.
type ActivityHandling ¶
type ActivityHandling string
ActivityHandling controls what Gemini Live should do when new activity starts.
const ( ActivityHandlingUnspecified ActivityHandling = "" ActivityHandlingNoInterrupt ActivityHandling = "no_interrupt" ActivityHandlingStartOfActivityInterrupts ActivityHandling = "start_of_activity_interrupts" )
type Callbacks ¶
type Callbacks struct {
OnStateChange func(state State)
OnAudio func(audio []byte) // Audio chunk to play
OnText func(text string) // Text for display (speech bubble)
OnError func(err error)
OnInputTranscript func(text string, done bool) // User speech transcribed
OnOutputTranscript func(text string, done bool) // Model speech transcribed
OnToolCall func(call ToolCall)
OnToolCallCancellation func(ids []string)
// OnHostPrompt may reject a Started event by returning false. Sent and
// SendFailed are correlated terminal phases for an accepted prompt; their
// return values are ignored.
OnHostPrompt func(event HostPromptEvent) bool
OnInterrupted func() // User interrupted model (barge-in)
OnSessionEnd func() // Session ended (error, GoAway failure, or deactivation)
}
Callbacks are event handlers for UI integration with a low-level Session.
This is the rich Callbacks struct used by the low-level Session runtime. The higher-level [voiceagent.Callbacks] in the parent package carries only the three most-common handlers (OnAudio, OnText, OnError) and is the right shape for the embedded [voiceagent.Service]; reach for this struct only when wiring a custom realtime host.
type CapabilityRequirement ¶ added in v0.47.0
type CapabilityRequirement struct {
Capability LiveCapabilityFlag `json:"capability"`
Required bool `json:"required,omitempty"`
}
type ContextCompressionPolicy ¶
ContextCompressionPolicy defines how the live API should compress long sessions.
type EndSensitivity ¶
type EndSensitivity string
EndSensitivity controls how aggressively automatic activity detection commits speech end.
const ( EndSensitivityLow EndSensitivity = "low" EndSensitivityMedium EndSensitivity = "medium" EndSensitivityHigh EndSensitivity = "high" )
type HostPromptEvent ¶ added in v0.50.0
type HostPromptEvent struct {
ID uint64
Kind HostPromptKind
Type HostPromptEventType
}
HostPromptEvent correlates the synchronous authorization opening with a possible provider send failure. IDs are session-local and never reused.
type HostPromptEventType ¶ added in v0.50.0
type HostPromptEventType string
const ( HostPromptStarted HostPromptEventType = "started" HostPromptSent HostPromptEventType = "sent" HostPromptSendFailed HostPromptEventType = "send_failed" )
type HostPromptKind ¶ added in v0.50.0
type HostPromptKind string
HostPromptKind identifies a trusted, locally generated text turn. These prompts are distinct from user audio and let hosts open an explicit playback generation instead of treating arbitrary out-of-turn model audio as an idle response.
const ( HostPromptIdleReminder HostPromptKind = "idle_reminder" HostPromptIdleDeactivate HostPromptKind = "idle_deactivate" // HostPromptAgentProgress carries progress from a long-running host-side // tool (e.g. the external coding agent bridge) back into the dialogue. // Hosts deliver it only while the session is listening, so it never // collides with user speech or model playback. HostPromptAgentProgress HostPromptKind = "agent_progress" )
type IdleConfig ¶
type IdleConfig struct {
ReminderAfter time.Duration // Default: 5 minutes
DeactivateAfter time.Duration // Default: 15 minutes
}
IdleConfig configures the idle timer behavior.
func DefaultIdleConfig ¶
func DefaultIdleConfig() IdleConfig
DefaultIdleConfig returns sensible defaults.
type IdleTimer ¶
type IdleTimer struct {
// contains filtered or unexported fields
}
IdleTimer manages reminder and auto-deactivation for Voice Agent.
func NewIdleTimer ¶
func NewIdleTimer(cfg IdleConfig, session *Session) *IdleTimer
NewIdleTimer creates an idle timer bound to a session.
type LatencyProfile ¶ added in v0.47.0
type LatencyProfile string
const ( LatencyProfileInteractive LatencyProfile = "interactive" LatencyProfileBalanced LatencyProfile = "balanced" LatencyProfileAccuracy LatencyProfile = "accuracy" )
type LiveCapabilityFlag ¶ added in v0.47.0
type LiveCapabilityFlag string
const ( LiveCapabilityRealtimeAudio LiveCapabilityFlag = "realtime_audio" LiveCapabilityToolCalling LiveCapabilityFlag = "tool_calling" LiveCapabilityNativeWords LiveCapabilityFlag = "native_words" LiveCapabilityTranscript LiveCapabilityFlag = "transcript" LiveCapabilityInterruptions LiveCapabilityFlag = "interruptions" LiveCapabilitySessionResume LiveCapabilityFlag = "session_resume" LiveCapabilityNativeContextPrompt LiveCapabilityFlag = "native_context_prompt" LiveCapabilityNativeKeyterms LiveCapabilityFlag = "native_keyterms" LiveCapabilityLanguageHints LiveCapabilityFlag = "language_hints" LiveCapabilitySpeakerStreaming LiveCapabilityFlag = "speaker_streaming" LiveCapabilityPrivacyRedaction LiveCapabilityFlag = "privacy_redaction" LiveCapabilityVoiceFocus LiveCapabilityFlag = "voice_focus" LiveCapabilityMedicalDomain LiveCapabilityFlag = "medical_domain" LiveCapabilityReasoningEffort LiveCapabilityFlag = "reasoning_effort" LiveCapabilityTranslation LiveCapabilityFlag = "translation" LiveCapabilityTranscriptionOnly LiveCapabilityFlag = "transcription_only" )
type LiveConfig ¶
type LiveConfig struct {
Provider string // e.g. "google", "deepgram", "assemblyai", "openai"
ProfileID string // e.g. "realtime.google.gemini-native-audio"
Model string // e.g. "gemini-3.1-flash-live-preview"
// FallbackModel is tried when the primary Model's Connect fails. Empty
// disables the fallback. Typical pairing in 2026: a preview model as
// Model + the last GA model as FallbackModel, so transient preview
// outages don't take down a Voice Agent deployment.
FallbackModel string
APIKey string
Voice string // Voice name
FrameworkPrompt string
RefinementPrompt string
VocabularyHint string
Locale string
// Region is the Google Cloud region the caller's API key / project is
// pinned to (e.g. "europe-west3", "us-central1"). Used by providers that
// support regional endpoints. For Gemini Live (as of May 2026) the API
// exposes a single global WebSocket endpoint, so this field does NOT
// redirect traffic — it is logged at connect time for compliance evidence
// (byok.key_updated audit event) and reserved for future regional routing
// once Google publishes per-region hostnames. Data residency is controlled
// at the Google Cloud project level; both the project region AND this field
// must agree for audit records to be accurate.
// See docs/compliance/byok-gemini-region-pinning.md.
Region string
Policies LivePolicies
Tools []ToolDefinition
Workflow *WorkflowConfig
Speaker speaker.Options
Options provideropts.Values
ProviderOptions provideropts.Values
}
LiveConfig configures a real-time session.
func DefaultLiveConfigForProvider ¶ added in v0.47.0
func DefaultLiveConfigForProvider(providerOrProfile string) (LiveConfig, bool)
DefaultLiveConfigForProvider returns a provider/profile/model tuple suitable for constructing an embedded Voice Agent session. Callers still supply API keys, prompts, tools, and policies.
type LiveEndpointReporter ¶ added in v0.61.33
type LiveEndpointReporter interface {
EndpointURL() string
}
LiveEndpointReporter is implemented by providers that can report the endpoint they dial. Connect-time observability only — the reported URL must never carry credentials; providerEndpointForLog additionally strips any query string so tokens passed as query parameters cannot leak into logs.
type LiveEventType ¶ added in v0.47.0
type LiveEventType string
const ( LiveEventSessionReady LiveEventType = "session_ready" LiveEventInputPartial LiveEventType = "input_partial" LiveEventInputFinal LiveEventType = "input_final" LiveEventOutputAudio LiveEventType = "output_audio" LiveEventOutputText LiveEventType = "output_text" LiveEventToolCall LiveEventType = "tool_call" LiveEventToolResultAck LiveEventType = "tool_result_ack" LiveEventInterrupted LiveEventType = "interrupted" LiveEventTurnEnd LiveEventType = "turn_end" LiveEventSessionResumable LiveEventType = "session_resumable" LiveEventSessionEnd LiveEventType = "session_end" )
func InferLiveEventTypes ¶ added in v0.47.0
func InferLiveEventTypes(msg *LiveMessage) []LiveEventType
InferLiveEventTypes returns the provider-neutral event meanings represented by a LiveMessage. Providers should set EventType/EventTypes themselves when translating native frames; this helper gives custom providers and tests the same fallback semantics.
type LiveInstructionUpdater ¶
type LiveInstructionUpdater interface {
UpdateInstructions(ctx context.Context, cfg LiveConfig) error
}
LiveInstructionUpdater is optionally implemented by providers that can refresh active host instructions without treating the update as a user turn.
type LiveMessage ¶
type LiveMessage struct {
EventType LiveEventType // Provider-neutral event type for hosts that need precise routing.
EventTypes []LiveEventType // All provider-neutral event meanings when a provider combines multiple events in one frame.
ProviderMetadata map[string]any // Optional provider-native metadata for debugging and advanced hosts.
Audio []byte // PCM audio chunk (24kHz 16-bit mono)
Text string // Text transcript (may be partial or empty)
Done bool // True when the model's turn is complete
// Transcription fields (populated when transcription is enabled).
InputTranscript string // User speech transcribed by server
InputTranscriptDone bool // True when input transcription segment is final
InputSpeakerLabel string // Optional diarized speaker label for input transcript segments
InputPersonID string // Optional known person id for input transcript segments
InputDisplayName string // Optional known display name for input transcript segments
InputSpeakerConfidence float64 // Optional speaker-label confidence
OutputTranscript string // Model speech transcribed by server
OutputTranscriptDone bool // True when output transcription segment is final
ToolCalls []ToolCall
ToolCallCancellationIDs []string
Interrupted bool // True when user interrupted model (barge-in)
GoAway bool // True when server signals imminent session end
SessionResumable bool // True when the provider indicates reconnect/resume is possible.
}
LiveMessage is a message received from the real-time model.
func NormalizeMessageEvents ¶ added in v0.64.3
func NormalizeMessageEvents(msg *LiveMessage, providerEvent string) *LiveMessage
type LiveModelDescriptor ¶ added in v0.47.0
type LiveModelDescriptor struct {
Provider string `json:"provider"`
ModelID string `json:"modelId"`
Name string `json:"name"`
Lifecycle framework.ModelLifecycle `json:"lifecycle"`
Default bool `json:"default,omitempty"`
Recommended bool `json:"recommended,omitempty"`
SourceURL string `json:"sourceUrl"`
}
type LivePolicies ¶
type LivePolicies struct {
EnableInputAudioTranscription bool
EnableOutputAudioTranscription bool
EnableAffectiveDialog bool
Thinking ThinkingPolicy
ContextCompression ContextCompressionPolicy
ActivityDetection ActivityDetectionPolicy
}
LivePolicies configures Google Live API features that shape Voice Agent behavior.
type LiveProvider ¶
type LiveProvider interface {
// Connect establishes a WebSocket session to the real-time model.
Connect(ctx context.Context, cfg LiveConfig) error
// SendAudio streams PCM audio chunks to the model.
// Format: 16-bit signed int, little-endian, mono, 16kHz.
SendAudio(chunk []byte) error
// SendAudioStreamEnd signals that microphone input for the current turn ended.
SendAudioStreamEnd() error
// Receive blocks until the next server message arrives.
// Returns audio chunks and/or text from the model.
Receive(ctx context.Context) (*LiveMessage, error)
// SendText injects a text prompt into the session (for idle reminders).
SendText(text string) error
// SendToolResponse sends the result of a host-side tool invocation back to the model.
SendToolResponse(response ToolResponse) error
// Close terminates the WebSocket session.
Close() error
// Name returns the provider identifier.
Name() string
}
LiveProvider abstracts a real-time audio-to-audio model connection.
type LiveReconnector ¶
LiveReconnector is an optional interface for providers that support session reconnection.
type LiveSessionCapabilities ¶ added in v0.47.0
type LiveSessionCapabilities interface {
SessionCapabilities() SessionCapabilities
}
type ModelLifecyclePolicy ¶ added in v0.47.0
type ModelLifecyclePolicy string
const ( ModelLifecycleAny ModelLifecyclePolicy = "any" ModelLifecyclePreferGA ModelLifecyclePolicy = "prefer_ga" ModelLifecycleRequireGA ModelLifecyclePolicy = "require_ga" )
type ProviderDescriptor ¶ added in v0.47.0
type ProviderDescriptor struct {
Provider string `json:"provider"`
DisplayName string `json:"displayName"`
ProfileID string `json:"profileId"`
Capabilities []LiveCapabilityFlag `json:"capabilities"`
Models []LiveModelDescriptor `json:"models,omitempty"`
SupportedLocales []string `json:"supportedLocales,omitempty"`
NativeOptions []provideropts.OptionID `json:"nativeOptions,omitempty"`
AuthRequirement string `json:"authRequirement,omitempty"`
Transport string `json:"transport,omitempty"`
EvidenceURL string `json:"evidenceUrl,omitempty"`
}
func DefaultProviderDescriptors ¶ added in v0.47.0
func DefaultProviderDescriptors() []ProviderDescriptor
DefaultProviderDescriptors returns the public live-provider catalog for embedders that want to switch providers by name/profile/model at runtime.
func FindProviderDescriptor ¶ added in v0.47.0
func FindProviderDescriptor(providerOrProfile string) (ProviderDescriptor, bool)
FindProviderDescriptor resolves provider ids, provider aliases, and profile ids to the canonical public descriptor.
func (ProviderDescriptor) DefaultModel ¶ added in v0.47.0
func (d ProviderDescriptor) DefaultModel() (LiveModelDescriptor, bool)
DefaultModel returns the provider's default live model. If no model is explicitly marked as default, it falls back to the first advertised model.
func (ProviderDescriptor) HasCapability ¶ added in v0.47.0
func (d ProviderDescriptor) HasCapability(flag LiveCapabilityFlag) bool
HasCapability reports whether this provider advertises a capability.
type ProviderFallback ¶ added in v0.47.0
type ProviderFallback struct {
Kind ProviderFallbackKind `json:"kind"`
Provider string `json:"provider"`
ProfileID string `json:"profileId,omitempty"`
Model string `json:"model,omitempty"`
ModelLifecycle framework.ModelLifecycle `json:"modelLifecycle,omitempty"`
Reason string `json:"reason,omitempty"`
MissingRequiredCapabilities []LiveCapabilityFlag `json:"missingRequiredCapabilities,omitempty"`
MissingRequiredOptions []provideropts.OptionID `json:"missingRequiredOptions,omitempty"`
AuthRequirement string `json:"authRequirement,omitempty"`
Transport string `json:"transport,omitempty"`
EvidenceURL string `json:"evidenceUrl,omitempty"`
}
type ProviderFallbackKind ¶ added in v0.47.0
type ProviderFallbackKind string
const ( FallbackKindSameProviderModel ProviderFallbackKind = "same_provider_model" FallbackKindCrossProvider ProviderFallbackKind = "cross_provider" FallbackKindCascaded ProviderFallbackKind = "cascaded" FallbackKindCapabilityMissing ProviderFallbackKind = "capability_missing" )
type ProviderIntent ¶ added in v0.47.0
type ProviderIntent struct {
Mode string `json:"mode,omitempty"`
Provider string `json:"provider,omitempty"`
ProfileID string `json:"profileId,omitempty"`
Model string `json:"model,omitempty"`
RequiredCapabilities []LiveCapabilityFlag `json:"requiredCapabilities,omitempty"`
PreferredCapabilities []LiveCapabilityFlag `json:"preferredCapabilities,omitempty"`
Requirements []CapabilityRequirement `json:"requirements,omitempty"`
RequiredOptions []provideropts.OptionID `json:"requiredOptions,omitempty"`
PreferredOptions []provideropts.OptionID `json:"preferredOptions,omitempty"`
Locale string `json:"locale,omitempty"`
LanguageHints []string `json:"languageHints,omitempty"`
PrivacyRedaction bool `json:"privacyRedaction,omitempty"`
ResumePreferred bool `json:"resumePreferred,omitempty"`
LatencyProfile LatencyProfile `json:"latencyProfile,omitempty"`
SelectionPolicy ProviderSelectionPolicy `json:"selectionPolicy,omitempty"`
}
type ProviderIntentError ¶ added in v0.47.0
type ProviderIntentError struct {
Intent ProviderIntent `json:"intent"`
MissingRequiredCapabilities []LiveCapabilityFlag `json:"missingRequiredCapabilities,omitempty"`
MissingRequiredOptions []provideropts.OptionID `json:"missingRequiredOptions,omitempty"`
Fallbacks []ProviderFallback `json:"fallbacks,omitempty"`
RejectedProviders []ProviderRejection `json:"rejectedProviders,omitempty"`
}
func (*ProviderIntentError) Error ¶ added in v0.47.0
func (e *ProviderIntentError) Error() string
func (*ProviderIntentError) Unwrap ¶ added in v0.47.0
func (e *ProviderIntentError) Unwrap() error
type ProviderRejection ¶ added in v0.47.0
type ProviderRejection struct {
Provider string `json:"provider"`
ProfileID string `json:"profileId,omitempty"`
Model string `json:"model,omitempty"`
ModelLifecycle framework.ModelLifecycle `json:"modelLifecycle,omitempty"`
FallbackKind ProviderFallbackKind `json:"fallbackKind,omitempty"`
Reason string `json:"reason"`
MissingRequiredCapabilities []LiveCapabilityFlag `json:"missingRequiredCapabilities,omitempty"`
MissingRequiredOptions []provideropts.OptionID `json:"missingRequiredOptions,omitempty"`
AuthRequirement string `json:"authRequirement,omitempty"`
Transport string `json:"transport,omitempty"`
EvidenceURL string `json:"evidenceUrl,omitempty"`
UnsupportedLocale string `json:"unsupportedLocale,omitempty"`
}
type ProviderSelectionPolicy ¶ added in v0.47.0
type ProviderSelectionPolicy struct {
PreferredProviders []string `json:"preferredProviders,omitempty"`
AllowPreview bool `json:"allowPreview,omitempty"`
AllowLegacy bool `json:"allowLegacy,omitempty"`
ModelLifecycle ModelLifecyclePolicy `json:"modelLifecycle,omitempty"`
}
type ResolvedLiveOptions ¶ added in v0.45.0
type ResolvedLiveOptions struct {
Locale string
Voice string
ContextPrompt string
LanguageHints []string
Keyterms []string
ReasoningEffort string
Resume bool
TurnDetection bool
TurnDetectionSource provideropts.ValueSource
EndpointingMs int
EndpointingSource provideropts.ValueSource
Effective provideropts.EffectiveOptions
}
func ResolveLiveOptions ¶ added in v0.45.0
func ResolveLiveOptions(provider, profileID string, cfg LiveConfig, providerDefaults, providerOverrides provideropts.Values) ResolvedLiveOptions
func (ResolvedLiveOptions) HasEndpointingOverride ¶ added in v0.45.0
func (r ResolvedLiveOptions) HasEndpointingOverride() bool
func (ResolvedLiveOptions) HasTurnDetectionOverride ¶ added in v0.45.0
func (r ResolvedLiveOptions) HasTurnDetectionOverride() bool
type ResolvedProviderPlan ¶ added in v0.47.0
type ResolvedProviderPlan struct {
Provider string `json:"provider"`
ProfileID string `json:"profileId"`
Model string `json:"model"`
Descriptor ProviderDescriptor `json:"descriptor"`
ModelDescriptor LiveModelDescriptor `json:"modelDescriptor"`
SelectionReason string `json:"selectionReason,omitempty"`
SelectedFallbackKind ProviderFallbackKind `json:"selectedFallbackKind,omitempty"`
Fallbacks []ProviderFallback `json:"fallbacks,omitempty"`
RejectedProviders []ProviderRejection `json:"rejectedProviders,omitempty"`
MatchedRequiredCapabilities []LiveCapabilityFlag `json:"matchedRequiredCapabilities,omitempty"`
MatchedPreferredCapabilities []LiveCapabilityFlag `json:"matchedPreferredCapabilities,omitempty"`
UnsupportedPreferredCapabilities []LiveCapabilityFlag `json:"unsupportedPreferredCapabilities,omitempty"`
MatchedRequiredOptions []provideropts.OptionID `json:"matchedRequiredOptions,omitempty"`
MatchedPreferredOptions []provideropts.OptionID `json:"matchedPreferredOptions,omitempty"`
UnsupportedPreferredOptions []provideropts.OptionID `json:"unsupportedPreferredOptions,omitempty"`
AuthRequirement string `json:"authRequirement,omitempty"`
Transport string `json:"transport,omitempty"`
LatencyProfile LatencyProfile `json:"latencyProfile,omitempty"`
}
func ResolveProviderIntent ¶ added in v0.47.0
func ResolveProviderIntent(intent ProviderIntent, descriptors []ProviderDescriptor) (ResolvedProviderPlan, error)
func (ResolvedProviderPlan) LiveConfig ¶ added in v0.47.0
func (p ResolvedProviderPlan) LiveConfig() LiveConfig
func (ResolvedProviderPlan) String ¶ added in v0.47.0
func (p ResolvedProviderPlan) String() string
type ResumeHandle ¶ added in v0.64.3
type ResumeHandle struct {
// contains filtered or unexported fields
}
ResumeHandle stores a Gemini Live session resumption handle with a time-to- live and at-rest protection. On Windows the ciphertext is produced by DPAPI (CryptProtectData, user-scope) so a memory dump taken hours later cannot be replayed against the same session. On other platforms the handle is held in memory without encryption (there is no equivalent process-scoped primitive in the Go standard library) but the TTL still applies.
func NewResumeHandle ¶ added in v0.64.3
func NewResumeHandle() *ResumeHandle
func (*ResumeHandle) Clear ¶ added in v0.64.3
func (h *ResumeHandle) Clear()
Clear wipes any stored handle.
func (*ResumeHandle) Get ¶ added in v0.64.3
func (h *ResumeHandle) Get() string
Get returns the cleartext handle if present and not expired. Expired handles are discarded on read. An empty return value means "no handle available".
func (*ResumeHandle) Set ¶ added in v0.64.3
func (h *ResumeHandle) Set(raw string)
Set replaces the stored handle. An empty input clears it.
type Session ¶
type Session struct {
// contains filtered or unexported fields
}
Session manages a Voice Agent conversation.
func NewSession ¶
func NewSession(provider LiveProvider, callbacks Callbacks) *Session
NewSession creates a Voice Agent session with the given provider.
func (*Session) AdvanceWorkflowStep ¶
AdvanceWorkflowStep moves a configured local workflow to its next step and updates the active provider instructions. It returns nil when no workflow is active or the workflow has already completed.
func (*Session) CurrentState ¶
State returns the current session state.
func (*Session) EndAudioStream ¶
EndAudioStream tells the live provider that the current microphone stream ended.
func (*Session) ProviderName ¶
func (*Session) SendAgentProgress ¶ added in v0.53.0
SendAgentProgress delivers a trusted progress line from a long-running host-side tool as a host prompt. It is a no-op unless the session is listening: a progress line must never interrupt the user speaking or the model answering. Returns true when the prompt was accepted and sent.
func (*Session) SendToolResponse ¶
func (s *Session) SendToolResponse(response ToolResponse) error
SendToolResponse forwards the result of a host-side tool invocation to the model.
func (*Session) Start ¶
func (s *Session) Start(ctx context.Context, cfg LiveConfig, idleCfg IdleConfig) error
Start activates the Voice Agent session.
func (*Session) Stop ¶
func (s *Session) Stop()
Stop deactivates the Voice Agent session. It waits at most DefaultStopCloseTimeout for provider.Close so a hung websocket handshake cannot wedge the caller.
func (*Session) StopWithTimeout ¶ added in v0.60.41
StopWithTimeout is Stop with an explicit Close bound. A zero or negative timeout waits for Close without a deadline (tests and operators only).
type SessionCapabilities ¶ added in v0.47.0
type SessionCapabilities struct {
Provider string `json:"provider,omitempty"`
ProfileID string `json:"profileId,omitempty"`
Model string `json:"model,omitempty"`
Capabilities []LiveCapabilityFlag `json:"capabilities,omitempty"`
ProviderMetadata map[string]any `json:"providerMetadata,omitempty"`
}
func SessionCapabilitiesForProvider ¶ added in v0.64.3
func SessionCapabilitiesForProvider(provider string) SessionCapabilities
type StartSensitivity ¶
type StartSensitivity string
StartSensitivity controls how aggressively automatic activity detection commits speech start.
const ( StartSensitivityLow StartSensitivity = "low" StartSensitivityMedium StartSensitivity = "medium" StartSensitivityHigh StartSensitivity = "high" )
type ThinkingLevel ¶
type ThinkingLevel string
ThinkingLevel controls how much deliberate reasoning Gemini Live should spend.
const ( ThinkingLevelOff ThinkingLevel = "off" ThinkingLevelLow ThinkingLevel = "low" ThinkingLevelMedium ThinkingLevel = "medium" ThinkingLevelHigh ThinkingLevel = "high" )
type ThinkingPolicy ¶
type ThinkingPolicy struct {
Enabled bool
IncludeThoughts bool
ThinkingBudget int32
ThinkingLevel ThinkingLevel
}
ThinkingPolicy defines optional Gemini Live thinking behavior.
type ToolBehavior ¶
type ToolBehavior string
ToolBehavior controls whether the model waits for a tool result.
const ( ToolBehaviorUnspecified ToolBehavior = "" ToolBehaviorBlocking ToolBehavior = "blocking" ToolBehaviorNonBlocking ToolBehavior = "non_blocking" )
type ToolDefinition ¶
type ToolDefinition struct {
Name string
Description string
ParametersJSONSchema map[string]any
ResponseJSONSchema map[string]any
Behavior ToolBehavior
}
ToolDefinition exposes a host-side action the Voice Agent may call.
type ToolResponse ¶
type ToolResponse struct {
ID string
Name string
Response map[string]any
Scheduling ToolResponseScheduling
WillContinue *bool
}
ToolResponse resolves a previously emitted tool call.
type ToolResponseScheduling ¶
type ToolResponseScheduling string
ToolResponseScheduling controls how a non-blocking tool result is reintroduced into the conversation.
const ( ToolResponseSchedulingUnspecified ToolResponseScheduling = "" ToolResponseSchedulingSilent ToolResponseScheduling = "silent" ToolResponseSchedulingWhenIdle ToolResponseScheduling = "when_idle" ToolResponseSchedulingInterrupt ToolResponseScheduling = "interrupt" )
type TurnCoverage ¶
type TurnCoverage string
TurnCoverage controls how the live API builds a user turn from incoming activity.
const ( TurnCoverageUnspecified TurnCoverage = "" TurnCoverageTurnIncludesOnlyActivity TurnCoverage = "turn_includes_only_activity" TurnCoverageTurnIncludesAllInput TurnCoverage = "turn_includes_all_input" TurnCoverageTurnIncludesAudioActivity TurnCoverage = "turn_includes_audio_activity" )
type WorkflowConfig ¶
type WorkflowConfig struct {
SequenceID string
Completion string
MaxTurns int
BasePrompt string
Steps []WorkflowStep
// InitialStep selects the first active step. Out-of-range values fall
// back to zero.
InitialStep int
}
WorkflowConfig describes a deterministic, step-based Voice Agent behavior sequence. Durations are intentionally expressed as turns instead of wall clock time so tests and local installs can exercise long moderation flows quickly.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package livecontract provides reusable conformance checks for LiveProvider implementations.
|
Package livecontract provides reusable conformance checks for LiveProvider implementations. |