live

package
v0.9.49 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 17, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package live implements bidirectional voice ("live") sessions: the protocol client for the voice model, and a controller that turns the voice model's delegation requests into ordinary agent turns.

The package is host-agnostic. A host supplies the media peer (browser WebRTC for direct providers or a bounded PCM bridge for proxied providers) and a Delegator that runs delegated work; nothing here knows about HTTP handlers or a terminal UI.

Index

Constants

View Source
const (
	RoleUser      = "user"
	RoleAssistant = "assistant"
)

Roles used by transcript and turn events.

View Source
const (
	ChannelSpeakable  = "speakable"
	ChannelCommentary = "commentary"
)

Channels accepted by outbound context appends.

View Source
const DefaultInstructions = `` /* 1446-byte string literal not displayed */

DefaultInstructions is the default prompt for term-llm's conversational voice surface.

View Source
const ExecutionInstructions = `` /* 282-byte string literal not displayed */

ExecutionInstructions describes how the ordinary agent should handle work originating from a live voice turn.

View Source
const MaxAppendBytes = 500

MaxAppendBytes is the largest UTF-8 payload a single context append carries.

Variables

View Source
var ErrDelegationBusy = errors.New("live: session is busy")

ErrDelegationBusy tells the controller the host could not start the turn because another turn owns the chat session. The delegation stays queued and is retried.

View Source
var ErrForbidden = errors.New("live: access denied")

ErrForbidden reports that the provider refused the requested live session. This can indicate account access or an incompatible voice/protocol pairing, not just invalid credentials. Surface the message without refreshing tokens.

View Source
var ErrUnauthorized = errors.New("live: unauthorized")

ErrUnauthorized reports a stale or invalid token, so the caller may refresh the credentials and retry once.

Functions

func CapabilityContext

func CapabilityContext(capabilities Capabilities) string

CapabilityContext describes authoritative, credential-free host facts for the voice model. It is intended for SessionOptions.Context.

func ChunkText

func ChunkText(text string, limit int) []string

ChunkText splits text into pieces of at most limit UTF-8 bytes without splitting a rune. Empty input yields no chunks.

func DelegationPrompt

func DelegationPrompt(input, transcriptDelta string) string

DelegationPrompt returns only the bounded XML wrapper used to pass structured live input to an executing agent. Input keeps its head and transcript context keeps its tail; each escaped field is at most 4 KiB.

func ParseCallID

func ParseCallID(location string) string

ParseCallID extracts the call id from a Location header. The provider returns paths such as /v1/live/rtc_abc or /v1/live/<uuid>.

func SessionJSON

func SessionJSON(cfg config.LiveConfig, opts SessionOptions) (json.RawMessage, error)

SessionJSON builds the session object sent with a call creation request.

Types

type Auth

type Auth struct {
	AccessToken string
	AccountID   string
	// Headers carries the shared client identity (originator, User-Agent).
	Headers map[string]string
}

Auth carries the identity used for call creation and the control channel.

func ChatGPTAuth

func ChatGPTAuth(_ context.Context, refresh bool) (Auth, error)

ChatGPTAuth resolves the stored OAuth session, refreshing when the token is expired or the provider rejected it.

type AuthFunc

type AuthFunc func(ctx context.Context, refresh bool) (Auth, error)

AuthFunc resolves credentials. refresh asks for a forced token refresh after the provider rejected the previous token.

type CallRequest

type CallRequest struct {
	SDP     string          `json:"sdp"`
	Session json.RawMessage `json:"session"`
}

CallRequest is the body of a realtime call creation request.

type CallResponse

type CallResponse struct {
	AnswerSDP string
	CallID    string
}

CallResponse is the provider's answer to a call creation request.

func CreateCall

func CreateCall(ctx context.Context, client *http.Client, baseURL string, auth Auth, sessionID string, request CallRequest) (CallResponse, error)

CreateCall exchanges an SDP offer for the provider's answer and call id.

type Capabilities

type Capabilities struct {
	Provider    string   `json:"provider"`
	Model       string   `json:"model"`
	Voice       string   `json:"voice"`
	Voices      []string `json:"voices"`
	CanSetVoice bool     `json:"can_set_voice"`
}

Capabilities is a credential-free snapshot of the configured live voice transport. Hosts can expose it directly as JSON.

func ConfigCapabilities

func ConfigCapabilities(cfg config.LiveConfig) Capabilities

ConfigCapabilities resolves the live settings that apply to a new call.

type ChatGPTProvider

type ChatGPTProvider struct {
	// contains filtered or unexported fields
}

ChatGPTProvider runs gpt-live over the ChatGPT backend using the stored OAuth session. The browser owns the media path; this side owns auth, the control channel, and delegation.

func NewChatGPTProvider

func NewChatGPTProvider(cfg config.LiveConfig, auth AuthFunc, client *http.Client) *ChatGPTProvider

NewChatGPTProvider builds the ChatGPT live provider. A nil auth uses the stored OAuth credentials; a nil client uses a default with a call timeout.

func (*ChatGPTProvider) Name

func (p *ChatGPTProvider) Name() string

Name reports the provider name.

func (*ChatGPTProvider) Ready

func (p *ChatGPTProvider) Ready(ctx context.Context) error

Ready reports whether a live session could be started right now.

func (*ChatGPTProvider) Start

func (p *ChatGPTProvider) Start(ctx context.Context, offerSDP string, opts SessionOptions) (Session, error)

Start creates the call and joins the control channel.

type Controller

type Controller struct {
	// contains filtered or unexported fields
}

Controller owns the live conversation state: transcripts, the delegation queue, and streaming delegated output back to the voice model.

func NewController

func NewController(opts ControllerOptions) *Controller

NewController builds a controller for an open session.

func (*Controller) AppendUserText

func (c *Controller) AppendUserText(ctx context.Context, text string) error

AppendUserText injects text the user typed instead of speaking.

func (*Controller) Close

func (c *Controller) Close(ctx context.Context) error

Close ends the provider session and waits for both loops to finish.

func (*Controller) Start

func (c *Controller) Start(ctx context.Context)

Start runs the event and delegation loops until the session ends or Close is called.

type ControllerOptions

type ControllerOptions struct {
	Session   Session
	Delegator Delegator
	// Observer receives host-visible updates. It must not block for long.
	Observer func(Update)
	// FlushInterval batches streamed agent text before it is appended.
	FlushInterval time.Duration
	// BusyRetry is the delay between retries when the chat session is busy.
	BusyRetry time.Duration
	// BusyRetries caps how many times a delegation waits for a busy session.
	BusyRetries int
}

ControllerOptions configures a Controller.

type DelegationChunk

type DelegationChunk struct {
	Text    string
	Channel string
}

DelegationChunk is a piece of delegated-turn output sent back to the voice model. Channel is ChannelSpeakable or ChannelCommentary.

type DelegationCompletionSession

type DelegationCompletionSession interface {
	CompleteDelegation(ctx context.Context, delegationID string) error
}

DelegationCompletionSession is implemented by protocols requiring a complete function result rather than accepting an open-ended stream of context.

type DelegationRequest

type DelegationRequest struct {
	// ID is the provider's stable delegation identifier when one was supplied.
	ID              string
	Input           string
	TranscriptDelta string
}

DelegationRequest is the structured input for one delegated agent turn.

type DelegationState

type DelegationState string

DelegationState is the lifecycle of one delegated turn.

const (
	// DelegationQueued means the request is waiting for the runner.
	DelegationQueued DelegationState = "queued"
	// DelegationRunning means the chat session is executing it.
	DelegationRunning DelegationState = "running"
	// DelegationDone means the turn finished successfully.
	DelegationDone DelegationState = "done"
	// DelegationFailed means the turn failed or was cancelled.
	DelegationFailed DelegationState = "failed"
)

type Delegator

type Delegator interface {
	Run(ctx context.Context, request DelegationRequest, emit func(DelegationChunk)) error
}

Delegator runs delegated work in the bound chat session. Run streams output back through emit and returns when the turn finishes.

type Event

type Event struct {
	Kind EventKind
	// RawType preserves the wire type, mainly for Unknown events.
	RawType string
	Role    string
	Text    string
	// Voice is the acknowledged output voice for EventSessionUpdated.
	Voice string
	// ErrorHandled means an operation caller already received this error. Keep
	// it on the event stream without also failing the conversational UI.
	ErrorHandled bool
	// DelegationID identifies the delegation item for EventDelegationCreated.
	DelegationID string
}

Event is a provider-neutral live event.

func ParseEvent

func ParseEvent(data []byte) (Event, error)

ParseEvent normalises one frameless protocol frame. Unrecognised frames are reported as EventUnknown rather than an error so new server events never break a running session.

type EventKind

type EventKind string

EventKind enumerates the normalised inbound events a live provider reports.

const (
	// EventSessionStarted reports that the provider accepted the session.
	EventSessionStarted EventKind = "session.started"
	// EventSessionUpdated reports a session object change.
	EventSessionUpdated EventKind = "session.updated"
	// EventUserTranscript carries a delta of the user's speech.
	EventUserTranscript EventKind = "user.transcript"
	// EventUserTranscriptInterim carries a replaceable speech-recognition preview.
	// It must never be appended to authoritative transcript or delegation context.
	EventUserTranscriptInterim EventKind = "user.transcript.interim"
	// EventAssistantTranscript carries a delta of the model's speech.
	EventAssistantTranscript EventKind = "assistant.transcript"
	// EventTurnDone reports a completed turn with its full transcript.
	EventTurnDone EventKind = "turn.done"
	// EventDelegationCreated asks the host to run real work.
	EventDelegationCreated EventKind = "delegation.created"
	// EventInterrupted reports that provider-side barge-in stopped the current
	// model audio response. PCM transports use it to flush queued playback.
	EventInterrupted EventKind = "interrupted"
	// EventError carries a provider-reported error.
	EventError EventKind = "error"
	// EventEnded reports that the provider session finished.
	EventEnded EventKind = "ended"
	// EventUnknown is any event we deliberately ignore.
	EventUnknown EventKind = "unknown"
)

type GeminiProvider

type GeminiProvider struct {
	// contains filtered or unexported fields
}

GeminiProvider runs Gemini Live over a server-owned BidiGenerateContent WebSocket. Browser audio is proxied as PCM and the configured API key never leaves the server.

func NewGeminiProvider

func NewGeminiProvider(cfg config.LiveConfig) *GeminiProvider

NewGeminiProvider builds the Gemini Live provider.

func (*GeminiProvider) Name

func (p *GeminiProvider) Name() string

func (*GeminiProvider) Ready

func (p *GeminiProvider) Ready(ctx context.Context) error

func (*GeminiProvider) Start

func (p *GeminiProvider) Start(ctx context.Context, _ string, opts SessionOptions) (Session, error)

type InitialItem

type InitialItem struct {
	Role string
	Text string
}

InitialItem seeds a live session with one prior message.

type OpenAIProvider

type OpenAIProvider struct {
	// contains filtered or unexported fields
}

OpenAIProvider runs the public OpenAI GPT-Live or legacy Realtime API with WebRTC media and a server-owned sideband WebSocket. It intentionally does not use ChatGPT OAuth or proprietary headers.

func NewOpenAIProvider

func NewOpenAIProvider(cfg config.LiveConfig, client *http.Client) *OpenAIProvider

NewOpenAIProvider builds the public OpenAI live provider.

func (*OpenAIProvider) Name

func (p *OpenAIProvider) Name() string

Name reports the provider name.

func (*OpenAIProvider) Ready

func (p *OpenAIProvider) Ready(ctx context.Context) error

Ready reports whether an API key is available without making a network call.

func (*OpenAIProvider) Start

func (p *OpenAIProvider) Start(ctx context.Context, offerSDP string, opts SessionOptions) (Session, error)

Start routes explicitly configured Realtime models through the legacy Realtime API. All other models, including the default gpt-live-1, use the public GPT-Live transport.

type Outbound

type Outbound struct {
	Type             string          `json:"type"`
	DelegationItemID string          `json:"delegation_item_id,omitempty"`
	Channel          string          `json:"channel,omitempty"`
	Content          []wireContent   `json:"content,omitempty"`
	Session          json.RawMessage `json:"session,omitempty"`
}

Outbound is a message sent up the control channel.

func DelegationContextAppend

func DelegationContextAppend(delegationID, channel, text string) Outbound

DelegationContextAppend continues a delegation with more agent output.

func SessionClose

func SessionClose() Outbound

SessionClose ends the session from the client side.

func SessionContextAppend

func SessionContextAppend(channel, text string) Outbound

SessionContextAppend injects text into the conversation outside a delegation.

func SessionUpdate

func SessionUpdate(session json.RawMessage) Outbound

SessionUpdate replaces session settings mid-call.

type PCMFrame

type PCMFrame struct {
	Audio []byte
	Flush bool
}

PCMFrame is provider audio or a playback-control marker. Audio is mono, signed 16-bit little-endian PCM at 24 kHz. Flush discards queued playback.

type PCMSession

type PCMSession interface {
	Session
	SendPCM(ctx context.Context, pcm []byte) error
	PCMFrames() <-chan PCMFrame
}

PCMSession is implemented by providers whose media is proxied through the term-llm server instead of negotiated directly with browser WebRTC.

type Provider

type Provider interface {
	// Name reports the configured provider name.
	Name() string
	// Ready reports whether a session could be started right now.
	Ready(ctx context.Context) error
	// Start exchanges the caller's SDP offer for a provider answer and opens
	// the control channel.
	Start(ctx context.Context, offerSDP string, opts SessionOptions) (Session, error)
}

Provider creates live sessions for one vendor.

func NewProvider

func NewProvider(cfg config.LiveConfig) (Provider, error)

NewProvider builds the configured live provider.

func NewProviderWithClient

func NewProviderWithClient(cfg config.LiveConfig, client *http.Client) (Provider, error)

NewProviderWithClient builds the configured live provider with an explicit HTTP client. Tests use it to point at an in-process fake.

type Session

type Session interface {
	// AnswerSDP returns the provider's SDP answer for the media peer.
	AnswerSDP() string
	// Events streams normalised provider events until the session ends.
	Events() <-chan Event
	// AppendDelegation streams delegated-turn output back to the voice model.
	AppendDelegation(ctx context.Context, delegationID string, chunk DelegationChunk) error
	// AppendText injects text into the conversation outside a delegation.
	AppendText(ctx context.Context, text string) error
	// Close ends the session.
	Close(ctx context.Context) error
}

Session is one open live call.

type SessionOptions

type SessionOptions struct {
	// SessionID is the chat session the live call is bound to. It is sent as
	// x-session-id so provider-side logs line up with ours.
	SessionID string
	// Instructions replaces the default voice-model prompt when set.
	Instructions string
	// Context is authoritative host context appended to either the default or
	// custom instructions. It is kept separate so a custom prompt cannot erase
	// the current session capabilities supplied by the host.
	Context string
	// InitialItems seeds the voice model with prior conversation.
	InitialItems []InitialItem
	// Debug enables metadata-only live transport diagnostics. DebugRaw adds
	// sanitized inbound provider events and may include transcript text.
	Debug    bool
	DebugRaw bool
	// LiveID correlates provider diagnostics with host/browser media reports.
	LiveID string
}

SessionOptions carries the provider-neutral inputs for a new live session.

type SteeringDelegator

type SteeringDelegator interface {
	Steer(ctx context.Context, request DelegationRequest) (bool, error)
}

SteeringDelegator can admit a delegation as guidance for the canonical task already running in the bound chat session. A false result means there is no active task to steer, so the controller should start an ordinary Run.

type Update

type Update struct {
	Kind         UpdateKind
	Role         string
	Text         string
	Final        bool
	Interim      bool // replaceable UI-only recognition preview, not transcript history
	DelegationID string
	State        DelegationState
}

Update is one host-visible controller event.

type UpdateKind

type UpdateKind string

UpdateKind classifies a controller update for the host UI.

const (
	// UpdateStarted reports that the provider accepted the session.
	UpdateStarted UpdateKind = "started"
	// UpdateTranscript carries partial or final spoken text.
	UpdateTranscript UpdateKind = "transcript"
	// UpdateDelegation reports delegated-turn progress.
	UpdateDelegation UpdateKind = "delegation"
	// UpdateInterrupted tells the host to discard any buffered provider audio.
	UpdateInterrupted UpdateKind = "interrupted"
	// UpdateError carries a session error.
	UpdateError UpdateKind = "error"
	// UpdateEnded reports that the session finished.
	UpdateEnded UpdateKind = "ended"
)

type VoiceSession

type VoiceSession interface {
	// CurrentVoice returns the most recently acknowledged voice, including late
	// acknowledgements after a canceled SetVoice request.
	CurrentVoice() string
	SetVoice(ctx context.Context, voice string) error
}

VoiceSession is implemented by sessions whose provider can change the audio output voice during an authenticated current call.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL