Documentation
¶
Overview ¶
Package live implements bidirectional voice ("live") sessions: the protocol client for the voice model, and a controller that turns the voice model's delegation requests into ordinary agent turns.
The package is host-agnostic. A host supplies the media peer (browser WebRTC for direct providers or a bounded PCM bridge for proxied providers) and a Delegator that runs delegated work; nothing here knows about HTTP handlers or a terminal UI.
Index ¶
- Constants
- Variables
- func CapabilityContext(capabilities Capabilities) string
- func ChunkText(text string, limit int) []string
- func DelegationPrompt(input, transcriptDelta string) string
- func ParseCallID(location string) string
- func SessionJSON(cfg config.LiveConfig, opts SessionOptions) (json.RawMessage, error)
- type Auth
- type AuthFunc
- type CallRequest
- type CallResponse
- type Capabilities
- type ChatGPTProvider
- type Controller
- type ControllerOptions
- type DelegationChunk
- type DelegationCompletionSession
- type DelegationRequest
- type DelegationState
- type Delegator
- type Event
- type EventKind
- type GeminiProvider
- type InitialItem
- type OpenAIProvider
- type Outbound
- type PCMFrame
- type PCMSession
- type Provider
- type Session
- type SessionOptions
- type SteeringDelegator
- type Update
- type UpdateKind
- type VoiceSession
Constants ¶
const ( RoleUser = "user" RoleAssistant = "assistant" )
Roles used by transcript and turn events.
const ( ChannelSpeakable = "speakable" ChannelCommentary = "commentary" )
Channels accepted by outbound context appends.
const DefaultInstructions = `` /* 1446-byte string literal not displayed */
DefaultInstructions is the default prompt for term-llm's conversational voice surface.
const ExecutionInstructions = `` /* 282-byte string literal not displayed */
ExecutionInstructions describes how the ordinary agent should handle work originating from a live voice turn.
const MaxAppendBytes = 500
MaxAppendBytes is the largest UTF-8 payload a single context append carries.
Variables ¶
var ErrDelegationBusy = errors.New("live: session is busy")
ErrDelegationBusy tells the controller the host could not start the turn because another turn owns the chat session. The delegation stays queued and is retried.
var ErrForbidden = errors.New("live: access denied")
ErrForbidden reports that the provider refused the requested live session. This can indicate account access or an incompatible voice/protocol pairing, not just invalid credentials. Surface the message without refreshing tokens.
ErrUnauthorized reports a stale or invalid token, so the caller may refresh the credentials and retry once.
Functions ¶
func CapabilityContext ¶
func CapabilityContext(capabilities Capabilities) string
CapabilityContext describes authoritative, credential-free host facts for the voice model. It is intended for SessionOptions.Context.
func ChunkText ¶
ChunkText splits text into pieces of at most limit UTF-8 bytes without splitting a rune. Empty input yields no chunks.
func DelegationPrompt ¶
DelegationPrompt returns only the bounded XML wrapper used to pass structured live input to an executing agent. Input keeps its head and transcript context keeps its tail; each escaped field is at most 4 KiB.
func ParseCallID ¶
ParseCallID extracts the call id from a Location header. The provider returns paths such as /v1/live/rtc_abc or /v1/live/<uuid>.
func SessionJSON ¶
func SessionJSON(cfg config.LiveConfig, opts SessionOptions) (json.RawMessage, error)
SessionJSON builds the session object sent with a call creation request.
Types ¶
type Auth ¶
type Auth struct {
AccessToken string
AccountID string
// Headers carries the shared client identity (originator, User-Agent).
Headers map[string]string
}
Auth carries the identity used for call creation and the control channel.
type AuthFunc ¶
AuthFunc resolves credentials. refresh asks for a forced token refresh after the provider rejected the previous token.
type CallRequest ¶
type CallRequest struct {
SDP string `json:"sdp"`
Session json.RawMessage `json:"session"`
}
CallRequest is the body of a realtime call creation request.
type CallResponse ¶
CallResponse is the provider's answer to a call creation request.
func CreateCall ¶
func CreateCall(ctx context.Context, client *http.Client, baseURL string, auth Auth, sessionID string, request CallRequest) (CallResponse, error)
CreateCall exchanges an SDP offer for the provider's answer and call id.
type Capabilities ¶
type Capabilities struct {
Provider string `json:"provider"`
Model string `json:"model"`
Voice string `json:"voice"`
Voices []string `json:"voices"`
CanSetVoice bool `json:"can_set_voice"`
}
Capabilities is a credential-free snapshot of the configured live voice transport. Hosts can expose it directly as JSON.
func ConfigCapabilities ¶
func ConfigCapabilities(cfg config.LiveConfig) Capabilities
ConfigCapabilities resolves the live settings that apply to a new call.
type ChatGPTProvider ¶
type ChatGPTProvider struct {
// contains filtered or unexported fields
}
ChatGPTProvider runs gpt-live over the ChatGPT backend using the stored OAuth session. The browser owns the media path; this side owns auth, the control channel, and delegation.
func NewChatGPTProvider ¶
func NewChatGPTProvider(cfg config.LiveConfig, auth AuthFunc, client *http.Client) *ChatGPTProvider
NewChatGPTProvider builds the ChatGPT live provider. A nil auth uses the stored OAuth credentials; a nil client uses a default with a call timeout.
func (*ChatGPTProvider) Name ¶
func (p *ChatGPTProvider) Name() string
Name reports the provider name.
func (*ChatGPTProvider) Ready ¶
func (p *ChatGPTProvider) Ready(ctx context.Context) error
Ready reports whether a live session could be started right now.
func (*ChatGPTProvider) Start ¶
func (p *ChatGPTProvider) Start(ctx context.Context, offerSDP string, opts SessionOptions) (Session, error)
Start creates the call and joins the control channel.
type Controller ¶
type Controller struct {
// contains filtered or unexported fields
}
Controller owns the live conversation state: transcripts, the delegation queue, and streaming delegated output back to the voice model.
func NewController ¶
func NewController(opts ControllerOptions) *Controller
NewController builds a controller for an open session.
func (*Controller) AppendUserText ¶
func (c *Controller) AppendUserText(ctx context.Context, text string) error
AppendUserText injects text the user typed instead of speaking.
func (*Controller) Close ¶
func (c *Controller) Close(ctx context.Context) error
Close ends the provider session and waits for both loops to finish.
func (*Controller) Start ¶
func (c *Controller) Start(ctx context.Context)
Start runs the event and delegation loops until the session ends or Close is called.
type ControllerOptions ¶
type ControllerOptions struct {
Session Session
Delegator Delegator
// Observer receives host-visible updates. It must not block for long.
Observer func(Update)
// FlushInterval batches streamed agent text before it is appended.
FlushInterval time.Duration
// BusyRetry is the delay between retries when the chat session is busy.
BusyRetry time.Duration
// BusyRetries caps how many times a delegation waits for a busy session.
BusyRetries int
}
ControllerOptions configures a Controller.
type DelegationChunk ¶
DelegationChunk is a piece of delegated-turn output sent back to the voice model. Channel is ChannelSpeakable or ChannelCommentary.
type DelegationCompletionSession ¶
type DelegationCompletionSession interface {
CompleteDelegation(ctx context.Context, delegationID string) error
}
DelegationCompletionSession is implemented by protocols requiring a complete function result rather than accepting an open-ended stream of context.
type DelegationRequest ¶
type DelegationRequest struct {
// ID is the provider's stable delegation identifier when one was supplied.
ID string
Input string
TranscriptDelta string
}
DelegationRequest is the structured input for one delegated agent turn.
type DelegationState ¶
type DelegationState string
DelegationState is the lifecycle of one delegated turn.
const ( // DelegationQueued means the request is waiting for the runner. DelegationQueued DelegationState = "queued" // DelegationRunning means the chat session is executing it. DelegationRunning DelegationState = "running" // DelegationDone means the turn finished successfully. DelegationDone DelegationState = "done" // DelegationFailed means the turn failed or was cancelled. DelegationFailed DelegationState = "failed" )
type Delegator ¶
type Delegator interface {
Run(ctx context.Context, request DelegationRequest, emit func(DelegationChunk)) error
}
Delegator runs delegated work in the bound chat session. Run streams output back through emit and returns when the turn finishes.
type Event ¶
type Event struct {
Kind EventKind
// RawType preserves the wire type, mainly for Unknown events.
RawType string
Role string
Text string
// Voice is the acknowledged output voice for EventSessionUpdated.
Voice string
// ErrorHandled means an operation caller already received this error. Keep
// it on the event stream without also failing the conversational UI.
ErrorHandled bool
// DelegationID identifies the delegation item for EventDelegationCreated.
DelegationID string
}
Event is a provider-neutral live event.
func ParseEvent ¶
ParseEvent normalises one frameless protocol frame. Unrecognised frames are reported as EventUnknown rather than an error so new server events never break a running session.
type EventKind ¶
type EventKind string
EventKind enumerates the normalised inbound events a live provider reports.
const ( // EventSessionStarted reports that the provider accepted the session. EventSessionStarted EventKind = "session.started" // EventSessionUpdated reports a session object change. EventSessionUpdated EventKind = "session.updated" // EventUserTranscript carries a delta of the user's speech. EventUserTranscript EventKind = "user.transcript" // EventUserTranscriptInterim carries a replaceable speech-recognition preview. // It must never be appended to authoritative transcript or delegation context. EventUserTranscriptInterim EventKind = "user.transcript.interim" // EventAssistantTranscript carries a delta of the model's speech. EventAssistantTranscript EventKind = "assistant.transcript" // EventTurnDone reports a completed turn with its full transcript. EventTurnDone EventKind = "turn.done" // EventDelegationCreated asks the host to run real work. EventDelegationCreated EventKind = "delegation.created" // EventInterrupted reports that provider-side barge-in stopped the current // model audio response. PCM transports use it to flush queued playback. EventInterrupted EventKind = "interrupted" // EventError carries a provider-reported error. EventError EventKind = "error" // EventEnded reports that the provider session finished. EventEnded EventKind = "ended" // EventUnknown is any event we deliberately ignore. EventUnknown EventKind = "unknown" )
type GeminiProvider ¶
type GeminiProvider struct {
// contains filtered or unexported fields
}
GeminiProvider runs Gemini Live over a server-owned BidiGenerateContent WebSocket. Browser audio is proxied as PCM and the configured API key never leaves the server.
func NewGeminiProvider ¶
func NewGeminiProvider(cfg config.LiveConfig) *GeminiProvider
NewGeminiProvider builds the Gemini Live provider.
func (*GeminiProvider) Name ¶
func (p *GeminiProvider) Name() string
func (*GeminiProvider) Start ¶
func (p *GeminiProvider) Start(ctx context.Context, _ string, opts SessionOptions) (Session, error)
type InitialItem ¶
InitialItem seeds a live session with one prior message.
type OpenAIProvider ¶
type OpenAIProvider struct {
// contains filtered or unexported fields
}
OpenAIProvider runs the public OpenAI GPT-Live or legacy Realtime API with WebRTC media and a server-owned sideband WebSocket. It intentionally does not use ChatGPT OAuth or proprietary headers.
func NewOpenAIProvider ¶
func NewOpenAIProvider(cfg config.LiveConfig, client *http.Client) *OpenAIProvider
NewOpenAIProvider builds the public OpenAI live provider.
func (*OpenAIProvider) Name ¶
func (p *OpenAIProvider) Name() string
Name reports the provider name.
func (*OpenAIProvider) Ready ¶
func (p *OpenAIProvider) Ready(ctx context.Context) error
Ready reports whether an API key is available without making a network call.
func (*OpenAIProvider) Start ¶
func (p *OpenAIProvider) Start(ctx context.Context, offerSDP string, opts SessionOptions) (Session, error)
Start routes explicitly configured Realtime models through the legacy Realtime API. All other models, including the default gpt-live-1, use the public GPT-Live transport.
type Outbound ¶
type Outbound struct {
Type string `json:"type"`
DelegationItemID string `json:"delegation_item_id,omitempty"`
Channel string `json:"channel,omitempty"`
Content []wireContent `json:"content,omitempty"`
Session json.RawMessage `json:"session,omitempty"`
}
Outbound is a message sent up the control channel.
func DelegationContextAppend ¶
DelegationContextAppend continues a delegation with more agent output.
func SessionClose ¶
func SessionClose() Outbound
SessionClose ends the session from the client side.
func SessionContextAppend ¶
SessionContextAppend injects text into the conversation outside a delegation.
func SessionUpdate ¶
func SessionUpdate(session json.RawMessage) Outbound
SessionUpdate replaces session settings mid-call.
type PCMFrame ¶
PCMFrame is provider audio or a playback-control marker. Audio is mono, signed 16-bit little-endian PCM at 24 kHz. Flush discards queued playback.
type PCMSession ¶
type PCMSession interface {
Session
SendPCM(ctx context.Context, pcm []byte) error
PCMFrames() <-chan PCMFrame
}
PCMSession is implemented by providers whose media is proxied through the term-llm server instead of negotiated directly with browser WebRTC.
type Provider ¶
type Provider interface {
// Name reports the configured provider name.
Name() string
// Ready reports whether a session could be started right now.
Ready(ctx context.Context) error
// Start exchanges the caller's SDP offer for a provider answer and opens
// the control channel.
Start(ctx context.Context, offerSDP string, opts SessionOptions) (Session, error)
}
Provider creates live sessions for one vendor.
func NewProvider ¶
func NewProvider(cfg config.LiveConfig) (Provider, error)
NewProvider builds the configured live provider.
func NewProviderWithClient ¶
NewProviderWithClient builds the configured live provider with an explicit HTTP client. Tests use it to point at an in-process fake.
type Session ¶
type Session interface {
// AnswerSDP returns the provider's SDP answer for the media peer.
AnswerSDP() string
// Events streams normalised provider events until the session ends.
Events() <-chan Event
// AppendDelegation streams delegated-turn output back to the voice model.
AppendDelegation(ctx context.Context, delegationID string, chunk DelegationChunk) error
// AppendText injects text into the conversation outside a delegation.
AppendText(ctx context.Context, text string) error
// Close ends the session.
Close(ctx context.Context) error
}
Session is one open live call.
type SessionOptions ¶
type SessionOptions struct {
// SessionID is the chat session the live call is bound to. It is sent as
// x-session-id so provider-side logs line up with ours.
SessionID string
// Instructions replaces the default voice-model prompt when set.
Instructions string
// Context is authoritative host context appended to either the default or
// custom instructions. It is kept separate so a custom prompt cannot erase
// the current session capabilities supplied by the host.
Context string
// InitialItems seeds the voice model with prior conversation.
InitialItems []InitialItem
// Debug enables metadata-only live transport diagnostics. DebugRaw adds
// sanitized inbound provider events and may include transcript text.
Debug bool
DebugRaw bool
// LiveID correlates provider diagnostics with host/browser media reports.
LiveID string
}
SessionOptions carries the provider-neutral inputs for a new live session.
type SteeringDelegator ¶
type SteeringDelegator interface {
Steer(ctx context.Context, request DelegationRequest) (bool, error)
}
SteeringDelegator can admit a delegation as guidance for the canonical task already running in the bound chat session. A false result means there is no active task to steer, so the controller should start an ordinary Run.
type Update ¶
type Update struct {
Kind UpdateKind
Role string
Text string
Final bool
Interim bool // replaceable UI-only recognition preview, not transcript history
DelegationID string
State DelegationState
}
Update is one host-visible controller event.
type UpdateKind ¶
type UpdateKind string
UpdateKind classifies a controller update for the host UI.
const ( // UpdateStarted reports that the provider accepted the session. UpdateStarted UpdateKind = "started" // UpdateTranscript carries partial or final spoken text. UpdateTranscript UpdateKind = "transcript" // UpdateDelegation reports delegated-turn progress. UpdateDelegation UpdateKind = "delegation" // UpdateInterrupted tells the host to discard any buffered provider audio. UpdateInterrupted UpdateKind = "interrupted" // UpdateError carries a session error. UpdateError UpdateKind = "error" // UpdateEnded reports that the session finished. UpdateEnded UpdateKind = "ended" )
type VoiceSession ¶
type VoiceSession interface {
// CurrentVoice returns the most recently acknowledged voice, including late
// acknowledgements after a canceled SetVoice request.
CurrentVoice() string
SetVoice(ctx context.Context, voice string) error
}
VoiceSession is implemented by sessions whose provider can change the audio output voice during an authenticated current call.