llm

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 30, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Index

Constants

View Source
const (
	ThinkTagStart = "<think>"
	ThinkTagEnd   = "</think>"
)
View Source
const ConfirmDuplicateParam = "lk_agents_confirm_duplicate"
View Source
const MaxFunctionArgumentsBytes = 1 << 20

Variables

View Source
var ErrRunContextUnavailable = errors.New("tool run context is not attached to a voice runtime")

Functions

func ChatItemToJSON

func ChatItemToJSON(item ChatItem, excludeTimestamp bool) ([]byte, error)

ChatItemToJSON serializes a chat item with its native camelCase wire names.

func FormatChatHistory

func FormatChatHistory(chatContext *ChatContext, options FormatChatHistoryOptions) string

FormatChatHistory renders a stable, human-readable diagnostic view. It is intended for logs and deliberately avoids serializing frame payloads.

func HasResponse

func HasResponse(chunk ChatChunk) bool

func ParseFunctionArguments

func ParseFunctionArguments(raw any) (map[string]any, error)

ParseFunctionArguments accepts a JSON string or an already-decoded object. It first uses strict JSON, then applies a conservative repair pass for the malformed object syntax commonly produced by streaming language models (unquoted keys, single-quoted strings, trailing commas, and omitted closing delimiters). Template tokens are stripped only on the repaired path, so a legitimate strict-JSON value such as "<|safe|>" remains untouched.

func RunWithRetry

func RunWithRetry(ctx context.Context, base *Base, options agents.APIConnectOptions, operation func(context.Context, int) error) error

RunWithRetry centralizes provider retry behavior and error events.

func UpdateRealtimeSessionBestEffort

func UpdateRealtimeSessionBestEffort(ctx context.Context, session RealtimeSession, update RealtimeSessionUpdate)

UpdateRealtimeSessionBestEffort preserves the JS compatibility behavior: each update is attempted in order, failures are logged, and later updates still run. Providers should use the individual methods when errors matter.

Types

type AgentConfigUpdate

type AgentConfigUpdate struct {
	ID           string
	Instructions *Instructions
	ToolsAdded   []string
	ToolsRemoved []string
	CreatedAt    time.Time
}

func (*AgentConfigUpdate) ItemCreatedAt

func (a *AgentConfigUpdate) ItemCreatedAt() time.Time

func (*AgentConfigUpdate) ItemID

func (a *AgentConfigUpdate) ItemID() string

func (*AgentConfigUpdate) ItemType

func (a *AgentConfigUpdate) ItemType() ItemType

func (*AgentConfigUpdate) MarshalJSON

func (a *AgentConfigUpdate) MarshalJSON() ([]byte, error)

type AgentHandoff

type AgentHandoff struct {
	Agent   any
	Returns any
}

func Handoff

func Handoff(agent, returns any) AgentHandoff

type AgentHandoffItem

type AgentHandoffItem struct {
	ID         string
	OldAgentID string
	NewAgentID string
	CreatedAt  time.Time
}

func (*AgentHandoffItem) ItemCreatedAt

func (a *AgentHandoffItem) ItemCreatedAt() time.Time

func (*AgentHandoffItem) ItemID

func (a *AgentHandoffItem) ItemID() string

func (*AgentHandoffItem) ItemType

func (a *AgentHandoffItem) ItemType() ItemType

func (*AgentHandoffItem) MarshalJSON

func (a *AgentHandoffItem) MarshalJSON() ([]byte, error)

type AsyncToolOptions

type AsyncToolOptions struct {
	UpdateTemplate                string
	UpdateTemplateSet             bool
	UpdateTemplateFunc            func(UpdatePromptArgs) string
	DuplicateRejectTemplate       string
	DuplicateRejectTemplateSet    bool
	DuplicateRejectTemplateFunc   func(DuplicatePromptArgs) string
	DuplicateConfirmTemplate      string
	DuplicateConfirmTemplateSet   bool
	DuplicateConfirmTemplateFunc  func(DuplicatePromptArgs) string
	ReplyAtTailTemplate           string
	ReplyAtTailTemplateSet        bool
	ReplyAtTailTemplateFunc       func(ReplyPromptArgs) string
	ReplyMaybeCoveredTemplate     string
	ReplyMaybeCoveredTemplateSet  bool
	ReplyMaybeCoveredTemplateFunc func(ReplyPromptArgs) string
}

AsyncToolOptions contains the conversational templates used when a tool detaches from its originating reply after reporting progress. Empty fields inherit the voice session/activity defaults.

type AsyncToolset

type AsyncToolset struct {
	*Toolset
	// contains filtered or unexported fields
}

AsyncToolset is a Toolset marker with an independent execution scope. The voice package supplies the executor so llm remains free of an import cycle.

func MustAsyncToolset

func MustAsyncToolset(options AsyncToolsetOptions) *AsyncToolset

func NewAsyncToolset

func NewAsyncToolset(options AsyncToolsetOptions) (*AsyncToolset, error)

func (*AsyncToolset) ToolHandling

func (t *AsyncToolset) ToolHandling() *AsyncToolOptions

ToolHandling returns an independent copy of this scope's option override.

type AsyncToolsetOptions

type AsyncToolsetOptions struct {
	ID           string
	Tools        []Tool
	Setup        func(context.Context, ToolsetContext) error
	Close        func(context.Context) error
	ToolHandling *AsyncToolOptions
}

AsyncToolsetOptions creates an executor scope. Tools in a session-level AsyncToolset share a session-lifetime executor and therefore survive agent handoffs; agent-level sets are scoped to that activity.

type AudioContent

type AudioContent struct {
	Frames     []agents.AudioFrame
	Transcript string
}

type AvailabilityChangedEvent

type AvailabilityChangedEvent struct {
	LLM       LLM
	Available bool
}

AvailabilityChangedEvent reports a provider entering or leaving the fallback rotation. Availability is advisory: when every provider is marked unavailable, an adapter still tries all of them in order.

type Base

type Base struct {
	// contains filtered or unexported fields
}

func NewBase

func NewBase(label, provider, model string) *Base

func (*Base) Close

func (b *Base) Close(ctx context.Context) error

func (*Base) EmitError

func (b *Base) EmitError(event ErrorEvent)

func (*Base) EmitMetrics

func (b *Base) EmitMetrics(metric metrics.LLM)

func (*Base) Label

func (b *Base) Label() string

func (*Base) Model

func (b *Base) Model() string

func (*Base) OnError

func (b *Base) OnError(fn func(ErrorEvent)) func()

func (*Base) OnMetrics

func (b *Base) OnMetrics(fn func(metrics.LLM)) func()

func (*Base) Prewarm

func (b *Base) Prewarm(parent context.Context)

func (*Base) Provider

func (b *Base) Provider() string

func (*Base) SetModel

func (b *Base) SetModel(model string)

func (*Base) SetPrewarm

func (b *Base) SetPrewarm(fn func(context.Context) error)

type BaseStream

type BaseStream struct {
	// contains filtered or unexported fields
}

func NewBaseStream

func NewBaseStream(parent context.Context, base *Base, options ChatOptions, capacity int) *BaseStream

func (*BaseStream) ChatContext

func (s *BaseStream) ChatContext() *ChatContext

func (*BaseStream) Close

func (s *BaseStream) Close() error

func (*BaseStream) Collect

func (s *BaseStream) Collect(ctx context.Context) (CollectedResponse, error)

func (*BaseStream) Context

func (s *BaseStream) Context() context.Context

func (*BaseStream) Emit

func (s *BaseStream) Emit(ctx context.Context, chunk ChatChunk) error

func (*BaseStream) Finish

func (s *BaseStream) Finish(err error)

func (*BaseStream) Recv

func (s *BaseStream) Recv(ctx context.Context) (ChatChunk, error)

func (*BaseStream) ToolContext

func (s *BaseStream) ToolContext() *Context

type ChatChunk

type ChatChunk struct {
	ID    string
	Delta *ChoiceDelta
	Usage *CompletionUsage
}

type ChatContext

type ChatContext struct {
	// contains filtered or unexported fields
}

func EmptyChatContext

func EmptyChatContext() *ChatContext

func NewChatContext

func NewChatContext(items ...ChatItem) *ChatContext

func (*ChatContext) AddMessage

func (c *ChatContext) AddMessage(role ChatRole, text string) (*ChatMessage, error)

func (*ChatContext) AsReadonly

func (c *ChatContext) AsReadonly() *ChatContext

func (*ChatContext) Copy

func (c *ChatContext) Copy(options CopyOptions) *ChatContext

func (*ChatContext) GetByID

func (c *ChatContext) GetByID(id string) (ChatItem, bool)

func (*ChatContext) IndexByID

func (c *ChatContext) IndexByID(id string) (int, bool)

func (*ChatContext) Insert

func (c *ChatContext) Insert(items ...ChatItem) error

func (*ChatContext) IsEquivalent

func (c *ChatContext) IsEquivalent(other *ChatContext) bool

func (*ChatContext) Items

func (c *ChatContext) Items() []ChatItem

func (*ChatContext) Len

func (c *ChatContext) Len() int

func (*ChatContext) MarshalJSON

func (c *ChatContext) MarshalJSON() ([]byte, error)

MarshalJSON uses the same defaults as ChatContext.toJSON() in agents-js.

func (*ChatContext) Merge

func (c *ChatContext) Merge(other *ChatContext, options CopyOptions) *ChatContext

func (*ChatContext) Readonly

func (c *ChatContext) Readonly() bool

func (*ChatContext) Remove

func (c *ChatContext) Remove(id string) bool

func (*ChatContext) ToJSON

func (c *ChatContext) ToJSON(options ...ChatContextJSONOptions) ([]byte, error)

ToJSON returns the agents-js compatible JSON representation. At most one options value may be supplied.

func (*ChatContext) Truncate

func (c *ChatContext) Truncate(maxItems int) *ChatContext

func (*ChatContext) UnmarshalJSON

func (c *ChatContext) UnmarshalJSON(data []byte) error

UnmarshalJSON accepts a complete or privacy-filtered chat context.

type ChatContextDiff

type ChatContextDiff struct {
	ToRemove []string                   `json:"toRemove"`
	ToCreate []ChatContextDiffOperation `json:"toCreate"`
	ToUpdate []ChatContextDiffOperation `json:"toUpdate"`
}

func ComputeChatContextDiff

func ComputeChatContextDiff(oldContext, newContext *ChatContext) ChatContextDiff

ComputeChatContextDiff computes a minimal set of ordered remove/create operations plus content updates for same-ID messages. Its LCS implementation uses linear auxiliary memory, avoiding the quadratic RSS spike of the JS implementation for large histories.

type ChatContextDiffOperation

type ChatContextDiffOperation struct {
	PreviousItemID string `json:"previousItemId,omitempty"`
	ItemID         string `json:"itemId"`
}

ChatContextDiffOperation identifies an item and the item after which it belongs. PreviousItemID is empty for the root of the context.

type ChatContextJSONOptions

type ChatContextJSONOptions struct {
	ExcludeImage        agents.Override[bool]
	ExcludeAudio        agents.Override[bool]
	ExcludeTimestamp    agents.Override[bool]
	ExcludeFunctionCall bool
	ExcludeConfigUpdate bool
	StripMarkup         bool
}

ChatContextJSONOptions controls the portable chat-history representation. The zero value has the same privacy-safe defaults as agents-js: images, audio, and timestamps are excluded. Use agents.Use(false) to include one of those fields explicitly.

type ChatContextValidationCode

type ChatContextValidationCode string
const (
	ValidationDuplicateID             ChatContextValidationCode = "duplicate_id"
	ValidationTimestampOrder          ChatContextValidationCode = "timestamp_order"
	ValidationEmptyMessageContent     ChatContextValidationCode = "empty_message_content"
	ValidationEmptyTextTerm           ChatContextValidationCode = "empty_text_term"
	ValidationMissingImageTerm        ChatContextValidationCode = "missing_image_term"
	ValidationInvalidAudioTerm        ChatContextValidationCode = "invalid_audio_term"
	ValidationInvalidFunctionCall     ChatContextValidationCode = "invalid_function_call"
	ValidationInvalidFunctionCallArgs ChatContextValidationCode = "invalid_function_call_args"
	ValidationInvalidFunctionOutput   ChatContextValidationCode = "invalid_function_call_output"
	ValidationOrphanFunctionOutput    ChatContextValidationCode = "orphan_function_call_output"
)

type ChatContextValidationIssue

type ChatContextValidationIssue struct {
	Severity ChatContextValidationSeverity `json:"severity"`
	Code     ChatContextValidationCode     `json:"code"`
	Index    int                           `json:"index"`
	ItemID   string                        `json:"itemId"`
	Message  string                        `json:"message"`
}

type ChatContextValidationResult

type ChatContextValidationResult struct {
	Valid    bool                         `json:"valid"`
	Errors   int                          `json:"errors"`
	Warnings int                          `json:"warnings"`
	Issues   []ChatContextValidationIssue `json:"issues"`
}

func ValidateChatContextStructure

func ValidateChatContextStructure(chatContext *ChatContext) ChatContextValidationResult

ValidateChatContextStructure performs the same realtime structural checks as the TypeScript SDK without mutating the supplied history.

type ChatContextValidationSeverity

type ChatContextValidationSeverity string
const (
	ValidationError   ChatContextValidationSeverity = "error"
	ValidationWarning ChatContextValidationSeverity = "warning"
)

type ChatItem

type ChatItem interface {
	ItemID() string
	ItemType() ItemType
	ItemCreatedAt() time.Time
	// contains filtered or unexported methods
}

type ChatMessage

type ChatMessage struct {
	ID                   string
	Role                 ChatRole
	Content              []Content
	Interrupted          bool
	TranscriptConfidence *float64
	Extra                map[string]any
	Metrics              MetricsReport
	Hash                 []byte
	CreatedAt            time.Time
}

func NewChatMessage

func NewChatMessage(role ChatRole, text string) *ChatMessage

func (*ChatMessage) Clone

func (m *ChatMessage) Clone() *ChatMessage

func (*ChatMessage) ItemCreatedAt

func (m *ChatMessage) ItemCreatedAt() time.Time

func (*ChatMessage) ItemID

func (m *ChatMessage) ItemID() string

func (*ChatMessage) ItemType

func (m *ChatMessage) ItemType() ItemType

func (*ChatMessage) MarshalJSON

func (m *ChatMessage) MarshalJSON() ([]byte, error)

func (*ChatMessage) RawTextContent

func (m *ChatMessage) RawTextContent() (string, bool)

func (*ChatMessage) TextContent

func (m *ChatMessage) TextContent() (string, bool)

type ChatOptions

type ChatOptions struct {
	ChatContext       *ChatContext
	ToolContext       *Context
	ConnectOptions    agents.APIConnectOptions
	ParallelToolCalls bool
	ToolChoice        ToolChoice
	Extra             map[string]any
}

type ChatRole

type ChatRole string
const (
	RoleDeveloper ChatRole = "developer"
	RoleSystem    ChatRole = "system"
	RoleUser      ChatRole = "user"
	RoleAssistant ChatRole = "assistant"
)

type ChoiceDelta

type ChoiceDelta struct {
	Role      ChatRole
	Content   string
	ToolCalls []*FunctionCall
	Extra     map[string]any
}

type CollectedResponse

type CollectedResponse struct {
	Text      string
	ToolCalls []*FunctionCall
	Usage     *CompletionUsage
	Extra     map[string]any
}

type CompletionUsage

type CompletionUsage struct {
	CompletionTokens    int64
	PromptTokens        int64
	PromptCachedTokens  int64
	CacheCreationTokens int64
	TotalTokens         int64
	ServiceTier         string
}

type Content

type Content interface {
	// contains filtered or unexported methods
}

type Context

type Context struct {
	// contains filtered or unexported fields
}

func EmptyToolContext

func EmptyToolContext() *Context

func NewToolContext

func NewToolContext(entries ...any) (*Context, error)

func (*Context) Copy

func (c *Context) Copy() *Context

Copy returns an independent context that retains toolset identity and setup hooks while sharing the immutable tool implementations themselves.

func (*Context) Flatten

func (c *Context) Flatten() []Tool

func (*Context) FunctionTool

func (c *Context) FunctionTool(id string) (ExecutableTool, bool)

func (*Context) HasTool

func (c *Context) HasTool(id string) bool

func (*Context) SortedFunctionTools

func (c *Context) SortedFunctionTools() []ExecutableTool

func (*Context) SortedToolNames

func (c *Context) SortedToolNames() []string

func (*Context) Tool

func (c *Context) Tool(id string) (Tool, bool)

func (*Context) Toolsets

func (c *Context) Toolsets() []*Toolset

func (*Context) Update

func (c *Context) Update(entries ...any) error

func (*Context) UpdateTools

func (c *Context) UpdateTools(tools []Tool) error

type CopyOptions

type CopyOptions struct {
	ExcludeFunctionCall bool
	ExcludeInstructions bool
	ExcludeEmptyMessage bool
	ExcludeHandoff      bool
	ExcludeConfigUpdate bool
	// ToolContext removes function calls/outputs for tools that are not
	// currently active, matching agents-js copy({ toolCtx }).
	ToolContext *Context
}

type DuplicateMode

type DuplicateMode string
const (
	DuplicateAllow   DuplicateMode = "allow"
	DuplicateReject  DuplicateMode = "reject"
	DuplicateReplace DuplicateMode = "replace"
	DuplicateConfirm DuplicateMode = "confirm"
)

type DuplicatePromptArgs

type DuplicatePromptArgs struct {
	FunctionName      string
	FunctionCallsJSON []string
	FunctionCallsText string
}

DuplicatePromptArgs is supplied to duplicate-template callbacks.

type ErrorEvent

type ErrorEvent struct {
	Timestamp   time.Time
	Label       string
	Err         error
	Recoverable bool
}

type ExecutableTool

type ExecutableTool interface {
	Tool
	Name() string
	Description() string
	Parameters() json.RawMessage
	Flags() ToolFlag
	OnDuplicate() DuplicateMode
	Execute(context.Context, json.RawMessage, ToolOptions) (any, error)
}

type FallbackAdapter

type FallbackAdapter struct {
	// contains filtered or unexported fields
}

FallbackAdapter implements ordered LLM failover with shared health state, bounded streams, and at most one recovery probe per provider. It forwards child metrics and errors without synthesizing duplicate adapter metrics.

func NewFallbackAdapter

func NewFallbackAdapter(options FallbackOptions) (*FallbackAdapter, error)

func (*FallbackAdapter) Availability

func (a *FallbackAdapter) Availability() []bool

func (*FallbackAdapter) Chat

func (a *FallbackAdapter) Chat(parent context.Context, options ChatOptions) (LLMStream, error)

func (*FallbackAdapter) Close

func (a *FallbackAdapter) Close(ctx context.Context) error

func (*FallbackAdapter) Label

func (a *FallbackAdapter) Label() string

func (*FallbackAdapter) Model

func (a *FallbackAdapter) Model() string

func (*FallbackAdapter) OnAvailabilityChanged

func (a *FallbackAdapter) OnAvailabilityChanged(fn func(AvailabilityChangedEvent)) func()

func (*FallbackAdapter) OnError

func (a *FallbackAdapter) OnError(fn func(ErrorEvent)) func()

func (*FallbackAdapter) OnMetrics

func (a *FallbackAdapter) OnMetrics(fn func(metrics.LLM)) func()

func (*FallbackAdapter) Prewarm

func (a *FallbackAdapter) Prewarm(ctx context.Context)

func (*FallbackAdapter) Provider

func (a *FallbackAdapter) Provider() string

func (*FallbackAdapter) Providers

func (a *FallbackAdapter) Providers() []LLM

type FallbackOptions

type FallbackOptions struct {
	LLMs             []LLM
	AttemptTimeout   time.Duration
	MaxRetriesPerLLM int
	RetryInterval    time.Duration
	RetryOnChunkSent bool
	StreamCapacity   int
	// KeepProvidersOpen transfers provider lifetime ownership to the caller.
	// By default Close closes every provider after in-flight recovery ends.
	KeepProvidersOpen bool
}

FallbackOptions configures a FallbackAdapter. Providers are attempted in slice order. A zero AttemptTimeout or RetryInterval uses the agents-js defaults; MaxRetriesPerLLM defaults to zero.

type FormatChatHistoryOptions

type FormatChatHistoryOptions struct {
	IncludeIDs        bool
	IncludeTimestamps bool
}

type FunctionCall

type FunctionCall struct {
	ID               string
	CallID           string
	Name             string
	Arguments        string
	CreatedAt        time.Time
	Extra            map[string]any
	GroupID          string
	ThoughtSignature string
}

func NewFunctionCall

func NewFunctionCall(callID, name, arguments string) *FunctionCall

func (*FunctionCall) Clone

func (f *FunctionCall) Clone() *FunctionCall

func (*FunctionCall) DecodeArguments

func (f *FunctionCall) DecodeArguments(target any) error

func (*FunctionCall) ItemCreatedAt

func (f *FunctionCall) ItemCreatedAt() time.Time

func (*FunctionCall) ItemID

func (f *FunctionCall) ItemID() string

func (*FunctionCall) ItemType

func (f *FunctionCall) ItemType() ItemType

func (*FunctionCall) MarshalJSON

func (f *FunctionCall) MarshalJSON() ([]byte, error)

type FunctionCallOutput

type FunctionCallOutput struct {
	ID        string
	CallID    string
	Name      string
	Output    string
	IsError   bool
	CreatedAt time.Time
}

func NewFunctionCallOutput

func NewFunctionCallOutput(callID, name, output string, isError bool) *FunctionCallOutput

func (*FunctionCallOutput) ItemCreatedAt

func (f *FunctionCallOutput) ItemCreatedAt() time.Time

func (*FunctionCallOutput) ItemID

func (f *FunctionCallOutput) ItemID() string

func (*FunctionCallOutput) ItemType

func (f *FunctionCallOutput) ItemType() ItemType

func (*FunctionCallOutput) MarshalJSON

func (f *FunctionCallOutput) MarshalJSON() ([]byte, error)

type FunctionTool

type FunctionTool[I any, O any] struct {
	// contains filtered or unexported fields
}

func MustTool

func MustTool[I any, O any](options FunctionToolOptions[I, O]) *FunctionTool[I, O]

func NewTool

func NewTool[I any, O any](options FunctionToolOptions[I, O]) (*FunctionTool[I, O], error)

func (*FunctionTool[I, O]) Description

func (t *FunctionTool[I, O]) Description() string

func (*FunctionTool[I, O]) Execute

func (t *FunctionTool[I, O]) Execute(ctx context.Context, raw json.RawMessage, options ToolOptions) (any, error)

func (*FunctionTool[I, O]) Flags

func (t *FunctionTool[I, O]) Flags() ToolFlag

func (*FunctionTool[I, O]) ID

func (t *FunctionTool[I, O]) ID() string

func (*FunctionTool[I, O]) Name

func (t *FunctionTool[I, O]) Name() string

func (*FunctionTool[I, O]) OnDuplicate

func (t *FunctionTool[I, O]) OnDuplicate() DuplicateMode

func (*FunctionTool[I, O]) Parameters

func (t *FunctionTool[I, O]) Parameters() json.RawMessage

func (*FunctionTool[I, O]) Type

func (*FunctionTool[I, O]) Type() ToolType

type FunctionToolOptions

type FunctionToolOptions[I any, O any] struct {
	Name        string
	Description string
	Parameters  json.RawMessage
	Flags       ToolFlag
	OnDuplicate DuplicateMode
	Validate    func(*I) error
	Execute     func(context.Context, I, ToolOptions) (O, error)
}

type GenerateRealtimeReplyOptions

type GenerateRealtimeReplyOptions struct {
	Instructions string
}

type GenerationCreatedEvent

type GenerationCreatedEvent struct {
	MessageStream  stream.Reader[MessageGeneration]
	FunctionStream stream.Reader[*FunctionCall]
	UserInitiated  bool
	ResponseID     string
}

type ImageContent

type ImageContent struct {
	ID              string
	Image           any
	InferenceDetail ImageDetail
	InferenceWidth  int
	InferenceHeight int
	MIMEType        string
}

func NewImageContent

func NewImageContent(image any) ImageContent

type ImageDetail

type ImageDetail string
const (
	ImageDetailAuto ImageDetail = "auto"
	ImageDetailHigh ImageDetail = "high"
	ImageDetailLow  ImageDetail = "low"
)

type InputSpeechStartedEvent

type InputSpeechStartedEvent struct{}

type InputSpeechStoppedEvent

type InputSpeechStoppedEvent struct {
	UserTranscriptionEnabled bool
}

type InputTranscriptionCompletedEvent

type InputTranscriptionCompletedEvent struct {
	ItemID        string
	Transcript    string
	IsFinal       bool
	TurnStartedAt *time.Time
}

type InstructionContent

type InstructionContent struct{ Instructions Instructions }

type Instructions

type Instructions struct {
	Audio string `json:"audio"`
	Text  string `json:"text,omitempty"`
	// contains filtered or unexported fields
}

func NewInstructions

func NewInstructions(audio, text string) Instructions

func (Instructions) AsModality

func (i Instructions) AsModality(modality Modality) Instructions

func (Instructions) Concat

func (i Instructions) Concat(other Instructions) Instructions

func (Instructions) MarshalJSON

func (i Instructions) MarshalJSON() ([]byte, error)

func (Instructions) String

func (i Instructions) String() string

func (Instructions) TextValue

func (i Instructions) TextValue() string

func (*Instructions) UnmarshalJSON

func (i *Instructions) UnmarshalJSON(data []byte) error

func (Instructions) Value

func (i Instructions) Value() string

type ItemType

type ItemType string
const (
	ItemMessage            ItemType = "message"
	ItemFunctionCall       ItemType = "function_call"
	ItemFunctionCallOutput ItemType = "function_call_output"
	ItemAgentHandoff       ItemType = "agent_handoff"
	ItemAgentConfigUpdate  ItemType = "agent_config_update"
)

type LLM

type LLM interface {
	Label() string
	Provider() string
	Model() string
	Chat(context.Context, ChatOptions) (LLMStream, error)
	Prewarm(context.Context)
	Close(context.Context) error
	OnMetrics(func(metrics.LLM)) func()
	OnError(func(ErrorEvent)) func()
}

type LLMStream

type LLMStream interface {
	stream.Reader[ChatChunk]
	Collect(context.Context) (CollectedResponse, error)
	Close() error
	ChatContext() *ChatContext
	ToolContext() *Context
}

type MessageGeneration

type MessageGeneration struct {
	MessageID   string
	TextStream  stream.Reader[RealtimeText]
	AudioStream stream.Reader[agents.AudioFrame]
	// Modalities is nil when the provider does not expose the asynchronous
	// modality result. Providers must return a fresh slice from the callback.
	Modalities func(context.Context) ([]Modality, error)
}

type MetricsReport

type MetricsReport struct {
	ProviderRequestIDs       []string
	StartedSpeakingAt        time.Time
	StoppedSpeakingAt        time.Time
	TranscriptionDelay       time.Duration
	EndOfTurnDelay           time.Duration
	OnUserTurnCompletedDelay time.Duration
	LLMNodeTTFT              time.Duration
	TTSNodeTTFB              time.Duration
	PlaybackLatency          time.Duration
	EndToEndLatency          time.Duration
}

type Modality

type Modality string
const (
	ModalityAudio Modality = "audio"
	ModalityText  Modality = "text"
)

func CloneModalities

func CloneModalities(values []Modality) []Modality

type ProviderTool

type ProviderTool struct {
	ToolID string
	Data   map[string]any
}

func (*ProviderTool) ID

func (t *ProviderTool) ID() string

func (*ProviderTool) Type

func (*ProviderTool) Type() ToolType

type RealtimeAudioInput

type RealtimeAudioInput struct {
	// contains filtered or unexported fields
}

RealtimeAudioInput owns the single replaceable input-audio pump used by a realtime session. It creates no goroutine until a stream is attached, never polls, and waits for the previous pump to stop before replacement so frames from two sources cannot be reordered.

func NewRealtimeAudioInput

func NewRealtimeAudioInput(parent context.Context, push func(context.Context, agents.AudioFrame) error, onError func(error)) (*RealtimeAudioInput, error)

func (*RealtimeAudioInput) Close

func (i *RealtimeAudioInput) Close(ctx context.Context) error

func (*RealtimeAudioInput) Replace

Replace attaches source, or detaches the current source when source is nil.

type RealtimeCapabilities

type RealtimeCapabilities struct {
	MessageTruncation            bool
	TurnDetection                bool
	UserTranscription            bool
	AutoToolReplyGeneration      bool
	AudioOutput                  bool
	ManualFunctionCalls          bool
	MidSessionChatContextUpdate  bool
	MidSessionInstructionsUpdate bool
	MidSessionToolsUpdate        bool
	PerResponseToolChoice        bool
	NativeTranscriptSync         bool // Deprecated: retained for provider compatibility.
}

type RealtimeError

type RealtimeError struct {
	Operation string
	Err       error
}

func (*RealtimeError) Error

func (e *RealtimeError) Error() string

func (*RealtimeError) Unwrap

func (e *RealtimeError) Unwrap() error

type RealtimeModel

type RealtimeModel interface {
	Capabilities() RealtimeCapabilities
	Model() string
	Provider() string
	Label() string
	Session(context.Context) (RealtimeSession, error)
	Close(context.Context) error
}

type RealtimeModelBase

type RealtimeModelBase struct {
	// contains filtered or unexported fields
}

RealtimeModelBase supplies the immutable provider metadata and capabilities for concrete realtime model implementations.

func NewRealtimeModelBase

func NewRealtimeModelBase(capabilities RealtimeCapabilities, label, provider, model string) *RealtimeModelBase

func (*RealtimeModelBase) Capabilities

func (b *RealtimeModelBase) Capabilities() RealtimeCapabilities

func (*RealtimeModelBase) Label

func (b *RealtimeModelBase) Label() string

func (*RealtimeModelBase) Model

func (b *RealtimeModelBase) Model() string

func (*RealtimeModelBase) Provider

func (b *RealtimeModelBase) Provider() string

func (*RealtimeModelBase) SetModel

func (b *RealtimeModelBase) SetModel(model string)

type RealtimeModelError

type RealtimeModelError struct {
	Type        string
	Timestamp   time.Time
	Label       string
	Err         error
	Recoverable bool
}

func (RealtimeModelError) Error

func (e RealtimeModelError) Error() string

func (RealtimeModelError) Unwrap

func (e RealtimeModelError) Unwrap() error

type RealtimeSession

type RealtimeSession interface {
	RealtimeModel() RealtimeModel
	ChatContext() *ChatContext
	Tools() *Context
	UpdateInstructions(context.Context, string) error
	UpdateChatContext(context.Context, *ChatContext) error
	UpdateTools(context.Context, *Context) error
	UpdateOptions(context.Context, RealtimeUpdateOptions) error
	PushAudio(context.Context, agents.AudioFrame) error
	GenerateReply(context.Context, GenerateRealtimeReplyOptions) (GenerationCreatedEvent, error)
	CommitAudio(context.Context) error
	ClearAudio(context.Context) error
	Interrupt(context.Context) error
	Truncate(context.Context, TruncateRealtimeMessageOptions) error
	StartUserActivity()
	SetInputAudioStream(context.Context, stream.Reader[agents.AudioFrame]) error
	Close(context.Context) error

	OnInputSpeechStarted(func(InputSpeechStartedEvent)) func()
	OnInputSpeechStopped(func(InputSpeechStoppedEvent)) func()
	OnInputTranscriptionCompleted(func(InputTranscriptionCompletedEvent)) func()
	OnGenerationCreated(func(GenerationCreatedEvent)) func()
	OnMetrics(func(metrics.Realtime)) func()
	OnError(func(RealtimeModelError)) func()
	OnReconnected(func(RealtimeSessionReconnectedEvent)) func()
}

type RealtimeSessionEvents

type RealtimeSessionEvents struct {
	// contains filtered or unexported fields
}

RealtimeSessionEvents is intended to be embedded in provider sessions. It provides ordered typed subscriptions and isolates callback panics.

func (*RealtimeSessionEvents) EmitError

func (e *RealtimeSessionEvents) EmitError(event RealtimeModelError)

func (*RealtimeSessionEvents) EmitGenerationCreated

func (e *RealtimeSessionEvents) EmitGenerationCreated(event GenerationCreatedEvent)

func (*RealtimeSessionEvents) EmitInputSpeechStarted

func (e *RealtimeSessionEvents) EmitInputSpeechStarted(event InputSpeechStartedEvent)

func (*RealtimeSessionEvents) EmitInputSpeechStopped

func (e *RealtimeSessionEvents) EmitInputSpeechStopped(event InputSpeechStoppedEvent)

func (*RealtimeSessionEvents) EmitInputTranscriptionCompleted

func (e *RealtimeSessionEvents) EmitInputTranscriptionCompleted(event InputTranscriptionCompletedEvent)

func (*RealtimeSessionEvents) EmitMetrics

func (e *RealtimeSessionEvents) EmitMetrics(metric metrics.Realtime)

func (*RealtimeSessionEvents) EmitReconnected

func (e *RealtimeSessionEvents) EmitReconnected(event RealtimeSessionReconnectedEvent)

func (*RealtimeSessionEvents) OnError

func (e *RealtimeSessionEvents) OnError(fn func(RealtimeModelError)) func()

func (*RealtimeSessionEvents) OnGenerationCreated

func (e *RealtimeSessionEvents) OnGenerationCreated(fn func(GenerationCreatedEvent)) func()

func (*RealtimeSessionEvents) OnInputSpeechStarted

func (e *RealtimeSessionEvents) OnInputSpeechStarted(fn func(InputSpeechStartedEvent)) func()

func (*RealtimeSessionEvents) OnInputSpeechStopped

func (e *RealtimeSessionEvents) OnInputSpeechStopped(fn func(InputSpeechStoppedEvent)) func()

func (*RealtimeSessionEvents) OnInputTranscriptionCompleted

func (e *RealtimeSessionEvents) OnInputTranscriptionCompleted(fn func(InputTranscriptionCompletedEvent)) func()

func (*RealtimeSessionEvents) OnMetrics

func (e *RealtimeSessionEvents) OnMetrics(fn func(metrics.Realtime)) func()

func (*RealtimeSessionEvents) OnReconnected

func (e *RealtimeSessionEvents) OnReconnected(fn func(RealtimeSessionReconnectedEvent)) func()

type RealtimeSessionReconnectedEvent

type RealtimeSessionReconnectedEvent struct{}

type RealtimeSessionUpdate

type RealtimeSessionUpdate struct {
	Instructions *string
	ChatContext  *ChatContext
	Tools        *Context
}

type RealtimeText

type RealtimeText struct {
	Text  string
	Timed *agents.TimedString
}

RealtimeText represents the string | TimedString union exposed by the TypeScript SDK without resorting to any. Timed is non-nil for aligned text.

func PlainRealtimeText

func PlainRealtimeText(text string) RealtimeText

func TimedRealtimeText

func TimedRealtimeText(value agents.TimedString) RealtimeText

type RealtimeUpdateOptions

type RealtimeUpdateOptions struct {
	// ToolChoice uses Override so Unset means no change and Null explicitly
	// resets the provider's tool choice.
	ToolChoice agents.Override[ToolChoice]
}

type RemoteChatContext

type RemoteChatContext struct {
	// contains filtered or unexported fields
}

RemoteChatContext maintains the ordered, ID-addressable history used by realtime providers. Insert is O(1), as are lookup and deletion. Returned values are defensive copies and may be safely retained by callers.

func NewRemoteChatContext

func NewRemoteChatContext() *RemoteChatContext

func (*RemoteChatContext) Delete

func (c *RemoteChatContext) Delete(itemID string) error

Delete removes itemID while preserving the order of its neighbors.

func (*RemoteChatContext) Get

func (c *RemoteChatContext) Get(itemID string) (RemoteChatItem, bool)

Get returns a snapshot of the node with itemID.

func (*RemoteChatContext) Insert

func (c *RemoteChatContext) Insert(previousItemID string, item ChatItem) error

Insert adds item immediately after previousItemID. An empty previousItemID inserts at the head, matching the TypeScript SDK's undefined/root behavior.

func (*RemoteChatContext) Len

func (c *RemoteChatContext) Len() int

func (*RemoteChatContext) ToChatContext

func (c *RemoteChatContext) ToChatContext() *ChatContext

ToChatContext returns an independent ChatContext in remote ordering.

type RemoteChatItem

type RemoteChatItem struct {
	Item           ChatItem
	PreviousItemID string
	NextItemID     string
}

RemoteChatItem is an immutable snapshot of an item in a RemoteChatContext. Empty previous/next IDs denote the head/tail respectively.

type ReplyPromptArgs

type ReplyPromptArgs struct {
	CallIDs []string
}

ReplyPromptArgs is supplied to deferred-reply template callbacks.

type RunContext

type RunContext struct {
	UserData     any
	ToolCallID   string
	Session      any
	FunctionCall *FunctionCall
	// contains filtered or unexported fields
}

func NewRunContext

func NewRunContext(userData any, call *FunctionCall, session any, runtime RunContextRuntime) *RunContext

func (*RunContext) CurrentSpeechHandle

func (r *RunContext) CurrentSpeechHandle() any

func (*RunContext) DisallowInterruptions

func (r *RunContext) DisallowInterruptions() error

func (*RunContext) Filler

func (r *RunContext) Filler(ctx context.Context, source any, options RunContextFillerOptions, fn func(context.Context) error) error

Filler accepts a string or the voice package's typed FillerSource callback.

func (*RunContext) Foreground

func (r *RunContext) Foreground(ctx context.Context, fn func(context.Context, any) error) error

func (*RunContext) Runtime

func (r *RunContext) Runtime() RunContextRuntime

Runtime exposes the cycle-breaking adapter for SDK integrations. Tool code should use the direct RunContext methods instead.

func (*RunContext) Update

func (r *RunContext) Update(ctx context.Context, message any, options ...RunContextUpdateOptions) error

func (*RunContext) WaitForPlayout

func (r *RunContext) WaitForPlayout(ctx context.Context) error

type RunContextFillerOptions

type RunContextFillerOptions struct {
	Delay    time.Duration
	Interval *time.Duration
	MaxSteps *int
}

type RunContextRuntime

type RunContextRuntime interface {
	ToolCurrentSpeechHandle() any
	ToolWaitForPlayout(context.Context) error
	ToolDisallowInterruptions() error
	ToolUpdate(context.Context, any, RunContextUpdateOptions) error
	ToolForeground(context.Context, func(context.Context, any) error) error
	ToolFiller(context.Context, any, RunContextFillerOptions, func(context.Context) error) error
}

RunContextRuntime is implemented by voice.RunContext. It keeps llm free of a voice import cycle while preserving direct ToolOptions.Context methods.

type RunContextUpdateOptions

type RunContextUpdateOptions struct {
	Template string
	// TemplateSet distinguishes an explicit empty template (deliver the raw
	// message) from an omitted template (inherit the executor default).
	TemplateSet  bool
	TemplateFunc func(UpdatePromptArgs) string
}

type TextContent

type TextContent string

type ThinkingTokenFilter

type ThinkingTokenFilter struct {
	StartTag string
	EndTag   string
	// contains filtered or unexported fields
}

ThinkingTokenFilter incrementally removes model reasoning delimited by StartTag and EndTag. It retains only a possible partial delimiter between calls, so memory remains bounded even when a provider emits an unterminated reasoning block.

A filter is stateful and intended for one stream. It is not safe for concurrent calls; provider stream readers are already serialized.

func NewThinkingTokenFilter

func NewThinkingTokenFilter(startTag, endTag string) *ThinkingTokenFilter

func (*ThinkingTokenFilter) BufferedBytes

func (f *ThinkingTokenFilter) BufferedBytes() int

BufferedBytes reports the retained partial delimiter length. It is useful for saturation diagnostics and tests.

func (*ThinkingTokenFilter) Reset

func (f *ThinkingTokenFilter) Reset()

Reset discards buffered visible/reasoning state.

func (*ThinkingTokenFilter) StripThinkingTokens

func (f *ThinkingTokenFilter) StripThinkingTokens(content *string, final bool) (string, bool)

StripThinkingTokens consumes one delta. The returned bool distinguishes no visible content from a deliberately visible empty provider delta. final flushes any visible partial text and resets the filter for reuse.

type Tool

type Tool interface {
	Type() ToolType
	ID() string
}

type ToolChoice

type ToolChoice struct {
	Kind ToolChoiceKind
	Name string
}

type ToolChoiceKind

type ToolChoiceKind string
const (
	ToolChoiceAuto     ToolChoiceKind = "auto"
	ToolChoiceNone     ToolChoiceKind = "none"
	ToolChoiceRequired ToolChoiceKind = "required"
	ToolChoiceFunction ToolChoiceKind = "function"
)

type ToolError

type ToolError struct{ Message string }

func (*ToolError) Error

func (e *ToolError) Error() string

type ToolFlag

type ToolFlag uint32
const (
	ToolFlagNone          ToolFlag = 0
	ToolFlagIgnoreOnEnter ToolFlag = 1 << 0
	ToolFlagCancellable   ToolFlag = 1 << 1
)

type ToolOptions

type ToolOptions struct {
	Context    *RunContext
	ToolCallID string
}

type ToolType

type ToolType string
const (
	ToolTypeFunction ToolType = "function"
	ToolTypeProvider ToolType = "provider"
)

type Toolset

type Toolset struct {
	// contains filtered or unexported fields
}

func NewToolset

func NewToolset(options ToolsetOptions) (*Toolset, error)

func (*Toolset) Async

func (t *Toolset) Async() (*AsyncToolset, bool)

Async returns the async execution-scope marker, when this toolset was created with NewAsyncToolset.

func (*Toolset) Close

func (t *Toolset) Close(ctx context.Context) error

func (*Toolset) ID

func (t *Toolset) ID() string

func (*Toolset) Setup

func (t *Toolset) Setup(ctx context.Context, toolContext ToolsetContext) error

func (*Toolset) Tools

func (t *Toolset) Tools() []Tool

type ToolsetContext

type ToolsetContext interface{ UpdateTools([]Tool) error }

type ToolsetOptions

type ToolsetOptions struct {
	ID    string
	Tools []Tool
	Setup func(context.Context, ToolsetContext) error
	Close func(context.Context) error
}

type TruncateRealtimeMessageOptions

type TruncateRealtimeMessageOptions struct {
	MessageID       string
	AudioEnd        time.Duration
	Modalities      []Modality
	AudioTranscript string
}

type UpdatePromptArgs

type UpdatePromptArgs struct {
	FunctionName string
	CallID       string
	Message      string
}

UpdatePromptArgs is supplied to a typed update-template callback. String templates use the equivalent {functionName}, {callId}, and {message} placeholders.

Directories

Path Synopsis
Package providerformat converts LiveKit chat contexts into the wire-neutral request shapes consumed by OpenAI-compatible, Google Gemini, and Mistral APIs.
Package providerformat converts LiveKit chat contexts into the wire-neutral request shapes consumed by OpenAI-compatible, Google Gemini, and Mistral APIs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL