agent

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 6, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package agent provides a full-featured LiveKit agent implementation.

This package contains LiveKit-specific agent functionality that goes beyond the generic omnimeet-core interfaces. Use this package when you need:

  • Video publishing (static images, webcam simulation)
  • Advanced media mode selection (audio-only, audio+image, audio+video)
  • LiveKit-specific configuration options
  • Direct access to LiveKit SDK features

For generic meeting participation that works across providers, use the omnimeet package which implements the omnimeet-core interfaces.

Media Modes

The agent supports three media modes:

  • AudioOnly: Agent publishes only audio (default)
  • AudioWithImage: Agent publishes audio + static image as video
  • AudioWithVideo: Agent publishes audio + video frames

Usage

agent, err := agent.New(agent.Options{
    Config: omnimeet.Config{
        APIKey:    "...",
        APISecret: "...",
        ServerURL: "wss://...",
    },
    MediaMode:   agent.AudioWithImage,
    Name:        "AI Assistant",
    ImagePath:   "/path/to/avatar.png",
    AudioConfig: agent.AudioConfig{
        SampleRate: 48000,
        Channels:   1,
    },
})
if err != nil {
    log.Fatal(err)
}

// Join a meeting
err = agent.Join(ctx, "room-name")
if err != nil {
    log.Fatal(err)
}
defer agent.Leave(ctx)

// Start publishing audio
writer, err := agent.StartAudio(ctx)
if err != nil {
    log.Fatal(err)
}
defer agent.StopAudio(ctx)

// Write PCM16 audio frames
writer.Write(pcmData)

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Agent

type Agent struct {
	// contains filtered or unexported fields
}

Agent is a full-featured LiveKit agent with audio and video capabilities.

func New

func New(opts Options) (*Agent, error)

New creates a new LiveKit agent with the given options.

func (*Agent) Events

func (a *Agent) Events() <-chan event.Event

Events returns a channel of real-time events.

func (*Agent) GetParticipant

func (a *Agent) GetParticipant(identity string) *participant.Participant

GetParticipant returns a specific participant by identity.

func (*Agent) Join

func (a *Agent) Join(ctx context.Context, roomName string) error

Join connects the agent to a meeting room.

func (*Agent) Leave

func (a *Agent) Leave(ctx context.Context) error

Leave disconnects the agent from the meeting.

func (*Agent) LocalParticipant

func (a *Agent) LocalParticipant() *participant.Participant

LocalParticipant returns the agent's participant info.

func (*Agent) Meeting

func (a *Agent) Meeting() *meeting.Meeting

Meeting returns the current meeting info.

func (*Agent) OnActiveSpeakerChanged

func (a *Agent) OnActiveSpeakerChanged(handler func([]participant.Participant))

func (*Agent) OnAudioFrame

func (a *Agent) OnAudioFrame(handler func(AudioFrame))

func (*Agent) OnDataMessage

func (a *Agent) OnDataMessage(handler func(DataMessage))

func (*Agent) OnParticipantJoined

func (a *Agent) OnParticipantJoined(handler func(participant.Participant))

func (*Agent) OnParticipantLeft

func (a *Agent) OnParticipantLeft(handler func(participant.Participant))

func (*Agent) OnTrackPublished

func (a *Agent) OnTrackPublished(handler func(participant.Participant, track.Track))

func (*Agent) OnTrackUnpublished

func (a *Agent) OnTrackUnpublished(handler func(participant.Participant, track.Track))

func (*Agent) RemoteParticipants

func (a *Agent) RemoteParticipants() []participant.Participant

RemoteParticipants returns all other participants in the meeting.

func (*Agent) SendData

func (a *Agent) SendData(ctx context.Context, msg DataMessage) error

SendData sends a data channel message.

func (*Agent) StartAudio

func (a *Agent) StartAudio(ctx context.Context) (AudioWriter, error)

StartAudio starts publishing audio and returns a writer for PCM16 data.

func (*Agent) StartVideo

func (a *Agent) StartVideo(ctx context.Context) (VideoWriter, error)

StartVideo starts publishing video and returns a writer for video frames. Only available in AudioWithVideo mode.

func (*Agent) State

func (a *Agent) State() ConnectionState

State returns the current connection state.

func (*Agent) StopAudio

func (a *Agent) StopAudio(ctx context.Context) error

StopAudio stops publishing audio.

func (*Agent) StopVideo

func (a *Agent) StopVideo(ctx context.Context) error

StopVideo stops publishing video.

func (*Agent) SubscribeToAllAudio

func (a *Agent) SubscribeToAllAudio(ctx context.Context) (<-chan AudioFrame, error)

SubscribeToAllAudio subscribes to all participants' audio.

func (*Agent) SubscribeToAudio

func (a *Agent) SubscribeToAudio(ctx context.Context, participantID string) (<-chan AudioFrame, error)

SubscribeToAudio subscribes to a participant's audio and returns a channel of frames.

func (*Agent) UpdateImage

func (a *Agent) UpdateImage(ctx context.Context, imageData []byte) error

UpdateImage updates the static image being published (AudioWithImage mode).

func (*Agent) WriteOpusDirect

func (a *Agent) WriteOpusDirect(data []byte, duration time.Duration) error

WriteOpusDirect writes pre-encoded Opus data directly to the audio track. This bypasses local Opus encoding - use when TTS returns Opus format. Each call should contain one Opus frame (typically 20ms of audio).

type AudioConfig

type AudioConfig struct {
	// SampleRate in Hz (default: 48000)
	SampleRate int
	// Channels: 1 = mono, 2 = stereo (default: 1)
	Channels int
	// FrameDuration for audio frames (default: 20ms)
	FrameDuration time.Duration
	// TrackName for the published audio track
	TrackName string
}

AudioConfig configures audio publishing.

type AudioFrame

type AudioFrame struct {
	ParticipantID   string
	ParticipantName string
	Data            []byte // PCM16 little-endian
	SampleRate      int
	Channels        int
	Timestamp       time.Time
	SequenceNumber  uint64
}

AudioFrame represents a received audio frame.

type AudioWriter

type AudioWriter interface {
	// Write writes PCM16 little-endian audio data.
	Write(data []byte) (int, error)
	// Close stops the audio writer.
	Close() error
}

AudioWriter writes PCM16 audio data to the meeting.

type ConnectionState

type ConnectionState string

ConnectionState represents the agent's connection state.

const (
	StateDisconnected ConnectionState = "disconnected"
	StateConnecting   ConnectionState = "connecting"
	StateConnected    ConnectionState = "connected"
	StateReconnecting ConnectionState = "reconnecting"
	StateFailed       ConnectionState = "failed"
)

type DataMessage

type DataMessage struct {
	Topic          string
	Payload        []byte
	From           *participant.Participant
	DestinationIDs []string
	Reliable       bool
	Timestamp      time.Time
}

DataMessage represents a data channel message.

type ImageConfig

type ImageConfig struct {
	// Path to the image file (PNG, JPEG, etc.)
	// Requires CGO for runtime encoding. For no-CGO deployment, use H264Path.
	Path string
	// Data is raw image data (alternative to Path)
	// Requires CGO for runtime encoding. For no-CGO deployment, use H264Data.
	Data []byte

	// H264Path is the path to a pre-encoded H.264 keyframe file.
	// Use the encode-avatar tool to create this file.
	// This is the recommended approach - no CGO required at runtime.
	H264Path string
	// H264Data is pre-encoded H.264 keyframe data (alternative to H264Path).
	// Use the encode-avatar tool to create this data.
	H264Data []byte

	// Width to resize to (0 = use original). Only used with Path/Data.
	Width int
	// Height to resize to (0 = use original). Only used with Path/Data.
	Height int
	// FrameRate for the static image video track (default: 1)
	// Lower values use less bandwidth for static content.
	FrameRate int
	// TrackName for the published video track
	TrackName string
}

ImageConfig configures static image video publishing.

type ImageWriter

type ImageWriter interface {
	// UpdateImage updates the image being published.
	UpdateImage(data []byte) error
	// Close stops the image writer.
	Close() error
}

ImageWriter is the interface for static image video publishing.

type MediaMode

type MediaMode string

MediaMode defines what media tracks the agent publishes.

const (
	// AudioOnly publishes only an audio track (default).
	AudioOnly MediaMode = "audio_only"
	// AudioWithImage publishes audio + a static image as video.
	AudioWithImage MediaMode = "audio_with_image"
	// AudioWithVideo publishes audio + video frames.
	AudioWithVideo MediaMode = "audio_with_video"
)

type Options

type Options struct {
	// LiveKit connection settings
	APIKey    string
	APISecret string
	ServerURL string

	// Agent identity
	Name     string // Display name in the meeting
	Identity string // Unique identity (defaults to generated UUID)
	Metadata map[string]string

	// Media mode selection
	MediaMode MediaMode // Defaults to AudioOnly

	// Audio configuration
	Audio AudioConfig

	// Video configuration (for AudioWithVideo mode)
	Video VideoConfig

	// Image configuration (for AudioWithImage mode)
	Image ImageConfig

	// Behavior
	AutoSubscribe bool // Auto-subscribe to all tracks (default: true)
}

Options configures a LiveKit agent.

type VideoConfig

type VideoConfig struct {
	// Width in pixels (default: 640)
	Width int
	// Height in pixels (default: 480)
	Height int
	// FrameRate in fps (default: 30)
	FrameRate int
	// Codec: "vp8" or "h264" (default: "vp8")
	Codec string
	// Bitrate in bps (default: auto)
	Bitrate int
	// TrackName for the published video track
	TrackName string
}

VideoConfig configures video publishing.

type VideoFrame

type VideoFrame struct {
	// Data contains encoded video data (VP8 or H264 depending on codec).
	Data []byte
	// Width of the frame in pixels.
	Width int
	// Height of the frame in pixels.
	Height int
	// Timestamp of the frame.
	Timestamp time.Time
	// Keyframe indicates if this is a keyframe.
	Keyframe bool
}

VideoFrame represents a video frame to publish.

type VideoWriter

type VideoWriter interface {
	// WriteFrame writes a video frame.
	WriteFrame(frame VideoFrame) error
	// Close stops the video writer.
	Close() error
}

VideoWriter writes video frames to the meeting.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL