omniavatar

OmniAvatar Core

Core interfaces for AI avatars. This package provides the interface definitions without any provider dependencies, across two surfaces:

• live — real-time streaming avatar sessions (rooms, PCM audio streaming for lip-sync) for conversational agents
• render — asynchronous batch avatar video generation (narration audio in, talking-head MP4 out) for offline pipelines

For a batteries-included package with all providers, see omniavatar.

Architecture

Adapters follow the PlexusOne convention: render adapters live in each provider SDK repo (depending only on this module), while live adapters live in the batteries-included omniavatar package (their LiveKit integration lives there).

omniavatar-core/              # Core interfaces + shared helpers (no provider deps)
├── live/                     # Real-time sessions
│   ├── provider.go           # Provider interface
│   ├── session.go            # Session interface + callbacks
│   ├── audio.go              # AudioDestination interface
│   └── errors.go             # Error types
├── render/                   # Batch video generation
│   ├── provider.go           # Provider + AudioUploader interfaces
│   ├── lister.go             # AvatarLister + AvatarInfo
│   ├── request.go            # GenerateRequest
│   ├── job.go                # Job, JobState, JobStatus, Wait
│   ├── mediatype.go / download.go  # AudioContentType / DownloadURL helpers
│   └── errors.go             # Error types
└── registry/
    └── registry.go           # Factory types + options

heygen-go/omniavatar/         # HeyGen RENDER adapter (core-only) — in the SDK repo
tavus-go/omniavatar/          # Tavus RENDER adapter
bithuman-go/omniavatar/       # bitHuman RENDER adapter

omniavatar/                   # Batteries-included
├── registry.go               # Global live + render registries
├── token.go / start_options.go  # LiveKit token + start options
└── providers/
    ├── heygen/               # HeyGen LIVE adapter; registers the SDK render adapter
    ├── tavus/                # Tavus LIVE adapter
    ├── bithuman/             # bitHuman LIVE adapter
    └── all/                  # Convenience import

Live Interfaces

Provider

Creates avatar sessions with provider-specific configuration.

type Provider interface {
    Name() string
    CreateSession(cfg SessionConfig) (Session, error)
}

Session

Manages the avatar lifecycle: start, audio streaming, and cleanup.

type Session interface {
    Identity() string
    Provider() string
    Start(ctx context.Context, opts any) error
    WaitForJoin(ctx context.Context, timeout time.Duration) error
    AudioOutput() AudioDestination
    Close(ctx context.Context) error
    SetCallbacks(callbacks *SessionCallbacks)
}

AudioDestination

Streams TTS audio to the avatar for lip-sync playback.

type AudioDestination interface {
    CaptureFrame(ctx context.Context, frame []byte) error
    Flush(ctx context.Context) error
    ClearBuffer(ctx context.Context) error
    SampleRate() int
    Channels() int
    Close() error
}

Render Interfaces

Provider

Generates avatar videos asynchronously: submit, poll, download.

type Provider interface {
    Name() string
    Generate(ctx context.Context, req GenerateRequest) (*Job, error)
    Status(ctx context.Context, jobID string) (*JobStatus, error)
    Download(ctx context.Context, jobID string, dst io.Writer) error
}

AudioUploader (optional capability)

Providers that can host local audio files implement this; feature-detect it:

if up, ok := provider.(render.AudioUploader); ok {
    audioURL, err = up.UploadAudio(ctx, "narration.mp3", f)
}

AvatarLister (optional capability)

Providers that can enumerate the account's avatars implement this; the returned AvatarInfo.ID values are directly usable as AvatarID:

if l, ok := provider.(render.AvatarLister); ok {
    avatars, err := l.ListAvatars(ctx, "abigail") // "" = all
}

GenerateRequest

job, err := provider.Generate(ctx, render.GenerateRequest{
    AvatarID: avatarID,        // heygen avatar_id / tavus replica_id / bithuman agent_id
    AudioURL: narrationURL,    // drives lip-sync from existing audio (primary path)
})

Wait

status, err := render.Wait(ctx, provider, job.ID, 5*time.Second)
// status.State: pending → processing → completed | failed

Usage

Import only the interfaces:

import (
    "github.com/plexusone/omniavatar-core/live"
    "github.com/plexusone/omniavatar-core/render"
)

func processAvatar(session live.Session) error {
    audioOut := session.AudioOutput()
    return audioOut.CaptureFrame(ctx, pcmData)
}

Import with all providers (batteries-included):

import (
    "github.com/plexusone/omniavatar"
    _ "github.com/plexusone/omniavatar/providers/all"
)

liveProvider, err := omniavatar.GetLiveProvider("heygen",
    omniavatar.WithAPIKey(os.Getenv("LIVEAVATAR_API_KEY")),
    omniavatar.WithExtension("avatar_id", avatarID),
    omniavatar.WithExtension("sandbox", true),
)

renderProvider, err := omniavatar.GetRenderProvider("heygen",
    omniavatar.WithAPIKey(os.Getenv("HEYGEN_API_KEY")),
)

Provider Registry Pattern

Providers register with a priority level:

Higher priority providers override lower priority registrations for the same name.

Registration via init()

The batteries omniavatar package registers providers by name via init(). The render provider is the SDK-hosted adapter; the live provider is local:

// In omniavatar/providers/heygen/register.go
import heygenrender "github.com/plexusone/heygen-go/omniavatar"

func init() {
    omniavatar.RegisterLiveProvider("heygen", NewProviderFromConfig, omniavatar.PriorityThick)
    omniavatar.RegisterRenderProvider("heygen", heygenrender.NewRenderProviderFromConfig, omniavatar.PriorityThick)
}

Supported Providers

Session Lifecycle (live)

1. Get Provider    → omniavatar.GetLiveProvider("heygen", opts...)
2. Create Session  → provider.CreateSession(cfg)
3. Start           → session.Start(ctx, startOptions)
4. Wait for Join   → session.WaitForJoin(ctx, timeout)
5. Stream Audio    → session.AudioOutput().CaptureFrame(ctx, pcm)
6. Close           → session.Close(ctx)

Job Lifecycle (render)

1. Get Provider    → omniavatar.GetRenderProvider("heygen", opts...)
2. Upload Audio    → provider.(render.AudioUploader).UploadAudio(...)  [optional]
3. Generate        → provider.Generate(ctx, render.GenerateRequest{...})
4. Wait            → render.Wait(ctx, provider, job.ID, interval)
5. Download        → provider.Download(ctx, job.ID, dst)

Session Callbacks (live)

session.SetCallbacks(&live.SessionCallbacks{
    OnAvatarJoined: func(identity string) {
        log.Printf("Avatar joined: %s", identity)
    },
    OnPlaybackStarted: func() {
        log.Print("Avatar started speaking")
    },
    OnPlaybackFinished: func(position float64, interrupted bool) {
        log.Printf("Avatar finished speaking at %.2fs (interrupted: %v)", position, interrupted)
    },
    OnError: func(err error) {
        log.Printf("Avatar error: %v", err)
    },
})

Audio Format (live)

Default audio configuration for avatar providers:

Resources

• omniavatar - Provider implementations
• HeyGen LiveAvatar
• HeyGen API
• Tavus
• bitHuman