omniavatar

OmniAvatar

Batteries-included package for AI avatars with two surfaces:

• live — real-time streaming avatars (LiveKit sessions, PCM audio streaming for lip-sync) for conversational agents
• render — asynchronous batch avatar video generation (narration audio in, talking-head MP4 out) for offline pipelines such as presentation videos

Provides provider implementations for HeyGen, Tavus, and bitHuman.

For core interfaces only (no provider dependencies), see omniavatar-core.

Quick Start (live)

import (
    "github.com/plexusone/omniavatar"
    "github.com/plexusone/omniavatar-core/live"
    _ "github.com/plexusone/omniavatar/providers/all"
)

func main() {
    provider, err := omniavatar.GetLiveProvider("heygen",
        omniavatar.WithAPIKey(os.Getenv("LIVEAVATAR_API_KEY")),
        omniavatar.WithExtension("avatar_id", avatarID),
        omniavatar.WithExtension("sandbox", true))
    if err != nil {
        log.Fatal(err)
    }

    session, err := provider.CreateSession(live.SessionConfig{
        AudioConfig: live.DefaultAudioConfig(),
    })
    if err != nil {
        log.Fatal(err)
    }

    // Start with LiveKit
    err = session.Start(ctx, &omniavatar.LiveKitStartOptions{
        Room:             room,
        AgentIdentity:    "agent-123",
        LiveKitURL:       os.Getenv("LIVEKIT_URL"),
        LiveKitAPIKey:    os.Getenv("LIVEKIT_API_KEY"),
        LiveKitAPISecret: os.Getenv("LIVEKIT_API_SECRET"),
    })
}

Quick Start (render)

import (
    "github.com/plexusone/omniavatar"
    "github.com/plexusone/omniavatar-core/render"
    _ "github.com/plexusone/omniavatar/providers/all"
)

func main() {
    provider, err := omniavatar.GetRenderProvider("bithuman",
        omniavatar.WithAPIKey(os.Getenv("BITHUMAN_API_KEY")),
        omniavatar.WithExtension("agent_id", agentID))
    if err != nil {
        log.Fatal(err)
    }

    // Providers with hosting support can upload local narration audio.
    audioURL := ""
    if up, ok := provider.(render.AudioUploader); ok {
        audioURL, err = up.UploadAudio(ctx, "narration.mp3", audioFile)
        if err != nil {
            log.Fatal(err)
        }
    }

    job, err := provider.Generate(ctx, render.GenerateRequest{
        AvatarID: agentID,
        AudioURL: audioURL,
    })
    if err != nil {
        log.Fatal(err)
    }

    status, err := render.Wait(ctx, provider, job.ID, 5*time.Second)
    if err != nil {
        log.Fatal(err)
    }
    log.Printf("video ready: %s (%.1fs)", status.VideoURL, status.Duration)

    out, err := os.Create("presenter.mp4")
    if err != nil {
        log.Fatal(err)
    }
    defer out.Close()

    if err := provider.Download(ctx, job.ID, out); err != nil {
        log.Fatal(err)
    }
}

Architecture

Adapters follow the PlexusOne convention: render adapters live in each provider SDK repo (heygen-go/omniavatar, …), depending only on omniavatar-core, so provider-specific knowledge stays with the SDK. The live adapters live here in the batteries-included package, because their LiveKit integration (LiveKitStartOptions, token generation) lives here. This package re-exports both, registered by name via providers/all.

omniavatar-core/              # Core interfaces + shared helpers (no provider deps)
├── live/                     # Real-time session interfaces
├── render/                   # Batch generation: Provider, AudioUploader,
│                             #   AvatarLister, GenerateRequest, Wait,
│                             #   AudioContentType/DownloadURL helpers
└── registry/                 # Factory types

heygen-go/omniavatar/         # HeyGen RENDER adapter (core-only) — in the SDK repo
tavus-go/omniavatar/          # Tavus RENDER adapter
bithuman-go/omniavatar/       # bitHuman RENDER adapter

omniavatar/                   # Batteries-included (this package)
├── registry.go               # Global live + render registries
├── token.go / start_options.go  # LiveKit token + start options
└── providers/
    ├── heygen/               # HeyGen LIVE adapter (LiveAvatar); registers the SDK render adapter
    ├── tavus/                # Tavus LIVE adapter (CVI)
    ├── bithuman/             # bitHuman LIVE adapter
    └── all/                  # Convenience import (registers every provider)

Provider Registry

Priority System

Providers register with a priority level:

Higher priority providers override lower priority registrations for the same name.

Auto-Registration

Providers auto-register both surfaces via init() when imported:

// Import specific provider
import _ "github.com/plexusone/omniavatar/providers/heygen"

// Or import all providers
import _ "github.com/plexusone/omniavatar/providers/all"

Registry Functions

// Live (real-time sessions)
provider, err := omniavatar.GetLiveProvider("heygen", opts...)
names := omniavatar.ListLiveProviders()
ok := omniavatar.HasLiveProvider("heygen")

// Render (batch video generation)
provider, err := omniavatar.GetRenderProvider("heygen", opts...)
names := omniavatar.ListRenderProviders()
ok := omniavatar.HasRenderProvider("heygen")

Supported Providers

HeyGen

Live: real-time avatar with lip-sync using HeyGen LiveAvatar LITE mode. Render: HeyGen Video Generation API (v2).

Note: the live surface uses the LiveAvatar API key (LIVEAVATAR_API_KEY); the render surface uses the HeyGen API key (HEYGEN_API_KEY). They are different credentials.

// Live
provider, err := omniavatar.GetLiveProvider("heygen",
    omniavatar.WithAPIKey(os.Getenv("LIVEAVATAR_API_KEY")),
    omniavatar.WithExtension("avatar_id", "josh_lite3_20230714"),
    omniavatar.WithExtension("sandbox", true),           // 60s limit, no credits
    omniavatar.WithExtension("video_quality", "high"),   // very_high, high, medium, low
)

// Render
provider, err := omniavatar.GetRenderProvider("heygen",
    omniavatar.WithAPIKey(os.Getenv("HEYGEN_API_KEY")),
    omniavatar.WithExtension("avatar_id", avatarID),
)

The HeyGen render provider implements render.AudioUploader via the HeyGen asset upload API (MP3/audio/mpeg is the documented audio asset type).

Tavus

Live: real-time avatar using Tavus PAL (Personalized AI Likeness). Render: Tavus Video Generation using replicas.

// Live
provider, err := omniavatar.GetLiveProvider("tavus",
    omniavatar.WithAPIKey(os.Getenv("TAVUS_API_KEY")),
    omniavatar.WithExtension("pal_id", "pal_xxx"),   // Optional
    omniavatar.WithExtension("face_id", "face_xxx"), // Optional
)

// Render
provider, err := omniavatar.GetRenderProvider("tavus",
    omniavatar.WithAPIKey(os.Getenv("TAVUS_API_KEY")),
    omniavatar.WithExtension("replica_id", "rep_xxx"),
)

Tavus has no audio upload API; supply a publicly fetchable GenerateRequest.AudioURL (.wav or .mp3).

bitHuman

Live: ultra-low latency real-time avatars. Render: bitHuman video generation, including audio upload support (render.AudioUploader).

// Live
provider, err := omniavatar.GetLiveProvider("bithuman",
    omniavatar.WithAPIKey(os.Getenv("BITHUMAN_API_KEY")),
    omniavatar.WithExtension("agent_id", "agent_xxx"),
)

// Render
provider, err := omniavatar.GetRenderProvider("bithuman",
    omniavatar.WithAPIKey(os.Getenv("BITHUMAN_API_KEY")),
    omniavatar.WithExtension("agent_id", "agent_xxx"),
)

Session Lifecycle (live)

1. Get Provider    → omniavatar.GetLiveProvider("heygen", opts...)
2. Create Session  → provider.CreateSession(cfg)
3. Start           → session.Start(ctx, &LiveKitStartOptions{...})
4. Wait for Join   → session.WaitForJoin(ctx, 30*time.Second)
5. Stream Audio    → session.AudioOutput().CaptureFrame(ctx, pcm)
6. Close           → session.Close(ctx)

Job Lifecycle (render)

1. Get Provider    → omniavatar.GetRenderProvider("heygen", opts...)
2. Upload Audio    → provider.(render.AudioUploader).UploadAudio(...)  [optional]
3. Generate        → provider.Generate(ctx, render.GenerateRequest{...})
4. Wait            → render.Wait(ctx, provider, job.ID, interval)
5. Download        → provider.Download(ctx, job.ID, dst)

LiveKit Integration

Token Generation

Generate tokens for avatar participants to join LiveKit rooms:

token, err := omniavatar.GenerateAvatarToken(omniavatar.TokenOptions{
    APIKey:        os.Getenv("LIVEKIT_API_KEY"),
    APISecret:     os.Getenv("LIVEKIT_API_SECRET"),
    RoomName:      "my-room",
    Identity:      "avatar-heygen-abc123",
    Provider:      "heygen",
    AgentIdentity: "agent-123",
    TTL:           time.Hour,
})

Start Options

type LiveKitStartOptions struct {
    Room             *lksdk.Room  // LiveKit room reference
    AgentIdentity    string       // Agent's participant identity
    LiveKitURL       string       // LiveKit server URL
    LiveKitAPIKey    string       // API key for token generation
    LiveKitAPISecret string       // API secret for token generation
}

Audio Format (live)

Default audio configuration:

Provider Comparison

Specs

• Render PRD
• Render TRD
• Render Plan
• Render Roadmap

Resources

• omniavatar-core - Core interfaces
• HeyGen LiveAvatar
• HeyGen API
• Tavus
• bitHuman