generation

package
v0.68.21 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 4, 2026 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package generation defines SpeechKit's provider-neutral text generation boundary. Provider SDK types stay behind adapters so workflows can select models, enforce privacy policy, and handle context limits consistently.

Index

Constants

View Source
const (
	DefaultContextWindowTokens = 8192
	DefaultPromptReserveTokens = 1024
)
View Source
const LocalContextWindowTokens = 4096

LocalContextWindowTokens is the context the bundled llama-server is started with (localllm.DefaultContextSize, `--ctx-size 4096`). Budgets derived from the 8192 default sized meeting batches for a window the local server does not have; the server answered 400 and the batch never fit.

Variables

This section is empty.

Functions

func ConservativeContextWindow

func ConservativeContextWindow(provider, model string) int

func EstimateTokens

func EstimateTokens(text string) int

EstimateTokens intentionally overestimates multilingual meeting text. Exact provider tokenizers are not available for every selectable model, so a conservative common estimate is safer than retrying oversized prompts.

func SafeInputBudget

func SafeInputBudget(model Model, outputTokens int) int

Types

type Catalog

type Catalog struct {
	Models []Model
}

type Chain

type Chain struct {
	// contains filtered or unexported fields
}

func NewChain

func NewChain(generators ...Generator) *Chain

func (*Chain) Generate

func (c *Chain) Generate(ctx context.Context, request Request) (Result, error)

func (*Chain) Models

func (c *Chain) Models(ctx context.Context, query ModelQuery) (Catalog, error)

type Error

type Error struct {
	Kind      ErrorKind
	Operation string
	Provider  string
	Model     string
	Retryable bool
	Err       error
}

func (*Error) Error

func (e *Error) Error() string

func (*Error) Unwrap

func (e *Error) Unwrap() error

type ErrorKind

type ErrorKind string
const (
	ErrorConfiguration  ErrorKind = "configuration"
	ErrorAuthentication ErrorKind = "authentication"
	ErrorConsent        ErrorKind = "consent"
	ErrorQuota          ErrorKind = "quota"
	ErrorContextLimit   ErrorKind = "context_limit"
	ErrorTransient      ErrorKind = "transient"
	ErrorInvalidOutput  ErrorKind = "invalid_output"
	ErrorCancelled      ErrorKind = "cancelled"
	ErrorPermanent      ErrorKind = "permanent"
)

func Kind

func Kind(err error) ErrorKind

type Generator

type Generator interface {
	Generate(context.Context, Request) (Result, error)
	Models(context.Context, ModelQuery) (Catalog, error)
}

type GenkitGenerator

type GenkitGenerator struct {
	// contains filtered or unexported fields
}

func NewGenkit

func NewGenkit(runtime *genkit.Genkit, models []GenkitModel) *GenkitGenerator

func (*GenkitGenerator) Generate

func (g *GenkitGenerator) Generate(ctx context.Context, request Request) (Result, error)

func (*GenkitGenerator) Models

func (g *GenkitGenerator) Models(_ context.Context, query ModelQuery) (Catalog, error)

type GenkitModel

type GenkitModel struct {
	Model firebaseai.Model
	Info  Model
}

type Message

type Message struct {
	Role    Role
	Content string
}

type Model

type Model struct {
	ID                       string
	Provider                 string
	Name                     string
	Purposes                 []Purpose
	ContextWindowTokens      int
	SupportsStructuredOutput bool
	Cloud                    bool
}

func (Model) Supports

func (m Model) Supports(purpose Purpose) bool

type ModelQuery

type ModelQuery struct {
	Purpose Purpose
}

type Purpose

type Purpose string
const (
	PurposeUtility           Purpose = "utility"
	PurposeAssist            Purpose = "assist"
	PurposeMeetingExtraction Purpose = "meeting_extraction"
	PurposeMeetingSynthesis  Purpose = "meeting_synthesis"
	PurposeVoiceAgentThink   Purpose = "voice_agent_think"
)

type Request

type Request struct {
	Purpose         Purpose
	Locale          string
	System          string
	Prompt          string
	Messages        []Message
	ModelID         string
	AffinityKey     string
	StructuredHint  string
	MaxOutputTokens int
	Temperature     float64
}

type Result

type Result struct {
	Text         string
	Provider     string
	Model        string
	FinishReason string
	Usage        *Usage
	Latency      time.Duration
}

type Role

type Role string
const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
)

type Usage

type Usage struct {
	InputTokens  int
	OutputTokens int
	TotalTokens  int
}

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL