Documentation
¶
Overview ¶
Package generation defines SpeechKit's provider-neutral text generation boundary. Provider SDK types stay behind adapters so workflows can select models, enforce privacy policy, and handle context limits consistently.
Index ¶
- Constants
- func ConservativeContextWindow(provider, model string) int
- func EstimateTokens(text string) int
- func SafeInputBudget(model Model, outputTokens int) int
- type Catalog
- type Chain
- type Error
- type ErrorKind
- type Generator
- type GenkitGenerator
- type GenkitModel
- type Message
- type Model
- type ModelQuery
- type Purpose
- type Request
- type Result
- type Role
- type Usage
Constants ¶
const ( DefaultContextWindowTokens = 8192 DefaultPromptReserveTokens = 1024 )
const LocalContextWindowTokens = 4096
LocalContextWindowTokens is the context the bundled llama-server is started with (localllm.DefaultContextSize, `--ctx-size 4096`). Budgets derived from the 8192 default sized meeting batches for a window the local server does not have; the server answered 400 and the batch never fit.
Variables ¶
This section is empty.
Functions ¶
func EstimateTokens ¶
EstimateTokens intentionally overestimates multilingual meeting text. Exact provider tokenizers are not available for every selectable model, so a conservative common estimate is safer than retrying oversized prompts.
func SafeInputBudget ¶
Types ¶
type Chain ¶
type Chain struct {
// contains filtered or unexported fields
}
type Error ¶
type ErrorKind ¶
type ErrorKind string
const ( ErrorConfiguration ErrorKind = "configuration" ErrorAuthentication ErrorKind = "authentication" ErrorConsent ErrorKind = "consent" ErrorQuota ErrorKind = "quota" ErrorContextLimit ErrorKind = "context_limit" ErrorTransient ErrorKind = "transient" ErrorInvalidOutput ErrorKind = "invalid_output" ErrorCancelled ErrorKind = "cancelled" ErrorPermanent ErrorKind = "permanent" )
type GenkitGenerator ¶
type GenkitGenerator struct {
// contains filtered or unexported fields
}
func NewGenkit ¶
func NewGenkit(runtime *genkit.Genkit, models []GenkitModel) *GenkitGenerator
func (*GenkitGenerator) Models ¶
func (g *GenkitGenerator) Models(_ context.Context, query ModelQuery) (Catalog, error)
type GenkitModel ¶
type GenkitModel struct {
Model firebaseai.Model
Info Model
}
type Model ¶
type ModelQuery ¶
type ModelQuery struct {
Purpose Purpose
}