Documentation
¶
Overview ¶
Package runtime provides Runtime, the one value that owns an embedding product's provider state: the provider registry, the provider cache, the credential refresh coordinators, the catalog cache and provider health.
The package keeps no package state and reads no environment. A product supplies everything through Options, including its own settings type. The Runtime never interprets settings; it notices when their generation changes and hands the snapshot to the product's factories.
Import it under another name, such as coreruntime, in a file that also uses the standard library's runtime package.
Index ¶
- Constants
- type MemorySettings
- type Options
- type ProviderFactory
- type RefreshFactory
- type Runtime
- func (r *Runtime[S]) Health(instance string) core.ProviderHealthEvidence
- func (r *Runtime[S]) Invoke(ctx context.Context, caller core.Caller, instance string, request core.Request) (core.Response, error)
- func (r *Runtime[S]) ListModels(ctx context.Context, caller core.Caller, instance string) (core.CatalogRecord, error)
- func (r *Runtime[S]) Registry() *providers.Registry
- func (r *Runtime[S]) Stream(ctx context.Context, caller core.Caller, instance string, request core.Request) (core.StreamIter, error)
- type SettingsSource
Constants ¶
const DefaultCatalogTTL = 15 * time.Minute
DefaultCatalogTTL is how long a stored catalog is served before the Runtime rediscovers it, when Options.CatalogTTL is zero.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type MemorySettings ¶
type MemorySettings[S any] struct { // contains filtered or unexported fields }
MemorySettings is the in-memory reference SettingsSource.
func NewMemorySettings ¶
func NewMemorySettings[S any](settings S) *MemorySettings[S]
NewMemorySettings returns settings at generation 1.
func (*MemorySettings[S]) Snapshot ¶
func (m *MemorySettings[S]) Snapshot() (S, uint64)
Snapshot implements SettingsSource.
func (*MemorySettings[S]) Update ¶
func (m *MemorySettings[S]) Update(settings S) uint64
Update replaces the settings and returns the new generation.
type Options ¶
type Options[S any] struct { Settings SettingsSource[S] Providers ProviderFactory[S] // Credentials resolves and stores credentials. Nil sends every request // without one. Credentials core.CredentialStore // Refresh supplies each instance's OAuth refresh. Nil means credentials // never refresh. Refresh RefreshFactory[S] // Catalogs stores discovered catalogs. Nil keeps them in this Runtime's // memory only. Catalogs core.CatalogStore // Evidence receives account evidence. Nil discards it. Evidence core.EvidenceSink // Registry is the product's effective provider registry. Nil uses // providers.DefaultRegistry. Registry *providers.Registry // CatalogTTL bounds how long a stored catalog is served. Zero uses // DefaultCatalogTTL. CatalogTTL time.Duration // Now returns the current time. Nil uses time.Now. Now func() time.Time }
Options configures a Runtime. Settings and Providers are required.
type ProviderFactory ¶
ProviderFactory builds the provider of one instance from a settings snapshot. It constructs and must not block: the Runtime calls it under its lock, once per instance and settings generation.
type RefreshFactory ¶
type RefreshFactory[S any] func(settings S, instance string) tokenstore.RefreshFunc
RefreshFactory returns how one instance refreshes an OAuth credential, or nil when its credentials never refresh.
type Runtime ¶
type Runtime[S any] struct { // contains filtered or unexported fields }
Runtime performs provider operations for callers. It is safe for concurrent use; keep one per process.
func (*Runtime[S]) Health ¶
func (r *Runtime[S]) Health(instance string) core.ProviderHealthEvidence
Health returns the last observed health of an instance.
func (*Runtime[S]) Invoke ¶
func (r *Runtime[S]) Invoke(ctx context.Context, caller core.Caller, instance string, request core.Request) (core.Response, error)
Invoke performs one non-streaming operation for caller on instance. An OAuth credential the upstream rejects is refreshed once and the request replayed.
func (*Runtime[S]) ListModels ¶
func (r *Runtime[S]) ListModels(ctx context.Context, caller core.Caller, instance string) (core.CatalogRecord, error)
ListModels returns the catalog of instance for caller. A stored catalog is served while it is fresh: younger than the catalog TTL, observed after the last settings change, and not a failure. Another process's discovery counts, because catalogs are shared through the CatalogStore and keyed by the credential that discovered them. Discoveries of one catalog are serialized within this Runtime.
func (*Runtime[S]) Stream ¶
func (r *Runtime[S]) Stream(ctx context.Context, caller core.Caller, instance string, request core.Request) (core.StreamIter, error)
Stream performs one streaming operation for caller on instance. A rejected OAuth credential is refreshed and the stream reopened only while no frame has been delivered, that is when opening the stream fails.
type SettingsSource ¶
SettingsSource supplies a product's current settings and a generation that changes whenever they do. After a generation change the Runtime rebuilds providers and refresh coordinators on their next use, and rediscovers catalogs observed before the change, so hot-reloaded settings take effect without a restart.
Source Files