Documentation
¶
Overview ¶
Package runtime manages the lifecycle of a Provider with optional hooks and a post-processing Pipeline. It is the top-level entry point for traced, observable LLM interactions.
Index ¶
- type Runtime
- func (r *Runtime) Generate(ctx context.Context, req *niro.Request) (*niro.Stream, error)
- func (r *Runtime) WithCacheEngine(engine niro.CacheEngine) *Runtime
- func (r *Runtime) WithHook(h hook.Hook) *Runtime
- func (r *Runtime) WithPipeline(p *pipe.Pipeline) *Runtime
- func (r *Runtime) WithPrefixNormalizer(normalizer niro.PrefixNormalizer) *Runtime
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Runtime ¶
type Runtime struct {
// contains filtered or unexported fields
}
Runtime manages the lifecycle of a Provider with hooks and an optional post-processing Pipeline.
For simple use cases, calling Provider.Generate directly is fine. Runtime adds value when you want:
- Telemetry hooks on every generation
- Consistent post-processing pipeline
func (*Runtime) Generate ¶
Generate sends a request to the provider and returns a stream of frames, optionally processed through the attached pipeline. Hooks are invoked at each stage.
func (*Runtime) WithCacheEngine ¶
func (r *Runtime) WithCacheEngine(engine niro.CacheEngine) *Runtime
WithCacheEngine attaches an optional cache engine for provider adapters.
func (*Runtime) WithHook ¶
WithHook attaches a telemetry/observability Hook. Use hook.Compose() to combine multiple hooks.
func (*Runtime) WithPipeline ¶
WithPipeline attaches a post-processing Pipeline to the Runtime.
func (*Runtime) WithPrefixNormalizer ¶
func (r *Runtime) WithPrefixNormalizer(normalizer niro.PrefixNormalizer) *Runtime
WithPrefixNormalizer sets a deterministic prefix normalizer for cache key derivation.
Source Files