Documentation
¶
Index ¶
- func WaitFinish(ctx context.Context, timeout time.Duration)
- func WithFinishBarrier(ctx context.Context) (context.Context, chan struct{})
- func WithObserver(ctx context.Context, ob Observer) context.Context
- func WithRecorderObserver(ctx context.Context, client apiconv.Client) context.Context
- func WithRecorderObserverWithPrice(ctx context.Context, client apiconv.Client, provider TokenPriceProvider) context.Context
- type Buffer
- type Info
- type Observer
- type TokenPriceProvider
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func WaitFinish ¶
WaitFinish blocks until the barrier is signaled, ctx is done, or timeout elapses. When no barrier is present it returns immediately.
func WithFinishBarrier ¶
WithFinishBarrier attaches a barrier channel to ctx so that callers can wait until a model call has fully finished (including persistence) before proceeding with dependent actions (e.g., emitting final assistant message).
func WithObserver ¶
WithObserver stores a concrete Observer in context so providers can call it directly.
func WithRecorderObserver ¶
WithRecorderObserver injects a recorder-backed Observer into context.
func WithRecorderObserverWithPrice ¶ added in v0.2.9
Types ¶
type Info ¶
type Info struct {
Provider string
Model string
ModelKind string
RequestJSON []byte
ResponseJSON []byte
Payload []byte
LLMRequest *llm.GenerateRequest
LLMResponse *llm.GenerateResponse
StreamText string
Usage *llm.Usage
StartedAt time.Time
CompletedAt time.Time
Err string
ErrorCode string
FinishReason string
Cost *float64
}
Info carries a single model-call snapshot.
type Observer ¶
type Observer interface {
OnCallStart(ctx context.Context, info Info) (context.Context, error)
OnCallEnd(ctx context.Context, info Info) error
// OnStreamDelta delivers raw streamed chunks (provider-specific encoding).
// Implementations may aggregate plain text or persist progressive payloads.
// Returns error when persistence fails; callers may choose to abort stream.
OnStreamDelta(ctx context.Context, data []byte) error
}
Observer exposes OnCallStart/OnCallEnd used by providers.
func ObserverFromContext ¶
ObserverFromContext returns the explicitly injected Observer stored in ctx (or nil). Callers must inject an Observer (for example, WithRecorderObserver) before invoking LLM providers so that OnCallStart/OnCallEnd are delivered.
type TokenPriceProvider ¶ added in v0.2.9
type TokenPriceProvider interface {
TokenPrices(model string) (in float64, out float64, cached float64, ok bool)
}
WithRecorderObserverWithPrice injects a recorder-backed Observer with an optional price resolver used to compute per-call cost from token usage. TokenPriceProvider exposes per-1k token pricing for a model id/name.
func NewFixedModelPriceProvider ¶ added in v0.2.9
func NewFixedModelPriceProvider(base TokenPriceProvider, declared string) TokenPriceProvider
NewFixedModelPriceProvider creates a provider that ignores the lookup key and always returns prices for the declared model.
Source Files