Documentation
¶
Index ¶
- func RedactGenerateRequestForTranscript(req *llm.GenerateRequest) []byte
- func WaitFinish(ctx context.Context, timeout time.Duration)
- func WithFinishBarrier(ctx context.Context) (context.Context, chan struct{})
- func WithObserver(ctx context.Context, ob Observer) context.Context
- func WithRecorderObserver(ctx context.Context, client apiconv.Client) context.Context
- func WithRecorderObserverWithPrice(ctx context.Context, client apiconv.Client, provider TokenPriceProvider) context.Context
- func WithStreamPublisher(ctx context.Context, p StreamPublisher) context.Context
- type Buffer
- type Info
- type Observer
- type StreamEvent
- type StreamPublisher
- type TokenPriceProvider
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func RedactGenerateRequestForTranscript ¶ added in v0.2.30
func RedactGenerateRequestForTranscript(req *llm.GenerateRequest) []byte
RedactGenerateRequestForTranscript returns a JSON snapshot of the request suitable for persisting into conversation transcripts.
It removes large base64 payloads from message items (for example image/PDF attachments) while keeping enough metadata to understand that an attachment was present. This avoids exploding transcript size and log views.
func WaitFinish ¶
WaitFinish blocks until the barrier is signaled, ctx is done, or timeout elapses. When no barrier is present it returns immediately.
func WithFinishBarrier ¶
WithFinishBarrier attaches a barrier channel to ctx so that callers can wait until a model call has fully finished (including persistence) before proceeding with dependent actions (e.g., emitting final assistant message).
func WithObserver ¶
WithObserver stores a concrete Observer in context so providers can call it directly.
func WithRecorderObserver ¶
WithRecorderObserver injects a recorder-backed Observer into context.
func WithRecorderObserverWithPrice ¶ added in v0.2.9
func WithStreamPublisher ¶ added in v0.2.38
func WithStreamPublisher(ctx context.Context, p StreamPublisher) context.Context
WithStreamPublisher injects a StreamPublisher into context.
Types ¶
type Info ¶
type Info struct {
Provider string
Model string
ModelKind string
RequestJSON []byte
ResponseJSON []byte
Payload []byte
LLMRequest *llm.GenerateRequest
LLMResponse *llm.GenerateResponse
StreamText string
Usage *llm.Usage
StartedAt time.Time
CompletedAt time.Time
Err string
ErrorCode string
FinishReason string
Cost *float64
}
Info carries a single model-call snapshot.
type Observer ¶
type Observer interface {
OnCallStart(ctx context.Context, info Info) (context.Context, error)
OnCallEnd(ctx context.Context, info Info) error
// OnStreamDelta delivers raw streamed chunks (provider-specific encoding).
// Implementations may aggregate plain text or persist progressive payloads.
// Returns error when persistence fails; callers may choose to abort stream.
OnStreamDelta(ctx context.Context, data []byte) error
}
Observer exposes OnCallStart/OnCallEnd used by providers.
func ObserverFromContext ¶
ObserverFromContext returns the explicitly injected Observer stored in ctx (or nil). Callers must inject an Observer (for example, WithRecorderObserver) before invoking LLM providers so that OnCallStart/OnCallEnd are delivered.
type StreamEvent ¶ added in v0.2.38
type StreamEvent struct {
ConversationID string
Message *agconv.MessageView
ContentType string
Content interface{}
}
StreamEvent describes a streaming-only message envelope for instant updates. It is not persisted; consumers should treat it as ephemeral.
type StreamPublisher ¶ added in v0.2.38
type StreamPublisher interface {
Publish(ctx context.Context, ev *StreamEvent) error
}
StreamPublisher publishes ephemeral stream events (e.g., token deltas).
func StreamPublisherFromContext ¶ added in v0.2.38
func StreamPublisherFromContext(ctx context.Context) (StreamPublisher, bool)
StreamPublisherFromContext returns a StreamPublisher from context.
type TokenPriceProvider ¶ added in v0.2.9
type TokenPriceProvider interface {
TokenPrices(model string) (in float64, out float64, cached float64, ok bool)
}
WithRecorderObserverWithPrice injects a recorder-backed Observer with an optional price resolver used to compute per-call cost from token usage. TokenPriceProvider exposes per-1k token pricing for a model id/name.
func NewFixedModelPriceProvider ¶ added in v0.2.9
func NewFixedModelPriceProvider(base TokenPriceProvider, declared string) TokenPriceProvider
NewFixedModelPriceProvider creates a provider that ignores the lookup key and always returns prices for the declared model.
Source Files