Documentation
¶
Overview ¶
Package lib holds the parts of a Provider that do not depend on which API answered: reading a server sent event stream, keeping a deadline on one, and handing a caller events while the wire keeps moving.
Hosts use Options, passed to a provider's WithTransport to tune the idle deadline and the frame size cap; ErrIdle can reach them inside an api.EventErr.
Index ¶
- Constants
- Variables
- func Drain(events ...api.Event) <-chan api.Event
- func MarshalRaw(v any) ([]byte, error)
- func Omitted(c api.Content, why string) string
- func Stream(parentCtx context.Context, client *http.Client, req *http.Request, ...) <-chan api.Event
- func Unknown(provider string, message int, c api.Content) string
- type Body
- type Converter
- type Frame
- type HTTPError
- type Options
Constants ¶
const DefaultIdle = 2 * time.Minute
DefaultIdle is the idle deadline when Options.Idle is unset. There is no "never" value; pass a long duration instead.
const DefaultMaxFrame = 64 << 20
DefaultMaxFrame is the frame cap when Options.Max is unset. It is generous because the frame that ends a turn can carry the whole response, and exists only to keep a broken peer from exhausting memory. There is no "no cap" value; pass a larger number instead.
const EmptyResult = "(the tool returned no output)"
EmptyResult is the text an adapter sends for a tool result that carries no blocks, so the model reads that the call succeeded and returned nothing rather than an empty output or a note about content that never existed.
Variables ¶
var ErrIdle = errors.New("neui: stream went idle")
ErrIdle is the cancellation cause Stream sets when Options.Idle elapses, and what End is usually handed then; a read that fails from the same cancellation can reach End first with its own error.
Functions ¶
func Drain ¶
Drain is a turn that ended before it began, for a request that could not be built out of what the caller asked for. The events are already in the channel and it is already closed, so nothing blocks on it and a caller that never reads it simply drops it.
func MarshalRaw ¶
MarshalRaw is json.Marshal without HTML escaping: <, > and & stay as they are, including inside json.RawMessage values, which json.Marshal would rewrite as \u escapes. Raw messages are still compacted. There is no trailing newline.
func Omitted ¶
Omitted returns the placeholder text an adapter sends in place of content c that this wire cannot carry, such as "[image omitted]", so the turn keeps its shape instead of failing. It logs nothing: an omission is a fixed consequence of the stored thread and recurs on every turn. why says at the call site why the content cannot go and never reaches the model.
func Stream ¶
func Stream(parentCtx context.Context, client *http.Client, req *http.Request, conv Converter, opt Options) <-chan api.Event
Stream sends req with client and returns the turn's events as conv produces them from the response, applying the opt defaults. The channel closes after Convert returns false or End has run, once their events are delivered. Cancelling parentCtx ends the turn, and the channel closes without waiting for queued events to be read; otherwise the caller must read the channel until it closes, or the streaming goroutine leaks.
func Unknown ¶
Unknown returns the placeholder an adapter sends in place of content c whose type it has no case for, and logs that at Error once per provider and type per process: a missing type is a bug in the adapter, and the stored thread would otherwise repeat the record on every turn. message is the index of the api.Message holding c.
Types ¶
type Body ¶
type Body struct {
// contains filtered or unexported fields
}
Body is a request body encoded the way MarshalRaw encodes, into a pooled buffer. Hand the request built by Request to Stream, which recycles the buffer once the exchange is over; a Body that never reaches Stream needs no cleanup.
func MarshalBody ¶
MarshalBody encodes v as MarshalRaw does, into a pooled buffer.
type Converter ¶
type Converter interface {
// Convert turns one frame into the events it carries. The frame's Data is
// reused once Convert returns, so neither the converter nor the events
// may keep any of it but copies. Returning false says the turn is over
// and nothing further will be read, which is what bounds a peer that
// keeps writing after saying how the reply ended. End is not called after
// it: the converter has already said how the turn ended.
Convert(Frame) ([]api.Event, bool)
// End is every other ending, never called with nil: io.EOF for a body that
// ended between events, io.ErrUnexpectedEOF for one cut mid event, an
// *HTTPError for a non-200 response, ErrIdle (or the read error it caused)
// for the idle deadline, the cancellation error when parentCtx is
// cancelled, or whatever reading failed with, including an oversized
// frame. Only the adapter can classify any of them, so only the adapter
// builds the event; returning nothing ends the turn with no terminal event,
// which says the reply is incomplete.
End(err error) []api.Event
}
Converter is the whole of an adapter that Stream needs: the wire format, and what a failure means in it. Stream itself constructs no event.
Both methods run on the driving goroutine and are never called concurrently, so a converter may keep whatever state its wire format forces on it — an index remapping, a tool call being assembled — without a lock.
type Frame ¶
type Frame struct {
Event string // value of the event: line; "" when absent
Data []byte // data: lines joined with "\n"; never empty
}
Frame is one dispatched server sent event: the value of its event: line and its data: lines joined with newlines. Data is valid only until Converter.Convert returns, so keep only copies of it; json.Unmarshal into a string, []byte or json.RawMessage already copies.
type HTTPError ¶
type HTTPError struct {
Status int // HTTP status code, never 200
Header http.Header // response headers, for retry-after and request ids
Body []byte // response body, at most 1 MiB
}
HTTPError is a response the API refused, handed to Converter.End as raw material rather than as a verdict: the status classifies, the headers carry retry-after and whatever request id the API uses, and the body is the API's own account of it. Every one of those is spelled differently per API, so the adapter decodes it and Stream does not look inside.
type Options ¶
type Options struct {
// Idle ends a turn when no complete frame or keepalive has arrived for
// this long, counting from the request until response headers and then
// from each frame; a frame that takes longer than Idle to arrive in full
// trips it even while bytes flow. A slow consumer does not count. Zero or
// less is DefaultIdle.
Idle time.Duration
// Max caps the bytes of one frame's data and of any single line; going
// over ends the reading and End receives an error. Zero or less is
// DefaultMaxFrame.
Max int
}
Options are the transport limits Stream keeps on a turn. Hosts pass them to a provider's WithTransport option, which hands them here unread; the zero value takes both defaults.