Documentation
¶
Overview ¶
Package lib holds the parts of a Provider that do not depend on which API answered: reading a server sent event stream, keeping a deadline on one, and handing a caller events while the wire keeps moving.
Hosts use Options, passed to a provider's WithTransport to tune the idle deadline and the frame size cap; ErrIdle can reach them inside an api.EventErr.
Index ¶
- Constants
- Variables
- func Drain(events ...api.Event) <-chan api.Event
- func ErrorMessage(body []byte) string
- func ImageMime(mime string) (string, bool)
- func MarshalRaw(v any) ([]byte, error)
- func MeasuredTooLarge(model string, sent int, declared int, err error) error
- func Omitted(c api.Content, why string) string
- func RetryAfter(h http.Header) time.Duration
- func Stream(parentCtx context.Context, client *http.Client, req *http.Request, ...) <-chan api.Event
- func Unknown(provider string, message int, c api.Content) string
- type APIError
- type Body
- type Converter
- type Frame
- type HTTPError
- type Options
Constants ¶
const DefaultIdle = 2 * time.Minute
DefaultIdle is the idle deadline when Options.Idle is unset. There is no "never" value; pass a long duration instead.
const DefaultMaxFrame = 64 << 20
DefaultMaxFrame is the frame cap when Options.Max is unset. It is generous because the frame that ends a turn can carry the whole response, and exists only to keep a broken peer from exhausting memory. There is no "no cap" value; pass a larger number instead.
const EmptyResult = "(the tool returned no output)"
EmptyResult is the text an adapter sends for a tool result that carries no blocks, so the model reads that the call succeeded and returned nothing rather than an empty output or a note about content that never existed.
Variables ¶
var ErrIdle = errors.New("neui: stream went idle")
ErrIdle is the cancellation cause Stream sets when Options.Idle elapses, and what End is usually handed then; a read that fails from the same cancellation can reach End first with its own error.
Functions ¶
func Drain ¶
Drain is a turn that ended before it began, for a request that could not be built out of what the caller asked for. The events are already in the channel and it is already closed, so nothing blocks on it and a caller that never reads it simply drops it.
func ErrorMessage ¶ added in v0.1.12
ErrorMessage returns an error response body as the Message of an APIError, for a body that was not the JSON the API documents: trimmed of surrounding space and cut to 512 characters, with "…" marking the cut. At a gateway the body is often a whole HTML page.
func ImageMime ¶ added in v0.1.12
ImageMime normalizes the media type of an image for the wire: lower case, surrounding space trimmed. It reports whether the result is one every provider takes, which is the four api.ImageMime values and "image/jpg", the common misspelling of image/jpeg; an adapter sends a note in place of any other image.
func MarshalRaw ¶
MarshalRaw is json.Marshal without HTML escaping: <, > and & stay as they are, including inside json.RawMessage values, which json.Marshal would rewrite as \u escapes. Raw messages are still compacted. There is no trailing newline.
func MeasuredTooLarge ¶ added in v0.1.12
MeasuredTooLarge wraps err, the API's refusal of a request by its size, in api.ErrMeasuredTooLarge when the model declares a request limit: the body of sent bytes was measured against the declared limit and sent because it fit, so the limit over-promises and no smaller request fixes it. With declared at zero or less the request was never measured and err is returned as it is.
func Omitted ¶
Omitted returns the placeholder text an adapter sends in place of content c that this wire cannot carry, such as "[image omitted]", so the turn keeps its shape instead of failing. It logs nothing: an omission is a fixed consequence of the stored thread and recurs on every turn. why says at the call site why the content cannot go and never reaches the model.
func RetryAfter ¶ added in v0.1.12
RetryAfter reads the Retry-After header in either form RFC 9110 gives it, a whole number of seconds or an HTTP date. It returns zero for a header that is absent, unreadable, negative or in the past.
func Stream ¶
func Stream(parentCtx context.Context, client *http.Client, req *http.Request, conv Converter, opt Options) <-chan api.Event
Stream sends req with client and returns the turn's events as conv produces them from the response, applying the opt defaults. The channel closes after Convert returns false or End has run, once their events are delivered. Cancelling parentCtx ends the turn, and the channel closes without waiting for queued events to be read; otherwise the caller must read the channel until it closes, or the streaming goroutine leaks.
func Unknown ¶
Unknown returns the placeholder an adapter sends in place of content c whose type it has no case for, and logs that at Error once per provider and type per process: a missing type is a bug in the adapter, and the stored thread would otherwise repeat the record on every turn. message is the index of the api.Message holding c.
Types ¶
type APIError ¶ added in v0.1.12
type APIError struct {
// Provider is the name of the provider that built the error, such as
// "claude".
Provider string
// Status is the HTTP status, zero for a failure that arrived inside the
// stream.
Status int
// Type is the API's error type, such as "invalid_request_error".
Type string
// Code is the API's error code, such as "context_length_exceeded". For a
// turn the API ended without a reply it is the stop reason exactly as the
// API sent it.
Code string
// Message is the API's explanation. For a response body that was not the
// documented JSON it is the trimmed body, cut to 512 characters, and for
// a stop reason it is the adapter's own description.
Message string
// Param is the request parameter the error is about, where the API names
// one.
Param string
// RequestID is the API's id for the request, to quote to its support.
RequestID string
// RetryAfter is the Retry-After header's wait, zero when the header is
// absent or unreadable.
RetryAfter time.Duration
// NoRetry says the response advised against retrying, which only OpenAI
// does, with x-should-retry: false.
NoRetry bool
}
APIError is a failure the API itself reported, as opposed to one the connection produced: a non 200 response, an error the API put inside a stream that had already returned 200, or a turn the API ended without a reply worth keeping, such as one cut at the token cap or refused. Each provider package names it as its own APIError.
It arrives as the Err of an api.EventErr, wrapped after a 413 on a request measured against Model.RequestBytes; read it with errors.As. Branch on the event's Code, and use these fields for logs and support requests. Which fields an API fills differs, and an empty one means it gave none.
type Body ¶
type Body struct {
// contains filtered or unexported fields
}
Body is a request body encoded the way MarshalRaw encodes, into a pooled buffer. Hand the request built by Request to Stream, which recycles the buffer once the exchange is over; a Body that never reaches Stream needs no cleanup.
func MarshalBody ¶
MarshalBody encodes v as MarshalRaw does, into a pooled buffer.
type Converter ¶
type Converter interface {
// Convert turns one frame into the events it carries. The frame's Data is
// reused once Convert returns, so neither the converter nor the events
// may keep any of it but copies. Returning false says the turn is over
// and nothing further will be read, which is what bounds a peer that
// keeps writing after saying how the reply ended. End is not called after
// it: the converter has already said how the turn ended.
Convert(Frame) ([]api.Event, bool)
// End is every other ending, never called with nil: io.EOF for a body that
// ended between events, io.ErrUnexpectedEOF for one cut mid event, an
// *HTTPError for a non-200 response, ErrIdle (or the read error it caused)
// for the idle deadline, the cancellation error when parentCtx is
// cancelled, or whatever reading failed with, including an oversized
// frame. Only the adapter can classify any of them, so only the adapter
// builds the event; returning nothing ends the turn with no terminal event,
// which says the reply is incomplete.
End(err error) []api.Event
}
Converter is the whole of an adapter that Stream needs: the wire format, and what a failure means in it. Stream itself constructs no event.
Both methods run on the driving goroutine and are never called concurrently, so a converter may keep whatever state its wire format forces on it — an index remapping, a tool call being assembled — without a lock.
type Frame ¶
type Frame struct {
Event string // value of the event: line; "" when absent
Data []byte // data: lines joined with "\n"; never empty
}
Frame is one dispatched server sent event: the value of its event: line and its data: lines joined with newlines. Data is valid only until Converter.Convert returns, so keep only copies of it; json.Unmarshal into a string, []byte or json.RawMessage already copies.
type HTTPError ¶
type HTTPError struct {
Status int // HTTP status code, never 200
Header http.Header // response headers, for retry-after and request ids
Body []byte // response body, at most 1 MiB
}
HTTPError is a response the API refused, handed to Converter.End as raw material rather than as a verdict: the status classifies, the headers carry retry-after and whatever request id the API uses, and the body is the API's own account of it. Every one of those is spelled differently per API, so the adapter decodes it and Stream does not look inside.
type Options ¶
type Options struct {
// Idle ends a turn when no complete frame or keepalive has arrived for
// this long, counting from the request until response headers and then
// from each frame; a frame that takes longer than Idle to arrive in full
// trips it even while bytes flow. A slow consumer does not count. Zero or
// less is DefaultIdle.
Idle time.Duration
// Max caps the bytes of one frame's data and of any single line; going
// over ends the reading and End receives an error. Zero or less is
// DefaultMaxFrame.
Max int
}
Options are the transport limits Stream keeps on a turn. Hosts pass them to a provider's WithTransport option, which hands them here unread; the zero value takes both defaults.