Documentation
¶
Overview ¶
Package sse implements Server-Sent Events for Iris, in both directions: a push-based response stream, and a tolerant reader for consuming somebody else's.
It is transport only. It ships no application vocabulary, no replay store and no pub/sub fan-out. Those are application concerns that vary per service, and keeping them out is what lets this package stay small enough to be correct. The ai package builds the AI-shaped layer on top.
The writer half exists because the details that break a stream are neither obvious nor local. Compression buffers; a response recorder buffers; the server's absolute WriteTimeout guillotines a long generation; reverse proxies buffer unless told not to; and a client that vanishes without closing the connection stays invisible until the kernel gives up retransmitting, which takes about fifteen minutes. Open handles all of it for one response.
The reader half exists because every Go SDK that consumes a model's output reimplements event-stream parsing, usually by matching a "data: " prefix, and then breaks against a producer that omits the space.
stream, err := sse.Open(ctx)
if err != nil {
return
}
defer stream.Close(nil)
for token := range tokens {
if err := stream.Send(sse.Event{Name: "token", Data: []byte(token)}); err != nil {
return
}
}
Index ¶
- Constants
- Variables
- func Read(r io.Reader, opts ...ReadOptions) iter.Seq2[Event, error]
- type Coalescer
- type ErrorFrameFunc
- type Event
- type Options
- type ReadOptions
- type Stream
- func (s *Stream) Close(err error) error
- func (s *Stream) Coalesce(interval time.Duration, names ...string) *Coalescer
- func (s *Stream) Comment(text string) error
- func (s *Stream) Commit() error
- func (s *Stream) Committed() bool
- func (s *Stream) Context() stdcontext.Context
- func (s *Stream) Err() error
- func (s *Stream) LastEventID() string
- func (s *Stream) Send(ev Event) error
- func (s *Stream) SendJSON(name string, v any) error
Constants ¶
const DefaultMaxEventBytes = 1 << 20
DefaultMaxEventBytes caps one event when ReadOptions does not say otherwise.
Variables ¶
var ( // ErrNotStreamable reports that this response cannot flush, so nothing // written to it would reach the client until the handler returned. // // The usual cause is Timeout.HandlerTimeout, which wraps the router in // http.TimeoutHandler; its response writer implements neither http.Flusher // nor http.Hijacker. Open reports it before writing a byte, so the handler // can still answer with an ordinary status code. ErrNotStreamable = errors.New("iris: sse: response writer cannot flush") // ErrInvalidField reports a CR, LF or NUL in a field that is terminated by a // newline on the wire. Without the check a caller-supplied value forges // frames, and LastEventID echoes something the client chose. ErrInvalidField = errors.New("iris: sse: field contains CR, LF or NUL") // ErrSlowClient reports that a single Send exceeded SendTimeout, meaning the // client is not draining the connection. ErrSlowClient = errors.New("iris: sse: write deadline exceeded") // ErrClosed reports a Send on a stream that has already been closed. ErrClosed = errors.New("iris: sse: stream closed") // ErrTooLarge reports an event larger than ReadOptions.MaxEventBytes. ErrTooLarge = errors.New("iris: sse: event exceeds the maximum size") // ErrNoTerminator reports a stream that ended cleanly without any of the // configured ReadOptions.Terminators while RequireTerminator was set. It // matches io.ErrUnexpectedEOF as well, because that is what it is: the // producer stopped before it said it was done. ErrNoTerminator = fmt.Errorf("iris: sse: stream ended without a terminator: %w", io.ErrUnexpectedEOF) )
Errors reported by this package. Match them with errors.Is.
Functions ¶
func Read ¶
Read parses an event stream and yields its events.
It implements the WHATWG parsing algorithm rather than a prefix match, which matters more than it sounds: "data:" with and without a space are both legal, and a reader that matches "data: " sees an empty stream from a producer that omits it. It also handles multi-line data joined with "\n", comments, unknown fields, a leading byte order mark, and lines terminated by CR, LF or CRLF.
It never yields a partially framed event. An event is dispatched by the blank line that ends it, so a stream cut mid-event ends the iteration with io.ErrUnexpectedEOF instead of yielding half an event, or worse, ending quietly as if the answer were complete. The terminal error, if any, comes last and io.EOF is not one: a clean end of stream simply ends the iteration, unless RequireTerminator says otherwise.
Lenient on parse, strict on emit. A closed parser rejects a valid-but-newer producer, and an event stream is exactly the kind of format producers extend.
resp, err := client.Do(req)
// ...
defer resp.Body.Close()
for ev, err := range sse.Read(resp.Body, sse.ReadOptions{Terminators: []string{"[DONE]"}}) {
if err != nil {
return err
}
// ev.Name, ev.Data
}
Types ¶
type Coalescer ¶
type Coalescer struct {
// contains filtered or unexported fields
}
Coalescer batches named events on a stream. See Stream.Coalesce.
func (*Coalescer) Close ¶
Close flushes what is pending, stops the ticker and hands the stream back. It is idempotent.
type ErrorFrameFunc ¶
ErrorFrameFunc renders the event Close writes when a stream fails after it has already committed. It reports false to write no frame at all.
type Event ¶
type Event struct {
// ID is the "id" field. It becomes the client's Last-Event-ID.
ID string
// Name is the "event" field. Empty means the default, "message", which is
// what an EventSource onmessage handler receives.
Name string
// Data is the payload. A multi-line value is split across several "data"
// fields and rejoined by the client with "\n" between them.
//
// A nil Data with a Comment writes a comment-only frame, which dispatches no
// event. A non-nil but empty Data dispatches an event with an empty payload.
Data []byte
// Retry sets the client's reconnection time for this and later events.
Retry time.Duration
// Comment is written as an SSE comment before the fields. Conforming clients
// ignore it; it exists for keep-alives and for human-readable markers.
Comment string
}
Event is one frame. A zero field is absent from the wire.
func RawErrorFrame ¶
RawErrorFrame renders the error's own text, with no mapping, no redaction and no reporting.
It is the shape this package used before ReportErrorFrame became the default, and it is appropriate only when the error text is already known to be safe for the client to read. Anything derived from an upstream response is not.
func ReportErrorFrame ¶
ReportErrorFrame is the default renderer for a failure that arrives after the first event.
It runs the error through rest/errors.Report, which is what makes a mid-stream failure behave like every other failure in the application: the route's error mappings apply, the logger and the collector fire so a cloud sink sees it, a correlation id is generated, and an internal or private error is redacted to a generic message before anything reaches the client.
That redaction is the part worth noticing. An unmapped error becomes "server error" on the wire, so an upstream message naming a vendor, a model or a host cannot leak into a stream by default. Map the errors you do want to describe, through errors.Options.MapErrors and its relatives, exactly as for an ordinary response.
The payload is the rest/errors Error envelope, so a client parses the same shape whether the failure happened before the first byte or after it.
It writes no frame when Report has nothing to report, which is the case for a client that disconnected while errors.SkipCanceled is on: there is nobody left to tell, and nobody should be paged.
type Options ¶
type Options struct {
// Heartbeat is the longest the stream may go without writing anything.
// Default 15s. Zero disables it.
//
// 15s is not a round number, it is the largest value that survives every
// common intermediary at once: nginx proxy_read_timeout is 60s, an AWS
// Application Load Balancer idles out at 60s, Cloudflare returns a 524 at
// 100s, and Heroku's router applies a rolling 55s. Note that a Google Cloud
// Application Load Balancer applies its backend timeout to the whole request
// and response, so no heartbeat rescues it; raise timeoutSec there instead.
//
// Its larger job is liveness, not keep-alive. Only a real FIN or RST, or a
// failing write, cancels a request context. A client that disappears without
// one - a closed lid, a dropped Wi-Fi link, an evicted NAT entry - stays
// invisible until TCP gives up, which on Linux means tcp_retries2 = 15 and
// roughly 924 seconds. Writing something is what lets the kernel fail and
// the handler find out, and it is why this bounds idle time rather than
// firing on a fixed tick.
Heartbeat time.Duration
// HeartbeatFrame is written verbatim when the stream has been idle for
// Heartbeat. Default ": ping\n\n", an SSE comment, which conforming clients
// ignore.
//
// It is configurable rather than fixed because not every client conforms:
// sashabaranov/go-openai, widely used against OpenAI-compatible servers,
// counts comment frames toward emptyMessagesLimit and fails the stream
// instead of ignoring them. Serving such a client, send a real event.
HeartbeatFrame []byte
// Retry advertises how long the client should wait before reconnecting,
// written once with the first event. Default 3s. Zero omits the field and
// leaves the client's own default in place.
Retry time.Duration
// AutoID stamps a monotonically increasing id on every event that does not
// carry one. Default true.
//
// It makes gaps detectable within one connection, for the cost of a few
// bytes per frame: a client that sees id 7 after id 5 knows it missed one.
// It is not resumption. The counter starts at 1 on every stream, so a
// Last-Event-ID from a previous connection means nothing to it; a resumable
// stream needs ids of the application's own, tied to its store.
AutoID bool
// Terminator is the event name of the frame Close writes. Default "done".
// Empty writes no terminal frame.
//
// A reserved event name, never an in-band data sentinel. A sentinel such as
// [DONE] travels on the same channel as the payload, so a model that happens
// to emit the literal string ends the client's stream.
Terminator string
// ErrorFrame renders the event Close writes when the stream failed after it
// had already committed. Nil selects ReportErrorFrame.
//
// The default runs the error through rest/errors, so a mid-stream failure is
// mapped, logged, collected and redacted exactly like any other failure in
// the application. Set RawErrorFrame to send the error's own text instead,
// which is safe only when that text did not come from an upstream.
ErrorFrame ErrorFrameFunc
// SendTimeout bounds one Send with a write deadline. Default 30s; zero
// disables it.
//
// This is the backpressure mechanism, and it is a deadline rather than a
// queue on purpose. Send is synchronous, so there is no buffer of ours to
// budget; a client that will not drain shows up as a write that does not
// complete. Exceeding it fails the stream with ErrSlowClient and never skips
// an event, because a dropped delta corrupts the assembled message with no
// way for the client to detect it.
SendTimeout time.Duration
}
Options configure one stream. Start from DefaultOptions and change what you need: a zero field means "off" rather than "default", so that disabling the heartbeat or the terminator stays expressible.
func DefaultOptions ¶
func DefaultOptions() Options
DefaultOptions returns the recommended settings. See each field for why.
type ReadOptions ¶
type ReadOptions struct {
// Terminators end the iteration when an event's data equals one of them
// exactly. Set []string{"[DONE]"} for an OpenAI-dialect upstream.
//
// Empty by default. An in-band sentinel is a compatibility concession on the
// reading side and never a mechanism this package emits: it travels on the
// same channel as the payload, so a producer that legitimately emits the
// literal string ends the stream.
Terminators []string
// RequireTerminator ends the iteration with ErrNoTerminator when the stream
// ends cleanly without one of the Terminators. It has no effect when
// Terminators is empty.
//
// An upstream that promised a sentinel and closed without it was cut - by a
// proxy idle timeout, by a crashed worker - however clean the close looked to
// the transport. Off by default because a plain event stream has no
// sentinel to miss; Relay turns it on whenever Terminators are configured.
RequireTerminator bool
// MaxEventBytes caps one event. Default DefaultMaxEventBytes. Exceeding it
// ends the iteration with ErrTooLarge.
MaxEventBytes int
// KeepComments yields comment-only frames, such as an upstream's own
// heartbeat, instead of skipping them. Off by default, because a conforming
// client is required to ignore them.
KeepComments bool
}
ReadOptions configure Read.
type Stream ¶
type Stream struct {
// contains filtered or unexported fields
}
Stream is one open Server-Sent Events response.
It is safe for concurrent use: Send, Comment and Close take the same lock the heartbeat does, so a keep-alive can never interleave with a partially written frame.
func Open ¶
Open prepares an SSE response and writes nothing.
The protocol headers and every adjustment below are applied on the first Send, not here, so an authorization check, a validation failure or a policy veto after Open still produces an ordinary status code with an ordinary body, and a handler that emits nothing and returns leaves the response untouched. That one decision is what every complaint about a framework's SSE support comes back to: open the stream before the handler runs and you can no longer return a 401, set a header, or let an error handler fire.
Open does verify up front that this response can flush, because the answer is only available at flush time otherwise, which is far too late: under Timeout.HandlerTimeout the writer silently buffers into a 503 HTML page. ErrNotStreamable is reported with zero bytes written.
On the first Send, and for this response only, it then does the five things no middleware can guess:
- removes every buffering response-writer wrapper, so compression and the response recorder that accesslog installs stop holding the events;
- clears the server's absolute WriteTimeout, which is a total deadline and would otherwise guillotine a long generation mid-sentence;
- enables full duplex on HTTP/1 so a POST body can still be read after the response has started, which is how every LLM API is actually called;
- sets Cache-Control: no-cache, no-transform and X-Accel-Buffering: no, and deletes any Content-Length;
- sets Connection: keep-alive only on HTTP/1, where it is meaningful. It is a protocol error on HTTP/2.
Close the stream, normally with defer.
func (*Stream) Close ¶
Close writes the terminal frame and releases the stream. It is idempotent, so a deferred Close and an explicit one coexist, and a panicking handler still terminates the stream cleanly.
The terminal frame is written on success too, and that is the point. If the terminator is always present, its absence is how a client detects truncation. Nothing below it provides that signal: chunked framing catches a torn connection, but an nginx or load-balancer idle timeout closes the stream cleanly and the client cannot tell a finished answer from a severed one.
When err is non-nil and nothing has been committed, Close writes nothing and returns err, so the caller renders an ordinary HTTP error. After the first Send it flushes any text an attached Coalescer still holds, writes the error event Options.ErrorFrame renders, and then the terminator. This package has no view on which errors are safe to show a client: the default frame runs the error through rest/errors, which redacts what was not deliberately mapped; use ai.Open, or map the error yourself before passing it here.
func (*Stream) Coalesce ¶
Coalesce batches the named events and emits at most one frame per name per interval, concatenating their payloads.
It exists because a fast model is a syscall amplifier. One frame per token means one write, one flush, one TCP segment and one client-side render per token, and the tokens arrive faster than any of those are worth doing.
The payloads are concatenated, so this is for text-like deltas - the running prose of an answer, a reasoning trace - and not for events whose data is a self-contained JSON document. Anything not named here passes straight through, after the pending buffers are flushed, so ordering is preserved.
A background ticker drains the buffers, which is what makes the interval a ceiling rather than a floor. Without one, a buffered delta is only emitted when the next one arrives, so text that already reached the server sits unsent through a mid-answer pause - and a pause is exactly when a reader is most likely to be waiting.
The returned Coalescer owns the stream until it is closed. Calling Stream.Send directly still works and flushes the pending buffers first, which is why the ordering hazard a hand-rolled batcher has to document - close the batcher before writing to the same writer, or the two interleave - cannot be expressed here.
func (*Stream) Commit ¶
Commit writes the status line and the stream headers now, with no event.
Send does this on its own, so most handlers never call it. It exists for a response that is a stream whether or not an event ever arrives - an answer that turned out to be empty - so that Close can still write the terminator and the client can tell an empty answer from a connection that dropped before the first byte. The ai package uses it for a zero-part turn.
Everything Open defers to the first Send happens here instead. The reconnection time is still advertised with the first event.
func (*Stream) Committed ¶
Committed reports whether any byte has reached the client.
It is the line between "this is still an error response" and "this is a truncated answer", and it is what a failover ladder must consult before retrying an upstream: after the first delta has gone out, a second attempt splices two different answers into one response.
func (*Stream) Context ¶
func (s *Stream) Context() stdcontext.Context
Context is done when the client disconnects, when a write fails, or when the stream is closed. Select on it together with Context.ShuttingDown so a graceful shutdown ends the stream cleanly rather than waiting for its deadline.
func (*Stream) Err ¶
Err reports why the stream ended: the error Close was given or the write failure that ended it, nil after a clean Close, and nil while the stream is healthy. Before Close, a client that went away shows up here as the request context's cause.
func (*Stream) LastEventID ¶
LastEventID reports the resume cursor the client sent, from the Last-Event-ID header or the lastEventId query parameter. It is empty on a fresh connection and can never contain CR, LF or NUL.