Documentation
¶
Overview ¶
Package outbox keeps small measurement rows in a local append-only file and hands them to a destination: an HTTP endpoint that takes newline-delimited JSON, or another file. One line is one row, so the rows survive a restart and Pending() is answered by reading the file back. Nothing here gets in the caller's way — a full disk, a dead server, a destination that is nonsense all surface as error values — and a row whose batch arrived is marked in the file, so it is never transmitted twice.
The file holds rows and, interleaved with them, one-line sent and dropped markers that retire rows; a line that is not a row is skipped on read. That is what keeps the file append-only while a cap on pending rows still drops from the old end and a mark outlives closing the outbox and opening the same path again.
Append-only also means the file grows a line for every row ever judged, and a long-lived run would read the whole of that back on every Send and every Pending. So a Send whose file has grown mostly into rows it can no longer need rewrites it to the pending rows alone — through a temporary file in the same directory, never in place — and the markers for rows that are gone go with the rows.
Index ¶
Constants ¶
const DefaultMaxPending = 5000
DefaultMaxPending is the cap on pending rows when Outbox.MaxPending is zero or below.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Drop ¶
Drop is one row a dropped marker retired without sending it: the row's nonce and the reason it was dropped — a destination's refusal text, or the cap's own word. A file written before reasons were kept answers with the reason empty.
type Outbox ¶
type Outbox struct {
// Now answers the time a row was appended; nil means time.Now.
Now func() time.Time
// Rand is read for nonce bytes; nil means crypto/rand.Reader.
Rand io.Reader
// Keep decides, at Send time, which payloads are transmitted; nil keeps
// every row. A payload Keep declines leaves the outbox without reaching
// the destination.
Keep func(payload json.RawMessage) bool
// Budget bounds one Send call as a whole; zero means 2s.
Budget time.Duration
// Client sends the batches; nil means a client the package makes.
Client *http.Client
// MaxPending caps the pending rows, dropping from the old end; zero or
// below means DefaultMaxPending.
MaxPending int
// Install is the install's own nonce, carried as the X-Codeaf-Install
// header on every batch sent over http; empty sends no header. A
// destination that requires the header names it in its error, so a batch
// sent without one is a batch the destination refuses — the caller sets
// this from its own store before the first Send.
Install string
// contains filtered or unexported fields
}
Outbox keeps rows in one append-only file until they are sent. The six exported fields are read every time they are used, so a caller may set them after Open and before the first Append or Send.
func Open ¶
Open prepares the outbox file at path, creating the parent directory with mode 0700 if it is missing and the file itself with mode 0600. A path that already holds rows keeps them, markers included. An empty path is an error.
func (*Outbox) Append ¶
func (o *Outbox) Append(payload json.RawMessage) error
Append writes one row holding payload. The row is refused, and nothing is written, when the payload is not valid JSON, when it is longer than 64 KiB once compacted, or when it still holds a newline once compacted. The payload is stored compacted, so one row is always one line.
func (*Outbox) Close ¶
Close releases whatever the outbox holds open. Calling it more than once is not an error. A closed outbox refuses Append and Send; Pending still answers by reading the file.
func (*Outbox) Dropped ¶
Dropped answers the rows a dropped marker retired without sending them, in the order they were dropped — oldest first, so the last is the most recent drop and the reason beside it is the one a reading form shows. It never fails: a line that does not parse is skipped, and a file that cannot be read answers as no drops at all. A file that has been compacted holds only its pending rows, so the dropped markers of rows a rewrite took go with them.
func (*Outbox) Pending ¶
Pending answers the rows that have not been sent, in the order they were appended. It never fails: a line that does not parse, or that is not a row, is skipped, and a file that cannot be read answers as no rows at all.
func (*Outbox) Send ¶
Send hands the pending rows to dest and answers with how many rows it transmitted. A dest beginning https:// or http:// receives one POST per batch of at most 200 rows, content-type application/x-ndjson, the body one row per line in pending order, each line the same JSON object the file holds; a 2xx answer means that batch arrived. Any other dest is a filesystem path, plain or written file://, that the rows are appended to, created with mode 0600 and a parent directory of mode 0700 if either is missing. An empty dest, or a Send with nothing pending, is a no-op: 0 and a nil error, the rows staying pending.
The whole call lives inside Budget, which is 2s when it is zero, and inside ctx as well, whichever ends first. Rows whose batch arrived are marked sent and leave the outbox; a row Keep declines is marked sent without being transmitted and is not counted. A batch the destination refused by line — a 400 naming `line N` — is refused on one row it has said it will never accept: the named row is marked dropped, the rows around it are sent within the same call, and Send counts the sent ones. Rows whose batch did not arrive, and every row after it, stay pending, and Send returns an error that says what failed.
Source Files