Documentation
¶
Overview ¶
Package runnerspool stores proxy decisions until the authenticated runner stream acknowledges their durable org ingestion. The proxy and daemon use the same private directory, serialized by a process-shared file lock.
Index ¶
- Constants
- Variables
- type Disposition
- type Drained
- type Grant
- type Liability
- type Loss
- type LossBasis
- type PendingLoss
- type Produced
- type Producer
- type Record
- type Session
- type Spool
- func (s *Spool) Accept(ctx context.Context, instance string) (out Session, err error)
- func (s *Spool) AckOverflow(ctx context.Context, row Record) error
- func (s *Spool) After(ctx context.Context, seq int64, limit int) (rows []Record, err error)
- func (s *Spool) Append(ctx context.Context, decision egress.DecisionLog) error
- func (s *Spool) AppendAdmitted(ctx context.Context, decision egress.DecisionLog) error
- func (s *Spool) CanRecord(ctx context.Context) error
- func (s *Spool) Close() error
- func (s *Spool) EmitLoss(ctx context.Context) (emitted bool, err error)
- func (s *Spool) Get(ctx context.Context) (pos ackcursor.Pos, err error)
- func (s *Spool) HoldWriter() (*os.File, error)
- func (s *Spool) Liability(ctx context.Context) (out Liability, err error)
- func (s *Spool) Overflow(ctx context.Context) (row *Record, err error)
- func (s *Spool) Produce(ctx context.Context, p Produced) (disp Disposition, fresh bool, err error)
- func (s *Spool) ProducerRefused(ctx context.Context, gen uint64, instance string, total uint64) error
- func (s *Spool) Receipt(ctx context.Context) (out types.RunnerSpoolReceipt, err error)
- func (s *Spool) Refuse(ctx context.Context) error
- func (s *Spool) Regrant(ctx context.Context, gen uint64, instance string) (g Grant, changed bool, err error)
- func (s *Spool) Replay(ctx context.Context, send func(context.Context, []Record) error) error
- func (s *Spool) Seal(ctx context.Context, expected types.RunnerSpoolReceipt) (sealed bool, err error)
- func (s *Spool) Set(ctx context.Context, pos ackcursor.Pos) error
- func (s *Spool) SettleLoss(ctx context.Context, drained *Drained) (loss Loss, err error)
- func (s *Spool) Stats(ctx context.Context) (depth int, bytes int64, drops uint64, err error)
- func (s *Spool) Succeed(ctx context.Context) error
Constants ¶
const ( // ReplayBatch bounds the uncommitted local ACK window retained by org receipts. ReplayBatch = 64 MaxRecords = 50000 MaxBytes = 32 << 20 // MaxRecordBytes leaves the record envelope below the wire control bound. MaxRecordBytes = (1 << 20) - 4096 )
const ( GrantRecords = 5000 GrantReservations = 16 GrantBytes = GrantReservations * MaxRecordBytes LossReserveBytes = 4 << 10 )
Remote-sink grant ceilings (design §3.2 item 2). The last record slot and the last LossReserveBytes stay free for the builtin:spool-lost row. The byte credit covers GrantReservations requests reserved at the largest record (a scanned body-bearing request reserves MaxRecordBytes), as the local spool admits them, within the spool's own MaxBytes.
Variables ¶
var ( // ErrProducer refuses a frame from a foreign, settled or out-of-order producer. ErrProducer = errors.New("spool refuses this producer sequence") // ErrStaleGen refuses a frame from a superseded sink session. ErrStaleGen = errors.New("spool sink session superseded") )
var ErrFull = errors.New("runner decision spool is full")
var ErrSealed = errors.New("runner spool is durably sealed")
ErrSealed refuses new evidence production after exact settled retirement.
var ErrWriterHeld = errors.New("spool writer lock is held")
ErrWriterHeld means another handle owns the spool's writer lock.
Functions ¶
This section is empty.
Types ¶
type Disposition ¶ added in v0.9.3
type Disposition string
Disposition is the one durable outcome of a producer sequence.
const ( Appended Disposition = "appended" Refused Disposition = "refused" Lost Disposition = "lost" )
type Drained ¶ added in v0.9.3
Drained is a proxy's drained reply: its last sequence and the credit units it consumed (sequences plus reservations still held).
type Grant ¶ added in v0.9.3
type Grant struct {
Gen uint64 `json:"gen"`
Base int64 `json:"base"`
Through int64 `json:"through"`
Bytes int64 `json:"bytes"`
}
Grant is write-ahead credit: sequences Base+1..Through carrying at most Bytes of record bodies. Its liability is Through - Producer.Seq.
type Liability ¶ added in v0.9.3
type Liability struct {
Gen uint64
Producer *Producer
Grant *Grant
PendingLoss *PendingLoss
Sealed bool
}
Liability is the remote producer's durable state, and whether the spool is sealed.
type LossBasis ¶ added in v0.9.3
type LossBasis string
LossBasis says whether a pending loss count is exact or an upper bound.
type PendingLoss ¶ added in v0.9.3
PendingLoss is settled loss not yet emitted as a builtin:spool-lost row.
type Produced ¶ added in v0.9.3
type Produced struct {
Gen uint64
Instance string
Seq int64
Hash string
Admitted bool
Decision egress.DecisionLog
}
Produced is one record frame from the remote proxy.
type Producer ¶ added in v0.9.3
type Producer struct {
Instance string `json:"instance"`
Seq int64 `json:"seq"`
Hash string `json:"hash,omitempty"`
Disposition Disposition `json:"disposition,omitempty"`
Refused uint64 `json:"refused,omitempty"`
}
Producer is the remote proxy's durable position: the last sequence with a disposition, its body hash, and the refusal total already counted in Drops.
type Record ¶
type Record struct {
SpoolID uuid.UUID `json:"spool_id"`
Seq int64 `json:"seq"`
Hash string `json:"hash"`
Decision egress.DecisionLog `json:"decision"`
Count uint64 `json:"count,omitempty"`
DropEnd uint64 `json:"drop_end,omitempty"`
}
Record is one stable, replayable decision. Seq zero is an overflow summary, whose durable identity is its hash; ordinary records have positive sequences.
func (Record) Marshal ¶
Marshal is the bounded EVENT.spool payload; no run token authenticates replay.
type Spool ¶
type Spool struct {
// contains filtered or unexported fields
}
Spool is safe across goroutines and independently opened proxy/daemon handles.
func Open ¶
Open refuses unsafe or corrupt existing state. A missing directory starts a new private stream; an existing stream never silently resets its cursor.
func (*Spool) Accept ¶ added in v0.9.3
Accept starts a session for instance at a new generation and, in the same write, issues its grant or re-issues the outstanding one no lower. A settled producer is never reopened; only Succeed opens the spool to a new instance.
func (*Spool) AckOverflow ¶
AckOverflow records only the frozen interval, retaining later refusals.
func (*Spool) After ¶
After implements ackcursor.Source. The acknowledged row's identity lives in the manifest, so reclaimed payloads do not make ackcursor infer a reset.
func (*Spool) Append ¶
Append commits before returning. An unreserved producer must refuse on any error; the proxy separately reserves capacity before forwarding requests.
func (*Spool) AppendAdmitted ¶
AppendAdmitted finishes a request whose proxy already reserved capacity. Overflow refuses new requests, never discards this previously admitted tail.
func (*Spool) CanRecord ¶
CanRecord checks durable capacity. Concurrent producers must also reserve capacity; checking alone cannot guarantee space for a later append. Refusals not yet reported do not refuse a spool with room (see append).
func (*Spool) EmitLoss ¶ added in v0.9.3
EmitLoss turns pending_loss into one ordinary builtin:spool-lost row in the loss reserve, clearing pending_loss in the same write. Unreported Drops never refuse it; while an earlier unacknowledged row still fills the reserve it waits (emitted=false) for the next call.
func (*Spool) HoldWriter ¶ added in v0.9.3
HoldWriter takes the writer lock for a remote producer's lifetime, as a live local proxy holds it: Seal defers until the returned file is closed.
func (*Spool) Overflow ¶
Overflow freezes one cumulative refused-attempt interval before transmission. A crash or new refusals cannot change the identity/count of an in-flight row.
func (*Spool) Produce ¶ added in v0.9.3
Produce gives a producer sequence its one durable disposition, in the same manifest write as its effect: appended (Last/Bytes), refused (Drops, an unreserved record only) or lost (pending_loss, an admitted record that still did not fit). A resend of the last sequence returns the stored disposition with fresh=false; an older one returns "" (durable, nothing to redo).
func (*Spool) ProducerRefused ¶ added in v0.9.3
func (s *Spool) ProducerRefused(ctx context.Context, gen uint64, instance string, total uint64) error
ProducerRefused counts the instance's cumulative refused admissions into Drops once: a resent or older total changes nothing.
func (*Spool) Regrant ¶ added in v0.9.3
func (s *Spool) Regrant(ctx context.Context, gen uint64, instance string) (g Grant, changed bool, err error)
Regrant persists fresh credit for the current session before it is sent, returning changed=false when the outstanding grant already equals it.
func (*Spool) Replay ¶
Replay uses the shared acknowledgement cursor. send gets each batch in seq order and must return nil only after durable checked org ingestion of every row in it (H2 Peer.EventsAck), not after a write.
Replay ends once it has delivered the head (rows and refused attempts) seen when it started. Rows a live proxy appends meanwhile wait for the next call, so a producer faster than the drain cannot keep Replay running.
func (*Spool) Seal ¶
func (s *Spool) Seal(ctx context.Context, expected types.RunnerSpoolReceipt) (sealed bool, err error)
Seal requires the proxy's lifetime writer lock and an exact fully accepted current head. A still-running producer or unsettled evidence defers sealing.
func (*Spool) Set ¶
Set commits an acknowledged cursor before reclaiming any payload. A crash during reclamation leaves harmless acknowledged files, never missing history.
func (*Spool) SettleLoss ¶ added in v0.9.3
SettleLoss writes off the remote producer's liability once it is proven gone (completion evidence; never a NotFound alone): exact when drained answered at the current generation with every sequence committed, else the upper bound Grant.Through - Producer.Seq. One write adds it to pending_loss and clears the grant; a settled producer is never accepted again.
func (*Spool) Succeed ¶ added in v0.9.3
Succeed opens the spool to a new producer once the last one is settled: a Docker runner's revive starts a new proxy process, which is a new instance. The settled producer's pending loss stays until EmitLoss. A producer still holding a grant is refused, so the new one never overlaps it. The caller hands the run to the new proxy under a new key, so the settled instance cannot come back.
Source Files