Documentation
¶
Overview ¶
Package gc implements the mark-and-sweep collector of architecture/mark-sweep-gc.md, a port of Mic92's bitmap GC (draganm/amber-store#9): a cycle marks every key reachable from the references' roots into a packstore.MarkSet — one bit per sealed record, slotted by the packs' own footer indexes — and sweeps by rewriting the packs whose dead ratio crosses the line (packstore.Compact). Nothing is persisted between cycles: no closure files, no union, no refcounts. A write barrier (packstore.BeginBarrier) keeps ingests running while the mark walks; reference publication and the sweep serialize on the collector's reference lock. Wire protocol and pack formats are unchanged.
Index ¶
- Constants
- Variables
- type Collector
- func (c *Collector) BeginWrite() (done func())
- func (c *Collector) Close() error
- func (c *Collector) PrepareRef(root key.Key) (commit, abort func(), err error)
- func (c *Collector) ReleaseRef(root key.Key) error
- func (c *Collector) Run(ctx context.Context, garbage float64) (CycleStats, error)
- func (c *Collector) Status(ctx context.Context) (Status, error)
- func (c *Collector) Why(k key.Key) ([]string, error)
- func (c *Collector) Wipe(reset func() error) error
- type CycleStats
- type Options
- type PackStatus
- type Status
Constants ¶
const ( // DefaultGrace is the minimum age of a sealed pack before it can be // reaped. DefaultGrace = time.Hour // DefaultGarbage is the selection line: an eligible pack with at // least this fraction of garbage is reaped. DefaultGarbage = 0.5 )
Variables ¶
var ErrCycleRunning = errors.New("gc: a cycle is already running")
ErrCycleRunning reports an overlapping Run; cycles never overlap.
Functions ¶
This section is empty.
Types ¶
type Collector ¶
type Collector struct {
// contains filtered or unexported fields
}
Collector implements the cycle and the reference hooks over an open packstore and refstore pair. Single-owner, like the stores it sits next to; close it before them.
func Open ¶
func Open(dir string, objects *packstore.Store, refs *refstore.Store, opts Options) (*Collector, error)
Open opens the collector next to an already-open packstore and refstore. dir is the layout slot the simple-gc collector kept closure files in (<store>/closures): it is created empty and any leftover closure state from a previous collector is swept — closures were derived data. With opts.Interval > 0 a goroutine runs a cycle per interval until Close.
func (*Collector) BeginWrite ¶ added in v0.0.6
func (c *Collector) BeginWrite() (done func())
BeginWrite gates one object-write span (an ingest, a pull, an inbox drain) against the sweep. The span holds the reference lock shared: the sweep waits out in-flight writes, and a write stalls while a sweep runs — never during the mark, which writers pass behind the write barrier. Without the gate a dedup hit against a record in a condemned pack could report success and then lose the record to the pack's removal (packstore.Compact must not overlap ingests). Returns the release, idempotent; call it when the span's writes are durable.
func (*Collector) Close ¶
Close stops the background loop and waits out a running cycle. Close the collector before the stores under it.
func (*Collector) PrepareRef ¶
PrepareRef readies a reference PUT naming root under the reference lock, held shared until commit or abort: the tree is walked for completeness (a missing object aborts with the error naming it — the caller's 404) and the walked closure is handed to the write barrier, so a PUT landing while a mark runs cannot lose its objects to the sweep. Exactly one of commit (after the reference record is stored) or abort (it was not) must be called. Release of an old root needs no bookkeeping; ReleaseRef exists for symmetry.
func (*Collector) ReleaseRef ¶
ReleaseRef records that one reference naming root was deleted or overwritten. The mark-and-sweep collector keeps no per-root state, so this is a no-op: the next cycle simply no longer marks from the root.
func (*Collector) Run ¶
Run marks from every reference root and sweeps the eligible packs above the line: garbage >= 0 forces that line, garbage < 0 uses policy — 0.5, or 0.1 under min-free pressure. Ingests and reads run through the mark (the write barrier keeps them); reference publication and the sweep stall each other on the reference lock.
func (*Collector) Status ¶
Status marks from the current references and scores every sealed pack — a full mark walk, the cost of keeping no persistent liveness state.
type CycleStats ¶
type CycleStats struct {
Start time.Time
Duration time.Duration
MarkDuration time.Duration // roots snapshot + mark walk
SweepDuration time.Duration // Compact: select, copy, delete
Threshold float64
Marked int // distinct live objects marked
Scored int // sealed packs considered
Reaped []uint64 // victim ids, ascending
CopiedRecords int
CopiedBytes int64
FreedBytes int64 // victim file bytes freed on disk, footers included
}
CycleStats describes one cycle.
type Options ¶
type Options struct {
Grace time.Duration // pack eligibility age
Garbage float64 // reap packs with at least this fraction of garbage
MinFree uint64 // free-space floor in bytes; 0 = 5 % of the filesystem
Rate int64 // copier bandwidth cap in bytes/s; 0 = unlimited
Interval time.Duration // time between background cycles; 0 = none
Jobs int // PrepareRef walk parallelism; <= 0 = GOMAXPROCS
}
Options configure a Collector. The zero value means: 1 h grace, 0.5 garbage line, min-free at 5 % of the filesystem, unlimited copy rate, no background cycles, GOMAXPROCS walk parallelism.
type PackStatus ¶
type PackStatus struct {
ID uint64
Sealed time.Time
Body int64 // record bytes
Keys uint64
Live int64 // Σ (46 + slen) over marked entries
Garbage float64
Eligible bool // sealed before the horizon (now − grace)
}
PackStatus is one sealed pack's score against a mark.
type Status ¶
type Status struct {
Packs []PackStatus
LiveBytes int64
GarbageBytes int64
Refs int // reference names
Marked int // distinct live objects marked
Last *CycleStats
LastError string
}
Status is the gc report: per-pack scores against a fresh advisory mark, totals, the last cycle. The mark runs without quiescing writers, so concurrent churn can skew the numbers; a cycle's own mark is exact.
Source Files