Documentation
¶
Overview ¶
Package generation publishes immutable graph snapshots (typically csr.CSR views) under a refcount-protected pointer so readers can observe a consistent generation while a new one is being prepared in the background.
The pattern is the read-mostly equivalent of an MVCC snapshot: every reader Acquires the current generation, uses it, and Releases it; a publisher prepares the next generation in a fresh allocation and atomically swaps the pointer. Old generations are reclaimed only after every outstanding reader has Released them.
How a generation relates to an MVCC snapshot (rmp #2311) ¶
They answer different questions and neither replaces the other.
An [lpg.Snapshot] is an INSTANT: a commit timestamp a reader resolves each object against, so the reader observes exactly the transactions committed at or before it. Visibility is decided per record, at read time, by [mvcc.Visible]. That is the module's isolation mechanism and its only one.
A generation is a PUBLISHED ARTEFACT: one immutable derived structure — typically a CSR built for whole-graph analytics — held alive by refcount while readers use it. It decides nothing about visibility; it is a cached result whose contents were fixed when it was built.
The two are related by exactly one rule: A GENERATION IS ONLY MEANINGFUL AS OF THE INSTANT IT WAS BUILT AT. Build it with csr.BuildFromAdjListAsOf from a snapshot, and it describes that snapshot's graph for as long as it is held. Build it from the present while writers commit and it describes no instant at all — which is the defect class rmp #2293 recorded for the CSR pair cache, and rmp #2310 for the checkpoint capture.
This package is NOT a second snapshot mechanism in the engine ¶
Nothing in the module uses it: the engine's analytics reads go through the versioned accessors like every other read, and the only caller is examples/33_generation_swap, which demonstrates the publish-and-refcount pattern itself. It is a utility a consumer may use to cache a derived structure, and the paragraph above is the contract for doing so safely. It is not a competing notion of "a consistent view", because it never decides what is visible.
Index ¶
- Variables
- type Generation
- type Publisher
- func (p *Publisher[W]) Acquire() *Generation[W]
- func (p *Publisher[W]) Close()
- func (p *Publisher[W]) Current() *Generation[W]
- func (p *Publisher[W]) Publish(c *csr.CSR[W]) (*Generation[W], error)
- func (p *Publisher[W]) PublishWithDrain(c *csr.CSR[W], timeout time.Duration) (*Generation[W], error)
- func (p *Publisher[W]) Release(g *Generation[W])
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrClosed = errors.New("generation: publisher closed")
ErrClosed is returned by Publish when the Publisher has been closed.
var ErrDrainTimeout = errors.New("generation: drain timeout")
ErrDrainTimeout is returned by Publisher.PublishWithDrain when the old generation still has outstanding readers after the configured timeout.
Functions ¶
This section is empty.
Types ¶
type Generation ¶
type Generation[W any] struct { // contains filtered or unexported fields }
Generation wraps an immutable csr.CSR snapshot with a refcount. Generation is safe for concurrent use; Acquire/Release on the same generation can run from any number of goroutines.
func (*Generation[W]) CSR ¶
func (g *Generation[W]) CSR() *csr.CSR[W]
CSR returns the underlying snapshot. The pointer is valid only while at least one Acquire is outstanding on g.
func (*Generation[W]) Refcount ¶
func (g *Generation[W]) Refcount() int64
Refcount returns the current refcount. Intended for observability; callers must not infer correctness from it because the value can race with concurrent Acquire / Release.
type Publisher ¶
type Publisher[W any] struct { // contains filtered or unexported fields }
Publisher owns the current generation and serialises publication of new ones. It is safe for concurrent reads (Acquire/Release) from any number of goroutines, and for any number of concurrent publishers (Publish/PublishWithDrain): publishMu serialises the load-prev/store-next swap so every superseded generation is observed as exactly one publisher's predecessor and is therefore drained on. The read path (Acquire/Release/Current/CSR) never takes publishMu and stays fully lock-free.
Example ¶
ExamplePublisher shows the read side of the MVCC-style snapshot pattern: a reader Acquires the current generation, uses its immutable CSR, and Releases it. The refcount tracks outstanding readers so an old generation is reclaimed only once every reader has let go.
package main
import (
"fmt"
"github.com/FlavioCFOliveira/GoGraph/graph/adjlist"
"github.com/FlavioCFOliveira/GoGraph/graph/csr"
"github.com/FlavioCFOliveira/GoGraph/graph/generation"
)
// snapshot is a helper that freezes a directed graph with the given
// edges into a CSR view, the immutable unit a Publisher hands out.
func snapshot(edges [][2]string) *csr.CSR[int] {
g := adjlist.New[string, int](adjlist.Config{Directed: true})
for _, e := range edges {
_ = g.AddEdge(e[0], e[1], 1)
}
return csr.BuildFromAdjList(g)
}
func main() {
pub := generation.New(snapshot([][2]string{{"a", "b"}}))
defer pub.Close()
gen := pub.Acquire()
fmt.Println("edges:", gen.CSR().Size())
fmt.Println("readers while held:", gen.Refcount())
pub.Release(gen)
fmt.Println("readers after release:", pub.Current().Refcount())
}
Output: edges: 1 readers while held: 1 readers after release: 0
func (*Publisher[W]) Acquire ¶
func (p *Publisher[W]) Acquire() *Generation[W]
Acquire returns the current generation with its refcount incremented. The caller must eventually call Release on the returned generation to allow stale generations to be reclaimed.
func (*Publisher[W]) Close ¶
func (p *Publisher[W]) Close()
Close marks the Publisher as closed, waits for all outstanding acquisitions to drain (with a 30 s safety deadline), and returns. After Close returns, Acquire returns nil and Publish returns ErrClosed.
Close is safe to call from any goroutine. Calling Close more than once is safe; subsequent calls are no-ops.
func (*Publisher[W]) Current ¶
func (p *Publisher[W]) Current() *Generation[W]
Current returns the current generation without incrementing its refcount. Intended for introspection only; never use the returned pointer to access the CSR.
func (*Publisher[W]) Publish ¶
func (p *Publisher[W]) Publish(c *csr.CSR[W]) (*Generation[W], error)
Publish atomically swaps in a fresh generation built from c and returns the new generation. Returns (nil, ErrClosed) when the Publisher has been closed. The previous generation is not reclaimed until its refcount drains to zero (which happens naturally as readers Release).
Publish is safe for concurrent use with other Publish, PublishWithDrain, and Close calls: the load-prev/store-next swap is serialised so generations are never lost.
Example ¶
ExamplePublisher_Publish shows the write side: Publish installs a new immutable generation and makes it the one future Acquire calls see, without disturbing readers still holding the previous generation.
package main
import (
"fmt"
"github.com/FlavioCFOliveira/GoGraph/graph/adjlist"
"github.com/FlavioCFOliveira/GoGraph/graph/csr"
"github.com/FlavioCFOliveira/GoGraph/graph/generation"
)
// snapshot is a helper that freezes a directed graph with the given
// edges into a CSR view, the immutable unit a Publisher hands out.
func snapshot(edges [][2]string) *csr.CSR[int] {
g := adjlist.New[string, int](adjlist.Config{Directed: true})
for _, e := range edges {
_ = g.AddEdge(e[0], e[1], 1)
}
return csr.BuildFromAdjList(g)
}
func main() {
pub := generation.New(snapshot([][2]string{{"a", "b"}}))
defer pub.Close()
fmt.Println("before:", pub.Current().CSR().Size())
next, err := pub.Publish(snapshot([][2]string{{"a", "b"}, {"b", "c"}}))
if err != nil {
panic(err) // a healthy (non-closed) publisher never errors here
}
fmt.Println("published:", next.CSR().Size())
fmt.Println("current:", pub.Current().CSR().Size())
}
Output: before: 1 published: 2 current: 2
func (*Publisher[W]) PublishWithDrain ¶
func (p *Publisher[W]) PublishWithDrain(c *csr.CSR[W], timeout time.Duration) (*Generation[W], error)
PublishWithDrain swaps in a fresh generation and blocks until the previous generation's refcount reaches zero, or returns ErrDrainTimeout when the timeout elapses first. Returns (nil, ErrClosed) when the Publisher has been closed. Callers that need to recycle the previous generation's backing storage (e.g. to unmap a Tier 2 file) should prefer this variant.
A timeout of zero disables the deadline; PublishWithDrain then blocks indefinitely.
PublishWithDrain is safe for concurrent use with other publishers and with Close: each concurrent publisher captures a distinct predecessor under publishMu, so every superseded generation is drained on by exactly one caller and none is silently leaked.
func (*Publisher[W]) Release ¶
func (p *Publisher[W]) Release(g *Generation[W])
Release decrements g's refcount and signals any goroutine waiting inside Publisher.PublishWithDrain for the refcount to hit zero.
Source Files