snapshot

func ApplyCSRToGraph ¶

func ApplyCSRToGraph[N comparable, W any](g *lpg.Graph[N, W], rb *CSRReadback) error

ApplyCSRToGraph replays the adjacency in rb into g. The pre- condition is that g's underlying mapper has already been populated with every NodeID referenced by rb — typically by an immediately- preceding ApplyMapperToGraph (v3 snapshots) or by a WAL replay (v2 snapshots that pair with a WAL prefix). Records whose endpoints the mapper cannot resolve are skipped and counted via `store.snapshot.ApplyCSR.unresolved`; the function does not return an error for them so a partial mapper degrades cleanly rather than aborting recovery mid-way.

Weight decoding is supported for the common int/float weight types (int8/uint8/bool, int16/uint16, int32/uint32/float32, int/uint/ int64/uint64/float64/uintptr). Other W types apply zero weights; the metric `store.snapshot.ApplyCSR.weightFallback` reports the fallback count for observability.

ApplyCSRToGraph is idempotent against a freshly-loaded mapper but not against a graph that already contains edges: re-applying a CSR to a graph with existing edges may duplicate them in multigraph mode or no-op in simple-graph mode. Callers should run this exactly once per recovery, immediately after the mapper restore and before any WAL replay.

rb is passed by pointer to avoid copying the three slices in the readback (vertices, edges, weight bytes) on every call. The function does not mutate rb.

func ApplyEdgeHandlesToGraph ¶

func ApplyEdgeHandlesToGraph[N comparable, W any](g *lpg.Graph[N, W], rb EdgeHandlesReadback)

ApplyEdgeHandlesToGraph replays rb into a live g, re-attaching every per-handle edge label and property keyed by its stable handle and the endpoint NodeID pair. It MUST run AFTER the mapper and CSR (with its handle column) are applied so the handle the record references is already live on the adjacency slot — though the per-handle metadata stores are keyed by (NodeID pair, handle) directly and do not require the adjacency edge to be present, so a record whose edge the CSR did not materialise is still re-attached harmlessly. The handle high-water counter is re-seeded for every record so a post-recovery edge creation never re-mints a live handle (invariant I5).

func ApplyLabelsToGraph ¶

func ApplyLabelsToGraph[N comparable, W any](g *lpg.Graph[N, W], rb LabelsReadback) error

ApplyLabelsToGraph replays rb into a live g. The pre-condition is that g's underlying mapper has already been populated with every NodeID referenced by rb — typically by replaying the WAL prefix covered by the snapshot, or by re-issuing the original AddNode / AddEdge calls. Records whose NodeID cannot be resolved by the mapper are skipped and counted via the `store.snapshot.ApplyLabels.unresolved` metric counter; the function does not return an error for them so a partial mapper degrades cleanly rather than aborting recovery mid-way.

Edge label records whose endpoints are resolvable but whose edge is absent from the adjacency list (e.g., the CSR was not yet applied) are likewise skipped and counted under `store.snapshot.ApplyLabels.edgeMissing`; this matches lpg.Graph.SetEdgeLabel's own no-op-on-missing-edge contract.

func ApplyMapperToGraph ¶

func ApplyMapperToGraph[N comparable, W any](g *lpg.Graph[N, W], rb MapperReadback) error

ApplyMapperToGraph rebuilds g's underlying graph.Mapper from the snapshot readback. It is only meaningful for string-keyed graphs: any other N type returns nil without touching g, because no v3 mapper.bin is ever produced for non-string graphs. The caller is expected to invoke this function before ApplyCSRToGraph, ApplyLabelsToGraph, or ApplyPropertiesToGraph so subsequent resolution calls see the restored interning table.

Pre-condition: g must hold a fresh (empty) mapper. Calling on a graph that already has interned values returns ErrMapperApply wrapping graph.ErrMapperNotEmpty so the caller can distinguish a programmer error from a corruption error.

Concurrency: ApplyMapperToGraph is not safe to call concurrently with mutations or reads on g. It is intended for the one-shot snapshot-load phase of recovery.

func ApplyMapperToGraphWithCodec ¶

func ApplyMapperToGraphWithCodec[N comparable, W any](g *lpg.Graph[N, W], rb MapperReadback, codec keyDecoder[N]) error

ApplyMapperToGraphWithCodec rebuilds g's underlying graph.Mapper from a version-2 (codec) snapshot readback for ANY comparable key type N. Each MapperRawPair carries the codec-encoded key bytes the snapshot writer produced via WriteMapper; this function decodes them back into N via the supplied codec (the same one the store uses on the WAL) and seeds the interning table through graph.Mapper.LoadFrom.

It is the codec-aware dual of ApplyMapperToGraph: recovery calls this when the loaded readback carries RawPairs (non-string keys) and the string-specialised path when it carries Pairs. An empty readback is a no-op.

Pre-condition and concurrency contract match ApplyMapperToGraph: g must hold a fresh (empty) mapper, and the call must not race with any other access to g. A decode failure surfaces as ErrMapperApply wrapping the codec error; a structural violation surfaces as ErrMapperApply wrapping the relevant [graph.ErrMapper…] sentinel.

func ApplyPropertiesToGraph ¶

func ApplyPropertiesToGraph[N comparable, W any](g *lpg.Graph[N, W], rb PropertiesReadback) error

ApplyPropertiesToGraph replays rb into a live g. The pre-condition is that g's underlying mapper has already been populated with every NodeID referenced by rb — typically by replaying the WAL prefix covered by the snapshot, or by re-issuing the original AddNode / AddEdge calls. Records whose NodeID cannot be resolved by the mapper are skipped and counted via the `store.snapshot.ApplyProperties.unresolved` metric counter; the function does not return an error for them so a partial mapper degrades cleanly rather than aborting recovery mid-way.

Edge property records whose endpoints are resolvable but whose edge is absent from the adjacency list (e.g., the CSR was not yet applied) are likewise skipped and counted under `store.snapshot.ApplyProperties.edgeMissing`; this matches lpg.Graph.SetEdgeProperty's own no-op-on-missing-edge contract.

func ApplyTombstonesToGraph ¶

func ApplyTombstonesToGraph[N comparable, W any](g *lpg.Graph[N, W], rb TombstonesReadback)

ApplyTombstonesToGraph replays rb into a live g, re-tombstoning every NodeID the snapshot recorded as removed. It must run AFTER the snapshot nodes are loaded (mapper + CSR) so the ids it restores reference the same stable slots; it re-tombstones by id directly via lpg.Graph.RestoreTombstones and so does not require the natural keys to be resolvable.

A later WAL re-create (OpAddNode) for any of these ids still revives it, preserving the chronology of a delete→recreate cycle that straddles the snapshot boundary.

func WriteCSR ¶

func WriteCSR[W any](w io.Writer, c *csr.CSR[W]) (size int64, crc uint32, err error)

WriteCSR serialises c to w, returning the number of bytes written and the CRC32C of the serialised payload. The on-disk layout is:

uint64 nVertices      (little-endian)
uint64 nEdges
uint8  hasWeights     (1 = weights array present)
uint8  weightSizeBytes (0 when hasWeights = 0)
[vertices]            (nVertices * 8 bytes)
[edges]               (nEdges * 8 bytes)
[weights]             (nEdges * weightSizeBytes bytes, when present)
uint8  hasHandles      (OPTIONAL trailing block; 1 = handle array present)
[handles]             (nEdges * 8 bytes, when hasHandles = 1)

The trailing handles block (Stage 2 of the stable-edge-handle work) is emitted ONLY when the source CSR carries a per-slot handle column (csr.CSR.HandlesSlice != nil). A graph that never used AddEdgeH produces no trailing block, so its csr.bin is byte-identical to one written before this column existed — the v1 golden and the cross-process byte-equality fixtures are unaffected. [readCSRLimited] detects the block by attempting to read one more byte after the weights array: present → handles follow, EOF → none (the backward-compatible read branch).

func WriteEdgeHandles ¶

func WriteEdgeHandles[N comparable, W any](w io.Writer, g *lpg.Graph[N, W]) (size int64, crc uint32, emitted bool, err error)

WriteEdgeHandles serialises every per-handle edge label and property attached to g into w in the edgehandles.bin format. It returns the number of bytes written and the CRC32C of the serialised payload — both stored in the manifest's FileEntry so LoadSnapshotFull can verify integrity at load time. It returns (0, 0, nil, false) when the graph carries no per-handle metadata, signalling the caller to omit the component entirely.

Records are emitted in the deterministic order lpg.Graph.WalkEdgeHandles yields (the same source-node order csr.bin / labels.bin use), and within a record the label names and property keys are sorted, so the component is byte-stable across writes of the same logical state — the cross-process byte-equality contract the snapshot relies on.

func WriteLabels ¶

func WriteLabels[N comparable, W any](w io.Writer, g *lpg.Graph[N, W]) (size int64, crc uint32, err error)

WriteLabels serialises every node and edge label attached to g into w in the labels.bin format documented at the top of this file. It returns the number of bytes written and the CRC32C of the serialised payload — both stored in the manifest's FileEntry for the labels.bin component so Open / LoadSnapshotFull can verify integrity at load time.

The CRC32C covers the entire on-disk file, including the magic header. This lets the manifest's CRC field validate every byte of labels.bin end-to-end without a separate inner-payload checksum.

The on-disk string table is populated by walking g's lpg.LabelRegistry in interning order; the labelStringIdx written for each (node | edge) record indexes into that table. Because LabelID is itself assigned in interning order, this preserves the registry's identity across save and load: the reader interns each name back in the same order and observes the same LabelID values without an extra remap step.

The walk holds the registry's RLock for the duration of the string table emission; node/edge enumeration uses the same lock-free / RLock-only primitives the public LPG accessors expose.

func WriteManifest ¶

func WriteManifest(w io.Writer, m Manifest) error

WriteManifest writes m to w in canonical (pretty-printed) JSON.

func WriteMapper ¶

func WriteMapper[N comparable](w io.Writer, m *graph.Mapper[N], codec keyEncoder[N]) (size int64, crc uint32, err error)

WriteMapper serialises every (NodeID -> key) pair held by m into w, generalising WriteMapperString to any comparable key type N via the supplied codec. It returns the number of bytes written and the CRC32C of the serialised payload, both recorded in the manifest's FileEntry for the mapper.bin component so LoadSnapshotFull can verify integrity at load time.

Back-compatibility: when N is the canonical string type the function delegates to WriteMapperString, which emits the frozen version-1 layout (raw UTF-8 key bytes, no codec framing). The on-disk image is therefore byte-identical to every string mapper.bin produced before the codec generalisation, regardless of which string codec is supplied. For any other N the function emits a version-2 layout whose per-record key bytes are the output of codec.Encode.

On-disk layout (version 2, all little-endian):

uint32  magic           ('GMAP', 0x50414D47)
uint16  formatVersion   (2)
uint64  pairCount
for each pair:
    uint64  nodeID
    uint32  keyLen       (length of the codec-encoded key bytes)
    [keyLen]byte key     (codec.Encode output for the natural key)

Pairs are emitted in graph.Mapper.Walk order (shard-major, intra-index-major) so the read side reconstructs the mapper deterministically. The CRC32C covers the entire on-disk file, including the magic header.

func WriteMapperString ¶

func WriteMapperString(w io.Writer, m *graph.Mapper[string]) (size int64, crc uint32, err error)

WriteMapperString serialises every (NodeID -> string key) pair held by m into w in the mapper.bin format documented below. It returns the number of bytes written and the CRC32C of the serialised payload — both stored in the manifest's FileEntry for the mapper.bin component so LoadSnapshotFull can verify integrity at load time.

On-disk layout (all little-endian):

uint32  magic           ('GMAP', 0x50414D47)
uint16  formatVersion   (1)
uint64  pairCount
for each pair:
    uint64  nodeID
    uint32  keyLen
    [keyLen]byte key

Pairs are emitted in graph.Mapper.Walk order (shard-major, intra-index-major) so the read side can reconstruct the mapper deterministically.

The CRC32C covers the entire on-disk file, including the magic header. The reader recomputes the CRC end-to-end at load time.

func WriteProperties ¶

func WriteProperties[N comparable, W any](w io.Writer, g *lpg.Graph[N, W]) (size int64, crc uint32, err error)

WriteProperties serialises every node and edge property attached to g into w in the properties.bin format documented at the top of this file. It returns the number of bytes written and the CRC32C of the serialised payload — both stored in the manifest's FileEntry for the properties.bin component so LoadSnapshotFull can verify integrity at load time.

The CRC32C covers the entire on-disk file, including the magic header. This lets the manifest's CRC field validate every byte of properties.bin end-to-end without a separate inner-payload checksum.

The on-disk key string table is populated by walking g's lpg.PropertyKeyRegistry in interning order; the keyIdx written for each (node | edge) record indexes into that table.

Concurrency contract: the walk relies on the same lock-free / RLock-only primitives the public LPG accessors expose. Properties added by a concurrent mutator race with the snapshot writer in the same way labels do — the writer either observes the new property (and the matching node/edge entry) or it does not, but never an inconsistent fragment.

func WriteSnapshotCSR ¶

func WriteSnapshotCSR[W any](dir string, c *csr.CSR[W]) error

WriteSnapshotCSR is the legacy high-level helper that lays a snapshot directory containing a v1 manifest plus the CSR. It is retained for backward compatibility: callers that also need LPG label durability must use WriteSnapshotFull which writes a v2 manifest with both csr.bin and labels.bin. Atomic publication is achieved by assembling the snapshot under dir + ".tmp" and renaming it to dir on success.

package main

import (
	"fmt"
	"os"
	"path/filepath"

	"github.com/FlavioCFOliveira/GoGraph/graph/adjlist"
	"github.com/FlavioCFOliveira/GoGraph/graph/csr"
	"github.com/FlavioCFOliveira/GoGraph/store/snapshot"
)

func main() {
	dir, err := os.MkdirTemp("", "snapshot-csr-example")
	if err != nil {
		panic(err)
	}
	defer func() { _ = os.RemoveAll(dir) }()

	a := adjlist.New[string, int64](adjlist.Config{Directed: true})
	if err := a.AddEdge("a", "b", 1); err != nil {
		panic(err)
	}
	if err := a.AddEdge("a", "c", 2); err != nil {
		panic(err)
	}
	c := csr.BuildFromAdjList(a)

	snapDir := filepath.Join(dir, "snapshot")
	if err := snapshot.WriteSnapshotCSR(snapDir, c); err != nil {
		panic(err)
	}

	loaded, err := snapshot.Open(snapDir)
	if err != nil {
		panic(err)
	}
	fmt.Printf("manifest version=%d\n", loaded.Manifest.Version)
	fmt.Printf("csr edges=%d\n", len(loaded.CSR.Edges))

}

Output:
manifest version=1
csr edges=2

func WriteSnapshotCSRCtx ¶

func WriteSnapshotCSRCtx[W any](ctx context.Context, dir string, c *csr.CSR[W]) error

WriteSnapshotCSRCtx is the context-aware variant of WriteSnapshotCSR. ctx.Err() is checked at three stage boundaries: before the CSR write, before the manifest write, and before the atomic rename. On cancellation the temporary staging directory is cleaned up and the wrapped ctx.Err is returned.

func WriteSnapshotFull ¶

func WriteSnapshotFull[N comparable, W any](dir string, c *csr.CSR[W], g *lpg.Graph[N, W]) error

WriteSnapshotFull is the v2/v3 high-level helper: it lays out a snapshot directory containing csr.bin (legacy v1 component), labels.bin (v2 component), properties.bin (v2 component) and a manifest indexing them. When the underlying graph.Mapper is string-keyed (N=string) the writer additionally emits mapper.bin — the durable (NodeID -> natural key) interning table — and the manifest is stamped at ManifestVersion (v3). For any other N the writer falls back to the v2 layout (no mapper.bin) and the manifest records [manifestVersionV2]; recovery from a v2 snapshot continues to rely on WAL replay to re-intern keys.

Atomic publication is achieved by assembling the snapshot under dir + ".tmp" and renaming it to dir on success — the same protocol used by WriteSnapshotCSR.

When g carries a non-nil index.Manager (set via lpg.Graph.SetIndexManager) with at least one registered index that implements index.Serializer, an indexes/ sub-directory is also produced — one file per registered serializable index, each referenced from the manifest's Indexes field. Subscribers that do not implement index.Serializer are skipped (rebuild-on-restart).

Callers that do not need durable LPG labels or properties can keep using WriteSnapshotCSR; it writes a v1-shaped directory that future readers (including this one) accept transparently.

func WriteSnapshotFullCtx ¶

func WriteSnapshotFullCtx[N comparable, W any](
	ctx context.Context,
	dir string,
	c *csr.CSR[W],
	g *lpg.Graph[N, W],
) error

WriteSnapshotFullCtx is the context-aware variant of WriteSnapshotFull. ctx.Err() is checked at five stage boundaries: before the CSR write, before the labels write, before the properties write, before the manifest write, and before the atomic rename. On cancellation the temporary staging directory is cleaned up and the wrapped ctx.Err is returned.

func WriteSnapshotFullWithMapperCodec ¶

func WriteSnapshotFullWithMapperCodec[N comparable, W any](
	dir string,
	c *csr.CSR[W],
	g *lpg.Graph[N, W],
	codec keyEncoder[N],
) error

WriteSnapshotFullWithMapperCodec is the codec-aware variant of WriteSnapshotFull: it threads codec (the same [txn.Codec] the store uses to serialise node identifiers onto the WAL) into the mapper.bin writer so the durable NodeID->key interning table is emitted for ANY comparable key type N, not just string. A snapshot written this way is self-sufficient on load for every key type, which lets the checkpointer truncate the WAL instead of retaining it unboundedly (audit gap F3).

For string-keyed graphs the mapper bytes remain byte-identical to the version-1 layout (see WriteMapper), so this entry point is a safe drop-in for the existing WriteSnapshotFull on string stores too.

codec must not be nil; pass the store's [txn.Store.Codec].

func WriteSnapshotFullWithMapperCodecCtx ¶

func WriteSnapshotFullWithMapperCodecCtx[N comparable, W any](
	ctx context.Context,
	dir string,
	c *csr.CSR[W],
	g *lpg.Graph[N, W],
	codec keyEncoder[N],
) error

WriteSnapshotFullWithMapperCodecCtx is the context-aware variant of WriteSnapshotFullWithMapperCodec. ctx cancellation is honoured at the same stage boundaries as WriteSnapshotFullCtx; the only difference is that the mapper.bin component is emitted for every key type via codec rather than for string alone.

func WriteTombstones ¶

func WriteTombstones[N comparable, W any](w io.Writer, g *lpg.Graph[N, W]) (size int64, crc uint32, err error)

WriteTombstones serialises g's current tombstone set (the NodeIDs removed via lpg.Graph.RemoveNode) into w in the tombstones.bin format. It returns the number of bytes written and the CRC32C of the serialised payload — both stored in the manifest's FileEntry for the component so LoadSnapshotFull can verify integrity at load time.

The CRC32C covers the entire on-disk file, including the magic header, matching the labels.bin / properties.bin discipline. The id list is emitted in ascending order (lpg.Graph.TombstonedIDs sorts it) so the component is deterministic across writes of the same logical state.