leyline

package
v0.22.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 15, 2026 License: Apache-2.0 Imports: 25 Imported by: 0

Documentation

Overview

Package leyline — semantic.go provides typed methods for the ley-line daemon's semantic search operations (embedding-based KNN search).

SemanticClient wraps a SocketClient and translates between mache's search requests and the daemon's semantic_search / embed_status / embed_content ops. All methods are no-ops when the underlying SocketClient is nil, making the integration fully optional.

Package leyline — sheaf.go provides typed methods for the ley-line daemon's sheaf (topology-aware cache invalidation) operations.

SheafClient wraps a SocketClient and translates between mache's community detection output and the daemon's sheaf_* UDS ops. All methods are no-ops when the underlying SocketClient is nil, making the integration fully optional.

Package leyline — trigger.go pushes graph content to the ley-line daemon for embedding. Called as a fire-and-forget goroutine after graph construction.

Package leyline consumes LLO's published daemon JSON wire contract.

Index

Constants

View Source 
const (
	// ScopeChangedOnly is the fine-grained per-region contract:
	// invalidate only the caches for regions listed in RegionIDs.
	// Emitted by the consumer-driven path (mache-triggered
	// `op_sheaf_invalidate`) and by the future fine-grained
	// daemon-driven emit (LLO bead `e40566`, not yet shipped).
	ScopeChangedOnly = "changed-only"

	// ScopeAllKnown is the coarse-v1 contract shipped by LLO v0.6:
	// evict every sheaf-scoped cache entry. Region IDs are still
	// carried (they list every known region) but callers should NOT
	// treat them as a whitelist — the daemon is reporting the whole
	// working set as stale.
	ScopeAllKnown = "all-known"
)

Sheaf-invalidation scope values carried on the wire.

The scope field is emitted by LLO v0.6+ on the unified `daemon.sheaf.invalidate` topic (LLO PR #147 folded consumer- and watcher-driven emits together; bead `ley-line-open-2191e1`). An empty or absent scope is treated as ScopeChangedOnly so future emit paths that omit the field default to the fine-grained contract.

View Source 
const BinaryOverrideEnv = "MACHE_LEYLINE_BINARY"

BinaryOverrideEnv is the explicit developer escape hatch for pointing mache at a leyline build other than the pinned release — a local `cargo build`, a bisect, a patched daemon.

It exists because the pin is deliberately strict: ResolveBinary accepts ONLY leylineBinaryVersion at every tier (PATH, ~/.mache/bin, download), since LLO ships _ast schema changes in patch releases and a mismatched producer silently diverges local runs from CI (mache-608a3c). That strictness is correct for normal use and hostile to LLO development, which previously left no option but editing the pin constant.

Contract, deliberately narrow:

  • It is an explicit opt-in. Nothing sets it implicitly, and it is never consulted as a fallback after a pin miss — an unset override must not change resolution at all.
  • It logs on every use. A silently-honoured override would reintroduce the exact class of failure the pin prevents: a wrong-version producer that nobody notices until numbers diverge.
  • It fails loudly if the path does not exist, rather than falling back to the pinned resolution. Someone who sets this wants THIS binary; quietly substituting another is worse than an error.
View Source 
const BinaryVersion = leylineBinaryVersion

BinaryVersion is the exact ley-line-open release mache pins, downloads and verifies. Exported so artifact generators (tools/server-json-gen) can derive the dependency version mache publishes rather than restating it — server.json previously carried an independently-maintained "0.4.5" that had fallen four minors behind this pin.

Variables

This section is empty.

Functions

func AdoptedDaemonVersion added in v0.22.0 

func AdoptedDaemonVersion() (string, bool)

AdoptedDaemonVersion reports the leyline version a reachable daemon claims, and whether it could be determined. Exported for `mache doctor`, which otherwise reports the RESOLVED BINARY's pin and says nothing about the daemon actually serving — accurate about what it measured, silent about what the reader cares about.

func CachedPinnedBinary added in v0.21.0 

func CachedPinnedBinary() string

CachedPinnedBinary returns a cached leyline binary matching this build's pin, or "" when none is cached. It NEVER downloads.

Exported for gated tests (internal/lltest, internal/fixturedb), which must resolve the pin the same way production does. Resolving it by hand meant hardcoding the LEGACY unversioned ~/.mache/bin/leyline path — which pinnedCachePath deliberately stopped writing, so those gates skipped unconditionally on any machine that had moved past the legacy file.

func CanonicalSourceRoot added in v0.22.0 

func CanonicalSourceRoot(dir string) string

CanonicalSourceRoot resolves dir the way leyline's own check does, so mache's reset decision and leyline's refusal decision agree on whether two paths name the same tree.

EvalSymlinks (not just Abs) is load-bearing: leyline compares with Rust's `Path::canonicalize`, which resolves symlinks. This repo is routinely reached through both ~/github/art/mache and ~/remotes/art/mache — the same tree behind a symlink — and comparing unresolved paths would report a mismatch leyline itself would not, cold-starting on every alternation. A path that cannot be resolved (does not exist yet) falls back to Abs, and then to the input, so this never fails the spawn.

func ComputeStalk added in v0.9.0 

func ComputeStalk(memberCount int, crossTokens []string) []float32

ComputeStalk derives a single region's 32-D δ⁰ stalk from its community membership + the cross-community tokens it touches. Exported so the watcher path (mache-c11848) can compute a fresh stalk to pass to InvalidateWithStalk.

The agreement-dim portion is purely a function of `crossTokens`, so a content change that does not alter which cross-community tokens the region exposes will produce the same agreement coords and the daemon's δ⁰ check will (correctly) skip the cascade.

func DaemonCDC added in v0.21.1 

func DaemonCDC() bool

DaemonCDC reports whether the next auto-spawned leyline daemon should receive --cdc.

func DaemonSource added in v0.10.0 

func DaemonSource() string

DaemonSource returns the configured daemon --source dir, or "" if unset.

func DiscoverOrStart 

func DiscoverOrStart() (string, error)

DiscoverOrStart finds a running ley-line daemon socket, or auto-starts a managed daemon subprocess if the leyline binary is on PATH.

The managed daemon uses ~/.mache/ as its data directory: 

~/.mache/default.arena  — arena file
~/.mache/default.ctrl   — control block
~/.mache/default.sock   — UDS socket (what we connect to)
~/.mache/mount/         — FUSE/NFS mount point

The subprocess is killed when the mache process exits (via atexit cleanup registered on first spawn). Only one managed daemon per process.

Dedup (mache-52a23a): every "is there already a socket?" check verifies the socket is *live* (a Unix dial connects) before returning the path. A stale socket file left behind by a SIGKILL'd daemon would otherwise be returned by findExistingSocket and cause the caller to fail at first DialSocket. Worse, *between* the stat and the spawn, the entire managed daemon path can also be re-entered, orphaning daemons under concurrent callers within the same process. The full DiscoverOrStart now runs under managed.mu so the discover-then-spawn sequence is atomic across goroutines; stale on-disk sockets are removed before spawning so the daemon can bind the well-known path cleanly.

func DiscoverSocket 

func DiscoverSocket() (string, error)

DiscoverSocket finds the ley-line socket path by checking:

  1. LEYLINE_SOCKET environment variable
  2. ~/.mache/default.sock (well-known kiln deployment path)

Returns the path and nil error if a socket file exists, or an error if no socket can be found. Does NOT auto-start a daemon.

func EnsureCachedBinary added in v0.20.0 

func EnsureCachedBinary() (string, error)

EnsureCachedBinary returns the exact pinned Leyline release from ~/.mache/bin, downloading and SHA-verifying it when the cache is absent or stale. Unlike ResolveBinary, this deliberately does not consult PATH: conformance tests need to exercise the published artifact rather than a same-version developer build.

func ExtractSemver added in v0.22.0 

func ExtractSemver(s string) string

ExtractSemver pulls the first "MAJOR.MINOR[.PATCH]" token from a version line such as "leyline 0.7.0 (open)". Returns "" when none is present.

Exported because lltest must parse the same line when resolving an override binary, and a second implementation of "read leyline's version" is exactly the drift this repo keeps paying for elsewhere.

func OverrideBinary added in v0.22.0 

func OverrideBinary() (string, bool, error)

OverrideBinary returns the developer-specified leyline path when MACHE_LEYLINE_BINARY is set, and ("", false, nil) when it is not.

Exported so the gated test resolvers (internal/lltest) honour the SAME override production does. They previously called CachedPinnedBinary directly and so ignored it entirely — meaning `mache build` would run a candidate while the conformance and parity gates silently kept testing the pin (mache-cc1a70).

func PinnedBinaryVersion added in v0.18.0 

func PinnedBinaryVersion() string

PinnedBinaryVersion returns the ley-line-open release tag this mache build pins (e.g. "v0.7.8"). Exported for test gates that must verify a local leyline binary matches the pin without ever downloading one.

func RecordResolved added in v0.20.0 

func RecordResolved(path, source string)

RecordResolved publishes the provenance of a leyline binary resolved outside the managed-daemon path, so Provenance() reports it too.

Provenance was populated only by DiscoverOrStart (socket.go), the daemon route. Every other caller of ResolveBinary — notably `mache build` via autoInvokeLeylineParse — resolved a binary, shelled out to it, and left Provenance() reporting "mache hasn't resolved one this process". That is a lie by omission in exactly the situation the provenance record exists for: mache had just run a leyline binary and could not say which.

It matters beyond diagnostics. `mache build` writes a persisted .db, and the leyline that produced it determines how merkle-AST node addresses are DERIVED — LLO v0.11.0 bumped IR_SCHEMA_VERSION from merkle-ast-v1 to merkle-ast-v2, rewriting every Rust trait signature's node_hash from BYTE-IDENTICAL sources (ley-line-open #282). No content- or time-based staleness check can see that: mache's cache lockfiles hash raw source bytes, the parse skip is mtime+size, and _mache_meta recorded nothing about leyline at all. Two .db files built from identical sources under different leyline lineages were indistinguishable (mache-438104).

Recording the resolved binary is the part of that mache can fix alone, today. Consuming a lineage tag published by LLO is the durable fix and is separate (mache-43d63d, blocked on ley-line-open-348de6).

func ResolveBinary added in v0.13.0 

func ResolveBinary(allowDownload bool) (string, error)

downloadLeyline fetches the pinned-version leyline binary from GitHub releases (see leylineBinaryVersion) to the specified path. Returns the path on success. ResolveBinary locates a usable leyline binary and returns its path. Resolution order: PATH → ~/.mache/bin/leyline → (when allowDownload) auto-download the pinned ley-line-open release. This is the single provisioning chokepoint shared by the serve path (autoInvokeLeylineParse) and the build path, so `mache build` and `mache serve` acquire leyline identically instead of the build path silently degrading to tree-sitter.

Auto-download is skipped — returning an error rather than fetching — when allowDownload is false, or when MACHE_NO_LEYLINE is set (the CI/offline opt-out). A downloaded binary is cached at ~/.mache/bin/leyline for reuse.

func SetDaemonCDC added in v0.21.1 

func SetDaemonCDC(enabled bool)

SetDaemonCDC configures CDC for the auto-spawned leyline daemon. It is read at spawn time, so a daemon that is already running is not restarted or reconfigured.

func SetDaemonSource added in v0.10.0 

func SetDaemonSource(dir string)

SetDaemonSource configures the source directory passed to the auto-spawned leyline daemon. Call once at serve startup when serving a source tree. Safe for concurrent use; takes effect on the next daemon spawn (a daemon already running is not restarted).

func ShutdownManaged added in v0.22.0 

func ShutdownManaged()

ShutdownManaged stops the managed daemon for good: it is StopManaged plus a refusal to spawn again, for the process's exit paths.

StopManaged alone is not enough there. The daemon can be spawned by a background trigger once the graph finishes loading; a session that closed first stopped nothing, the trigger then spawned a daemon, and the process exited underneath it. The daemon leads its own process group, so it survived as an orphan. The flag is set BEFORE stopping, so no spawn can slip in between (mache-25f49e).

func StopManaged 

func StopManaged()

StopManaged gracefully stops the auto-spawned leyline daemon, if any. Sends SIGTERM first so leyline can unmount NFS cleanly, then waits up to 3 seconds before falling back to SIGKILL. Without this, macOS shows a "Server connections interrupted" dialog for the stale NFS mount. Safe to call multiple times. Called automatically by cleanup hooks.

func TriggerEmbedding added in v0.6.1 

func TriggerEmbedding(g graph.Graph, batchSize int)

TriggerEmbedding walks all file nodes in the graph and pushes their content to the ley-line daemon for embedding via the embed_content op. Batches nodes in groups of batchSize. Logs errors but does not fail.

func VerifyReachableDaemonVersion added in v0.11.0 

func VerifyReachableDaemonVersion() error

VerifyReachableDaemonVersion is the startup wire-compat handshake: if a leyline daemon is ALREADY reachable (it never auto-starts one), it queries the leyline_version op and refuses to proceed on a structural mismatch. It returns nil when no daemon is reachable, when the daemon predates the leyline_version op, or when the versions are compatible — a version probe must never be stricter than the actual wire decode.

Types

type ASTDef added in v0.18.0 

type ASTDef struct {
	Token           string  `json:"token"`
	NodeID          string  `json:"node_id"`
	SourceID        string  `json:"source_id"`
	ContainerNodeID *string `json:"container_node_id"`
	CanonicalKind   *string `json:"canonical_kind"`
}

ASTDef mirrors one `node_defs` row from emit_ast.

type ASTImport added in v0.18.0 

type ASTImport struct {
	Alias    string `json:"alias"`
	Path     string `json:"path"`
	SourceID string `json:"source_id"`
}

ASTImport mirrors one `_imports` row from emit_ast.

type ASTPayload added in v0.18.0 

type ASTPayload struct {
	SourceID    string      `json:"source_id"`
	Language    string      `json:"language"`
	ContentHash string      `json:"content_hash"`
	AST         []ASTRow    `json:"ast"`
	Defs        []ASTDef    `json:"defs"`
	Refs        []ASTRef    `json:"refs"`
	Imports     []ASTImport `json:"imports"`
}

ASTPayload is the `ast` object the daemon folds into a validate response when emit_ast is requested: SQL-shaped `_ast`/`node_defs`/`node_refs`/ `_imports` rows from the SAME parse that produced the syntax verdict.

type ASTRef added in v0.18.0 

type ASTRef struct {
	Token           string  `json:"token"`
	NodeID          string  `json:"node_id"`
	SourceID        string  `json:"source_id"`
	ContainerNodeID *string `json:"container_node_id"`
}

ASTRef mirrors one `node_refs` row from emit_ast.

type ASTRow added in v0.18.0 

type ASTRow struct {
	NodeID    string `json:"node_id"`
	SourceID  string `json:"source_id"`
	NodeKind  string `json:"node_kind"`
	StartByte uint32 `json:"start_byte"`
	EndByte   uint32 `json:"end_byte"`
	StartRow  uint32 `json:"start_row"`
	StartCol  uint32 `json:"start_col"`
	EndRow    uint32 `json:"end_row"`
	EndCol    uint32 `json:"end_col"`
	NodeHash  string `json:"node_hash"`
}

ASTRow mirrors one `_ast` table row emitted by the daemon's emit_ast extension. NodeID is the hierarchical merkle-AST path (parent node_id + "/" + child kind, with a `_N` index suffix when a parent has multiple same-kind named children), rooted at the request's source_id.

type ArenaState added in v0.22.0 

type ArenaState struct {
	// Path is the shared arena file. It may not exist yet.
	Path string
	// ConfigPath is mache's sidecar record beside it.
	ConfigPath string
	// Exists reports whether the arena file itself is present.
	Exists bool
	// HasRecord reports whether mache has a parseable record of the config it
	// last spawned with. False means "cannot vouch for this arena" — which is
	// the reset trigger, not an error.
	HasRecord bool
	// SourceRoot is the canonicalized tree the arena is bound to, when known.
	SourceRoot string
	// CDCTarget is the activation target recorded for it, when known.
	CDCTarget string
}

ArenaState is what `mache doctor` needs to explain a warm-start refusal without the operator reading JSON out of ~/.mache by hand.

The incident this exists for: two stale daemons squatting the shared arena presented as three unrelated symptoms (empty sheaf cache, get_overview timeout, killed leyline parse), and nothing in mache could report that the arena was pinned to a different tree than the one being served.

func InspectArena added in v0.22.0 

func InspectArena() (ArenaState, error)

InspectArena reports the shared arena's state without mutating anything.

A missing arena, a missing record, and an unparseable record are all reported rather than returned as errors: each is a legitimate state a diagnostic must describe, and only a failure to locate the home directory is a genuine error.

func (ArenaState) ArenaBoundElsewhere added in v0.22.0 

func (s ArenaState) ArenaBoundElsewhere(want string) bool

ArenaBoundElsewhere reports whether the arena is recorded against a tree other than want — the exact condition leyline's verify_source_root_matches refuses to warm-start on. Comparison is canonicalized on both sides so a symlinked checkout does not read as a mismatch leyline itself would not see.

Returns false when there is no record: unknown is not disagreement.

type DaemonResponseError added in v0.20.0 

type DaemonResponseError struct {
	Message string
}

DaemonResponseError is an error envelope returned by the ley-line daemon.

Its message is kept separately so graph adapters can retain their public error contracts (for example, mapping a missing node to graph.ErrNotFound) while SendOpInto still rejects an unsuccessful typed response loudly.

func (*DaemonResponseError) Error added in v0.20.0 

func (e *DaemonResponseError) Error() string

type EmbedStatus added in v0.6.1 

type EmbedStatus struct {
	Ready      bool   `json:"ready"`
	Count      int    `json:"count"`
	Dimensions int    `json:"dimensions"`
	Model      string `json:"model"`
}

EmbedStatus reports the state of the embedding index.

type EventHandler added in v0.9.0 

type EventHandler func(SheafInvalidateEvent)

EventHandler is called once per pushed sheaf.invalidate event. Handler may be invoked concurrently across reconnect boundaries — implementations must guard their own state.

type FindCalleesResponse added in v0.9.0 

type FindCalleesResponse = daemonwire.FindCalleesResponse

These aliases preserve mache's internal call sites while making the LLO schema module the single definition of the daemon response contract.

type FindCallersResponse added in v0.9.0 

type FindCallersResponse = daemonwire.FindCallersResponse

These aliases preserve mache's internal call sites while making the LLO schema module the single definition of the daemon response contract.

type GetNodeResponse added in v0.9.0 

type GetNodeResponse = daemonwire.GetNodeResponse

These aliases preserve mache's internal call sites while making the LLO schema module the single definition of the daemon response contract.

type LeylineProvenance added in v0.10.0 

type LeylineProvenance struct {
	Path            string `json:"path"`
	Version         string `json:"version"`          // as reported by `<bin> --version` (best-effort)
	Source          string `json:"source"`           // PATH | cached | downloaded
	ExpectedVersion string `json:"expected_version"` // the compile-time pin (leylineBinaryVersion)
	VersionMatches  bool   `json:"version_matches"`  // reported version contains the pinned tag
}

LeylineProvenance is the observable provenance of the leyline binary mache resolved for the managed daemon. Surfaced so "which leyline is mache using?" is a log grep / MCP tool call instead of an archaeology dig (the failure mode that hid the stale-cached-0.4.1 skew, mache-0acdf6).

func Provenance added in v0.10.0 

func Provenance() (LeylineProvenance, bool)

Provenance returns the resolved leyline binary's provenance, or ok=false if mache hasn't resolved one this process. The returned ExpectedVersion is always populated (the compile-time pin) even when ok is false.

type ListChildrenResponse added in v0.9.0 

type ListChildrenResponse = daemonwire.ListChildrenResponse

These aliases preserve mache's internal call sites while making the LLO schema module the single definition of the daemon response contract.

type Node added in v0.9.0 

type Node = daemonwire.Node

These aliases preserve mache's internal call sites while making the LLO schema module the single definition of the daemon response contract.

type NodeContent added in v0.6.1 

type NodeContent struct {
	ID      string `json:"id"`
	Content string `json:"content"`
}

NodeContent is a node to embed: id + text content.

type OwnerRecord added in v0.22.0 

type OwnerRecord struct {
	// Pin is the leyline release the spawner required (leylineBinaryVersion
	// at spawn time), which is NOT necessarily what the binary reports —
	// asking the daemon remains the authority (a file named v0.18.2 reported
	// 0.18.1 this week). The record is provenance, not identity.
	Pin string `json:"pin"`
	// DaemonPID is the spawned daemon process.
	DaemonPID int `json:"daemon_pid"`
	// SpawnerPID is the mache process that spawned it. A dead spawner beside
	// a live daemon is the orphaned-but-alive state that used to accumulate
	// silently.
	SpawnerPID int `json:"spawner_pid"`
	// StartedAt is when the daemon reported ready.
	StartedAt time.Time `json:"started_at"`
}

OwnerRecord is who started a leyline daemon, with what, and when — written next to the socket so "is this ours and current" is answerable without a round trip, and so a stale daemon can be ATTRIBUTED instead of merely detected.

Why it exists (mache-967cff): drift detection can say "a daemon reporting 0.18.1 is running against a v0.19.0 pin" but not whose it is, how old it is, or whether its spawner died long ago. Observed twice in one week: an 11-day-21-hour daemon nobody could date without `ps etime`, and a leaked test daemon that ran 1h43m before the drift warning caught it — caught, but unattributable. Cleanup stayed manual `kill` by a human reading ps.

func ReadOwnerRecord added in v0.22.0 

func ReadOwnerRecord(sockPath string) (OwnerRecord, bool)

ReadOwnerRecord loads the record for a socket, reporting whether one exists and parses. Absence is normal — every daemon spawned before this existed, and daemons started by hand, have no record. A record proves provenance; its absence proves nothing.

func WellKnownOwnerRecord added in v0.22.0 

func WellKnownOwnerRecord() (OwnerRecord, bool)

WellKnownOwnerRecord reads the record for the default shared socket. Exported for `mache doctor`, which reports provenance beside the daemon's self-reported version.

func (OwnerRecord) Age added in v0.22.0 

func (r OwnerRecord) Age(now time.Time) time.Duration

Age is how long the daemon has been running per the record.

func (OwnerRecord) Orphaned added in v0.22.0 

func (r OwnerRecord) Orphaned() bool

Orphaned reports whether the daemon has outlived its spawner: daemon alive, spawner dead. This is precisely the state that used to accumulate silently (the 11-day daemon), distinct from a STALE record where the daemon itself is gone and only the file remains.

func (OwnerRecord) Stale added in v0.22.0 

func (r OwnerRecord) Stale() bool

Stale reports a record whose daemon is dead — the file outlived the process and should be reported rather than silently trusted.

type ReadContentResponse added in v0.9.0 

type ReadContentResponse = daemonwire.ReadContentResponse

These aliases preserve mache's internal call sites while making the LLO schema module the single definition of the daemon response contract.

type Ref added in v0.9.0 

type Ref = daemonwire.Ref

These aliases preserve mache's internal call sites while making the LLO schema module the single definition of the daemon response contract.

type SemanticClient added in v0.6.1 

type SemanticClient struct {
	// contains filtered or unexported fields
}

SemanticClient provides typed access to ley-line's embedding operations. A nil SemanticClient is safe — all methods return zero values without error.

func NewSemanticClient added in v0.6.1 

func NewSemanticClient(sock *SocketClient) *SemanticClient

NewSemanticClient wraps an existing SocketClient. sock may be nil.

func (*SemanticClient) EmbedContent added in v0.6.1 

func (sc *SemanticClient) EmbedContent(nodes []NodeContent) (int, error)

EmbedContent pushes node content to the daemon for embedding. Returns the number of nodes successfully embedded.

func (*SemanticClient) Search added in v0.6.1 

func (sc *SemanticClient) Search(query string, k int) ([]SemanticResult, error)

Search performs a semantic search over embedded nodes. Returns results sorted by distance (closest first).

func (*SemanticClient) Status added in v0.6.1 

func (sc *SemanticClient) Status() (EmbedStatus, error)

Status returns the current embedding index status.

type SemanticResult added in v0.6.1 

type SemanticResult struct {
	ID       string  `json:"id"`
	Distance float64 `json:"distance"`
}

SemanticResult is a single KNN search result.

type SheafClient added in v0.6.1 

type SheafClient struct {
	// contains filtered or unexported fields
}

SheafClient provides typed access to ley-line's sheaf operations. A nil SheafClient is safe — all methods return zero values without error.

func NewSheafClient added in v0.6.1 

func NewSheafClient(sock *SocketClient) *SheafClient

NewSheafClient wraps an existing SocketClient. sock may be nil.

func (*SheafClient) Defect added in v0.6.1 

func (sc *SheafClient) Defect() (float64, error)

Defect queries the global consistency defect score. Returns 0.0 when the daemon is unavailable.

func (*SheafClient) Invalidate added in v0.6.1 

func (sc *SheafClient) Invalidate(regionID int) ([]int, error)

Invalidate marks a region as stale without pushing new stalk data. The daemon's cascade falls back to the heuristic XOR-of-endpoints pre-filter for this op. Callers that have the post-change stalk available should prefer InvalidateWithStalk so the daemon's δ⁰ check decides whether the change actually crossed the agreement plane.

func (*SheafClient) InvalidateWithStalk added in v0.9.0 

func (sc *SheafClient) InvalidateWithStalk(regionID int, newHash string, newData []float32) ([]int, error)

InvalidateWithStalk reports regionID changed and pushes the new 32-D f32 stalk so the daemon can evaluate δ⁰ against the baseline recorded at sheaf_set_topology time. Returns the BFS-ordered list of regions the daemon determined are transitively affected.

newHash may be empty — the daemon will infer the hash from data when both `hash` and `data` are present. When both hash and data are empty, this is equivalent to Invalidate.

Returns an error when newData is non-empty but len(newData) is not stalkDim — a wrong-sized vector would either be rejected daemon-side with a confusing error or, worse, cause the δ⁰ projection to silently mis-align with restriction agreement_dim and skip the cascade entirely.

func (*SheafClient) PushTopology added in v0.6.1 

func (sc *SheafClient) PushTopology(cr *graph.CommunityResult, refs map[string][]string) error

PushTopology converts Louvain community detection output into a sheaf_set_topology op and sends it to the daemon.

Each community maps to a region whose hash is the SHA-256 of the sorted, concatenated member node IDs. Each region also carries a 32-D f32 stalk: the first 30 dims are SHA-256 of the region's cross-community tokens (so two regions agree iff they share the same set of cross-boundary refs), and the trailing 2 dims are private discriminators (member count + cross-degree). Restriction edges carry `agreement_dim=30` so the daemon's δ⁰ check projects the agreement subspace.

With these inputs the daemon engages δ⁰-driven invalidation (response advertises `delta_zero_mode: true`). Without the f32 data the daemon silently falls back to heuristic-only mode, which is correct but loses the precision guarantee from LLO PR #16.

func (*SheafClient) Status added in v0.6.1 

func (sc *SheafClient) Status() (SheafStatus, error)

Status returns the full sheaf status from the daemon.

type SheafInvalidateEvent added in v0.9.0 

type SheafInvalidateEvent struct {
	// Invalidated carries the region identifiers the daemon named in
	// the payload. Populated from either `invalidated` or `region_ids`
	// (the parser reads both — see the type docstring). Interpretation
	// depends on Scope (see the constants above): under
	// ScopeChangedOnly this is a whitelist to invalidate; under
	// ScopeAllKnown it's a snapshot of every known region and callers
	// should evict everything.
	//
	// The field keeps the `Invalidated` name to preserve the public-API
	// contract of the mache event surface.
	Invalidated []int

	// Generation is the daemon's monotonic counter after the emit.
	// Agents compare this to a previously-cached value to decide
	// whether their snapshot is still fresh.
	Generation uint64

	// PriorGeneration is the counter value before this event's bump.
	// Populated on both topics.
	PriorGeneration uint64

	// Count mirrors len(Invalidated); kept as a separate wire field
	// because the daemon emits it that way.
	Count int

	// Scope classifies the event: [ScopeChangedOnly] or [ScopeAllKnown].
	// See the constants for semantics + wire-emission provenance. The
	// dispatcher treats an unknown non-empty value as ScopeAllKnown
	// (over-invalidate rather than serve stale) and logs a warning so
	// a new future scope value isn't silently dropped.
	//
	// Empty when the emit path omits the field — cmd routing infers
	// ScopeChangedOnly for the empty case.
	Scope string

	// ChangedFiles lists the files whose reparse triggered the emit.
	// Watcher-driven emits populate this; consumer-driven emits (from
	// mache's own `op_sheaf_invalidate` calls) leave it empty.
	ChangedFiles []string

	// CurrentRoot is the 64-hex BLAKE3 root of the substrate state
	// after the emit. Watcher-driven emits populate this; may be empty
	// when the daemon's controller read failed (best-effort, degrades
	// honestly per LLO PR #140).
	CurrentRoot string

	// TimestampMs is now_ms() at emit time. Watcher-driven emits only.
	TimestampMs int64
}

SheafInvalidateEvent is the parsed shape of a sheaf-invalidate event pushed by the daemon on the `daemon.sheaf.invalidate` topic (LLO v0.6+, unified by LLO PR #147 / bead `ley-line-open-2191e1`).

Payload shape: 

{invalidated | region_ids, count, scope, changed_files,
 current_root, generation, prior_generation, timestamp_ms}

The parser reads both `invalidated` and `region_ids` for the region list so a wire-name flip in LLO doesn't break decode — the forward-compat tolerance is free and covered by TestSheafSubscriber_DispatchesEvent (`invalidated`) and TestSheafSubscriber_DispatchesWatcherDrivenEvent (`region_ids`).

`Generation` and `PriorGeneration` are parsed from either raw JSON numbers or capnp-json quoted-string Int64 (see parseUint64) so a single struct handles both encodings without a per-shape parse.

type SheafStatus added in v0.6.1 

type SheafStatus struct {
	Generation uint64  `json:"generation"`
	Valid      int     `json:"valid"`
	Total      int     `json:"total"`
	Defect     float64 `json:"defect"`
}

SheafStatus mirrors the response from sheaf_status / sheaf_defect.

type SheafSubscriber added in v0.9.0 

type SheafSubscriber struct {
	// contains filtered or unexported fields
}

SheafSubscriber owns a long-lived `subscribe` connection to the ley-line daemon, dispatching pushed sheaf-invalidate events to a handler.

Subscribes to a single topic: `daemon.sheaf.invalidate` (LLO v0.6+). LLO PR #147 (bead `ley-line-open-2191e1`) unified both the consumer-driven emit (from `op_sheaf_invalidate` when a client calls SheafClient.Invalidate) and the watcher-driven emit (from the daemon's own enrichment cycle) under this topic. The pre-v0.6 `sheaf.invalidate` topic is no longer emitted by LLO — mache pins to LLO tip, so there is no v0.5.x daemon to be compatible with.

Events decode to SheafInvalidateEvent; the routing layer in cmd/sheaf_subscribe.go dispatches on the Scope field.

FIXED UPSTREAM (LLO v0.4.3, ley-line-open-5caa59): pre-v0.4.3 LLO daemons did NOT actually deliver sheaf.invalidate events to UDS subscribers — `handle_connection` was a pure request/response loop that never drained `ConnectionState.event_rx`, so emitted events silently accumulated. The cascade math + op response were always correct; only the event-bus push was broken. LLO v0.4.3 shipped a per-connection writer task plus a per-subscribe event-relay task; mache requires v0.4.3 or later for daemon-pushed cascades.

tools/sheaf-subscribe-probe/main.go is the canonical repro tool; internal/leyline/sheaf_subscriber_e2e_test.go is the regression guard that catches a future re-emergence of this bug.

Design constraints (locked in by PR for mache-c14c43):

  • Dedicated SocketClient (not shared with the watcher's SheafClient). The Subscribe primitive owns the read side of the conn after activation, and would race any concurrent SendOp — see PR #383's SocketClient docstring. Mache should hold one SocketClient for ops and one for events.

  • One subscriber per process. Spawned at serve startup, routes events through a single handler that the caller (cmd/serve) uses to fan out to all per-graph invalidators.

  • Reconnect with exponential backoff. The daemon may restart; the subscriber must redial transparently without leaking goroutines or spinning when the socket is permanently absent. Backoff starts at 1s and caps at 30s in production; tests override via SetBackoffForTest.

  • Observable state. Status() returns the current connection state + last-event time + last-seen generation. cmd/serve_handlers' new get_sheaf_status MCP tool will surface this so agents have an honest "cascade engaged?" signal.

  • Unmapped events log + skip, not buffer. The handler is responsible for mapping region IDs → node IDs via the current CommunityResult; when no result is installed yet, drop the event (the cascade for that window is unreachable anyway, and any cached results are equally pre-result-installation stale).

func NewSheafSubscriber added in v0.9.0 

func NewSheafSubscriber(sockPath string, handler EventHandler) *SheafSubscriber

NewSheafSubscriber creates a subscriber that will dial sockPath and dispatch events to handler. Use Start to begin the loop; Stop to shut it down cleanly.

Both lifecycle channels are constructed here so Stop is safe to call in any order relative to Start (see Stop's contract).

func (*SheafSubscriber) SetBackoffForTest added in v0.9.0 

func (s *SheafSubscriber) SetBackoffForTest(min, max time.Duration)

SetBackoffForTest overrides the dial-retry backoff bounds. Test-only; production callers must not touch this. Kept on the public surface (rather than exposed via build tag) because the alternative — making the bounds a constructor option — would force every caller to thread production defaults through.

func (*SheafSubscriber) Start added in v0.9.0 

func (s *SheafSubscriber) Start(ctx context.Context)

Start spawns the subscribe loop in a background goroutine. Returns immediately. The loop runs until Stop is called or ctx is cancelled.

Idempotent — repeated calls after the first are no-ops (sync.Once gate). Concurrent Start calls collapse to a single loop spawn so two goroutines can't accidentally race for the conn.

func (*SheafSubscriber) Status added in v0.9.0 

func (s *SheafSubscriber) Status() SubscriberStatus

Status returns a snapshot of the subscriber's current state. Cheap; callers (e.g. the get_sheaf_status MCP handler) can poll freely.

func (*SheafSubscriber) Stop added in v0.9.0 

func (s *SheafSubscriber) Stop()

Stop signals the subscribe loop to terminate and blocks until it has.

Contract:

  • When Start was never called, returns immediately (no goroutine to stop, no channel to wait on).
  • When Start was called, every Stop call (including concurrent and repeated ones) waits until run() has actually returned. Returning before that would let callers observe a Disconnected state too early — the contract is "by the time Stop returns, the loop is gone." Multiple Stop calls all observe the same close on doneCh (a closed channel is permanently ready) so the wait is cheap after the first one completes.

Fixed in response to PR #384 Copilot #1: the prior shape blocked forever pre-Start and short-circuited on a "stopped" flag for repeated calls, violating both guarantees.

type SocketClient 

type SocketClient struct {
	// contains filtered or unexported fields
}

SocketClient communicates with a running ley-line daemon over its Unix domain control socket ({ctrl}.sock).

SendOp / SendOpInto are safe for concurrent use: sendMu serializes the write-then-read pair so the line-delimited JSON protocol can't get crossed when multiple callers (e.g. the file watcher firing multiple debounce timers after a save burst) hit the same client simultaneously. The protocol is request/response — interleaving produces a response for the wrong caller, which the daemon can't detect since there's no per-message correlation ID.

Subscribe is the exception: once a Subscribe goroutine takes ownership of the read side, ad-hoc SendOp on the same connection is unsupported and will race the subscription's event reads. The canonical usage is one SocketClient per role (one for ops, one for subscribe).

func DialSocket 

func DialSocket(sockPath string) (*SocketClient, error)

DialSocket connects to the ley-line control socket at sockPath. The socket path is typically derived from the control path: e.g. /tmp/leyline.ctrl → /tmp/leyline.sock

func (*SocketClient) Close 

func (c *SocketClient) Close() error

Close closes the underlying connection. Safe to call multiple times from any number of goroutines — the sync.Once gates the underlying conn.Close so concurrent shutdown can't race the Subscribe read goroutine (see closeOnce docstring on SocketClient for the history). The conn pointer is intentionally NOT zeroed; closed conn errors from subsequent reads/writes propagate normally to their callers.

func (*SocketClient) Enrich added in v0.7.0 

func (c *SocketClient) Enrich(pass string, files []string) (map[string]any, error)

Enrich runs a daemon enrichment pass (e.g. "lsp", "embed") over the given files via the `enrich` op, populating the corresponding arena tables (for "lsp": _lsp_hover / _lsp / _lsp_defs / _lsp_refs). `files` are paths relative to the daemon's --source dir; nil/empty enriches every file. The op is SYNCHRONOUS on the daemon side — it runs the pass and snapshots the living db to the arena before responding — so a subsequent Query observes the populated tables.

This replaces the prior `tool` op: the leyline daemon's UDS dispatch speaks named ops directly ({"op":"enrich","pass":...}); the MCP tools/call envelope ({"op":"tool","name":...}) only exists on the HTTP /mcp endpoint, so sending it over UDS returned "unknown op: tool" (mache-303036).

func (*SocketClient) Prioritize added in v0.7.0 

func (c *SocketClient) Prioritize(files []string) error

Prioritize requests the daemon to parse the given files on the next pass. Non-blocking from the daemon's perspective: it acknowledges the request and schedules the files for priority parsing asynchronously.

func (*SocketClient) Query 

func (c *SocketClient) Query(sql string) ([][]any, error)

Query runs a SQL query against the active arena buffer via the `query` op and returns the rows as [][]any (one slice per row, column order matching the SELECT / the response's `columns` field).

The daemon's `query` op returns each row as a JSON OBJECT keyed by column name ({"node_id":..,"hover_text":..}), with a separate ordered `columns` array — NOT an array of values. (A legacy array-of-arrays shape is also accepted for forward/back-compat.) The prior parser only handled the array shape and silently dropped every object row, so any query against the daemon returned zero rows — which is exactly why LSP get_type_info surfaced no hover even though enrichment populated _lsp_hover (mache-303036).

func (*SocketClient) SendOp 

func (c *SocketClient) SendOp(req map[string]any) (map[string]any, error)

SendOp sends a JSON request and reads the JSON response. Both are line-delimited (newline-terminated JSON).

func (*SocketClient) SendOpInto added in v0.9.0 

func (c *SocketClient) SendOpInto(req map[string]any, dest any) error

SendOpInto sends a JSON request and decodes the response into dest. Use this for ops with typed response structs (see wire.go); SendOp's map[string]any return silently zeros Int64 fields under the post-b0ea2e daemon wire (capnp-json codec emits int64 as JSON strings).

func (*SocketClient) SetDeadline 

func (c *SocketClient) SetDeadline(t time.Time) error

SetDeadline sets the read/write deadline on the underlying connection.

func (*SocketClient) Subscribe added in v0.7.0 

func (c *SocketClient) Subscribe(topics []string) (<-chan map[string]any, error)

Subscribe sends a subscribe op and returns a channel that receives pushed events from the daemon. The subscription runs until the connection closes.

Topics use dot-separated hierarchical matching:

  • "daemon.snapshot" — exact match
  • "daemon.*" — one segment wildcard
  • "daemon.**" — any segments wildcard

The returned channel is buffered (64). If the consumer falls behind, events are dropped (the daemon handles overflow per its policy).

func (*SocketClient) SubscribeDropped added in v0.9.0 

func (c *SocketClient) SubscribeDropped() uint64

SubscribeDropped returns the cumulative number of events the Subscribe goroutine has dropped because the consumer's channel buffer was full. Non-zero values mean the consumer is falling behind the daemon's event rate; investigate before treating cache state as authoritative.

func (*SocketClient) Validate added in v0.18.0 

func (c *SocketClient) Validate(content []byte, language, path string, emitAST bool) (*ValidateResult, error)

Validate runs the daemon's validate op on caller-supplied content. language is a leyline extension key ("go", "py", ...); path is optional and (a) lets the daemon infer the language when language is empty, (b) becomes the source_id on emitted AST rows. emitAST folds ONE parse into both the syntax verdict and SQL-shaped AST rows (ValidateResult.AST).

A daemon response without the `errors` key, or without the `ast` payload when emitAST was requested on a clean parse, is a hard error: the daemon predates the v0.7.8 wire contract and mache must not silently skip validation or linting.

func (*SocketClient) VerifyVersion added in v0.11.0 

func (c *SocketClient) VerifyVersion(clientVersion string) error

VerifyVersion queries the daemon's leyline_version op and checks wire-format and schema compatibility against clientVersion (this build's leyline schema-client version). A daemon that predates the op (SendOp errors) is not an error — older daemons simply can't be verified, and we must not refuse a daemon we might still decode fine.

type SubscriberState added in v0.9.0 

type SubscriberState int

SubscriberState classifies the subscriber's connection state. Surfaced via Status() and ultimately through get_sheaf_status. 

const (
	// StateDisconnected: not currently connected; either not yet started
	// or the loop exited (Stop called, ctx cancelled, fatal error).
	StateDisconnected SubscriberState = iota
	// StateConnecting: between dial attempts (in backoff) or actively dialing.
	StateConnecting
	// StateConnected: subscribed and listening for events.
	StateConnected
)

func (SubscriberState) String added in v0.9.0 

func (s SubscriberState) String() string

String renders the state for logging / MCP responses. Lower-case since it appears in agent-consumable JSON.

type SubscriberStatus added in v0.9.0 

type SubscriberStatus struct {
	State          SubscriberState
	Reason         string    // non-empty when State != Connected, explains why
	LastEvent      time.Time // zero when no event seen yet
	LastGeneration uint64    // 0 when no event seen yet
}

SubscriberStatus is the snapshot returned by SheafSubscriber.Status().

type ValidateResult added in v0.18.0 

type ValidateResult struct {
	OK     bool
	Errors []ValidateSyntaxError
	AST    *ASTPayload // non-nil only when emit_ast was requested and honored
}

ValidateResult is the decoded validate response.

func ValidateContent added in v0.18.0 

func ValidateContent(content []byte, language, path string, emitAST bool) (*ValidateResult, error)

ValidateContent is the write-back path's convenience wrapper: it acquires a daemon via DiscoverOrStart (reusing a live socket when one exists — env LEYLINE_SOCKET or ~/.mache/default.sock — else lazily spawning a managed daemon), dials, runs one validate op, and closes the connection.

Latency: the UDS dial + round trip is sub-millisecond when a daemon is already running. The FIRST write after mount may pay a cold daemon spawn (typically 0.5–2s: arena init + socket bind); subsequent writes reuse that daemon via the well-known socket. Dial-per-call is deliberate — it avoids a cached client going stale when the daemon restarts between writes.

type ValidateSyntaxError added in v0.18.0 

type ValidateSyntaxError struct {
	Row       uint32 `json:"row"`
	Col       uint32 `json:"col"`
	ByteStart uint32 `json:"byte_start"`
	ByteEnd   uint32 `json:"byte_end"`
	Message   string `json:"message"`
}

ValidateSyntaxError is one ERROR/MISSING node from the daemon's validate op. Row and Col are 0-based; Col is the byte offset within the row. ByteStart == ByteEnd for zero-width MISSING nodes.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.