Documentation
¶
Overview ¶
Package merkle is an implementation of a Merkel DAG
Index ¶
- Variables
- func ProjectContent(blocks []llm.ContentBlock) []llm.ContentBlock
- type Bucket
- type Dag
- func (d *Dag) AddNode(node *Node) (*DagNode, error)
- func (d *Dag) Ancestors(hash string) []*DagNode
- func (d *Dag) BranchPoints() []*DagNode
- func (d *Dag) Descendants(hash string) []*DagNode
- func (d *Dag) Get(hash string) *DagNode
- func (d *Dag) IsBranching(hash string) bool
- func (d *Dag) Leaves() []*DagNode
- func (d *Dag) Size() int
- func (d *Dag) Walk(f func(*DagNode) (bool, error)) error
- type DagNode
- type Node
- type NodeOptions
Constants ¶
This section is empty.
Variables ¶
var HarnessTags = []string{
"system-reminder",
"command-name",
"command-message",
"command-args",
"local-command-stdout",
"local-command-stderr",
"local-command-caveat",
"session",
"conversation",
"new-diagnostics",
"task-notification",
"status",
"summary",
"transcript",
"event",
"tool-use-id",
"output-file",
"task-id",
}
HarnessTags lists the XML-like tags emitted by agent harnesses (e.g. Claude Code) that wrap mutable metadata into user-role content blocks. Text wrapped in these tags drifts between turns of the same conversation — the clock ticks, a skill loads, an MCP inventory changes — yet none of it is part of the user's intent. Letting it participate in the content-addressed hash fractures otherwise continuous conversations into multiple disconnected roots. See PCC-562.
Tags are matched verbatim, so "system-reminder" matches <system-reminder>…</system-reminder> but does NOT match <local-command-stdout>; each suffixed variant must be listed explicitly.
Functions ¶
func ProjectContent ¶ added in v0.10.0
func ProjectContent(blocks []llm.ContentBlock) []llm.ContentBlock
ProjectContent returns the projection of content blocks used when computing a node's chain hash. The projection:
- Strips every <tag>…</tag> span whose tag is in HarnessTags from each block's Text field.
- Normalizes whitespace in what remains so blank-line drift inside the surviving prose cannot fork the chain (see PCC-562's "57a58 >" case).
- Drops any block that became empty after stripping.
- For tool_use blocks, drops zero-valued keys from ToolInput. The streamed assistant capture sees every key the model emitted including optional defaults like Edit's "replace_all": false; when that same turn is re-sent as history the harness strips those defaults. Without this step the two captures hash to different nodes and the chain branches.
- For thinking blocks, drops ThinkingSignature. Anthropic emits the signature on the live stream but the harness omits it on re-send; same shape as (4).
The input slice and its blocks are not mutated; the returned slice is independent. Bucket.Content itself stays unprojected so display, labels, and search continue to see the raw text the model received.
Types ¶
type Bucket ¶
type Bucket struct {
// Type identifies the kind of content (e.g., "message")
Type string `json:"type"`
// Role indicates who produced this message ("system", "user", "assistant", "tool")
Role string `json:"role"`
// Content holds the message content blocks
Content []llm.ContentBlock `json:"content"`
// Model identifies the LLM model (e.g., "gpt-4", "claude-3-sonnet")
Model string `json:"model"`
// Provider identifies the API provider (e.g., "openai", "anthropic", "ollama")
Provider string `json:"provider"`
// AgentName identifies the agent harness (e.g., "claude", "opencode", "codex")
AgentName string `json:"agent_name,omitempty"`
}
Bucket represents the hashable content stored in a Merkle DAG node. This is the tapes canonical content-addressable hashing structure for all LLM conversation turns.
func (*Bucket) Clone ¶ added in v0.19.1
Clone returns a deep copy of the bucket with every string field and content block reallocated, so the copy shares no backing array with the zero-copy raw request buffer the bucket was parsed from. The bytes are identical to the original, so a node hashed from a cloned bucket produces the same Hash — cloning is a pure memory optimization, never a content change. See llm.ContentBlock.Clone for why the alias matters.
func (*Bucket) ExtractText ¶
ExtractText returns the concatenated text content from the bucket's content blocks. This is useful for generating embeddings for semantic search. It extracts text from text blocks, tool outputs, and tool use requests, joining them with newlines.
type Dag ¶
type Dag struct {
// Root is the single root node of this directed graph
Root *DagNode
// contains filtered or unexported fields
}
Dag is an in-memory view of a single-rooted Merkle DAG (i.e., a single branch within the graph's plane).
It is loaded on-demand from a provided BranchLoader.
func (*Dag) AddNode ¶ added in v0.5.0
AddNode adds a node to the given DAG. The node's ParentHash must reference an existing node in the DAG, or be nil (making it the root).
This method returns an error if:
- The provided node is nil: this is a programmer error.
- The parent hash references a node not in the DAG (this node does not belong in this branch)
- Adding a root node (where node.Parent is empty) when one already exists
This method is a noop if:
- The node already exists in the DAG
func (*Dag) Ancestors ¶
Ancestors returns the path from the given node up to the root. The returned slice is ordered from the node to root (node first, root last). Returns nil if the hash is not found.
func (*Dag) BranchPoints ¶
BranchPoints returns all nodes that have more than one child.
func (*Dag) Descendants ¶
Descendants returns all descendants of the given node (children, grandchildren, etc.). The returned slice is ordered by depth-first traversal. Returns nil if the hash is not found.
func (*Dag) IsBranching ¶
IsBranching returns true if the node with the given hash has multiple children. Returns false if the hash is not found or has 0-1 children.
type DagNode ¶
type DagNode struct {
*Node
// Parent is the parent node in the DAG (nil for root)
Parent *DagNode
// Children are the child nodes (empty for leaves)
Children []*DagNode
}
DagNode wraps a "Node" with structural relationships for efficient traversal.
type Node ¶
type Node struct {
// Hash is the content-addressed identifier (SHA-256, hex-encoded)
Hash string `json:"hash"`
// ParentHash links to the previous node hash.
// This will be nil for root nodes.
ParentHash *string `json:"parent_hash"`
// Bucket is the hashable content for the node.
Bucket Bucket `json:"bucket"`
// StopReason indicates why generation stopped (only for responses)
// Values: "stop", "length", "tool_use", "end_turn", etc.
StopReason string `json:"stop_reason,omitempty"`
// Usage contains token counts and timing (only for responses)
Usage *llm.Usage `json:"usage,omitempty"`
// Project is the git repository or project name that produced this node
Project string `json:"project,omitempty"`
// Kind is the semantic classification of the call (or injected
// block) that produced this node — 'main', 'offshoot:…',
// 'injected:…' per the design taxonomy. Derived metadata, NOT part
// of the content-addressed hash, and recomputable from the raw
// layer at any time.
Kind string `json:"node_kind,omitempty"`
// ThreadID is the harness sub-thread that fired the capturing
// call (Claude Code: the subagent agent-id), "" for main-thread
// calls. Captured deterministically at the wire; non-hashed.
ThreadID string `json:"thread_id,omitempty"`
// ParentToolUseID is the semantic fork/attach edge: the tool_use id
// this node's call relates to (a permission verdict points at the
// tool_use it judged; a subagent fork points at its Task tool_use).
// Derived, non-hashed, recomputable.
ParentToolUseID string `json:"parent_tool_use_id,omitempty"`
// Request carries the request-envelope parameters of the API call
// that captured this node (system prompt, max_tokens, temperature,
// stream, tool count). Like StopReason/Usage it is call metadata,
// NOT part of the content-addressed hash: the same logical turn
// keeps the same hash regardless of which kind of call sent it.
Request *llm.RequestParams `json:"request_params,omitempty"`
// CreatedAt is the time the node was persisted to storage. It is populated
// by the storage layer (not by NewNode) and is NOT part of the content hash.
// Zero value means "unknown" — typically for nodes constructed in-memory
// that have not yet been Put.
CreatedAt time.Time `json:"created_at,omitzero"`
}
Node represents a single content-addressed node in a Merkle DAG
func NewNode ¶
func NewNode(bucket Bucket, parent *Node, opts ...NodeOptions) *Node
NewNode creates a new node with the computed hash for the provided bucket. The optional NodeOptions parameter allows for setting metadata (StopReason, Usage, etc.) outside of the content addressable Bucket
func (*Node) CloneRetained ¶ added in v0.19.1
func (n *Node) CloneRetained(req *llm.RequestParams)
CloneRetained breaks every alias between this node's retained fields and the zero-copy raw request/response buffers it was parsed from, so those buffers can be freed once their turn is processed. It reallocates every retained string and byte field in place.
Cloning is all-or-nothing: a single surviving alias pins the whole backing array, so EVERY retained string/byte field must be copied here. Usage (a *Usage of integer counters parsed straight from the response with a standalone decoder) is measured not to pin its buffer, so it is left as-is — unlike RequestParams, whose scalars come from the provider request decoder and CAN sit in the buffer's arena (see RequestParams.Clone).
req replaces the node's Request. A parser shares one *RequestParams across every node of a single call, so the caller clones it once per call (RequestParams.Clone) and passes the result here, rather than re-cloning the (potentially large) system prompt for every node.
The copied bytes are identical to the originals, so node.Hash is unchanged — clone-on-retain is a pure memory optimization.
type NodeOptions ¶ added in v0.4.0
type NodeOptions struct {
StopReason string
Usage *llm.Usage
Project string
Request *llm.RequestParams
}
NodeOptions contains optional metadata for a new node that is stored but does not affect the content-addressable hashing.
Source Files