shadow

package
v0.0.59 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 18, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Overview

Package shadow provides block-by-block comparison between a shadow chain node and a canonical chain node. The comparison is layered:

  • Layer 0: Block header hashes (AppHash, LastResultsHash, gas). If these match, the block is identical and deeper layers are skipped.
  • Layer 1: Transaction receipt comparison (status, gas, logs, etc.). Run only when Layer 0 fails, to isolate which transactions diverged.
  • Layer 2: State diff comparison — logical EVM state (storage/code/nonce) for the keys a block touched, read via EVM RPC on both sides. The load-bearing check for an AppHash-breaking migration shadow, where the committed root diverges by design and only logical state can be compared.
  • Layer 3: Execution trace comparison (future).

Index

Constants

This section is empty.

Variables

View Source
var (
	// BlocksCompared counts blocks the comparator has processed.
	// rate(...)==0 indicates the comparator has stopped advancing — typically
	// shadow RPC unreachable or the local node has stopped producing blocks.
	// pod_name differentiates two shadow candidates for the same chain so
	// alerts can route to a specific candidate image.
	BlocksCompared = prometheus.NewCounterVec(
		prometheus.CounterOpts{
			Name: "seictl_shadow_blocks_compared_total",
			Help: "Total blocks compared between the shadow node and the canonical chain.",
		},
		[]string{"chain_id", "pod_name"},
	)

	// Divergences counts app-hash divergences detected. Increments at most
	// once per process lifetime — the comparison loop exits on first divergence.
	// divergence_layer is "0" for header-hash mismatch, "1" when Layer 1
	// isolated specific tx-receipt mismatches.
	Divergences = prometheus.NewCounterVec(
		prometheus.CounterOpts{
			Name: "seictl_shadow_divergences_total",
			Help: "App-hash divergences detected by the shadow comparator. Increments once per process lifetime since the loop exits on first divergence.",
		},
		[]string{"chain_id", "pod_name", "divergence_layer"},
	)
)

Functions

func RenderMarkdown added in v0.0.25

func RenderMarkdown(r *DivergenceReport) string

RenderMarkdown produces a human-readable investigation report from a DivergenceReport. The output is designed to be consumed by engineers or LLM agents analyzing why a shadow node diverged from the canonical chain.

Types

type ChainSnapshot added in v0.0.25

type ChainSnapshot struct {
	Block        json.RawMessage `json:"block"`
	BlockResults json.RawMessage `json:"blockResults"`
}

ChainSnapshot captures the raw RPC responses from one chain at a specific height. The JSON is preserved verbatim for offline analysis.

type Comparator

type Comparator struct {
	// contains filtered or unexported fields
}

Comparator performs block-by-block comparison between a shadow node and a canonical chain node via their RPC endpoints.

func NewComparator

func NewComparator(shadowRPC, canonicalRPC string, opts ...Option) *Comparator

NewComparator creates a Comparator that queries shadowRPC for the local shadow node and canonicalRPC for the reference chain.

func (*Comparator) BuildDivergenceReport added in v0.0.25

func (c *Comparator) BuildDivergenceReport(ctx context.Context, height int64, comparison CompareResult) (*DivergenceReport, error)

BuildDivergenceReport captures a complete investigation artifact at the given height. It pairs the comparison result with the full raw RPC responses from both chains so engineers can diagnose offline.

func (*Comparator) Close added in v0.0.59

func (c *Comparator) Close()

Close releases resources held by configured Layer 2 readers / key source. go-ethereum's *ethclient.Client and *rpc.Client expose Close() with NO return, so they do not satisfy io.Closer — assert the no-return shape instead, or the connections leak silently.

func (*Comparator) CompareBlock

func (c *Comparator) CompareBlock(ctx context.Context, height int64) (*CompareResult, error)

CompareBlock performs a layered comparison for the given block height. Layer 0 (block headers) always runs. Layer 1 (transaction receipts) runs when a real divergence is detected, and always in migration mode — where AppHash, the cheap Layer 0 signal, is expected to differ, so the receipt check is the real correctness signal.

type CompareResult

type CompareResult struct {
	// Height is the block height that was compared.
	Height int64 `json:"height"`

	// Timestamp is the UTC time the comparison was performed.
	Timestamp string `json:"timestamp"`

	// Match is true when all checked layers agree between shadow and canonical.
	// In migration mode an expected AppHash divergence does not clear Match.
	Match bool `json:"match"`

	// MigrationMode records that this comparison treated AppHash divergence as
	// expected (an AppHash-breaking migration shadow), keying the verdict on
	// execution-results equivalence instead.
	MigrationMode bool `json:"migrationMode,omitempty"`

	// DivergenceLayer is the first layer that detected a mismatch.
	// Nil when Match is true.
	DivergenceLayer *int `json:"divergenceLayer,omitempty"`

	// Layer0 holds the block-level hash comparison. Always populated.
	Layer0 Layer0Result `json:"layer0"`

	// Layer1 holds the transaction receipt comparison.
	// Only populated when Layer 0 detected a divergence.
	Layer1 *Layer1Result `json:"layer1,omitempty"`

	// Layer2 holds the logical state-diff comparison. Populated only when a
	// state reader and key source are configured (see WithLayer2).
	Layer2 *Layer2Result `json:"layer2,omitempty"`
}

CompareResult holds the comparison output for a single block.

func (*CompareResult) Diverged

func (r *CompareResult) Diverged() bool

Diverged returns true when the comparison detected a mismatch at any layer.

type DivergenceReport added in v0.0.25

type DivergenceReport struct {
	Height     int64         `json:"height"`
	Timestamp  string        `json:"timestamp"`
	Comparison CompareResult `json:"comparison"`
	Shadow     ChainSnapshot `json:"shadow"`
	Canonical  ChainSnapshot `json:"canonical"`
}

DivergenceReport is a self-contained investigation artifact for a single app-hash divergence event. It includes the layered comparison result plus the full block and block_results from both chains, giving engineers all the context needed to diagnose why the shadow node diverged without querying external systems.

func FetchReport added in v0.0.30

func FetchReport(ctx context.Context, downloader seis3.Downloader, bucket, key string) (*DivergenceReport, error)

FetchReport downloads and decodes a DivergenceReport from S3.

type FieldDivergence

type FieldDivergence struct {
	Field     string `json:"field"`
	Shadow    any    `json:"shadow"`
	Canonical any    `json:"canonical"`
}

FieldDivergence records a single field-level mismatch in a tx receipt.

type KeySource added in v0.0.59

type KeySource interface {
	TouchedAccounts(ctx context.Context, height int64) ([]TouchedAccount, error)
}

KeySource yields the accounts (and their slots) a block touched, so Layer 2 compares exactly the state real transactions read or wrote at that height.

type Layer0Result

type Layer0Result struct {
	AppHashMatch         bool `json:"appHashMatch"`
	LastResultsHashMatch bool `json:"lastResultsHashMatch"`
	GasUsedMatch         bool `json:"gasUsedMatch"`

	// Raw values are included when there is a mismatch, for diagnostics.
	ShadowAppHash    string `json:"shadowAppHash,omitempty"`
	CanonicalAppHash string `json:"canonicalAppHash,omitempty"`

	ShadowLastResultsHash    string `json:"shadowLastResultsHash,omitempty"`
	CanonicalLastResultsHash string `json:"canonicalLastResultsHash,omitempty"`

	ShadowGasUsed    int64 `json:"shadowGasUsed,omitempty"`
	CanonicalGasUsed int64 `json:"canonicalGasUsed,omitempty"`
}

Layer0Result compares block-level hashes. This is the cheapest check; if all fields match, the block is identical and no further comparison is needed.

func (Layer0Result) Match

func (r Layer0Result) Match() bool

Match returns true when all Layer 0 fields agree.

type Layer1Result

type Layer1Result struct {
	// TotalTxs is the number of transactions in the block.
	TotalTxs int `json:"totalTxs"`

	// TxCountMatch is true when both chains have the same number of txs.
	TxCountMatch bool `json:"txCountMatch"`

	// Divergences lists the per-transaction differences found.
	Divergences []TxDivergence `json:"divergences,omitempty"`

	// Indeterminate is set when the receipt comparison could not run (RPC error).
	// In migration mode Layer 1 is a load-bearing check, so an indeterminate
	// Layer 1 forces the block to fail closed rather than pass silently.
	Indeterminate bool   `json:"indeterminate,omitempty"`
	Error         string `json:"error,omitempty"`
}

Layer1Result compares individual transaction receipts within a block. Only populated when Layer 0 fails.

type Layer2Result added in v0.0.59

type Layer2Result struct {
	// AccountsChecked is the number of touched accounts compared.
	AccountsChecked int `json:"accountsChecked"`

	// KeysChecked is the number of individual state reads compared (storage
	// slots plus per-account balance/code/nonce checks).
	KeysChecked int `json:"keysChecked"`

	// Divergences lists the logical-state mismatches found.
	Divergences []StateDivergence `json:"divergences,omitempty"`

	// Indeterminate is set when the layer could not be evaluated (a key source
	// or state read failed). A migration shadow's load-bearing check could not
	// run, so the block must NOT be counted clean — Error carries the cause.
	Indeterminate bool   `json:"indeterminate,omitempty"`
	Error         string `json:"error,omitempty"`
}

Layer2Result compares logical EVM state (storage slots, code, nonce) for the keys a block touched, read via EVM RPC on both chains. This is the logical- content truth check; it never compares the committed root, which is schedule-dependent for a migration shadow and so not a correctness oracle.

type Option added in v0.0.59

type Option func(*Comparator)

Option configures a Comparator.

func WithLayer2 added in v0.0.59

func WithLayer2(shadowState, canonicalState StateReader, keySource KeySource) Option

WithLayer2 enables logical state-diff comparison: the keySource yields the accounts/slots each block touched, and the two StateReaders (EVM RPC on the shadow and canonical chains) supply their logical values to compare.

func WithMigrationMode added in v0.0.59

func WithMigrationMode() Option

WithMigrationMode treats AppHash divergence as expected (not a mismatch) and keys the verdict on execution-results equivalence. Use for a shadow running an AppHash-breaking state migration against an un-migrated canonical chain.

type StateDivergence added in v0.0.59

type StateDivergence struct {
	Kind      string `json:"kind"` // storage | code | nonce
	Addr      string `json:"addr"`
	Slot      string `json:"slot,omitempty"` // set only for storage
	Shadow    string `json:"shadow"`
	Canonical string `json:"canonical"`
}

StateDivergence records a single logical-state mismatch between the shadow and canonical chains. Values are hex for legibility in reports.

type StateReader added in v0.0.59

type StateReader interface {
	StorageAt(ctx context.Context, account common.Address, key common.Hash, blockNumber *big.Int) ([]byte, error)
	CodeAt(ctx context.Context, account common.Address, blockNumber *big.Int) ([]byte, error)
	NonceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (uint64, error)
	BalanceAt(ctx context.Context, account common.Address, blockNumber *big.Int) (*big.Int, error)
}

StateReader reads logical EVM state at a height. Its method set matches go-ethereum's *ethclient.Client, so a real client satisfies it directly; the shadow and canonical sides are two instances. A nil blockNumber means latest.

type StaticKeySource added in v0.0.59

type StaticKeySource struct {
	Accounts []TouchedAccount
}

StaticKeySource compares a fixed, curated set of accounts on every block — the fallback when prestate tracing (debug_) is unavailable, and the hook for a periodic cold-key / hot-contract sweep that the trace source does not cover.

func (StaticKeySource) TouchedAccounts added in v0.0.59

func (s StaticKeySource) TouchedAccounts(_ context.Context, _ int64) ([]TouchedAccount, error)

type TouchedAccount added in v0.0.59

type TouchedAccount struct {
	Addr         common.Address
	Slots        []common.Hash
	CheckCode    bool
	CheckNonce   bool
	CheckBalance bool
}

TouchedAccount is the set of state a block touched for one address: the storage slots to compare, and whether the account's balance, code, and nonce should be checked. A KeySource produces these per block (e.g. from a trace).

type TraceKeySource added in v0.0.59

type TraceKeySource struct {
	// contains filtered or unexported fields
}

TraceKeySource derives a block's touched accounts and storage slots from a diff-mode prestate trace (debug_traceBlockByNumber, prestateTracer with diffMode) on an EVM JSON-RPC endpoint. diffMode reports both the pre-state (slots read) and post-state (slots written), so the union covers keys the block read OR modified — not just those read. Requires the debug_ namespace enabled on the endpoint (a non-public, operator-owned node).

Coverage boundary: this is the per-block TOUCHED set. Keys migrated but never touched by any transaction (cold state), and non-EVM Cosmos-module state, are not covered here — that breadth is the corpus harness's job (Arm A) plus a periodic StaticKeySource sweep. Layer 2 over a trace source is a hot-state sampling oracle, not a full-keyspace check.

func NewTraceKeySource added in v0.0.59

func NewTraceKeySource(evmRPC string) (*TraceKeySource, error)

NewTraceKeySource dials the EVM JSON-RPC endpoint used to fetch prestate traces. The same touched set applies to both chains (identical transactions), so a single endpoint (typically the canonical node) is sufficient.

func (*TraceKeySource) Close added in v0.0.59

func (t *TraceKeySource) Close()

Close releases the underlying RPC connection. No return value, matching go-ethereum's *ethclient.Client / *rpc.Client Close() so Comparator.Close can treat all closeables uniformly.

func (*TraceKeySource) TouchedAccounts added in v0.0.59

func (t *TraceKeySource) TouchedAccounts(ctx context.Context, height int64) ([]TouchedAccount, error)

type TxDivergence

type TxDivergence struct {
	// TxIndex is the position of the transaction within the block.
	TxIndex int `json:"txIndex"`

	// Fields lists which receipt fields diverged.
	Fields []FieldDivergence `json:"fields"`
}

TxDivergence records a mismatch for a single transaction within a block.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL