parallel

package
v1.99.25 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 18, 2026 License: GPL-3.0, LGPL-3.0, LGPL-3.0 Imports: 10 Imported by: 0

Documentation

Overview

Package parallel defines interfaces for optional parallel block execution and GPU acceleration in the Lux EVM.

By default (no build tags), a sequential executor is used. When built with -tags parallel, the evmgpu Block-STM engine is linked. When built with -tags parallel,gpu, GPU-accelerated hashing and ecrecover are also enabled.

This package lives in lux/evm so that the state processor can call into it without importing evmgpu. The evmgpu package provides the real implementations that satisfy these interfaces.

Package parallel defines interfaces for optional parallel block execution and GPU acceleration in the Lux EVM.

No build tags required. GPU acceleration is auto-detected at init time:

  • darwin + CGo: Metal GPU via gpu_bridge.go
  • linux + CGo + CUDA: NVIDIA GPU (future)
  • otherwise: CPU sequential (zero overhead)

The registration pattern allows platform-specific init() functions to register GPU backends without import cycles.

Index

Constants

View Source
const DChainSettlementHook = "" /* 133-byte string literal not displayed */

DChainSettlementHook documents the boundary between this package (settlement on the reused Block-STM engine) and dex/pkg/dchain (the matcher + ledger). This is the documented adapter seam: the d-chain VM owns the other side and is out of scope for this track (another track owns dex/pkg/dchain).

On the d-chain side, Block.Verify already matches orders against a versiondb overlay (see dchain-vm-design). To settle the resulting fills in parallel:

  1. After matching, build []Fill from the matched trades: fills[k] = Fill{ Maker: trade.MakerOwner, Taker: trade.TakerOwner, Market: poolAddr, Index: k, // matcher sequence == deterministic order Apply: func(sdb *state.StateDB, f Fill) error { // debit/credit both legs in the d-chain ledger overlay, // update market fee/oracle slot; fixed-point (Q64.64), // never float -- the determinism rule from dchain-vm-design }, }
  2. Settle: conflicts, err := NewSettlementExecutor(0).SettleBatch(overlay, fills)
  3. Optionally assert CPU==GPU edges at scale: cpuEdges := ConflictEdges(fills) gpuEdges := <gpu-kernels op_cevm_conflict_detect over the same rwSets> require cpuEdges == gpuEdges // the consensus-relevant invariant
  4. Block.Accept commits the overlay -> zapdb atomically (unchanged).

The hook is deliberately a doc, not code: wiring it lives in dex/pkg/dchain, which this track must not modify.

Variables

This section is empty.

Functions

func ConflictEdges added in v1.99.22

func ConflictEdges(fills []Fill) [][2]int

ConflictEdges returns the GPU-equivalent conflict edge list for a fill batch (canonical (lo,hi) ascending). Exposed so the d-chain side can assert CPU==GPU by diffing this against the GPU op_cevm_conflict_detect output over the same footprints, and to drive a sharded merge.

func IsGPUAvailable added in v0.8.50

func IsGPUAvailable() bool

IsGPUAvailable returns true if a GPU backend was detected at startup.

func RegisterExecutor

func RegisterExecutor(e BlockExecutor)

RegisterExecutor sets the parallel block executor.

func RegisterGPU

func RegisterGPU(g GPUAccelerator)

RegisterGPU sets the GPU accelerator.

func RegisterTransactionExecutor added in v0.8.46

func RegisterTransactionExecutor(backend EVMBackend, e TransactionExecutor)

RegisterTransactionExecutor registers a per-tx executor for a given backend. Call from init() in backend packages (e.g., revmbackend, cevmbackend).

func SelectedGPU added in v0.8.50

func SelectedGPU() string

SelectedGPU returns the detected GPU name ("Metal", "CUDA", "none").

func SetBackend added in v0.8.46

func SetBackend(backend EVMBackend)

SetBackend selects the active EVM backend. Use AutoEVM to select the best available.

Types

type BlockExecutor

type BlockExecutor interface {
	// ExecuteBlock processes all transactions in a block.
	// Returns receipts in original transaction order, or an error.
	// A nil return (nil, nil) means "not handled, fall through to sequential."
	ExecuteBlock(
		config *ethparams.ChainConfig,
		header *types.Header,
		txs types.Transactions,
		statedb *state.StateDB,
		vmCfg vm.Config,
	) ([]*types.Receipt, error)
}

BlockExecutor processes all transactions in a block. The default implementation delegates to sequential per-tx execution. The parallel implementation uses Block-STM speculative execution.

func DefaultExecutor

func DefaultExecutor() BlockExecutor

DefaultExecutor returns the registered parallel executor, or a no-op sequential fallback if none was registered.

type BlockSTMExecutor added in v0.8.46

type BlockSTMExecutor struct {
	// contains filtered or unexported fields
}

BlockSTMExecutor implements BlockExecutor using the Block-STM algorithm. It executes all transactions in a block speculatively in parallel, detects read-write conflicts, and re-executes conflicting transactions.

func NewBlockSTMExecutor added in v0.8.46

func NewBlockSTMExecutor(workers int, applyFn TxApplyFunc) *BlockSTMExecutor

NewBlockSTMExecutor creates a Block-STM executor. Pass 0 for workers to use runtime.NumCPU(). applyFn is the function that executes a single tx against a StateDB.

func (*BlockSTMExecutor) ExecuteBlock added in v0.8.46

func (e *BlockSTMExecutor) ExecuteBlock(
	config *ethparams.ChainConfig,
	header *types.Header,
	txs types.Transactions,
	statedb *state.StateDB,
	vmCfg vm.Config,
) ([]*types.Receipt, error)

ExecuteBlock processes all transactions in the block using Block-STM.

Returns (nil, nil) when:

  • The block has fewer than 2 transactions (no benefit from parallelism).
  • No apply function was configured.

type EVMBackend added in v0.8.46

type EVMBackend string

EVMBackend identifies which EVM implementation to use.

const (
	// GoEVM is the default Go EVM from luxfi/geth (geth interpreter).
	GoEVM EVMBackend = "gevm"

	// RustEVM uses revm (Rust EVM) via FFI for execution.
	// Native Block-STM parallel execution, memory-safe.
	RustEVM EVMBackend = "revm"

	// CppEVM is the Lux C++ EVM via CGo.
	// Fastest single-threaded interpreter, SIMD opcodes, GPU kernel dispatch.
	CppEVM EVMBackend = "cevm"

	// AutoEVM selects the best available backend at runtime.
	AutoEVM EVMBackend = "auto"
)

func ActiveBackend added in v0.8.46

func ActiveBackend() EVMBackend

ActiveBackend returns the currently selected EVM backend.

func AvailableBackends added in v0.8.46

func AvailableBackends() []EVMBackend

AvailableBackends returns all registered backend names.

func SelectedBackend added in v0.8.50

func SelectedBackend() EVMBackend

SelectedBackend returns the auto-detected backend for metrics/observability.

type Fill added in v1.99.22

type Fill struct {
	// Maker and Taker are the two accounts whose balances move. For a spot
	// trade these are the resting-order owner and the incoming-order owner.
	Maker common.Address
	Taker common.Address

	// Market identifies the order book (e.g. the pool/pair address). Two fills
	// in the same market that adjust shared market-level state (fee accumulator,
	// last-price oracle slot) conflict; fills in different markets do not.
	Market common.Address

	// Index is the fill's position in the deterministic settlement order
	// (matcher sequence). Commit and re-execution honour ascending Index.
	Index int

	// Apply settles this fill against the state. Return an error to abort the
	// batch (e.g. insufficient balance that the matcher should have caught).
	Apply FillApplyFunc
}

Fill is one matched DEX fill to settle. It is a value: the accounts and market it touches (its conflict footprint) plus the index that fixes its deterministic commit order (the matcher's sequence number within the batch).

Apply performs the actual balance mutation against the supplied state. It is injected so this package never needs to know asset/ledger internals -- the same inversion TxApplyFunc uses for the EVM path. Apply MUST be a pure function of the state it is given (no shared mutable capture) so speculative execution on isolated copies is sound.

type FillApplyFunc added in v1.99.22

type FillApplyFunc func(statedb *state.StateDB, f Fill) error

FillApplyFunc settles a single fill against a state copy. The d-chain matcher supplies this; it debits/credits the two accounts and updates market state.

type GPUAccelerator

type GPUAccelerator interface {
	// Available reports whether a GPU backend is detected.
	Available() bool

	// BatchEcrecover recovers sender addresses for a batch of transactions.
	// Returns a map from tx hash to recovered sender address.
	BatchEcrecover(txs []*types.Transaction) (map[common.Hash]common.Address, error)

	// BatchKeccak hashes multiple inputs on GPU.
	BatchKeccak(inputs [][]byte) ([]common.Hash, error)
}

GPUAccelerator provides optional GPU-offloaded crypto operations. The default implementation returns Available() == false.

func DefaultGPU

func DefaultGPU() GPUAccelerator

DefaultGPU returns the registered GPU accelerator, or a no-op if none was registered.

type Metrics added in v0.8.46

type Metrics struct {
	BlocksProcessed atomic.Int64
	TxsProcessed    atomic.Int64
	TxsReExecuted   atomic.Int64
}

Metrics exposes Block-STM conflict statistics for observability.

var DefaultMetrics Metrics

DefaultMetrics is the global Block-STM metrics instance.

type SettlementExecutor added in v1.99.22

type SettlementExecutor struct {
	// contains filtered or unexported fields
}

SettlementExecutor settles batches of DEX fills. On a single state it applies in deterministic order with cheap conflict detection (SettleBatch); across independent state shards it settles concurrently (SettleSharded) -- the latter is where DEX-settlement parallelism actually lives (geth StateDB cannot be mutated concurrently even for disjoint accounts; see SettleBatch).

func NewSettlementExecutor added in v1.99.22

func NewSettlementExecutor(workers int) *SettlementExecutor

NewSettlementExecutor creates a settlement executor. Pass 0 for workers to use runtime.NumCPU().

func (*SettlementExecutor) SettleBatch added in v1.99.22

func (e *SettlementExecutor) SettleBatch(base *state.StateDB, fills []Fill) (int, error)

SettleBatch settles fills against a single state in deterministic index order and returns the conflict count (for observability). It does NOT copy state per fill: a DEX fill is a ~3-account balance mutation (sub-microsecond), so the optimistic copy-the-world-per-item model the EVM block path uses (where each tx is heavy enough to amortise a StateDB.Copy) is catastrophically wrong for fine-grained fills -- measured 311x SLOWER than sequential on this host.

MEASURED FINDING (M1 Max, 10k fills): per-fill StateDB.Copy() => 750 fills/s (70 GB allocated); sequential apply on the shared state => 233k fills/s. The conflict-detection kernel itself is cheap (2.2M fills/s) -- so the right shape is "detect cheaply, then settle without per-item world-copies".

Single-state parallelism is impossible here: geth's StateDB.stateObjects is a plain map mutated even on reads (getOrNewStateObject), so concurrent goroutines touching DISJOINT accounts still race the map and panic. The parallelism for DEX settlement therefore lives ONE level up -- across independent state shards (SettleSharded), which is also the LAN / multi-node story. On one state, the correct, fastest path is the deterministic sequential apply this method does.

fills MUST be ordered by their matcher sequence; the order is preserved so the committed state is identical for every validator.

func (*SettlementExecutor) SettleSharded added in v1.99.22

func (e *SettlementExecutor) SettleSharded(
	base *state.StateDB,
	shardStates []*state.StateDB,
	fills []Fill,
) (ShardResult, error)

SettleSharded settles a fill batch by partitioning accounts across N independent state shards and settling each shard CONCURRENTLY. This is the real parallelism for DEX settlement: each shard is its own *state.StateDB (no shared map), so the goroutines never race. It is also the in-process model of the LAN multi-node settlement -- a node == a shard, the merge == cross-shard reconciliation.

Contract:

  • shardStates[i] is an independent state holding the accounts that trade ONLY in markets assigned to shard i; the caller seeds each such account into that shard, and cross-shard accounts into base (see seedSharded).
  • base is the authoritative state for fills touching a cross-shard account (an account that trades in markets on >1 shard) and the merge target. After this returns, the canonical post-batch state = the union of the shard states + the cross-shard fills applied to base, exactly as a single sequential settlement would produce (TestSettleShardedEqualsSequential).

Partition: fills are sharded BY MARKET (fillShard), so a fill is always intra-shard for its market. The hazard is an ACCOUNT trading across markets that landed on different shards -- two shard goroutines would mutate it concurrently. SettleSharded detects those cross-shard accounts up front and routes every fill touching one to the serial base tail, so the concurrent shard phase only ever touches shard-local accounts -- race-free WITHOUT locks.

Determinism: a shard's fills apply in ascending Index; shards are disjoint by construction so their relative order is irrelevant; the cross-shard tail applies last in ascending Index. Every validator runs the same shardOf, so the partition and the final state are identical everywhere.

type ShardResult added in v1.99.22

type ShardResult struct {
	Shards        int // number of intra-shard states settled concurrently
	IntraShard    int // fills settled inside a single shard (parallel)
	CrossShard    int // fills touching >1 shard (settled serially after)
	Conflicts     int // total conflict edges across all fills (consensus artefact)
	CrossShardCfl int // conflicts among the cross-shard fills
}

ShardResult reports a sharded settlement outcome.

type TransactionExecutor added in v0.8.46

type TransactionExecutor interface {
	// Backend returns which EVM implementation this executor uses.
	Backend() EVMBackend

	// ExecuteTransaction runs a single transaction against the state.
	// Returns the execution result or error.
	ExecuteTransaction(
		config *ethparams.ChainConfig,
		header *types.Header,
		tx *types.Transaction,
		statedb *state.StateDB,
		vmCfg vm.Config,
		gasPool uint64,
	) (*types.Receipt, error)

	// SupportsParallel returns true if this backend can execute
	// transactions in parallel (e.g., Block-STM in revm, GPU in cevm).
	SupportsParallel() bool

	// SupportsGPU returns true if this backend can offload computation
	// to GPU (e.g., evmone with CUDA/Metal kernel dispatch).
	SupportsGPU() bool
}

TransactionExecutor processes a single transaction against a state. This is the per-tx abstraction point for swappable EVM backends.

Backends implement this to replace the default Go EVM interpreter:

  • GoEVM: delegates to geth's vm.EVM.Call()/Create() (default)
  • RustEVM: calls revm via FFI (faster single-thread, native Block-STM)
  • CppEVM: calls evmone via CGo (fastest interpreter, GPU offload)

The StateDB interface is the bridge — all backends read/write state through the same Go StateDB, ensuring consensus compatibility.

func DefaultTransactionExecutor added in v0.8.46

func DefaultTransactionExecutor() TransactionExecutor

DefaultTransactionExecutor returns the tx executor for the active backend.

type TxApplyFunc added in v0.8.46

type TxApplyFunc func(
	config *ethparams.ChainConfig,
	header *types.Header,
	tx *types.Transaction,
	statedb *state.StateDB,
	vmCfg vm.Config,
	txIndex int,
) (*types.Receipt, error)

TxApplyFunc executes a single transaction against the given state and returns the receipt. This is the injection point that breaks the circular dependency between core and parallel: core.applyTransaction satisfies this signature.

The function must call statedb.SetTxContext before execution.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL