iterator

package
v1.32.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 11, 2026 License: GPL-3.0 Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const MaxEpochsPerBatch = 64

MaxEpochsPerBatch bounds iterator.backfill.epochsPerBatch: a batch is held in memory until it is written.

Variables

View Source 
var (
	ErrLocationUpToDate = errors.New("location up to date")
)

Functions

func DeriveAndSendEpochs 

func DeriveAndSendEpochs(
	ctx context.Context,
	epochs []phase0.Epoch,
	workers int,
	derive EpochDeriveFunc,
	send func(ctx context.Context, events []*xatu.DecoratedEvent) error,
) error

DeriveAndSendEpochs derives and sends every epoch of a step. With one worker it derives them all and sends them in one call, as DeriveEpochs does; with more, up to workers epochs are derived and sent at once, each as soon as it is derived. Either way the caller only moves the marker once it returns nil.

func DeriveEpochs 

func DeriveEpochs(ctx context.Context, epochs []phase0.Epoch, derive EpochDeriveFunc) ([]*xatu.DecoratedEvent, error)

DeriveEpochs derives every epoch in order and returns all their events, or none if any epoch fails.

func FailedEpoch 

func FailedEpoch(err error, fallback phase0.Epoch) phase0.Epoch

FailedEpoch is the epoch a DeriveEpochs error came from, or fallback for a single-epoch step.

Types

type BackFillingBlockNextResponse added in v1.18.0 

type BackFillingBlockNextResponse struct {
	From      uint64
	To        uint64
	Direction BackfillingBlockDirection
}

BackFillingBlockNextResponse is an inclusive block range to process.

type BackFillingCheckpointNextResponse added in v0.0.179 

type BackFillingCheckpointNextResponse struct {
	Next       phase0.Epoch
	LookAheads []phase0.Epoch
	Direction  BackfillingCheckpointDirection
	// contains filtered or unexported fields
}

func (*BackFillingCheckpointNextResponse) Epochs 

func (r *BackFillingCheckpointNextResponse) Epochs() []phase0.Epoch

Epochs returns the step's epochs in processing order: Next alone for head steps, up to epochsPerBatch walking down for backfill.

func (*BackFillingCheckpointNextResponse) LastEpoch 

func (r *BackFillingCheckpointNextResponse) LastEpoch() phase0.Epoch

LastEpoch is the epoch to pass to UpdateLocation, only after every epoch of the step is durably written.

type BackfillingBlock added in v1.18.0 

type BackfillingBlock struct {
	// contains filtered or unexported fields
}

BackfillingBlock walks execution-block-number space, gated on the consensus layer: the head frontier never runs past the execution block embedded in the CL-finalized beacon block. Backfill walks immutable history down to a floor.

func NewBackfillingBlock added in v1.18.0 

func NewBackfillingBlock(
	log observability.ContextualLogger, networkName, networkID string,
	cannonType xatu.CannonType,
	coordinatorClient *coordinator.Client,
	wallclock *ethwallclock.EthereumBeaconChain,
	metrics *BackfillingBlockMetrics,
	beacon CeilingBeaconNode,
	config *BackfillingBlockConfig,
) *BackfillingBlock

NewBackfillingBlock creates an EL cannon block iterator.

func (*BackfillingBlock) GetMarker added in v1.18.0 

func (b *BackfillingBlock) GetMarker(location *xatu.CannonLocation) (*xatu.BackfillingBlockMarker, error)

GetMarker extracts the block marker from a location.

func (*BackfillingBlock) Next added in v1.18.0 

func (b *BackfillingBlock) Next(ctx context.Context) (*BackFillingBlockNextResponse, error)

Next returns the next inclusive block range to process. It blocks (sleeping until the next epoch) when caught up to the CL-finalized ceiling and backfill is complete.

func (*BackfillingBlock) Start added in v1.18.0 

func (b *BackfillingBlock) Start(ctx context.Context) error

Start logs the iterator configuration.

func (*BackfillingBlock) UpdateLocation added in v1.18.0 

func (b *BackfillingBlock) UpdateLocation(ctx context.Context, from, to uint64, direction BackfillingBlockDirection) error

UpdateLocation advances the cursor after a range has been processed. Head advances finalized_block to the top of the range; backfill lowers backfill_block to the bottom of the range.

type BackfillingBlockConfig added in v1.18.0 

type BackfillingBlockConfig struct {
	// MaxRangeSize is the maximum number of blocks returned by a single Next().
	MaxRangeSize uint64 `yaml:"maxRangeSize" default:"50"`
	Backfill     struct {
		Enabled bool   `yaml:"enabled" default:"false"`
		ToBlock uint64 `yaml:"toBlock" default:"0"`
	} `yaml:"backfill"`
}

BackfillingBlockConfig configures an EL cannon block iterator.

type BackfillingBlockDirection added in v1.18.0 

type BackfillingBlockDirection string

BackfillingBlockDirection is the direction an EL cannon iterator is moving. 

const (
	BackfillingBlockDirectionHead     BackfillingBlockDirection = "head"
	BackfillingBlockDirectionBackfill BackfillingBlockDirection = "backfill"
)

type BackfillingBlockMetrics added in v1.18.0 

type BackfillingBlockMetrics struct {
	BackfillBlock  *prometheus.GaugeVec
	FinalizedBlock *prometheus.GaugeVec
	CeilingBlock   *prometheus.GaugeVec
	Lag            *prometheus.GaugeVec
}

BackfillingBlockMetrics exposes the position of an EL cannon block iterator.

func NewBackfillingBlockMetrics added in v1.18.0 

func NewBackfillingBlockMetrics(namespace string) BackfillingBlockMetrics

NewBackfillingBlockMetrics registers and returns the block iterator metrics.

func (*BackfillingBlockMetrics) SetBackfillBlock added in v1.18.0 

func (s *BackfillingBlockMetrics) SetBackfillBlock(cannonType, network string, block float64)

SetBackfillBlock records the backfill position.

func (*BackfillingBlockMetrics) SetCeilingBlock added in v1.18.0 

func (s *BackfillingBlockMetrics) SetCeilingBlock(network string, block float64)

SetCeilingBlock records the CL-finalized execution block ceiling.

func (*BackfillingBlockMetrics) SetFinalizedBlock added in v1.18.0 

func (s *BackfillingBlockMetrics) SetFinalizedBlock(cannonType, network string, block float64)

SetFinalizedBlock records the head position.

func (*BackfillingBlockMetrics) SetLag added in v1.18.0 

func (s *BackfillingBlockMetrics) SetLag(cannonType, network string, direction BackfillingBlockDirection, lag float64)

SetLag records the iterator lag in blocks for a direction.

type BackfillingCheckpoint added in v0.0.136 

type BackfillingCheckpoint struct {
	// contains filtered or unexported fields
}

func NewBackfillingCheckpoint added in v0.0.136 

func NewBackfillingCheckpoint(
	log observability.ContextualLogger, networkName, networkID string,
	cannonType xatu.CannonType,
	coordinatorClient *coordinator.Client,
	wallclock *ethwallclock.EthereumBeaconChain,
	metrics *BackfillingCheckpointMetrics,
	beacon *ethereum.BeaconNode,
	checkpoint string,
	lookAheadDistance int,
	config *BackfillingCheckpointConfig,
) *BackfillingCheckpoint

func (*BackfillingCheckpoint) GetMarker added in v0.0.179 

func (c *BackfillingCheckpoint) GetMarker(location *xatu.CannonLocation) (*xatu.BackfillingCheckpointMarker, error)

func (*BackfillingCheckpoint) Next added in v0.0.136 

func (c *BackfillingCheckpoint) Next(ctx context.Context) (rsp *BackFillingCheckpointNextResponse, err error)

func (*BackfillingCheckpoint) Start added in v0.0.179 

func (c *BackfillingCheckpoint) Start(ctx context.Context, activationFork spec.DataVersion) error

func (*BackfillingCheckpoint) StartEarly 

func (c *BackfillingCheckpoint) StartEarly(ctx context.Context, activationFork spec.DataVersion, epochsBeforeFork phase0.Epoch) error

StartEarly starts the iterator for a type whose data starts epochsBeforeFork epochs before its activation fork.

func (*BackfillingCheckpoint) StartUntil 

func (c *BackfillingCheckpoint) StartUntil(ctx context.Context, activationFork, endFork spec.DataVersion) error

StartUntil starts the iterator for a type that only exists before endFork.

func (*BackfillingCheckpoint) UpdateLocation added in v0.0.136 

func (c *BackfillingCheckpoint) UpdateLocation(ctx context.Context, epoch phase0.Epoch, direction BackfillingCheckpointDirection) error

UpdateLocation records epoch as processed in direction; callers only call it once the epoch's events are written.

type BackfillingCheckpointConfig added in v1.0.8 

type BackfillingCheckpointConfig struct {
	Backfill struct {
		Enabled bool         `yaml:"enabled" default:"false"`
		ToEpoch phase0.Epoch `yaml:"toEpoch" default:"0"`
		// EpochsPerBatch is how many epochs one backfill step derives and writes before the marker advances.
		EpochsPerBatch uint64 `yaml:"epochsPerBatch" default:"1"`
	} `yaml:"backfill"`
}

func (*BackfillingCheckpointConfig) Validate 

func (c *BackfillingCheckpointConfig) Validate() error

type BackfillingCheckpointDirection added in v0.0.179 

type BackfillingCheckpointDirection string
const (
	BackfillingCheckpointDirectionBackfill BackfillingCheckpointDirection = "backfill"
	BackfillingCheckpointDirectionHead     BackfillingCheckpointDirection = "head"
)

type BackfillingCheckpointMetrics added in v0.0.136 

type BackfillingCheckpointMetrics struct {
	BackfillEpoch            *prometheus.GaugeVec
	FinalizedEpoch           *prometheus.GaugeVec
	FinalizedCheckpointEpoch *prometheus.GaugeVec
	Lag                      *prometheus.GaugeVec
	BackfillBatchFallbacks   *prometheus.CounterVec
}

func NewBackfillingCheckpointMetrics added in v0.0.136 

func NewBackfillingCheckpointMetrics(namespace string) BackfillingCheckpointMetrics

func (*BackfillingCheckpointMetrics) IncBackfillBatchFallbacks 

func (s *BackfillingCheckpointMetrics) IncBackfillBatchFallbacks(cannonType, network string)

func (*BackfillingCheckpointMetrics) SetBackfillEpoch added in v0.0.136 

func (s *BackfillingCheckpointMetrics) SetBackfillEpoch(cannonType, network, checkpoint string, epoch float64)

func (*BackfillingCheckpointMetrics) SetFinalizedCheckpointEpoch added in v0.0.179 

func (s *BackfillingCheckpointMetrics) SetFinalizedCheckpointEpoch(network string, epoch float64)

func (*BackfillingCheckpointMetrics) SetFinalizedEpoch added in v0.0.136 

func (s *BackfillingCheckpointMetrics) SetFinalizedEpoch(cannonType, network, checkpoint string, epoch float64)

func (*BackfillingCheckpointMetrics) SetLag added in v0.0.179 

func (s *BackfillingCheckpointMetrics) SetLag(cannonType, network string, direction BackfillingCheckpointDirection, lag float64)

type CeilingBeaconNode 

type CeilingBeaconNode interface {
	// Finality returns the current finality checkpoints.
	Finality() (*apiv1.Finality, error)
	// GetBeaconBlock returns a beacon block by an immutable identifier
	// (slot or block root). Results are cached by identifier, so moving
	// aliases such as "finalized" or "head" must never be passed here.
	GetBeaconBlock(ctx context.Context, identifier string, ignoreMetrics ...bool) (*spec.VersionedSignedBeaconBlock, error)
	// GetExecutionPayloadEnvelope returns a Gloas block's execution payload
	// envelope, or nil when the builder withheld the payload.
	GetExecutionPayloadEnvelope(ctx context.Context, blockID string) (*gloas.SignedExecutionPayloadEnvelope, error)
}

CeilingBeaconNode is the subset of *ethereum.BeaconNode the EL block iterator needs to resolve its head ceiling.

type EpochDeriveFunc 

type EpochDeriveFunc func(ctx context.Context, epoch phase0.Epoch) ([]*xatu.DecoratedEvent, error)

EpochDeriveFunc derives the events of one epoch.

type EpochError 

type EpochError struct {
	Epoch phase0.Epoch
	Err   error
}

EpochError is a failure to derive one epoch of a multi-epoch step.

func (*EpochError) Error 

func (e *EpochError) Error() string

func (*EpochError) Unwrap 

func (e *EpochError) Unwrap() error

type SlotMetrics 

type SlotMetrics struct {
	TrailingSlots *prometheus.GaugeVec
	CurrentSlot   *prometheus.GaugeVec
}

func NewSlotMetrics 

func NewSlotMetrics(namespace string) SlotMetrics

func (*SlotMetrics) SetCurrentSlot 

func (s *SlotMetrics) SetCurrentSlot(cannonType, network string, currentSlot float64)

func (*SlotMetrics) SetTrailingSlots 

func (s *SlotMetrics) SetTrailingSlots(cannonType, network string, trailingSlots float64)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.