extended

package
v0.48.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 9, 2026 License: AGPL-3.0 Imports: 32 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// DefaultBackfillDelay defines the delay between consecutive backfill attempts.
	// Set carefully to avoid overwhelming the system.
	// With the default value of 5ms, the maximum catch-up rate is approximately
	// 200 blocks per second.
	DefaultBackfillDelay = 5 * time.Millisecond
)

Variables

View Source 
var (
	ErrAlreadyIndexed = errors.New("data already indexed for height")
	ErrFutureHeight   = errors.New("cannot index future height")
)

Functions

func DecodeTransactionDataResults added in v0.48.0 

func DecodeTransactionDataResults(response []byte, ids []uint64) (map[uint64]*access.ScheduledTransaction, error)

DecodeTransactionDataResults decodes the JSON-CDC response from a batch GetTransactionData script execution. The ids slice must match the order of IDs passed when the script was called.

Returns a map from scheduled transaction ID to decoded access.ScheduledTransaction. IDs for which the contract returned nil (not found on-chain) are omitted from the map. The returned entries have access.ScheduledTxStatusScheduled status, since TransactionData reflects the initially scheduled state.

Any error indicates that the response is malformed.

func EncodeGetTransactionDataArg added in v0.48.0 

func EncodeGetTransactionDataArg(ids []uint64) ([]byte, error)

EncodeGetTransactionDataArg encodes a slice of scheduled transaction IDs as a JSON-CDC [UInt64] array suitable for passing as the script argument when executing a script generated from [getTransactionDataScriptTemplate].

No error returns are expected during normal operation.

func GetTransactionDataScript added in v0.48.0 

func GetTransactionDataScript(chainID flow.ChainID) []byte

GetTransactionDataScript returns the Cadence script used for JIT scheduled transaction lookups on the given chain. Exposed for testing.

Types

type AccountTransactions 

type AccountTransactions struct {
	// contains filtered or unexported fields
}

AccountTransactions indexes account-transaction associations for a block.

func NewAccountTransactions 

func NewAccountTransactions(
	log zerolog.Logger,
	store storage.AccountTransactionsBootstrapper,
	chainID flow.ChainID,
	lockManager storage.LockManager,
) (*AccountTransactions, error)

func (*AccountTransactions) IndexBlockData 

func (a *AccountTransactions) IndexBlockData(lctx lockctx.Proof, data BlockData, batch storage.ReaderBatchWriter) error

IndexBlockData indexes the block data for the given height. If the header in `data` does not match the expected height, an error is returned.

The caller must hold the storage.LockIndexAccountTransactions lock until the batch is committed.

CAUTION: Not safe for concurrent use.

Expected error returns during normal operations:

func (*AccountTransactions) Name 

func (a *AccountTransactions) Name() string

Name returns the name of the indexer.

func (*AccountTransactions) NextHeight 

func (a *AccountTransactions) NextHeight() (uint64, error)

NextHeight returns the next height that the indexer will index.

No error returns are expected during normal operation.

func (*AccountTransactions) ProcessBlockData added in v0.48.0 

func (a *AccountTransactions) ProcessBlockData(data BlockData) ([]access.AccountTransaction, AccountTransactionsMetadata, error)

ProcessBlockData processes the block data and returns the indexed account transaction entries.

No error returns are expected during normal operation.

type AccountTransactionsMetadata added in v0.48.0 

type AccountTransactionsMetadata struct{}

type BlockData 

type BlockData struct {
	Header       *flow.Header
	Transactions []*flow.TransactionBody
	Events       []flow.Event
}

type Contracts added in v0.48.0 

type Contracts struct {
	// contains filtered or unexported fields
}

Contracts indexes contract deployment lifecycle events and writes to the contract deployments index. Handles flow.AccountContractAdded and flow.AccountContractUpdated events. flow.AccountContractRemoved is not currently permitted; encountering one returns an error.

On first bootstrapping, the indexer will load all deployed contracts from storage at the bootstrap height and include placeholder deployment objects for all contracts that are deployed at that height. Any contracts that are deployed in the same block as the bootstrap height will not have placeholder deployment objects created since there are already deployment objects for those contracts.

CAUTION: Not safe for concurrent use.

func NewContracts added in v0.48.0 

func NewContracts(
	log zerolog.Logger,
	store storage.ContractDeploymentsIndexBootstrapper,
	registers registerScanner,
	scriptExecutor snapshotProvider,
	metrics module.ExtendedIndexingMetrics,
) *Contracts

NewContracts creates a new Contracts indexer backed by store.

func (*Contracts) IndexBlockData added in v0.48.0 

func (c *Contracts) IndexBlockData(lctx lockctx.Proof, data BlockData, rw storage.ReaderBatchWriter) error

IndexBlockData processes one block's events and transactions and updates the contract deployments index.

The caller must hold the storage.LockIndexContractDeployments lock until the batch is committed.

CAUTION: Not safe for concurrent use.

Expected error returns during normal operations:

func (*Contracts) Name added in v0.48.0 

func (c *Contracts) Name() string

Name returns the name of this indexer.

func (*Contracts) NextHeight added in v0.48.0 

func (c *Contracts) NextHeight() (uint64, error)

NextHeight returns the next block height to index.

No error returns are expected during normal operation.

func (*Contracts) ProcessBlockData added in v0.48.0 

func (c *Contracts) ProcessBlockData(data BlockData) ([]access.ContractDeployment, ContractsMetadata, error)

ProcessBlockData processes the block data and returns the indexed contract deployment entries along with metadata containing the counts of created and updated contracts.

No error returns are expected during normal operation.

type ContractsMetadata added in v0.48.0 

type ContractsMetadata struct {
	Created int
	Updated int
}

ContractsMetadata contains indexing metrics for a single block's contract events.

type ExtendedIndexer 

type ExtendedIndexer struct {
	component.Component
	// contains filtered or unexported fields
}

ExtendedIndexer orchestrates indexing for all extended indexers.

Indexing is performed in a single-threaded loop, where each iteration indexes the next height for all indexers. Indexers are grouped by their next height to reduce database lookups. All data for each height is written to a batch and committed at once.

NOT CONCURRENCY SAFE.

func NewExtendedIndexer 

func NewExtendedIndexer(
	log zerolog.Logger,
	metrics module.ExtendedIndexingMetrics,
	db storage.DB,
	lockManager storage.LockManager,
	state protocol.State,
	index storage.Index,
	headers storage.Headers,
	guarantees storage.Guarantees,
	collections storage.Collections,
	events storage.Events,
	results storage.LightTransactionResults,
	indexers []Indexer,
	chainID flow.ChainID,
	backfillDelay time.Duration,
) (*ExtendedIndexer, error)

func (*ExtendedIndexer) IndexBlockData 

func (c *ExtendedIndexer) IndexBlockData(
	header *flow.Header,
	transactions []*flow.TransactionBody,
	events []flow.Event,
) error

IndexBlockData stores block data and exposes it to the indexers. It must be called sequentially, with blocks provided in strictly increasing height order.

Typically, this method is invoked when the latest block is received. If the indexer is fully caught up, this latest block will be the next one to process, and indexing it will advance the indexed height.

If the indexer is still catching up, however, the latest block is not immediately needed because the indexer must first process older blocks.

For this reason, we do not index the latest block right away. Instead, we cache it and notify the worker to proceed with the next job.

If the next job is to process the latest block, the cached c.latestBlockData will be used. Otherwise, if the job is to process older blocks, the cache is ignored and the worker fetches the required block data for indexing.

No error returns are expected during normal operation.

func (*ExtendedIndexer) IndexBlockExecutionData 

func (c *ExtendedIndexer) IndexBlockExecutionData(
	data *execution_data.BlockExecutionDataEntity,
) error

IndexBlockExecutionData captures the block data and makes it available to the indexers.

No error returns are expected during normal operation.

type FungibleTokenTransfers added in v0.48.0 

type FungibleTokenTransfers struct {
	// contains filtered or unexported fields
}

FungibleTokenTransfers indexes fungible token transfer events for a block.

func NewFungibleTokenTransfers added in v0.48.0 

func NewFungibleTokenTransfers(
	log zerolog.Logger,
	chainID flow.ChainID,
	ftStore storage.FungibleTokenTransfersBootstrapper,
	metrics module.ExtendedIndexingMetrics,
) *FungibleTokenTransfers

NewFungibleTokenTransfers creates a new FungibleTokenTransfers indexer.

func (*FungibleTokenTransfers) IndexBlockData added in v0.48.0 

func (a *FungibleTokenTransfers) IndexBlockData(lctx lockctx.Proof, data BlockData, batch storage.ReaderBatchWriter) error

IndexBlockData indexes FT transfer data for the given height. If the header in `data` does not match the expected height, an error is returned.

The caller must hold the storage.LockIndexFungibleTokenTransfers lock until the batch is committed.

CAUTION: Not safe for concurrent use.

Expected error returns during normal operations:

func (*FungibleTokenTransfers) Name added in v0.48.0 

func (a *FungibleTokenTransfers) Name() string

Name returns the name of the indexer.

func (*FungibleTokenTransfers) NextHeight added in v0.48.0 

func (a *FungibleTokenTransfers) NextHeight() (uint64, error)

NextHeight returns the next height that the indexer will index.

No error returns are expected during normal operation.

func (*FungibleTokenTransfers) ProcessBlockData added in v0.48.0 

func (a *FungibleTokenTransfers) ProcessBlockData(data BlockData) ([]access.FungibleTokenTransfer, FungibleTokenTransfersMetadata, error)

ProcessBlockData processes the block data and returns the indexed fungible token transfer entries.

No error returns are expected during normal operation.

type FungibleTokenTransfersMetadata added in v0.48.0 

type FungibleTokenTransfersMetadata struct {
	// FilteredCount is the number of transfers that were omitted.
	FilteredCount int
}

type IndexProcessor added in v0.48.0 

type IndexProcessor[T any, M any] interface {
	// ProcessBlockData processes the block data and returns the indexed data for the given type
	// along with indexer-specific metadata.
	//
	// No error returns are expected during normal operation.
	ProcessBlockData(data BlockData) ([]T, M, error)
}

IndexProcessor is a helper interface for indexers that need to process the block data and return indexed data for a specific type. The second type parameter M allows each indexer to return indexer-specific metadata alongside the indexed entries.

Safe for concurrent use.

type Indexer 

type Indexer interface {
	// Name returns the name of the indexer.
	Name() string

	// IndexBlockData indexes the block data for the given height.
	// If the header in `data` does not match the expected height, an error is returned.
	//
	// CAUTION: Not safe for concurrent use.
	//
	// Expected error returns during normal operations:
	//   - [ErrAlreadyIndexed]: if the data is already indexed for the height.
	//   - [ErrFutureHeight]: if the data is for a future height.
	IndexBlockData(lctx lockctx.Proof, data BlockData, batch storage.ReaderBatchWriter) error

	// NextHeight returns the next height that the indexer will index.
	//
	// No error returns are expected during normal operation.
	NextHeight() (uint64, error)
}

type IndexerManager 

type IndexerManager interface {
	// IndexBlockExecutionData indexes the block data for the given height.
	// If the header in `data` does not match the expected height, an error is returned.
	//
	// Not safe for concurrent use.
	//
	// No error returns are expected during normal operation.
	IndexBlockExecutionData(data *execution_data.BlockExecutionDataEntity) error

	// IndexBlockData indexes the block data for the given height.
	// If the header in `data` does not match the expected height, an error is returned.
	//
	// Not safe for concurrent use.
	//
	// No error returns are expected during normal operation.
	IndexBlockData(header *flow.Header, transactions []*flow.TransactionBody, events []flow.Event) error
}

IndexerManager orchestrates indexing for all extended indexers. It handles both indexing from the latest block submitted via the Index methods, and backfilling from storage.

type NonFungibleTokenTransfers added in v0.48.0 

type NonFungibleTokenTransfers struct {
	// contains filtered or unexported fields
}

NonFungibleTokenTransfers indexes non-fungible token transfer events for a block.

func NewNonFungibleTokenTransfers added in v0.48.0 

func NewNonFungibleTokenTransfers(
	log zerolog.Logger,
	chainID flow.ChainID,
	nftStore storage.NonFungibleTokenTransfersBootstrapper,
	metrics module.ExtendedIndexingMetrics,
) *NonFungibleTokenTransfers

NewNonFungibleTokenTransfers creates a new NonFungibleTokenTransfers indexer.

func (*NonFungibleTokenTransfers) IndexBlockData added in v0.48.0 

func (a *NonFungibleTokenTransfers) IndexBlockData(lctx lockctx.Proof, data BlockData, batch storage.ReaderBatchWriter) error

IndexBlockData indexes NFT transfer data for the given height. If the header in `data` does not match the expected height, an error is returned.

The caller must hold the storage.LockIndexNonFungibleTokenTransfers lock until the batch is committed.

CAUTION: Not safe for concurrent use.

Expected error returns during normal operations:

func (*NonFungibleTokenTransfers) Name added in v0.48.0 

func (a *NonFungibleTokenTransfers) Name() string

Name returns the name of the indexer.

func (*NonFungibleTokenTransfers) NextHeight added in v0.48.0 

func (a *NonFungibleTokenTransfers) NextHeight() (uint64, error)

NextHeight returns the next height that the indexer will index.

No error returns are expected during normal operation.

func (*NonFungibleTokenTransfers) ProcessBlockData added in v0.48.0 

func (a *NonFungibleTokenTransfers) ProcessBlockData(data BlockData) ([]access.NonFungibleTokenTransfer, NonFungibleTokenTransfersMetadata, error)

ProcessBlockData processes the block data and returns the indexed non-fungible token transfer entries.

No error returns are expected during normal operation.

type NonFungibleTokenTransfersMetadata added in v0.48.0 

type NonFungibleTokenTransfersMetadata struct{}

type ScheduledTransactionRequester added in v0.48.0 

type ScheduledTransactionRequester struct {
	// contains filtered or unexported fields
}

ScheduledTransactionRequester fetches scheduled transaction data from on-chain state by executing Cadence scripts against the FlowTransactionScheduler contract.

Not safe for concurrent use.

func NewScheduledTransactionRequester added in v0.48.0 

func NewScheduledTransactionRequester(executor scriptExecutor, chainID flow.ChainID) *ScheduledTransactionRequester

NewScheduledTransactionRequester creates a new ScheduledTransactionRequester.

func (*ScheduledTransactionRequester) Fetch added in v0.48.0 

func (r *ScheduledTransactionRequester) Fetch(
	ctx context.Context,
	lookupIDs []uint64,
	lookupHeight uint64,
	meta ScheduledTransactionsMetadata,
) ([]access.ScheduledTransaction, error)

Fetch fetches scheduled transaction data for the given IDs from on-chain state at lookupHeight, and applies the status updates from the collected block data.

No error returns are expected during normal operation.

type ScheduledTransactions added in v0.48.0 

type ScheduledTransactions struct {
	// contains filtered or unexported fields
}

ScheduledTransactions indexes scheduled transaction lifecycle events from the FlowTransactionScheduler system contract.

It processes Scheduled, PendingExecution, Executed, and Canceled events and writes corresponding entries to the scheduled transactions storage index.

A scheduled transaction that appeared in a PendingExecution event but has no matching Executed event in the same block is considered failed. The corresponding Flow transaction that was submitted by the scheduled executor account is identified by its authorizer (the scheduled executor account) and an empty payer address.

This indexer will automatically backfill any scheduled transactions that are executed or cancelled which were scheduled before the indexer was initialized. This is done by executing scripts when an unknown transaction is executed or cancelled within a block. There are a couple important considerations to keep in mind:

  1. If there are many unknown transactions with a block, the script execution may be slow and block the indexing process until it completes. Since the extended indexers are run in a batch, this will block all other indexers that are indexing the same block. In general, there should be relatively few unknown transactions executed. However, if this becomes a problem, we will need to consider a more efficient way to backfill the index.
  2. Since script executions are required to backfill the index, the indexer must be started after the registers db is initialized.

Not safe for concurrent use.

func NewScheduledTransactions added in v0.48.0 

func NewScheduledTransactions(
	log zerolog.Logger,
	store storage.ScheduledTransactionsIndexBootstrapper,
	scriptExecutor scriptExecutor,
	metrics module.ExtendedIndexingMetrics,
	chainID flow.ChainID,
) *ScheduledTransactions

NewScheduledTransactions creates a new ScheduledTransactions indexer.

No error returns are expected during normal operation.

func (*ScheduledTransactions) IndexBlockData added in v0.48.0 

func (s *ScheduledTransactions) IndexBlockData(lctx lockctx.Proof, data BlockData, rw storage.ReaderBatchWriter) error

IndexBlockData processes one block's events and transactions, and updates the scheduled transactions index.

The caller must hold the storage.LockIndexScheduledTransactionsIndex lock until the batch is committed.

CAUTION: Not safe for concurrent use.

Expected error returns during normal operations:

func (*ScheduledTransactions) Name added in v0.48.0 

func (s *ScheduledTransactions) Name() string

Name returns the indexer name.

func (*ScheduledTransactions) NextHeight added in v0.48.0 

func (s *ScheduledTransactions) NextHeight() (uint64, error)

NextHeight returns the next block height to index.

No error returns are expected during normal operation.

func (*ScheduledTransactions) ProcessBlockData added in v0.48.0 

func (s *ScheduledTransactions) ProcessBlockData(data BlockData) ([]access.ScheduledTransaction, ScheduledTransactionsMetadata, error)

ProcessBlockData processes the block data and returns event-derived metadata for the block's scheduled transaction lifecycle events.

The returned []access.ScheduledTransaction slice is always nil because all updates in the block cannot be represented by a single slice of objects. Instead, data is passed via ScheduledTransactionsMetadata, partitioned into their respective lifecycle events.

No error returns are expected during normal operation.

type ScheduledTransactionsMetadata added in v0.48.0 

type ScheduledTransactionsMetadata struct {
	NewTxs          []access.ScheduledTransaction
	ExecutedEntries []executedEntry
	CanceledEntries []canceledEntry
	FailedEntries   []failedEntry
}

ScheduledTransactionsMetadata collects all event-derived data for a single block's scheduled transaction lifecycle. It contains the newly scheduled transactions as well as the executed, canceled, and failed lifecycle entries.

Note: the complete indexed dataset is NOT available from [IndexProcessor.ProcessBlockData] alone because backfilling missing transactions requires storage and script execution. The full dataset is only assembled inside ScheduledTransactions.IndexBlockData.

Source Files

View all Source files

Directories

Path Synopsis
testutil
Package testutil provides shared event-building helpers for FT transfer tests.
 Package testutil provides shared event-building helpers for FT transfer tests.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.