Documentation
¶
Overview ¶
Package deferred holds the bulk-load deferred-index manifest.
The parent metadata package owns the DeferredIndexManager interface that callers type-assert against. This package owns the data: what to drop, what to rebuild, and the sync_state key that records crash-recovery state.
Index ¶
Constants ¶
const SyncStateKey = "metadata_indexes_pending"
SyncStateKey is the sync_state row that marks an in-flight (or interrupted) deferred-index drop/rebuild cycle. The value is the string "true" while the rebuild is outstanding and is removed once every manifest entry is present.
const SyncStateValue = "true"
SyncStateValue is the literal sync_state value written while a drop/rebuild cycle is outstanding.
Variables ¶
var Manifest = []Index{ { Name: "idx_utxo_payment_key", Table: "utxo", Columns: []string{"payment_key"}, Notes: "API address lookup", Critical: true, }, { Name: "idx_utxo_staking_key", Table: "utxo", Columns: []string{"staking_key"}, Notes: "API stake lookup", Critical: true, }, { Name: "idx_utxo_spent_at_tx_id", Table: "utxo", Columns: []string{"spent_at_tx_id"}, Notes: "Consumer transaction lookup and rollback repair", Critical: true, }, { Name: "idx_utxo_referenced_by_tx_id", Table: "utxo", Columns: []string{"referenced_by_tx_id"}, Notes: "Reference-input lookup and rollback repair", Critical: true, }, { Name: "idx_utxo_collateral_by_tx_id", Table: "utxo", Columns: []string{"collateral_by_tx_id"}, Notes: "Collateral lookup and rollback repair", Critical: true, }, { Name: "idx_utxo_added_slot", Table: "utxo", Columns: []string{"added_slot"}, Notes: "Rollback range scan", Critical: true, }, { Name: "idx_utxo_transaction_id", Table: "utxo", Columns: []string{"transaction_id"}, Notes: "Foreign-key reverse lookup", }, { Name: "idx_utxo_deleted_staking_amount", Table: "utxo", Columns: []string{ "deleted_slot", "credential_tag", "staking_key", "amount", }, Notes: "Primary UTxO RPC search path", Critical: true, }, { Name: "idx_utxo_deleted_payment_script", Table: "utxo", Columns: []string{"deleted_slot", "payment_script", "amount"}, Notes: "Script-locked supply", Critical: true, }, { Name: "idx_transaction_block_hash", Table: "transaction", Columns: []string{"block_hash"}, Notes: "Block transaction grouping", Critical: true, }, { Name: "idx_transaction_slot", Table: "transaction", Columns: []string{"slot"}, Notes: "Rollback and transaction history ordering", Critical: true, }, { Name: "idx_asset_name_hex", Table: "asset", Columns: []string{"name_hex"}, Notes: "Asset name lookup", }, { Name: "idx_asset_policy_id", Table: "asset", Columns: []string{"policy_id"}, Notes: "Policy lookup", Critical: true, }, { Name: "idx_asset_fingerprint", Table: "asset", Columns: []string{"fingerprint"}, Notes: "Fingerprint lookup", }, { Name: "idx_asset_amount", Table: "asset", Columns: []string{"amount"}, Notes: "Amount range scan", }, { Name: "idx_datum_added_slot", Table: "datum", Columns: []string{"added_slot"}, Notes: "Datum rollback scan", }, { Name: "idx_certs_block_hash", Table: "certs", Columns: []string{"block_hash"}, Notes: "Block certificate lookup", }, { Name: "idx_certs_certificate_id", Table: "certs", Columns: []string{"certificate_id"}, Notes: "Certificate reverse lookup", }, { Name: "idx_certs_slot", Table: "certs", Columns: []string{"slot"}, Notes: "Certificate rollback scan", Critical: true, }, { Name: "idx_certs_cert_type", Table: "certs", Columns: []string{"cert_type"}, Notes: "Certificate type filter", }, { Name: "idx_redeemer_transaction_id", Table: "redeemer", Columns: []string{"transaction_id"}, Notes: "Redeemer transaction lookup", }, { Name: "idx_redeemer_index", Table: "redeemer", Columns: []string{"index"}, Notes: "Redeemer index lookup", }, { Name: "idx_redeemer_tag", Table: "redeemer", Columns: []string{"tag"}, Notes: "Redeemer tag filter", }, { Name: "idx_key_witness_transaction_id", Table: "key_witness", Columns: []string{"transaction_id"}, Notes: "Witness transaction lookup", }, { Name: "idx_key_witness_type", Table: "key_witness", Columns: []string{"type"}, Notes: "Witness type filter", }, { Name: "idx_witness_scripts_script_hash", Table: "witness_scripts", Columns: []string{"script_hash"}, Notes: "Script hash lookup", }, { Name: "idx_witness_scripts_transaction_id", Table: "witness_scripts", Columns: []string{"transaction_id"}, Notes: "Script transaction lookup", }, { Name: "idx_witness_scripts_type", Table: "witness_scripts", Columns: []string{"type"}, Notes: "Script type filter", }, }
Manifest is the canonical list of metadata-store indexes that are dropped before bulk load and rebuilt before the database is marked ready.
The list is intentionally conservative: it targets the heaviest write paths (utxo, transaction, asset, datum, witness, certs/redeemer secondary indexes) where API backfill spends the bulk of its time.
Order matters at rebuild time only as a logging convenience; SQLite builds each index in a single statement and does not benefit from re-ordering.
Functions ¶
This section is empty.
Types ¶
type Index ¶
type Index struct {
// Name is the explicit index name in the versioned schema.
Name string
// Table is the SQL table name.
Table string
// Columns is the ordered SQL column list.
Columns []string
// Notes documents why this index is safe to defer. Surfaces
// in the manifest test failure message when the
// classification is questioned.
Notes string
// Critical marks indexes that must be present before the API
// can serve traffic. Critical indexes are rebuilt first so
// that the node can accept queries while the remaining lazy
// indexes finish in the background.
//
// Criteria for Critical=true:
// - Any WHERE predicate on the index column used by a live
// API query path (blockfrost, utxorpc, ledger queries).
// - Any WHERE predicate used by the rollback path
// (DeleteXAfterSlot), since rollbacks can occur as soon
// as live sync resumes.
//
// Everything else is lazy: FK reverse-lookups,
// witness/redeemer secondary indexes, and any column that is
// only SELECTed or SET but never filtered.
Critical bool
}
Index is one entry in the deferred-index manifest. Each entry names an index that is safe to drop while the database is in bulk-load mode (Mithril sync ledger-state import, immutable blob load, API-mode historical metadata backfill) and rebuild before the database is marked ready.
The manifest deliberately excludes:
- Primary keys (autoincrement IDs).
- Unique indexes that back ON CONFLICT clauses used during import (e.g. utxo.tx_id_output_idx, transaction.hash, asset.idx_asset_unique, datum.hash, script.hash, certs.uniq_tx_cert).
- Indexes on resume-checkpoint tables (import_checkpoint.import_key, backfill_checkpoint.phase).
- The utxo (tx_id, output_idx) lookup index, required to resolve transaction inputs during backfill UTxO spending.
- Cross-row uniqueness constraints used by ledger-state import (pool_stake_snapshot, reward_snapshot, reward_pool_input, network_state, account.staking_key, drep.credential, etc.).
Adding a new index to the versioned metadata schema requires deciding its bulk-load behavior at the same time:
- Does any import path (ledger-state import, immutable blob load, backfill block replay) rely on the index for an ON CONFLICT target, FK enforcement, or constraint lookup? If yes, leave it out of the manifest.
- Does the index only serve API/query/rollback paths that do not run during Mithril sync? If yes, add it here.
- Composite indexes share state with their constituent columns. If a field has both a deferrable single-column query index and a protected composite unique index, give the single-column index an explicit name and list that name here instead of the field.
See deferred_test.go for manifest invariants.
func CriticalManifest ¶ added in v0.51.0
func CriticalManifest() []Index
CriticalManifest returns the subset of Manifest entries that are marked Critical=true. These are the indexes that must be present before the API can serve traffic.
Source Files