chainsync

package
v0.163.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 20, 2026 License: Apache-2.0 Imports: 14 Imported by: 22

README

ChainSync Protocol

The ChainSync protocol synchronizes blockchain state between nodes by streaming headers (NtN) or blocks (NtC) from a producer to a consumer.

Protocol Identifiers

Property Value
Protocol Name chain-sync
Protocol ID (NtN) 2
Protocol ID (NtC) 5
Mode Node-to-Node / Node-to-Client

State Machine

                    RequestNext
              ┌──────────────────┐
              │                  ▼
┌──────┐      │           ┌──────────┐
│ Idle │──────┴──────────►│ CanAwait │◄────────┐
└──┬───┘                  └────┬─────┘         │
   │                           │               │
   │ FindIntersect    AwaitReply│               │ RollForward
   │                           │               │ RollBackward
   │                           ▼               │ (pipeline > 1)
   │                    ┌───────────┐          │
   │                    │ MustReply │──────────┘
   │                    └───────────┘
   │                           │
   │                           │ RollForward
   │                           │ RollBackward
   │                           │ (pipeline = 1)
   │                           │
   ▼                           ▼
┌───────────┐           ┌──────┐
│ Intersect │──────────►│ Idle │
└───────────┘           └──────┘
      │
      │ IntersectFound
      │ IntersectNotFound
      ▼
   ┌──────┐       Done      ┌──────┐
   │ Idle │ ───────────────►│ Done │
   └──────┘                 └──────┘

States

State ID Agency Description
Idle 1 Client Waiting for client request
CanAwait 2 Server Server may provide block or signal wait
MustReply 3 Server Server must provide block (no await allowed)
Intersect 4 Server Processing intersection request
Done 5 None Terminal state

Messages

Message Type ID Direction Description
RequestNext 0 Client → Server Request next block/header
AwaitReply 1 Server → Client Signal to wait for new blocks
RollForward 2 Server → Client Provide next block/header
RollBackward 3 Server → Client Signal chain rollback
FindIntersect 4 Client → Server Find common chain point
IntersectFound 5 Server → Client Intersection found
IntersectNotFound 6 Server → Client No intersection found
Done 7 Client → Server Terminate protocol

State Transitions

From Idle (Client Agency)
Message New State Notes
RequestNext CanAwait Increments pipeline count
FindIntersect Intersect
Done Done
From CanAwait (Server Agency)
Message New State Notes
RequestNext CanAwait Increments pipeline count
AwaitReply MustReply
RollForward Idle When pipeline count = 1
RollForward CanAwait When pipeline count > 1
RollBackward Idle When pipeline count = 1
RollBackward CanAwait When pipeline count > 1
From MustReply (Server Agency)
Message New State Notes
RollForward Idle When pipeline count = 1
RollForward CanAwait When pipeline count > 1
RollBackward Idle When pipeline count = 1
RollBackward CanAwait When pipeline count > 1
From Intersect (Server Agency)
Message New State
IntersectFound Idle
IntersectNotFound Idle

Timeouts (NtN only, per spec Table 3.8)

NtC mode has no timeouts per Ouroboros spec Table 3.9.

State Timeout Description
Idle 3673 seconds Client must send next request
CanAwait 10 seconds Server must provide block or await
Intersect 10 seconds Server must respond to intersect
MustReply random(135-269 seconds) Server must provide block

Limits

Limit Value Description
Max Pipeline Limit 100 Maximum pipelined requests
Default Pipeline Limit 75 Default pipeline limit
Max Recv Queue Size 100 Maximum receive queue size
Default Recv Queue Size 75 Default queue size
Max Pending Message Bytes 100 KB Maximum pending message bytes

Pipelining

ChainSync supports request pipelining for improved throughput. The client can send multiple RequestNext messages before receiving responses. The pipeline count tracks outstanding requests.

Configuration Options

chainsync.NewConfig(
    chainsync.WithRollBackwardFunc(rollBackwardCallback),
    chainsync.WithRollForwardFunc(rollForwardCallback),
    chainsync.WithRollForwardRawFunc(rollForwardRawCallback),
    chainsync.WithFindIntersectFunc(findIntersectCallback),
    chainsync.WithRequestNextFunc(requestNextCallback),
    chainsync.WithIntersectTimeout(10 * time.Second),
    chainsync.WithBlockTimeout(269 * time.Second),
    chainsync.WithPipelineLimit(75),
    chainsync.WithRecvQueueSize(75),
)

Usage Example

// Find intersection point
client.FindIntersect(knownPoints)

// Stream blocks
for {
    client.RequestNext()
    // Handle RollForward/RollBackward in callbacks
}

Notes

  • Node-to-Node mode streams headers only (use BlockFetch for full blocks)
  • Node-to-Client mode streams full blocks
  • Pipelining significantly improves sync performance
  • The protocol supports chain rollbacks during reorganizations

Documentation

Overview

Package chainsync implements the Ouroboros chain-sync protocol

Index

Constants

View Source 
const (
	ProtocolName         = "chain-sync"
	ProtocolIdNtN uint16 = 2
	ProtocolIdNtC uint16 = 5
)

Protocol identifiers

View Source 
const (
	MaxPipelineLimit            = 100    // Max pipelined requests
	MaxRecvQueueSize            = 100    // Max receive queue size (messages)
	DefaultPipelineLimit        = 75     // Default pipeline limit
	DefaultRecvQueueSize        = 75     // Default queue size
	MaxPendingMessageBytes      = 462000 // NtN mux ingress buffer per spec Table 3.15 (462KB)
	DefaultPipelineDrainTimeout = 30 * time.Second
)

Protocol limits per Ouroboros Network Specification

View Source 
const (
	IdleTimeout         = 3673 * time.Second  // Timeout for client to send next request
	CanAwaitTimeout     = 10 * time.Second    // Timeout for server to provide next block or await
	IntersectTimeout    = 10 * time.Second    // Timeout for server to respond to intersect request
	MustReplyTimeoutMin = 135 * time.Second   // Minimum random timeout for MustReply state
	MustReplyTimeoutMax = 269 * time.Second   // Maximum random timeout for MustReply state
	MustReplyTimeout    = MustReplyTimeoutMax // Fixed timeout for backward compatibility
)

Protocol state timeout constants per Ouroboros Network Specification (Table 3.8).

View Source 
const (
	MessageTypeRequestNext       = 0
	MessageTypeAwaitReply        = 1
	MessageTypeRollForward       = 2
	MessageTypeRollBackward      = 3
	MessageTypeFindIntersect     = 4
	MessageTypeIntersectFound    = 5
	MessageTypeIntersectNotFound = 6
	MessageTypeDone              = 7
)

Message types

Variables

View Source 
var ErrIntersectNotFound = errors.New("chain intersection not found")
View Source 
var ErrStopSyncProcess = errors.New("stop sync process")

StopChainSync is used as a special return value from a RollForward or RollBackward handler function to signify that the sync process should be stopped

View Source 
var StateMap = StateMapNtN.Copy()

StateMap is a copy of StateMapNtN for backward compatibility.

View Source 
var StateMapNtC = protocol.StateMap{
	// contains filtered or unexported fields
}

StateMapNtC is the N2C ChainSync state machine with no timeouts and no size limits per spec Section 3.7.5: "There are no size-limits nor timeouts".

View Source 
var StateMapNtN = protocol.StateMap{
	// contains filtered or unexported fields
}

StateMapNtN is the N2N ChainSync state machine with timeouts per spec Table 3.8.

Functions

func MustReplyTimeoutFunc added in v0.158.1 

func MustReplyTimeoutFunc() time.Duration

MustReplyTimeoutFunc returns a random timeout in [MustReplyTimeoutMin, MustReplyTimeoutMax) per Ouroboros Network Specification Table 3.8.

func NewMsgFromCbor 

func NewMsgFromCbor(
	protoMode protocol.ProtocolMode,
	msgType uint,
	data []byte,
) (protocol.Message, error)

NewMsgFromCbor parses a ChainSync message from CBOR

func NewMsgFromCborNtC 

func NewMsgFromCborNtC(msgType uint, data []byte) (protocol.Message, error)

NewMsgFromCborNtC parses a NtC ChainSync message from CBOR

func NewMsgFromCborNtN 

func NewMsgFromCborNtN(msgType uint, data []byte) (protocol.Message, error)

NewMsgFromCborNtN parses a NtC ChainSync message from CBOR

Types

type CallbackContext added in v0.78.0 

type CallbackContext struct {
	ConnectionId connection.ConnectionId
	Client       *Client
	Server       *Server
}

Callback context

type ChainSync 

type ChainSync struct {
	Client *Client
	Server *Server
}

ChainSync is a wrapper object that holds the client and server instances

func New 

func New(protoOptions protocol.ProtocolOptions, cfg *Config) *ChainSync

New returns a new ChainSync object

type ChainSyncOptionFunc 

type ChainSyncOptionFunc func(*Config)

ChainSyncOptionFunc represents a function used to modify the ChainSync protocol config

func WithBlockTimeout 

func WithBlockTimeout(timeout time.Duration) ChainSyncOptionFunc

WithBlockTimeout specifies the timeout for block fetch operations

func WithFindIntersectFunc added in v0.66.0 

func WithFindIntersectFunc(
	findIntersectFunc FindIntersectFunc,
) ChainSyncOptionFunc

WithFindIntersectFunc specifies the FindIntersect callback function

func WithIdleTimeout added in v0.158.2 

func WithIdleTimeout(timeout time.Duration) ChainSyncOptionFunc

WithIdleTimeout specifies the timeout for the Idle state (client agency). A non-zero value overrides the protocol default. This is primarily useful for the server side where the client may take a long time to send the next request.

func WithIntersectTimeout 

func WithIntersectTimeout(timeout time.Duration) ChainSyncOptionFunc

WithIntersectTimeout specifies the timeout for intersect operations

func WithPipeline added in v0.154.0 

func WithPipeline(p *pipeline.BlockPipeline) ChainSyncOptionFunc

WithPipeline sets the block processing pipeline for parallel block handling.

func WithPipelineBufferSize added in v0.154.0 

func WithPipelineBufferSize(size int) ChainSyncOptionFunc

WithPipelineBufferSize sets the prefetch buffer size when pipeline is enabled.

func WithPipelineDrainTimeout added in v0.154.0 

func WithPipelineDrainTimeout(timeout time.Duration) ChainSyncOptionFunc

WithPipelineDrainTimeout sets the timeout for draining the pipeline before rollback.

func WithPipelineLimit added in v0.36.0 

func WithPipelineLimit(limit int) ChainSyncOptionFunc

WithPipelineLimit specifies the maximum number of block requests to pipeline

func WithRecvQueueSize added in v0.114.0 

func WithRecvQueueSize(size int) ChainSyncOptionFunc

WithRecvQueueSize specifies the size of the received messages queue

func WithRequestNextFunc added in v0.66.0 

func WithRequestNextFunc(requestNextFunc RequestNextFunc) ChainSyncOptionFunc

WithRequestNextFunc specifies the RequestNext callback function

func WithRollBackwardFunc 

func WithRollBackwardFunc(
	rollBackwardFunc RollBackwardFunc,
) ChainSyncOptionFunc

WithRollBackwardFunc specifies the RollBackward callback function

func WithRollForwardFunc 

func WithRollForwardFunc(rollForwardFunc RollForwardFunc) ChainSyncOptionFunc

WithRollForwardFunc specifies the RollForward callback function. This will provided a parsed header or block

func WithRollForwardRawFunc added in v0.107.0 

func WithRollForwardRawFunc(
	rollForwardRawFunc RollForwardRawFunc,
) ChainSyncOptionFunc

WithRollForwardRawFunc specifies the RollForwardRaw callback function. This will provide the raw header or block

type Client 

type Client struct {
	*protocol.Protocol
	// contains filtered or unexported fields
}

Client implements the ChainSync client

func NewClient 

func NewClient(
	protoOptions protocol.ProtocolOptions,
	cfg *Config,
) *Client

NewClient returns a new ChainSync client object

func (*Client) GetAvailableBlockRange added in v0.38.0 

func (c *Client) GetAvailableBlockRange(
	intersectPoints []pcommon.Point,
) (pcommon.Point, pcommon.Point, error)

GetAvailableBlockRange returns the start and end of the range of available blocks given the provided intersect point(s). Empty start/end points will be returned if there are no additional blocks available.

func (*Client) GetCurrentTip 

func (c *Client) GetCurrentTip() (*Tip, error)

GetCurrentTip returns the current chain tip

func (*Client) ProtocolInstance added in v0.160.2 

func (c *Client) ProtocolInstance() *protocol.Protocol

func (*Client) Start added in v0.73.3 

func (c *Client) Start()

func (*Client) Stop 

func (c *Client) Stop() error

Stop sends a Done message and transitions the client to the Stopped state.

func (*Client) Sync 

func (c *Client) Sync(intersectPoints []pcommon.Point) error

Sync begins a chain-sync operation using the provided intersect point(s). Incoming blocks will be delivered via the RollForward callback function specified in the protocol config

type Config 

type Config struct {
	RollBackwardFunc    RollBackwardFunc
	RollForwardFunc     RollForwardFunc
	RollForwardRawFunc  RollForwardRawFunc
	FindIntersectFunc   FindIntersectFunc
	RequestNextFunc     RequestNextFunc
	IntersectTimeout    time.Duration
	IdleTimeout         time.Duration
	BlockTimeout        time.Duration
	PipelineLimit       int
	RecvQueueSize       int
	SkipBlockValidation bool // Skip block validation during parsing

	// Pipeline enables the block processing pipeline.
	// The pipeline should be configured with desired buffer sizes when created.
	Pipeline *pipeline.BlockPipeline

	// PipelineBufferSize is reserved for future use.
	// Currently, buffer sizes are configured when creating the BlockPipeline.
	PipelineBufferSize int

	// PipelineDrainTimeout is the timeout for draining the pipeline before rollback.
	// If zero, defaults to DefaultPipelineDrainTimeout.
	PipelineDrainTimeout time.Duration
}

Config is used to configure the ChainSync protocol instance

func NewConfig 

func NewConfig(options ...ChainSyncOptionFunc) Config

NewConfig returns a new ChainSync config object with the provided options

type FindIntersectFunc added in v0.66.0 

type FindIntersectFunc func(CallbackContext, []pcommon.Point) (pcommon.Point, Tip, error)

type MsgAwaitReply 

type MsgAwaitReply struct {
	protocol.MessageBase
}

func NewMsgAwaitReply 

func NewMsgAwaitReply() *MsgAwaitReply

type MsgDone 

type MsgDone struct {
	protocol.MessageBase
}

func NewMsgDone 

func NewMsgDone() *MsgDone

type MsgFindIntersect 

type MsgFindIntersect struct {
	protocol.MessageBase
	Points []pcommon.Point
}

func NewMsgFindIntersect 

func NewMsgFindIntersect(points []pcommon.Point) *MsgFindIntersect

type MsgIntersectFound 

type MsgIntersectFound struct {
	protocol.MessageBase
	Point pcommon.Point
	Tip   Tip
}

func NewMsgIntersectFound 

func NewMsgIntersectFound(point pcommon.Point, tip Tip) *MsgIntersectFound

type MsgIntersectNotFound 

type MsgIntersectNotFound struct {
	protocol.MessageBase
	Tip Tip
}

func NewMsgIntersectNotFound 

func NewMsgIntersectNotFound(tip Tip) *MsgIntersectNotFound

type MsgRequestNext 

type MsgRequestNext struct {
	protocol.MessageBase
}

func NewMsgRequestNext 

func NewMsgRequestNext() *MsgRequestNext

type MsgRollBackward 

type MsgRollBackward struct {
	protocol.MessageBase
	Point pcommon.Point
	Tip   Tip
}

func NewMsgRollBackward 

func NewMsgRollBackward(point pcommon.Point, tip Tip) *MsgRollBackward

type MsgRollForwardNtC 

type MsgRollForwardNtC struct {
	protocol.MessageBase
	WrappedBlock cbor.Tag
	Tip          Tip
	// contains filtered or unexported fields
}

MsgRollForwardNtC is the NtC version of the RollForward message

func NewMsgRollForwardNtC 

func NewMsgRollForwardNtC(
	blockType uint,
	blockCbor []byte,
	tip Tip,
) (*MsgRollForwardNtC, error)

NewMsgRollForwardNtC returns a MsgRollForwardNtC with the provided parameters

func (*MsgRollForwardNtC) BlockCbor 

func (m *MsgRollForwardNtC) BlockCbor() []byte

BlockCbor returns the block CBOR

func (*MsgRollForwardNtC) BlockType 

func (m *MsgRollForwardNtC) BlockType() uint

BlockType returns the block type

func (*MsgRollForwardNtC) UnmarshalCBOR 

func (m *MsgRollForwardNtC) UnmarshalCBOR(data []byte) error

type MsgRollForwardNtN 

type MsgRollForwardNtN struct {
	protocol.MessageBase
	WrappedHeader WrappedHeader
	Tip           Tip
}

MsgRollForwardNtN is the NtN version of the RollForward message

func NewMsgRollForwardNtN 

func NewMsgRollForwardNtN(
	era uint,
	byronType uint,
	blockCbor []byte,
	tip Tip,
) (*MsgRollForwardNtN, error)

NewMsgRollForwardNtN returns a MsgRollForwardNtN with the provided parameters

type RequestNextFunc added in v0.66.0 

type RequestNextFunc func(CallbackContext) error

type RollBackwardFunc 

type RollBackwardFunc func(CallbackContext, pcommon.Point, Tip) error

Callback function types

type RollForwardFunc 

type RollForwardFunc func(CallbackContext, uint, any, Tip) error

Callback function types

type RollForwardRawFunc added in v0.107.0 

type RollForwardRawFunc func(CallbackContext, uint, []byte, Tip) error

Callback function types

type Server 

type Server struct {
	*protocol.Protocol
	// contains filtered or unexported fields
}

Server implements the ChainSync server

func NewServer 

func NewServer(
	protoOptions protocol.ProtocolOptions,
	cfg *Config,
) *Server

NewServer returns a new ChainSync server object

func (*Server) AwaitReply added in v0.66.0 

func (s *Server) AwaitReply() error

func (*Server) ProtocolInstance added in v0.160.2 

func (s *Server) ProtocolInstance() *protocol.Protocol

func (*Server) RollBackward added in v0.66.0 

func (s *Server) RollBackward(point pcommon.Point, tip Tip) error

func (*Server) RollForward added in v0.66.0 

func (s *Server) RollForward(blockType uint, blockData []byte, tip Tip) error

type Tip 

type Tip = pcommon.Tip

Tip is an alias to keep historical code from breaking after moving this elsewhere

type WrappedBlock 

type WrappedBlock struct {
	cbor.StructAsArray
	BlockType uint
	BlockCbor cbor.RawMessage
}

WrappedBlock represents a block returned via a NtC RollForward message

func NewWrappedBlock 

func NewWrappedBlock(blockType uint, blockCbor []byte) *WrappedBlock

NewWrappedBlock returns a new WrappedBlock

type WrappedHeader 

type WrappedHeader struct {
	cbor.StructAsArray
	Era        uint
	RawMessage cbor.RawMessage
	// contains filtered or unexported fields
}

WrappedHeader represents a block header returned via NtN RollForward message

func NewWrappedHeader 

func NewWrappedHeader(
	era uint,
	byronType uint,
	blockCbor []byte,
) (*WrappedHeader, error)

NewWrappedHeader returns a new WrappedHeader

func (*WrappedHeader) ByronType 

func (w *WrappedHeader) ByronType() uint

ByronType returns the block type for Byron blocks

func (*WrappedHeader) HeaderCbor 

func (w *WrappedHeader) HeaderCbor() []byte

HeaderCbor returns the header CBOR

func (*WrappedHeader) MarshalCBOR 

func (w *WrappedHeader) MarshalCBOR() ([]byte, error)

func (*WrappedHeader) UnmarshalCBOR 

func (w *WrappedHeader) UnmarshalCBOR(data []byte) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.