Documentation
¶
Overview ¶
Package queue implements the data transmission queue.
Index ¶
- func ClearChunkedSyncFailureInjector()
- func RegisterChunkedSyncFailureInjector(injector ChunkedSyncFailureInjector)
- type BatchPublisher
- type ChunkedSyncClient
- type ChunkedSyncFailureInjector
- type ChunkedSyncHandler
- type ChunkedSyncPartContext
- type Client
- type FailedPart
- type FileData
- type FileInfo
- type PartHandler
- type Queue
- type Server
- type StreamingPartData
- type SyncMetadata
- type SyncResult
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ClearChunkedSyncFailureInjector ¶ added in v0.9.0
func ClearChunkedSyncFailureInjector()
ClearChunkedSyncFailureInjector removes the registered failure injector.
func RegisterChunkedSyncFailureInjector ¶ added in v0.9.0
func RegisterChunkedSyncFailureInjector(injector ChunkedSyncFailureInjector)
RegisterChunkedSyncFailureInjector registers a failure injector for testing.
Types ¶
type BatchPublisher ¶ added in v0.5.0
BatchPublisher is the interface for publishing data in batch.
type ChunkedSyncClient ¶ added in v0.9.0
type ChunkedSyncClient interface {
// SyncStreamingParts sends streaming parts using chunked transfer.
SyncStreamingParts(ctx context.Context, parts []StreamingPartData) (*SyncResult, error)
// Close releases resources associated with the client.
Close() error
}
ChunkedSyncClient provides methods for chunked data synchronization with streaming support.
type ChunkedSyncFailureInjector ¶ added in v0.9.0
type ChunkedSyncFailureInjector interface {
// BeforeSync returns whether the current SyncStreamingParts invocation should
// short-circuit. The returned slice provides the failed parts that should be
// reported back to the caller when a failure is injected.
BeforeSync(parts []StreamingPartData) (bool, []FailedPart, error)
}
ChunkedSyncFailureInjector allows tests to deterministically inject failures into chunked sync operations. It is only intended for test code.
func GetChunkedSyncFailureInjector ¶ added in v0.9.0
func GetChunkedSyncFailureInjector() ChunkedSyncFailureInjector
GetChunkedSyncFailureInjector returns the currently registered failure injector.
type ChunkedSyncHandler ¶ added in v0.9.0
type ChunkedSyncHandler interface {
// HandleFileChunk processes incoming file chunks as they arrive.
// This method is called for each chunk of file data to enable streaming processing.
HandleFileChunk(ctx *ChunkedSyncPartContext, chunk []byte) error
// CreatePartHandler creates a new part handler for the given context.
CreatePartHandler(ctx *ChunkedSyncPartContext) (PartHandler, error)
}
ChunkedSyncHandler handles incoming chunked sync requests on the server side.
type ChunkedSyncPartContext ¶ added in v0.9.0
type ChunkedSyncPartContext struct {
Handler PartHandler
Group string
FileName string
PartType string
ID uint64
CompressedSizeBytes uint64
UncompressedSizeBytes uint64
TotalCount uint64
BlocksCount uint64
MinTimestamp int64
MaxTimestamp int64
MinKey int64
MaxKey int64
ShardID uint32
}
ChunkedSyncPartContext represents the context for a chunked sync operation.
func (*ChunkedSyncPartContext) Close ¶ added in v0.9.0
func (c *ChunkedSyncPartContext) Close() error
Close releases resources associated with the context.
type Client ¶ added in v0.5.0
type Client interface {
run.Unit
bus.Publisher
bus.Broadcaster
NewBatchPublisher(timeout time.Duration) BatchPublisher
NewChunkedSyncClient(node string, chunkSize uint32) (ChunkedSyncClient, error)
Register(bus.Topic, schema.EventHandler)
OnAddOrUpdate(md schema.Metadata)
GracefulStop()
HealthyNodes() []string
}
Client is the interface for publishing data to the queue.
type FailedPart ¶ added in v0.9.0
FailedPart contains information about a part that failed to sync.
type FileInfo ¶ added in v0.9.0
FileInfo represents information about an individual file within a part.
type PartHandler ¶ added in v0.9.0
type PartHandler interface {
NewPartType(ctx *ChunkedSyncPartContext) error
FinishSync() error
Close() error
}
PartHandler handles individual parts during sync operations.
type Server ¶ added in v0.5.0
type Server interface {
run.Unit
bus.Subscriber
RegisterChunkedSyncHandler(topic bus.Topic, handler ChunkedSyncHandler)
GetPort() *uint32
}
Server is the interface for receiving data from the queue.
type StreamingPartData ¶ added in v0.9.0
type StreamingPartData struct {
Group string
Topic string
PartType string
Files []FileInfo
ID uint64
CompressedSizeBytes uint64
UncompressedSizeBytes uint64
TotalCount uint64
BlocksCount uint64
MinTimestamp int64
MaxTimestamp int64
MinKey int64
MaxKey int64
ShardID uint32
}
StreamingPartData represents streaming part data for syncing.
type SyncMetadata ¶ added in v0.9.0
type SyncMetadata = clusterv1.SyncMetadata
SyncMetadata is an alias for clusterv1.SyncMetadata.
type SyncResult ¶ added in v0.9.0
type SyncResult struct {
SessionID string
FailedParts []FailedPart
TotalBytes uint64
DurationMs int64
ChunksCount uint32
PartsCount uint32
Success bool
}
SyncResult represents the result of a sync operation.