engine

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 27, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ParseSize 

func ParseSize(s string) (int64, error)

ParseSize parses a human-readable size string like "25GB" into bytes. Supports B, KB, MB, GB, TB suffixes (case-insensitive). A plain number is treated as bytes.

Types

type ArchiveInfo 

type ArchiveInfo struct {
	Name   string
	Size   int64
	SHA256 string
	Files  []string
}

ArchiveInfo describes one split archive.

type ExportOptions 

type ExportOptions struct {
	OutputDir   string
	Providers   []string
	SplitSize   int64
	Compression string
}

ExportOptions configures an export operation.

type ExportReport 

type ExportReport struct {
	Archives     []ArchiveInfo
	TotalFiles   int
	TotalSize    int64
	ManifestPath string
	Duration     time.Duration
}

ExportReport summarizes a completed export.

type FileEvent 

type FileEvent struct {
	Path   string `json:"path"`
	Status string `json:"status"` // "completed", "failed"
	Error  string `json:"error,omitempty"`
	Size   int64  `json:"size,omitempty"`
}

FileEvent records a completed or failed file for the recent activity log.

type FileProgress 

type FileProgress struct {
	Path            string `json:"path"`
	BytesDownloaded int64  `json:"bytes_downloaded"`
	TotalBytes      int64  `json:"total_bytes"`
	Done            bool   `json:"done"`
	Failed          bool   `json:"failed"`
}

FileProgress tracks the download state of an individual file.

type ImportOptions 

type ImportOptions struct {
	SourceDir     string
	VerifyOnly    bool
	Force         bool
	SkipValidated bool
}

ImportOptions configures an import operation.

type ImportReport 

type ImportReport struct {
	ArchivesValidated int
	ArchivesFailed    int
	ArchivesSkipped   int
	FilesExtracted    int
	TotalSize         int64
	Duration          time.Duration
	Errors            []string
}

ImportReport summarizes a completed import.

type ManifestArchive 

type ManifestArchive struct {
	Name   string   `json:"name"`
	Size   int64    `json:"size"`
	SHA256 string   `json:"sha256"`
	Files  []string `json:"files"`
}

ManifestArchive describes a single split archive in the export.

type ManifestFile 

type ManifestFile struct {
	Provider string `json:"provider"`
	Path     string `json:"path"`
	Size     int64  `json:"size"`
	SHA256   string `json:"sha256"`
}

ManifestFile is one entry in the full file inventory.

type ManifestProvider 

type ManifestProvider struct {
	Type      string `json:"type,omitempty"`
	FileCount int    `json:"file_count"`
	TotalSize int64  `json:"total_size"`
}

ManifestProvider summarizes one provider's contribution to the export.

type ProviderFactory 

type ProviderFactory func(typeName, dataDir string, logger *slog.Logger) (provider.Provider, error)

ProviderFactory creates a provider instance given a type name and data dir.

type ProviderStatus 

type ProviderStatus struct {
	Name        string
	Enabled     bool
	FileCount   int
	TotalSize   int64
	LastSync    time.Time
	LastStatus  string
	FailedFiles int
}

ProviderStatus summarizes a provider's state.

type RegistryPushOptions 

type RegistryPushOptions struct {
	SourceProvider string
	TargetProvider string
	DryRun         bool
}

RegistryPushOptions configures pushing mirrored container images to a target registry.

type RegistryPushReport 

type RegistryPushReport struct {
	SourceProvider  string
	TargetProvider  string
	ImagesTotal     int
	ImagesPushed    int
	BlobsProcessed  int
	ManifestsPushed int
	Failures        []string
	Duration        time.Duration
}

RegistryPushReport summarizes an image push operation.

type ScanReport 

type ScanReport struct {
	Provider string `json:"provider"`
	Found    int    `json:"found"`
	New      int    `json:"new"`
	Updated  int    `json:"updated"`
	Duration string `json:"duration"`
}

ScanReport summarizes the results of a local file scan.

type SyncManager 

type SyncManager struct {
	// contains filtered or unexported fields
}

SyncManager orchestrates providers and connects them to the download client and store.

func NewSyncManager 

func NewSyncManager(
	registry *provider.Registry,
	st *store.Store,
	client *download.Client,
	cfg *config.Config,
	logger *slog.Logger,
) *SyncManager

NewSyncManager creates a new SyncManager.

func (*SyncManager) ActiveProgress 

func (m *SyncManager) ActiveProgress() *SyncTracker

ActiveProgress returns the tracker for the currently running sync, or nil.

func (*SyncManager) ClearActiveTracker 

func (m *SyncManager) ClearActiveTracker()

ClearActiveTracker removes the active tracker.

func (*SyncManager) Client 

func (m *SyncManager) Client() *download.Client

Client returns the download client for reuse by retry operations.

func (*SyncManager) Export 

func (m *SyncManager) Export(ctx context.Context, opts ExportOptions) (*ExportReport, error)

Export creates split tar.zst archives of synced content for air-gapped transfer.

func (*SyncManager) Import 

func (m *SyncManager) Import(ctx context.Context, opts ImportOptions) (*ImportReport, error)

Import reads an airgap transfer package and extracts its contents.

func (*SyncManager) PushContainerImages 

func (m *SyncManager) PushContainerImages(ctx context.Context, opts RegistryPushOptions) (*RegistryPushReport, error)

PushContainerImages pushes mirrored container images from a container_images provider to a configured registry target.

func (*SyncManager) ReconfigureProviders 

func (m *SyncManager) ReconfigureProviders(configs []store.ProviderConfig) error

ReconfigureProviders rebuilds the provider registry from the given configs. Only enabled providers with a configured factory are instantiated. Acquires a write lock to prevent races with running syncs.

func (*SyncManager) ScanLocal 

func (m *SyncManager) ScanLocal(ctx context.Context, providerName string) (*ScanReport, error)

ScanLocal walks the data directory on disk for a given provider, computes SHA256 for each file, and upserts FileRecords in the store. This allows rebuilding store state from files already on disk without downloading.

func (*SyncManager) SetActiveTracker 

func (m *SyncManager) SetActiveTracker(t *SyncTracker)

SetActiveTracker installs an external tracker (e.g. for retry operations).

func (*SyncManager) SetProviderFactory 

func (m *SyncManager) SetProviderFactory(f ProviderFactory)

SetProviderFactory sets the factory used by ReconfigureProviders.

func (*SyncManager) Status 

func (m *SyncManager) Status() map[string]ProviderStatus

Status returns a summary of each provider's state by querying the store.

func (*SyncManager) SyncAll 

func (m *SyncManager) SyncAll(ctx context.Context, opts provider.SyncOptions) (map[string]*provider.SyncReport, error)

SyncAll synchronizes all enabled providers. It continues even if one provider fails, collecting all reports and errors.

func (*SyncManager) SyncProvider 

func (m *SyncManager) SyncProvider(ctx context.Context, name string, opts provider.SyncOptions) (*provider.SyncReport, error)

SyncProvider synchronizes a single provider. It orchestrates planning, downloading, storing, and cleanup operations.

func (*SyncManager) ValidateAll 

func (m *SyncManager) ValidateAll(ctx context.Context) (map[string]*provider.ValidationReport, error)

ValidateAll validates all enabled providers. It continues even if one provider fails, collecting all reports.

func (*SyncManager) ValidateProvider 

func (m *SyncManager) ValidateProvider(ctx context.Context, name string) (*provider.ValidationReport, error)

ValidateProvider validates a single provider.

type SyncPhase 

type SyncPhase string

SyncPhase represents the current phase of a sync operation. 

const (
	PhasePlanning    SyncPhase = "planning"
	PhaseDownloading SyncPhase = "downloading"
	PhaseComplete    SyncPhase = "complete"
	PhaseFailed      SyncPhase = "failed"
	PhaseCancelled   SyncPhase = "cancelled"
)

type SyncProgress 

type SyncProgress struct {
	Provider        string         `json:"provider"`
	Phase           SyncPhase      `json:"phase"`
	TotalFiles      int            `json:"total_files"`
	CompletedFiles  int            `json:"completed_files"`
	FailedFiles     int            `json:"failed_files"`
	SkippedFiles    int            `json:"skipped_files"`
	TotalBytes      int64          `json:"total_bytes"`
	BytesDownloaded int64          `json:"bytes_downloaded"`
	Percent         float64        `json:"percent"`
	CurrentFiles    []FileProgress `json:"current_files,omitempty"`
	RecentEvents    []FileEvent    `json:"recent_events,omitempty"`
	TotalRetries    int            `json:"total_retries"`
	BytesPerSecond  int64          `json:"bytes_per_second"`
	ETA             string         `json:"eta,omitempty"`
	StartTime       time.Time      `json:"start_time"`
	Elapsed         string         `json:"elapsed"`
	Message         string         `json:"message,omitempty"`
}

SyncProgress is a snapshot of the current sync state, safe for JSON serialization.

type SyncTracker 

type SyncTracker struct {
	// contains filtered or unexported fields
}

SyncTracker accumulates progress from pool workers in a thread-safe manner. SSE handlers use Wait() to block until new updates are available.

func NewSyncTracker 

func NewSyncTracker(providerName string) *SyncTracker

NewSyncTracker creates a tracker for the given provider.

func (*SyncTracker) AddRetries 

func (t *SyncTracker) AddRetries(count int)

AddRetries increments the total retry counter.

func (*SyncTracker) FileCompleted 

func (t *SyncTracker) FileCompleted(destPath string, bytesDownloaded int64)

FileCompleted marks a file as successfully downloaded.

func (*SyncTracker) FileFailed 

func (t *SyncTracker) FileFailed(destPath string, errMsg string)

FileFailed marks a file as failed with an error reason.

func (*SyncTracker) FileSkipped 

func (t *SyncTracker) FileSkipped()

FileSkipped increments the skipped file counter.

func (*SyncTracker) SetMessage 

func (t *SyncTracker) SetMessage(msg string)

SetMessage sets a human-readable status message.

func (*SyncTracker) SetPhase 

func (t *SyncTracker) SetPhase(phase SyncPhase)

SetPhase updates the current sync phase.

func (*SyncTracker) SetSkippedFiles 

func (t *SyncTracker) SetSkippedFiles(count int)

SetSkippedFiles sets the skipped file count in bulk (one signal instead of N).

func (*SyncTracker) SetTotals 

func (t *SyncTracker) SetTotals(totalFiles int, totalBytes int64)

SetTotals sets the total file count and byte count after planning.

func (*SyncTracker) Snapshot 

func (t *SyncTracker) Snapshot() SyncProgress

Snapshot returns a copy of the current progress state.

func (*SyncTracker) UpdateFileProgress 

func (t *SyncTracker) UpdateFileProgress(destPath string, bytesDownloaded, totalBytes int64)

UpdateFileProgress updates the byte-level progress for a single file. Throttled to ~250ms per file to reduce lock contention.

func (*SyncTracker) Wait 

func (t *SyncTracker) Wait() <-chan struct{}

Wait returns a channel that will be closed when the next update occurs. Callers should select on this channel alongside a timeout for heartbeats.

type TransferManifest 

type TransferManifest struct {
	Version       string                      `json:"version"`
	Created       time.Time                   `json:"created"`
	SourceHost    string                      `json:"source_host"`
	Providers     map[string]ManifestProvider `json:"providers"`
	Archives      []ManifestArchive           `json:"archives"`
	TotalArchives int                         `json:"total_archives"`
	TotalSize     int64                       `json:"total_size"`
	FileInventory []ManifestFile              `json:"file_inventory"`
}

TransferManifest describes a complete export for transfer to an air-gapped environment.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.