manifest

package
v0.4.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 9, 2026 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package manifest provides loading and validation of gonimbus job manifests.

A job manifest is a YAML or JSON file that configures all aspects of a crawl job: provider connection, pattern matching, crawl behavior, and output.

Manifests are validated against a JSON Schema to ensure correctness before execution. The schema enforces strict typing and disallows unknown properties.

Example manifest (YAML): 

version: "1.0"
connection:
  provider: s3
  bucket: my-data-bucket
  region: us-east-1
match:
  includes:
    - "data/2024/**/*.parquet"
  excludes:
    - "**/_temporary/**"
crawl:
  concurrency: 4
output:
  destination: stdout
  progress: true

Index

Constants

View Source 
const (
	// DefaultIndexVersion is the current index manifest schema version.
	DefaultIndexVersion = "1.0"

	// DefaultIndexSource is the default index source type.
	DefaultIndexSource = "crawl"

	// DefaultIndexConcurrency is the default number of concurrent list operations for index build.
	DefaultIndexConcurrency = 4

	// DefaultIndexProgressEvery is the default progress emission frequency for index build.
	DefaultIndexProgressEvery = 1000

	// DefaultIndexIncludes is the default include pattern (index everything).
	DefaultIndexIncludes = "**"
)

Index manifest defaults.

View Source 
const (
	// DefaultVersion is the current manifest schema version.
	DefaultVersion = "1.0"

	// DefaultConcurrency is the default number of concurrent list operations.
	DefaultConcurrency = 4

	// DefaultRateLimit is the default rate limit (0 = unlimited).
	DefaultRateLimit = 0.0

	// DefaultProgressEvery is the default progress emission frequency.
	DefaultProgressEvery = 1000

	// DefaultDestination is the default output destination.
	DefaultDestination = "stdout"

	// DefaultProgress is the default value for progress emission.
	DefaultProgress = true

	// DefaultPreflightMode is the default preflight mode.
	DefaultPreflightMode = "read-safe"

	// DefaultProbeStrategy is the default provider probe strategy.
	DefaultProbeStrategy = "multipart-abort"

	// DefaultProbePrefix is the default prefix under which probe keys are created.
	DefaultProbePrefix = "_gonimbus/probe/"
)

Default values for optional configuration fields.

View Source 
const (
	// DefaultTransferConcurrency is the default concurrent transfer workers.
	DefaultTransferConcurrency = 16

	// DefaultTransferMode is the default transfer mode.
	DefaultTransferMode = "copy"

	// DefaultOnExists is the default on-exists behavior.
	DefaultOnExists = "skip"

	// DefaultDedupEnabled is the default for dedup enabled.
	DefaultDedupEnabled = true

	// DefaultDedupStrategy is the default dedup strategy.
	DefaultDedupStrategy = "etag"

	DefaultShardDepth           = 1
	DefaultShardListConcurrency = 16
	DefaultShardDelimiter       = "/"
)
View Source 
const SchemaID = "gonimbus/v1.0.0/job-manifest"

SchemaID is the schema identifier for job manifests.

View Source 
const TransferSchemaID = "gonimbus/v1.0.0/transfer-manifest"

TransferSchemaID is the schema identifier for transfer manifests.

Variables

View Source 
var (
	// ErrTransferSchemaNotFound indicates the transfer schema file could not be located.
	ErrTransferSchemaNotFound = errors.New("transfer manifest schema not found")

	// ErrTransferValidationFailed indicates the manifest failed schema validation.
	ErrTransferValidationFailed = errors.New("transfer manifest validation failed")
)

Transfer validation errors.

View Source 
var (
	// ErrSchemaNotFound indicates the schema file could not be located.
	ErrSchemaNotFound = errors.New("manifest schema not found")

	// ErrValidationFailed indicates the manifest failed schema validation.
	ErrValidationFailed = errors.New("manifest validation failed")
)

Validation errors

Functions

func Validate 

func Validate(m *Manifest) error

Validate checks the manifest against the JSON schema.

Returns nil if validation succeeds, or a ValidationErrors with details about all validation failures.

Note: This validates the struct representation, which loses unknown fields. For strict validation including additionalProperties checks, use ValidateRaw on the original input data.

func ValidateRaw 

func ValidateRaw(jsonData []byte) error

ValidateRaw checks raw JSON data against the manifest schema.

This function should be used when strict validation is needed, including rejection of unknown fields (additionalProperties: false). The raw JSON preserves all fields from the original input.

The schema is embedded at compile time, so validation works correctly in installed binaries and library consumers without requiring schema files to be present on disk.

Returns nil if validation succeeds, or a ValidationErrors with details about all validation failures.

func ValidateTransfer 

func ValidateTransfer(m *TransferManifest) error

ValidateTransfer validates a typed TransferManifest by round-tripping to JSON.

func ValidateTransferRaw 

func ValidateTransferRaw(jsonData []byte) error

ValidateTransferRaw checks raw JSON data against the transfer manifest schema.

Types

type ConnectionConfig 

type ConnectionConfig struct {
	// Provider is the storage provider type.
	Provider string `json:"provider" yaml:"provider"`

	// Bucket is the bucket name to crawl for object-store providers.
	Bucket string `json:"bucket,omitempty" yaml:"bucket,omitempty"`

	// BaseDir is the absolute local root for provider=file crawl manifests.
	BaseDir string `json:"base_dir,omitempty" yaml:"base_dir,omitempty"`

	// Region is the AWS region (e.g., "us-east-1"). Optional.
	Region string `json:"region,omitempty" yaml:"region,omitempty"`

	// Project is the optional GCP project hint for provider=gcs.
	Project string `json:"project,omitempty" yaml:"project,omitempty"`

	// Endpoint is a custom endpoint URL for S3-compatible storage. Optional.
	// Example: "https://s3.wasabisys.com"
	Endpoint string `json:"endpoint,omitempty" yaml:"endpoint,omitempty"`

	// Profile is the AWS credential profile name. Optional.
	Profile string `json:"profile,omitempty" yaml:"profile,omitempty"`
}

ConnectionConfig configures the storage provider connection.

type CrawlConfig 

type CrawlConfig struct {
	// Concurrency is the number of concurrent list operations.
	// Range: 1-32. Default: 4.
	Concurrency int `json:"concurrency,omitempty" yaml:"concurrency,omitempty"`

	// RateLimit is the maximum requests per second (0 = unlimited).
	// Default: 0.
	RateLimit float64 `json:"rate_limit,omitempty" yaml:"rate_limit,omitempty"`

	// ProgressEvery controls progress record frequency.
	// A progress record is emitted every N matched objects.
	// Default: 1000.
	ProgressEvery int `json:"progress_every,omitempty" yaml:"progress_every,omitempty"`

	// Preflight configures permission checks and provider probes.
	//
	// This is part of the plan/inspect/execute model for long-running jobs.
	Preflight PreflightConfig `json:"preflight,omitempty" yaml:"preflight,omitempty"`
}

CrawlConfig configures crawl behavior.

All fields are optional with sensible defaults applied during loading.

type DateFilterConfig 

type DateFilterConfig struct {
	// After filters to objects modified at or after this time (inclusive).
	After string `json:"after,omitempty" yaml:"after,omitempty"`

	// Before filters to objects modified before this time (exclusive end).
	Before string `json:"before,omitempty" yaml:"before,omitempty"`
}

DateFilterConfig specifies date range constraints.

type DedupConfig 

type DedupConfig struct {
	Enabled  *bool  `json:"enabled,omitempty" yaml:"enabled,omitempty"`
	Strategy string `json:"strategy,omitempty" yaml:"strategy,omitempty"`
}

DedupConfig controls deduplication behavior.

func (DedupConfig) DedupEnabled 

func (d DedupConfig) DedupEnabled() bool

DedupEnabled returns whether deduplication is enabled.

type FilterConfig 

type FilterConfig struct {
	// Size specifies min/max size constraints.
	// Supports human-readable values: "1KB", "100MiB", "1GB".
	Size *SizeFilterConfig `json:"size,omitempty" yaml:"size,omitempty"`

	// Modified specifies last-modified date range constraints.
	// Dates are in ISO 8601 format: "2024-01-15" or "2024-01-15T10:30:00Z".
	Modified *DateFilterConfig `json:"modified,omitempty" yaml:"modified,omitempty"`

	// ContentType specifies allowed MIME types.
	// Requires enrichment (HEAD calls) to evaluate.
	ContentType []string `json:"content_type,omitempty" yaml:"content_type,omitempty"`

	// KeyRegex is a regex pattern applied to object keys after glob matching.
	// Use for patterns not expressible with globs, e.g., "TXN-\\d{8}".
	KeyRegex string `json:"key_regex,omitempty" yaml:"key_regex,omitempty"`
}

FilterConfig specifies metadata-based object filters. All filters are optional and compose with AND semantics.

type IndexBuildConfig 

type IndexBuildConfig struct {
	// Source is the index source type (crawl|inventory|auto).
	// v0.1.3: crawl-only, inventory reserved for future.
	Source string `json:"source,omitempty" yaml:"source,omitempty"`

	// Scope constrains what prefixes are listed during an index build.
	// v0.1.4: scope partitions (path scoping).
	Scope *IndexScopeConfig `json:"scope,omitempty" yaml:"scope,omitempty"`

	// Match configures to object filtering during build (defaults to index everything).
	Match *IndexMatchConfig `json:"match,omitempty" yaml:"match,omitempty"`

	// Crawl configures to crawl-specific build settings.
	Crawl *IndexCrawlBuildConfig `json:"crawl,omitempty" yaml:"crawl,omitempty"`
}

IndexBuildConfig configures to index build behavior.

type IndexConnectionConfig 

type IndexConnectionConfig struct {
	// Provider is the storage provider type.
	Provider string `json:"provider" yaml:"provider"`

	// Bucket is the bucket name to index.
	Bucket string `json:"bucket" yaml:"bucket"`

	// BaseURI is the base prefix URI for the index. Must end with '/'.
	BaseURI string `json:"base_uri" yaml:"base_uri"`

	// Region is the AWS region. Optional.
	Region string `json:"region,omitempty" yaml:"region,omitempty"`

	// Project is the optional GCP project hint for provider=gcs.
	Project string `json:"project,omitempty" yaml:"project,omitempty"`

	// Endpoint is a custom endpoint URL for S3-compatible storage. Optional.
	Endpoint string `json:"endpoint,omitempty" yaml:"endpoint,omitempty"`

	// Profile is the AWS credential profile name. Optional.
	Profile string `json:"profile,omitempty" yaml:"profile,omitempty"`
}

IndexConnectionConfig configures the index source storage connection.

type IndexCrawlBuildConfig 

type IndexCrawlBuildConfig struct {
	// Concurrency is the number of concurrent list operations.
	// Range: 1-32. Default: 4.
	Concurrency int `json:"concurrency,omitempty" yaml:"concurrency,omitempty"`

	// RateLimit is the maximum requests per second (0 = unlimited).
	// Default: 0.
	RateLimit float64 `json:"rate_limit,omitempty" yaml:"rate_limit,omitempty"`

	// ProgressEvery controls the progress emission frequency.
	// A progress record is emitted every N indexed objects.
	// Default: 1000.
	ProgressEvery int `json:"progress_every,omitempty" yaml:"progress_every,omitempty"`
}

IndexCrawlBuildConfig configures to crawl-specific build behavior.

type IndexIdentityConfig 

type IndexIdentityConfig struct {
	// StorageProvider is the canonical backend/variant (e.g., aws_s3, r2, wasabi).
	StorageProvider string `json:"storage_provider,omitempty" yaml:"storage_provider,omitempty"`

	// CloudProvider is the broader cloud (e.g., aws, gcp, azure).
	CloudProvider string `json:"cloud_provider,omitempty" yaml:"cloud_provider,omitempty"`

	// RegionKind disambiguates region naming schemes (e.g., aws, gcp, azure).
	RegionKind string `json:"region_kind,omitempty" yaml:"region_kind,omitempty"`

	// Region is the region name.
	Region string `json:"region,omitempty" yaml:"region,omitempty"`

	// EndpointHost is the host part of the endpoint URL.
	EndpointHost string `json:"endpoint_host,omitempty" yaml:"endpoint_host,omitempty"`
}

IndexIdentityConfig provides explicit provider identity.

type IndexManifest 

type IndexManifest struct {
	// Schema is an optional JSON Schema reference for editor support.
	Schema string `json:"$schema,omitempty" yaml:"$schema,omitempty"`

	// Version is the manifest schema version. Must be "1.0".
	Version string `json:"version" yaml:"version"`

	// Connection configures cloud storage provider and base URI.
	Connection IndexConnectionConfig `json:"connection" yaml:"connection"`

	// Identity provides explicit provider identity (ENTARCH: never infer as authoritative).
	Identity *IndexIdentityConfig `json:"identity,omitempty" yaml:"identity,omitempty"`

	// Build configures to index build behavior.
	Build *IndexBuildConfig `json:"build,omitempty" yaml:"build,omitempty"`

	// PathDate configures to date extraction from object key paths.
	PathDate *PathDateConfig `json:"path_date,omitempty" yaml:"path_date,omitempty"`
}

IndexManifest represents a validated index build manifest.

A manifest configures all aspects of an index build job. Required fields are Version and Connection.

func LoadIndexManifest 

func LoadIndexManifest(path string) (*IndexManifest, error)

LoadIndexManifest loads and validates an index manifest from given file path.

func LoadIndexManifestFromBytes 

func LoadIndexManifestFromBytes(data []byte, path string) (*IndexManifest, error)

LoadIndexManifestFromBytes parses and validates an index manifest from raw bytes.

func (*IndexManifest) ApplyDefaults 

func (m *IndexManifest) ApplyDefaults()

ApplyDefaults fills in default values for optional index manifest fields.

type IndexMatchConfig 

type IndexMatchConfig struct {
	// Includes is a list of glob patterns for objects to include.
	// Default: ["**"] (index everything under base_uri).
	Includes []string `json:"includes,omitempty" yaml:"includes,omitempty"`

	// Excludes is a list of glob patterns for objects to exclude. Optional.
	Excludes []string `json:"excludes,omitempty" yaml:"excludes,omitempty"`

	// IncludeHidden includes hidden files (starting with .). Default: false.
	IncludeHidden bool `json:"include_hidden,omitempty" yaml:"include_hidden,omitempty"`

	// Filters specifies additional metadata-based filters. Optional.
	Filters *FilterConfig `json:"filters,omitempty" yaml:"filters,omitempty"`
}

IndexMatchConfig configures to object matching for index build.

type IndexScopeConfig 

type IndexScopeConfig struct {
	Type string `json:"type" yaml:"type"`

	// Shared options
	BasePrefix string `json:"base_prefix,omitempty" yaml:"base_prefix,omitempty"`
	Delimiter  string `json:"delimiter,omitempty" yaml:"delimiter,omitempty"`

	// prefix_list
	Prefixes []string `json:"prefixes,omitempty" yaml:"prefixes,omitempty"`

	// union
	Scopes []IndexScopeConfig `json:"scopes,omitempty" yaml:"scopes,omitempty"`

	// date_partitions
	Discover *IndexScopeDiscoverConfig `json:"discover,omitempty" yaml:"discover,omitempty"`
	Date     *IndexScopeDateConfig     `json:"date,omitempty" yaml:"date,omitempty"`

	// segments (reserved; proposed design in docs/architecture/adr/ADR-0009)
	Constraints []IndexScopeSegmentConstraint `json:"constraints,omitempty" yaml:"constraints,omitempty"`
	DepthMode   string                        `json:"depth_mode,omitempty" yaml:"depth_mode,omitempty"`
}

IndexScopeConfig constrains what prefixes are listed during an index build.

ENTARCH: scope is a scoper (provider-cost lever) and must compile to an explicit prefix plan. Match patterns remain ingest predicates.

type IndexScopeDateConfig 

type IndexScopeDateConfig struct {
	SegmentIndex int                  `json:"segment_index" yaml:"segment_index"`
	Format       string               `json:"format,omitempty" yaml:"format,omitempty"`
	Range        *IndexScopeDateRange `json:"range,omitempty" yaml:"range,omitempty"`
	Glob         string               `json:"glob,omitempty" yaml:"glob,omitempty"`
}

type IndexScopeDateRange 

type IndexScopeDateRange struct {
	After  string `json:"after" yaml:"after"`
	Before string `json:"before" yaml:"before"`
}

type IndexScopeDiscoverConfig 

type IndexScopeDiscoverConfig struct {
	Segments []IndexScopeDiscoverSegment `json:"segments,omitempty" yaml:"segments,omitempty"`
}

type IndexScopeDiscoverSegment 

type IndexScopeDiscoverSegment struct {
	Index     int      `json:"index" yaml:"index"`
	Allow     []string `json:"allow,omitempty" yaml:"allow,omitempty"`
	Deny      []string `json:"deny,omitempty" yaml:"deny,omitempty"`
	GlobAllow []string `json:"glob_allow,omitempty" yaml:"glob_allow,omitempty"`
	GlobDeny  []string `json:"glob_deny,omitempty" yaml:"glob_deny,omitempty"`
}

type IndexScopeSegmentConstraint 

type IndexScopeSegmentConstraint struct {
	Index int    `json:"index" yaml:"index"`
	Glob  string `json:"glob" yaml:"glob"`
}

type Manifest 

type Manifest struct {
	// Schema is an optional JSON Schema reference for editor support.
	// Example: "https://schemas.3leaps.dev/gonimbus/v1.0.0/job-manifest.schema.json"
	Schema string `json:"$schema,omitempty" yaml:"$schema,omitempty"`

	// Version is the manifest schema version. Must be "1.0".
	Version string `json:"version" yaml:"version"`

	// Connection configures the cloud storage provider.
	Connection ConnectionConfig `json:"connection" yaml:"connection"`

	// Match configures object filtering by glob patterns.
	Match MatchConfig `json:"match" yaml:"match"`

	// Crawl configures crawl behavior (optional).
	Crawl CrawlConfig `json:"crawl,omitempty" yaml:"crawl,omitempty"`

	// Output configures output destination and format (optional).
	Output OutputConfig `json:"output,omitempty" yaml:"output,omitempty"`
}

Manifest represents a validated job manifest.

A manifest configures all aspects of a crawl job. Required fields are Version, Connection, and Match. Crawl and Output are optional with sensible defaults.

func Load 

func Load(path string) (*Manifest, error)

Load reads and validates a manifest from the given file path.

The file format is determined by extension: .yaml/.yml for YAML, .json for JSON. If the extension is unrecognized, YAML is attempted first, then JSON.

After loading, the manifest is validated against the JSON schema, and defaults are applied to optional fields.

Returns an error if:

  • The file cannot be read (not found, permission denied, etc.)
  • The file content is not valid YAML or JSON
  • The manifest fails schema validation

func LoadFromBytes 

func LoadFromBytes(data []byte, path string) (*Manifest, error)

LoadFromBytes parses and validates a manifest from raw bytes.

The path parameter is used for error messages and format detection. If path is empty, format detection falls back to trying YAML first.

Validation is performed on the raw data (converted to JSON) before parsing into the typed struct. This ensures strict validation including rejection of unknown fields (additionalProperties: false in the schema).

func LoadFromReader 

func LoadFromReader(r io.Reader, path string) (*Manifest, error)

LoadFromReader reads and validates a manifest from an io.Reader.

The path parameter is used for error messages and format detection. If path is empty, format detection falls back to trying YAML first.

func (*Manifest) ApplyDefaults 

func (m *Manifest) ApplyDefaults()

ApplyDefaults fills in default values for optional fields.

This should be called after loading and validating the manifest to ensure all optional fields have sensible values.

type MatchConfig 

type MatchConfig struct {
	// Includes is a list of glob patterns for objects to include.
	// At least one pattern is required.
	Includes []string `json:"includes" yaml:"includes"`

	// Excludes is a list of glob patterns for objects to exclude. Optional.
	Excludes []string `json:"excludes,omitempty" yaml:"excludes,omitempty"`

	// IncludeHidden includes hidden files (starting with .). Default: false.
	IncludeHidden bool `json:"include_hidden,omitempty" yaml:"include_hidden,omitempty"`

	// Filters specifies additional metadata-based filters. Optional.
	// Filters are applied after glob pattern matching with AND semantics.
	Filters *FilterConfig `json:"filters,omitempty" yaml:"filters,omitempty"`
}

MatchConfig configures object filtering by glob patterns and metadata filters.

type OutputConfig 

type OutputConfig struct {
	// Destination is the output target.
	// Values: "stdout" or "file:/path/to/output.jsonl"
	// Default: "stdout".
	Destination string `json:"destination,omitempty" yaml:"destination,omitempty"`

	// Progress enables progress record emission during crawl.
	// Default: true.
	Progress *bool `json:"progress,omitempty" yaml:"progress,omitempty"`
}

OutputConfig configures output destination and format.

All fields are optional with sensible defaults applied during loading.

func (*OutputConfig) ProgressEnabled 

func (o *OutputConfig) ProgressEnabled() bool

ProgressEnabled returns whether progress records should be emitted. Returns the configured value, or DefaultProgress if not set.

type PathDateConfig 

type PathDateConfig struct {
	// Method is the extraction method (regex or segment).
	Method string `json:"method,omitempty" yaml:"method,omitempty"`

	// Regex is the regex pattern for extracting date from key path.
	Regex string `json:"regex,omitempty" yaml:"regex,omitempty"`

	// SegmentIndex is the zero-indexed segment number containing date.
	SegmentIndex int `json:"segment_index,omitempty" yaml:"segment_index,omitempty"`
}

PathDateConfig configures to date extraction from object key paths.

type PreflightConfig 

type PreflightConfig struct {
	Mode          string `json:"mode,omitempty" yaml:"mode,omitempty"`
	ProbeStrategy string `json:"probe_strategy,omitempty" yaml:"probe_strategy,omitempty"`
	ProbePrefix   string `json:"probe_prefix,omitempty" yaml:"probe_prefix,omitempty"`
}

PreflightConfig controls how aggressively gonimbus probes permissions.

Preflight is a capability contract, not a data operation. - plan-only: no provider calls - read-safe: no writes/deletes - write-probe: explicit opt-in minimal side effects (for transfer operations)

Note: write-probe fields are included for schema consistency, but crawl jobs should generally use plan-only/read-safe.

Values are schema-validated.

type ShardingConfig 

type ShardingConfig struct {
	Enabled         bool   `json:"enabled,omitempty" yaml:"enabled,omitempty"`
	Depth           int    `json:"depth,omitempty" yaml:"depth,omitempty"`
	MaxShards       int    `json:"max_shards,omitempty" yaml:"max_shards,omitempty"`
	ListConcurrency int    `json:"list_concurrency,omitempty" yaml:"list_concurrency,omitempty"`
	Delimiter       string `json:"delimiter,omitempty" yaml:"delimiter,omitempty"`
}

ShardingConfig configures delimiter-based prefix sharding for enumeration.

This allows large buckets to be enumerated in parallel without relying on a single flat list stream.

type SizeFilterConfig 

type SizeFilterConfig struct {
	// Min is the minimum size (inclusive).
	// Supports: raw bytes "1024", base-10 "1KB", base-2 "1KiB".
	Min string `json:"min,omitempty" yaml:"min,omitempty"`

	// Max is the maximum size (inclusive).
	Max string `json:"max,omitempty" yaml:"max,omitempty"`
}

SizeFilterConfig specifies size constraints.

type TransferConfig 

type TransferConfig struct {
	// Mode controls transfer semantics.
	// Values: "copy" or "move".
	Mode string `json:"mode,omitempty" yaml:"mode,omitempty"`

	// Concurrency controls concurrent transfer workers.
	Concurrency int `json:"concurrency,omitempty" yaml:"concurrency,omitempty"`

	// OnExists controls behavior when the target key already exists.
	// Values: "skip" | "overwrite" | "fail".
	OnExists string `json:"on_exists,omitempty" yaml:"on_exists,omitempty"`

	// Dedup configures deduplication.
	Dedup DedupConfig `json:"dedup,omitempty" yaml:"dedup,omitempty"`

	// PathTemplate optionally maps source keys to target keys.
	PathTemplate string `json:"path_template,omitempty" yaml:"path_template,omitempty"`

	// Sharding configures delimiter-based prefix sharding for enumeration.
	Sharding ShardingConfig `json:"sharding,omitempty" yaml:"sharding,omitempty"`

	// Preflight configures permission checks and provider probes.
	Preflight PreflightConfig `json:"preflight,omitempty" yaml:"preflight,omitempty"`
}

TransferConfig configures copy/move behavior.

type TransferManifest 

type TransferManifest struct {
	// Schema is an optional JSON Schema reference for editor support.
	Schema string `json:"$schema,omitempty" yaml:"$schema,omitempty"`

	// Version is the manifest schema version. Must be "1.0".
	Version string `json:"version" yaml:"version"`

	// Source configures the source storage provider.
	Source ConnectionConfig `json:"source" yaml:"source"`

	// Target configures the target storage provider.
	Target ConnectionConfig `json:"target" yaml:"target"`

	// Match configures which source objects are eligible for transfer.
	Match MatchConfig `json:"match" yaml:"match"`

	// Transfer configures transfer behavior.
	Transfer TransferConfig `json:"transfer" yaml:"transfer"`

	// Output configures output destination and format.
	Output OutputConfig `json:"output,omitempty" yaml:"output,omitempty"`
}

TransferManifest represents a validated transfer job manifest.

This is the contract for multi-step copy/move operations. Fields are schema-validated using the embedded transfer-manifest schema.

func LoadTransferFromBytes 

func LoadTransferFromBytes(data []byte, path string) (*TransferManifest, error)

LoadTransferFromBytes parses and validates a transfer manifest from raw bytes.

func (*TransferManifest) ApplyDefaults 

func (m *TransferManifest) ApplyDefaults()

ApplyDefaults fills in default values for optional fields.

type ValidationError 

type ValidationError struct {
	// Path is the JSON pointer to the problematic field (e.g., "/match/includes").
	Path string

	// Message describes the validation failure.
	Message string
}

ValidationError represents a single validation issue.

func (ValidationError) Error 

func (e ValidationError) Error() string

Error implements error interface.

type ValidationErrors 

type ValidationErrors []ValidationError

ValidationErrors is a collection of validation errors.

func (ValidationErrors) Error 

func (e ValidationErrors) Error() string

Error implements error interface.

func (ValidationErrors) Unwrap 

func (e ValidationErrors) Unwrap() error

Unwrap returns the underlying error type.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.