datasync

package
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 3, 2026 License: MIT Imports: 21 Imported by: 0

README

DataSync

Parity grade: A · SDK aws-sdk-go-v2/service/datasync@v1.59.2 · last audited 2026-07-23 (8379d347)

Coverage

Metric Value
Operations audited 53 (53 ok)
Feature families 5 (5 ok)
Known gaps 6
Deferred items 1
Resource leaks clean
Known gaps
  • LocationUri scheme prefixes for ObjectStorage ("object-storage://") and AzureBlob ("azure-blob://") technically violate AWS's own published LocationUri pattern (^(efs|nfs|s3|smb|hdfs|fsx[a-z0-9-]+)://...$, identical text on every DescribeLocation*Output doc page including these two), same as the now-fixed Lustre/ONTAP bugs -- but no positive evidence exists for what AWS actually returns for these two location types (both are comparatively recent additions; the shared regex may itself be stale doc-generation cruft that predates them and isn't enforced server-side for newer types, unlike the FSx family where the regex was clearly extended on purpose to add the fsx[a-z0-9-]+ alternative). Left unchanged pending confirmation via real AWS output or LocalStack; do not "fix" to a guessed scheme without evidence.
  • LocationUri scheme "fsxl://" for FSx Lustre (fixed this sweep from the confirmed-wrong "lustre://") was chosen by analogy with FSx OpenZFS's confirmed "fsxz://" (real AWS CLI doc example: fsxz://us-west-2.fs-.../fsx/folderA/folder) but is not independently confirmed against real AWS output. Medium confidence: matches the regex, matches the single-letter-suffix convention, but Lustre could plausibly use a different fsx-prefixed string.
  • CancelTaskExecution succeeds unconditionally, including on an already-terminal (SUCCESS/ERROR) execution, overwriting its terminal status with ERROR. Real AWS likely rejects cancelling a finished execution, but the exact error behavior was not confirmed and existing test coverage (TestDataSync_TaskExecution) exercises cancel-after-success expecting a 200. Left unfixed pending confirmation of the real error contract (unchanged from prior sweep).
  • Managed-secret config fields (CmkSecretConfig/CustomSecretConfig/ManagedSecretConfig) present on the real DescribeLocation*Output for Smb/Hdfs/ObjectStorage/AzureBlob/FsxWindows are not modeled at all -- gopherstack only supports the plaintext Password/SecretKey/SasToken credential flow, not the KMS/Secrets-Manager-backed secret-config flow. Honestly omitted from output (never fabricated) rather than implemented; a real feature gap for a future sweep.
  • SMB Kerberos authentication is not modeled: AuthenticationType now accepts/echoes "KERBEROS" as a string (added this sweep), but the corresponding real fields (KerberosPrincipal, DnsIpAddresses on DescribeLocationSmbOutput; KerberosKeytab-equivalent secret config) are not collected or returned -- only the NTLM (User/Password) flow is functionally implemented.
  • DescribeTaskOutput omits ErrorCode, ErrorDetail, DestinationNetworkInterfaceArns, SourceNetworkInterfaceArns, and ScheduleDetails, all present on the real output. gopherstack doesn't simulate task-execution failures or ENI provisioning, so these would always be empty/absent -- an honest omission rather than a fabricated value, but a gap if a future sweep adds failure simulation.
Deferred
  • StorageSystem / DiscoveryJob / TaskReport-as-standalone-ops: not present in aws-sdk-go-v2/service/datasync v1.59.2 at all (re-verified this sweep: full api_op_.go listing has no such ops -- grepped for storagesystem/discoveryjob/taskreport case-insensitively across services/datasync/.go, zero matches). TaskReportConfig itself IS a real member of CreateTaskInput/UpdateTaskInput/DescribeTaskOutput and is now modeled (see Task family fix) -- it's a configuration blob on the task, not a standalone set of ops. The family mentioned in the audit brief (dedicated StorageSystem/DiscoveryJob CRUD + Start/Stop/DescribeDiscoveryJob ops) does not exist in the pinned SDK.

More

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrNotFound is returned when a resource does not exist.
	ErrNotFound = awserr.New(resourceNotFoundType, awserr.ErrNotFound)
	// ErrAlreadyExists is returned when a resource already exists.
	ErrAlreadyExists = awserr.New(conflictExceptionType, awserr.ErrAlreadyExists)
	// ErrInvalidParameter is returned for invalid input.
	ErrInvalidParameter = awserr.New(invalidRequestType, awserr.ErrInvalidParameter)
)
View Source
var ErrNilAppContext = errors.New("datasync: nil app context")

ErrNilAppContext is returned when Init is called with a nil AppContext.

Functions

This section is empty.

Types

type Agent

type Agent struct {
	CreationTime time.Time
	Tags         map[string]string
	AgentArn     string
	Name         string
	Status       string
	EndpointType string
}

Agent represents a DataSync agent. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type AgentListEntry

type AgentListEntry struct {
	AgentArn string
	Name     string
	Status   string
}

AgentListEntry is an agent entry in a list response.

type Ec2Config

type Ec2Config struct {
	SubnetArn         string
	SecurityGroupArns []string
}

Ec2Config holds EC2 configuration for an EFS location.

type FilterRule added in v1.2.0

type FilterRule struct {
	FilterType string
	Value      string
}

FilterRule is a DataSync include/exclude filter (SIMPLE_PATTERN rules only).

type FsxNfsProtocol

type FsxNfsProtocol struct {
	MountOptions *MountOptions
}

FsxNfsProtocol holds FSx NFS protocol configuration.

type FsxProtocol

type FsxProtocol struct {
	NFS *FsxNfsProtocol
	SMB *FsxSmbProtocol
}

FsxProtocol holds FSx protocol configuration (NFS or SMB).

type FsxSmbProtocol

type FsxSmbProtocol struct {
	MountOptions *MountOptions
	Domain       string
	Password     string
	User         string
}

FsxSmbProtocol holds FSx SMB protocol configuration.

type Handler

type Handler struct {
	Backend StorageBackend
	// contains filtered or unexported fields
}

Handler handles DataSync HTTP requests.

func NewHandler

func NewHandler(b StorageBackend) *Handler

NewHandler constructs a new Handler.

func (*Handler) ExtractOperation

func (h *Handler) ExtractOperation(c *echo.Context) string

ExtractOperation extracts the operation name from the X-Amz-Target header.

func (*Handler) ExtractResource

func (h *Handler) ExtractResource(_ *echo.Context) string

ExtractResource extracts the resource identifier from the request body.

func (*Handler) GetSupportedOperations

func (h *Handler) GetSupportedOperations() []string

GetSupportedOperations returns the list of supported operations.

func (*Handler) Handler

func (h *Handler) Handler() echo.HandlerFunc

Handler returns the Echo handler function for DataSync requests.

func (*Handler) MatchPriority

func (h *Handler) MatchPriority() int

MatchPriority returns the routing priority.

func (*Handler) Name

func (h *Handler) Name() string

Name returns the service name.

func (*Handler) Reset

func (h *Handler) Reset()

Reset resets the backend and rebuilds the dispatch table.

func (*Handler) Restore

func (h *Handler) Restore(ctx context.Context, data []byte) error

Restore implements persistence.Persistable by delegating to the backend. See the Snapshot doc comment above for why this delegation is new.

func (*Handler) RouteMatcher

func (h *Handler) RouteMatcher() service.Matcher

RouteMatcher returns a function that matches DataSync API requests.

func (*Handler) Snapshot

func (h *Handler) Snapshot(ctx context.Context) []byte

Snapshot implements persistence.Persistable by delegating to the backend.

Prior to Phase 3.3, Handler had no Snapshot/Restore of its own even though InMemoryBackend implemented both (dead wiring: nothing ever called them). This delegation is what actually wires DataSync into the persistence Manager.

type HdfsNameNode

type HdfsNameNode struct {
	Hostname string
	Port     int32
}

HdfsNameNode is an HDFS name node endpoint.

type InMemoryBackend

type InMemoryBackend struct {
	// contains filtered or unexported fields
}

InMemoryBackend implements StorageBackend using pkgs/store tables.

agents, locations, and tasks are flat ARN-keyed resources with globally unique identity, so each is registered directly on b.registry (see store_setup.go). executions is likewise keyed by its own globally-unique TaskExecutionArn (which embeds the owning TaskArn), so it too is a direct registration; the former taskArn → executionArn → execution nesting is replaced by the executionsByTask secondary index, additively derived from TaskExecutionArn via extractTaskArnFromExecution. tags is left as a raw map because its values are plain string maps, not *T.

func NewInMemoryBackend

func NewInMemoryBackend(accountID, region string) *InMemoryBackend

NewInMemoryBackend constructs a new InMemoryBackend.

func (*InMemoryBackend) AccountID

func (b *InMemoryBackend) AccountID() string

AccountID returns the account ID.

func (*InMemoryBackend) CancelTaskExecution

func (b *InMemoryBackend) CancelTaskExecution(taskExecutionArn string) error

CancelTaskExecution cancels a running task execution.

AWS has no CANCELLED task-execution status (see types.TaskExecutionStatus); a cancelled execution settles into ERROR, matching the documented "some transient states... interrupt a task execution" behavior. Unlike a truly "current" (actively running) marker, CurrentTaskExecutionArn on the parent task is documented as "the ARN of the most recent task execution", so it is left pointing at the cancelled execution rather than cleared.

func (*InMemoryBackend) CreateAgent

func (b *InMemoryBackend) CreateAgent(name, _ string, tags map[string]string) (*Agent, error)

CreateAgent creates a new DataSync agent.

func (*InMemoryBackend) CreateLocationAzureBlob

func (b *InMemoryBackend) CreateLocationAzureBlob(
	containerURL, subdirectory, blobType, accessTier, authenticationType string,
	sasConfig *SasConfiguration,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationEfs

func (b *InMemoryBackend) CreateLocationEfs(
	efsFilesystemArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
	ec2Config *Ec2Config,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxLustre

func (b *InMemoryBackend) CreateLocationFsxLustre(
	fsxFilesystemArn, subdirectory string,
	securityGroupArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxOntap

func (b *InMemoryBackend) CreateLocationFsxOntap(
	storageVirtualMachineArn, subdirectory string,
	protocol *FsxProtocol,
	securityGroupArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxOpenZfs

func (b *InMemoryBackend) CreateLocationFsxOpenZfs(
	fsxFilesystemArn, subdirectory string,
	protocol *FsxProtocol,
	securityGroupArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxWindows

func (b *InMemoryBackend) CreateLocationFsxWindows(
	fsxFilesystemArn, subdirectory, domain, user, password string,
	securityGroupArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationHdfs

func (b *InMemoryBackend) CreateLocationHdfs(
	subdirectory, authenticationType, simpleUser string,
	kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
	nameNodes []HdfsNameNode,
	blockSize int64,
	replicationFactor int32,
	qopConfig *QopConfiguration,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationNfs

func (b *InMemoryBackend) CreateLocationNfs(
	serverHostname, subdirectory string,
	mountOptions *MountOptions,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationObjectStorage

func (b *InMemoryBackend) CreateLocationObjectStorage(
	serverHostname, serverProtocol, bucketName, subdirectory, accessKey, secretKey string,
	serverPort int32,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationS3

func (b *InMemoryBackend) CreateLocationS3(
	subdirectory, s3BucketArn, s3StorageClass string,
	s3Config S3Config,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

CreateLocationS3 creates a new S3 location.

func (*InMemoryBackend) CreateLocationSmb

func (b *InMemoryBackend) CreateLocationSmb(
	serverHostname, subdirectory, domain, user, password, authenticationType string,
	mountOptions *MountOptions,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateTask

func (b *InMemoryBackend) CreateTask(
	sourceLocationArn, destinationLocationArn, name, cloudWatchLogGroupArn string,
	settings TaskSettings,
	tags map[string]string,
) (*Task, error)

CreateTask creates a new DataSync task.

func (*InMemoryBackend) DeleteAgent

func (b *InMemoryBackend) DeleteAgent(agentArn string) error

DeleteAgent deletes an agent.

func (*InMemoryBackend) DeleteLocation

func (b *InMemoryBackend) DeleteLocation(locationArn string) error

DeleteLocation deletes a location.

func (*InMemoryBackend) DeleteTask

func (b *InMemoryBackend) DeleteTask(taskArn string) error

DeleteTask deletes a task.

func (*InMemoryBackend) DescribeAgent

func (b *InMemoryBackend) DescribeAgent(agentArn string) (*Agent, error)

DescribeAgent returns agent details.

func (*InMemoryBackend) DescribeLocationAzureBlob

func (b *InMemoryBackend) DescribeLocationAzureBlob(locationArn string) (*LocationAzureBlob, error)

func (*InMemoryBackend) DescribeLocationEfs

func (b *InMemoryBackend) DescribeLocationEfs(locationArn string) (*LocationEfs, error)

func (*InMemoryBackend) DescribeLocationFsxLustre

func (b *InMemoryBackend) DescribeLocationFsxLustre(locationArn string) (*LocationFsxLustre, error)

func (*InMemoryBackend) DescribeLocationFsxOntap

func (b *InMemoryBackend) DescribeLocationFsxOntap(locationArn string) (*LocationFsxOntap, error)

func (*InMemoryBackend) DescribeLocationFsxOpenZfs

func (b *InMemoryBackend) DescribeLocationFsxOpenZfs(locationArn string) (*LocationFsxOpenZfs, error)

func (*InMemoryBackend) DescribeLocationFsxWindows

func (b *InMemoryBackend) DescribeLocationFsxWindows(locationArn string) (*LocationFsxWindows, error)

func (*InMemoryBackend) DescribeLocationHdfs

func (b *InMemoryBackend) DescribeLocationHdfs(locationArn string) (*LocationHdfs, error)

func (*InMemoryBackend) DescribeLocationNfs

func (b *InMemoryBackend) DescribeLocationNfs(locationArn string) (*LocationNfs, error)

func (*InMemoryBackend) DescribeLocationObjectStorage

func (b *InMemoryBackend) DescribeLocationObjectStorage(locationArn string) (*LocationObjectStorage, error)

func (*InMemoryBackend) DescribeLocationS3

func (b *InMemoryBackend) DescribeLocationS3(locationArn string) (*LocationS3, error)

DescribeLocationS3 returns S3 location details.

func (*InMemoryBackend) DescribeLocationSmb

func (b *InMemoryBackend) DescribeLocationSmb(locationArn string) (*LocationSmb, error)

func (*InMemoryBackend) DescribeTask

func (b *InMemoryBackend) DescribeTask(taskArn string) (*Task, error)

DescribeTask returns task details.

func (*InMemoryBackend) DescribeTaskExecution

func (b *InMemoryBackend) DescribeTaskExecution(taskExecutionArn string) (*TaskExecution, error)

DescribeTaskExecution returns task execution details. Executions in LAUNCHING state are lazily advanced to SUCCESS on first describe. When the execution that finishes is still the task's current one, the parent task's Status reverts from RUNNING to AVAILABLE, matching AWS (task Status is RUNNING only while a task execution is in progress).

func (*InMemoryBackend) ListAgents

func (b *InMemoryBackend) ListAgents(maxResults int32, nextToken string) ([]*AgentListEntry, string, error)

ListAgents returns agents, sorted by ARN.

func (*InMemoryBackend) ListLocations

func (b *InMemoryBackend) ListLocations(maxResults int32, nextToken string) ([]*LocationListEntry, string, error)

ListLocations returns locations, sorted by ARN.

func (*InMemoryBackend) ListTagsForResource

func (b *InMemoryBackend) ListTagsForResource(
	resourceArn string,
	maxResults int32,
	nextToken string,
) (map[string]string, string, error)

ListTagsForResource returns tags for a resource with pagination.

func (*InMemoryBackend) ListTaskExecutions

func (b *InMemoryBackend) ListTaskExecutions(
	taskArn string,
	maxResults int32,
	nextToken string,
) ([]*TaskExecutionListEntry, string, error)

ListTaskExecutions returns executions for a task (or, when taskArn is empty, every execution across all tasks -- TaskArn is an optional filter per the real ListTaskExecutions API), sorted by ARN.

func (*InMemoryBackend) ListTasks

func (b *InMemoryBackend) ListTasks(maxResults int32, nextToken string) ([]*TaskListEntry, string, error)

ListTasks returns tasks, sorted by ARN.

func (*InMemoryBackend) Region

func (b *InMemoryBackend) Region() string

Region returns the region.

func (*InMemoryBackend) Reset

func (b *InMemoryBackend) Reset()

Reset clears all state.

func (*InMemoryBackend) Restore

func (b *InMemoryBackend) Restore(ctx context.Context, data []byte) error

Restore deserializes backend state from a snapshot.

func (*InMemoryBackend) Snapshot

func (b *InMemoryBackend) Snapshot(ctx context.Context) []byte

Snapshot serializes the backend state to JSON.

func (*InMemoryBackend) StartTaskExecution

func (b *InMemoryBackend) StartTaskExecution(taskArn string) (*TaskExecution, error)

StartTaskExecution starts a new task execution.

AWS allows only one task execution in progress per task at a time ("For each task, you can only run one task execution at a time.", per the StartTaskExecution API docs); attempting to start another while one is still running is rejected as an invalid request.

func (*InMemoryBackend) TagResource

func (b *InMemoryBackend) TagResource(resourceArn string, tags map[string]string) error

TagResource adds or updates tags on a resource.

func (*InMemoryBackend) UntagResource

func (b *InMemoryBackend) UntagResource(resourceArn string, keys []string) error

UntagResource removes tags from a resource.

func (*InMemoryBackend) UpdateAgent

func (b *InMemoryBackend) UpdateAgent(agentArn, name string) error

UpdateAgent updates the agent's name.

func (*InMemoryBackend) UpdateLocationAzureBlob

func (b *InMemoryBackend) UpdateLocationAzureBlob(
	locationArn, containerURL, subdirectory, blobType, accessTier, authenticationType string,
	sasConfig *SasConfiguration,
	agentArns []string,
) error

func (*InMemoryBackend) UpdateLocationEfs

func (b *InMemoryBackend) UpdateLocationEfs(
	locationArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
	ec2Config *Ec2Config,
) error

func (*InMemoryBackend) UpdateLocationFsxLustre

func (b *InMemoryBackend) UpdateLocationFsxLustre(locationArn, subdirectory string) error

func (*InMemoryBackend) UpdateLocationFsxOntap

func (b *InMemoryBackend) UpdateLocationFsxOntap(locationArn, subdirectory string, protocol *FsxProtocol) error

func (*InMemoryBackend) UpdateLocationFsxOpenZfs

func (b *InMemoryBackend) UpdateLocationFsxOpenZfs(locationArn, subdirectory string, protocol *FsxProtocol) error

func (*InMemoryBackend) UpdateLocationFsxWindows

func (b *InMemoryBackend) UpdateLocationFsxWindows(locationArn, subdirectory, domain, user, password string) error

func (*InMemoryBackend) UpdateLocationHdfs

func (b *InMemoryBackend) UpdateLocationHdfs(
	locationArn, subdirectory, authenticationType, simpleUser string,
	kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
	nameNodes []HdfsNameNode,
	blockSize int64,
	replicationFactor int32,
	qopConfig *QopConfiguration,
	agentArns []string,
) error

func (*InMemoryBackend) UpdateLocationNfs

func (b *InMemoryBackend) UpdateLocationNfs(
	locationArn, subdirectory string,
	mountOptions *MountOptions,
	agentArns []string,
) error

func (*InMemoryBackend) UpdateLocationObjectStorage

func (b *InMemoryBackend) UpdateLocationObjectStorage(
	locationArn, serverProtocol, subdirectory, accessKey, secretKey string,
	serverPort int32,
	agentArns []string,
) error

func (*InMemoryBackend) UpdateLocationS3

func (b *InMemoryBackend) UpdateLocationS3(locationArn, subdirectory, s3StorageClass string, s3Config S3Config) error

UpdateLocationS3 updates an S3 location's subdirectory, storage class, and S3 config.

func (*InMemoryBackend) UpdateLocationSmb

func (b *InMemoryBackend) UpdateLocationSmb(
	locationArn, subdirectory, domain, user, password, authenticationType string,
	mountOptions *MountOptions,
	agentArns []string,
) error

func (*InMemoryBackend) UpdateTask

func (b *InMemoryBackend) UpdateTask(taskArn, name, cloudWatchLogGroupArn string, settings TaskSettings) error

UpdateTask updates a task's mutable settings. AWS's UpdateTaskInput has no TaskArn-only "clear the log group" convention like some other fields, so CloudWatchLogGroupArn is always assigned (empty clears it, matching prior behavior). Options/Excludes/Includes/Schedule/ManifestConfig/ TaskReportConfig follow AWS's "only supplied fields change" semantics: a nil field in settings means "not supplied, leave unchanged"; a non-nil (possibly empty) field means "set to this value" -- which also covers the documented "specify this parameter as empty to remove" behavior for ManifestConfig/TaskReportConfig, since an explicit empty map is non-nil.

func (*InMemoryBackend) UpdateTaskExecution

func (b *InMemoryBackend) UpdateTaskExecution(taskExecutionArn string, options map[string]any) error

UpdateTaskExecution updates a task execution (no-op: options are advisory only).

type Location

type Location struct {
	CreationTime time.Time
	LocationArn  string
	LocationURI  string
}

Location is a generic DataSync location. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationAzureBlob

type LocationAzureBlob struct {
	CreationTime       time.Time
	SasConfiguration   *SasConfiguration
	LocationArn        string
	LocationURI        string
	ContainerURL       string
	BlobType           string
	AccessTier         string
	AuthenticationType string
	Subdirectory       string
	AgentArns          []string
}

LocationAzureBlob is a DataSync Azure Blob location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationEfs

type LocationEfs struct {
	CreationTime            time.Time
	Ec2Config               *Ec2Config
	LocationArn             string
	LocationURI             string
	EfsFilesystemArn        string
	AccessPointArn          string
	FileSystemAccessRoleArn string
	InTransitEncryption     string
	Subdirectory            string
}

LocationEfs is a DataSync EFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxLustre

type LocationFsxLustre struct {
	CreationTime      time.Time
	LocationArn       string
	LocationURI       string
	FsxFilesystemArn  string
	Subdirectory      string
	SecurityGroupArns []string
}

LocationFsxLustre is a DataSync FSx Lustre location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxOntap

type LocationFsxOntap struct {
	CreationTime             time.Time
	Protocol                 *FsxProtocol
	LocationArn              string
	LocationURI              string
	StorageVirtualMachineArn string
	FsxFilesystemArn         string
	Subdirectory             string
	SecurityGroupArns        []string
}

LocationFsxOntap is a DataSync FSx ONTAP location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxOpenZfs

type LocationFsxOpenZfs struct {
	CreationTime      time.Time
	Protocol          *FsxProtocol
	LocationArn       string
	LocationURI       string
	FsxFilesystemArn  string
	Subdirectory      string
	SecurityGroupArns []string
}

LocationFsxOpenZfs is a DataSync FSx OpenZFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxWindows

type LocationFsxWindows struct {
	CreationTime      time.Time
	LocationArn       string
	LocationURI       string
	FsxFilesystemArn  string
	Domain            string
	User              string
	Subdirectory      string
	SecurityGroupArns []string
}

LocationFsxWindows is a DataSync FSx Windows location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationHdfs

type LocationHdfs struct {
	CreationTime       time.Time
	QopConfiguration   *QopConfiguration
	KmsKeyProviderURI  string
	LocationArn        string
	LocationURI        string
	AuthenticationType string
	SimpleUser         string
	KerberosPrincipal  string
	Subdirectory       string
	AgentArns          []string
	NameNodes          []HdfsNameNode
	BlockSize          int64
	ReplicationFactor  int32
}

LocationHdfs is a DataSync HDFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationListEntry

type LocationListEntry struct {
	CreationTime time.Time
	LocationArn  string
	LocationURI  string
}

LocationListEntry is a location entry in a list response.

type LocationNfs

type LocationNfs struct {
	CreationTime   time.Time
	MountOptions   *MountOptions
	LocationArn    string
	LocationURI    string
	ServerHostname string
	Subdirectory   string
	AgentArns      []string
}

LocationNfs is a DataSync NFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationObjectStorage

type LocationObjectStorage struct {
	CreationTime   time.Time
	LocationArn    string
	LocationURI    string
	ServerHostname string
	ServerProtocol string
	BucketName     string
	AccessKey      string
	Subdirectory   string
	AgentArns      []string
	ServerPort     int32
}

LocationObjectStorage is a DataSync object storage location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationS3

type LocationS3 struct {
	CreationTime   time.Time
	S3Config       S3Config
	LocationArn    string
	LocationURI    string
	S3BucketArn    string
	Subdirectory   string
	S3StorageClass string
	AgentArns      []string
}

LocationS3 is a DataSync S3 location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationSmb

type LocationSmb struct {
	CreationTime       time.Time
	MountOptions       *MountOptions
	LocationArn        string
	LocationURI        string
	ServerHostname     string
	Domain             string
	User               string
	AuthenticationType string
	Subdirectory       string
	AgentArns          []string
}

LocationSmb is a DataSync SMB location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type MountOptions

type MountOptions struct {
	Version string
}

MountOptions holds NFS/SMB mount version.

type Provider

type Provider struct{}

Provider implements service.Provider for AWS DataSync.

func (*Provider) Init

Init initializes the DataSync service backend and handler.

func (*Provider) Name

func (p *Provider) Name() string

Name returns the provider name.

type QopConfiguration

type QopConfiguration struct {
	DataTransferProtection string
	RPCProtection          string
}

QopConfiguration holds HDFS QOP configuration.

type S3Config

type S3Config struct {
	BucketAccessRoleArn string
}

S3Config holds IAM role configuration for an S3 location.

type SasConfiguration

type SasConfiguration struct {
	Token string
}

SasConfiguration holds Azure Blob SAS token configuration.

type StorageBackend

type StorageBackend interface {
	// Agent operations
	CreateAgent(name string, activationKey string, tags map[string]string) (*Agent, error)
	DescribeAgent(agentArn string) (*Agent, error)
	UpdateAgent(agentArn, name string) error
	DeleteAgent(agentArn string) error
	ListAgents(maxResults int32, nextToken string) ([]*AgentListEntry, string, error)

	// Location operations (S3)
	CreateLocationS3(
		subdirectory, s3BucketArn, s3StorageClass string,
		s3Config S3Config,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationS3(locationArn string) (*LocationS3, error)
	DeleteLocation(locationArn string) error
	ListLocations(maxResults int32, nextToken string) ([]*LocationListEntry, string, error)

	// Task operations
	CreateTask(
		sourceLocationArn, destinationLocationArn, name, cloudWatchLogGroupArn string,
		settings TaskSettings,
		tags map[string]string,
	) (*Task, error)
	DescribeTask(taskArn string) (*Task, error)
	UpdateTask(taskArn, name, cloudWatchLogGroupArn string, settings TaskSettings) error
	DeleteTask(taskArn string) error
	ListTasks(maxResults int32, nextToken string) ([]*TaskListEntry, string, error)

	// Task execution operations
	StartTaskExecution(taskArn string) (*TaskExecution, error)
	CancelTaskExecution(taskExecutionArn string) error
	DescribeTaskExecution(taskExecutionArn string) (*TaskExecution, error)
	ListTaskExecutions(taskArn string, maxResults int32, nextToken string) ([]*TaskExecutionListEntry, string, error)

	// Location operations (S3 update)
	UpdateLocationS3(locationArn, subdirectory, s3StorageClass string, s3Config S3Config) error

	// Task execution update
	UpdateTaskExecution(taskExecutionArn string, options map[string]any) error

	// Location operations (Azure Blob)
	CreateLocationAzureBlob(
		containerURL, subdirectory, blobType, accessTier, authenticationType string,
		sasConfig *SasConfiguration,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationAzureBlob(locationArn string) (*LocationAzureBlob, error)
	UpdateLocationAzureBlob(
		locationArn, containerURL, subdirectory, blobType, accessTier, authenticationType string,
		sasConfig *SasConfiguration,
		agentArns []string,
	) error

	// Location operations (EFS)
	CreateLocationEfs(
		efsFilesystemArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
		ec2Config *Ec2Config,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationEfs(locationArn string) (*LocationEfs, error)
	UpdateLocationEfs(
		locationArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
		ec2Config *Ec2Config,
	) error

	// Location operations (FSx Lustre)
	CreateLocationFsxLustre(
		fsxFilesystemArn, subdirectory string,
		securityGroupArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationFsxLustre(locationArn string) (*LocationFsxLustre, error)
	UpdateLocationFsxLustre(locationArn, subdirectory string) error

	// Location operations (FSx ONTAP)
	CreateLocationFsxOntap(
		storageVirtualMachineArn, subdirectory string,
		protocol *FsxProtocol,
		securityGroupArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationFsxOntap(locationArn string) (*LocationFsxOntap, error)
	UpdateLocationFsxOntap(locationArn, subdirectory string, protocol *FsxProtocol) error

	// Location operations (FSx OpenZFS)
	CreateLocationFsxOpenZfs(
		fsxFilesystemArn, subdirectory string,
		protocol *FsxProtocol,
		securityGroupArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationFsxOpenZfs(locationArn string) (*LocationFsxOpenZfs, error)
	UpdateLocationFsxOpenZfs(locationArn, subdirectory string, protocol *FsxProtocol) error

	// Location operations (FSx Windows)
	CreateLocationFsxWindows(
		fsxFilesystemArn, subdirectory, domain, user, password string,
		securityGroupArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationFsxWindows(locationArn string) (*LocationFsxWindows, error)
	UpdateLocationFsxWindows(locationArn, subdirectory, domain, user, password string) error

	// Location operations (HDFS)
	CreateLocationHdfs(
		subdirectory, authenticationType, simpleUser string,
		kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
		nameNodes []HdfsNameNode,
		blockSize int64,
		replicationFactor int32,
		qopConfig *QopConfiguration,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationHdfs(locationArn string) (*LocationHdfs, error)
	UpdateLocationHdfs(
		locationArn, subdirectory, authenticationType, simpleUser string,
		kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
		nameNodes []HdfsNameNode,
		blockSize int64,
		replicationFactor int32,
		qopConfig *QopConfiguration,
		agentArns []string,
	) error

	// Location operations (NFS)
	CreateLocationNfs(
		serverHostname, subdirectory string,
		mountOptions *MountOptions,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationNfs(locationArn string) (*LocationNfs, error)
	UpdateLocationNfs(locationArn, subdirectory string, mountOptions *MountOptions, agentArns []string) error

	// Location operations (Object Storage)
	CreateLocationObjectStorage(
		serverHostname, serverProtocol, bucketName, subdirectory, accessKey, secretKey string,
		serverPort int32,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationObjectStorage(locationArn string) (*LocationObjectStorage, error)
	UpdateLocationObjectStorage(
		locationArn, serverProtocol, subdirectory, accessKey, secretKey string,
		serverPort int32,
		agentArns []string,
	) error

	// Location operations (SMB)
	CreateLocationSmb(
		serverHostname, subdirectory, domain, user, password, authenticationType string,
		mountOptions *MountOptions,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationSmb(locationArn string) (*LocationSmb, error)
	UpdateLocationSmb(
		locationArn, subdirectory, domain, user, password, authenticationType string,
		mountOptions *MountOptions,
		agentArns []string,
	) error

	// Tag operations
	TagResource(resourceArn string, tags map[string]string) error
	UntagResource(resourceArn string, keys []string) error
	ListTagsForResource(resourceArn string, maxResults int32, nextToken string) (map[string]string, string, error)

	AccountID() string
	Region() string
	Reset()
	Snapshot(ctx context.Context) []byte
	Restore(ctx context.Context, data []byte) error
}

StorageBackend is the interface for DataSync storage operations.

type Task

type Task struct {
	CreationTime            time.Time
	Tags                    map[string]string
	Options                 map[string]any
	Schedule                *TaskSchedule
	ManifestConfig          map[string]any
	TaskReportConfig        map[string]any
	TaskArn                 string
	Name                    string
	Status                  string
	SourceLocationArn       string
	DestinationLocationArn  string
	CloudWatchLogGroupArn   string
	CurrentTaskExecutionArn string
	TaskMode                string
	Excludes                []FilterRule
	Includes                []FilterRule
}

Task represents a DataSync transfer task. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type TaskExecution

type TaskExecution struct {
	StartTime                time.Time
	Options                  map[string]any
	TaskExecutionArn         string
	Status                   string
	EstimatedFilesToTransfer int64
	EstimatedBytesToTransfer int64
	FilesTransferred         int64
	BytesTransferred         int64
}

TaskExecution represents a DataSync task execution. StartTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type TaskExecutionListEntry

type TaskExecutionListEntry struct {
	TaskExecutionArn string
	Status           string
}

TaskExecutionListEntry is a task execution entry in a list response.

type TaskListEntry

type TaskListEntry struct {
	TaskArn string
	Name    string
	Status  string
}

TaskListEntry is a task entry in a list response.

type TaskSchedule added in v1.2.0

type TaskSchedule struct {
	ScheduleExpression string
	Status             string
}

TaskSchedule holds a task's cron/rate schedule expression and enabled status.

type TaskSettings added in v1.2.0

type TaskSettings struct {
	Options          map[string]any
	Schedule         *TaskSchedule
	ManifestConfig   map[string]any
	TaskReportConfig map[string]any
	TaskMode         string
	Excludes         []FilterRule
	Includes         []FilterRule
}

TaskSettings groups the optional CreateTask/UpdateTask configuration knobs that mirror AWS's own grouping in CreateTaskInput/UpdateTaskInput (Options, Schedule, Excludes, Includes, ManifestConfig, TaskReportConfig, TaskMode). A nil/zero field means "not supplied" (leave unchanged on UpdateTask); Options/ManifestConfig/TaskReportConfig are opaque pass-through blobs (echoed back verbatim on Describe) since their AWS shapes are deep and advisory rather than FK-validated.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL