datasync

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 19, 2026 License: MIT Imports: 21 Imported by: 0

README

DataSync

Parity grade: A · SDK aws-sdk-go-v2/service/datasync@v1.59.2 · last audited 2026-07-12 (8379d347)

Coverage

Operations audited 53 (42 ok, 11 partial)
Feature families 5 (5 ok)
Known gaps 5
Deferred items 1
Resource leaks clean
Known gaps
  • DescribeLocation* outputs (all types except AzureBlob, fixed this sweep) include an extra Subdirectory field that does not exist on the real AWS wire (confirmed: grepped every DescribeLocation*Output struct in aws-sdk-go-v2/service/datasync v1.59.2 -- none have a Subdirectory member; the path is folded into LocationUri only). Harmless (real JSON-1.1 clients ignore unknown fields) but not wire-exact. Left unfixed this sweep to bound the diff to genuine-impact bugs; next audit should strip the field from the other 10 location output structs the same way AzureBlob's was fixed.
  • CreateLocationAzureBlob / UpdateLocationAzureBlob / DescribeLocationAzureBlob have no AuthenticationType field at all (real API: required on Create, returned on Describe). gopherstack only supports the SAS-token flow implicitly; adding real support requires plumbing a new field through the Create/Update input structs, storedAzureBlobConfig, the backend method signatures (interfaces.go), and the Describe output -- larger than a validation-only fix, deferred.
  • LocationUri scheme prefixes for FSx Lustre ("lustre://"), FSx ONTAP ("ontap://"), FSx Windows ("smb://"), and several other location types were NOT independently confirmed against real AWS output (only S3 "s3://" and FSx OpenZFS "fsxz://" were confirmed via SDK/doc evidence this sweep, and OpenZFS was fixed from "openzfs://"). The real scheme names for the others are plausible but unverified guesses inherited from before this audit -- worth confirming against actual aws datasync describe-location-fsx-* output or LocalStack's implementation in a future pass.
  • CancelTaskExecution succeeds unconditionally, including on an already-terminal (SUCCESS/ERROR) execution, overwriting its terminal status with ERROR. Real AWS likely rejects cancelling a finished execution (the API doc frames Cancel as stopping something "in progress"), but the exact error behavior was not confirmed and existing test coverage (TestDataSync_TaskExecution) exercises cancel-after-success expecting a 200. Left unfixed pending confirmation of the real error contract.
  • storedAgent.Tags is kept in sync by TagResource/UntagResource but storedLocation.Tags/storedTask.Tags are not (only the canonical b.tags map is updated for those). Harmless dead-code asymmetry since no Describe output reads *.Tags and ListTagsForResource sources from b.tags for every resource type, but worth cleaning up for consistency in a future pass.
Deferred
  • StorageSystem / DiscoveryJob operations: not present in aws-sdk-go-v2/service/datasync v1.59.2 at all (verified: full api_op_*.go listing has no such ops), so there is nothing to implement against this SDK version -- the family mentioned in the audit brief does not exist in the pinned SDK.

More

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrNotFound is returned when a resource does not exist.
	ErrNotFound = awserr.New(resourceNotFoundType, awserr.ErrNotFound)
	// ErrAlreadyExists is returned when a resource already exists.
	ErrAlreadyExists = awserr.New(conflictExceptionType, awserr.ErrAlreadyExists)
	// ErrInvalidParameter is returned for invalid input.
	ErrInvalidParameter = awserr.New(invalidRequestType, awserr.ErrInvalidParameter)
)
View Source
var ErrNilAppContext = errors.New("datasync: nil app context")

ErrNilAppContext is returned when Init is called with a nil AppContext.

Functions

This section is empty.

Types

type Agent

type Agent struct {
	CreationTime time.Time
	Tags         map[string]string
	AgentArn     string
	Name         string
	Status       string
	EndpointType string
}

Agent represents a DataSync agent. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type AgentListEntry

type AgentListEntry struct {
	AgentArn string
	Name     string
	Status   string
}

AgentListEntry is an agent entry in a list response.

type Ec2Config

type Ec2Config struct {
	SubnetArn         string
	SecurityGroupArns []string
}

Ec2Config holds EC2 configuration for an EFS location.

type FsxNfsProtocol

type FsxNfsProtocol struct {
	MountOptions *MountOptions
}

FsxNfsProtocol holds FSx NFS protocol configuration.

type FsxProtocol

type FsxProtocol struct {
	NFS *FsxNfsProtocol
	SMB *FsxSmbProtocol
}

FsxProtocol holds FSx protocol configuration (NFS or SMB).

type FsxSmbProtocol

type FsxSmbProtocol struct {
	MountOptions *MountOptions
	Domain       string
	Password     string
	User         string
}

FsxSmbProtocol holds FSx SMB protocol configuration.

type Handler

type Handler struct {
	Backend StorageBackend
	// contains filtered or unexported fields
}

Handler handles DataSync HTTP requests.

func NewHandler

func NewHandler(b StorageBackend) *Handler

NewHandler constructs a new Handler.

func (*Handler) ExtractOperation

func (h *Handler) ExtractOperation(c *echo.Context) string

ExtractOperation extracts the operation name from the X-Amz-Target header.

func (*Handler) ExtractResource

func (h *Handler) ExtractResource(_ *echo.Context) string

ExtractResource extracts the resource identifier from the request body.

func (*Handler) GetSupportedOperations

func (h *Handler) GetSupportedOperations() []string

GetSupportedOperations returns the list of supported operations.

func (*Handler) Handler

func (h *Handler) Handler() echo.HandlerFunc

Handler returns the Echo handler function for DataSync requests.

func (*Handler) MatchPriority

func (h *Handler) MatchPriority() int

MatchPriority returns the routing priority.

func (*Handler) Name

func (h *Handler) Name() string

Name returns the service name.

func (*Handler) Reset

func (h *Handler) Reset()

Reset resets the backend and rebuilds the dispatch table.

func (*Handler) Restore

func (h *Handler) Restore(ctx context.Context, data []byte) error

Restore implements persistence.Persistable by delegating to the backend. See the Snapshot doc comment above for why this delegation is new.

func (*Handler) RouteMatcher

func (h *Handler) RouteMatcher() service.Matcher

RouteMatcher returns a function that matches DataSync API requests.

func (*Handler) Snapshot

func (h *Handler) Snapshot(ctx context.Context) []byte

Snapshot implements persistence.Persistable by delegating to the backend.

Prior to Phase 3.3, Handler had no Snapshot/Restore of its own even though InMemoryBackend implemented both (dead wiring: nothing ever called them). This delegation is what actually wires DataSync into the persistence Manager.

type HdfsNameNode

type HdfsNameNode struct {
	Hostname string
	Port     int32
}

HdfsNameNode is an HDFS name node endpoint.

type InMemoryBackend

type InMemoryBackend struct {
	// contains filtered or unexported fields
}

InMemoryBackend implements StorageBackend using pkgs/store tables.

agents, locations, and tasks are flat ARN-keyed resources with globally unique identity, so each is registered directly on b.registry (see store_setup.go). executions is likewise keyed by its own globally-unique TaskExecutionArn (which embeds the owning TaskArn), so it too is a direct registration; the former taskArn → executionArn → execution nesting is replaced by the executionsByTask secondary index, additively derived from TaskExecutionArn via extractTaskArnFromExecution. tags is left as a raw map because its values are plain string maps, not *T.

func NewInMemoryBackend

func NewInMemoryBackend(accountID, region string) *InMemoryBackend

NewInMemoryBackend constructs a new InMemoryBackend.

func (*InMemoryBackend) AccountID

func (b *InMemoryBackend) AccountID() string

AccountID returns the account ID.

func (*InMemoryBackend) CancelTaskExecution

func (b *InMemoryBackend) CancelTaskExecution(taskExecutionArn string) error

CancelTaskExecution cancels a running task execution.

AWS has no CANCELLED task-execution status (see types.TaskExecutionStatus); a cancelled execution settles into ERROR, matching the documented "some transient states... interrupt a task execution" behavior. Unlike a truly "current" (actively running) marker, CurrentTaskExecutionArn on the parent task is documented as "the ARN of the most recent task execution", so it is left pointing at the cancelled execution rather than cleared.

func (*InMemoryBackend) CreateAgent

func (b *InMemoryBackend) CreateAgent(name, _ string, tags map[string]string) (*Agent, error)

CreateAgent creates a new DataSync agent.

func (*InMemoryBackend) CreateLocationAzureBlob

func (b *InMemoryBackend) CreateLocationAzureBlob(
	containerURL, subdirectory, blobType, accessTier string,
	sasConfig *SasConfiguration,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationEfs

func (b *InMemoryBackend) CreateLocationEfs(
	efsFilesystemArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
	ec2Config *Ec2Config,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxLustre

func (b *InMemoryBackend) CreateLocationFsxLustre(
	fsxFilesystemArn, subdirectory string,
	securityGroupArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxOntap

func (b *InMemoryBackend) CreateLocationFsxOntap(
	storageVirtualMachineArn, subdirectory string,
	protocol *FsxProtocol,
	securityGroupArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxOpenZfs

func (b *InMemoryBackend) CreateLocationFsxOpenZfs(
	fsxFilesystemArn, subdirectory string,
	protocol *FsxProtocol,
	securityGroupArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxWindows

func (b *InMemoryBackend) CreateLocationFsxWindows(
	fsxFilesystemArn, subdirectory, domain, user, password string,
	securityGroupArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationHdfs

func (b *InMemoryBackend) CreateLocationHdfs(
	subdirectory, authenticationType, simpleUser string,
	kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
	nameNodes []HdfsNameNode,
	blockSize int64,
	replicationFactor int32,
	qopConfig *QopConfiguration,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationNfs

func (b *InMemoryBackend) CreateLocationNfs(
	serverHostname, subdirectory string,
	mountOptions *MountOptions,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationObjectStorage

func (b *InMemoryBackend) CreateLocationObjectStorage(
	serverHostname, serverProtocol, bucketName, subdirectory, accessKey, secretKey string,
	serverPort int32,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationS3

func (b *InMemoryBackend) CreateLocationS3(
	subdirectory, s3BucketArn, s3StorageClass string,
	s3Config S3Config,
	tags map[string]string,
) (*Location, error)

CreateLocationS3 creates a new S3 location.

func (*InMemoryBackend) CreateLocationSmb

func (b *InMemoryBackend) CreateLocationSmb(
	serverHostname, subdirectory, domain, user, password string,
	mountOptions *MountOptions,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateTask

func (b *InMemoryBackend) CreateTask(
	sourceLocationArn, destinationLocationArn, name, cloudWatchLogGroupArn string,
	tags map[string]string,
) (*Task, error)

CreateTask creates a new DataSync task.

func (*InMemoryBackend) DeleteAgent

func (b *InMemoryBackend) DeleteAgent(agentArn string) error

DeleteAgent deletes an agent.

func (*InMemoryBackend) DeleteLocation

func (b *InMemoryBackend) DeleteLocation(locationArn string) error

DeleteLocation deletes a location.

func (*InMemoryBackend) DeleteTask

func (b *InMemoryBackend) DeleteTask(taskArn string) error

DeleteTask deletes a task.

func (*InMemoryBackend) DescribeAgent

func (b *InMemoryBackend) DescribeAgent(agentArn string) (*Agent, error)

DescribeAgent returns agent details.

func (*InMemoryBackend) DescribeLocationAzureBlob

func (b *InMemoryBackend) DescribeLocationAzureBlob(locationArn string) (*LocationAzureBlob, error)

func (*InMemoryBackend) DescribeLocationEfs

func (b *InMemoryBackend) DescribeLocationEfs(locationArn string) (*LocationEfs, error)

func (*InMemoryBackend) DescribeLocationFsxLustre

func (b *InMemoryBackend) DescribeLocationFsxLustre(locationArn string) (*LocationFsxLustre, error)

func (*InMemoryBackend) DescribeLocationFsxOntap

func (b *InMemoryBackend) DescribeLocationFsxOntap(locationArn string) (*LocationFsxOntap, error)

func (*InMemoryBackend) DescribeLocationFsxOpenZfs

func (b *InMemoryBackend) DescribeLocationFsxOpenZfs(locationArn string) (*LocationFsxOpenZfs, error)

func (*InMemoryBackend) DescribeLocationFsxWindows

func (b *InMemoryBackend) DescribeLocationFsxWindows(locationArn string) (*LocationFsxWindows, error)

func (*InMemoryBackend) DescribeLocationHdfs

func (b *InMemoryBackend) DescribeLocationHdfs(locationArn string) (*LocationHdfs, error)

func (*InMemoryBackend) DescribeLocationNfs

func (b *InMemoryBackend) DescribeLocationNfs(locationArn string) (*LocationNfs, error)

func (*InMemoryBackend) DescribeLocationObjectStorage

func (b *InMemoryBackend) DescribeLocationObjectStorage(locationArn string) (*LocationObjectStorage, error)

func (*InMemoryBackend) DescribeLocationS3

func (b *InMemoryBackend) DescribeLocationS3(locationArn string) (*LocationS3, error)

DescribeLocationS3 returns S3 location details.

func (*InMemoryBackend) DescribeLocationSmb

func (b *InMemoryBackend) DescribeLocationSmb(locationArn string) (*LocationSmb, error)

func (*InMemoryBackend) DescribeTask

func (b *InMemoryBackend) DescribeTask(taskArn string) (*Task, error)

DescribeTask returns task details.

func (*InMemoryBackend) DescribeTaskExecution

func (b *InMemoryBackend) DescribeTaskExecution(taskExecutionArn string) (*TaskExecution, error)

DescribeTaskExecution returns task execution details. Executions in LAUNCHING state are lazily advanced to SUCCESS on first describe. When the execution that finishes is still the task's current one, the parent task's Status reverts from RUNNING to AVAILABLE, matching AWS (task Status is RUNNING only while a task execution is in progress).

func (*InMemoryBackend) ListAgents

func (b *InMemoryBackend) ListAgents(maxResults int32, nextToken string) ([]*AgentListEntry, string, error)

ListAgents returns agents, sorted by ARN.

func (*InMemoryBackend) ListLocations

func (b *InMemoryBackend) ListLocations(maxResults int32, nextToken string) ([]*LocationListEntry, string, error)

ListLocations returns locations, sorted by ARN.

func (*InMemoryBackend) ListTagsForResource

func (b *InMemoryBackend) ListTagsForResource(
	resourceArn string,
	maxResults int32,
	nextToken string,
) (map[string]string, string, error)

ListTagsForResource returns tags for a resource with pagination.

func (*InMemoryBackend) ListTaskExecutions

func (b *InMemoryBackend) ListTaskExecutions(
	taskArn string,
	maxResults int32,
	nextToken string,
) ([]*TaskExecutionListEntry, string, error)

ListTaskExecutions returns executions for a task (or, when taskArn is empty, every execution across all tasks -- TaskArn is an optional filter per the real ListTaskExecutions API), sorted by ARN.

func (*InMemoryBackend) ListTasks

func (b *InMemoryBackend) ListTasks(maxResults int32, nextToken string) ([]*TaskListEntry, string, error)

ListTasks returns tasks, sorted by ARN.

func (*InMemoryBackend) Region

func (b *InMemoryBackend) Region() string

Region returns the region.

func (*InMemoryBackend) Reset

func (b *InMemoryBackend) Reset()

Reset clears all state.

func (*InMemoryBackend) Restore

func (b *InMemoryBackend) Restore(ctx context.Context, data []byte) error

Restore deserializes backend state from a snapshot.

func (*InMemoryBackend) Snapshot

func (b *InMemoryBackend) Snapshot(ctx context.Context) []byte

Snapshot serializes the backend state to JSON.

func (*InMemoryBackend) StartTaskExecution

func (b *InMemoryBackend) StartTaskExecution(taskArn string) (*TaskExecution, error)

StartTaskExecution starts a new task execution.

AWS allows only one task execution in progress per task at a time ("For each task, you can only run one task execution at a time.", per the StartTaskExecution API docs); attempting to start another while one is still running is rejected as an invalid request.

func (*InMemoryBackend) TagResource

func (b *InMemoryBackend) TagResource(resourceArn string, tags map[string]string) error

TagResource adds or updates tags on a resource.

func (*InMemoryBackend) UntagResource

func (b *InMemoryBackend) UntagResource(resourceArn string, keys []string) error

UntagResource removes tags from a resource.

func (*InMemoryBackend) UpdateAgent

func (b *InMemoryBackend) UpdateAgent(agentArn, name string) error

UpdateAgent updates the agent's name.

func (*InMemoryBackend) UpdateLocationAzureBlob

func (b *InMemoryBackend) UpdateLocationAzureBlob(
	locationArn, containerURL, subdirectory, blobType, accessTier string,
	sasConfig *SasConfiguration,
	agentArns []string,
) error

func (*InMemoryBackend) UpdateLocationEfs

func (b *InMemoryBackend) UpdateLocationEfs(
	locationArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
	ec2Config *Ec2Config,
) error

func (*InMemoryBackend) UpdateLocationFsxLustre

func (b *InMemoryBackend) UpdateLocationFsxLustre(locationArn, subdirectory string) error

func (*InMemoryBackend) UpdateLocationFsxOntap

func (b *InMemoryBackend) UpdateLocationFsxOntap(locationArn, subdirectory string, protocol *FsxProtocol) error

func (*InMemoryBackend) UpdateLocationFsxOpenZfs

func (b *InMemoryBackend) UpdateLocationFsxOpenZfs(locationArn, subdirectory string, protocol *FsxProtocol) error

func (*InMemoryBackend) UpdateLocationFsxWindows

func (b *InMemoryBackend) UpdateLocationFsxWindows(locationArn, subdirectory, domain, user, password string) error

func (*InMemoryBackend) UpdateLocationHdfs

func (b *InMemoryBackend) UpdateLocationHdfs(
	locationArn, subdirectory, authenticationType, simpleUser string,
	kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
	nameNodes []HdfsNameNode,
	blockSize int64,
	replicationFactor int32,
	qopConfig *QopConfiguration,
	agentArns []string,
) error

func (*InMemoryBackend) UpdateLocationNfs

func (b *InMemoryBackend) UpdateLocationNfs(
	locationArn, subdirectory string,
	mountOptions *MountOptions,
	agentArns []string,
) error

func (*InMemoryBackend) UpdateLocationObjectStorage

func (b *InMemoryBackend) UpdateLocationObjectStorage(
	locationArn, serverProtocol, subdirectory, accessKey, secretKey string,
	serverPort int32,
	agentArns []string,
) error

func (*InMemoryBackend) UpdateLocationS3

func (b *InMemoryBackend) UpdateLocationS3(locationArn, subdirectory, s3StorageClass string, s3Config S3Config) error

UpdateLocationS3 updates an S3 location's subdirectory, storage class, and S3 config.

func (*InMemoryBackend) UpdateLocationSmb

func (b *InMemoryBackend) UpdateLocationSmb(
	locationArn, subdirectory, domain, user, password string,
	mountOptions *MountOptions,
	agentArns []string,
) error

func (*InMemoryBackend) UpdateTask

func (b *InMemoryBackend) UpdateTask(taskArn, name, cloudWatchLogGroupArn string) error

UpdateTask updates the task's name and CloudWatch log group.

func (*InMemoryBackend) UpdateTaskExecution

func (b *InMemoryBackend) UpdateTaskExecution(taskExecutionArn string, options map[string]any) error

UpdateTaskExecution updates a task execution (no-op: options are advisory only).

type Location

type Location struct {
	CreationTime time.Time
	LocationArn  string
	LocationURI  string
}

Location is a generic DataSync location. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationAzureBlob

type LocationAzureBlob struct {
	CreationTime     time.Time
	SasConfiguration *SasConfiguration
	LocationArn      string
	LocationURI      string
	ContainerURL     string
	BlobType         string
	AccessTier       string
	Subdirectory     string
	AgentArns        []string
}

LocationAzureBlob is a DataSync Azure Blob location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationEfs

type LocationEfs struct {
	CreationTime            time.Time
	Ec2Config               *Ec2Config
	LocationArn             string
	LocationURI             string
	EfsFilesystemArn        string
	AccessPointArn          string
	FileSystemAccessRoleArn string
	InTransitEncryption     string
	Subdirectory            string
}

LocationEfs is a DataSync EFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxLustre

type LocationFsxLustre struct {
	CreationTime      time.Time
	LocationArn       string
	LocationURI       string
	FsxFilesystemArn  string
	Subdirectory      string
	SecurityGroupArns []string
}

LocationFsxLustre is a DataSync FSx Lustre location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxOntap

type LocationFsxOntap struct {
	CreationTime             time.Time
	Protocol                 *FsxProtocol
	LocationArn              string
	LocationURI              string
	StorageVirtualMachineArn string
	Subdirectory             string
	SecurityGroupArns        []string
}

LocationFsxOntap is a DataSync FSx ONTAP location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxOpenZfs

type LocationFsxOpenZfs struct {
	CreationTime      time.Time
	Protocol          *FsxProtocol
	LocationArn       string
	LocationURI       string
	FsxFilesystemArn  string
	Subdirectory      string
	SecurityGroupArns []string
}

LocationFsxOpenZfs is a DataSync FSx OpenZFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxWindows

type LocationFsxWindows struct {
	CreationTime      time.Time
	LocationArn       string
	LocationURI       string
	FsxFilesystemArn  string
	Domain            string
	User              string
	Subdirectory      string
	SecurityGroupArns []string
}

LocationFsxWindows is a DataSync FSx Windows location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationHdfs

type LocationHdfs struct {
	CreationTime       time.Time
	QopConfiguration   *QopConfiguration
	KmsKeyProviderURI  string
	LocationArn        string
	LocationURI        string
	AuthenticationType string
	SimpleUser         string
	KerberosPrincipal  string
	Subdirectory       string
	AgentArns          []string
	NameNodes          []HdfsNameNode
	BlockSize          int64
	ReplicationFactor  int32
}

LocationHdfs is a DataSync HDFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationListEntry

type LocationListEntry struct {
	CreationTime time.Time
	LocationArn  string
	LocationURI  string
}

LocationListEntry is a location entry in a list response.

type LocationNfs

type LocationNfs struct {
	CreationTime   time.Time
	MountOptions   *MountOptions
	LocationArn    string
	LocationURI    string
	ServerHostname string
	Subdirectory   string
	AgentArns      []string
}

LocationNfs is a DataSync NFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationObjectStorage

type LocationObjectStorage struct {
	CreationTime   time.Time
	LocationArn    string
	LocationURI    string
	ServerHostname string
	ServerProtocol string
	BucketName     string
	AccessKey      string
	Subdirectory   string
	AgentArns      []string
	ServerPort     int32
}

LocationObjectStorage is a DataSync object storage location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationS3

type LocationS3 struct {
	CreationTime   time.Time
	S3Config       S3Config
	LocationArn    string
	LocationURI    string
	S3BucketArn    string
	Subdirectory   string
	S3StorageClass string
}

LocationS3 is a DataSync S3 location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationSmb

type LocationSmb struct {
	CreationTime   time.Time
	MountOptions   *MountOptions
	LocationArn    string
	LocationURI    string
	ServerHostname string
	Domain         string
	User           string
	Subdirectory   string
	AgentArns      []string
}

LocationSmb is a DataSync SMB location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type MountOptions

type MountOptions struct {
	Version string
}

MountOptions holds NFS/SMB mount version.

type Provider

type Provider struct{}

Provider implements service.Provider for AWS DataSync.

func (*Provider) Init

Init initializes the DataSync service backend and handler.

func (*Provider) Name

func (p *Provider) Name() string

Name returns the provider name.

type QopConfiguration

type QopConfiguration struct {
	DataTransferProtection string
	RPCProtection          string
}

QopConfiguration holds HDFS QOP configuration.

type S3Config

type S3Config struct {
	BucketAccessRoleArn string
}

S3Config holds IAM role configuration for an S3 location.

type SasConfiguration

type SasConfiguration struct {
	Token string
}

SasConfiguration holds Azure Blob SAS token configuration.

type StorageBackend

type StorageBackend interface {
	// Agent operations
	CreateAgent(name string, activationKey string, tags map[string]string) (*Agent, error)
	DescribeAgent(agentArn string) (*Agent, error)
	UpdateAgent(agentArn, name string) error
	DeleteAgent(agentArn string) error
	ListAgents(maxResults int32, nextToken string) ([]*AgentListEntry, string, error)

	// Location operations (S3)
	CreateLocationS3(
		subdirectory, s3BucketArn, s3StorageClass string,
		s3Config S3Config,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationS3(locationArn string) (*LocationS3, error)
	DeleteLocation(locationArn string) error
	ListLocations(maxResults int32, nextToken string) ([]*LocationListEntry, string, error)

	// Task operations
	CreateTask(
		sourceLocationArn, destinationLocationArn, name, cloudWatchLogGroupArn string,
		tags map[string]string,
	) (*Task, error)
	DescribeTask(taskArn string) (*Task, error)
	UpdateTask(taskArn, name, cloudWatchLogGroupArn string) error
	DeleteTask(taskArn string) error
	ListTasks(maxResults int32, nextToken string) ([]*TaskListEntry, string, error)

	// Task execution operations
	StartTaskExecution(taskArn string) (*TaskExecution, error)
	CancelTaskExecution(taskExecutionArn string) error
	DescribeTaskExecution(taskExecutionArn string) (*TaskExecution, error)
	ListTaskExecutions(taskArn string, maxResults int32, nextToken string) ([]*TaskExecutionListEntry, string, error)

	// Location operations (S3 update)
	UpdateLocationS3(locationArn, subdirectory, s3StorageClass string, s3Config S3Config) error

	// Task execution update
	UpdateTaskExecution(taskExecutionArn string, options map[string]any) error

	// Location operations (Azure Blob)
	CreateLocationAzureBlob(
		containerURL, subdirectory, blobType, accessTier string,
		sasConfig *SasConfiguration,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationAzureBlob(locationArn string) (*LocationAzureBlob, error)
	UpdateLocationAzureBlob(
		locationArn, containerURL, subdirectory, blobType, accessTier string,
		sasConfig *SasConfiguration,
		agentArns []string,
	) error

	// Location operations (EFS)
	CreateLocationEfs(
		efsFilesystemArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
		ec2Config *Ec2Config,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationEfs(locationArn string) (*LocationEfs, error)
	UpdateLocationEfs(
		locationArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
		ec2Config *Ec2Config,
	) error

	// Location operations (FSx Lustre)
	CreateLocationFsxLustre(
		fsxFilesystemArn, subdirectory string,
		securityGroupArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationFsxLustre(locationArn string) (*LocationFsxLustre, error)
	UpdateLocationFsxLustre(locationArn, subdirectory string) error

	// Location operations (FSx ONTAP)
	CreateLocationFsxOntap(
		storageVirtualMachineArn, subdirectory string,
		protocol *FsxProtocol,
		securityGroupArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationFsxOntap(locationArn string) (*LocationFsxOntap, error)
	UpdateLocationFsxOntap(locationArn, subdirectory string, protocol *FsxProtocol) error

	// Location operations (FSx OpenZFS)
	CreateLocationFsxOpenZfs(
		fsxFilesystemArn, subdirectory string,
		protocol *FsxProtocol,
		securityGroupArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationFsxOpenZfs(locationArn string) (*LocationFsxOpenZfs, error)
	UpdateLocationFsxOpenZfs(locationArn, subdirectory string, protocol *FsxProtocol) error

	// Location operations (FSx Windows)
	CreateLocationFsxWindows(
		fsxFilesystemArn, subdirectory, domain, user, password string,
		securityGroupArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationFsxWindows(locationArn string) (*LocationFsxWindows, error)
	UpdateLocationFsxWindows(locationArn, subdirectory, domain, user, password string) error

	// Location operations (HDFS)
	CreateLocationHdfs(
		subdirectory, authenticationType, simpleUser string,
		kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
		nameNodes []HdfsNameNode,
		blockSize int64,
		replicationFactor int32,
		qopConfig *QopConfiguration,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationHdfs(locationArn string) (*LocationHdfs, error)
	UpdateLocationHdfs(
		locationArn, subdirectory, authenticationType, simpleUser string,
		kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
		nameNodes []HdfsNameNode,
		blockSize int64,
		replicationFactor int32,
		qopConfig *QopConfiguration,
		agentArns []string,
	) error

	// Location operations (NFS)
	CreateLocationNfs(
		serverHostname, subdirectory string,
		mountOptions *MountOptions,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationNfs(locationArn string) (*LocationNfs, error)
	UpdateLocationNfs(locationArn, subdirectory string, mountOptions *MountOptions, agentArns []string) error

	// Location operations (Object Storage)
	CreateLocationObjectStorage(
		serverHostname, serverProtocol, bucketName, subdirectory, accessKey, secretKey string,
		serverPort int32,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationObjectStorage(locationArn string) (*LocationObjectStorage, error)
	UpdateLocationObjectStorage(
		locationArn, serverProtocol, subdirectory, accessKey, secretKey string,
		serverPort int32,
		agentArns []string,
	) error

	// Location operations (SMB)
	CreateLocationSmb(
		serverHostname, subdirectory, domain, user, password string,
		mountOptions *MountOptions,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationSmb(locationArn string) (*LocationSmb, error)
	UpdateLocationSmb(
		locationArn, subdirectory, domain, user, password string,
		mountOptions *MountOptions,
		agentArns []string,
	) error

	// Tag operations
	TagResource(resourceArn string, tags map[string]string) error
	UntagResource(resourceArn string, keys []string) error
	ListTagsForResource(resourceArn string, maxResults int32, nextToken string) (map[string]string, string, error)

	AccountID() string
	Region() string
	Reset()
	Snapshot(ctx context.Context) []byte
	Restore(ctx context.Context, data []byte) error
}

StorageBackend is the interface for DataSync storage operations.

type Task

type Task struct {
	CreationTime            time.Time
	Tags                    map[string]string
	TaskArn                 string
	Name                    string
	Status                  string
	SourceLocationArn       string
	DestinationLocationArn  string
	CloudWatchLogGroupArn   string
	CurrentTaskExecutionArn string
}

Task represents a DataSync transfer task. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type TaskExecution

type TaskExecution struct {
	StartTime                time.Time
	Options                  map[string]any
	TaskExecutionArn         string
	Status                   string
	EstimatedFilesToTransfer int64
	EstimatedBytesToTransfer int64
	FilesTransferred         int64
	BytesTransferred         int64
}

TaskExecution represents a DataSync task execution. StartTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type TaskExecutionListEntry

type TaskExecutionListEntry struct {
	TaskExecutionArn string
	Status           string
}

TaskExecutionListEntry is a task execution entry in a list response.

type TaskListEntry

type TaskListEntry struct {
	TaskArn string
	Name    string
	Status  string
}

TaskListEntry is a task entry in a list response.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL