datasync

package
v1.3.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 16, 2026 License: MIT Imports: 21 Imported by: 0

README

DataSync

Parity grade: A · SDK aws-sdk-go-v2/service/datasync@v1.61.4 · last audited 2026-08-10 (5eee2c54)

Coverage

Metric Value
Operations audited 53 (53 ok)
Feature families 5 (5 ok)
Known gaps 4
Deferred items 1
Resource leaks clean
Known gaps
  • LocationUri scheme prefixes for ObjectStorage ("object-storage://") and AzureBlob ("azure-blob://") technically violate AWS's own published LocationUri pattern (^(efs|nfs|s3|smb|hdfs|fsx[a-z0-9-]+)://...$, identical text on every DescribeLocation*Output doc page including these two), same as the now-fixed Lustre/ONTAP bugs -- but no positive evidence exists for what AWS actually returns for these two location types (both are comparatively recent additions; the shared regex may itself be stale doc-generation cruft that predates them and isn't enforced server-side for newer types, unlike the FSx family where the regex was clearly extended on purpose to add the fsx[a-z0-9-]+ alternative). Re-checked this sweep from two independent sources -- the installed botocore 1.43.56 model (data/datasync/2018-11-09/service-2.json.gz, LocationUri shape, pinned to the one version directory present) and AWS's live API_DescribeLocationObjectStorage.html/API_DescribeLocationAzureBlob.html doc pages -- both give the identical pattern text with no scheme-prefix example anywhere for either location type, so the "no positive evidence" verdict stands: there is proof the current prefixes violate the published pattern, but no proof of what a compliant replacement should be (both prefixes follow the same type-name-as-scheme convention as every other location type, including the fsxl:// fix below, which is itself only an analogy-based guess -- see next gap). Left unchanged; do not "fix" to a guessed scheme without evidence.
  • LocationUri scheme "fsxl://" for FSx Lustre (fixed this sweep from the confirmed-wrong "lustre://") was chosen by analogy with FSx OpenZFS's confirmed "fsxz://" (real AWS CLI doc example: fsxz://us-west-2.fs-.../fsx/folderA/folder) but is not independently confirmed against real AWS output. Medium confidence: matches the regex, matches the single-letter-suffix convention, but Lustre could plausibly use a different fsx-prefixed string.
  • ManagedSecretConfig (distinct from CmkSecretConfig/CustomSecretConfig, which are now modeled -- see Location family note) stays absent from every DescribeLocation*Output this sweep confirmed it on (Smb/Hdfs/ObjectStorage/AzureBlob/FsxWindows). This is correct, not a gap: the botocore model's own CmkSecretConfig-in-FsxProtocolSmb documentation states outright "Do not provide this for a CreateLocation request. ManagedSecretConfig is a ReadOnly property and is only be populated in the DescribeLocation response" -- AWS populates it itself when the client supplies a plaintext credential (Password/SecretKey/SasConfiguration) without CmkSecretConfig/CustomSecretConfig, by auto-provisioning a Secrets Manager secret. gopherstack has no Secrets Manager integration to back a real SecretArn, and fabricating one would violate the no-fabricated-IDs rule -- correctly left absent.
  • DescribeTaskOutput omits ErrorCode, ErrorDetail, DestinationNetworkInterfaceArns, SourceNetworkInterfaceArns, and ScheduleDetails, all present on the real output. Re-checked this sweep, including whether CancelTaskExecution's ERROR-status path (the one place this backend models a task-execution outcome other than SUCCESS) gives ErrorCode/ErrorDetail anything to surface: it doesn't -- CancelTaskExecution only flips storedTaskExecution.Status to the bare "ERROR" enum value, it never records failure text anywhere, so there is no error-message state to promote, and DataSync's own ErrorCode strings (e.g. its internal troubleshooting codes) aren't published anywhere this sweep could cite. gopherstack also has no ENI provisioning state at all. Both stay an honest omission rather than a fabricated value; a gap only if a future sweep adds real failure-text tracking or ENI simulation.
Deferred
  • StorageSystem / DiscoveryJob / TaskReport-as-standalone-ops: not present in aws-sdk-go-v2/service/datasync v1.59.2 at all (re-verified this sweep: full api_op_.go listing has no such ops -- grepped for storagesystem/discoveryjob/taskreport case-insensitively across services/datasync/.go, zero matches). TaskReportConfig itself IS a real member of CreateTaskInput/UpdateTaskInput/DescribeTaskOutput and is now modeled (see Task family fix) -- it's a configuration blob on the task, not a standalone set of ops. The family mentioned in the audit brief (dedicated StorageSystem/DiscoveryJob CRUD + Start/Stop/DescribeDiscoveryJob ops) does not exist in the pinned SDK.

More

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrNotFound is returned when a resource does not exist.
	ErrNotFound = awserr.New(resourceNotFoundType, awserr.ErrNotFound)
	// ErrAlreadyExists is returned when a resource already exists.
	ErrAlreadyExists = awserr.New(conflictExceptionType, awserr.ErrAlreadyExists)
	// ErrInvalidParameter is returned for invalid input.
	ErrInvalidParameter = awserr.New(invalidRequestType, awserr.ErrInvalidParameter)
)
View Source
var ErrNilAppContext = errors.New("datasync: nil app context")

ErrNilAppContext is returned when Init is called with a nil AppContext.

Functions

This section is empty.

Types

type Agent

type Agent struct {
	CreationTime time.Time
	Tags         map[string]string
	AgentArn     string
	Name         string
	Status       string
	EndpointType string
}

Agent represents a DataSync agent. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type AgentListEntry

type AgentListEntry struct {
	AgentArn string
	Name     string
	Status   string
}

AgentListEntry is an agent entry in a list response.

type CmkSecretConfig added in v1.3.1

type CmkSecretConfig struct {
	SecretArn string
	KmsKeyArn string
}

CmkSecretConfig holds configuration for a DataSync-managed secret encrypted with a customer-managed KMS key (CmkSecretConfig, botocore datasync 2018-11-09 model).

type CustomSecretConfig added in v1.3.1

type CustomSecretConfig struct {
	SecretArn           string
	SecretAccessRoleArn string
}

CustomSecretConfig holds configuration for a customer-managed Secrets Manager secret (CustomSecretConfig, botocore datasync 2018-11-09 model).

type Ec2Config

type Ec2Config struct {
	SubnetArn         string
	SecurityGroupArns []string
}

Ec2Config holds EC2 configuration for an EFS location.

type FilterRule added in v1.2.0

type FilterRule struct {
	FilterType string
	Value      string
}

FilterRule is a DataSync include/exclude filter (SIMPLE_PATTERN rules only).

type FsxNfsProtocol

type FsxNfsProtocol struct {
	MountOptions *MountOptions
}

FsxNfsProtocol holds FSx NFS protocol configuration.

type FsxProtocol

type FsxProtocol struct {
	NFS *FsxNfsProtocol
	SMB *FsxSmbProtocol
}

FsxProtocol holds FSx protocol configuration (NFS or SMB).

type FsxSmbProtocol

type FsxSmbProtocol struct {
	MountOptions *MountOptions
	Domain       string
	Password     string
	User         string
}

FsxSmbProtocol holds FSx SMB protocol configuration.

type Handler

type Handler struct {
	Backend StorageBackend
	// contains filtered or unexported fields
}

Handler handles DataSync HTTP requests.

func NewHandler

func NewHandler(b StorageBackend) *Handler

NewHandler constructs a new Handler.

func (*Handler) ExtractOperation

func (h *Handler) ExtractOperation(c *echo.Context) string

ExtractOperation extracts the operation name from the X-Amz-Target header.

func (*Handler) ExtractResource

func (h *Handler) ExtractResource(_ *echo.Context) string

ExtractResource extracts the resource identifier from the request body.

func (*Handler) GetSupportedOperations

func (h *Handler) GetSupportedOperations() []string

GetSupportedOperations returns the list of supported operations.

func (*Handler) Handler

func (h *Handler) Handler() echo.HandlerFunc

Handler returns the Echo handler function for DataSync requests.

func (*Handler) MatchPriority

func (h *Handler) MatchPriority() int

MatchPriority returns the routing priority.

func (*Handler) Name

func (h *Handler) Name() string

Name returns the service name.

func (*Handler) Reset

func (h *Handler) Reset()

Reset resets the backend and rebuilds the dispatch table.

func (*Handler) Restore

func (h *Handler) Restore(ctx context.Context, data []byte) error

Restore implements persistence.Persistable by delegating to the backend. See the Snapshot doc comment above for why this delegation is new.

func (*Handler) RouteMatcher

func (h *Handler) RouteMatcher() service.Matcher

RouteMatcher returns a function that matches DataSync API requests.

func (*Handler) Snapshot

func (h *Handler) Snapshot(ctx context.Context) []byte

Snapshot implements persistence.Persistable by delegating to the backend.

Prior to Phase 3.3, Handler had no Snapshot/Restore of its own even though InMemoryBackend implemented both (dead wiring: nothing ever called them). This delegation is what actually wires DataSync into the persistence Manager.

type HdfsNameNode

type HdfsNameNode struct {
	Hostname string
	Port     int32
}

HdfsNameNode is an HDFS name node endpoint.

type InMemoryBackend

type InMemoryBackend struct {
	// contains filtered or unexported fields
}

InMemoryBackend implements StorageBackend using pkgs/store tables.

agents, locations, and tasks are flat ARN-keyed resources with globally unique identity, so each is registered directly on b.registry (see store_setup.go). executions is likewise keyed by its own globally-unique TaskExecutionArn (which embeds the owning TaskArn), so it too is a direct registration; the former taskArn → executionArn → execution nesting is replaced by the executionsByTask secondary index, additively derived from TaskExecutionArn via extractTaskArnFromExecution. tags is left as a raw map because its values are plain string maps, not *T.

func NewInMemoryBackend

func NewInMemoryBackend(accountID, region string) *InMemoryBackend

NewInMemoryBackend constructs a new InMemoryBackend.

func (*InMemoryBackend) AccountID

func (b *InMemoryBackend) AccountID() string

AccountID returns the account ID.

func (*InMemoryBackend) CancelTaskExecution

func (b *InMemoryBackend) CancelTaskExecution(taskExecutionArn string) error

CancelTaskExecution cancels a running task execution.

AWS has no CANCELLED task-execution status (see types.TaskExecutionStatus); a cancelled execution settles into ERROR, matching the documented "some transient states... interrupt a task execution" behavior. Unlike a truly "current" (actively running) marker, CurrentTaskExecutionArn on the parent task is documented as "the ARN of the most recent task execution", so it is left pointing at the cancelled execution rather than cleared.

func (*InMemoryBackend) CreateAgent

func (b *InMemoryBackend) CreateAgent(name, _ string, tags map[string]string) (*Agent, error)

CreateAgent creates a new DataSync agent.

func (*InMemoryBackend) CreateLocationAzureBlob

func (b *InMemoryBackend) CreateLocationAzureBlob(
	containerURL, subdirectory, blobType, accessTier, authenticationType string,
	sasConfig *SasConfiguration,
	agentArns []string,
	tags map[string]string,
	secretConfig SecretConfig,
) (*Location, error)

func (*InMemoryBackend) CreateLocationEfs

func (b *InMemoryBackend) CreateLocationEfs(
	efsFilesystemArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
	ec2Config *Ec2Config,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxLustre

func (b *InMemoryBackend) CreateLocationFsxLustre(
	fsxFilesystemArn, subdirectory string,
	securityGroupArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxOntap

func (b *InMemoryBackend) CreateLocationFsxOntap(
	storageVirtualMachineArn, subdirectory string,
	protocol *FsxProtocol,
	securityGroupArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxOpenZfs

func (b *InMemoryBackend) CreateLocationFsxOpenZfs(
	fsxFilesystemArn, subdirectory string,
	protocol *FsxProtocol,
	securityGroupArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationFsxWindows

func (b *InMemoryBackend) CreateLocationFsxWindows(
	fsxFilesystemArn, subdirectory, domain, user, password string,
	securityGroupArns []string,
	tags map[string]string,
	secretConfig SecretConfig,
) (*Location, error)

func (*InMemoryBackend) CreateLocationHdfs

func (b *InMemoryBackend) CreateLocationHdfs(
	subdirectory, authenticationType, simpleUser string,
	kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
	nameNodes []HdfsNameNode,
	blockSize int64,
	replicationFactor int32,
	qopConfig *QopConfiguration,
	agentArns []string,
	tags map[string]string,
	secretConfig SecretConfig,
) (*Location, error)

func (*InMemoryBackend) CreateLocationNfs

func (b *InMemoryBackend) CreateLocationNfs(
	serverHostname, subdirectory string,
	mountOptions *MountOptions,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

func (*InMemoryBackend) CreateLocationObjectStorage

func (b *InMemoryBackend) CreateLocationObjectStorage(
	serverHostname, serverProtocol, bucketName, subdirectory, accessKey, secretKey string,
	serverPort int32,
	agentArns []string,
	tags map[string]string,
	secretConfig SecretConfig,
) (*Location, error)

func (*InMemoryBackend) CreateLocationS3

func (b *InMemoryBackend) CreateLocationS3(
	subdirectory, s3BucketArn, s3StorageClass string,
	s3Config S3Config,
	agentArns []string,
	tags map[string]string,
) (*Location, error)

CreateLocationS3 creates a new S3 location.

func (*InMemoryBackend) CreateLocationSmb

func (b *InMemoryBackend) CreateLocationSmb(
	serverHostname, subdirectory, domain, user, password, authenticationType string,
	mountOptions *MountOptions,
	agentArns []string,
	tags map[string]string,
	smbKerberos SmbKerberosConfig,
	secretConfig SecretConfig,
) (*Location, error)

func (*InMemoryBackend) CreateTask

func (b *InMemoryBackend) CreateTask(
	sourceLocationArn, destinationLocationArn, name, cloudWatchLogGroupArn string,
	settings TaskSettings,
	tags map[string]string,
) (*Task, error)

CreateTask creates a new DataSync task.

func (*InMemoryBackend) DeleteAgent

func (b *InMemoryBackend) DeleteAgent(agentArn string) error

DeleteAgent deletes an agent.

func (*InMemoryBackend) DeleteLocation

func (b *InMemoryBackend) DeleteLocation(locationArn string) error

DeleteLocation deletes a location.

func (*InMemoryBackend) DeleteTask

func (b *InMemoryBackend) DeleteTask(taskArn string) error

DeleteTask deletes a task.

func (*InMemoryBackend) DescribeAgent

func (b *InMemoryBackend) DescribeAgent(agentArn string) (*Agent, error)

DescribeAgent returns agent details.

func (*InMemoryBackend) DescribeLocationAzureBlob

func (b *InMemoryBackend) DescribeLocationAzureBlob(locationArn string) (*LocationAzureBlob, error)

func (*InMemoryBackend) DescribeLocationEfs

func (b *InMemoryBackend) DescribeLocationEfs(locationArn string) (*LocationEfs, error)

func (*InMemoryBackend) DescribeLocationFsxLustre

func (b *InMemoryBackend) DescribeLocationFsxLustre(locationArn string) (*LocationFsxLustre, error)

func (*InMemoryBackend) DescribeLocationFsxOntap

func (b *InMemoryBackend) DescribeLocationFsxOntap(locationArn string) (*LocationFsxOntap, error)

func (*InMemoryBackend) DescribeLocationFsxOpenZfs

func (b *InMemoryBackend) DescribeLocationFsxOpenZfs(locationArn string) (*LocationFsxOpenZfs, error)

func (*InMemoryBackend) DescribeLocationFsxWindows

func (b *InMemoryBackend) DescribeLocationFsxWindows(locationArn string) (*LocationFsxWindows, error)

func (*InMemoryBackend) DescribeLocationHdfs

func (b *InMemoryBackend) DescribeLocationHdfs(locationArn string) (*LocationHdfs, error)

func (*InMemoryBackend) DescribeLocationNfs

func (b *InMemoryBackend) DescribeLocationNfs(locationArn string) (*LocationNfs, error)

func (*InMemoryBackend) DescribeLocationObjectStorage

func (b *InMemoryBackend) DescribeLocationObjectStorage(locationArn string) (*LocationObjectStorage, error)

func (*InMemoryBackend) DescribeLocationS3

func (b *InMemoryBackend) DescribeLocationS3(locationArn string) (*LocationS3, error)

DescribeLocationS3 returns S3 location details.

func (*InMemoryBackend) DescribeLocationSmb

func (b *InMemoryBackend) DescribeLocationSmb(locationArn string) (*LocationSmb, error)

func (*InMemoryBackend) DescribeTask

func (b *InMemoryBackend) DescribeTask(taskArn string) (*Task, error)

DescribeTask returns task details.

func (*InMemoryBackend) DescribeTaskExecution

func (b *InMemoryBackend) DescribeTaskExecution(taskExecutionArn string) (*TaskExecution, error)

DescribeTaskExecution returns task execution details. Executions in LAUNCHING state are lazily advanced to SUCCESS on first describe. When the execution that finishes is still the task's current one, the parent task's Status reverts from RUNNING to AVAILABLE, matching AWS (task Status is RUNNING only while a task execution is in progress).

func (*InMemoryBackend) ListAgents

func (b *InMemoryBackend) ListAgents(maxResults int32, nextToken string) ([]*AgentListEntry, string, error)

ListAgents returns agents, sorted by ARN.

func (*InMemoryBackend) ListLocations

func (b *InMemoryBackend) ListLocations(maxResults int32, nextToken string) ([]*LocationListEntry, string, error)

ListLocations returns locations, sorted by ARN.

func (*InMemoryBackend) ListTagsForResource

func (b *InMemoryBackend) ListTagsForResource(
	resourceArn string,
	maxResults int32,
	nextToken string,
) (map[string]string, string, error)

ListTagsForResource returns tags for a resource with pagination.

func (*InMemoryBackend) ListTaskExecutions

func (b *InMemoryBackend) ListTaskExecutions(
	taskArn string,
	maxResults int32,
	nextToken string,
) ([]*TaskExecutionListEntry, string, error)

ListTaskExecutions returns executions for a task (or, when taskArn is empty, every execution across all tasks -- TaskArn is an optional filter per the real ListTaskExecutions API), sorted by ARN.

func (*InMemoryBackend) ListTasks

func (b *InMemoryBackend) ListTasks(maxResults int32, nextToken string) ([]*TaskListEntry, string, error)

ListTasks returns tasks, sorted by ARN.

func (*InMemoryBackend) Region

func (b *InMemoryBackend) Region() string

Region returns the region.

func (*InMemoryBackend) Reset

func (b *InMemoryBackend) Reset()

Reset clears all state.

func (*InMemoryBackend) Restore

func (b *InMemoryBackend) Restore(ctx context.Context, data []byte) error

Restore deserializes backend state from a snapshot.

func (*InMemoryBackend) Snapshot

func (b *InMemoryBackend) Snapshot(ctx context.Context) []byte

Snapshot serializes the backend state to JSON.

func (*InMemoryBackend) StartTaskExecution

func (b *InMemoryBackend) StartTaskExecution(taskArn string) (*TaskExecution, error)

StartTaskExecution starts a new task execution.

AWS allows only one task execution in progress per task at a time ("For each task, you can only run one task execution at a time.", per the StartTaskExecution API docs); attempting to start another while one is still running is rejected as an invalid request.

func (*InMemoryBackend) TagResource

func (b *InMemoryBackend) TagResource(resourceArn string, tags map[string]string) error

TagResource adds or updates tags on a resource.

func (*InMemoryBackend) TaggedResources added in v1.3.1

func (b *InMemoryBackend) TaggedResources() []TaggedEntry

TaggedResources returns every DataSync resource ARN that currently has at least one tag applied via TagResource.

func (*InMemoryBackend) UntagResource

func (b *InMemoryBackend) UntagResource(resourceArn string, keys []string) error

UntagResource removes tags from a resource.

func (*InMemoryBackend) UpdateAgent

func (b *InMemoryBackend) UpdateAgent(agentArn, name string) error

UpdateAgent updates the agent's name.

func (*InMemoryBackend) UpdateLocationAzureBlob

func (b *InMemoryBackend) UpdateLocationAzureBlob(
	locationArn, containerURL, subdirectory, blobType, accessTier, authenticationType string,
	sasConfig *SasConfiguration,
	agentArns []string,
	secretConfig SecretConfig,
) error

func (*InMemoryBackend) UpdateLocationEfs

func (b *InMemoryBackend) UpdateLocationEfs(
	locationArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
	ec2Config *Ec2Config,
) error

func (*InMemoryBackend) UpdateLocationFsxLustre

func (b *InMemoryBackend) UpdateLocationFsxLustre(locationArn, subdirectory string) error

func (*InMemoryBackend) UpdateLocationFsxOntap

func (b *InMemoryBackend) UpdateLocationFsxOntap(locationArn, subdirectory string, protocol *FsxProtocol) error

func (*InMemoryBackend) UpdateLocationFsxOpenZfs

func (b *InMemoryBackend) UpdateLocationFsxOpenZfs(locationArn, subdirectory string, protocol *FsxProtocol) error

func (*InMemoryBackend) UpdateLocationFsxWindows

func (b *InMemoryBackend) UpdateLocationFsxWindows(
	locationArn, subdirectory, domain, user, password string,
	secretConfig SecretConfig,
) error

func (*InMemoryBackend) UpdateLocationHdfs

func (b *InMemoryBackend) UpdateLocationHdfs(
	locationArn, subdirectory, authenticationType, simpleUser string,
	kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
	nameNodes []HdfsNameNode,
	blockSize int64,
	replicationFactor int32,
	qopConfig *QopConfiguration,
	agentArns []string,
	secretConfig SecretConfig,
) error

func (*InMemoryBackend) UpdateLocationNfs

func (b *InMemoryBackend) UpdateLocationNfs(
	locationArn, serverHostname, subdirectory string,
	mountOptions *MountOptions,
	agentArns []string,
) error

func (*InMemoryBackend) UpdateLocationObjectStorage

func (b *InMemoryBackend) UpdateLocationObjectStorage(
	locationArn, serverHostname, serverProtocol, subdirectory, accessKey, secretKey string,
	serverPort int32,
	agentArns []string,
	secretConfig SecretConfig,
) error

func (*InMemoryBackend) UpdateLocationS3

func (b *InMemoryBackend) UpdateLocationS3(locationArn, subdirectory, s3StorageClass string, s3Config S3Config) error

UpdateLocationS3 updates an S3 location's subdirectory, storage class, and S3 config.

func (*InMemoryBackend) UpdateLocationSmb

func (b *InMemoryBackend) UpdateLocationSmb(
	locationArn, serverHostname, subdirectory, domain, user, password, authenticationType string,
	mountOptions *MountOptions,
	agentArns []string,
	smbKerberos SmbKerberosConfig,
	secretConfig SecretConfig,
) error

func (*InMemoryBackend) UpdateTask

func (b *InMemoryBackend) UpdateTask(taskArn, name, cloudWatchLogGroupArn string, settings TaskSettings) error

UpdateTask updates a task's mutable settings. AWS's UpdateTaskInput has no TaskArn-only "clear the log group" convention like some other fields, so CloudWatchLogGroupArn is always assigned (empty clears it, matching prior behavior). Options/Excludes/Includes/Schedule/ManifestConfig/ TaskReportConfig follow AWS's "only supplied fields change" semantics: a nil field in settings means "not supplied, leave unchanged"; a non-nil (possibly empty) field means "set to this value" -- which also covers the documented "specify this parameter as empty to remove" behavior for ManifestConfig/TaskReportConfig, since an explicit empty map is non-nil.

func (*InMemoryBackend) UpdateTaskExecution

func (b *InMemoryBackend) UpdateTaskExecution(taskExecutionArn string, options map[string]any) error

UpdateTaskExecution updates a task execution (no-op: options are advisory only).

type Location

type Location struct {
	CreationTime time.Time
	LocationArn  string
	LocationURI  string
}

Location is a generic DataSync location. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationAzureBlob

type LocationAzureBlob struct {
	CreationTime       time.Time
	SasConfiguration   *SasConfiguration
	CmkSecretConfig    *CmkSecretConfig
	CustomSecretConfig *CustomSecretConfig
	LocationArn        string
	LocationURI        string
	ContainerURL       string
	BlobType           string
	AccessTier         string
	AuthenticationType string
	Subdirectory       string
	AgentArns          []string
}

LocationAzureBlob is a DataSync Azure Blob location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationEfs

type LocationEfs struct {
	CreationTime            time.Time
	Ec2Config               *Ec2Config
	LocationArn             string
	LocationURI             string
	EfsFilesystemArn        string
	AccessPointArn          string
	FileSystemAccessRoleArn string
	InTransitEncryption     string
	Subdirectory            string
}

LocationEfs is a DataSync EFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxLustre

type LocationFsxLustre struct {
	CreationTime      time.Time
	LocationArn       string
	LocationURI       string
	FsxFilesystemArn  string
	Subdirectory      string
	SecurityGroupArns []string
}

LocationFsxLustre is a DataSync FSx Lustre location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxOntap

type LocationFsxOntap struct {
	CreationTime             time.Time
	Protocol                 *FsxProtocol
	LocationArn              string
	LocationURI              string
	StorageVirtualMachineArn string
	FsxFilesystemArn         string
	Subdirectory             string
	SecurityGroupArns        []string
}

LocationFsxOntap is a DataSync FSx ONTAP location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxOpenZfs

type LocationFsxOpenZfs struct {
	CreationTime      time.Time
	Protocol          *FsxProtocol
	LocationArn       string
	LocationURI       string
	FsxFilesystemArn  string
	Subdirectory      string
	SecurityGroupArns []string
}

LocationFsxOpenZfs is a DataSync FSx OpenZFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationFsxWindows

type LocationFsxWindows struct {
	CreationTime       time.Time
	CmkSecretConfig    *CmkSecretConfig
	CustomSecretConfig *CustomSecretConfig
	LocationArn        string
	LocationURI        string
	FsxFilesystemArn   string
	Domain             string
	User               string
	Subdirectory       string
	SecurityGroupArns  []string
}

LocationFsxWindows is a DataSync FSx Windows location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationHdfs

type LocationHdfs struct {
	CreationTime       time.Time
	QopConfiguration   *QopConfiguration
	CmkSecretConfig    *CmkSecretConfig
	CustomSecretConfig *CustomSecretConfig
	KmsKeyProviderURI  string
	LocationArn        string
	LocationURI        string
	AuthenticationType string
	SimpleUser         string
	KerberosPrincipal  string
	Subdirectory       string
	AgentArns          []string
	NameNodes          []HdfsNameNode
	BlockSize          int64
	ReplicationFactor  int32
}

LocationHdfs is a DataSync HDFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationListEntry

type LocationListEntry struct {
	CreationTime time.Time
	LocationArn  string
	LocationURI  string
}

LocationListEntry is a location entry in a list response.

type LocationNfs

type LocationNfs struct {
	CreationTime   time.Time
	MountOptions   *MountOptions
	LocationArn    string
	LocationURI    string
	ServerHostname string
	Subdirectory   string
	AgentArns      []string
}

LocationNfs is a DataSync NFS location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationObjectStorage

type LocationObjectStorage struct {
	CreationTime       time.Time
	CmkSecretConfig    *CmkSecretConfig
	CustomSecretConfig *CustomSecretConfig
	LocationArn        string
	LocationURI        string
	ServerHostname     string
	ServerProtocol     string
	BucketName         string
	AccessKey          string
	Subdirectory       string
	AgentArns          []string
	ServerPort         int32
}

LocationObjectStorage is a DataSync object storage location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationS3

type LocationS3 struct {
	CreationTime   time.Time
	S3Config       S3Config
	LocationArn    string
	LocationURI    string
	S3BucketArn    string
	Subdirectory   string
	S3StorageClass string
	AgentArns      []string
}

LocationS3 is a DataSync S3 location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type LocationSmb

type LocationSmb struct {
	CreationTime       time.Time
	MountOptions       *MountOptions
	CmkSecretConfig    *CmkSecretConfig
	CustomSecretConfig *CustomSecretConfig
	LocationArn        string
	LocationURI        string
	ServerHostname     string
	Domain             string
	User               string
	AuthenticationType string
	KerberosPrincipal  string
	Subdirectory       string
	AgentArns          []string
	DNSIPAddresses     []string
}

LocationSmb is a DataSync SMB location with full details. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type MountOptions

type MountOptions struct {
	Version string
}

MountOptions holds NFS/SMB mount version.

type Provider

type Provider struct{}

Provider implements service.Provider for AWS DataSync.

func (*Provider) Init

Init initializes the DataSync service backend and handler.

func (*Provider) Name

func (p *Provider) Name() string

Name returns the provider name.

type QopConfiguration

type QopConfiguration struct {
	DataTransferProtection string
	RPCProtection          string
}

QopConfiguration holds HDFS QOP configuration.

type S3Config

type S3Config struct {
	BucketAccessRoleArn string
}

S3Config holds IAM role configuration for an S3 location.

type SasConfiguration

type SasConfiguration struct {
	Token string
}

SasConfiguration holds Azure Blob SAS token configuration.

type SecretConfig added in v1.3.1

type SecretConfig struct {
	Cmk    *CmkSecretConfig
	Custom *CustomSecretConfig
}

SecretConfig groups the two mutually exclusive secret-config alternatives that CreateLocation*/UpdateLocation* requests accept for location types that support DataSync-managed secrets. A zero value means neither was supplied.

type SmbKerberosConfig added in v1.3.1

type SmbKerberosConfig struct {
	KerberosPrincipal string
	KerberosKeytab    string
	KerberosKrb5Conf  string
	DNSIPAddresses    []string
}

SmbKerberosConfig groups the SMB location fields that apply only when AuthenticationType is KERBEROS (DnsIpAddresses, KerberosPrincipal, KerberosKeytab, KerberosKrb5Conf; botocore datasync 2018-11-09 model, CreateLocationSmbRequest). KerberosKeytab/KerberosKrb5Conf are write-only credential blobs, matching AWS: DescribeLocationSmbResponse has neither.

type StorageBackend

type StorageBackend interface {
	// Agent operations
	CreateAgent(name string, activationKey string, tags map[string]string) (*Agent, error)
	DescribeAgent(agentArn string) (*Agent, error)
	UpdateAgent(agentArn, name string) error
	DeleteAgent(agentArn string) error
	ListAgents(maxResults int32, nextToken string) ([]*AgentListEntry, string, error)

	// Location operations (S3)
	CreateLocationS3(
		subdirectory, s3BucketArn, s3StorageClass string,
		s3Config S3Config,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationS3(locationArn string) (*LocationS3, error)
	DeleteLocation(locationArn string) error
	ListLocations(maxResults int32, nextToken string) ([]*LocationListEntry, string, error)

	// Task operations
	CreateTask(
		sourceLocationArn, destinationLocationArn, name, cloudWatchLogGroupArn string,
		settings TaskSettings,
		tags map[string]string,
	) (*Task, error)
	DescribeTask(taskArn string) (*Task, error)
	UpdateTask(taskArn, name, cloudWatchLogGroupArn string, settings TaskSettings) error
	DeleteTask(taskArn string) error
	ListTasks(maxResults int32, nextToken string) ([]*TaskListEntry, string, error)

	// Task execution operations
	StartTaskExecution(taskArn string) (*TaskExecution, error)
	CancelTaskExecution(taskExecutionArn string) error
	DescribeTaskExecution(taskExecutionArn string) (*TaskExecution, error)
	ListTaskExecutions(taskArn string, maxResults int32, nextToken string) ([]*TaskExecutionListEntry, string, error)

	// Location operations (S3 update)
	UpdateLocationS3(locationArn, subdirectory, s3StorageClass string, s3Config S3Config) error

	// Task execution update
	UpdateTaskExecution(taskExecutionArn string, options map[string]any) error

	// Location operations (Azure Blob)
	CreateLocationAzureBlob(
		containerURL, subdirectory, blobType, accessTier, authenticationType string,
		sasConfig *SasConfiguration,
		agentArns []string,
		tags map[string]string,
		secretConfig SecretConfig,
	) (*Location, error)
	DescribeLocationAzureBlob(locationArn string) (*LocationAzureBlob, error)
	UpdateLocationAzureBlob(
		locationArn, containerURL, subdirectory, blobType, accessTier, authenticationType string,
		sasConfig *SasConfiguration,
		agentArns []string,
		secretConfig SecretConfig,
	) error

	// Location operations (EFS)
	CreateLocationEfs(
		efsFilesystemArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
		ec2Config *Ec2Config,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationEfs(locationArn string) (*LocationEfs, error)
	UpdateLocationEfs(
		locationArn, subdirectory, accessPointArn, fileSystemAccessRoleArn, inTransitEncryption string,
		ec2Config *Ec2Config,
	) error

	// Location operations (FSx Lustre)
	CreateLocationFsxLustre(
		fsxFilesystemArn, subdirectory string,
		securityGroupArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationFsxLustre(locationArn string) (*LocationFsxLustre, error)
	UpdateLocationFsxLustre(locationArn, subdirectory string) error

	// Location operations (FSx ONTAP)
	CreateLocationFsxOntap(
		storageVirtualMachineArn, subdirectory string,
		protocol *FsxProtocol,
		securityGroupArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationFsxOntap(locationArn string) (*LocationFsxOntap, error)
	UpdateLocationFsxOntap(locationArn, subdirectory string, protocol *FsxProtocol) error

	// Location operations (FSx OpenZFS)
	CreateLocationFsxOpenZfs(
		fsxFilesystemArn, subdirectory string,
		protocol *FsxProtocol,
		securityGroupArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationFsxOpenZfs(locationArn string) (*LocationFsxOpenZfs, error)
	UpdateLocationFsxOpenZfs(locationArn, subdirectory string, protocol *FsxProtocol) error

	// Location operations (FSx Windows)
	CreateLocationFsxWindows(
		fsxFilesystemArn, subdirectory, domain, user, password string,
		securityGroupArns []string,
		tags map[string]string,
		secretConfig SecretConfig,
	) (*Location, error)
	DescribeLocationFsxWindows(locationArn string) (*LocationFsxWindows, error)
	UpdateLocationFsxWindows(
		locationArn, subdirectory, domain, user, password string,
		secretConfig SecretConfig,
	) error

	// Location operations (HDFS)
	CreateLocationHdfs(
		subdirectory, authenticationType, simpleUser string,
		kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
		nameNodes []HdfsNameNode,
		blockSize int64,
		replicationFactor int32,
		qopConfig *QopConfiguration,
		agentArns []string,
		tags map[string]string,
		secretConfig SecretConfig,
	) (*Location, error)
	DescribeLocationHdfs(locationArn string) (*LocationHdfs, error)
	UpdateLocationHdfs(
		locationArn, subdirectory, authenticationType, simpleUser string,
		kerberosPrincipal, kerberosKeytab, kerberosKrb5Conf, kmsKeyProviderURI string,
		nameNodes []HdfsNameNode,
		blockSize int64,
		replicationFactor int32,
		qopConfig *QopConfiguration,
		agentArns []string,
		secretConfig SecretConfig,
	) error

	// Location operations (NFS)
	CreateLocationNfs(
		serverHostname, subdirectory string,
		mountOptions *MountOptions,
		agentArns []string,
		tags map[string]string,
	) (*Location, error)
	DescribeLocationNfs(locationArn string) (*LocationNfs, error)
	UpdateLocationNfs(
		locationArn, serverHostname, subdirectory string,
		mountOptions *MountOptions,
		agentArns []string,
	) error

	// Location operations (Object Storage)
	CreateLocationObjectStorage(
		serverHostname, serverProtocol, bucketName, subdirectory, accessKey, secretKey string,
		serverPort int32,
		agentArns []string,
		tags map[string]string,
		secretConfig SecretConfig,
	) (*Location, error)
	DescribeLocationObjectStorage(locationArn string) (*LocationObjectStorage, error)
	UpdateLocationObjectStorage(
		locationArn, serverHostname, serverProtocol, subdirectory, accessKey, secretKey string,
		serverPort int32,
		agentArns []string,
		secretConfig SecretConfig,
	) error

	// Location operations (SMB)
	CreateLocationSmb(
		serverHostname, subdirectory, domain, user, password, authenticationType string,
		mountOptions *MountOptions,
		agentArns []string,
		tags map[string]string,
		smbKerberos SmbKerberosConfig,
		secretConfig SecretConfig,
	) (*Location, error)
	DescribeLocationSmb(locationArn string) (*LocationSmb, error)
	UpdateLocationSmb(
		locationArn, serverHostname, subdirectory, domain, user, password, authenticationType string,
		mountOptions *MountOptions,
		agentArns []string,
		smbKerberos SmbKerberosConfig,
		secretConfig SecretConfig,
	) error

	// Tag operations
	TagResource(resourceArn string, tags map[string]string) error
	UntagResource(resourceArn string, keys []string) error
	ListTagsForResource(resourceArn string, maxResults int32, nextToken string) (map[string]string, string, error)

	AccountID() string
	Region() string
	Reset()
	Snapshot(ctx context.Context) []byte
	Restore(ctx context.Context, data []byte) error
}

StorageBackend is the interface for DataSync storage operations.

type TaggedEntry added in v1.3.1

type TaggedEntry struct {
	Tags map[string]string
	ARN  string
}

TaggedEntry pairs a resource ARN with its tags.

type Task

type Task struct {
	CreationTime            time.Time
	Tags                    map[string]string
	Options                 map[string]any
	Schedule                *TaskSchedule
	ManifestConfig          map[string]any
	TaskReportConfig        map[string]any
	TaskArn                 string
	Name                    string
	Status                  string
	SourceLocationArn       string
	DestinationLocationArn  string
	CloudWatchLogGroupArn   string
	CurrentTaskExecutionArn string
	TaskMode                string
	Excludes                []FilterRule
	Includes                []FilterRule
}

Task represents a DataSync transfer task. CreationTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type TaskExecution

type TaskExecution struct {
	StartTime                time.Time
	Options                  map[string]any
	TaskExecutionArn         string
	Status                   string
	TaskMode                 string
	EstimatedFilesToTransfer int64
	EstimatedBytesToTransfer int64
	FilesTransferred         int64
	BytesTransferred         int64
}

TaskExecution represents a DataSync task execution. StartTime is first: time.Time's non-pointer prefix reduces GC pointer bytes.

type TaskExecutionListEntry

type TaskExecutionListEntry struct {
	TaskExecutionArn string
	Status           string
	TaskMode         string
}

TaskExecutionListEntry is a task execution entry in a list response.

type TaskListEntry

type TaskListEntry struct {
	TaskArn  string
	Name     string
	Status   string
	TaskMode string
}

TaskListEntry is a task entry in a list response.

type TaskSchedule added in v1.2.0

type TaskSchedule struct {
	ScheduleExpression string
	Status             string
}

TaskSchedule holds a task's cron/rate schedule expression and enabled status.

type TaskSettings added in v1.2.0

type TaskSettings struct {
	Options          map[string]any
	Schedule         *TaskSchedule
	ManifestConfig   map[string]any
	TaskReportConfig map[string]any
	TaskMode         string
	Excludes         []FilterRule
	Includes         []FilterRule
}

TaskSettings groups the optional CreateTask/UpdateTask configuration knobs that mirror AWS's own grouping in CreateTaskInput/UpdateTaskInput (Options, Schedule, Excludes, Includes, ManifestConfig, TaskReportConfig, TaskMode). A nil/zero field means "not supplied" (leave unchanged on UpdateTask); Options/ManifestConfig/TaskReportConfig are opaque pass-through blobs (echoed back verbatim on Describe) since their AWS shapes are deep and advisory rather than FK-validated.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL