Documentation
¶
Overview ¶
Package rclone talks to the rclone daemon this distribution already installs.
The installer puts rclone on every box and runs it as a service:
ExecStart=/usr/bin/rclone rcd --rc-addr unix:///var/run/rclone/rclone.sock --rc-no-auth
which means S3, SFTP, FTP, the retries, the resume after a broken connection and the incremental comparison are already here, already running, and already driven over an API. None of that is worth writing again in Go.
Two things about that daemon are worth knowing before trusting it with credentials. It listens with --rc-no-auth, so any local process can read and write every remote it holds; that is already true of the cloud drives CasaOS stores there today. And it is pinned at v1.61.1, which is old but long predates every backend used here.
Index ¶
- Constants
- Variables
- func Destination(name, remotePath string) string
- type Client
- func (c *Client) CheckDestination(name string) (DestinationSpace, error)
- func (c *Client) CopyDirectory(ctx context.Context, source, destinationName, destinationPath string) error
- func (c *Client) CopyFile(ctx context.Context, source, destinationName, destinationPath string) error
- func (c *Client) CreateDestination(name, backend, root string, parameters map[string]string) error
- func (c *Client) CreateEncryptedDestination(name, backend, root string, parameters map[string]string, password string) error
- func (c *Client) DeleteDestination(name string) error
- func (c *Client) Destinations() ([]string, error)
- func (c *Client) Fetch(ctx context.Context, destinationName, remotePath string) ([]byte, error)
- func (c *Client) JobStatus(id int64) (JobStatus, error)
- func (c *Client) Purge(ctx context.Context, destinationName, remotePath string) error
- func (c *Client) RestoreDirectory(ctx context.Context, destinationName, remotePath, hostPath string) error
- func (c *Client) RestoreFile(ctx context.Context, destinationName, remotePath, hostPath string) error
- func (c *Client) Runs(destinationName, appPath string) ([]string, error)
- func (c *Client) StopJob(id int64) error
- type DestinationSpace
- type JobStatus
Constants ¶
const DestinationPrefix = "casaos-backup-"
DestinationPrefix marks a remote as one of ours.
Backup destinations live in rclone's own config, next to the cloud drives CasaOS mounts -- which is deliberate: the credentials then sit where this project already keeps credentials of exactly that kind, rather than in a second store invented for the purpose. The prefix is what tells the two apart, and it is readable in `rclone config` by anyone wondering what CasaOS put there.
const RawSuffix = "-raw"
RawSuffix marks the remote underneath a destination that has a root or is encrypted: the bucket or the server itself (which, when encrypted, holds only ciphertext), never listed as a destination in its own right. A name ending in it is refused, so the two can always be told apart.
const SocketPath = "/var/run/rclone/rclone.sock"
SocketPath is where the unit file puts the daemon's socket.
Variables ¶
var ErrDaemonUnreachable = errors.New("the rclone daemon is not answering on " + SocketPath)
ErrDaemonUnreachable is returned when the socket is not there or not answering. It is worth its own error: it means rclone is not running, which is a different problem from a destination being misconfigured, and the two get different advice.
Functions ¶
func Destination ¶
Destination turns a destination name and a path inside it into the `remote:path` rclone expects.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client drives the rclone daemon over its unix socket.
func NewClient ¶
func NewClient() *Client
NewClient returns a client bound to the daemon's socket.
The timeout here covers control calls only -- listing remotes, creating one, checking one answers. A transfer is started as a job and polled, so no copy ever has to finish inside one request.
func (*Client) CheckDestination ¶
func (c *Client) CheckDestination(name string) (DestinationSpace, error)
CheckDestination asks a destination whether it is reachable, and what it can say about its space.
Worth doing at the moment somebody configures a destination rather than at the moment a backup runs: a typo in a bucket name found at 3am by a scheduled job nobody is watching is a backup that never existed.
func (*Client) CopyDirectory ¶
func (c *Client) CopyDirectory(ctx context.Context, source, destinationName, destinationPath string) error
CopyDirectory copies a folder and everything under it, and blocks until it is done.
This is rclone's `copy`, not `sync`: it adds and updates, and never deletes anything at the destination. A backup that deletes is a backup that can destroy the previous one when the source is wrong -- an app whose folder was unmounted at the moment of the run looks exactly like an app whose files were all deleted.
func (*Client) CopyFile ¶
func (c *Client) CopyFile(ctx context.Context, source, destinationName, destinationPath string) error
CopyFile copies a single file.
rclone addresses a file as a folder plus a name, on both sides, which is why this takes the whole path and splits it: handing `/etc/app/php.ini` to the directory call above would copy `/etc/app` entire.
func (*Client) CreateDestination ¶
CreateDestination adds or replaces a backup destination.
`backend` is an rclone backend name -- "s3", "sftp", "ftp" -- and `parameters` are that backend's own options. Neither is interpreted here: rclone validates them, rclone knows what each backend needs, and a list of required fields maintained on this side would go stale the first time rclone gained an option.
`root` is where on the backend everything is put, handed to rclone as it is: a folder, or for an object store the bucket (rclone has no bucket option; the first folder of a path is the bucket, which left to itself is named after the app, and a name at a provider such as AWS belongs to whoever took it first). Empty, the destination is the backend's own root.
func (*Client) CreateEncryptedDestination ¶ added in v0.4.44
func (c *Client) CreateEncryptedDestination(name, backend, root string, parameters map[string]string, password string) error
CreateEncryptedDestination makes a destination whose contents -- names and bytes both -- are encrypted before they leave this box.
Two remotes: the backend itself under the raw name, and rclone's crypt backend on top of it under the destination's name. Everything else in this package talks to the top one and never knows the difference; a bucket at a provider holds ciphertext it cannot read. The password is obscured by rclone into its config the way it obscures every password it keeps, and is not readable back through this API.
func (*Client) DeleteDestination ¶
DeleteDestination forgets a destination and its credentials. Whatever was already copied to it stays where it is: this removes the way in, not the backup.
func (*Client) Destinations ¶
Destinations lists the remotes this project put there, by their short name -- the prefix is an implementation detail and does not belong in an interface.
func (*Client) Fetch ¶ added in v0.4.41
Fetch reads one file held at a destination.
rclone's API has no "read this file" call; the file is copied into a folder of ours and read from there. Small files only -- a manifest, a compose file -- which is all a restore needs to read before it starts copying.
func (*Client) Purge ¶
Purge deletes one run and everything under it.
The only call in this package that removes anything, kept separate from the copies for that reason: nothing that copies can reach it by accident.
func (*Client) RestoreDirectory ¶ added in v0.4.41
func (c *Client) RestoreDirectory(ctx context.Context, destinationName, remotePath, hostPath string) error
RestoreDirectory makes hostPath hold what the destination holds under remotePath.
sync/sync rather than sync/copy: a restore means "as it was", and a file that appeared in that folder after the backup was taken is not part of "as it was". This is the one call in the package that deletes something on this host, and it deletes exactly what a restore is asked to undo.
func (*Client) RestoreFile ¶ added in v0.4.41
func (c *Client) RestoreFile(ctx context.Context, destinationName, remotePath, hostPath string) error
RestoreFile puts one file from the destination at hostPath.
type DestinationSpace ¶
type DestinationSpace struct {
// Used and Free are bytes, and either may be absent: plenty of backends,
// object stores especially, have no answer to "how much room is left", and
// inventing one would be worse than saying nothing.
Used *int64 `json:"used,omitempty"`
Free *int64 `json:"free,omitempty"`
Total *int64 `json:"total,omitempty"`
}
DestinationSpace is what a destination says about itself when asked.
Source Files