Documentation
¶
Overview ¶
Package rclone talks to the rclone daemon this distribution already installs.
The installer puts rclone on every box and runs it as a service:
ExecStart=/usr/bin/rclone rcd --rc-addr unix:///var/run/rclone/rclone.sock --rc-no-auth
which means S3, SFTP, FTP, the retries, the resume after a broken connection and the incremental comparison are already here, already running, and already driven over an API. None of that is worth writing again in Go.
Two things about that daemon are worth knowing before trusting it with credentials. It listens with --rc-no-auth, so any local process can read and write every remote it holds; that is already true of the cloud drives CasaOS stores there today. And it is pinned at v1.61.1, which is old but long predates every backend used here.
Index ¶
- Constants
- Variables
- func CompleteSignIn(redirected string) (string, map[string]string, error)
- func Destination(name, remotePath string) string
- func SignIn(backend string) (string, error)
- func SignInBackends() []string
- type BadSignIn
- type Client
- func (c *Client) CheckDestination(name string) (DestinationSpace, error)
- func (c *Client) CopyBack(ctx context.Context, destinationName, remotePath, sub, hostPath string) error
- func (c *Client) CopyDirectory(ctx context.Context, source, destinationName, destinationPath string) error
- func (c *Client) CopyFile(ctx context.Context, source, destinationName, destinationPath string) error
- func (c *Client) CreateDestination(name, backend, root string, parameters map[string]string) error
- func (c *Client) CreateEncryptedDestination(name, backend, root string, parameters map[string]string, password string) error
- func (c *Client) DeleteDestination(name string) error
- func (c *Client) Destinations() ([]string, error)
- func (c *Client) Fetch(ctx context.Context, destinationName, remotePath string) ([]byte, error)
- func (c *Client) FetchLimited(ctx context.Context, destination, remotePath string, maxBytes int64) (data []byte, fetchErr error)
- func (c *Client) InspectBackupPath(ctx context.Context, destination, remotePath string, directory bool) (VerificationSize, error)
- func (c *Client) JobStatus(id int64) (JobStatus, error)
- func (c *Client) List(_ context.Context, destinationName, remotePath, sub string) ([]File, error)
- func (c *Client) Purge(ctx context.Context, destinationName, remotePath string) error
- func (c *Client) RestoreDirectory(ctx context.Context, destinationName, remotePath, hostPath string) error
- func (c *Client) RestoreFile(ctx context.Context, destinationName, remotePath, hostPath string) error
- func (c *Client) Runs(destinationName, appPath string) ([]string, error)
- func (c *Client) StagingFreeSpace(ctx context.Context, stagingDir string) (int64, error)
- func (c *Client) StopJob(id int64) error
- func (c *Client) VerifyBackupPath(ctx context.Context, destination, remotePath, stagingPath string, ...) (verifyErr error)
- type DestinationSpace
- type File
- type JobStatus
- type NotFoundError
- type VerificationSize
Constants ¶
const DestinationPrefix = "casaos-backup-"
DestinationPrefix marks a remote as one of ours.
Backup destinations live in rclone's own config, next to the cloud drives CasaOS mounts -- which is deliberate: the credentials then sit where this project already keeps credentials of exactly that kind, rather than in a second store invented for the purpose. The prefix is what tells the two apart, and it is readable in `rclone config` by anyone wondering what CasaOS put there.
const RawSuffix = "-raw"
RawSuffix marks the remote underneath a destination that has a root or is encrypted: the bucket or the server itself (which, when encrypted, holds only ciphertext), never listed as a destination in its own right. A name ending in it is refused, so the two can always be told apart.
const SocketPath = "/var/run/rclone/rclone.sock"
SocketPath is where the unit file puts the daemon's socket.
Variables ¶
var ErrDaemonUnreachable = errors.New("the rclone daemon is not answering on " + SocketPath)
ErrDaemonUnreachable is returned when the socket is not there or not answering. It is worth its own error: it means rclone is not running, which is a different problem from a destination being misconfigured, and the two get different advice.
var ErrNoSignIn = BadSignIn("no sign-in is waiting: start it again")
ErrNoSignIn is what completing a sign-in says when none is waiting: it was never started, it timed out, or a newer one took its place.
var ErrVerificationJobUncertain = errors.New("the verification job could not be confirmed stopped")
ErrVerificationJobUncertain means a daemon job may still access its staging directory. The caller must retain that directory instead of removing it.
Functions ¶
func CompleteSignIn ¶ added in v0.4.84
CompleteSignIn takes the address of the page the provider redirected to -- the one that failed to load -- and returns the token rclone then produced, with the other options that backend needs (OneDrive's drive_id and drive_type), if any.
func Destination ¶
Destination turns a destination name and a path inside it into the `remote:path` rclone expects.
func SignIn ¶ added in v0.4.84
SignIn starts a sign-in to backend and returns the address to open in a browser. It replaces any sign-in already waiting.
func SignInBackends ¶ added in v0.4.84
func SignInBackends() []string
SignInBackends lists the backends SignIn handles.
Types ¶
type BadSignIn ¶ added in v0.4.84
type BadSignIn string
BadSignIn is a sign-in error that comes from what was asked or pasted, not from rclone or the box: the person can fix it, so it is reported as a bad request.
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client drives the rclone daemon over its unix socket.
func NewClient ¶
func NewClient() *Client
NewClient returns a client bound to the daemon's socket.
The timeout here covers control calls only -- listing remotes, creating one, checking one answers. A transfer is started as a job and polled, so no copy ever has to finish inside one request.
func (*Client) CheckDestination ¶
func (c *Client) CheckDestination(name string) (DestinationSpace, error)
CheckDestination asks a destination whether it is reachable, and what it can say about its space.
Worth doing at the moment somebody configures a destination rather than at the moment a backup runs: a typo in a bucket name found at 3am by a scheduled job nobody is watching is a backup that never existed.
func (*Client) CopyBack ¶ added in v0.4.83
func (c *Client) CopyBack(ctx context.Context, destinationName, remotePath, sub, hostPath string) error
CopyBack puts `sub` of the copy at remotePath on hostPath: a file where the file goes, a folder's contents into the folder.
sync/copy rather than sync/sync, unlike RestoreDirectory: this is a few things taken back out of a backup, and what else is in the folder now is not part of the request. Nothing is deleted here.
func (*Client) CopyDirectory ¶
func (c *Client) CopyDirectory(ctx context.Context, source, destinationName, destinationPath string) error
CopyDirectory copies a folder and everything under it, and blocks until it is done.
This is rclone's `copy`, not `sync`: it adds and updates, and never deletes anything at the destination. A backup that deletes is a backup that can destroy the previous one when the source is wrong -- an app whose folder was unmounted at the moment of the run looks exactly like an app whose files were all deleted.
func (*Client) CopyFile ¶
func (c *Client) CopyFile(ctx context.Context, source, destinationName, destinationPath string) error
CopyFile copies a single file.
rclone addresses a file as a folder plus a name, on both sides, which is why this takes the whole path and splits it: handing `/etc/app/php.ini` to the directory call above would copy `/etc/app` entire.
func (*Client) CreateDestination ¶
CreateDestination adds or replaces a backup destination.
`backend` is an rclone backend name -- "s3", "sftp", "ftp" -- and `parameters` are that backend's own options. Neither is interpreted here: rclone validates them, rclone knows what each backend needs, and a list of required fields maintained on this side would go stale the first time rclone gained an option.
`root` is where on the backend everything is put, handed to rclone as it is: a folder, or for an object store the bucket (rclone has no bucket option; the first folder of a path is the bucket, which left to itself is named after the app, and a name at a provider such as AWS belongs to whoever took it first). Empty, the destination is the backend's own root.
func (*Client) CreateEncryptedDestination ¶ added in v0.4.44
func (c *Client) CreateEncryptedDestination(name, backend, root string, parameters map[string]string, password string) error
CreateEncryptedDestination makes a destination whose contents -- names and bytes both -- are encrypted before they leave this box.
Two remotes: the backend itself under the raw name, and rclone's crypt backend on top of it under the destination's name. Everything else in this package talks to the top one and never knows the difference; a bucket at a provider holds ciphertext it cannot read. The password is obscured by rclone into its config the way it obscures every password it keeps, and is not readable back through this API.
func (*Client) DeleteDestination ¶
DeleteDestination forgets a destination and its credentials. Whatever was already copied to it stays where it is: this removes the way in, not the backup.
func (*Client) Destinations ¶
Destinations lists the remotes this project put there, by their short name -- the prefix is an implementation detail and does not belong in an interface.
func (*Client) Fetch ¶ added in v0.4.41
Fetch reads one file held at a destination.
rclone's API has no "read this file" call; the file is copied into a folder of ours and read from there. Small files only -- a manifest, a compose file -- which is all a restore needs to read before it starts copying.
func (*Client) FetchLimited ¶ added in v0.4.92
func (c *Client) FetchLimited(ctx context.Context, destination, remotePath string, maxBytes int64) (data []byte, fetchErr error)
FetchLimited bounds both transfer and the final local read. It never follows a symlink and retains its private directory if a job might still write there.
func (*Client) InspectBackupPath ¶ added in v0.4.92
func (c *Client) InspectBackupPath(ctx context.Context, destination, remotePath string, directory bool) (VerificationSize, error)
InspectBackupPath refuses missing objects, type mismatches and unknown sizes. Empty directories must actually exist; an absent prefix is not an empty backup.
func (*Client) List ¶ added in v0.4.83
List lists one level of `sub` inside the copy at remotePath: its files and its folders. An empty sub is the copy itself.
func (*Client) Purge ¶
Purge deletes one run and everything under it.
The only call in this package that removes anything, kept separate from the copies for that reason: nothing that copies can reach it by accident.
func (*Client) RestoreDirectory ¶ added in v0.4.41
func (c *Client) RestoreDirectory(ctx context.Context, destinationName, remotePath, hostPath string) error
RestoreDirectory makes hostPath hold what the destination holds under remotePath.
sync/sync rather than sync/copy: a restore means "as it was", and a file that appeared in that folder after the backup was taken is not part of "as it was". This is the one call in the package that deletes something on this host, and it deletes exactly what a restore is asked to undo.
func (*Client) RestoreFile ¶ added in v0.4.41
func (c *Client) RestoreFile(ctx context.Context, destinationName, remotePath, hostPath string) error
RestoreFile puts one file from the destination at hostPath.
func (*Client) Runs ¶
Runs lists the runs an app has at a destination, by their stamp.
Directories only, one level down: the layout puts every run of an app in its own folder under the app's, and nothing else lives there.
func (*Client) StagingFreeSpace ¶ added in v0.4.92
func (*Client) VerifyBackupPath ¶ added in v0.4.92
func (c *Client) VerifyBackupPath(ctx context.Context, destination, remotePath, stagingPath string, directory bool, maxBytes int64) (verifyErr error)
VerifyBackupPath copies into a caller-owned private staging path and compares all bytes with a second remote read. This proves current readability, not an application's ability to start or equality with unrecorded historical hashes.
type DestinationSpace ¶
type DestinationSpace struct {
// Used and Free are bytes, and either may be absent: plenty of backends,
// object stores especially, have no answer to "how much room is left", and
// inventing one would be worse than saying nothing.
Used *int64 `json:"used,omitempty"`
Free *int64 `json:"free,omitempty"`
Total *int64 `json:"total,omitempty"`
}
DestinationSpace is what a destination says about itself when asked.
type File ¶ added in v0.4.83
type File struct {
Name string `json:"name"`
Directory bool `json:"directory"`
Size int64 `json:"size"`
Modified time.Time `json:"modified"`
}
File is one file or folder held in a backup.
type JobStatus ¶
type JobStatus struct {
ID int64 `json:"id"`
Finished bool `json:"finished"`
Success bool `json:"success"`
Error string `json:"error"`
Duration float64 `json:"duration"`
// Output is the endpoint's own answer, which can say failure while the job succeeds.
Output json.RawMessage `json:"output"`
}
JobStatus is what the daemon says about a job.
type NotFoundError ¶ added in v0.4.83
type NotFoundError struct{ Remote string }
NotFoundError is a path the destination does not hold.
func (*NotFoundError) Error ¶ added in v0.4.83
func (e *NotFoundError) Error() string
Source Files