Documentation
¶
Overview ¶
Package destinations — destination.go: the Destination interface, Parse and the kind registry.
Purpose: one interface for every place a backup can be written to or read from, so callers stop switching on URI schemes. Inputs: a destination URI and, for host://, the controlplane inventory. Outputs: a Destination of one of three kinds: rclone (s3 r2 minio b2 gcs az and bare rclone remotes written name:path), path (path://<abs dir>) and host (host://<inventory server>/<abs dir>). Constraints: Parse never touches the network or starts a process.
host.go — the host:// destination kind (a controlplane inventory server).
Purpose: store backups on a server we own over SSH, without rclone. Inputs: host://<server>/<absolute dir>; the server comes from the inventory. Outputs: objects under the directory on that server. Constraints: every ssh and scp process is started by sdk/go/remote (Run, Start, CopyTo); no argv is built here. Server name and paths are validated before any exec (remote.ValidateCopyPath allowlist, no ".."), remote paths are also single-quoted. Host keys are pinned: the D4 option set sets StrictHostKeyChecking=yes against a known_hosts file holding a key for the alias "nself-ci-<server>"; an unpinned server is refused before ssh runs.
hostio.go — I/O helpers for the host:// destination.
Purpose: stream a remote file through an ssh session and hash local files. Inputs: a started remote.Session; a local path. Outputs: an io.ReadCloser that reports the remote exit status on Close. Constraints: no process is started here; sdk/go/remote owns every exec.
path.go — the path:// destination kind (local or mounted disk).
Purpose: store backups under an absolute directory without rclone. Inputs: path://<absolute dir>, local files and keys. Outputs: objects under the directory, files 0600, directories 0700. Constraints: every access goes through os.Root, so a ".." or a symlink cannot leave the directory; the directory itself and every component below it must not be a symlink; writes go to <key>.tmp, are fsynced, renamed and re-read to compare sha256. A symlink at the final component is refused, never followed.
rclone.go — the rclone destination kind and the pre-interface helpers.
Purpose: drive rclone for s3, r2, minio, b2, gcs and az remotes. IsRemoteKey, FetchRemote and toRclonePath predate the Destination interface and keep their exact behaviour and argv for existing callers (pitr restore). Inputs: remote URIs such as s3://bucket/path/to/object. Outputs: files downloaded or uploaded through rclone. Constraints: rclone argv is frozen (see rclone_golden_test.go).
Remote keys follow rclone's URI scheme:
s3://bucket/path/to/object r2://bucket/path/to/object minio://bucket/path/to/object b2://bucket/path/to/object gcs://bucket/path/to/object az://container/path/to/object
The minio:// scheme is an nSelf convention: it is rewritten to a "minio" rclone remote automatically (rclone remote name "minio"). All other schemes are passed through to rclone verbatim (scheme becomes the rclone remote name).
rclone must be installed and configured (rclone.conf or environment variables) before FetchRemote is called.
Index ¶
Constants ¶
const ( KindRclone = "rclone" KindPath = "path" KindHost = "host" )
Kind names.
const KnownHostsEnv = "NSELF_BACKUP_KNOWN_HOSTS"
KnownHostsEnv overrides the pinned known_hosts file host:// uses.
Variables ¶
This section is empty.
Functions ¶
func FetchRemote ¶
FetchRemote downloads the object at remoteKey to a local file in destDir. It returns the local file path on success.
remoteKey must be a URI understood by rclone (or a minio:// URI — see package-level documentation for the rewrite rules).
If remoteKey ends in ".age" the caller is responsible for decrypting the returned file; FetchRemote copies the raw (still encrypted) bytes.
func IsNativeKey ¶ added in v1.4.13
IsNativeKey reports whether uri is a path:// or host:// destination.
func IsRemoteKey ¶
IsRemoteKey reports whether key is a supported remote destination URI (i.e. not a local filesystem path).
func KindOf ¶ added in v1.4.13
KindOf reports the kind a destination URI selects. An empty URI is "".
func KnownHostsPath ¶ added in v1.4.13
func KnownHostsPath() string
KnownHostsPath is the pinned known_hosts file host:// reads.
Types ¶
type Destination ¶ added in v1.4.13
type Destination interface {
Kind() string
Put(ctx context.Context, localPath, key string) error
Get(ctx context.Context, key, localPath string) error
List(ctx context.Context, prefix string) ([]Object, error)
}
Destination is a place backups are stored. key is a relative, slash separated name inside the destination.
func Parse ¶ added in v1.4.13
func Parse(uri string, inv *Inventory, rcloneEnv ...string) (Destination, error)
Parse returns the Destination uri names. rcloneEnv is appended to the environment of every rclone process (nil: none); inv is used by host:// only and may be nil for the other kinds.
func ParseObject ¶ added in v1.4.13
func ParseObject(uri string, inv *Inventory) (Destination, string, error)
ParseObject parses a URI that names one object (restore-remote --from). For path:// and host:// the last path element is the key and the rest is the destination directory. Other URIs are returned whole with an empty key.
type Inventory ¶ added in v1.4.13
type Inventory = controlplane.Inventory
Inventory is the controlplane inventory host:// resolves servers from.
Source Files