Documentation
¶
Overview ¶
Package backupfile reads and writes the folder a backup is kept in:
manifest.json what the backup holds, written last
<kind>.jsonl the records of one kind, a JSON line each
content/aa/bb/<name> file content, by its checksum or, encrypted, by a
keyed hash of it
A record file begins with a header line. With a key the records follow in encrypted blocks of about a thousand lines, so a backup of any size is read and written a block at a time. Content is stored once however many files share it, so repeating a backup into the same folder copies only what is new; with a key it is encrypted too, under names that do not give away its checksum (see Content).
Index ¶
- Constants
- Variables
- func ClearUnfinished(dir string) error
- func ContentPath(dir, sha string) string
- func DownloadContent(dir, sha string, fetch Fetch) (int64, error)
- func IsBackupDir(dir string) bool
- func MarkUnfinished(dir string) error
- func PartialPath(dir, sha string) string
- func Prune(dir string, keep func(sha string) bool) (removed int, freed int64, err error)
- func RecordPath(dir, kind string) string
- func WriteManifest(dir string, m *Manifest) error
- type Content
- func (c *Content) Download(sha string, fetch Fetch) (int64, error)
- func (c *Content) Encrypted() bool
- func (c *Content) Has(sha string) bool
- func (c *Content) HasOfSize(sha string, size int64) bool
- func (c *Content) Name(sha string) string
- func (c *Content) Open(sha string) (io.ReadCloser, int64, error)
- func (c *Content) Prune(wanted map[string]struct{}) (int, int64, error)
- func (c *Content) StoredSize(size int64) int64
- type Fetch
- type Manifest
- type Reader
- type Writer
Constants ¶
const (
// Version is the folder format.
Version = 1
)
Variables ¶
var ErrWrongKey = errors.New("cannot decrypt the backup: wrong key, or the file is damaged")
ErrWrongKey is returned when a block does not decrypt.
Functions ¶
func ClearUnfinished ¶
ClearUnfinished removes the mark once the backup is complete.
func ContentPath ¶
ContentPath is where the content with a checksum is kept.
func DownloadContent ¶
DownloadContent copies the content with a checksum into the backup, carrying on from a partial copy if there is one, and checking the whole against the checksum before keeping it. A failure leaves the partial copy for the next attempt; content that fails its checksum is discarded. It returns the bytes copied in this call.
func IsBackupDir ¶
IsBackupDir reports whether dir holds a backup, finished or not.
func MarkUnfinished ¶
MarkUnfinished records that a backup into dir has begun.
func PartialPath ¶
PartialPath is where content with a checksum is kept while it is being copied. A copy that fails leaves it, so the next attempt, in this run or the next, carries on from there.
func Prune ¶
Prune removes content that no file refers to: keep reports whether a content file's name (see Content.Name) is still wanted. It also clears partial copies. It returns how many files it removed and the bytes they held.
func RecordPath ¶
RecordPath is the file the records of a kind are kept in.
func WriteManifest ¶
WriteManifest writes the manifest into dir.
Types ¶
type Content ¶ added in v0.42.0
type Content struct {
// contains filtered or unexported fields
}
Content is the file content of a backup folder. Without a key it is kept as it is, under its checksum. With a key it is encrypted, so a folder that leaves the server gives away neither what its files hold nor, by their names, whether it holds a file someone already has:
- each content file is named by an HMAC of its checksum under a key derived from the backup key, rather than by the checksum;
- its content is encrypted with AES-256-GCM under a key derived from the backup key and its checksum, in chunks of 64 KiB, each checked on its own. A chunk's nonce is its number, with the last one marked, so a file cut short, or with its chunks in another order, does not decrypt.
The same content always encrypts the same way under the same key, so content shared by many files is still kept once, and a backup repeated into the same folder copies only what is new. A key is never used for two different contents, so a nonce is never used twice under one key.
An encrypted content file is a header, magic and version, then its chunks, each its ciphertext and tag:
"KNBC" 1 chunk... (64 KiB + 16 bytes each, the last shorter)
func NewContent ¶ added in v0.42.0
NewContent is the content of the backup in dir: encrypted with key when encrypted is set, as the manifest's ContentEncrypted says or, for a new backup, whenever there is a key.
func (*Content) Download ¶ added in v0.42.0
Download copies the content with a checksum into the backup, carrying on from a partial copy if there is one, and checking the whole against the checksum before keeping it. A failure leaves the partial copy for the next attempt; content that fails its checksum is discarded. It returns the bytes of content copied in this call.
func (*Content) Has ¶ added in v0.42.0
Has reports whether the backup holds the content with a checksum.
func (*Content) HasOfSize ¶ added in v0.42.0
HasOfSize is Has that also checks the size, so content a crash left empty or cut short is copied again rather than trusted. Content is not synced file by file, which would make a backup of many small files crawl.
func (*Content) Open ¶ added in v0.42.0
Open opens the content with a checksum, returning it as it was backed up and its size. Encrypted content is decrypted as it is read: a chunk that does not decrypt, or a file cut short, fails the read.
func (*Content) Prune ¶ added in v0.42.0
Prune removes content that no file refers to: wanted holds the checksums still referred to. It also clears partial copies, and content kept under another key or none, which no file of this backup can be read from. It returns how many files it removed and the bytes they held.
func (*Content) StoredSize ¶ added in v0.42.0
StoredSize is the size, in the folder, of content of size bytes.
type Fetch ¶
type Fetch func(offset int64) (rc io.ReadCloser, resumed bool, err error)
Fetch opens content from byte offset. It returns the stream and whether the stream starts at offset: false means it starts at the beginning, because the server sent everything again.
type Manifest ¶
type Manifest struct {
Version int `json:"version" msgpack:"version"`
Created time.Time `json:"created" msgpack:"created"`
Server string `json:"server" msgpack:"server"` // the knot version that made it
Encrypted bool `json:"encrypted" msgpack:"encrypted"`
Counts map[string]int `json:"counts" msgpack:"counts"`
Content bool `json:"content" msgpack:"content"` // file content was copied
// ContentEncrypted says the content is encrypted with the key, as it is
// in a backup with a key since knot 0.42 (see Content); in one before,
// only the records were.
ContentEncrypted bool `json:"content_encrypted,omitempty" msgpack:"content_encrypted,omitempty"`
}
Manifest says what a backup holds. It is written when the backup is complete, so a folder without one is unfinished.
func ReadManifest ¶
ReadManifest reads the manifest of the backup in dir.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader reads the records of one kind.
Source Files