Documentation
¶
Index ¶
- Constants
- Variables
- func ComputeHash(data []byte) string
- func ComputeJSONHash(v interface{}) (string, []byte, error)
- func FileMetaRef(f *FileMeta) (string, []byte, error)
- func IsSelfAddressed(ref string) bool
- func VerifyRef(ref string, data []byte) error
- type Content
- type ErrRefMismatch
- type FileMeta
- type FileType
- type Index
- type ObjectType
- type RepoConfig
- type Snapshot
- type SnapshotSummary
- type SourceInfo
Constants ¶
const ( // RepoFormatVersion is stamped into every repository this build creates. // // It tracks MaxSupportedRepoFormat deliberately. The version is not only a // claim about the bytes currently present — it is the signal that tells // other machines sharing this repository to upgrade. A heterogeneous fleet // is the dangerous state, so a repository touched by a build that can seal // says so, and older builds are told to catch up rather than left writing // alongside it. RepoFormatVersion = 2 // MaxSupportedRepoFormat is the highest version this build can read. A // repository above it is refused rather than misread. // // 2 covers a sealed pack index and a sharded one. Builds before that read // the sealed catalog as unparseable and, without the fixes released in // v1.15.0, as empty — which is how a prune deletes a live repository. They // would also read the pre-shard monolithic catalog as complete when it is // merely stale, which is the same failure by a different route. // // Both changes ship in the same release, so one version covers them. MaxSupportedRepoFormat = 2 // FramedCompressionFormat is the lowest recorded format at which the // compression layer may write framed objects (see pkg/store/compressed.go). // // It gates writes rather than reads: a framed object is always readable, // but a build predating the frame reads one as opaque bytes and returns // them, which is a misread rather than a clean refusal. The version gate // cannot prevent that on its own, because the stamp is applied after a // mutation completes — so a repository still recording format 1 would // otherwise be handed framed objects that an older build sails straight // into. Framing only once the repository already records this version // closes that window without stamping repositories speculatively. FramedCompressionFormat = 2 )
RepoConfig is the repository marker written by "init". It is stored as plaintext at key "config" so it can be read without the encryption key. Key: config Repository format versioning. See docs/compatibility.md for the contract these constants enforce.
RepoFormatVersion is stamped into every repository this build creates. MaxSupportedRepoFormat is the highest version this build can read; a repository above it is refused rather than misread.
Raise both together when a change makes a repository unreadable by earlier builds. Do not raise them for a change earlier builds can still read: a needless bump locks users out of their own data for no benefit.
const ( FileTypeFile = source.FileTypeFile FileTypeFolder = source.FileTypeFolder )
FileType values.
Variables ¶
var SelfAddressedPrefixes = []string{"filemeta/", "node/", "snapshot/"}
SelfAddressedPrefixes are the object namespaces whose key is the SHA-256 of the bytes stored under it. For these, and only these, the key is a checksum the reader can verify without holding any secret.
The other namespaces are deliberately absent. A chunk is named by an HMAC of its bytes when the repository is encrypted, so verifying it needs the dedup key rather than a bare hash; `check -read-data` does that where the key is available. A content object is named by an HMAC of the *file's* hash, not of its own bytes, so it is verified through the file it describes — restore hashes the reconstructed stream against FileMeta.ContentHash.
Functions ¶
func ComputeHash ¶
ComputeHash computes the SHA-256 hash of the given data and returns it as a hex string.
func ComputeJSONHash ¶
ComputeJSONHash computes the SHA-256 hash of the JSON representation of the object. This uses encoding/json which sorts map keys, providing a canonical form for basic structs.
func FileMetaRef ¶ added in v1.18.0
FileMetaRef returns the object key and encoded bytes for a file's metadata.
It is a function rather than a method on FileMeta because FileMeta is now defined in pkg/source: "filemeta/<hash>" is repository-format naming, which a source implementation has no business knowing about.
func IsSelfAddressed ¶ added in v1.17.0
IsSelfAddressed reports whether ref lives in a namespace VerifyRef can check.
func VerifyRef ¶ added in v1.17.0
VerifyRef checks that data is the object ref names.
Repository objects form a Merkle tree: a snapshot names a root node, nodes name child nodes and filemeta refs, and a filemeta names its content. Every link in that chain is a SHA-256 of the bytes on the other end, so verifying a ref on the way in authenticates the whole tree beneath it against anything short of rewriting the snapshot ref itself.
Objects outside SelfAddressedPrefixes are not self-checking, and passing one here is a programming error rather than a corrupt repository — hence the distinct error. Callers that handle mixed keys should test the prefix first.
Types ¶
type Content ¶
type Content struct {
Type ObjectType `json:"type"` // "content"
Size int64 `json:"size"`
Chunks []string `json:"chunks,omitempty"` // List of "chunk/<sha256>"
DataInlineB64 []byte `json:"data_inline_b64,omitempty"` // For small files
}
Content represents a file's content as a list of chunks Object key: content/<sha256>
type ErrRefMismatch ¶ added in v1.17.0
ErrRefMismatch reports an object whose bytes are not what its key names.
It is deliberately one error for every such object type: whether the cause is bit rot, a truncated write, or a store that substituted the object, the caller's correct response is the same — refuse the bytes.
func (*ErrRefMismatch) Error ¶ added in v1.17.0
func (e *ErrRefMismatch) Error() string
type FileMeta ¶
The source-facing domain types are defined in pkg/source and aliased here.
The definitions moved so that the public Source contract does not depend on an internal package; the aliases stay so that the engine, the HAMT and the stored JSON keep spelling them core.FileMeta and core.SourceInfo. A Go alias denotes the identical type, so nothing about the on-disk format changes.
type FileType ¶
The source-facing domain types are defined in pkg/source and aliased here.
The definitions moved so that the public Source contract does not depend on an internal package; the aliases stay so that the engine, the HAMT and the stored JSON keep spelling them core.FileMeta and core.SourceInfo. A Go alias denotes the identical type, so nothing about the on-disk format changes.
type Index ¶
type Index struct {
LatestSnapshot string `json:"latest_snapshot"` // "snapshot/<sha256>"
Seq int `json:"seq"`
}
Index represents a pointer to the latest snapshot Key: index/latest
type ObjectType ¶
type ObjectType string
ObjectType defines the type of the object in the system
const (
ObjectTypeContent ObjectType = "content"
)
type RepoConfig ¶
type Snapshot ¶
type Snapshot struct {
Version int `json:"version"`
Created string `json:"created"` // ISO8601
Root string `json:"root"` // "node/<sha256>"
Seq int `json:"seq"`
Source *SourceInfo `json:"source,omitempty"`
Meta map[string]string `json:"meta,omitempty"`
Tags []string `json:"tags,omitempty"`
ChangeToken string `json:"change_token,omitempty"`
ExcludeHash string `json:"exclude_hash,omitempty"`
}
Snapshot represents a backup checkpoint Object key: snapshot/<sha256>
type SnapshotSummary ¶
type SnapshotSummary struct {
Ref string `json:"ref"` // "snapshot/<hash>"
Seq int `json:"seq"`
Created string `json:"created"` // ISO8601
Root string `json:"root"` // "node/<hash>"
Source *SourceInfo `json:"source,omitempty"`
Tags []string `json:"tags,omitempty"`
ChangeToken string `json:"change_token,omitempty"`
ExcludeHash string `json:"exclude_hash,omitempty"`
}
SnapshotSummary is a lightweight representation of a snapshot stored in the snapshot catalog index. It contains enough metadata for listing, filtering, and finding the previous snapshot without having to fetch the full object.
type SourceInfo ¶
type SourceInfo = source.SourceInfo
The source-facing domain types are defined in pkg/source and aliased here.
The definitions moved so that the public Source contract does not depend on an internal package; the aliases stay so that the engine, the HAMT and the stored JSON keep spelling them core.FileMeta and core.SourceInfo. A Go alias denotes the identical type, so nothing about the on-disk format changes.
Source Files