Documentation
¶
Overview ¶
Package contentid derives identities for files so the SAME content can be recognised across torrents and mounts without re-downloading it. Two flavours, matched to the two situations the dedup-on-add feature (#23) faces:
Fingerprint — a PROBABLE match. Hashes size + the first and last SampleBytes, never the middle, so on an rclone/Drive (FUSE) mount it costs two small ranged reads instead of fetching the whole object. Cheap enough to run against cloud files; strong enough to SUGGEST "you already have this". It is a fingerprint, not a full hash: files sharing size + head + tail but differing only in the middle collide (astronomically unlikely for real media, but never trusted for a silent auto-link).
VerifyInteriorPieces — an EXACT match for local files. Re-hashes the candidate against a v1 torrent's own SHA-1 piece hashes for every piece that lies fully inside the file. Zero false positives (it is the torrent's own integrity check), at the cost of reading the file's bytes — so it is reserved for local candidates, never the cloud.
Index ¶
- Constants
- func CertainMatch(interior, matched int) bool
- func FileMatchesPieces(path string, pc PieceCheck) bool
- func Fingerprint(abs string, size int64) (string, error)
- func FingerprintAt(ra io.ReaderAt, size int64) (string, error)
- func FingerprintReadSeeker(rs io.ReadSeeker, size int64) (string, error)
- func VerifyInteriorPieces(ra io.ReaderAt, pc PieceCheck) (interior, matched int, err error)
- type PieceCheck
Constants ¶
const SampleBytes = 64 << 10 // 64 KiB
SampleBytes is how much is read from each end of a file for a Fingerprint.
Variables ¶
This section is empty.
Functions ¶
func CertainMatch ¶
CertainMatch reports an exact identity from a VerifyInteriorPieces result: at least one interior piece existed and every one matched.
func FileMatchesPieces ¶
func FileMatchesPieces(path string, pc PieceCheck) bool
FileMatchesPieces reports whether the file at path is byte-identical to the torrent file described by pc, verifying every interior piece. A missing or unreadable file, or one with no interior pieces, is not a match.
func Fingerprint ¶
Fingerprint opens abs and returns its Fingerprint (see package doc). size is the file's declared length; it is folded into the hash so length alone always distinguishes, and it bounds the head/tail reads.
func FingerprintAt ¶
FingerprintAt is Fingerprint over an already-open io.ReaderAt of the given size. Files at or under 2*SampleBytes are hashed whole (already tiny).
func FingerprintReadSeeker ¶
func FingerprintReadSeeker(rs io.ReadSeeker, size int64) (string, error)
FingerprintReadSeeker is Fingerprint over an io.ReadSeeker (e.g. a torrent file reader, which is not a ReaderAt). It writes the exact same bytes to the hash as FingerprintAt, so a torrent file and an on-disk file of identical content yield the SAME fingerprint — the basis for matching a torrent's file against a cloud/library file without a piece-by-piece check.
func VerifyInteriorPieces ¶
func VerifyInteriorPieces(ra io.ReaderAt, pc PieceCheck) (interior, matched int, err error)
VerifyInteriorPieces re-hashes the candidate file (read through ra, where offset 0 is the start of the file) against the torrent's own SHA-1 piece hashes for every piece fully inside the file. It returns how many interior pieces exist and how many matched. interior > 0 && matched == interior means the file's interior bytes are byte-identical to what the torrent expects — an exact match with no false positives (use CertainMatch). Boundary pieces shared with neighbouring files are skipped: they can't be judged from this file alone.
Types ¶
type PieceCheck ¶
type PieceCheck struct {
PieceLen int64 // torrent piece length (must be > 0)
FileStart int64 // byte offset of the file within the torrent's concatenated stream
FileLen int64 // length of the file in bytes
PieceHashes [][]byte // every piece's 20-byte SHA-1 hash, indexed by global piece index
}
PieceCheck describes where a single file sits inside a v1 torrent's piece grid.
v1-ONLY: PieceHashes must be 20-byte SHA-1 hashes (BEP3). A v2 (BEP52) torrent uses per-file SHA-256 merkle hashes that are not comparable here — callers must detect v2 and fall back to Fingerprint instead of calling VerifyInteriorPieces (which returns an error if handed non-20-byte hashes for an interior piece).
Source Files