lfsscan

package
v0.16.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 12, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Overview

Package lfsscan is the leaf reachability layer of unreferenced-LFS-object reclamation (issue #73): a lenient Git LFS pointer-file decoder and a bounded `git rev-list`/`cat-file` walk collecting the OIDs reachable pointer files reference. It deliberately sits below internal/gitread — internal/jobs consumes it, and gitread's githttp dependency would cycle.

Index

Constants

View Source
const LFSPointerMaxSize = 1024

LFSPointerMaxSize is the pointer-decode window: the prefix of a candidate blob the reclamation scan parses, mirroring git-lfs's blobSizeCutoff — its decoder reads at most this many bytes and accepts a blob of ANY total size whose window decodes (a padded pointer smudges fine, Codex security round 19). The scan therefore parses this window of every reachable blob rather than filtering on total size; a false positive only protects an object from retirement, never endangers one.

Variables

View Source
var ErrLFSScanBudget = errors.New("lfs reachability scan budget exceeded")

ErrLFSScanBudget marks a reachability walk aborted because it charged more work units than the caller's budget allows. The partial OID set is unusable for reclamation — an incomplete reachable set must never retire anything — so callers skip the repository and retry when budget allows.

Functions

func ParseLFSPointer

func ParseLFSPointer(data []byte) (oid string, ok bool)

ParseLFSPointer decodes data as a Git LFS pointer file and returns the referenced sha256 OID (64 lowercase hex characters). ok is false when data is not a pointer.

The decoder is deliberately MORE lenient than git-lfs: reclamation deletes registered objects that no reachable pointer references, so treating a dubious blob as a pointer merely spares an object, while rejecting a real pointer could retire live data. It therefore requires only the spec's two load-bearing properties — a `version` line and an `oid sha256:` key line, each ANYWHERE in the file — and ignores line ordering entirely, blank lines, per-line indentation, unknown keys, and a missing size line. Ordering blindness is deliberate (Codex security rounds 8-10): git-lfs itself decodes blank-led files, whitespace-indented version lines, and ext-* metadata lines preceding `version`, and each positional requirement this parser carried became a false negative that could retire live data — order-free matching closes the class instead of the instance. It never panics on arbitrary input.

func ReachableLFSOIDs

func ReachableLFSOIDs(ctx context.Context, repoPath string, budget int64) (map[string]struct{}, int64, error)

ReachableLFSOIDs walks every object reachable from the cache repo's refs (the DB-authority set Materialize just synced), decodes every non-empty reachable blob's first LFSPointerMaxSize bytes as a Git LFS pointer file (the window git-lfs's own decoder reads, whatever the blob's total size), and returns the set of referenced sha256 OIDs plus the work units the walk charged: one per listed object plus one per lfsScanBytesPerUnit of its raw size — a charge that also prices the content streaming.

The walk streams three git subprocesses — rev-list --objects, cat-file --batch-check (type/size), cat-file --batch (candidate contents) — so memory stays O(pointers found), not O(objects). Aborting past budget kills the pipeline and returns ErrLFSScanBudget; any subprocess failure (including an object reported missing) fails the walk, because reclamation must only ever act on a provably complete scan.

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL