gitread

package
v0.18.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 13, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Overview

Package gitread is the read-side git plumbing layer for the /api/v1 git:read endpoints (docs/reference/http-api.md): ref resolution, ls-tree, log, diff, blame, grep, and tar.gz archive streaming over a hydrated bare cache repo.

FROZEN CONTRACT: the exported types and function signatures are the shared surface the phase-3 endpoint slices build on. gitread stays HTTP-agnostic (no status codes, no wire envelopes): handlers own parameter validation, cursoring, and response shaping. All plumbing is `os/exec git` over a repoPath produced by Materialize — consistent with the rest of the repo, no go-git.

Error mapping: ErrNotFound (wrapped) marks "the requested ref/object/path does not exist" outcomes that handlers turn into 404; every other error is internal (500).

Index

Constants

View Source
const (
	// HeavyHeapBytes caps the heap (RLIMIT_DATA, Linux) of a child whose
	// work grows with history length, tree size, blob size, or output size.
	// Measured peak VmData on a 300,000-commit linear history (git 2.55):
	// `log -1 -- <file from the root commit>` 184 MB, `blame` of that file
	// 126 MB, `merge-base <tip> <root>` 130 MB, `log --skip=299990` 75 MB.
	// So a 300k history fits; histories past roughly 400k commits can fail
	// the per-file walks. Read-only pack mappings are shared page cache and
	// are not counted.
	HeavyHeapBytes = gitlimit.HeavyHeapBytes
	// LightHeapBytes caps a child whose work is bounded by one object:
	// ref resolution and single-commit reads. A commit object larger than
	// this cannot be read through the API.
	LightHeapBytes int64 = 32 << 20
	// DefaultOutputBytes is the buffered-stdout allowance of one request (a
	// Budget), shared by all of its children. Streaming children (CatBlobTo
	// into a caller's writer) are not counted; they never buffer in memory.
	DefaultOutputBytes int64 = 64 << 20
	// MaxBlobBytes is the largest blob the API streams (file, archive).
	// git reads a blob below core.bigFileThreshold (512 MiB) fully into its
	// heap, and a deltified blob also needs its base, so larger blobs would
	// hit HeavyHeapBytes part-way; callers check sizes up front instead.
	MaxBlobBytes int64 = 100 << 20
)

Resource bounds for read-side git children (#313). Every git child that gitread starts runs through run, which applies them before git does any work. docs/reference/http-api.md ("Resource limits on git reads") is the client-facing description and ADR-049 the design; keep them in step.

View Source
const (
	GrepLineMatch   = "match"
	GrepLineContext = "context"
)

Grep line types (docs/reference/http-api.md POST /repos/grep lines[].type).

View Source
const (
	EntryBlob      = "blob"
	EntryTree      = "tree"
	EntrySymlink   = "symlink"
	EntrySubmodule = "submodule"
)

TreeEntry types (docs/reference/http-api.md GET /repos/files entry `type`): symlinks and submodules are surfaced distinctly from plain blobs/trees.

View Source
const EmptyTreeSHA = "4b825dc642cb6eb9a060e54bf8d69288fbee4904"

EmptyTreeSHA is git's well-known empty tree object id (SHA-1 repos): the diff base for root commits (`git hash-object -t tree /dev/null`).

Variables

View Source
var (
	// ErrBusy: no admission slot freed up in time; git was not started.
	ErrBusy = gitlimit.ErrBusy
	// ErrTimeBudget: the request's git time budget ran out; the running
	// child (if any) was killed and reaped.
	ErrTimeBudget = errors.New("git reads exceeded the request time budget")
	// ErrOutputLimit: git's buffered output exceeded the request's
	// allowance; git was killed and reaped as soon as that happened.
	ErrOutputLimit = errors.New("git output exceeded the request output budget")
	// ErrBlobTooLarge: a blob is larger than MaxBlobBytes; git was not
	// started for it.
	ErrBlobTooLarge = errors.New("blob exceeds the API read size limit")
	// ErrMemoryLimit: git needed more heap than its class's cap and was
	// stopped by its own allocation failure (Linux only).
	ErrMemoryLimit = gitlimit.ErrMemoryLimit
	// ErrStop may be returned by a streaming Cmd.Stdout's Write to end git
	// early (a history walk that has seen enough): Run then kills and reaps
	// git and returns no error.
	ErrStop = errors.New("stop streaming git output")
)
View Source
var ErrNotFound = errors.New("not found")

ErrNotFound marks a missing ref, object, or path (handlers map it to 404 per docs/reference/http-api.md, e.g. unknown branch and unborn-branch reads).

Functions

func Archive

func Archive(ctx context.Context, repoPath, commit string, opts ArchiveOptions, w io.Writer) error

Archive is PrepareArchive + Stream in one call, for callers that do not need the resolve-before-stream split.

func CatBlobTo

func CatBlobTo(ctx context.Context, repoPath, sha string, w io.Writer) error

CatBlobTo streams the raw bytes of blob sha into w without buffering the whole blob in memory (file/archive streaming path). git itself runs as a Heavy child: a blob it must inflate or rebuild from deltas in its heap can hit HeavyHeapBytes, so callers check CheckBlobSize first.

func CheckBlobSize

func CheckBlobSize(ctx context.Context, path string, size int64) error

CheckBlobSize fails with ErrBlobTooLarge when a blob of size bytes is over the blob limit (MaxBlobBytes, or the Budget's in ctx). Streaming callers check before they start git or send response headers.

func Diff

func Diff(ctx context.Context, repoPath, base, target string, opts DiffOptions) ([]FileDiff, DiffStats, error)

Diff computes the tree diff base..target (both resolved full SHAs; pass EmptyTreeSHA as base for root commits). Rename/copy detection is always on (-M) so renames surface as one R entry with OldPath, matching the docs/reference/http-api.md FileDiff shape.

func HeapLimitSupported

func HeapLimitSupported() bool

HeapLimitSupported reports whether Run can cap a git child's heap on this platform (Linux: RLIMIT_DATA). Outside Linux, Run runs git without a heap cap, under the admission, time, and output limits only (ADR-049); it never refuses a read because the cap is unavailable.

func IsBudgetError

func IsBudgetError(err error) bool

IsBudgetError reports whether err is a resource-limit outcome (ErrBusy, ErrTimeBudget, ErrOutputLimit, ErrBlobTooLarge, ErrMemoryLimit). Callers that turn git failures into "not found" or "no match" must check it first so a limit is never reported as missing data.

func MatchGlob

func MatchGlob(pattern, p string) bool

MatchGlob matches a slash-separated path against one glob pattern:

  • a pattern without "/" matches against the path's base name ("*.md" matches docs/x.md), gitignore-style;
  • a pattern with "/" matches against the full path, segment by segment with path.Match semantics, where a "**" segment matches any number of segments (including zero).

Malformed patterns match nothing.

func Materialize

func Materialize(ctx context.Context, st ref.RefStore, bl blob.BlobStore, cacheDir, repo, defaultBranch string) (string, error)

Materialize rebuilds the cache repo for repo from the ref/blob stores (githttp.Hydrate: ensure repo, hydrate packs, sync refs from the DB authority, point HEAD at defaultBranch) and returns its bare-repo path. Unlike the receive path, a ref-sync failure is fatal here: read plumbing over stale refs would serve wrong answers.

func Note

func Note(ctx context.Context, repoPath, notesRef, sha string) (string, error)

Note reads the git note attached to object sha from notesRef in the repo at repoPath. Callers validate inputs: sha must be a full object id (ref.ValidSHA) and notesRef a refs/notes/ ref, so neither can smuggle options or rev expressions into git. A missing note wraps ErrNotFound — `git notes show` exits 1 for an unborn notes ref, an unknown object, and an object without a note alike (anything else is internal).

func ReadBlob

func ReadBlob(ctx context.Context, repoPath, sha string) ([]byte, error)

ReadBlob returns the full contents of blob sha. Use only for small blobs (symlink targets); file streaming goes through CatBlobTo. The bytes count against the output allowance.

func ResolveRef

func ResolveRef(ctx context.Context, repoPath, spec string, ephemeral bool) (string, error)

ResolveRef resolves a request ref to a full commit SHA in the repo at repoPath. Resolution order (docs/reference/http-api.md "branch name, short or full SHA"):

  1. branch: refs/heads/<spec> — or the ephemeral namespace name (auth.EphemeralRefName) when ephemeral is true
  2. tag: refs/tags/<spec> (annotated tags peel to their commit)
  3. full ref: <spec> verbatim when it already starts with "refs/"
  4. hex: short or full SHA (no rev expressions; a full SHA may be uppercase, an abbreviation must be lowercase)

Everything unresolvable wraps ErrNotFound — including the empty spec and the unborn-branch case, where the DB knows no such ref yet (invariant 3: handlers map it to 404).

func Run

func Run(ctx context.Context, c Cmd) (string, error)

Run starts one read-side git child under the limits above and waits for it (git is always reaped before Run returns). Limit outcomes are returned as ErrBusy, ErrTimeBudget, ErrOutputLimit, or ErrMemoryLimit, and never wrap *exec.ExitError, so exit-status checks cannot mistake them for git's own "no match" or "not found" statuses. Other failures wrap git's *exec.ExitError.

func TopoRanks

func TopoRanks(ctx context.Context, repoPath, rev string, want []string) (map[string]int, error)

TopoRanks returns the history-order rank of every commit in want relative to rev (a resolved full SHA): the commit's 0-based position in `git rev-list --topo-order rev`. rev itself ranks 0 and every commit ranks higher than all of its descendants.

Ties (commits that ancestry does not order) follow git's topo-order walk, which ignores author and committer dates: a commit becomes eligible once all of its children reachable from rev are ranked, and the most recently eligible commit is ranked next. When ranking a merge makes several of its parents eligible at once, they become eligible in parent order, so the line of history behind the last of them is ranked first. The same history therefore always yields the same ranks.

Every entry of want must be reachable from rev; an unreachable commit is an error, not a missing entry. Duplicates in want are allowed. The walk stops once every wanted commit is ranked; without a commit-graph, git still reads the whole history before it prints the first commit. The walk is a Heavy child of the request's read budget (see Run): it fails with ErrBusy, ErrTimeBudget or ErrMemoryLimit rather than exceed it.

func ValidArchivePrefix

func ValidArchivePrefix(prefix string) bool

ValidArchivePrefix reports whether prefix is safe to prepend to tar entry names: no NUL bytes, no backslashes (Windows extractors treat them as separators), not absolute, and no "." or ".." segments — so extracted entries cannot escape the destination root (tar slip).

func WithBudget

func WithBudget(ctx context.Context, b *Budget) context.Context

WithBudget attaches b to ctx for every gitread call made with it.

Types

type ArchiveOptions

type ArchiveOptions struct {
	// Prefix is prepended verbatim to every entry name (callers include the
	// trailing "/", e.g. "repo-main/"). It must satisfy ValidArchivePrefix;
	// callers accepting untrusted input must validate it first.
	Prefix string
	// IncludeGlobs whitelists paths when non-empty; ExcludeGlobs always
	// wins. See MatchGlob for the dialect.
	IncludeGlobs []string
	ExcludeGlobs []string
	// MaxBlobSize (>0) skips file blobs larger than this many bytes —
	// the reason this is a custom walk instead of `git archive`.
	MaxBlobSize int64
}

ArchiveOptions tune one Archive stream. The zero value archives the whole tree with no prefix.

type BlameLine

type BlameLine struct {
	LineNumber         int    // 1-based line number in the file at the blamed commit
	CommitSHA          string // commit that introduced the line
	OriginalLineNumber int    // line number in CommitSHA's version of the file
	OriginalPath       string // path in CommitSHA (differs across renames)
	PreviousCommitSHA  string // CommitSHA's parent per blame ("" for root/boundary)
	AuthorName         string
	AuthorEmail        string
	AuthorTime         time.Time
	CommitterName      string
	CommitterEmail     string
	CommitterTime      time.Time
	Summary            string // commit subject line
}

BlameLine is one attributed line (docs/reference/http-api.md GET /repos/blame line objects).

func Blame

func Blame(ctx context.Context, repoPath, commit, path string, ranges []string, detectMoves bool) ([]BlameLine, error)

Blame attributes every line of path at commit (a resolved full SHA). ranges are raw `git blame -L` specs (docs/reference/http-api.md: repeatable `range` params like "1,10" — the handler validates their shape); detectMoves adds -M. A path absent at commit wraps ErrNotFound.

type BlobInfo

type BlobInfo struct {
	SHA        string // blob object id
	Size       int64  // bytes
	Mode       string // git octal mode string
	Symlink    bool   // mode 120000: content is the link target
	LastCommit Commit // most recent commit touching the path, reachable from commit
}

BlobInfo locates one blob at commit:path plus the last commit that touched the path — the /repos/file streaming metadata (X-Blob-Sha, X-Last-Commit-Sha, ETag, Last-Modified sources).

func StatBlob

func StatBlob(ctx context.Context, repoPath, commit, path string) (BlobInfo, error)

StatBlob resolves commit:path to its blob. A missing path (or a path that names a tree/submodule) wraps ErrNotFound.

type Budget

type Budget struct {
	// contains filtered or unexported fields
}

Budget is one request's share of the read-side git limits. A handler middleware creates it, attaches it with WithBudget, and calls Close when the request has finished, including delivering its response. The request's first git child takes one admission slot, and the request keeps it until Close: the slot bounds everything the request retains (git children, buffered output, parsed results, spools), not just its git processes. Without a Budget in the context (write endpoints), Run applies none of these limits.

func BudgetFrom

func BudgetFrom(ctx context.Context) *Budget

BudgetFrom returns the Budget attached to ctx, or nil.

func NewBudget

func NewBudget(slots *Slots, key string, timeout time.Duration, outputBytes int64) *Budget

NewBudget returns a request budget drawing admission from slots under key (the request's repository), with a git time budget of timeout (counted from the first child, so repository hydration before it is not included) and outputBytes of buffered output.

func (*Budget) Close

func (b *Budget) Close()

Close ends the budget and releases its slot. Every child has been reaped by then (Run waits for git before returning).

func (*Budget) Limits

func (b *Budget) Limits() Limits

Limits returns the per-child caps b applies.

func (*Budget) WithLimits

func (b *Budget) WithLimits(l Limits) *Budget

WithLimits replaces the per-child caps (tests lower them). It returns b.

type Class

type Class int

Class sorts read-side git children by what their cost scales with.

const (
	// Light: bounded by one object (ref resolution, single-commit reads).
	// Runs under LightHeapBytes.
	Light Class = iota + 1
	// Heavy: grows with history, tree, blob, or output size. Runs under
	// HeavyHeapBytes.
	Heavy
)

type Cmd

type Cmd struct {
	Class Class
	Dir   string    // repository (or scratch) directory; required
	Env   []string  // nil inherits the server's environment (see childEnv)
	Stdin io.Reader // nil: no stdin
	// Stdout, when set, receives git's output as it is produced and is not
	// counted against the output allowance (streaming callers, such as a
	// `rev-list` walk read incrementally). Its Write may return ErrStop to
	// end git early. When nil, output is buffered, counted, and returned.
	Stdout io.Writer
	Args   []string
}

Cmd is one read-side git child for Run.

type Commit

type Commit struct {
	SHA            string
	TreeSHA        string
	ParentSHAs     []string // empty (never nil) for root commits — invariant 2
	AuthorName     string
	AuthorEmail    string
	AuthorDate     time.Time
	CommitterName  string
	CommitterEmail string
	CommitterDate  time.Time
	Message        string // full raw message (%B): subject + body, usually newline-terminated
}

Commit is one parsed commit — the docs/reference/http-api.md CommitInfo source data ({sha, parent_shas[], message, author_*, committer_*, date}).

func Log

func Log(ctx context.Context, repoPath, rev string, opts LogOptions) ([]Commit, error)

Log lists commits reachable from rev (a resolved full SHA), newest first in commit-graph order (git log default — invariant 9).

type CommitDetail

type CommitDetail struct {
	Commit
	Signature string // armored PGP/SSH signature; "" when unsigned
	Payload   string // exact signed bytes (raw commit minus the gpgsig header); "" when unsigned
}

CommitDetail is GET /repos/commit's payload: the commit plus, for signed commits only, the detached signature and the exact payload bytes it signs.

func GetCommit

func GetCommit(ctx context.Context, repoPath, sha string) (CommitDetail, error)

GetCommit loads one commit (a resolved full SHA) with signature extraction.

type DiffOptions

type DiffOptions struct {
	Paths []string // optional pathspec filter (`-- <path>...`, repeatable `path` params)
}

DiffOptions tune one Diff invocation. The zero value diffs the full trees.

type DiffStats

type DiffStats struct {
	Files     int
	Additions int
	Deletions int
	Changes   int // Additions + Deletions
}

DiffStats aggregates a diff (docs/reference/http-api.md stats envelope). Binary files count toward Files but contribute 0 additions/deletions.

type FileDiff

type FileDiff struct {
	Path      string // post-image path (the old path for deletions)
	OldPath   string // pre-image path for renames/copies; "" otherwise
	Status    string // "A", "M", "D", "R", "C", or "T"
	Additions int    // added lines; -1 for binary files (numstat "-")
	Deletions int    // deleted lines; -1 for binary files
	Binary    bool
	// Patch is this file's slice of `git diff --patch` output, starting at
	// its `diff --git` header line. Endpoint slices own the final `raw`
	// formatting (hunks-only vs git-apply-compatible, docs/explanation/domain-model.md:93-95).
	Patch string
}

FileDiff is one changed file between two trees. Status is git's raw status letter (A/M/D/R/C/T — docs/reference/http-api.md "state (raw letter)"); rename/copy similarity scores are stripped.

type GrepFile

type GrepFile struct {
	Path  string
	Lines []GrepLine
}

GrepFile groups all matched/context lines of one file, ascending by line number (docs/reference/http-api.md matches[] entries).

func Grep

func Grep(ctx context.Context, repoPath, commit string, opts GrepOptions) ([]GrepFile, error)

Grep searches the tree at commit (a resolved full SHA). Binary blobs are skipped (-I). No matches is a successful empty result, not an error. Result limiting/pagination (docs/reference/http-api.md limits, incomplete_results) is the endpoint slice's job on top of this full result.

Match/context typing: `--null` (needed for unambiguous parsing of paths containing ':' or '-') erases git's ':' vs '-' delimiter distinction, so with context enabled Grep runs a second match-only pass and classifies every line by (path, line) membership in that match set.

type GrepLine

type GrepLine struct {
	LineNumber int
	Text       string
	Type       string // GrepLineMatch | GrepLineContext
}

GrepLine is one output line: a pattern match or surrounding context.

type GrepOptions

type GrepOptions struct {
	Pattern         string
	CaseInsensitive bool     // -i
	Pathspecs       []string // git pathspecs incl. :(glob)/:(exclude) magic; empty = whole tree
	BeforeContext   int      // -B
	AfterContext    int      // -A
}

GrepOptions tune one Grep invocation. Pattern is required (POSIX basic regex, git grep's default dialect).

type Limits

type Limits struct {
	HeavyHeap int64 // RLIMIT_DATA of Heavy children
	LightHeap int64 // RLIMIT_DATA of Light children
	MaxBlob   int64 // CheckBlobSize limit
}

Limits are the per-child caps a Budget applies.

func DefaultLimits

func DefaultLimits() Limits

DefaultLimits returns HeavyHeapBytes, LightHeapBytes and MaxBlobBytes.

type LogOptions

type LogOptions struct {
	Path     string // non-empty limits to commits touching Path (`-- <path>`, no --follow)
	Skip     int    // --skip offset (tip-anchored cursor pagination)
	MaxCount int    // >0 caps the returned commits (-n); 0 = unlimited
}

LogOptions tune one Log invocation. The zero value lists the full history.

type PreparedArchive

type PreparedArchive struct {
	// contains filtered or unexported fields
}

PreparedArchive is a resolved, filtered archive plan. Everything that can fail without touching the output stream (commit lookup, tree walk, entry filtering) happens in PrepareArchive, so HTTP callers can commit response headers only after resolution succeeded and mid-stream failures are limited to blob streaming.

func PrepareArchive

func PrepareArchive(ctx context.Context, repoPath, commit string, opts ArchiveOptions) (*PreparedArchive, error)

PrepareArchive resolves the tree at commit (a resolved full SHA) and applies the include/exclude globs and MaxBlobSize filters up front. A kept blob over the blob limit (CheckBlobSize) fails it with ErrBlobTooLarge, before any archive byte is written.

func (*PreparedArchive) Stream

func (p *PreparedArchive) Stream(ctx context.Context, w io.Writer) error

Stream writes the prepared entries as a tar.gz into w: regular blobs (mode 0644, or 0755 for git mode 100755) and symlinks (target = blob content); entry mtimes are the commit's committer time. Errors here can only truncate the stream — HTTP callers have already sent headers.

type Slots

type Slots = gitlimit.Slots

Slots is a process-wide pool of request admission slots. A request holds at most one slot, from its first git child until it ends (see Budget). One key (a repository) may hold at most perKey of them, so slow requests against one repository cannot take the whole pool from the others. It is gitlimit's pool type, shared with cache maintenance.

func NewSlots

func NewSlots(n, perKey int, wait time.Duration) *Slots

NewSlots returns a pool of n slots, at most perKey of them per key. A request waits at most wait for a free slot before failing with ErrBusy.

type Tag

type Tag struct {
	Name string // short tag name (refs/tags/ stripped)
	SHA  string // peeled commit sha
}

Tag is one refs/tags entry. SHA is the PEELED commit sha for annotated tags (invariant 10, docs/explanation/domain-model.md:78-80) and the target itself for lightweight tags.

func PeeledTags

func PeeledTags(ctx context.Context, repoPath string) ([]Tag, error)

PeeledTags lists all tags of the repo at repoPath in lexical refname order (for-each-ref's default — the docs/reference/http-api.md GET /repos/tags order). Invariant 10 wants the peeled COMMIT sha, so tags that reference (or peel to) a non-commit object — trees or blobs, pushable with plain git — are skipped rather than surfaced with a non-commit sha.

type TreeEntry

type TreeEntry struct {
	Path string // full slash-separated path relative to the repo root
	Mode string // git octal mode string, e.g. "100644", "100755", "120000"
	Type string // EntryBlob | EntryTree | EntrySymlink | EntrySubmodule
	SHA  string // object id (blob sha for files — the X-Blob-Sha source)
	Size int64  // blob size in bytes; -1 for trees and submodules
}

TreeEntry is one ls-tree row.

func LsTree

func LsTree(ctx context.Context, repoPath, commit, path string, recursive bool) ([]TreeEntry, error)

LsTree lists the tree at commit (a resolved full SHA). path filters with git's `ls-tree <commit> -- <path>` semantics: "" lists the root, "dir/" lists a directory's contents, "dir" lists the entry itself. recursive descends into subtrees (blobs only, git -r semantics). A path that matches nothing yields an empty slice, NOT an error — the handler decides whether that is a 404.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL