git

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 11, 2026 License: MIT Imports: 26 Imported by: 0

README

ermite-git

A fast, read-only Git repository reader for Go: refs, objects, packfiles, trees and history, using commit-graph and changed-path Bloom filters. Standard library only.

Status: early development, no stable API. Phase A reads refs, loose and packed objects, trees, blobs and commits; phase B adds the commit-graph, changed-path Bloom filters and history walks.

import git "bnema.dev/ermite-git"

repo, err := git.Open("/srv/repos/project.git") // bare, or a ".git" directory
if err != nil {
	return err // errors.Is(err, git.ErrUnsupported) → use the git binary
}
defer repo.Close()

id, _ := repo.Resolve("main")
commitID, _ := repo.PeelToCommit(id)

var c git.Commit // reuse it: no allocation per commit after warm-up
_ = repo.ReadCommit(commitID, &c)
_ = repo.ReadTree(c.Tree, func(e git.Entry) bool {
	fmt.Printf("%s %s\n", e.Mode, e.Name) // e.Name is valid during the callback only
	return true
})

History in the order of git log, with the same path simplification as git log -- path:

for c, err := range repo.Walk(ctx, commitID, git.WalkOptions{Path: "pkg/gui", Skip: 100, Limit: 50}) {
	if err != nil {
		return err
	}
	fmt.Printf("%s\n", c.ID) // c is reused: valid until the next iteration
}
// the last commit of every entry of a directory, in one walk
ids, err := repo.LastCommits(ctx, commitID, "pkg", []string{"gui", "commands"})

A Repo is safe for concurrent use. Call Refresh after the repository changed (a push, a repack) to see new refs and packs; reads in flight keep working while packs are replaced.

Repositories the package does not read are rejected by Open with ErrUnsupported: SHA-256, reftable, other extensions.*, shallow, grafts, replace refs, alternates, partial clones, incremental or stale multi-pack-index.

Limits

  • A pack truncated in place can crash the process. Packs and indexes are memory-mapped; if another process shrinks one of them while it is mapped, a read of the missing part ends with SIGBUS, which Go cannot recover from. Git always replaces a pack by renaming a new file over it (repack, gc, fetch), and a mapping keeps the old file alive, so this takes an external actor that edits packs in place, or a failing disk. Not handled.
  • Without mmap, packs are read into memory. Linux, macOS, FreeBSD, OpenBSD, NetBSD and DragonFly map packs. On Windows and other systems the whole .pack and .idx are read into memory when a pack is opened, so memory grows with the size of the repository. Not handled.
  • Revision names. Resolve follows git rev-parse for plain names: a tag wins over a branch of the same name, and a full 40 digit id is never read as a ref. Use ResolveRef for a full ref name and ResolveCommitPrefix for an abbreviation that must designate a commit.

Test

The conformance tests compare with the git binary, 2.45 or newer (they are skipped without git).

go test -race ./...
# also compare every object of a real repository with `git cat-file`:
ERMITE_GIT_BENCH_REPO=/path/to/repo/.git go test -run TestRealRepository -v
# benchmarks (the same variable runs them on that repository):
go test -run '^$' -bench . -benchmem
# fuzz targets:
go test -run '^$' -fuzz FuzzParseCommit -fuzztime 30s

The tests build their fixtures with the git binary (2.45 or later) and compare with it.

License

MIT. See LICENSE.

Documentation

Overview

Package git reads Git repositories without the git binary: refs, loose and packed objects, trees, blobs and commit history. It uses the commit-graph and changed-path Bloom filters when present. It never writes to a repository.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNotFound reports a missing object, ref or path, or a directory that
	// is not a Git repository.
	ErrNotFound = errors.New("git: not found")
	// ErrUnsupported reports a repository feature the package does not read:
	// the error is wrapped with the reason.
	ErrUnsupported = errors.New("git: unsupported repository")
	// ErrAmbiguous reports an abbreviated object id that matches several
	// objects.
	ErrAmbiguous = errors.New("git: ambiguous object id")
	// ErrCorrupt reports malformed repository data. Reading never panics on
	// such data.
	ErrCorrupt = errors.New("git: corrupt data")
	// ErrInvalidID reports a malformed object id or abbreviation.
	ErrInvalidID = errors.New("git: invalid object id")
	// ErrWrongType reports an object that does not have the expected type.
	ErrWrongType = errors.New("git: wrong object type")
	// ErrTooLarge reports an object bigger than the configured bound
	// (see WithMaxObjectSize).
	ErrTooLarge = errors.New("git: object too large")
	// ErrClosed reports the use of a closed Repo.
	ErrClosed = errors.New("git: repository closed")
)

Errors returned by the package. They are wrapped with context, so test them with errors.Is.

View Source 
var ErrInvalidPath = errors.New("git: invalid path")

ErrInvalidPath reports a path that cannot designate anything in a repository: an absolute path or one that leaves the root with "..".

Functions

func ParseCommit 

func ParseCommit(data []byte, c *Commit) error

ParseCommit parses the content of a commit object into c. The fields of c point into data, which must stay unchanged while they are used.

It reads headers the way Git does: the first line must be the tree, the parent lines are the ones that immediately follow it, and any tree or parent line further down is an ordinary, ignored header. It returns ErrCorrupt if the first line is not a valid tree or a parent is malformed.

func ParseTag 

func ParseTag(data []byte, t *Tag) error

ParseTag parses the content of a tag object into t, whose byte fields point into data. It returns ErrCorrupt if the object or its type is missing.

func ParseTree 

func ParseTree(data []byte, fn func(Entry) bool) error

ParseTree calls fn for each entry of a tree object in order, until fn returns false. The entries are not copied, so iterating allocates nothing. It returns ErrCorrupt on a malformed tree.

Types

type Commit 

type Commit struct {
	ID        ObjectID // set by ReadCommit
	Tree      ObjectID
	Parents   []ObjectID
	Author    Signature
	Committer Signature
	// Encoding is the value of the encoding header, nil when absent. The
	// other headers (gpgsig, mergetag...) are skipped.
	Encoding []byte
	// Message is everything after the blank line that ends the headers,
	// as raw bytes: it may not be valid UTF-8.
	Message []byte
	// contains filtered or unexported fields
}

Commit is a parsed commit. A Commit is meant to be reused: ReadCommit and ParseCommit overwrite it and reuse its storage, so a hot loop allocates nothing after the first few commits. Every []byte field is valid until the next call that fills this Commit.

type Entry 

type Entry struct {
	// Name is the file name. It points into the tree buffer and is only
	// valid during the callback that receives the entry; copy it to keep it.
	Name []byte
	Mode Mode
	ID   ObjectID
}

Entry is an entry of a tree.

type Mode 

type Mode uint32

Mode is the file mode of a tree entry. 

const (
	ModeTree    Mode = 0o040000
	ModeFile    Mode = 0o100644
	ModeExec    Mode = 0o100755
	ModeSymlink Mode = 0o120000
	ModeGitlink Mode = 0o160000 // a submodule
)

Tree entry modes. 

func (m Mode) IsGitlink() bool

IsGitlink reports whether the entry is a submodule commit.

func (Mode) IsRegular 

func (m Mode) IsRegular() bool

IsRegular reports whether the entry is a regular file, executable or not. 

func (m Mode) IsSymlink() bool

IsSymlink reports whether the entry is a symbolic link.

func (Mode) IsTree 

func (m Mode) IsTree() bool

IsTree reports whether the entry is a directory.

func (Mode) String 

func (m Mode) String() string

String returns the mode as Git prints it: six octal digits (five for a directory is printed as "040000").

func (Mode) Type 

func (m Mode) Type() Type

Type returns the type of the object the entry designates: a tree for a directory, a commit for a submodule, a blob otherwise.

type ObjectID 

type ObjectID [20]byte

ObjectID is a SHA-1 object id.

func ParseID 

func ParseID(s string) (ObjectID, error)

ParseID parses 40 hexadecimal digits.

func (ObjectID) AppendHex 

func (id ObjectID) AppendHex(dst []byte) []byte

AppendHex appends the 40 lowercase hexadecimal digits of the id to dst. It does not allocate when dst has room.

func (ObjectID) IsZero 

func (id ObjectID) IsZero() bool

IsZero reports whether the id is all zeros (the null id).

func (ObjectID) String 

func (id ObjectID) String() string

String returns the 40 lowercase hexadecimal digits of the id.

type Option 

type Option func(*options)

Option configures Open.

func WithDeltaCacheSize 

func WithDeltaCacheSize(n int64) Option

WithDeltaCacheSize sets the budget in bytes of the cache of delta bases (LRU). The default is 32 MiB. Zero or less disables the cache.

func WithMaxObjectSize 

func WithMaxObjectSize(n int64) Option

WithMaxObjectSize bounds the size of an object that is held in memory: commits, trees, tags and blobs that are stored as deltas. Bigger ones fail with ErrTooLarge. Undeltified blobs stream and are not bounded. The default is 128 MiB. The bound is capped at the largest int of the platform (2 GiB - 1 on 32-bit systems), since such an object must fit in a slice.

type Ref 

type Ref struct {
	// Name is the full name, such as "refs/heads/main".
	Name string
	// ID is the object the ref designates, after following symbolic refs.
	// It may be an annotated tag; see Repo.PeelRef.
	ID ObjectID
	// Symref is the name of the ref this one points to when it is a symbolic
	// ref, empty otherwise.
	Symref string
	// Peeled is the object that ID designates once annotated tags are
	// followed. It is only set when PeeledKnown, which is the case for refs
	// read from packed-refs that carry peeled values.
	Peeled      ObjectID
	PeeledKnown bool
}

Ref is a reference.

type Repo 

type Repo struct {
	// contains filtered or unexported fields
}

Repo is an open Git repository. It is read-only and safe for concurrent use by multiple goroutines.

Loose refs and loose objects are always read live. The list of packs and the content of packed-refs are loaded snapshots: a missing object triggers one rescan of the pack directory, packed-refs is stat-checked on each ref read, and Refresh reloads everything explicitly.

func Open 

func Open(dir string, opts ...Option) (*Repo, error)

Open opens the repository whose Git directory is dir: a bare repository or a ".git" directory. If dir contains a ".git" directory, that one is used.

Open returns an error wrapping ErrNotFound when dir is not a repository and one wrapping ErrUnsupported (with the reason) when the repository uses a feature the package does not read: an object format other than sha1, a ref storage other than files, other extensions, a shallow file, info/grafts, replace refs, alternates, a linked worktree, a partial clone, or a multi-pack-index whose packs lack their .idx file. A multi-pack-index is otherwise ignored: the packs are read through their own .idx files.

func (*Repo) Close 

func (r *Repo) Close() error

Close releases the repository. Readers still streaming a blob keep their packs mapped until they are closed. Close is idempotent.

func (*Repo) Dir 

func (r *Repo) Dir() string

Dir returns the Git directory that was opened.

func (*Repo) HasObject 

func (r *Repo) HasObject(id ObjectID) (bool, error)

HasObject reports whether the object exists. The error is not nil when the answer is unknown: the repository is closed, or the pack list could not be reloaded while the object was missing from the current one (during a gc, for example).

func (*Repo) Head 

func (r *Repo) Head() (target string, id ObjectID, err error)

Head returns the full name of the ref HEAD points to ("refs/heads/main") and the object it designates. An unborn branch (the target does not exist yet) is not an error: id is then the zero ObjectID. For a detached HEAD target is empty and id is the commit.

func (*Repo) LastCommit 

func (r *Repo) LastCommit(ctx context.Context, start ObjectID, path string) (ObjectID, bool, error)

LastCommit returns the first commit of Walk with the given path and Limit 1, that is the answer of "git log -n 1 <start> -- <path>". The boolean is false when no commit has the path.

func (*Repo) LastCommits 

func (r *Repo) LastCommits(ctx context.Context, start ObjectID, dir string, names []string) ([]ObjectID, error)

LastCommits finds, in one walk, the last commit that touched each entry of a directory: for every name, the same commit as 

git log -n 1 <start> -- <dir>/<name>

with the same history simplification. The result has one id per name, in the order of names; the zero ObjectID means that no commit reachable from start has the entry (it never existed). Read the commits you need with ReadCommit; many names share the same commit.

dir is relative to the root of the repository ("" or "." for the root itself). Each name is one path component, taken literally.

Algorithm. The history of one path is a walk where a merge that is TREESAME to a parent follows only the first such parent, so two paths can follow different parents at the same merge and then visit commits in different orders. A single shared walk would therefore not give the per-path answers. Instead the names travel as groups. All names start in one group, which owns the queue and the set of seen commits of their (so far identical) walks. For each popped commit the group computes, per name, the first parent whose entry is identical to the commit's; a name with no such parent is changed by the commit: it is resolved and leaves the group. The other names follow exactly the parent they matched (the only parent for a commit that has one). When the names of a group match different parents of a merge, the group splits: the queue and the seen set are cloned, each part follows its parent. Every group therefore holds the exact state of the individual walk of each of its names, and the answers are those of N separate walks, but a commit is loaded and compared once per group instead of once per name. The walk ends when every name has an answer. A directory that did not change (same tree id, or a Bloom filter that says so) settles all the names of a group at once.

func (*Repo) LookupPath 

func (r *Repo) LookupPath(root ObjectID, path string) (Entry, error)

LookupPath finds the entry at path (slash separated, no leading slash) below the tree root. It returns ErrNotFound when a component does not exist or is not a directory. An empty path designates root itself, as a directory.

func (*Repo) LookupRef 

func (r *Repo) LookupRef(name string) (Ref, error)

LookupRef is like ResolveRef but returns the whole Ref.

func (*Repo) LookupRefID 

func (r *Repo) LookupRefID(name string) (id ObjectID, ok bool, err error)

LookupRefID is like ResolveRef but a ref that does not exist is not an error: ok is false, and nothing is allocated, which makes it the call for trying several candidate names. err is for what went wrong while looking (a closed Repo, an unreadable or corrupt ref).

func (*Repo) ObjectHeader 

func (r *Repo) ObjectHeader(id ObjectID) (Type, int64, error)

ObjectHeader returns the type and the size of an object without reading its content. For a deltified object the size is the result size announced by the delta header. It returns ErrNotFound if the object does not exist.

func (*Repo) OpenBlob 

func (r *Repo) OpenBlob(id ObjectID) (int64, io.ReadCloser, error)

OpenBlob opens the blob id for reading and returns its size. Undeltified blobs stream from the memory-mapped pack or the loose file, inflated as they are read, so a large file is never held in memory. A deltified blob is rebuilt in a pooled buffer first and is bounded by WithMaxObjectSize (ErrTooLarge). The caller must close the reader, which also releases the pack it reads from. Reads detect truncated or oversized data and return ErrCorrupt. The checksum of the stream is verified when the last byte is read: that Read returns (0, ErrCorrupt) instead of the final chunk, so io.ReadFull and io.CopyN of exactly size bytes report a corrupt trailer too. Close returns ErrCorrupt if all size bytes were consumed (or the blob is empty) but the end of the stream was not checked. A reader closed before size bytes were read cannot verify the checksum.

func (*Repo) Peel 

func (r *Repo) Peel(id ObjectID) (ObjectID, Type, error)

Peel follows annotated tags from id until it reaches an object that is not a tag, and returns that object and its type. Chains of tags longer than 64 are corrupt.

func (*Repo) PeelRef 

func (r *Repo) PeelRef(ref Ref) (ObjectID, error)

PeelRef returns the commit a ref designates, following annotated tags. It uses the peeled value of packed-refs when the ref carries one. It returns ErrWrongType if the ref designates something else than a commit.

func (*Repo) PeelToCommit 

func (r *Repo) PeelToCommit(id ObjectID) (ObjectID, error)

PeelToCommit follows annotated tags from id and returns the commit they designate. It returns ErrWrongType if the result is not a commit (a tag of a tree, for example).

func (*Repo) ReadCommit 

func (r *Repo) ReadCommit(id ObjectID, c *Commit) error

ReadCommit reads and parses the commit id into c.

func (*Repo) ReadObject 

func (r *Repo) ReadObject(id ObjectID, dst []byte) (Type, []byte, error)

ReadObject reads the whole content of an object. The content is appended to dst[:0], which is grown if needed; pass the previous result to reuse its storage. Objects over the WithMaxObjectSize bound fail with ErrTooLarge.

func (*Repo) ReadTag 

func (r *Repo) ReadTag(id ObjectID, t *Tag) error

ReadTag reads and parses the annotated tag id into t. The byte fields of t are copies, owned by the caller.

func (*Repo) ReadTree 

func (r *Repo) ReadTree(id ObjectID, fn func(Entry) bool) error

ReadTree reads the tree id and calls fn for each entry, in tree order, until fn returns false. Entry.Name is only valid during the call. The tree is read into a pooled buffer, so listing allocates nothing per entry.

func (*Repo) Refresh 

func (r *Repo) Refresh() error

Refresh reloads the list of packs and packed-refs when they changed, and checks again that the repository is still supported (it returns an error wrapping ErrUnsupported when, for example, a replace ref was pushed). Packs that disappeared stay mapped until the readers using them are done, so a concurrent "git repack -ad" never breaks a read in flight.

func (*Repo) Refs 

func (r *Repo) Refs(prefix string) iter.Seq2[Ref, error]

Refs iterates over the refs whose full name starts with prefix, such as "refs/heads/" or "refs/tags/" ("" for all of them), sorted by full name. Loose refs override packed ones; the peeled lines of packed-refs are used. Symbolic refs are resolved; a ref that cannot be read is skipped. HEAD is not part of it, see Head. An error ends the iteration.

func (*Repo) Resolve 

func (r *Repo) Resolve(rev string) (ObjectID, error)

Resolve turns a revision name into an object id the way "git rev-parse" does for plain names. The result is not peeled; use PeelToCommit. Revision operators (~, ^, @{}, :path) are not supported.

The name is tried in git's order, and the first match wins:

  1. a full 40 digit id. It is taken as an id even when a ref has that name (git warns that the ref is ambiguous and ignores it), and it must exist in the repository, which "git rev-parse" does not check;
  2. a ref, as "<n>", "refs/<n>", "refs/tags/<n>", "refs/heads/<n>", "refs/remotes/<n>" and "refs/remotes/<n>/HEAD", in that order. A tag therefore wins over a branch of the same name, whereas "git checkout" prefers the branch;
  3. an abbreviated id of 4 to 39 digits, which loses against any ref of that name. It matches any object, whatever its type, and fails with ErrAmbiguous if several do; see ResolveCommitPrefix.

A caller that wants a branch or a tag, or a commit, should not use Resolve: ResolveRef takes the full ref name and ResolveCommitPrefix only considers commits. Resolve returns an error wrapping ErrNotFound if nothing matches.

func (*Repo) ResolveCommitPrefix 

func (r *Repo) ResolveCommitPrefix(hex string) (ObjectID, error)

ResolveCommitPrefix is like ResolvePrefix but only considers the objects that designate a commit: commits and tags that peel to a commit. It is what "git rev-parse <hex>^{commit}" does.

func (*Repo) ResolvePrefix 

func (r *Repo) ResolvePrefix(hex string) (ObjectID, error)

ResolvePrefix returns the object whose id starts with the hexadecimal digits in hex (4 to 40, either case). It returns ErrNotFound when none matches and ErrAmbiguous when several do, whatever their type; see ResolveCommitPrefix. A full id must exist.

func (*Repo) ResolveRef 

func (r *Repo) ResolveRef(name string) (ObjectID, error)

ResolveRef returns the object the ref with this full name (such as "refs/heads/main", "refs/tags/v1" or "HEAD") designates, following symbolic refs. The result may be an annotated tag; see PeelToCommit. It returns ErrNotFound if the ref does not exist.

func (*Repo) Walk 

func (r *Repo) Walk(ctx context.Context, start ObjectID, opts WalkOptions) iter.Seq2[*Commit, error]

Walk yields the commits reachable from start in the order of "git log <start>", restricted as opts says. The yielded *Commit is reused: it and the bytes it points to are valid until the next iteration. On failure the sequence yields (nil, err) once and ends. start must be a commit id; use PeelToCommit first for a tag.

A Walk uses the commit-graph (parents, dates, root trees, changed-path Bloom filters) when the repository has one, and reads the objects for the commits it does not cover. The graph in use when the walk starts is kept for its whole duration, even if Refresh replaces it.

type Signature 

type Signature struct {
	Name   []byte
	Email  []byte
	Time   int64 // seconds since the Unix epoch
	Offset int   // time zone, in seconds east of UTC
}

Signature is the author or committer line of a commit or the tagger of a tag. Name and Email point into the buffer of the object they come from.

func (Signature) Date 

func (s Signature) Date() time.Time

Date returns the time of the signature, in its own time zone. It allocates.

type Tag 

type Tag struct {
	Object  ObjectID
	Type    Type // the type of Object
	Name    []byte
	Tagger  Signature
	Message []byte
}

Tag is a parsed annotated tag. Name, Tagger and Message point into the data that was parsed.

type Type 

type Type uint8

Type is the type of a Git object. The values are those of pack entries. 

const (
	TypeCommit Type = 1
	TypeTree   Type = 2
	TypeBlob   Type = 3
	TypeTag    Type = 4
)

Object types.

func (Type) String 

func (t Type) String() string

String returns the name Git uses for the type ("commit", "tree", "blob", "tag").

type WalkOptions 

type WalkOptions struct {
	// Skip is the number of commits to leave out, counted after path
	// simplification, like "git log --skip".
	Skip int
	// Limit is the number of commits to yield, like "git log -n". Zero or
	// less means no limit.
	Limit int
	// Path restricts the history like "git log -- <path>": a file, a
	// directory or a submodule, written relative to the root of the
	// repository, taken literally (no globs). The empty path means no
	// restriction. "." and "dir/" are accepted; a path outside the
	// repository gives ErrInvalidPath.
	Path string
	// IDOnly makes Walk fill only ID, Tree, Parents and Committer.Time when
	// the commit-graph has the commit, which then costs no object read.
	// Without it every field is filled.
	IDOnly bool
}

WalkOptions selects what Walk yields.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.