git

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 21, 2026 License: Apache-2.0 Imports: 38 Imported by: 0

Documentation

Overview

Package git implements a protocol-aware Git caching proxy strategy.

Index

Constants

View Source 
const EnsureRefsPath = "/ensure-refs"

EnsureRefsPath is the URL suffix for the ref-freshness endpoint. Clients POST to /git/{host}/{repo}/ensure-refs to force cachew to ensure the local mirror contains the listed refs before they fetch.

Variables

View Source 
var ErrSpoolFailed = errors.New("spool failed before response started")

ErrSpoolFailed is returned by ServeTo when the spool failed before any headers were written to the client, allowing the caller to fall back to upstream.

Functions

func ExtractRepoPath 

func ExtractRepoPath(pathValue string) string

func Register 

func Register(r *strategy.Registry, scheduler jobscheduler.Provider, cloneManagerProvider gitclone.ManagerProvider, tokenManagerProvider githubapp.TokenManagerProvider)

func RequestIsClone 

func RequestIsClone(pathValue string, r *http.Request) (bool, error)

RequestIsClone reports whether r is an initial clone of a Git repo.

Detection: POST /git-upload-pack whose pkt-line body contains no "have <oid>" line. v2 command=ls-refs (discovery) is also rejected. The body is buffered and replayed via io.NopCloser.

func SpoolKeyForRequest 

func SpoolKeyForRequest(pathValue string, r *http.Request) (string, error)

SpoolKeyForRequest returns the spool key for a request, or empty string if the request is not spoolable. For POST requests, the body is hashed to differentiate protocol v2 commands (e.g. ls-refs vs fetch) that share the same URL. The request body is buffered and replaced so it can still be read by the caller.

Types

type Config 

type Config struct {
	SnapshotInterval       time.Duration `` /* 127-byte string literal not displayed */
	SnapshotMaxAge         time.Duration `` /* 228-byte string literal not displayed */
	MirrorSnapshotInterval time.Duration `` /* 159-byte string literal not displayed */
	RepackInterval         time.Duration `hcl:"repack-interval,optional" help:"How often to run full repack. 0 disables." default:"0"`
	ZstdThreads            int           `` /* 248-byte string literal not displayed */
	BundleCacheTTL         time.Duration `hcl:"bundle-cache-ttl,optional" help:"TTL of cached server-side git bundles." default:"2h"`
}

type EnsureRefsRequest 

type EnsureRefsRequest struct {
	Refs    map[string]string `json:"refs,omitempty"`
	Commits []string          `json:"commits,omitempty"`
}

EnsureRefsRequest is the JSON request body for POST .../ensure-refs. At least one of Refs or Commits must be non-empty.

Refs maps each required ref (e.g. "refs/heads/main") to the expected SHA. An empty SHA means "require the ref to exist, at any SHA".

Commits lists individual commit SHAs that must exist in the mirror's object database, regardless of which ref points at them.

type EnsureRefsResponse 

type EnsureRefsResponse struct {
	Refs           map[string]string `json:"refs,omitempty"`
	MissingCommits []string          `json:"missing_commits,omitempty"`
	Fetched        bool              `json:"fetched"`
}

EnsureRefsResponse is the JSON response body for POST .../ensure-refs.

Refs contains the resolved local SHA for each requested ref (empty if the ref is still missing after the fetch). MissingCommits lists the requested commits that are still absent from the local object database after the fetch. Fetched reports whether an upstream fetch was performed.

type RepoCount 

type RepoCount struct {
	Repo  string `json:"repo"`
	Count int64  `json:"count"`
}

RepoCount is one row of the histogram.

type RepoCounts 

type RepoCounts struct {
	// contains filtered or unexported fields
}

RepoCounts tracks per-repository clone counts in a daily-bucketed IntMap. All methods are nil-safe.

func NewRepoCounts 

func NewRepoCounts(ns *metadatadb.Namespace) *RepoCounts

NewRepoCounts returns nil if ns is nil so callers don't need a separate "no metadata configured" code path.

func (*RepoCounts) IncrementClone 

func (r *RepoCounts) IncrementClone(upstreamURL string) error

IncrementClone bumps today's bucket (UTC) for upstreamURL.

func (*RepoCounts) Reap 

func (r *RepoCounts) Reap() (int, error)

Reap deletes buckets older than the retention window and any malformed keys, returning the number of entries deleted.

func (*RepoCounts) TopRepos 

func (r *RepoCounts) TopRepos(windowDays, limit int) []RepoCount

TopRepos aggregates buckets over the last windowDays days (UTC) and returns rows sorted by count descending then repo name ascending. windowDays <= 0 means no window; limit <= 0 means no truncation.

type RepoSpools 

type RepoSpools struct {
	// contains filtered or unexported fields
}

RepoSpools manages all response spools for a single repository.

func NewRepoSpools 

func NewRepoSpools(dir string) *RepoSpools

func (*RepoSpools) Close 

func (rp *RepoSpools) Close() error

Close marks the repo spools as closed, waits for all readers to finish, and removes spool files from disk.

func (*RepoSpools) GetOrCreate 

func (rp *RepoSpools) GetOrCreate(key string) (spool *ResponseSpool, isWriter bool, err error)

GetOrCreate returns an existing spool for the key, or creates a new one. isWriter is true if the caller created the spool and should act as the writer.

type ResponseSpool 

type ResponseSpool struct {
	// contains filtered or unexported fields
}

ResponseSpool captures a single HTTP response (headers + body) to a file on disk, allowing one writer and multiple concurrent readers. Readers follow the writer, blocking when caught up until the write completes.

func NewResponseSpool 

func NewResponseSpool(filePath string) (*ResponseSpool, error)

func (*ResponseSpool) CaptureHeader 

func (rs *ResponseSpool) CaptureHeader(status int, header http.Header)

func (*ResponseSpool) Failed 

func (rs *ResponseSpool) Failed() bool

func (*ResponseSpool) MarkComplete 

func (rs *ResponseSpool) MarkComplete()

func (*ResponseSpool) MarkError 

func (rs *ResponseSpool) MarkError(err error)

func (*ResponseSpool) ServeTo 

func (rs *ResponseSpool) ServeTo(w http.ResponseWriter) error

ServeTo streams the spooled response to w, blocking when caught up to the writer.

func (*ResponseSpool) WaitForReaders 

func (rs *ResponseSpool) WaitForReaders()

WaitForReaders blocks until all active spool readers have finished.

func (*ResponseSpool) Write 

func (rs *ResponseSpool) Write(data []byte) error

func (*ResponseSpool) Written 

func (rs *ResponseSpool) Written() int64

Written returns the total number of bytes written to the spool.

type SnapshotCoordinator added in v0.4.0 

type SnapshotCoordinator struct {
	// contains filtered or unexported fields
}

SnapshotCoordinator shares per-artifact generation state across replicas so that each interval one replica regenerates a given snapshot instead of all of them. Coordination is advisory: the metadata store is last-write-wins and syncs asynchronously, so replicas that decide within the same sync window can still generate concurrently — jittered schedules make that rare, and a duplicate generation is wasteful but harmless.

All methods are nil-safe; without a metadata store every replica generates.

func NewSnapshotCoordinator added in v0.4.0 

func NewSnapshotCoordinator(ns *metadatadb.Namespace) *SnapshotCoordinator

NewSnapshotCoordinator returns nil if ns is nil so callers don't need a separate "no metadata configured" code path.

func (*SnapshotCoordinator) Claim added in v0.4.0 

func (c *SnapshotCoordinator) Claim(job, upstreamURL string, interval time.Duration) (bool, error)

Claim reports whether this replica should generate the artifact now, and records the claim when it should. It declines when another replica completed a generation or checked-and-skipped within the interval, or holds an unexpired in-progress claim.

func (*SnapshotCoordinator) Complete added in v0.4.0 

func (c *SnapshotCoordinator) Complete(job, upstreamURL, commit string) error

Complete records a successful generation and the commit it captured so other replicas skip the artifact until it goes stale again.

func (*SnapshotCoordinator) Prime added in v0.4.0 

func (c *SnapshotCoordinator) Prime(ctx context.Context) error

Prime forces a synchronous refresh of shared state. Backends that sync asynchronously (S3) populate a fresh replica's local view lazily, so without a prime the first claim after startup could run against an empty view and regenerate an artifact a peer completed recently.

func (*SnapshotCoordinator) Skip added in v0.4.0 

func (c *SnapshotCoordinator) Skip(job, upstreamURL string) error

Skip records a claim that ended without an upload (unchanged artifact or nothing to snapshot). It clears the in-progress claim and stamps CheckedAt so peers stay suppressed for the freshness interval rather than the claim TTL, without counting as a completion: CompletedAt must keep tracking the last actual upload so Unchanged's maxAge bound holds.

func (*SnapshotCoordinator) Unchanged added in v0.4.0 

func (c *SnapshotCoordinator) Unchanged(job, upstreamURL, commit string, maxAge time.Duration) bool

Unchanged reports whether the last completed generation captured the same commit and is recent enough (within maxAge) that its cache entry cannot have expired. Callers skipping on this must not call Complete, so CompletedAt keeps tracking the last actual upload and maxAge bounds both cache-entry age and drift of non-HEAD refs captured in the artifact.

type SpoolTeeWriter 

type SpoolTeeWriter struct {
	// contains filtered or unexported fields
}

SpoolTeeWriter wraps an http.ResponseWriter to capture the response into a spool while simultaneously streaming it to the original client.

func NewSpoolTeeWriter 

func NewSpoolTeeWriter(inner http.ResponseWriter, spool *ResponseSpool) *SpoolTeeWriter

NewSpoolTeeWriter creates a new SpoolTeeWriter that tees writes to both the inner ResponseWriter and the given spool.

func (*SpoolTeeWriter) Flush 

func (w *SpoolTeeWriter) Flush()

func (*SpoolTeeWriter) Header 

func (w *SpoolTeeWriter) Header() http.Header

func (*SpoolTeeWriter) Write 

func (w *SpoolTeeWriter) Write(data []byte) (int, error)

func (*SpoolTeeWriter) WriteHeader 

func (w *SpoolTeeWriter) WriteHeader(code int)

type Strategy 

type Strategy struct {
	// contains filtered or unexported fields
}

func New 

func New(
	ctx context.Context,
	config Config,
	schedulerProvider jobscheduler.Provider,
	cache cache.Cache,
	mux strategy.Mux,
	cloneManagerProvider gitclone.ManagerProvider,
	tokenManagerProvider githubapp.TokenManagerProvider,
) (*Strategy, error)

func (*Strategy) Ready 

func (s *Strategy) Ready() bool

Ready reports whether startup warm-up has completed.

func (*Strategy) SetHTTPTransport 

func (s *Strategy) SetHTTPTransport(t http.RoundTripper)

SetHTTPTransport overrides the HTTP transport used for upstream requests. This is intended for testing.

func (*Strategy) SetMetadataStore 

func (s *Strategy) SetMetadataStore(store *metadatadb.Store)

SetMetadataStore enables the per-repo clone histogram (and schedules its daily reaper) and cross-replica snapshot generation coordination. Called by config.Load after the metadata backend is built. It also releases the warm-up goroutine, which waits for wiring so warmed repos are scheduled with coordination in place.

func (*Strategy) String 

func (s *Strategy) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.