git

package
v0.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 9, 2026 License: Apache-2.0 Imports: 43 Imported by: 0

Documentation

Overview

Package git implements a protocol-aware Git caching proxy strategy.

Index

Constants

View Source 
const EnsureRefsPath = "/ensure-refs"

EnsureRefsPath is the URL suffix for the ref-freshness endpoint. Clients POST to /git/{host}/{repo}/ensure-refs to force cachew to ensure the local mirror contains the listed refs before they fetch.

Variables

This section is empty.

Functions

func ExtractRepoPath 

func ExtractRepoPath(pathValue string) string

func Register 

func Register(r *strategy.Registry, scheduler jobscheduler.Provider, cloneManagerProvider gitclone.ManagerProvider, tokenManagerProvider githubapp.TokenManagerProvider, spools spool.Provider)

func RequestIsClone 

func RequestIsClone(pathValue string, r *http.Request) (bool, error)

RequestIsClone reports whether r is an initial clone of a Git repo.

Detection: POST /git-upload-pack whose pkt-line body contains no "have <oid>" line. v2 command=ls-refs (discovery) is also rejected. The body is buffered and replayed via io.NopCloser.

func SpoolKeyForRequest 

func SpoolKeyForRequest(pathValue string, r *http.Request) (string, error)

SpoolKeyForRequest returns the spool key for a request, or empty string if the request is not spoolable. For POST requests, the body is hashed to differentiate protocol v2 commands (e.g. ls-refs vs fetch) that share the same URL. The request body is buffered and replaced so it can still be read by the caller.

Types

type Config 

type Config struct {
	SnapshotInterval       time.Duration `` /* 127-byte string literal not displayed */
	SnapshotMaxAge         time.Duration `` /* 228-byte string literal not displayed */
	MirrorSnapshotInterval time.Duration `` /* 159-byte string literal not displayed */
	RepackInterval         time.Duration `hcl:"repack-interval,optional" help:"How often to run full repack. 0 disables." default:"0"`
	ZstdThreads            int           `` /* 248-byte string literal not displayed */
	BundleCacheTTL         time.Duration `hcl:"bundle-cache-ttl,optional" help:"TTL of cached server-side git bundles." default:"2h"`
	MaxRequestBodyMB       int           `` /* 260-byte string literal not displayed */

	SnapshotFilters map[string]string `` /* 346-byte string literal not displayed */
}

type EnsureRefsRequest 

type EnsureRefsRequest struct {
	Refs    map[string]string `json:"refs,omitempty"`
	Commits []string          `json:"commits,omitempty"`
}

EnsureRefsRequest is the JSON request body for POST .../ensure-refs. At least one of Refs or Commits must be non-empty.

Refs maps each required ref (e.g. "refs/heads/main") to the expected SHA. An empty SHA means "require the ref to exist, at any SHA".

Commits lists individual commit SHAs that must exist in the mirror's object database, regardless of which ref points at them.

type EnsureRefsResponse 

type EnsureRefsResponse struct {
	Refs           map[string]string `json:"refs,omitempty"`
	MissingCommits []string          `json:"missing_commits,omitempty"`
	Fetched        bool              `json:"fetched"`
}

EnsureRefsResponse is the JSON response body for POST .../ensure-refs.

Refs contains the resolved local SHA for each requested ref (empty if the ref is still missing after the fetch). MissingCommits lists the requested commits that are still absent from the local object database after the fetch. Fetched reports whether an upstream fetch was performed.

type RepoCount 

type RepoCount struct {
	Repo  string `json:"repo"`
	Count int64  `json:"count"`
}

RepoCount is one row of the histogram.

type RepoCounts 

type RepoCounts struct {
	// contains filtered or unexported fields
}

RepoCounts tracks per-repository clone counts in a daily-bucketed IntMap. All methods are nil-safe.

func NewRepoCounts 

func NewRepoCounts(ns *metadatadb.Namespace) *RepoCounts

NewRepoCounts returns nil if ns is nil so callers don't need a separate "no metadata configured" code path.

func (*RepoCounts) IncrementClone 

func (r *RepoCounts) IncrementClone(upstreamURL string) error

IncrementClone bumps today's bucket (UTC) for upstreamURL.

func (*RepoCounts) Reap 

func (r *RepoCounts) Reap() (int, error)

Reap deletes buckets older than the retention window and any malformed keys, returning the number of entries deleted.

func (*RepoCounts) TopRepos 

func (r *RepoCounts) TopRepos(windowDays, limit int) []RepoCount

TopRepos aggregates buckets over the last windowDays days (UTC) and returns rows sorted by count descending then repo name ascending. windowDays <= 0 means no window; limit <= 0 means no truncation.

type SnapshotCoordinator added in v0.4.0 

type SnapshotCoordinator struct {
	// contains filtered or unexported fields
}

SnapshotCoordinator shares per-artifact generation state across replicas so that each interval one replica regenerates a given snapshot instead of all of them. Coordination is advisory: the metadata store is last-write-wins and syncs asynchronously, so replicas that decide within the same sync window can still generate concurrently — jittered schedules make that rare, and a duplicate generation is wasteful but harmless.

All methods are nil-safe; without a metadata store every replica generates.

func NewSnapshotCoordinator added in v0.4.0 

func NewSnapshotCoordinator(ns *metadatadb.Namespace) *SnapshotCoordinator

NewSnapshotCoordinator returns nil if ns is nil so callers don't need a separate "no metadata configured" code path.

func (*SnapshotCoordinator) Claim added in v0.4.0 

func (c *SnapshotCoordinator) Claim(job, upstreamURL string, interval time.Duration) (bool, error)

Claim reports whether this replica should generate the artifact now, and records the claim when it should. It declines when another replica completed a generation or checked-and-skipped within the interval, or holds an unexpired in-progress claim.

func (*SnapshotCoordinator) ClaimWithTTL added in v0.5.1 

func (c *SnapshotCoordinator) ClaimWithTTL(job, upstreamURL string, interval, claimTTL time.Duration) (string, bool, error)

ClaimWithTTL lets the caller set the claim duration. The caller must use the returned claim ID to complete or release the claim.

func (*SnapshotCoordinator) Complete added in v0.4.0 

func (c *SnapshotCoordinator) Complete(job, upstreamURL, commit string) error

Complete records a successful generation and the commit it captured so other replicas skip the artifact until it goes stale again.

func (*SnapshotCoordinator) CompleteClaim added in v0.5.1 

func (c *SnapshotCoordinator) CompleteClaim(ctx context.Context, job, upstreamURL, claimID, commit string) error

CompleteClaim reads the shared state again. It completes the claim only if the claim ID still matches.

func (*SnapshotCoordinator) Fail added in v0.5.1 

func (c *SnapshotCoordinator) Fail(ctx context.Context, job, upstreamURL, claimID string) error

Fail releases a failed claim only if the claim ID still matches the shared state. It does not record a successful check, so another caller can try again immediately.

func (*SnapshotCoordinator) Prime added in v0.4.0 

func (c *SnapshotCoordinator) Prime(ctx context.Context) error

Prime forces a synchronous refresh of shared state. Backends that sync asynchronously (S3) populate a fresh replica's local view lazily, so without a prime the first claim after startup could run against an empty view and regenerate an artifact a peer completed recently.

func (*SnapshotCoordinator) Skip added in v0.4.0 

func (c *SnapshotCoordinator) Skip(job, upstreamURL string) error

Skip records a claim that ended without an upload (unchanged artifact or nothing to snapshot). It clears the in-progress claim and stamps CheckedAt so peers stay suppressed for the freshness interval rather than the claim TTL, without counting as a completion: CompletedAt must keep tracking the last actual upload so Unchanged's maxAge bound holds.

func (*SnapshotCoordinator) SkipClaim added in v0.5.1 

func (c *SnapshotCoordinator) SkipClaim(ctx context.Context, job, upstreamURL, claimID string) error

SkipClaim reads the shared state again. It records a skip only if the claim ID still matches.

func (*SnapshotCoordinator) Unchanged added in v0.4.0 

func (c *SnapshotCoordinator) Unchanged(job, upstreamURL, commit string, maxAge time.Duration) bool

Unchanged reports whether the last completed generation captured the same commit and is recent enough (within maxAge) that its cache entry cannot have expired. Callers skipping on this must not call Complete, so CompletedAt keeps tracking the last actual upload and maxAge bounds both cache-entry age and drift of non-HEAD refs captured in the artifact.

type Strategy 

type Strategy struct {
	// contains filtered or unexported fields
}

func New 

func New(
	ctx context.Context,
	config Config,
	schedulerProvider jobscheduler.Provider,
	cache cache.Cache,
	mux strategy.Mux,
	cloneManagerProvider gitclone.ManagerProvider,
	tokenManagerProvider githubapp.TokenManagerProvider,
	spools *spool.Registry,
) (*Strategy, error)

func (*Strategy) Ready 

func (s *Strategy) Ready() bool

Ready reports whether startup warm-up has completed.

func (*Strategy) SetHTTPTransport 

func (s *Strategy) SetHTTPTransport(t http.RoundTripper)

SetHTTPTransport overrides the HTTP transport used for upstream requests. This is intended for testing.

func (*Strategy) SetMetadataStore 

func (s *Strategy) SetMetadataStore(store *metadatadb.Store)

SetMetadataStore enables the per-repo clone histogram (and schedules its daily reaper) and cross-replica snapshot generation coordination. Called by config.Load after the metadata backend is built. It also releases the warm-up goroutine, which waits for wiring so warmed repos are scheduled with coordination in place.

func (*Strategy) String 

func (s *Strategy) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.