gitclone

package
v0.0.0-...-e308f56 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 10, 2026 License: Apache-2.0 Imports: 15 Imported by: 0

Documentation

Overview

Package gitclone provides reusable git clone management with lifecycle control, concurrency management, and large repository optimizations.

Index

Constants

View Source 
const CloneTimeout = 30 * time.Minute

CloneTimeout bounds `git clone --mirror` so a stuck clone cannot block the repo indefinitely. This is deliberately generous: large repos may take 10-20 minutes for GitHub to compute the server-side pack.

Variables

This section is empty.

Functions

func ParseGitRefs 

func ParseGitRefs(output []byte) map[string]string

func RepoPathFromURL 

func RepoPathFromURL(upstreamURL string) (string, error)

RepoPathFromURL extracts a normalised "host/path" from an upstream Git URL, stripping any ".git" suffix.

func WithReadLockReturn 

func WithReadLockReturn[T any](repo *Repository, fn func() (T, error)) (T, error)

Types

type Config 

type Config struct {
	MirrorRoot       string        `hcl:"mirror-root,optional" help:"Directory to store git clones." default:"${CACHEW_STATE}/git-mirrors"`
	FetchInterval    time.Duration `hcl:"fetch-interval,optional" help:"How often to fetch from upstream in minutes." default:"15m"`
	RefCheckInterval time.Duration `hcl:"ref-check-interval,optional" help:"How long to cache ref checks." default:"10s"`
	Maintenance      bool          `hcl:"maintenance,optional" help:"Enable git maintenance scheduling for mirror repos." default:"false"`
	PackThreads      int           `hcl:"pack-threads,optional" help:"Threads for git pack operations (0 = all CPU cores)." default:"0"`
}

type CredentialProvider 

type CredentialProvider interface {
	GetTokenForURL(ctx context.Context, url string) (string, error)
}

CredentialProvider provides credentials for git operations.

type CredentialProviderProvider 

type CredentialProviderProvider func() (CredentialProvider, error)

type GitTuningConfig 

type GitTuningConfig struct {
	PostBuffer    int           // http.postBuffer size in bytes
	LowSpeedLimit int           // http.lowSpeedLimit in bytes/sec
	LowSpeedTime  time.Duration // http.lowSpeedTime
}

func DefaultGitTuningConfig 

func DefaultGitTuningConfig() GitTuningConfig

type Manager 

type Manager struct {
	// contains filtered or unexported fields
}

func NewManager 

func NewManager(ctx context.Context, config Config, credentialProvider CredentialProvider) (*Manager, error)

func (*Manager) Config 

func (m *Manager) Config() Config

func (*Manager) DiscoverExisting 

func (m *Manager) DiscoverExisting(ctx context.Context) ([]*Repository, error)

func (*Manager) Get 

func (m *Manager) Get(upstreamURL string) *Repository

func (*Manager) GetOrCreate 

func (m *Manager) GetOrCreate(_ context.Context, upstreamURL string) (*Repository, error)

type ManagerProvider 

type ManagerProvider func() (*Manager, error)

ManagerProvider is a function that lazily creates a singleton Manager.

func NewManagerProvider 

func NewManagerProvider(ctx context.Context, config Config, credentialProviderProvider CredentialProviderProvider) ManagerProvider

type Repository 

type Repository struct {
	// contains filtered or unexported fields
}

func (*Repository) Clone 

func (r *Repository) Clone(ctx context.Context) error

func (*Repository) ConfigureMirror 

func (r *Repository) ConfigureMirror(ctx context.Context, repoPath string) error

ConfigureMirror configures a git directory at repoPath as a mirror with the repository's pack threads and optional maintenance settings.

func (*Repository) EnsureRefsUpToDate 

func (r *Repository) EnsureRefsUpToDate(ctx context.Context) (needsFetch bool, err error)

EnsureRefsUpToDate checks whether the local mirror's refs match upstream. If refs are stale it returns NeedsFetch=true so the caller can schedule a background fetch via the job scheduler, rather than fetching synchronously on the request path (which would acquire a write lock and block all serving).

func (*Repository) Fetch 

func (r *Repository) Fetch(ctx context.Context) error

func (*Repository) FetchLenient 

func (r *Repository) FetchLenient(ctx context.Context, timeout time.Duration) error

FetchLenient fetches from upstream with the given timeout but without the low-speed transfer check. Use this for post-restore catch-up fetches where the delta may be very large and GitHub's server-side pack computation can stall at near-zero transfer rate for minutes — the same situation that executeClone handles by omitting lowSpeedLimit.

func (*Repository) FetchWithTimeout 

func (r *Repository) FetchWithTimeout(ctx context.Context, timeout time.Duration) error

FetchWithTimeout fetches from upstream with a configurable timeout. Use this for catch-up fetches after snapshot restore where the delta may be large and the default fetchTimeout is too short.

func (*Repository) GetLocalRefs 

func (r *Repository) GetLocalRefs(ctx context.Context) (map[string]string, error)

func (*Repository) GetUpstreamRefs 

func (r *Repository) GetUpstreamRefs(ctx context.Context) (map[string]string, error)

func (*Repository) GitCommand 

func (r *Repository) GitCommand(ctx context.Context, args ...string) (*exec.Cmd, error)

GitCommand returns a git subprocess configured with repository-scoped authentication and any per-URL git config overrides disabled.

func (*Repository) HasCommit 

func (r *Repository) HasCommit(ctx context.Context, ref string) bool

func (*Repository) LastFetch 

func (r *Repository) LastFetch() time.Time

func (*Repository) MarkReady 

func (r *Repository) MarkReady()

MarkReady transitions the repository to StateReady, indicating it is up-to-date and can serve requests from the local mirror.

func (*Repository) NeedsFetch 

func (r *Repository) NeedsFetch(fetchInterval time.Duration) bool

func (*Repository) Path 

func (r *Repository) Path() string

func (*Repository) Repack 

func (r *Repository) Repack(ctx context.Context) error

Repack consolidates pack files using geometric repacking. Unlike a full repack (-a), geometric repacking only merges packs when there is significant fragmentation (many small packs), making it orders of magnitude faster on large repositories in steady state. The --write-midx and --write-bitmap-index flags maintain the multi-pack index and reachability bitmaps for efficient serving via git http-backend.

func (*Repository) ResetToEmpty 

func (r *Repository) ResetToEmpty()

ResetToEmpty transitions the repository back to StateEmpty so that a subsequent call to TryStartCloning can re-attempt the clone. Use this when a restored snapshot turns out to be corrupt or empty and needs to be replaced with a fresh clone.

func (*Repository) State 

func (r *Repository) State() State

func (*Repository) TryStartCloning 

func (r *Repository) TryStartCloning() bool

TryStartCloning atomically transitions the repository from StateEmpty to StateCloning. Returns true if this goroutine won the transition and should proceed with the clone/restore; false if another goroutine already claimed it.

func (*Repository) UpstreamURL 

func (r *Repository) UpstreamURL() string

func (*Repository) WithFetchExclusion 

func (r *Repository) WithFetchExclusion(ctx context.Context, fn func() error) error

WithFetchExclusion runs fn while holding the fetch semaphore, preventing concurrent git fetch operations. Use this for operations like tar that read the repository directory non-atomically and need a consistent view.

func (*Repository) WithReadLock 

func (r *Repository) WithReadLock(fn func() error) error

type State 

type State int
const (
	StateEmpty   State = iota // Not cloned yet
	StateCloning              // Clone in progress
	StateReady                // Ready to use
)

func (State) String 

func (s State) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.