gradlecache

package
v0.6.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 1, 2026 License: Apache-2.0 Imports: 31 Imported by: 0

Documentation

Overview

Package gradlecache provides a Go library for restoring Gradle build cache bundles from S3 or cachew. It wraps the same logic used by the gradle-cache CLI in an importable API.

Index

Constants

This section is empty.

Variables

View Source
var AtomicCacheParents = map[string]bool{
	"transforms":             true,
	"groovy-dsl":             true,
	"kotlin-dsl":             true,
	"dependencies-accessors": true,
	"jars-9":                 true,
}

AtomicCacheParents lists directory names whose descendants include hash-keyed workspace directories that must be captured atomically. The walker recurses through intermediate subdirectories until it finds children whose names look like hex hashes, then treats those as atomic units: if ANY file inside has mtime > marker, ALL files are included.

These directories contain hash-keyed subdirectories where Gradle expects all files to be present together. Some are true immutable workspaces (transforms, groovy-dsl, kotlin-dsl, dependencies-accessors) created via atomic directory renames. Others are persistent caches with receipt-based completion markers (jars-9). In both cases, a partial directory (e.g. metadata.bin without output files, or a .receipt without the transformed classes) causes Gradle to crash.

Mtime skew across delta cycles can produce partial directories: after restoring base + delta, the base provides output files (old mtime) while the delta overwrites metadata/receipt files (new mtime). A per-file mtime check captures only the newer files, producing a partial directory in the next delta. When applied to a base that lacks the hash, Gradle crashes.

View Source
var CacheExclusions = []string{

	"daemon",

	".tmp",

	"gc.properties",

	"*.lock",

	"cc-keystore",

	"file-changes",

	"journal-1",

	"user-id.txt",
}

CacheExclusions are patterns for files and directories that should never be included in cache bundles (base or delta).

View Source
var DeltaExclusions = []string{

	"fileHashes",

	"module-metadata.bin",
}

DeltaExclusions are additional file/directory names excluded only from delta bundles. These files are already present in the base bundle and get rewritten every build (Gradle's embedded BTree DB flushes on close even for read-only access), so the delta copy adds negligible value at significant size cost.

Functions

func CollectNewFiles

func CollectNewFiles(realCaches string, since time.Time, gradleHome string) ([]string, error)

CollectNewFiles walks realCaches in parallel and returns paths of regular files with mtime strictly after since. For directories listed in AtomicCacheParents, if any file in a child hash directory is newer than since, all files in that directory are included to prevent partial restores.

func ConventionBuildDirs

func ConventionBuildDirs(root string, includedBuilds []string) []string

ConventionBuildDirs returns the relative paths of included build output directories.

func CreateDeltaTarZstd

func CreateDeltaTarZstd(_ context.Context, w io.Writer, baseDir string, relPaths []string) error

CreateDeltaTarZstd creates a zstd-compressed tar archive containing the files at relPaths (relative to baseDir).

func CreateTarZstd

func CreateTarZstd(ctx context.Context, w io.Writer, sources []TarSource) error

CreateTarZstd creates a zstd-compressed tar archive from the given sources. If pzstd is available it is used to produce a multi-frame archive that can be decompressed in parallel on restore. Otherwise klauspost is used.

func DetectStatsdAddr

func DetectStatsdAddr() string

DetectStatsdAddr returns the DogStatsD address from the environment, or empty if DD_AGENT_HOST is not set.

func IsDeltaExcluded

func IsDeltaExcluded(name string) bool

IsDeltaExcluded reports whether a name matches delta-only exclusion patterns.

func IsExcludedCache

func IsExcludedCache(name string) bool

IsExcludedCache reports whether a file or directory name matches any cache exclusion pattern.

func IsFullSHA

func IsFullSHA(s string) bool

IsFullSHA returns true if s is a 40-character lowercase hex string.

func Restore

func Restore(ctx context.Context, cfg RestoreConfig) error

Restore downloads and extracts a Gradle cache bundle, routing tar entries directly to their final destinations. It walks git history to find the most recent cached bundle, downloads it with parallel range requests, and streams it through zstd decompression into the filesystem.

If Branch is set, a delta bundle is downloaded concurrently and applied after the base extraction.

func RestoreDelta

func RestoreDelta(ctx context.Context, cfg RestoreDeltaConfig) error

RestoreDelta downloads and applies a branch-specific delta bundle on top of an already-restored base cache.

func Save

func Save(ctx context.Context, cfg SaveConfig) error

Save archives GRADLE_USER_HOME/caches and uploads it to the configured backend.

func SaveDelta

func SaveDelta(ctx context.Context, cfg SaveDeltaConfig) error

SaveDelta packs all cache files created since the last restore and uploads them as a branch delta bundle.

func WriteDeltaTar

func WriteDeltaTar(w io.Writer, baseDir string, relPaths []string) error

WriteDeltaTar writes a tar stream for the specified files to w.

Types

type DatadogAPIClient

type DatadogAPIClient struct {
	// contains filtered or unexported fields
}

DatadogAPIClient sends metrics via the Datadog HTTP API.

func NewDatadogAPIClient

func NewDatadogAPIClient(apiKey string, baseTags []string) *DatadogAPIClient

NewDatadogAPIClient creates a new Datadog API client.

func (*DatadogAPIClient) Close

func (d *DatadogAPIClient) Close()

func (*DatadogAPIClient) Distribution

func (d *DatadogAPIClient) Distribution(name string, value float64, tags ...string)

type MetricsClient

type MetricsClient interface {
	Distribution(name string, value float64, tags ...string)
	Close()
}

MetricsClient emits distribution metrics to a backend.

type MetricsFlags

type MetricsFlags struct {
	StatsdAddr    string
	DatadogAPIKey string
	MetricsTags   []string
}

MetricsFlags are CLI flags for configuring metrics emission.

func (*MetricsFlags) NewMetricsClient

func (f *MetricsFlags) NewMetricsClient() MetricsClient

NewMetricsClient returns a MetricsClient based on the configured flags.

type NoopMetrics

type NoopMetrics struct{}

NoopMetrics is a no-op MetricsClient used when no backend is configured.

func (NoopMetrics) Close

func (NoopMetrics) Close()

func (NoopMetrics) Distribution

func (NoopMetrics) Distribution(string, float64, ...string)

type RestoreConfig

type RestoreConfig struct {
	// Bucket is the S3 bucket name. Mutually exclusive with CachewURL.
	Bucket string
	// Region is the AWS region. Falls back to AWS_REGION / AWS_DEFAULT_REGION env vars, then "us-west-2".
	Region string
	// CachewURL is the cachew server URL. Mutually exclusive with Bucket.
	CachewURL string
	// CacheKey is the bundle identifier (e.g. "my-project:assembleRelease").
	CacheKey string
	// GitDir is the path to the git repository for history walking. Defaults to ".".
	GitDir string
	// Ref is the git ref used to search for a base bundle. When Branch is set,
	// history walks from the merge-base of HEAD and Ref. Defaults to "HEAD".
	Ref string
	// Commit is a specific commit SHA to try directly, skipping history walk.
	Commit string
	// MaxBlocks is the number of distinct-author commit blocks to search. Defaults to 20.
	MaxBlocks int
	// GradleUserHome is the path to GRADLE_USER_HOME. Defaults to ~/.gradle.
	GradleUserHome string
	// IncludedBuilds lists included build directories whose build/ output to
	// restore (relative to project root). Defaults to ["buildSrc"].
	IncludedBuilds []string
	// Branch is an optional branch name to also apply a delta bundle for.
	Branch string
	// Metrics is an optional metrics client. If nil, a no-op client is used.
	Metrics MetricsClient
	// Logger is an optional structured logger. If nil, slog.Default() is used.
	Logger *slog.Logger
}

RestoreConfig holds the parameters for a cache restore operation.

type RestoreDeltaConfig

type RestoreDeltaConfig struct {
	Bucket         string
	Region         string
	CachewURL      string
	CacheKey       string
	Branch         string
	GradleUserHome string
	Metrics        MetricsClient
	Logger         *slog.Logger
}

RestoreDeltaConfig holds the parameters for a delta restore operation.

type SaveConfig

type SaveConfig struct {
	Bucket         string
	Region         string
	CachewURL      string
	CacheKey       string
	Commit         string
	GitDir         string
	GradleUserHome string
	IncludedBuilds []string
	Metrics        MetricsClient
	Logger         *slog.Logger
}

SaveConfig holds the parameters for a cache save operation.

type SaveDeltaConfig

type SaveDeltaConfig struct {
	Bucket         string
	Region         string
	CachewURL      string
	CacheKey       string
	Branch         string
	GradleUserHome string
	Metrics        MetricsClient
	Logger         *slog.Logger
}

SaveDeltaConfig holds the parameters for a delta save operation.

type StatsdClient

type StatsdClient struct {
	// contains filtered or unexported fields
}

StatsdClient sends metrics via DogStatsD UDP protocol.

func NewStatsdClient

func NewStatsdClient(addr string, baseTags []string) *StatsdClient

NewStatsdClient creates a new statsd client, or nil if connection fails.

func (*StatsdClient) Close

func (s *StatsdClient) Close()

func (*StatsdClient) Distribution

func (s *StatsdClient) Distribution(name string, value float64, tags ...string)

type TarSource

type TarSource struct {
	BaseDir string
	Path    string
}

TarSource specifies a (base directory, relative path) pair for inclusion in a tar archive.

func ProjectDirSources

func ProjectDirSources(projectDir string, includedBuilds []string) []TarSource

ProjectDirSources returns tarSource entries for project-specific dirs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL