Documentation
¶
Overview ¶
Package refs is the spec fabric slice for the remote-repo GIT primitives — pure git-exec + string parsing + cache-path computation, host primitives a plugin composes but the kernel itself needs (reconcile's --remote tag query, the collection walk). RELOCATED from sdk/kit (#55 fabric-primitive extraction), so charly core reaches them via spec/refs directly and the remaining sdk/candy callers keep a thin kit re-export. The FETCH ORCHESTRATION (override → cache → clone → migrate) still lives in the refs plugin (candy/plugin-refs); these are the reusable mechanisms it composes. The only non-stdlib dependency is the advisory file lock (spec/lock) DownloadRepo serializes concurrent cache refresh with.
Index ¶
- Constants
- func CompareSemver(a, b string) int
- func DownloadRepo(repoPath string, version string) (string, error)
- func GitClone(repoURL string, ref string, commit string, targetDir string) error
- func GitDefaultBranch(repoURL string) (string, error)
- func GitLatestTag(repoURL string) (string, error)
- func GitResolveRef(repoURL string, ref string) (string, error)
- func IsMutableRef(version string) bool
- func IsRepoCached(repoPath, version string) (bool, error)
- func RepoCacheDir() (string, error)
- func RepoCachePath(repoPath, version string) (string, error)
- func RepoGitURL(repoPath string) string
- type GitClient
- func (g *GitClient) BypassCache()
- func (g *GitClient) CacheStatus() (string, int)
- func (g *GitClient) ClearCache() error
- func (g *GitClient) DefaultBranch(repoURL string) (string, error)
- func (g *GitClient) Download(repoPath, version string, ...) (string, error)
- func (g *GitClient) LatestTag(repoURL string) (string, error)
- func (g *GitClient) ResolveRef(repoURL, ref string) (string, error)
- func (g *GitClient) SetBypass(on bool) error
- func (g *GitClient) WarmUp(repoURLs []string, stderr *os.File)
Constants ¶
const ( DefaultRefsCacheTTL = time.Hour LatestTagTTL = DefaultRefsCacheTTL DefaultBranchTTL = DefaultRefsCacheTTL ResolveRefTTL = DefaultRefsCacheTTL )
Cache TTLs (see the freshness policy above). One default — the eval-batch reality: a 16-lane, multi-phase check run re-resolves the same refs hundreds of times, so a shorter TTL re-probes mid-batch (measured: 152 concurrent git ls-remote -> GitHub throttling -> the deploy-add stall). LatestTagTTL bounds ONLY the in-process batch-dedupe cache; the persisted latest_tags entries are the offline fallback (served on fetch failure only), never TTL-fresh data.
Variables ¶
This section is empty.
Functions ¶
func CompareSemver ¶
CompareSemver compares two semver-like version strings (e.g. "v1.2.3"). Returns -1 if a < b, 0 if equal, 1 if a > b. It delegates to spec/calver.CompareCalVer — the single dotted-version comparator (which strips the "v" prefix and falls back to lexical comparison for non-numeric parts) — so semver and CalVer strings share ONE implementation.
func DownloadRepo ¶
DownloadRepo downloads a remote repo to the cache. Returns the cache path where the repo was stored.
Freshness contract: the cache is reused ONLY when its recorded provenance (the commit the export was cloned from) equals the ref's CURRENTLY resolved commit — so a branch that advanced upstream (main) is re-downloaded instead of silently serving stale content. The check costs the one ls-remote GitResolveRef already performs; for immutable tags the provenance always matches, so their cache-hit behavior is unchanged. A cache written before this contract has no provenance and is re-downloaded once, then self-heals.
func GitClone ¶
GitClone clones a git repository at a specific ref into the target directory. Uses shallow clone for efficiency.
func GitDefaultBranch ¶
GitDefaultBranch detects the default branch of a remote repository. Uses git ls-remote --symref to find what HEAD points to. Returns the branch name (e.g., "main", "master").
This is the RAW primitive — it always hits the network. Caching is the GitClient's job (refs/git_client.go): every consumer that needs a cached answer goes through GitClient.DefaultBranch. The former per-primitive cache (latest-tags.json) is DELETED — one cache, one layer (R3/R5).
func GitLatestTag ¶
GitLatestTag queries a remote repo for tags and returns the highest semver tag. Looks for tags matching v* pattern, sorts by semver, returns the highest. Returns an error if no version tags are found.
This is the RAW primitive — it always hits the network. Caching is the GitClient's job (refs/git_client.go): every consumer that needs a cached answer goes through GitClient.LatestTag, which wraps this primitive with the persistent `cache:` section of the per-host charly.yml. The former per-primitive cache (latest-tags.json) is DELETED — one cache, one layer (R3/R5).
func GitResolveRef ¶
GitResolveRef resolves a git reference (tag, branch, or commit) to a full commit hash. Uses git ls-remote for tags/branches; for commit hashes, validates length and returns as-is.
NOTE: deliberately NOT cached. DownloadRepo's freshness contract resolves the ref to its CURRENT commit on every call, so a mutable branch (main) that moved upstream is re-downloaded instead of serving stale content (refs/git_test.go TestDownloadRepoFrom_RefreshesMovedBranch). A TTL cache here would break that contract — the freshness check must see the live commit.
func IsMutableRef ¶
IsMutableRef reports whether a repo ref version can ADVANCE upstream (a branch name such as main, or an empty version that resolves to the default branch). Immutable coordinates — CalVer/semver tags (v…) and full commit SHAs — never change what they point at; the project's tags are add-only. Mutable refs must re-resolve on every access: a cache hit on a branch otherwise freezes it at first-download content forever (the pre-#146 @main protocol skew — a stale main export served protocol-v1 plugin sources indefinitely).
func IsRepoCached ¶
IsRepoCached reports whether a repo version is already in the cache AS A USABLE EXPORT — the directory exists AND every submodule it declares has content.
The completeness half is load-bearing for IMMUTABLE refs. EnsureRepoDownloaded short-circuits on `cached && !IsMutableRef(version)`, returning RepoCachePath directly and never entering DownloadRepo — so the repoCacheFresh completeness check cannot see a tag's cache at all. Directory-exists alone therefore pinned every incomplete TAG export permanently: a tag never moves, so nothing would ever re-fetch it. On the machine this was found, `charly@v2026.183.1359` held 0 of its 9 declared submodules and `charly@v2026.201.0706` held 1 of 10, both unrepairable for the life of the cache. Checking content here routes an incomplete export down the download branch instead, so tag and branch caches self-heal by the SAME predicate rather than one growing its own copy.
func RepoCacheDir ¶
RepoCacheDir returns the cache directory for remote repos. Uses $CHARLY_REPO_CACHE env var if set, otherwise ~/.cache/charly/repos/.
func RepoCachePath ¶
RepoCachePath returns the cache path for a specific repo version. e.g. ~/.cache/charly/repos/github.com/org/repo@v1.0.0/
func RepoGitURL ¶
RepoGitURL converts a repo path to a git clone URL. e.g. "github.com/opencharly/ml-layers" -> "https://github.com/opencharly/ml-layers.git"
Types ¶
type GitClient ¶ added in v0.2026241.711
type GitClient struct {
// contains filtered or unexported fields
}
GitClient is the centralized git layer. Construct once per process (or per project) and share it — the cache is the point.
func NewGitClient ¶ added in v0.2026241.711
NewGitClient returns a GitClient whose cache lives in the `cache:` section of the per-host charly.yml at cacheFile. When cacheFile is empty, the default deploy config path (~/.config/charly/charly.yml, honoring the CHARLY_DEPLOY_CONFIG override) is used.
func (*GitClient) BypassCache ¶ added in v0.2026247.2350
func (g *GitClient) BypassCache()
save persists the `cache:` section into the per-host charly.yml under the advisory lock (best-effort). It reads the CURRENT file, updates only the `cache:` key (preserving every other key — deploy:, provides:, ledger:, system:, …), and writes back atomically (tempfile + rename). BypassCache disables every cached lookup — the next resolutions are fresh (an operator on-demand truth: a new tag just released, a moved branch). Runtime-only: the flag does not survive the process (see SetBypass for the persisted form).
func (*GitClient) CacheStatus ¶ added in v0.2026247.2350
CacheStatus reports the cache file path and the number of cached git answers (the operator-facing `charly cache status` surface).
func (*GitClient) ClearCache ¶ added in v0.2026247.2350
ClearCache drops every cached git answer (the in-memory entries and the persisted file), so the next resolutions are fresh — the `charly cache clear/refresh` surface.
func (*GitClient) DefaultBranch ¶ added in v0.2026241.711
func (*GitClient) Download ¶ added in v0.2026241.711
func (g *GitClient) Download(repoPath, version string, download func(repoPath, version string) (string, error)) (string, error)
Download fetches repoPath@version into the repo cache and returns the cache path, CACHED with a short TTL (ResolveRefTTL — aliasing DefaultRefsCacheTTL today, the const block above documents the one-default reality and the future-divergence seam). A mutable ref (a branch or the default branch) can move, so the freshness contract requires re-resolving eventually — but not on every invocation: the TTL means a command run twice in quick succession (e.g. the status fan-out resolving the envelope multiple times) pays the download once. The cached PATH is additionally validated against its CONTENT (dirUsable): a wiped or evicted repo-cache dir must never be served for the TTL (the content-validity principle — the same class the materialized-tree cache fixed with component drift detection).
func (*GitClient) LatestTag ¶ added in v0.2026241.711
LatestTag returns the highest semver tag of repoURL.
Freshness contract (the offline-fallback cutover): the IN-PROCESS cache is the only fresh-data cache (batch dedupe within one process — the 152-concurrent- ls-remote throttling storm was a within-one-process fanout, solved by this layer). The PERSISTED latest_tags entries are an OFFLINE FALLBACK, never fresh data: a fresh process with a reachable network re-probes, so a tag pushed minutes ago is always seen. (The former model — serving persisted entries as fresh for the 1h TTL — was the stale-latest-tag bug: a fresh run resolved plugin refs to a tag list up to an hour old.) The persisted entry is served ONLY when the network fetch fails, with a loud stderr line — resilience, not freshness.
func (*GitClient) ResolveRef ¶ added in v0.2026241.711
func (*GitClient) SetBypass ¶ added in v0.2026248.719
SetBypass persists the bypass flag in the cache: git: section of the per-host charly.yml — the `charly cache bypass` operator surface. When on, every cached lookup is disabled (fresh resolutions) until turned off, and a NEW process (a fresh GitClient) honors it at construction (load reads the flag). When off, the flag is cleared and the cache resumes.
Source Files