Documentation
¶
Overview ¶
Package fetch provides utilities for fetching GitHub repository metadata and computing checksums.
Index ¶
- Constants
- func DownloadTarball(ctx context.Context, target, url, expectedSha256 string) error
- func ExtractTarball(tarballPath, destDir string) error
- func LatestCommitAndChecksum(endpoints *Endpoints, repo *Repo) (commit, sha256 string, err error)
- func RepoDir(ctx context.Context, repo, commit, expectedSHA256 string) (string, error)
- func TarballLink(githubDownload string, repo *Repo, sha string) string
- type Endpoints
- type Repo
Constants ¶
const ( // DefaultBranchMaster represents the default git branch "master". DefaultBranchMaster = "master" // DefaultBranchMain represents the default git branch "main". DefaultBranchMain = "main" )
Variables ¶
This section is empty.
Functions ¶
func DownloadTarball ¶ added in v0.8.0
DownloadTarball downloads a tarball from the given url to the target path, verifying its SHA256 checksum matches expectedSha256. It retries up to maxDownloadRetries times with exponential backoff on failure.
func ExtractTarball ¶ added in v0.8.0
ExtractTarball extracts a gzipped tarball to the specified directory, stripping the top-level directory prefix that GitHub adds to tarballs.
func LatestCommitAndChecksum ¶ added in v0.8.0
LatestCommitAndChecksum fetches the latest commit SHA and the SHA256 of the tarball for that commit from the GitHub API for the given repository.
func RepoDir ¶ added in v0.8.0
RepoDir downloads a repository tarball and returns the path to the extracted directory.
The cache directory is determined by LIBRARIAN_CACHE environment variable, or defaults to $HOME/.cache/librarian if not set.
The diagrams below explains the structure of the librarian cache. For each path, $repo is a repository path (i.e. github.com/googleapis/googleapis), and $commit is a commit hash in that repository.
Cache structure:
$LIBRARIAN_CACHE/
├── download/ # Downloaded artifacts
│ └── $repo@$commit.tar.gz # Source tarball (kept for re-extraction)
└── $repo@$commit/ # Extracted source files
└── {files...}
Example for github.com/googleapis/googleapis at commit abc123:
$HOME/.cache/librarian/
├── download/
│ └── github.com/googleapis/googleapis@abc123.tar.gz
└── github.com/googleapis/googleapis@abc123/
└── google/
└── api/
└── annotations.proto
Cache lookup order:
- Check if extracted directory exists and contains files. If so, return it.
- Check if tarball exists. Verify its SHA256 matches expectedSHA256. If yes, extract tarball and return the directory. If the hash mismatches, fall through to step 3.
- Download tarball, compute SHA256, verify it matches expectedSHA256 from librarian.yaml, extract, and return the path.
Types ¶
type Endpoints ¶
type Endpoints struct {
// API defines the endpoint used to make API calls.
API string
// Download defines the endpoint to download tarballs.
Download string
}
Endpoints defines the endpoints used to access GitHub.
type Repo ¶
type Repo struct {
// Branch is the name of the repository branch, such as `master` or `preview`.
Branch string
// Org defines the GitHub organization (or user), that owns the repository.
Org string
// Repo is the name of the repository, such as `googleapis` or `google-cloud-rust`.
Repo string
}
Repo represents a GitHub repository name.
func RepoFromTarballLink ¶
RepoFromTarballLink extracts the gitHub account and repository (such as `googleapis/googleapis`, or `googleapis/google-cloud-rust`) from the tarball link. Note: This does **not** set [Repo.Branch] as it is not derivable from a commit-based archive URL.
Source Files