Documentation
¶
Overview ¶
Package mirror provides selective package mirroring for pre-populating the proxy cache.
Index ¶
- func ParsePlatforms(values []string) ([]handler.Platform, error)
- type BottleResolver
- type ChartMirrorer
- type ImageMirrorer
- type Job
- type JobRequest
- type JobState
- type JobStore
- type LatestFinder
- type Mirror
- type MirrorError
- type PURLSource
- type PackageFinder
- type PackageVersion
- type Progress
- type ProgressFunc
- type RegistryVersions
- func (r RegistryVersions) FindPackage(ctx context.Context, ecosystem, name string) (*registries.Package, error)
- func (r RegistryVersions) FindVersion(ctx context.Context, ecosystem, name, version string) (registries.Version, error)
- func (r RegistryVersions) LatestVersion(ctx context.Context, ecosystem, name string) (string, error)
- func (r RegistryVersions) ListVersions(ctx context.Context, ecosystem, name string) ([]string, error)
- type SBOMSource
- type Schedule
- type Scheduler
- type Source
- type UnavailableImages
- type VersionFinder
- type VersionLister
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type BottleResolver ¶ added in v0.11.0
type BottleResolver interface {
BottleImage(ctx context.Context, formula, version string) (handler.ImageRef, error)
}
BottleResolver resolves a Homebrew formula version to the image its bottles are published under.
type ChartMirrorer ¶ added in v0.11.0
type ChartMirrorer interface {
MirrorChart(ctx context.Context, repository, chart, version string) (*handler.CacheResult, error)
}
ChartMirrorer caches Helm charts from the configured repositories the way the Helm handler serves them.
type ImageMirrorer ¶ added in v0.10.0
type ImageMirrorer interface {
MirrorImage(ctx context.Context, ref handler.ImageRef, platforms []handler.Platform) (handler.ImageResult, error)
}
ImageMirrorer fetches container images into the cache.
type Job ¶
type Job struct {
ID string `json:"id"`
State JobState `json:"state"`
Progress Progress `json:"progress"`
CreatedAt time.Time `json:"created_at"`
Error string `json:"error,omitempty"`
// contains filtered or unexported fields
}
Job represents an async mirror operation.
type JobRequest ¶
type JobRequest struct {
PURLs []string `json:"purls,omitempty"`
SBOM json.RawMessage `json:"sbom,omitempty"`
Registry string `json:"registry,omitempty"`
// Platforms limit which platforms of container images are mirrored,
// as os/architecture[/variant].
Platforms []string `json:"platforms,omitempty"`
// Latest mirrors only the newest version of PURLs without a version.
Latest bool `json:"latest,omitempty"`
}
JobRequest is the JSON body for starting a mirror job via the API.
type JobStore ¶
type JobStore struct {
// contains filtered or unexported fields
}
JobStore manages in-memory mirror jobs.
func NewJobStore ¶
NewJobStore creates a new job store. The parent context is used as the base for all job contexts so that jobs are canceled when the server shuts down.
func (*JobStore) Cleanup ¶
func (js *JobStore) Cleanup()
Cleanup removes completed/failed/canceled jobs older than jobTTL.
func (*JobStore) Create ¶
func (js *JobStore) Create(req JobRequest) (string, error)
Create starts a new mirror job and returns its ID.
func (*JobStore) Get ¶
Get returns a snapshot of a job by ID. The returned copy is safe to serialize without holding the lock.
func (*JobStore) StartCleanup ¶
StartCleanup runs periodic cleanup of old jobs until the context is canceled.
type LatestFinder ¶ added in v0.11.0
type LatestFinder interface {
LatestVersion(ctx context.Context, ecosystem, name string) (string, error)
}
LatestFinder picks the latest version of a package, for unversioned PURLs when only the latest is mirrored. Listers without it fall back to registries.SelectLatestVersion over ListVersions.
type Mirror ¶
type Mirror struct {
// Images mirrors docker and oci items. Without it they fail like any
// other unsupported ecosystem.
Images ImageMirrorer
// Charts mirrors helm charts. Without it they fail.
Charts ChartMirrorer
// Bottles resolves Homebrew formula versions to the images their
// bottles are published under, which Images mirrors. Without it brew
// PURLs fail.
Bottles BottleResolver
// Versions lists the versions of unversioned PURLs in mirror jobs and
// schedules. Nil lists them from the public registries.
Versions VersionLister
// JuliaUpstream is the Pkg server Julia packages are fetched from. Empty
// uses https://pkg.julialang.org.
JuliaUpstream string
// RefusePackages, when set, fails every item Images does not take with
// this reason instead of fetching it, so nothing is cached.
RefusePackages string
// contains filtered or unexported fields
}
Mirror pre-populates the proxy cache from various input sources.
func New ¶
func New(proxy *handler.Proxy, db *database.DB, store storage.Storage, logger *slog.Logger, workers int) *Mirror
New creates a new Mirror with the given dependencies.
func (*Mirror) Run ¶
func (m *Mirror) Run(ctx context.Context, source Source, onProgress ...ProgressFunc) (*Progress, error)
Run mirrors all packages from the source using a bounded worker pool. It returns the final progress when complete. If onProgress is non-nil, it is called with progress snapshots as work proceeds.
type MirrorError ¶
type MirrorError struct {
Ecosystem string `json:"ecosystem"`
Name string `json:"name"`
Version string `json:"version"`
Error string `json:"error"`
}
MirrorError records a single failed mirror attempt.
type PURLSource ¶
type PURLSource struct {
PURLs []string
// Versions lists the versions of unversioned PURLs. Nil lists them from
// the public registries.
Versions VersionLister
// Latest mirrors only the newest version of unversioned PURLs.
Latest bool
// Platforms limit which platforms of container images are mirrored.
Platforms []string
}
PURLSource yields packages from PURL strings. Versioned PURLs produce a single item. Unversioned PURLs produce one item per version Versions lists, or only the newest with Latest.
func (*PURLSource) Enumerate ¶
func (s *PURLSource) Enumerate(ctx context.Context, fn func(PackageVersion) error) error
type PackageFinder ¶ added in v0.11.0
type PackageFinder interface {
FindPackage(ctx context.Context, ecosystem, name string) (*registries.Package, error)
}
PackageFinder finds a package's registry entry, which for Julia gives the UUID the package is registered under.
type PackageVersion ¶
type PackageVersion struct {
Ecosystem string
Name string
Version string
// Qualifiers are the PURL qualifiers. Images use repository_url, tag and
// arch; PyPI uses file_name, and conda subdir, build and type, to select
// which files of the version are mirrored. Julia uses uuid to identify
// the package.
Qualifiers map[string]string
// Platforms limit which platforms of an image index are mirrored.
Platforms []string
}
PackageVersion identifies a specific package version to mirror.
func (PackageVersion) String ¶
func (pv PackageVersion) String() string
type Progress ¶
type Progress struct {
Total int64 `json:"total"`
Completed int64 `json:"completed"`
Skipped int64 `json:"skipped"`
Failed int64 `json:"failed"`
Bytes int64 `json:"bytes"`
Errors []MirrorError `json:"errors,omitempty"`
StartedAt time.Time `json:"started_at"`
Phase string `json:"phase"`
}
Progress tracks the state of a mirror operation.
type ProgressFunc ¶
type ProgressFunc func(Progress)
ProgressFunc is called periodically with a snapshot of the current progress.
type RegistryVersions ¶ added in v0.11.0
type RegistryVersions struct {
// Client makes the requests. Nil uses registries.DefaultClient.
Client *registries.Client
// URL returns the registry base URL to list a package from, by PURL
// type. Nil, or an empty URL, uses the public registry.
URL func(purlType, name string) (string, error)
}
RegistryVersions lists versions through the registries clients.
func (RegistryVersions) FindPackage ¶ added in v0.11.0
func (r RegistryVersions) FindPackage(ctx context.Context, ecosystem, name string) (*registries.Package, error)
FindPackage implements PackageFinder.
func (RegistryVersions) FindVersion ¶ added in v0.11.0
func (r RegistryVersions) FindVersion(ctx context.Context, ecosystem, name, version string) (registries.Version, error)
FindVersion implements VersionFinder.
func (RegistryVersions) LatestVersion ¶ added in v0.11.0
func (r RegistryVersions) LatestVersion(ctx context.Context, ecosystem, name string) (string, error)
LatestVersion implements LatestFinder with registries.FetchLatestVersion: the version the registry advertises as latest where it has one, such as npm's dist-tags.latest, and otherwise the newest version by publish time that isn't yanked, retracted or deprecated.
func (RegistryVersions) ListVersions ¶ added in v0.11.0
func (r RegistryVersions) ListVersions(ctx context.Context, ecosystem, name string) ([]string, error)
ListVersions implements VersionLister.
type SBOMSource ¶
type SBOMSource struct {
Data []byte
Name string
// Versions lists the versions of unversioned PURLs. Nil lists them from
// the public registries.
Versions VersionLister
// Latest mirrors only the newest version of unversioned PURLs.
Latest bool
// Platforms limit which platforms of container images are mirrored.
Platforms []string
}
SBOMSource extracts package versions from CycloneDX or SPDX SBOM data.
func (*SBOMSource) Enumerate ¶
func (s *SBOMSource) Enumerate(ctx context.Context, fn func(PackageVersion) error) error
type Scheduler ¶ added in v0.10.0
type Scheduler struct {
// Claim, when set, reports whether this instance should run a schedule
// that no instance sharing the database started within minGap. Each
// instance keeps its own timer, so without it every instance would run
// every schedule.
Claim func(key string, minGap time.Duration) (bool, error)
// Release, when set, drops the claim of a run that was interrupted, so
// the next attempt runs it rather than waiting an interval.
Release func(key string) error
// contains filtered or unexported fields
}
Scheduler runs schedules in the background until its context ends.
func NewScheduler ¶ added in v0.10.0
NewScheduler creates a scheduler that runs the schedules through m.
type Source ¶
type Source interface {
Enumerate(ctx context.Context, fn func(PackageVersion) error) error
}
Source produces PackageVersion items for mirroring.
type UnavailableImages ¶ added in v0.10.0
type UnavailableImages struct {
}
UnavailableImages refuses every image with the given reason.
func (UnavailableImages) MirrorImage ¶ added in v0.10.0
func (u UnavailableImages) MirrorImage(context.Context, handler.ImageRef, []handler.Platform) (handler.ImageResult, error)
type VersionFinder ¶ added in v0.11.0
type VersionFinder interface {
FindVersion(ctx context.Context, ecosystem, name, version string) (registries.Version, error)
}
VersionFinder finds one version of a package, with the files it publishes, for ecosystems the mirror fetches from the version's metadata.