Documentation
¶
Overview ¶
Package imagecache rewrites image references so the nodes pull them through the platform's in-cluster image cache (the registry gate in front of zot).
The cache is served on every node as localhost:<port> by the node-agent's registry forwarder. A reference REG/repo:tag of a tenant becomes localhost:<port>/c/<tenant>/REG/repo:tag; the gate takes the tenant from the path (names.TenantSegment), checks the pull against the schedule and maps the REG segment back to the upstream registry of its list (allowed registries), so the layout needs no per-image configuration. The c/ segment keeps the cache apart from the snapshots, which the gate does not check against an upstream. The internal tree of the cache has no tenant: c/REG/repo (RepoOf).
Index ¶
- Constants
- Variables
- func CredentialSource(fingerprint string) string
- func DigestOf(ref string) string
- func PlatformScope(ps []v1.Platform) string
- func ProbeDigest(ctx context.Context, ref string, kc authn.Keychain, rt http.RoundTripper) (string, error)
- func SafeError(err error) string
- func StripPlatformHeaders(h http.Header)
- type AsyncResolver
- type DockerConfigKeychain
- type Lookup
- type PinError
- type RegistryResolver
- type Resolver
- type Rewriter
- type Scoped
- type TokenFile
Constants ¶
const ( HeaderGateCache = "Registry-Gate-Cache" GateCacheHit = "hit" )
HeaderGateCache is the response header with which the registry gate says where an answer on the cache path came from: GateCacheHit when the object was served from the cache's own storage (after the gate proved the pull). The prewarm asks for it to know that a blob is really stored and not only proved.
const ( // HeaderNodeToken carries the token. HeaderNodeToken = "X-Cib-Node-Token" // HeaderPrefix is the prefix of every header the platform defines on the way to the gate. HeaderPrefix = "X-Cib-" // TokenAudience is the audience of the projected token the gate accepts. TokenAudience = "laboratory-registry-gate" // DefaultTokenPath is where the chart projects the token into the pod. DefaultTokenPath = "/var/run/secrets/registry-gate/token" // TokenReread is how often the token file is read again (the kubelet rotates a projected token well before it expires). TokenReread = 30 * time.Second )
What the components of the laboratory present to the registry gate: a projected ServiceAccount token of the audience TokenAudience in the header HeaderNodeToken. The gate reviews it with the API server and learns who calls (the relay of a node, the operator, the agent); it trusts no other X-Cib-* header, and a component removes every one of them from what it relays (StripPlatformHeaders).
const CacheSegment = "c"
CacheSegment is the first path segment of the cache under the registry's address.
const OperatorSource = "operator"
OperatorSource is the credential source of the operator's own pull secrets, the only one the operator resolves tags with today.
Variables ¶
var DefaultRegistries = []string{"docker.io", "ghcr.io", "quay.io", "registry.k8s.io"}
DefaultRegistries are the upstream registries the chart configures by default.
var ErrNotCacheable = errors.New("the image cannot be pulled through the image cache")
ErrNotCacheable is what RewriteRequired wraps when the cache is on and the image cannot go through it: a reference that does not parse, or a registry the cache does not serve. The cache being on, such an image is refused, never pulled directly: a direct pull would bypass the gate, the credentials rule and the allowed list at once.
Functions ¶
func CredentialSource ¶ added in v1.10.0
CredentialSource is the name of the source for the credentials with this fingerprint (a hash of the pull documents, never the secret). Labs with the same credentials share it, and an answer got with one set of credentials is never given to another: the key of an entry carries the source.
func PlatformScope ¶ added in v1.10.0
PlatformScope is the scope of a resolution for the platforms of the nodes: the one platform when there is exactly one (the image is pinned to the manifest of that platform), "" otherwise (the index digest is pinned). It matches what RegistryResolver does.
func ProbeDigest ¶ added in v1.2.0
func ProbeDigest(ctx context.Context, ref string, kc authn.Keychain, rt http.RoundTripper) (string, error)
ProbeDigest asks the registry (never a cache) for the manifest of ref, a tag or a digest, with the keychain over transport (nil: the default), and returns the digest it answers with. Unlike RegistryResolver.Resolve it contacts the registry for a digest reference too.
func SafeError ¶ added in v1.10.0
SafeError is the text of err that may be written to the status of a Lab or a group, to an event or to a log: a PinError as it is (or one made from a transport error), and for any other error its text on one line, without control characters, at most 200 characters. Use it wherever an error that may come from an upstream registry is shown.
func StripPlatformHeaders ¶ added in v1.10.0
StripPlatformHeaders removes every X-Cib-* header: nothing a client sends may be taken for what the platform says.
Types ¶
type AsyncResolver ¶ added in v1.10.0
type AsyncResolver struct {
// Sources maps a credential source to the resolver that resolves with its credentials. Read only once the resolver is in use;
// EnsureSource adds the sources of a lab's own credentials.
Sources map[string]Resolver
// Scope returns the platform scope of a resolution (see PlatformScope); nil: none. It is part of the key.
Scope func(ctx context.Context) string
// Concurrency bounds the resolutions in flight; 8 when not positive.
Concurrency int
// SuccessTTL is how long an answer is kept; 30 minutes when not positive.
SuccessTTL time.Duration
// FailureTTL is the longest a failure is kept for those who wait for it; 2 minutes when not positive.
FailureTTL time.Duration
// Notify is called, once per waiter, when a resolution the waiter asked for is done (outside every lock). Nil: nobody is told.
Notify func(waiter string)
// Now is the clock; nil means time.Now.
Now func() time.Time
// contains filtered or unexported fields
}
AsyncResolver resolves tags to digests in the background, so a reconcile never waits for a registry. A caller asks with Lookup: the answer is there, or it is Pending and the caller is told through Notify when it arrives (and looks again).
- One resolution per (credential source, platform scope, reference) runs at a time, however many labs ask for it.
- At most Concurrency resolutions run at once.
- The key carries the credential source, so the answer got with one set of credentials is never given to a caller that would use another: each source has its own Resolver and its own entries.
- A success is kept for SuccessTTL (labs of one wave run one image even when the tag moves); a failure is kept until every caller that waited for it has seen it (Done) and at most FailureTTL, then the next Lookup tries again.
func (*AsyncResolver) Done ¶ added in v1.10.0
func (a *AsyncResolver) Done(ctx context.Context, source, waiter string, refs ...string)
Done says that waiter has used the answers for refs: a failure nobody waits for any more is forgotten, so the next lab that asks tries again at once.
func (*AsyncResolver) EnsureSource ¶ added in v1.10.0
func (a *AsyncResolver) EnsureSource(name string, make func() Resolver)
EnsureSource makes the source name known: make is called once, the first time, and builds the resolver that resolves with that credential set only. It is kept in memory (it holds the credentials) while it is asked for, and dropped when unused for twice the success time or when too many are held. The built-in sources are never replaced.
func (*AsyncResolver) Lookup ¶ added in v1.10.0
func (a *AsyncResolver) Lookup(ctx context.Context, source, ref, waiter string) Lookup
Lookup asks for the digest of ref resolved with the credentials of source. waiter names who is asking (it is told through Notify when a pending answer arrives). A reference that already is a digest is answered at once.
func (*AsyncResolver) Pending ¶ added in v1.10.0
func (a *AsyncResolver) Pending() int
Pending is the number of resolutions that are running or queued.
type DockerConfigKeychain ¶
type DockerConfigKeychain struct {
// contains filtered or unexported fields
}
DockerConfigKeychain is a keychain over dockerconfigjson documents (the data of kubernetes.io/dockerconfigjson pull Secrets).
func NewDockerConfigKeychain ¶
func NewDockerConfigKeychain(docs ...[]byte) (*DockerConfigKeychain, error)
NewDockerConfigKeychain parses dockerconfigjson documents; a malformed one is an error.
func (*DockerConfigKeychain) Has ¶ added in v1.2.0
func (k *DockerConfigKeychain) Has(host string) bool
Has says whether the keychain holds a login for the registry host, with the same normalization Resolve uses (docker.io, index.docker.io and URL forms are one registry).
func (*DockerConfigKeychain) Resolve ¶
func (k *DockerConfigKeychain) Resolve(res authn.Resource) (authn.Authenticator, error)
Resolve implements authn.Keychain.
type Lookup ¶ added in v1.10.0
type Lookup struct {
// Pending: the resolution is running (or queued); Digest and Err are empty.
Pending bool
Digest string
Err error
}
Lookup is the answer of AsyncResolver.Lookup.
type PinError ¶ added in v1.10.0
PinError is why a registry could not tell the digest of an image: the registry of the reference, the HTTP status it answered (0: none) and a reason in OUR words. It never carries text of the registry's answer (the body of a response, a message of a diagnostic, an address it redirected to): an upstream is not trusted with what an operator or a user reads in a Lab's status, in an event or in a log.
func NewPinError ¶ added in v1.10.0
NewPinError describes err, an error of a request to registry, in words of our own.
type RegistryResolver ¶
type RegistryResolver struct {
Keychain authn.Keychain
TTL time.Duration
// Timeout bounds one request; 15s when zero.
Timeout time.Duration
// Now is the clock; nil means time.Now.
Now func() time.Time
// Platforms lists the platforms of the nodes the images run on. With exactly
// one, a multi-platform image is pinned to the manifest of that platform, so the
// cache fetches that one image and not every architecture of the index. With
// several (or none known) the index digest is pinned.
Platforms func(ctx context.Context) []v1.Platform
// PlainHTTP lists registries (as they are named in an image reference: "registry.e2e.test", "host:5000") that are asked over plain
// http, as the registry gate asks the ones the chart lists with an http:// url (a plain registry inside the cluster). Any other is
// asked over https only.
PlainHTTP []string
// Transport carries every request to a registry. Nil: the default transport with the guard of netguard, which refuses a redirect
// to an internal address (see defaultTransport). A first request goes where the reference says, whatever the address.
Transport http.RoundTripper
// contains filtered or unexported fields
}
RegistryResolver resolves tags with a HEAD request to the upstream registry (never through the cache, which would answer with what it has stored). Answers are remembered for TTL, so labs created in the same wave, and the prepull, see one digest even when the upstream tag moves meanwhile.
type Rewriter ¶
type Rewriter struct {
// Prefix is host:port of the cache as the nodes see it, e.g. "localhost:5035".
Prefix string
// Registries are the upstream registries the cache serves (docker.io, ghcr.io, ...).
Registries []string
}
Rewriter rewrites image references for one cache.
func (Rewriter) Enabled ¶ added in v1.10.0
Enabled says whether images are pulled through a cache at all.
func (Rewriter) For ¶ added in v1.10.0
For binds the rewriter to a tenant: the segment is names.TenantSegment(tenant) (the name itself when it is safe, a hashed form otherwise).
func (Rewriter) ForPlatform ¶ added in v1.10.0
ForPlatform binds the rewriter to the platform's own components (the VPN and gateway pods of every group): the segment is names.SegmentPlatform.
func (Rewriter) RepoOf ¶
RepoOf returns the internal repository path of a reference that points at the cache (prefix stripped, tag or digest removed), "" if it does not. The tenant segment of a cache path is not part of it: <prefix>/c/<segment>/<registry>/<repo> gives c/<registry>/<repo>, the path of the shared tree, which is also what the snapshot paths (lab/...) keep as they are.
func (Rewriter) RewriteFor ¶ added in v1.10.0
RewriteFor is Rewrite for the pods of a tenant.
type Scoped ¶ added in v1.10.0
type Scoped struct {
// contains filtered or unexported fields
}
Scoped is a Rewriter bound to one tenant segment: every reference it writes carries that segment.
func (Scoped) Rewrite ¶ added in v1.10.0
Rewrite returns the reference to pull through the cache, <prefix>/c/<segment>/<registry>/<repo>. It returns ref unchanged when the rewriter is empty, the reference is malformed, it already points at the cache, or its registry is not one the cache serves (a private registry, or a snapshot image that is already local).
A reference without a registry is docker.io; one without a repository namespace gets "library/"; one without a tag gets ":latest" (Kubernetes treats the two alike). A reference with a digest keeps only the digest.
func (Scoped) RewritePinned ¶ added in v1.10.0
RewritePinned is Rewrite for a reference pinned to digest: the tag is replaced by the digest, so every pod of a lab pulls the very same image. A reference the cache does not serve is returned unchanged.
func (Scoped) RewriteRequired ¶ added in v1.10.0
RewriteRequired is Rewrite for a cache that is on: the reference to pull through the cache, or an error wrapping ErrNotCacheable when the reference is malformed or its registry is not served. A reference that already points at the cache (a snapshot) passes unchanged. With the cache off (no prefix) the reference is returned as is.
type TokenFile ¶ added in v1.10.0
type TokenFile struct {
Path string
// Now is the clock; nil means time.Now.
Now func() time.Time
// contains filtered or unexported fields
}
TokenFile is a projected token read from a file and read again every TokenReread.
func NewTokenFile ¶ added in v1.10.0
NewTokenFile returns a TokenFile of path.
func (*TokenFile) RoundTripper ¶ added in v1.10.0
func (t *TokenFile) RoundTripper(next http.RoundTripper) http.RoundTripper
RoundTripper gives every request the token of t (and no other X-Cib-* header) before it goes to next (nil: http.DefaultTransport).
func (*TokenFile) SetNodeToken ¶ added in v1.10.0
SetNodeToken makes h carry the token of t as its only X-Cib-* header.