artifactkit

package module
v0.24.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 20, 2026 License: MIT Imports: 26 Imported by: 29

Documentation

Index

Constants

View Source
const DefaultIndexTTL = 60 * 60 * time.Second

DefaultIndexTTL is how long upstream index responses are served before a re-fetch.

View Source
const DefaultMaxBody = 8 << 30

DefaultMaxBody is the default request body limit (8 GiB: large OCI layers legitimately reach multi-GB).

View Source
const DefaultNamespace = "default"

DefaultNamespace is the namespace of everything that carries no explicit namespace of its own.

View Source
const DefaultNegativeTTL = 60 * time.Second

DefaultNegativeTTL is how long a 404/410 is remembered, so a scanner cannot hammer the upstream for a missing object.

View Source
const DefaultPathTTL = 5 * time.Minute

DefaultPathTTL is how long a mutable path-tree entry is trusted before a conditional revalidation. It only matters when the origin returns a validator (ETag/Last-Modified): without one, cached bytes are served indefinitely rather than re-downloaded (a large index cannot be cheaply revalidated).

View Source
const DefaultURICacheTTL = 5 * time.Minute

DefaultURICacheTTL is how long a mutable (non-immutable) URI is served from cache before a revalidation request is issued.

View Source
const ExplicitRepoMarker = "-"

ExplicitRepoMarker introduces an explicit repository in a request path:

/artifacts/npm/-/@acme/ui        -> repo @acme, path ui
/artifacts/pypi/-/internal/simple -> repo internal, path simple

"-" is safe as a marker: npm reserves a bare "-/" itself, OCI repository names cannot be a lone "-", and no protocol has a real path segment that is exactly "-". Requests without the marker keep their native shape.

View Source
const ImmutableTag = "immutable"

ImmutableTag marks an entry whose key looks up content that can never change (a content-addressed digest). Immutable entries never expire.

View Source
const MountBase = targets.MountBase

MountBase is the HTTP prefix every non-OCI protocol is mounted under. It is re-exported from the targets package so callers of artifactkit need not import both.

View Source
const NegativeMediaType = "application/x-netcache-miss"

NegativeMediaType marks a remembered netcache negative (404/410) entry: it holds no blob and exists only to absorb repeated misses until it expires. It is stored in the MediaType column and is the one media type that is read BEFORE the digest/blob emptiness checks, because a negative entry has neither.

View Source
const SHA256 = "sha256"

Digest is the canonical content-addressed identifier of a blob. The OCI form is "sha256:<hex>". The BlobStore is keyed by digest strings, so stems of implementations are free to use any algorithm.

View Source
const UserAgent = "artifactkit/1.0 (pull-through mirror)"

UserAgent is sent on every upstream request. Registries rate-limit generic user agents (Maven Central 429s the default Go UA), so a stable bespoke one is used.

Variables

View Source
var (
	ErrArtifactUnknown = errors.New("artifact unknown")
	ErrBlobUnknown     = errors.New("blob unknown")
	ErrUploadUnknown   = errors.New("upload unknown")
)

Sentinels for mapping store errors onto protocol responses.

Functions

func AuthorizeAdmin added in v0.21.0

func AuthorizeAdmin(w http.ResponseWriter, r *http.Request, auth Auth, realm string) bool

AuthorizeAdmin gates the operator/admin surface (the /artifacts/system subtree: upstream/target/proxy config, package inventory, footprint). It requires a WRITE-level credential — the operator level in this codebase — on every method, including reads, because the response body is configuration, not package bytes. A realm names the challenge so a browser/proxy can prompt. auth == nil (dev mode) permits access, exactly like writes.

func AuthorizePublish added in v0.10.0

func AuthorizePublish(ctx context.Context, o Ownership, format, namespace, name string, tenantID int64) error

AuthorizePublish resolves the most specific ownership check available: NamespacedOwnership when the layer implements it, else the flat form over the canonical repository key.

func AuthorizeReadFor added in v0.7.0

func AuthorizeReadFor(w http.ResponseWriter, r *http.Request, auth Auth, reg *Registry, format, repository string) bool

AuthorizeReadFor gates a READ of one registry name on the ownership layer: public/unclaimed names are readable by everyone; a private name requires a role on its scope (mapped repo members, else the owning user). When the registry carries no Ownership layer the read is allowed. It writes a 404 on denial (not 403: a hidden name must not reveal its existence) and returns false so the caller stops.

Like the write path, the check is made against the request's RepoScope when one is present, so a private namespace is hidden in one step.

func AuthorizeReadScoped added in v0.10.0

func AuthorizeReadScoped(w http.ResponseWriter, r *http.Request, auth Auth, reg *Registry, format string) bool

AuthorizeReadScoped is the read-side counterpart of AuthorizeWriteScoped.

func AuthorizeWrite

func AuthorizeWrite(w http.ResponseWriter, r *http.Request, auth Auth) bool

AuthorizeWrite gates a write operation on an optional Auth, requiring a WRITE-level credential (a read-level token is not enough to publish). On success it returns true; on failure it writes a 401/403 challenge and returns false (the caller should stop). auth == nil permits the write (an open/dev instance).

func AuthorizeWriteFor added in v0.4.0

func AuthorizeWriteFor(w http.ResponseWriter, r *http.Request, auth Auth, reg *Registry, format, repository string) bool

AuthorizeWriteFor is the tenancy-aware publish gate used by protocol adapters that know their native package name (npm name, OCI repository, generic repo). It performs the write-credential check of AuthorizeWrite and then, when the registry carries an Ownership layer, resolves the caller's tenant and enforces publish rights for (format, repository): unclaimed names are claimed by the caller's tenant, claimed names must match it. The error text is protocol-neutral JSON, like AuthorizeWrite.

When the request carries a RepoScope (the /artifacts gate installs one for every format), the ownership check is made against the (namespace, name) pair, so a namespace — npm scope, OCI host, maven groupId — is a boundary in its own right. The repository argument is qualified with the scope for the flat fallback path.

func AuthorizeWriteScoped added in v0.10.0

func AuthorizeWriteScoped(w http.ResponseWriter, r *http.Request, auth Auth, reg *Registry, format string) bool

AuthorizeWrite gives the universal publish gate for adapters that do not track package names: it is AuthorizeWriteFor driven entirely by the request's RepoScope. The /artifacts middleware resolves (namespace, name) for every format, so an adapter can call this one function and get full namespace-aware ownership without knowing its own naming rules.

func BoundaryFromContentType

func BoundaryFromContentType(ct string) string

BoundaryFromContentType extracts the boundary parameter of a multipart Content-Type, or "" when absent.

func Build

func Build(name string, reg *Registry, cfg map[string]any) (http.Handler, error)

Build constructs a protocol handler from its registered constructor.

func CacheKey added in v0.14.0

func CacheKey(rawURL string) string

CacheKey normalizes a URL into the cache key: scheme and host lowercased, default ports dropped, query retained but credential-bearing parameters stripped and the rest sorted (so parameter order does not fragment the cache). The result is stable across the presigned/unsigned forms of the same object.

func CanRead added in v0.10.0

func CanRead(ctx context.Context, o Ownership, format, namespace, name string, tenantID int64) bool

CanRead resolves the most specific read check available.

func CanonicalHost added in v0.11.0

func CanonicalHost(host string) string

CanonicalHost lower-cases a host and folds known registry aliases.

func DigestOf

func DigestOf(b []byte) string

DigestOf returns "sha256:<hex>" for bytes.

func ErrProtocolUnknown

func ErrProtocolUnknown(name string) error

ErrProtocolUnknown is returned when cmd-artifact asks for a protocol that was not registered (i.e. no adapter package was blank-imported).

func Error

func Error(w http.ResponseWriter, status int, msg string)

Error writes an {"ok":false,"error":msg} JSON error response.

func ExtractField

func ExtractField(body []byte, contentType, field string) ([]byte, bool)

ExtractField returns the body of the first part whose Content-Disposition names `field` (trailing CRLF trimmed).

func ExtractFirstFile

func ExtractFirstFile(body []byte, contentType string) (string, []byte, bool)

ExtractFirstFile returns (filename, body) of the first file part.

func ExtractTextField

func ExtractTextField(body []byte, contentType, field string) (string, bool)

ExtractTextField extracts a simple (non-file) form field, whitespace-trimmed.

func FirstSegNamespace added in v0.10.0

func FirstSegNamespace(_, path string) (namespace, name, rest string)

FirstSegNamespace resolves a protocol whose first path segment is already the repository (apk/debian/rpm/conda and the hosted repo model): the first segment is the namespace, the whole remainder is the name (these protocols treat the artifact path as one opaque object inside the repo).

func FlatNamespace added in v0.10.0

func FlatNamespace(_, path string) (namespace, name, rest string)

FlatNamespace is the default resolver: the first path segment is the name and the rest is the in-package remainder. It matches the historical behavior of the flat formats (pypi, rubygems, composer, ...).

func Fresh added in v0.18.0

func Fresh(art Artifact, now time.Time) bool

Fresh reports whether a cached entry may be served without revalidation. It is the exported form of isFresh for adapters (OCI) that hold their own cache rows in the shared IndexStore.

func GoModuleNamespace added in v0.10.0

func GoModuleNamespace(_, path string) (namespace, name, rest string)

GoModuleNamespace resolves a Go module path into (namespace, module): the namespace is the hosting prefix (host + owner for the common two-segment case) and the name is the module path itself, so an upstream override keyed "go/github.com/acme" covers every module under that owner.

github.com/acme/tool/sub -> ("github.com/acme", "github.com/acme/tool/sub")
golang.org/x/text        -> ("golang.org/x", "golang.org/x/text")

func HighestVersion

func HighestVersion(versions []string) string

HighestVersion returns the highest semantic version among versions, or "".

func IsBodyTooLarge

func IsBodyTooLarge(err error) bool

IsBodyTooLarge reports whether err is a request-body size violation.

func IsImmutableURL added in v0.14.0

func IsImmutableURL(rawURL string) bool

IsImmutableURL reports whether a URL denotes content that can never change, so it is cached permanently: a digest/hash appears in the path or query, or the filename carries a version-like immutable marker.

func IsPublicHostname added in v0.11.0

func IsPublicHostname(host string) bool

IsPublicHostname reports whether host looks like a public DNS name (dotted, not an IP literal, not a cluster-local name, not localhost). The git and ivy mirrors use it to tell a real upstream server (github.com) from the gateway they are themselves running behind, since their upstream identity may come from the request origin rather than the path. It delegates to the shared targets guard so the egress catch-all and the adapters agree.

func IsRegistryHost added in v0.11.0

func IsRegistryHost(host string) bool

IsRegistryHost reports whether a Host header names an upstream container registry rather than the gateway itself. The gateway is reached by an in-cluster name ("easylab", "easylab.ns.svc.cluster.local") or a bare address; a public registry is a dotted hostname. The docker insecure-registry convention additionally makes "localhost:port" and "IP:port" registries, which is what the test/local-registry paths use.

func IsUnknown

func IsUnknown(err error) bool

IsUnknown reports whether a store error is a plain "not found".

func JSON

func JSON(w http.ResponseWriter, status int, v any)

JSON writes an application/json response with the given body value.

func LimitBody

func LimitBody(w http.ResponseWriter, r *http.Request)

LimitBody bounds a request body (http.MaxBytesReader). Adapters must call this on every upload entrypoint before reading r.Body; an over-limit read fails with *http.MaxBytesError which LimitBodyError maps to 413.

func LogMetaErr

func LogMetaErr(op string, err error)

LogMetaErr is used by adapter helper flows (storeVersion / removeVersion) where an index write failure must not abort a proxied download but must be visible in the server log. Returns the error for callers that CAN act.

func MavenNamespace added in v0.10.0

func MavenNamespace(_, path string) (namespace, name, rest string)

MavenNamespace resolves a maven path into (groupId, artifactId). Maven's layout is <group>/<artifact>/<version>/<file> with a variable-length group, plus <group>/<artifact>/maven-metadata.xml for enumeration, so the name and group are found from the right: strip the metadata/file tail, then the version, leaving group + artifact.

org/slf4j/slf4j-api/2.0.9/slf4j-api-2.0.9.pom -> ("org.slf4j", "slf4j-api")
org/slf4j/slf4j-api/maven-metadata.xml        -> ("org.slf4j", "slf4j-api")
com/google/guava/guava/33.0.0/guava-33.0.0.jar -> ("com.google.guava", "guava")

func MirrorOrigin added in v0.13.1

func MirrorOrigin(ctx context.Context) string

MirrorOrigin returns the scheme://host[/prefix] a request arrived on when it was routed through a NAMED target (a mirror/alias such as npm.jsr.io), else "". An adapter uses it to emit self-URLs in the shape the client dialed, so an alias that shares an adapter with its base protocol (JSR's npm-compatibility registry riding the npm adapter) advertises its own origin instead of the base protocol's. The default target returns "": its origin is the protocol's public home, and emitting that would bypass the gateway.

func OCIHostNamespace added in v0.10.0

func OCIHostNamespace(_, path string) (namespace, name, rest string)

OCIHostNamespace resolves an OCI repository reference into (registry-host, repository). A leading component is treated as a host when it contains a dot or colon or is "localhost" (the registry convention); otherwise there is no host and the repository is the whole name.

ghcr.io/acme/app          -> ("ghcr.io", "acme/app")
localhost:5000/acme/app   -> ("localhost:5000", "acme/app")
acme/app                  -> ("", "acme/app")

func OctetResponse

func OctetResponse(w http.ResponseWriter, r *http.Request, data []byte)

OctetResponse writes a small known-in-memory body as application/octet-stream. It honors HEAD (no body) and sets Content-Length. For large/streamed content use ServeBlob.

func ParseDigest

func ParseDigest(d string) (string, error)

ParseDigest validates an "algo:hex" digest and returns the hex portion. sha256/sha512 only (the algorithms registries actually use on the wire).

func ProtocolRegistered added in v0.13.0

func ProtocolRegistered(name string) bool

ProtocolRegistered reports whether a protocol name is registered.

func Register

func Register(name string, build func(reg *Registry, cfg map[string]any) (http.Handler, error))

Register adds a protocol constructor. It panics on protocol-name collisions at start-up time, which is a programmer error we want surfaced eagerly.

func RegisterNamespace added in v0.10.0

func RegisterNamespace(format string, fn NamespaceResolver)

RegisterNamespace installs a format's resolver. Called from adapter init() next to Register. Adapters that do not register get FlatNamespace.

func Registered

func Registered() []string

Registered returns the names of all registered protocols.

func RegistryHostFrom added in v0.11.0

func RegistryHostFrom(ctx context.Context) string

RegistryHostFrom returns the registry host for a request: the explicit value from WithRegistryHost when set, else the Host the scope middleware recorded (OCI's /v2 mount). "" means "not a registry host", i.e. the default registry.

func RepoKey added in v0.10.0

func RepoKey(namespace, name string) string

RepoKey encodes (namespace, name) into the canonical repository string. The default namespace is elided so unscoped names keep their historical key (and existing metadata rows keep resolving). It is the primitive behind RepoScope.ScopedKey, which adds the isolated-target prefix on top.

func ResolveNamespace added in v0.10.0

func ResolveNamespace(format, path string) (namespace, name, rest string)

ResolveNamespace applies the format's resolver, defaulting to FlatNamespace.

func ScopeMiddleware added in v0.10.0

func ScopeMiddleware(base string, next http.Handler) http.Handler

ScopeMiddleware resolves a request's RepoScope and stashes it in the request context. It runs for EVERY format, so no adapter needs to know about namespaces: the scoping IndexStore decorator applies the namespace when the adapter reads or writes metadata, and Registry.Fetch consults it to choose the upstream.

Two addressing forms are understood:

/artifacts/<fmt>/<native-path>            native (namespace from the resolver)
/artifacts/<fmt>/-/<repo>/<native-path>   explicit repository

The explicit form is rewritten to the native path with the repository in the context, so adapters only ever see their normal URL shape.

base is the mount prefix "/artifacts"; OCI's /v2 uses ScopeMiddlewareForFormat.

func ScopeMiddlewareForFormat added in v0.10.0

func ScopeMiddlewareForFormat(format, base string, next http.Handler) http.Handler

ScopeMiddlewareForFormat is ScopeMiddleware for a mount whose format is fixed by the route rather than present in the path (OCI's /v2): the whole path inside base is the native path, and only the explicit "/-/<repo>/" marker (if any) names a repository.

func ScopeNamespace added in v0.10.0

func ScopeNamespace(_, path string) (namespace, name, rest string)

ScopeNamespace resolves npm-style scoped names: a leading "@scope" is the namespace and the rest is the package name (which may itself contain a slash, e.g. "@scope/pkg/sub" for a tarball path).

func SelfURL added in v0.17.0

func SelfURL(selfBase, format string) string

SelfURL returns the external base URL an adapter should build self-referential links on: the configured self_base when non-empty, else a localhost fallback carrying the protocol mount. It replaces the per-adapter `base()` copies.

func ServeBlob added in v0.7.0

func ServeBlob(w http.ResponseWriter, r *http.Request, rd io.ReadSeeker, ct string, modTime time.Time)

ServeBlob streams a seekable blob to the client with full HTTP semantics: Range/206, If-Range, HEAD (no body), Content-Length, and the given content type. It never buffers the whole blob in memory. When ct is empty it resolves by filename via mime by extension, falling back to application/octet-stream.

ServeBlob is the single response path for artifact bytes: OCI layers, apt pool files, apk archives, conda packages, nix NARs, and LFS objects.

func ServeBlobAt added in v0.7.0

func ServeBlobAt(w http.ResponseWriter, r *http.Request, store BlobStore, ctx context.Context, digest, ct string) bool

ServeBlobAt serves a blob opened from a BlobStore by digest: (nil, nil) from Open means the blob is absent -> 404.

func ServeBlobAtNamed added in v0.9.0

func ServeBlobAtNamed(w http.ResponseWriter, r *http.Request, store BlobStore, ctx context.Context, digest, ct, filename string) bool

ServeBlobAtNamed is ServeBlobAt with an optional download filename: when non-empty the response carries Content-Disposition: attachment (package managers that save the file rely on it). It streams with full Range/206/HEAD semantics and never buffers the blob.

func ServeCachedBlob added in v0.17.0

func ServeCachedBlob(w http.ResponseWriter, r *http.Request, store BlobStore, ctx context.Context, art Artifact, ct, filename string) bool

ServeCachedBlob serves a cached artifact's blob with full HTTP semantics and replays the response metadata captured at fetch time. It is the single serve path for the caching adapters (netcache, pathcache trees) so a stored Content-Encoding / ETag / Last-Modified reaches the client verbatim — without Content-Encoding replay a transparently-gzipped body would be handed to the client unlabeled and fail to decode.

ct overrides the artifact's media type when non-empty (an adapter may know a better type from the path). filename, when non-empty, adds a Content- Disposition. It returns false when the blob is absent (caller decides 404).

func ServeData added in v0.9.0

func ServeData(w http.ResponseWriter, r *http.Request, reg *Registry, ctx context.Context, data []byte, ct, filename string)

ServeData stores data into the CAS (idempotent) and streams it to the client with Range/HEAD semantics and an attachment filename. It is the streaming replacement for BlobResponse on pull-through paths where the bytes were just fetched into memory: storing first means the response is served from the blob store rather than re-buffered.

func SetURICacheTTL added in v0.14.0

func SetURICacheTTL(ttl time.Duration)

SetURICacheTTL overrides the shared URI cache's mutable-entry TTL. It exists for tests and operator tuning; a non-positive value restores the default.

func SortSemver

func SortSemver(versions []string)

SortSemver sorts version strings in ascending semantic order. The sort is stable, so equal semantic keys keep their input order.

func SplitExplicitRepo added in v0.10.0

func SplitExplicitRepo(path string) (repo, rest string, explicit bool)

SplitExplicitRepo strips an explicit "/-/<repo>/" prefix from a format-relative path. It reports whether the marker was present; when it is absent the path is returned unchanged (native addressing).

func SplitRepo added in v0.10.0

func SplitRepo(p string) (repo, rest string, ok bool)

SplitRepo splits a format-relative path into a repository key (its first segment) and the remaining path inside it. rest may be empty (repo root).

func TargetAuthHeader added in v0.14.0

func TargetAuthHeader(a targets.Auth, clientAuth string) (string, error)

TargetAuthHeader resolves a target's auth policy into an Authorization header, reusing clientAuth for passthrough. It is exported so the netcache protocol (which fetches a bare URL rather than a Remote) can apply the same policy. "" means anonymous.

func TenantOfRequest added in v0.4.0

func TenantOfRequest(ctx context.Context, auth Auth, r *http.Request) int64

TenantOfRequest resolves the caller's tenant through the Auth (StoreAuth bridges TenantTokenStore); anonymous or unresolvable callers map to the default tenant 1.

func Text

func Text(w http.ResponseWriter, status int, body, contentType string)

Text writes a response with an explicit content type.

func URLencode

func URLencode(s string) string

URLencode percent-encodes a path segment (RFC 3986 unreserved set plus ~ stay literal, everything else is %XX uppercase).

func WithRegistryHost added in v0.11.0

func WithRegistryHost(ctx context.Context, host string) context.Context

WithRegistryHost stores the registry host implied by a request's Host header. OCI uses it as the repository namespace and the upstream selector.

func WithRepoScope added in v0.10.0

func WithRepoScope(ctx context.Context, s RepoScope) context.Context

WithRepoScope returns a context carrying the request's repository scope.

func WriteCapable added in v0.2.1

func WriteCapable(ctx context.Context, a Auth, r *http.Request) bool

WriteCapable reports whether the request carries a write-level credential in ANY of the protocol credential schemes. It is the scheme-agnostic privilege check behind AuthorizeWrite (default implementation).

func WriteReadErr

func WriteReadErr(w http.ResponseWriter, err error)

WriteReadErr maps a request-body read failure to a status. A body over the MaxBytesReader cap is 413; anything else is a 400.

Types

type Action

type Action string

Action is the coarse permission an adapter requests from upstream.

const (
	ActionPull   Action = "pull"
	ActionPush   Action = "push"
	ActionDelete Action = "delete"
)

type Artifact

type Artifact struct {
	Format      string
	Repository  string
	Version     string
	MediaType   string
	Proprietary []byte
	Digest      string
	Blobs       []Descriptor
	Source      string
	// Target is the target id the content was fetched through ("maven",
	// "maven.google", "ghcr.io"). It is provenance only: the primary key stays
	// (Format, Repository, Version), so mirrors of one protocol still share
	// and dedupe by digest. Empty when the adapter did not record one.
	Target string
	// HTTP response metadata, recorded for byte-exact replay and revalidation
	// of cached URIs (the netcache protocol). None of these are key material.
	ETag            string
	LastModified    string
	ContentEncoding string
	// CacheControl is "immutable" for content whose URL denotes a fixed
	// revision (a digest/hash in the URI), which is never revalidated.
	CacheControl string
	// ExpiresAt, when non-zero, is when a non-immutable entry must be
	// revalidated before being served again.
	ExpiresAt time.Time
}

type Auth

type Auth interface {
	// Authenticate extracts a username from the request, or "" when
	// anonymous. Implementations must understand the common Authorization
	// prefixes plus protocol-specific header forms.
	Authenticate(ctx context.Context, r *http.Request) string

	// CheckBearer validates an OCI bearer token against a scope.
	// wantedScope is like "repository:<name>:<action>".
	CheckBearer(ctx context.Context, token, wantedScope string) (string, bool)

	// CheckToken validates a bare/raw token (cargo, npm publish, ...).
	CheckToken(ctx context.Context, token string) (string, bool)

	// CheckBasic validates Basic credentials.
	CheckBasic(ctx context.Context, user, pass string) bool

	// IssueToken mints an OCI bearer token (returned to clients verbatim).
	IssueToken(ctx context.Context, username string, scopes []string, ttl time.Duration) string
}

Auth is the protocol-agnostic credential interface. The reference implementation maps a static `token=level` table (read/write) onto the per-protocol credential models (OCI bearer, cargo raw token, nuget api-key, basic auth, ...).

type BlobInfo added in v0.24.0

type BlobInfo struct {
	Digest  string
	Size    int64
	ModTime time.Time
}

BlobInfo is the metadata of one stored blob. ModTime lets the reaper skip blobs that may still be mid-write, so no separate ager capability is needed.

type BlobStore

type BlobStore interface {
	// Stat returns the blob's metadata, or (nil, nil) when absent.
	Stat(ctx context.Context, digest string) (*BlobInfo, error)
	// Open returns a reader (supporting Seek when possible), or (nil, nil).
	Open(ctx context.Context, digest string) (io.ReadSeekCloser, error)
	// Put streams r into the CAS, computing every hash in one pass. When
	// wantDigest is non-empty the content MUST hash to it (else an error);
	// when empty the computed digest is used. stored is false when the blob
	// already existed (dedup). The returned Stored carries the hashes, so a
	// caller never hashes the bytes again.
	Put(ctx context.Context, r io.Reader, wantDigest string) (Stored, bool, error)
	// Hashes returns the hash set recorded for a blob at write time. ok=false
	// when the store has no record (a blob written by an older tool).
	Hashes(ctx context.Context, digest string) (Hashes, bool, error)
	// Delete removes the blob.
	Delete(ctx context.Context, digest string) error
	// List returns every blob's metadata (digest, size, mtime).
	List(ctx context.Context) ([]BlobInfo, error)
}

BlobStore abstracts the CAS for immutable artifact content. The default implementation is a filesystem store (one file per digest); an S3-compatible store is provided by the s3blob module.

The store owns INTEGRITY: Put streams the body once, computes the full hash set in that pass, verifies it against the caller's expected digest, and persists the hashes. Callers therefore never hash a body themselves and never re-read a blob to learn its hashes (Hashes), which is what makes publish and checksum sidecars cheap.

type ClientFactory

type ClientFactory struct {
	// contains filtered or unexported fields
}

ClientFactory builds (and caches) an http.Client for a proxy policy:

  • nil -> follow the environment proxy configuration
  • Some("") -> direct connection (env proxy bypassed)
  • Some(url) -> always route through the given proxy

One factory is shared process-wide (sharedFactory) so every upstream fetch reuses the same transports and keep-alive pool; building a factory per fetch would open a fresh TCP+TLS connection every time. It is safe for concurrent use.

func NewClientFactory

func NewClientFactory() *ClientFactory

NewClientFactory returns an empty factory. Callers that fetch repeatedly should share one (see sharedFactory); this constructor is for tests and isolated embedders.

func SharedClientFactory added in v0.17.0

func SharedClientFactory() *ClientFactory

SharedClientFactory returns the process-wide client factory.

func (*ClientFactory) Client

func (f *ClientFactory) Client(proxy *string) *http.Client

Client returns a cached client for the proxy policy that does NOT follow redirects. Adapters use it when a 3xx must be surfaced to the caller (e.g. HuggingFace resolve → CDN, or a Debian mirror redirect) so the client decides: transparent pass-through, or an explicit re-fetch. Silently following such a redirect would download the whole object server-side and defeat the range/caching policy.

func (*ClientFactory) Redirecting added in v0.9.0

func (f *ClientFactory) Redirecting(proxy *string) *http.Client

Redirecting returns a cached client for the proxy policy that FOLLOWS redirects (Go's default policy: up to 10 hops). It is used by registries whose blob endpoint 307-redirects to a CDN (Docker Hub → CloudFront, MCR → Azure Blob, GHCR/Quay): the adapter must read the object through the redirect to verify and cache it server-side.

Cross-host redirects drop the Authorization header automatically (Go's shouldCopyHeaderOnRedirect), which is exactly what a presigned CDN URL needs; same-host redirects keep it.

type Descriptor

type Descriptor struct {
	Digest    string
	MediaType string
	Size      int64
	Name      string
}

func (Descriptor) Hex

func (d Descriptor) Hex() string

func (Descriptor) IsEmpty

func (d Descriptor) IsEmpty() bool

type Fetched

type Fetched struct {
	Data   []byte
	Hashes Hashes
	Size   int64
	// Digest is the CAS digest of the fetched bytes ("" on a legacy path that
	// did not store them).
	Digest string
}

Fetched is the result of a pull-through fetch: full upstream bytes (cached or fresh) plus hashes.

type GeneratedFile added in v0.8.0

type GeneratedFile struct {
	Body        []byte
	ContentType string
}

GeneratedFile is one index document produced by a Generator.

type Generator added in v0.8.0

type Generator func(ctx context.Context, files []HostedFile) (map[string]GeneratedFile, error)

Generator renders the repo's index document(s) from its file list: a map from hosted path name to file. Multi-file indexes (rpm's repomd.xml plus the primary metadata it references) return several entries. Regeneration is deterministic from `files`, so no caching layer is needed. ctx carries the request's cancellation/deadline into any blob reads the generator performs.

type Hashes

type Hashes struct {
	SHA256 string
	SHA1   string
	MD5    string
	SHA512 string
}

func ComputeHashes

func ComputeHashes(r io.Reader) (Hashes, error)

ComputeHashes streams data once, threading it through md5, sha1, sha256 and sha512 simultaneously. It returns the raw hex digests.

func ComputeHashesBytes

func ComputeHashesBytes(b []byte) (Hashes, error)

ComputeHashesBytes is the byte-slice convenience.

type HostedFile added in v0.8.0

type HostedFile struct {
	Name   string
	Digest string
	Size   int64
}

HostedFile is one stored file, as seen by a Generator.

type HostedHandler added in v0.8.0

type HostedHandler struct {
	Format    string
	Store     HostedStore
	Auth      Auth
	Generator Generator
}

HostedHandler serves an upload/generated-index repo for one format.

func (*HostedHandler) Delete added in v0.10.0

func (h *HostedHandler) Delete(w http.ResponseWriter, r *http.Request, repo, name string)

Delete removes a hosted file from repo.

func (*HostedHandler) Get added in v0.10.0

func (h *HostedHandler) Get(w http.ResponseWriter, r *http.Request, repo, name string) bool

Get serves a hosted file or generated index from repo. It reports whether the request was handled (true) or the caller should fall back to proxying (false) because the repo holds no hosted content.

repo  the repository key (first path segment)
name  the path inside the repo

func (*HostedHandler) Put added in v0.10.0

func (h *HostedHandler) Put(w http.ResponseWriter, r *http.Request, repo, name string)

Put stores an uploaded file into repo.

type HostedStore added in v0.8.0

type HostedStore struct {
	Registry *Registry
}

HostedStore is the minimal backend the hosted handler needs (satisfied by Registry).

func (HostedStore) HostedDelete added in v0.8.0

func (h HostedStore) HostedDelete(ctx context.Context, format, repo, name string) error

HostedDelete removes one hosted file and its CAS blob when unreferenced.

func (HostedStore) HostedFile added in v0.8.0

func (h HostedStore) HostedFile(ctx context.Context, format, repo, name string) (string, bool)

HostedFile returns the stored digest for one hosted file ("" when absent).

func (HostedStore) HostedFiles added in v0.19.0

func (h HostedStore) HostedFiles(ctx context.Context, format, repo string) ([]HostedFile, error)

HostedFiles returns every hosted file in the repo with its digest and size in ONE index pass, rather than a Get + Stat per name. Index generation uses it so a repo with N packages costs one query.

func (HostedStore) HostedUpload added in v0.8.0

func (h HostedStore) HostedUpload(ctx context.Context, format, repo, name string, r io.Reader) (string, int64, error)

HostedUpload streams uploaded bytes under (format, repo, name) into the CAS and indexes them, without buffering the whole body in memory. It returns the digest and size.

type IndexCache

type IndexCache = MemCache

IndexCache is the type returned by SharedIndexCache for adapter use.

type IndexStore

type IndexStore interface {
	// Put upserts an artifact's metadata by (format, repository, version).
	Put(ctx context.Context, a Artifact) error
	// Get fetches (format, repository, version) metadata.
	Get(ctx context.Context, format, repository, version string) (Artifact, error)
	// Delete removes one (format, repository, version).
	Delete(ctx context.Context, format, repository, version string) error
	// ListVersions returns all versions for a repository, ordered.
	ListVersions(ctx context.Context, format, repository string) ([]string, error)
	// ListArtifacts returns every artifact row for a repository in one pass
	// (rather than Get-per-version). Index generators use it so a repo with N
	// files costs one query, not N.
	ListArtifacts(ctx context.Context, format, repository string) ([]Artifact, error)
	// ListRepositoriesByFormat returns every repository that has ≥1 version.
	ListRepositoriesByFormat(ctx context.Context, format string) ([]string, error)
	// ListRepositories returns every repository across all formats.
	ListRepositories(ctx context.Context) ([]string, error)
	// ListPackages returns a compact summary for the catalog.
	ListPackages(ctx context.Context) ([]PackageSummary, error)
	// ReferencedDigests returns every CAS digest any artifact references
	// (primary Digest plus blob descriptors), so a caller can decide whether a
	// blob is still in use without an N+1 scan. One pass over the index.
	ReferencedDigests(ctx context.Context) (map[string]bool, error)
	// DeleteRepo removes an entire repository for a format.
	DeleteRepo(ctx context.Context, format, repository string) (int, error)
	// SaveUpload persists an upload session.
	SaveUpload(ctx context.Context, u UploadRecord) error
	// GetUpload returns a session by id.
	GetUpload(ctx context.Context, id string) (UploadRecord, error)
	// DeleteUpload removes a session.
	DeleteUpload(ctx context.Context, id string) error
	// ListUploads returns session ids.
	ListUploads(ctx context.Context) ([]string, error)
	// GetMeta returns a per-repository metadata blob (e.g. a resolved index).
	GetMeta(ctx context.Context, format, repository string) ([]byte, error)
	// SetMeta stores a per-repository metadata blob.
	SetMeta(ctx context.Context, format, repository string, data []byte) error
	// Close releases the underlying resources.
	Close() error
}

IndexStore abstracts the structured metadata layer (SQLite reference implementation). It holds the mutable index: repositories, versions, and the descriptors pointing at immutable blobs.

func NewScopedStore added in v0.10.0

func NewScopedStore(inner IndexStore) IndexStore

NewScopedStore wraps an IndexStore with request-scoped namespacing.

type MemCache

type MemCache struct {
	// contains filtered or unexported fields
}

MemCache is a process-wide TTL cache for upstream index/metadata documents. Caching the index layer (not just artifacts) is what makes a mirror fast for repeated builds; otherwise every client resolution re-fetches the same small index documents through the proxy.

func NewMemCache

func NewMemCache(ttl time.Duration) *MemCache

NewMemCache returns a TTL cache with a bound on entries.

func SharedIndexCache

func SharedIndexCache() *MemCache

SharedIndexCache returns a process-global index cache (1h TTL) used by the protocol adapters that have no per-call cache to thread.

func (*MemCache) Get

func (c *MemCache) Get(key string) (string, bool)

Get returns a cached body, or false when absent/expired.

func (*MemCache) Set

func (c *MemCache) Set(key, body string)

Set stores a body with a fresh expiry. At capacity it evicts one expired entry when present, else an arbitrary entry, so the bound is always honored (a cache that grows without bound under an all-fresh workload is a leak).

type NamespaceResolver added in v0.10.0

type NamespaceResolver func(format, path string) (namespace, name, rest string)

NamespaceResolver maps a format-relative request path onto a namespace, a package name and the remainder of the path (the part inside the package). Returning ns=="" means the default namespace.

type NamespacedOwnership added in v0.10.0

type NamespacedOwnership interface {
	Ownership
	// AuthorizePublishNS authorizes publishing name inside namespace.
	AuthorizePublishNS(ctx context.Context, format, namespace, name string, tenantID int64) error
	// CanReadNS reports whether tenant may read name in namespace.
	CanReadNS(ctx context.Context, format, namespace, name string, tenantID int64) bool
}

NamespacedOwnership is the namespace-aware extension of Ownership. A namespace (npm scope, OCI host, maven groupId, ...) is a boundary of its own: a tenant owning @acme may publish any @acme package; a tenant may also be granted read on a whole namespace.

type Ownership added in v0.4.0

type Ownership interface {
	// AuthorizePublish returns nil when tenant may publish (format, name).
	// An unclaimed name is claimed for the tenant on first publish; a claimed
	// name requires the same tenant (admins excepted per implementation).
	AuthorizePublish(ctx context.Context, format, repository string, tenantID int64) error
	// CanRead reports whether tenant may read (format, name): true for
	// public/unclaimed names, owning tenant only for private ones.
	CanRead(ctx context.Context, format, repository string, tenantID int64) bool
}

Ownership is the npm-official-style ownership model over the registry's global namespace: names are globally unique, publishing requires the caller's tenant to own (or first-claim) the name, and private packages are readable only by their owning tenant.

A repository is identified by (format, repository) where repository is the canonical RepoKey(namespace, name). Implementations that predate namespaces implement the base interface; NamespacedOwnership additionally exposes the namespace as its own boundary. Callers use the Authorize* helpers, which prefer the namespaced form when available.

type PackageSummary

type PackageSummary struct {
	Format     string `json:"format"`
	Repository string `json:"repository"`
	Version    string `json:"version"`
	MediaType  string `json:"media_type,omitempty"`
	Digest     string `json:"digest,omitempty"`
	Size       int64  `json:"size"`
}

type PathCachePolicy added in v0.15.0

type PathCachePolicy struct {
	// Immutable marks content that never changes (a versioned/digest file):
	// it is cached forever and never revalidated.
	Immutable bool
	// TTL is how long a mutable entry is trusted before revalidation. Zero
	// uses DefaultPathTTL.
	TTL time.Duration
	// MediaType is recorded and replayed for the entry.
	MediaType string
	// BlobName is the descriptor name; defaults to the path's base.
	BlobName string
	// NoStore proxies without reading or writing the cache (a per-client
	// credential must not be shared), matching netcache's NoStore.
	NoStore bool
}

PathCachePolicy tunes how one path-tree entry is kept fresh.

type PathCacheResult added in v0.17.0

type PathCacheResult struct {
	// Artifact carries the digest, size, media type and the replayable HTTP
	// metadata (Content-Encoding/ETag/Last-Modified). Serve it with
	// ServeCachedBlob so the stored headers reach the client.
	Artifact Artifact
	// Hit is true when the bytes were served from the cache with no upstream
	// request.
	Hit bool
	// OK is false when the object is absent upstream (or air-gapped/unreachable).
	OK bool
}

PathCacheResult is the outcome of FetchCachedPath.

func (PathCacheResult) Digest added in v0.17.0

func (p PathCacheResult) Digest() string

Digest is the CAS digest of the served bytes ("" when !OK).

func (PathCacheResult) Size added in v0.17.0

func (p PathCacheResult) Size() int64

Size is the byte length (0 when !OK).

type Principal added in v0.3.0

type Principal struct {
	Username string
	Level    TokenLevel
	TenantID int64
}

Principal is the resolved identity behind a credential. TenantID carries the caller's tenancy boundary (0 = unset/default); the registry ownership layer uses it to decide who may publish and who may read private packages.

type PrincipalFunc added in v0.3.0

type PrincipalFunc func(ctx context.Context, token string) (Principal, bool)

PrincipalFunc adapts a function to TokenStore (OpenInstance = false).

func (PrincipalFunc) LookupToken added in v0.3.0

func (f PrincipalFunc) LookupToken(ctx context.Context, token string) (Principal, bool)

func (PrincipalFunc) LookupUsername added in v0.3.0

func (f PrincipalFunc) LookupUsername(context.Context, string) (Principal, bool)

func (PrincipalFunc) OpenInstance added in v0.3.0

func (f PrincipalFunc) OpenInstance(context.Context) bool

type Protocol

type Protocol interface {
	// Name returns the protocol format key ("oci", "pypi", "cargo", ...).
	Name() string
	// New builds the adapter for a substrate and configuration, or errors.
	New(reg *Registry, cfg map[string]any) (http.Handler, error)
}

Protocol is the pluggable seam for third-party protocol adapters. An adapter implements Register so the registry knows its format name; the concrete handler is a http.Handler mounted at a caller-chosen prefix.

Implementations must only depend on *artifactkit.Registry and never on other protocol packages. This invariant is what keeps the multi-repository module graph free of version conflicts.

type ProtocolEntry

type ProtocolEntry struct {
	Name string
	New  func(reg *Registry, cfg map[string]any) (http.Handler, error)
}

ProtocolEntry couples a protocol's name to a constructor for the registry.

type ProxyState

type ProxyState struct {
	Key      string `json:"key"`
	Proxy    string `json:"proxy"`
	Explicit bool   `json:"explicit"`
}

ProxyState is the per-key proxy policy in ProxyStates().

type Registry

type Registry struct {
	Blobs     BlobStore
	Meta      IndexStore
	Upstreams *Upstreams
	// Owners is the optional package-ownership layer: a GLOBAL namespace per
	// format where every name belongs to exactly one tenant. Nil disables
	// ownership enforcement (single-tenant / dev mode).
	Owners Ownership
	// TargetStore persists user-declared targets. Nil means built-ins only,
	// and the admin API treats target writes as unsupported.
	TargetStore targets.Store
}

Registry is the composed substrate handed to every protocol adapter. It bundles the metadata store (mutable index), the blob store (immutable CAS) and the upstream table, plus the generic pull-through flows. Adapters only depend on the interfaces, so a caller can wire any implementation.

func (*Registry) BlobHashes added in v0.24.0

func (r *Registry) BlobHashes(ctx context.Context, digest string) (Hashes, error)

BlobHashes returns a blob's hash set. It reads the set recorded by the store at write time (no re-read); only a blob written before that feature existed falls back to one recompute from the stored bytes.

func (*Registry) Fetch

func (r *Registry) Fetch(ctx context.Context, format, upstreamBase, path string) (Fetched, error)

Fetch pulls `path` from an upstream and caches it in the blob store. A redirect from the upstream (3xx with Location) is surfaced as an error so callers can decide how to handle it instead of silently downloading the whole target server-side.

func (*Registry) FetchAbsolute

func (r *Registry) FetchAbsolute(ctx context.Context, url string) (Fetched, error)

FetchAbsolute pulls a full URL verbatim.

func (*Registry) FetchAbsoluteBytes added in v0.13.0

func (r *Registry) FetchAbsoluteBytes(ctx context.Context, url string) ([]byte, error)

FetchAbsoluteBytes pulls a full URL verbatim and returns its bytes.

func (*Registry) FetchAbsoluteToBlob added in v0.13.0

func (r *Registry) FetchAbsoluteToBlob(ctx context.Context, url, wantDigest string) (int64, error)

FetchAbsoluteToBlob streams an absolute URL into the CAS, verifying it matches wantDigest (when non-empty) and following redirects. It is used for presigned CDN hrefs (git LFS objects) that the caller learned out of band.

func (*Registry) FetchCachedPath added in v0.15.0

func (r *Registry) FetchCachedPath(ctx context.Context, format, repo, version, path, upstreamBase string, pol PathCachePolicy) PathCacheResult

FetchCachedPath returns the cached artifact for one path-tree object, serving local bytes when fresh and revalidating or fetching otherwise. OK is false when the object is absent upstream (or air-gapped / unreachable via a 4xx). Hit is true when the bytes were served from the cache without any upstream request. Serve the result with ServeCachedBlob to replay stored headers.

Revalidation rules:

  • fresh (immutable, or within TTL): served locally, no network.
  • stale WITH a validator: one conditional request; 304 extends the TTL and serves the cached bytes; 200 replaces them.
  • stale WITHOUT a validator: served locally. Re-downloading a large index that cannot be conditionally revalidated would violate "fetch once".

func (*Registry) FetchPath added in v0.11.0

func (r *Registry) FetchPath(ctx context.Context, format, path string) (Fetched, error)

FetchPath pulls a format-relative path using the request's full upstream priority (per-repo override → client origin → table default), prepending the client's stripped prefix so the upstream sees the original path. Adapters call this instead of assembling a base themselves.

func (*Registry) FetchPathFollow added in v0.12.0

func (r *Registry) FetchPathFollow(ctx context.Context, format, path string) (Fetched, error)

FetchPathFollow is FetchPath but follows redirects, caching the final body. It is for the plain-HTTP trees whose roots redirect to a regional mirror (pkg.julialang.org → in.pkg.julialang.org) and for registries that redirect to an artifact store the mirror should read through (Ivy).

func (*Registry) FetchToBlob added in v0.13.0

func (r *Registry) FetchToBlob(ctx context.Context, format, path string) (string, int64, bool)

FetchToBlob streams a remote object into the CAS and returns its digest and size. Unlike Fetch it never buffers the whole body in memory, so it is the right call for the large, range-addressable files of the plain-HTTP trees (Hackage's 100MB+ index) and for anything a client may request a Range of. Redirects are followed (a tree root may point at a regional mirror).

func (*Registry) FetchURIToBlob added in v0.14.0

func (r *Registry) FetchURIToBlob(ctx context.Context, rawURL string, opts URIOptions) (URICacheResult, error)

FetchURIToBlob fetches an absolute URL into the CAS, deduplicating concurrent requests for the same key and streaming the body (no full buffering). When a cached entry exists and is fresh (immutable, or within TTL), it is returned without touching the network. Redirects are followed.

func (*Registry) ReadBlobPrefix added in v0.19.0

func (r *Registry) ReadBlobPrefix(ctx context.Context, digest string, n int64) ([]byte, error)

ReadBlobPrefix reads at most n bytes from the start of a blob, streaming rather than loading it whole. Index generators use it to parse the small control/header region of a large package without reading every byte. (nil, nil) when the blob is absent.

func (*Registry) Remote

func (r *Registry) Remote(ctx context.Context, spec UpstreamSpec) (*Remote, error)

Remote resolves an upstream handle for a spec. A Base is honored verbatim; a Host addresses the source by hostname; a Sub targets a sub-endpoint; else the format/repository is resolved from the request context (per-repo override -> client origin -> table default) with the target's auth applied.

func (*Registry) StoreAndHash

func (r *Registry) StoreAndHash(ctx context.Context, data []byte) (Stored, error)

StoreAndHash writes into the blob store (dedup by sha256) and returns a summary. The store computes and records the hashes, so nothing is hashed twice.

func (*Registry) StorePathBlob added in v0.14.0

func (r *Registry) StorePathBlob(ctx context.Context, format, repository, version, blobName, mediaType string, data []byte) string

StorePathBlob is the shared "cache one fetched path" flow for the path-tree protocols (apk/debian/rpm/conda/nix/...): store the bytes in the CAS by digest, then index them under (format, repository, version). It is best-effort: a cache write failure never fails the response, so the caller keeps serving the bytes it already has.

blobName is recorded on the descriptor (adapters differ: some use the basename, some the full relative path). Empty falls back to version.

func (*Registry) StoreStream added in v0.19.0

func (r *Registry) StoreStream(ctx context.Context, body io.Reader) (Stored, error)

StoreStream consumes a reader into the CAS without buffering the whole body in memory. The store streams it once, computing and recording the hash set in that pass (no double hash, no post-write re-read). This is the upload-path counterpart to StoreAndHash.

func (*Registry) StoreVersion added in v0.23.0

func (r *Registry) StoreVersion(ctx context.Context, in VersionInput)

StoreVersion persists one artifact version: stream the content into the CAS, REPLACE any existing descriptor of the same filename (a re-publish of a filename supersedes it; other filenames of the same version, e.g. a jar's pom/sha1, are preserved), and upsert the index row. It is the single implementation behind every flat adapter's publish/cache write.

An orphaned blob from a replaced descriptor is NOT deleted eagerly: another version may share the digest, and the background reaper reclaims true orphans. Index errors are logged, never fatal to the response.

type Remote

type Remote struct {
	Base string
	// contains filtered or unexported fields
}

Remote is a proxy-aware upstream root URL. Adapters map their URL layout onto Get/GetBytes/GetCached.

func NewRemote

func NewRemote(factory *ClientFactory, base string, proxy *string) *Remote

NewRemote creates a Remote for a base URL (trailing slash trimmed).

func (*Remote) Do added in v0.11.0

func (r *Remote) Do(ctx context.Context, method, path string, body io.Reader, contentType, accept string) (*http.Response, error)

Do issues an arbitrary request against the remote root. body may be nil; a non-empty contentType is set as the request Content-Type. It is the general escape hatch for protocols (git smart-HTTP) whose verbs are not plain GET.

func (*Remote) Get

func (r *Remote) Get(ctx context.Context, path string) (*http.Response, error)

Get issues GET and returns the raw response (non-2xx returned as-is).

func (*Remote) GetBytes

func (r *Remote) GetBytes(ctx context.Context, path string) ([]byte, error)

GetBytes GETs and errors on non-2xx, returning the body bytes. Transient failures — transport errors or 502/503/504 responses (an egress proxy often surfaces upstream blips that way) — are retried: these are idempotent index/file GETs, and short backoff rides out multi-second proxy flaps.

Concurrent identical requests are collapsed by a process-wide single-flight group (keyed by URL + auth headers), so N clients resolving the same index document cause one upstream fetch.

func (*Remote) GetBytesFollow added in v0.11.0

func (r *Remote) GetBytesFollow(ctx context.Context, path string) ([]byte, error)

GetBytesFollow is GetBytes but follows redirects (up to the default policy). Ivy servers redirect to their artifact store (repo.scala-sbt.org → scala.jfrog.io) and the redirected body is what must be cached. Concurrent identical requests are collapsed by the same single-flight group.

func (*Remote) GetCached

func (r *Remote) GetCached(ctx context.Context, cache *MemCache, path string) (string, error)

GetCached GETs through a TTL cache keyed by absolute URL (for small index documents re-fetched on every client resolution).

func (*Remote) URL

func (r *Remote) URL(path string) string

URL joins a path onto the base.

func (*Remote) WithHeader

func (r *Remote) WithHeader(k, v string) *Remote

WithHeader returns a copy with an extra header (e.g. Accept, auth).

type RepoScope added in v0.10.0

type RepoScope struct {
	Format    string
	Namespace string
	Name      string
	Host      string
	Proto     string
	Prefix    string
	// Explicit is true when the request used the "/-/<repo>/" marker.
	Explicit bool
	// Target is the target id the request was routed through ("maven",
	// "maven.google"). Empty means the protocol's default target.
	Target string
	// TargetShared is true when the target stores into the protocol's shared
	// namespace (mirrors) rather than an isolated one (user-declared private
	// repositories). Only meaningful when Target is set.
	TargetShared bool
	// ClientAuth is the request's Authorization header, captured so a target
	// with passthrough auth can forward the caller's own credential upstream.
	// It is never logged or stored as key material.
	ClientAuth string
}

RepoScope is the repository a request targets, resolved once at the mount boundary. It is carried in the request context so the storage and upstream layers can isolate content and choose a remote WITHOUT every adapter threading the namespace through its calls.

Namespace resolution order:

  1. an explicit "/-/<repo>/" marker in the path, else
  2. the format's NamespaceResolver (npm scope, maven groupId, go module prefix, apk/debian/rpm/conda first segment, OCI registry host), else
  3. DefaultNamespace.

Host is the request's original Host header. It matters for OCI: a docker client puts the registry in SNI/Host, never in the path, so the host is the only place the registry can be learned. The OCI resolver reads it from here.

Proto and Prefix describe the origin the CLIENT reached us by, when the request came through the egress proxy (which sets X-Forwarded-Proto and X-Forwarded-Prefix next to preserving the Host). Together with Host they let the upstream be reconstructed without a per-ecosystem table:

<proto>://<host><prefix><format-relative-path>

func RepoScopeFrom added in v0.10.0

func RepoScopeFrom(ctx context.Context) RepoScope

RepoScopeFrom returns the request's repository scope, or the zero value.

func (RepoScope) ScopedKey added in v0.13.0

func (s RepoScope) ScopedKey(key string) string

ScopedKey is the canonical storage key for an adapter-supplied repository: the target prefix (when isolated) around the namespace-qualified name.

func (RepoScope) ScopedName added in v0.10.0

func (s RepoScope) ScopedName(key string) string

ScopedName maps an adapter-computed repository key onto its fully-qualified form for the scope. The namespace is prefixed only when the key does not already carry it, so npm's "@scope/name" (where the adapter already includes the scope) is left alone while maven's bare artifactId and a flat protocol's package name gain the namespace.

func (RepoScope) UnscopedKey added in v0.13.0

func (s RepoScope) UnscopedKey(key string) string

UnscopedKey is ScopedKey's inverse for metadata reads/list filtering: it strips the target prefix first, then the namespace.

func (RepoScope) UnscopedName added in v0.10.0

func (s RepoScope) UnscopedName(key string) string

UnscopedName is ScopedName's inverse for catalog/list filtering.

type RepoState added in v0.10.0

type RepoState struct {
	Format string `json:"format"`
	Repo   string `json:"repo"`
	Base   string `json:"base"`
	Proxy  string `json:"proxy,omitempty"`
}

RepoState is one repository override in RepoStates().

type RepoUpstream added in v0.10.0

type RepoUpstream struct {
	Base  string `json:"base"`
	Proxy string `json:"proxy,omitempty"`
}

RepoUpstream is the per-repository upstream policy.

type StoreAuth added in v0.3.0

type StoreAuth struct {
	// contains filtered or unexported fields
}

StoreAuth is an Auth backed by a TokenStore (hashed credentials in a database) plus an in-memory minted-token table for OCI bearer tokens. The long-lived database credential is never handed back to a client.

func NewStoreAuth added in v0.3.0

func NewStoreAuth(store TokenStore) *StoreAuth

NewStoreAuth builds an Auth over a TokenStore.

func (*StoreAuth) Authenticate added in v0.3.0

func (a *StoreAuth) Authenticate(ctx context.Context, r *http.Request) string

Authenticate implements Auth. It returns a username for any valid credential (read or write); callers needing a privilege decision must use CheckBearer/CheckToken.

func (*StoreAuth) CheckBasic added in v0.3.0

func (a *StoreAuth) CheckBasic(ctx context.Context, _, pass string) bool

CheckBasic implements Auth: the password carries the token.

func (*StoreAuth) CheckBearer added in v0.3.0

func (a *StoreAuth) CheckBearer(ctx context.Context, token, wantedScope string) (string, bool)

CheckBearer implements Auth. The requested action must be permitted by the credential's level: a read-level credential (or a pull-only mint) cannot push/delete.

func (*StoreAuth) CheckToken added in v0.3.0

func (a *StoreAuth) CheckToken(ctx context.Context, token string) (string, bool)

CheckToken implements Auth.

func (*StoreAuth) IssueToken added in v0.3.0

func (a *StoreAuth) IssueToken(ctx context.Context, username string, scopes []string, ttl time.Duration) string

IssueToken implements Auth. It mints a RANDOM bearer token bound to the grantee's privilege: a write-level principal gets a push-capable token, a read-level principal gets a pull-only token, and an anonymous caller gets a pull-only token too (public reads need no credential). The database credential is NEVER handed back to a client.

func (*StoreAuth) TenantID added in v0.4.0

func (a *StoreAuth) TenantID(ctx context.Context, r *http.Request) int64

TenantID resolves the request's tenant through the store when it models tenancy (TenantTokenStore); the default tenant otherwise.

type Stored

type Stored struct {
	Hashes Hashes
	Size   int64
	Digest string
}

Stored is the summary of bytes written into the CAS.

type TargetDispatcher added in v0.13.0

type TargetDispatcher struct {
	Registry *Registry
	// Handlers maps a protocol name to its adapter. Built once at startup.
	Handlers map[string]http.Handler
	// Base is the mount prefix (normally MountBase).
	Base string
}

TargetDispatcher routes /artifacts/<target-id>/... to the adapter for the target's protocol, resolving the target at REQUEST time. It replaces the startup-time per-target mux registrations, so a target created through the admin API is reachable immediately (no restart) and built-ins, persisted and default targets all flow through one code path.

The adapter never sees the target: the path is rewritten to the protocol mount (/artifacts/<protocol>/<rest>) and the target identity is carried in the request scope, where it selects the upstream and, for an isolated target, the storage prefix.

func NewTargetDispatcher added in v0.13.0

func NewTargetDispatcher(base string, reg *Registry, handlers map[string]http.Handler) *TargetDispatcher

NewTargetDispatcher builds the dispatcher. Handlers are keyed by protocol.

func (*TargetDispatcher) ServeHTTP added in v0.13.0

func (d *TargetDispatcher) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP resolves the first path segment as a target id. A segment that is not a known target falls through to the protocol of the same name (legacy behavior, and the OCI /v2 path is mounted separately).

type TenantResolver added in v0.4.0

type TenantResolver interface {
	TenantID(ctx context.Context, r *http.Request) int64
}

TenantResolver is implemented by Auth implementations that can resolve a request's tenant (StoreAuth does, bridging TenantTokenStore).

type TenantTokenStore added in v0.4.0

type TenantTokenStore interface {
	// TenantOfToken resolves the tenant id of a credential (0 → default).
	TenantOfToken(ctx context.Context, token string) int64
	// TenantOfUsername resolves the tenant id of a user (0 → default).
	TenantOfUsername(ctx context.Context, username string) int64
}

TenantTokenStore is the optional tenancy extension of TokenStore: it resolves the tenant a credential belongs to. Implementations that don't model tenants simply don't implement it (every caller falls back to the default tenant).

type TokenAuth

type TokenAuth struct {
	// Tokens maps a token string to its level.
	Tokens map[string]TokenLevel
	// contains filtered or unexported fields
}

TokenAuth is the reference Auth implementation for a `token=level` table.

func NewTokenAuth

func NewTokenAuth(spec string) *TokenAuth

NewTokenAuth builds a TokenAuth from a `level=token` or `token=level` comma/space separated string (e.g. "devtoken=write,readtoken=read").

func (*TokenAuth) Authenticate

func (t *TokenAuth) Authenticate(ctx context.Context, r *http.Request) string

Authenticate implements Auth. It accepts Bearer/token prefixes, a bare token, Basic (user:token), and the X-NuGet-ApiKey header. The returned username identifies ANY valid credential (read or write); callers that need a privilege decision must use CheckBearer/CheckToken/permits.

func (*TokenAuth) CheckBasic

func (t *TokenAuth) CheckBasic(ctx context.Context, _, pass string) bool

func (*TokenAuth) CheckBearer

func (t *TokenAuth) CheckBearer(ctx context.Context, token, wantedScope string) (string, bool)

CheckBearer implements Auth. The token may be a minted bearer token (whose grant bounds the allowed action) or a static token checked by level.

func (*TokenAuth) CheckToken

func (t *TokenAuth) CheckToken(ctx context.Context, token string) (string, bool)

func (*TokenAuth) IssueToken

func (t *TokenAuth) IssueToken(ctx context.Context, username string, scopes []string, ttl time.Duration) string

IssueToken implements Auth. It mints a RANDOM bearer token bound to the grantee's privilege: a write-level principal gets a push-capable token, a read-level (or anonymous) principal gets a pull-only token. The static write token is NEVER handed back to a client.

type TokenLevel

type TokenLevel string

TokenLevel is the privilege carried by a static token.

const (
	LevelRead  TokenLevel = "read"
	LevelWrite TokenLevel = "write"
)

type TokenStore added in v0.3.0

type TokenStore interface {
	// LookupToken resolves a presented token to its principal. ok=false when
	// unknown.
	LookupToken(ctx context.Context, token string) (Principal, bool)
	// LookupUsername resolves a principal by username, reporting the
	// strongest credential level registered for that user. ok=false when the
	// user has no credentials (IssueToken then mints pull-only).
	LookupUsername(ctx context.Context, username string) (Principal, bool)
	// OpenInstance reports whether the deployment has no users at all, in
	// which case anonymous write is permitted (dev/single-user mode).
	OpenInstance(ctx context.Context) bool
}

TokenStore is the credential database an Auth consults. It is deliberately tiny so a deployment can bridge its own user tables (e.g. easylab's easyvcs store) without adopting artifact's storage. Tokens are compared by the STORE's semantics — implementations are expected to store hashes and hash the presented value on lookup (never plaintext).

type URICacheResult added in v0.14.0

type URICacheResult struct {
	Digest string
	Size   int64
	Header http.Header
	// Immutable is true when the entry is cached forever.
	Immutable bool
	// Status is the upstream status the bytes came from (200/304).
	Status int
}

URICacheResult is the outcome of caching one URI.

type URIOptions added in v0.14.0

type URIOptions struct {
	// Headers are sent on the upstream request (passthrough credentials,
	// Accept). They never enter the cache key.
	Headers http.Header
	// KeyURL, when set, is the cache identity instead of the fetch URL. It is
	// the CLIENT-facing absolute URL: a declared target may fetch the same
	// asset from an internal mirror, but the asset's identity is what the
	// client asked for, so two clients of the same logical asset share one
	// entry regardless of which mirror served it.
	KeyURL string
	// NoStore skips both the cache read and the cache write, so a request
	// carrying a per-client credential is proxied but never shared. Immutable
	// public objects do not need this; credentialed private ones do.
	NoStore bool
}

URIOptions tunes one FetchURIToBlob call.

type UploadRecord

type UploadRecord struct {
	ID         string `json:"id"`
	Format     string `json:"format"`
	Repository string `json:"repository"`
	Digest     string `json:"digest,omitempty"`
	Bytes      int64  `json:"bytes"`
	Complete   bool   `json:"complete"`
}

UploadRecord is a persisted in-progress multi-chunk upload session.

type UpstreamEntry

type UpstreamEntry struct {
	Name string `json:"key"`
	URL  string `json:"url"`
}

UpstreamEntry is one effective mapping in All().

type UpstreamRedirectError added in v0.7.0

type UpstreamRedirectError struct {
	Path     string
	Status   int
	Location string
}

UpstreamRedirectError reports a 3xx from an upstream.

func (*UpstreamRedirectError) Error added in v0.7.0

func (e *UpstreamRedirectError) Error() string

type UpstreamSpec added in v0.22.0

type UpstreamSpec struct {
	Format string
	Repo   string
	Sub    string
	Host   string
	Base   string
}

UpstreamSpec selects which upstream a Remote targets. Exactly one selector is used, in this precedence:

Base  an absolute base URL, verbatim (no target auth; generic proxy)
Host  address the upstream by hostname (git/ivy source mirrors)
Sub   a format sub-endpoint on its own host (cargo index, nuget search)
Repo  one repository of Format (per-repo override + target auth)
zero  the request-context priority for Format (override -> client origin
      -> table default), applying the resolved target's auth

It is the ONE constructor for upstream handles; adapters never assemble a base themselves.

type UpstreamStatusError

type UpstreamStatusError struct {
	Path   string
	Status int
}

UpstreamStatusError is the shared non-2xx upstream error.

func (*UpstreamStatusError) Error

func (e *UpstreamStatusError) Error() string

type Upstreams

type Upstreams struct {
	// Targets is the target registry. Nil disables target-driven resolution.
	Targets *targets.Registry
	// Defaults maps a format to its built-in public upstream base.
	Defaults map[string]string
	// Overrides overrides Defaults; empty value reverts to the default.
	Overrides map[string]string
	// Proxy is per-key proxy URL. "" means direct; absence means env proxy.
	Proxy map[string]string
	// Repos overrides the upstream per (format, repository/namespace). Keys are
	// "format/repo" (e.g. "npm/@acme", "maven/org.apache", "go/github.com/acme")
	// and match the LONGEST repository prefix, so one entry covers a subtree.
	Repos map[string]RepoUpstream
	// AllowedHosts are extra upstream hosts that may be reached through the
	// X-Forwarded-Host path even though they are not in Defaults (container
	// registries a client addresses by name: ghcr.io, quay.io, ...). Empty
	// values ("host" or "scheme://host") default to https.
	AllowedHosts []string
	// AirGap, when true, returns no upstreams at all (local-only registry).
	AirGap bool
}

Upstreams resolves the effective remote for a format / sub-endpoint against per-key overrides and per-key proxy policy, honoring an air-gap flag.

Targets is the target registry (protocol -> upstream identity). When set it is the single source of truth for hostnames (the SSRF allow-list) and for host-driven resolution; Defaults/Overrides/Repos remain as per-deployment policy layered on top. A nil Targets keeps the legacy table-only behavior.

func (*Upstreams) All

func (u *Upstreams) All() []UpstreamEntry

All returns the effective upstream for every known format + sub-endpoint.

func (*Upstreams) AllowedSource added in v0.11.0

func (u *Upstreams) AllowedSource(host string) (string, bool)

AllowedSource reports the recorded scheme for a host that may be used as an upstream source. Unlike HostBase it does not require the host to be in the upstream table: any syntactically valid public hostname is accepted, because the caller (the git/ivy mirrors) treats the requested host as the source identity. Cluster-local names and bare addresses are rejected so the mirror cannot be pointed at the gateway itself.

func (*Upstreams) Get

func (u *Upstreams) Get(format string) string

Get returns the effective base URL for a format, or "" when disabled.

func (*Upstreams) HostBase added in v0.11.0

func (u *Upstreams) HostBase(proto, host, prefix string) (string, bool)

HostBase reconstructs an upstream base (scheme://host) from the origin a client reached us by (the egress proxy records the original host/scheme as X-Forwarded-Host/Proto). It succeeds only when host is one the table already knows, so a hostile client cannot aim the mirror at an arbitrary origin. The stored scheme is the fallback; an http/https X-Forwarded-Proto wins (deb.debian.org is reached over http).

The client's stripped path prefix (/maven2, /stable) is restored here because it belongs to the origin the client dialed; every other upstream base comes from the table and needs none.

When a target registry is configured, the target that owns the host is authoritative: its Base already carries the mirror's path prefix (Google Maven's /dl/android/maven2) and its scheme is used, so a client that did not strip anything still reaches the right place.

func (*Upstreams) IsOverride

func (u *Upstreams) IsOverride(key string) bool

IsOverride reports whether a key has an explicit (non-default) override.

func (*Upstreams) ProxyFactory

func (u *Upstreams) ProxyFactory() *ClientFactory

ProxyFactory returns a client factory honored by the Upstreams' proxy policy. Adapters use it to build Remote handles.

func (*Upstreams) ProxyStates

func (u *Upstreams) ProxyStates() []ProxyState

ProxyStates returns the proxy policy for every known key.

func (*Upstreams) ProxyURL

func (u *Upstreams) ProxyURL(key string) (string, bool)

ProxyURL returns the proxy policy for a key, walking dotted parents, then falling back to the "*" catch-all entry when present (applied to every upstream). "" means direct; absence means the env proxy.

func (*Upstreams) Repo added in v0.10.0

func (u *Upstreams) Repo(format, repo string) string

Repo returns the effective upstream for one format's repository, using the longest matching repository prefix, then the format default. "" means disabled (air-gap or no upstream).

func (*Upstreams) RepoOverride added in v0.10.0

func (u *Upstreams) RepoOverride(format, repo string) (RepoUpstream, bool)

RepoOverride returns the explicit repository override for (format, repo) (longest prefix), without falling back to the format default. Callers that already have their own default (OCI derives https://<host>) use this to let -repo-upstreams win.

func (*Upstreams) RepoProxy added in v0.10.0

func (u *Upstreams) RepoProxy(format, repo string) (string, bool)

RepoProxy returns the proxy policy for a repository: the repository entry's own proxy when it sets one, else the format's proxy policy.

func (*Upstreams) RepoStates added in v0.10.0

func (u *Upstreams) RepoStates() []RepoState

RepoStates lists every repository override (sorted by key).

func (*Upstreams) Reset

func (u *Upstreams) Reset(key string)

Reset reverts a key override to its default.

func (*Upstreams) ResetRepo added in v0.10.0

func (u *Upstreams) ResetRepo(format, repo string)

ResetRepo removes a repository override (exact key).

func (*Upstreams) Set

func (u *Upstreams) Set(key, url string)

Set overrides a format/sub-endpoint URL and persists it in the in-memory map.

func (*Upstreams) SetProxy

func (u *Upstreams) SetProxy(key, value string)

SetProxy stores a per-key proxy URL ("" = direct).

func (*Upstreams) SetRepo added in v0.10.0

func (u *Upstreams) SetRepo(format, repo, base, proxy string)

SetRepo overrides the upstream (and optionally the proxy) for a repository.

func (*Upstreams) Sub

func (u *Upstreams) Sub(format, sub string) string

Sub returns the effective base URL for a dotted sub-endpoint.

func (*Upstreams) TargetFor added in v0.13.0

func (u *Upstreams) TargetFor(format, host string) (targets.Target, bool)

TargetFor returns the target a (format, host) request resolves to, when a target registry is configured. It is the host-driven half of the upstream priority: the caller layers per-repo overrides on top.

func (*Upstreams) TargetsFor added in v0.13.0

func (u *Upstreams) TargetsFor(format string) []targets.Target

TargetsFor returns every target serving a protocol (empty when no registry).

type VersionInput added in v0.23.0

type VersionInput struct {
	Format     string
	Repository string
	Version    string
	// Filename is the descriptor name (e.g. "pkg-1.2.3.tar.gz"). Empty uses
	// the version as the name.
	Filename string
	// Source is "push" (published) or "pull" (cached upstream).
	Source string
	// Body is the artifact content, streamed into the CAS (no in-memory copy).
	// Empty stores no blob (index-only row). Mutually exclusive with Data.
	Body io.Reader
	// Data is an in-memory alternative to Body. Mutually exclusive with Body.
	Data []byte
	// ExtraBlobs are additional blobs to index alongside the primary one
	// (rare; e.g. a detached signature). Optional.
	ExtraBlobs []Descriptor
	// Proprietary is opaque per-format metadata (hex inner_checksum, composer
	// autoload, ...). Optional.
	Proprietary []byte
	// MediaType is recorded on the row. Optional.
	MediaType string
	// DefaultVersion is applied when Version is empty (per-format, e.g.
	// "0.1.0"). Empty leaves Version as-is.
	DefaultVersion string
}

VersionInput describes one artifact version to persist: the identity plus the bytes. It is the input to StoreVersion, the shared publish/pull cache write every flat adapter used to hand-roll.

Directories

Path Synopsis
Package store provides reference implementations of the artifactkit storage interfaces: a filesystem blob CAS, a SQLite/postgres/mysql metadata index, and the credential tables.
Package store provides reference implementations of the artifactkit storage interfaces: a filesystem blob CAS, a SQLite/postgres/mysql metadata index, and the credential tables.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL