transfer

package
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 3, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package transfer is the peer-facing OCI endpoint other Gantry agents pull from. It binds `:5001` over HTTP/2 cleartext (h2c) because NetworkPolicy restricts the port to inter-node traffic and TLS termination would duplicate the libp2p Noise security that Gantry already does for coordination RPCs.

Design (architecture.md the API contract):

- Same OCI URL shape as the mirror: `/v2/<repo>/blobs/<digest>` and `/v2/<repo>/manifests/<digest>`. Tag-shaped manifest requests at this endpoint return **404 unconditionally** - peers must already know the digest (the tag-resolution path runs through the mirror, not here). - Requires `Gantry-Mirrored: 1` request header. Without the header the server returns 400. With the header, the response semantics are: **serve from the local store or return 404**. No DHT lookup, no HRW probe, no `please_pull`, no origin contact. The header is the loop-breaker that prevents two agents from recursing into each other's miss paths. - `Range: bytes=N-M` returns `206 Partial Content` with the correct `Content-Range`. Resumable peer callers use this to continue a live stream from another provider without retransmitting the verified prefix; the contract is preserved for v2 striping. - Metric `p2p_peer_serve_total` is bumped per served body - not `p2p_cache_hit_total`, so cluster scrapes distinguish containerd- facing hits from peer-facing serves.

Index

Constants

View Source
const MirroredHeader = "Gantry-Mirrored"

MirroredHeader is the OCI-extension header peers MUST include on every transfer-endpoint request. It is the loop-breaker described in the design doc.

Variables

This section is empty.

Functions

This section is empty.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client implements ifaces.PeerDialer over HTTP/2 cleartext (h2c). Reuse a single Client across all peers - the underlying http2.Transport pools per-host connections internally.

func NewClient

func NewClient(opts ...ClientOption) *Client

NewClient builds a Client tuned for peer fetches.

func (*Client) FetchFromPeer

func (c *Client) FetchFromPeer(ctx context.Context, peerAddr string, ref ifaces.OriginRef) (io.ReadCloser, int64, error)

FetchFromPeer implements ifaces.PeerDialer.

type ClientOption

type ClientOption func(*clientOptions)

ClientOption tweaks Client construction.

func WithClientByteMetrics added in v0.2.1

func WithClientByteMetrics(onBytesRead func(kind string, bytes int64)) ClientOption

WithClientByteMetrics registers a callback for bytes actually read from peer response bodies, including partial failed transfers and retries.

func WithDialTimeout

func WithDialTimeout(d time.Duration) ClientOption

WithDialTimeout sets the TCP dial timeout per peer.

func WithReadIdleTimeout

func WithReadIdleTimeout(d time.Duration) ClientOption

WithReadIdleTimeout configures the h2 ping-based idle stall detector.

func WithRequestTimeout

func WithRequestTimeout(d time.Duration) ClientOption

WithRequestTimeout caps total time per request.

type Describer

type Describer interface {
	LookupMediaType(d digest.Digest) string
}

Describer is an optional capability that callers (typically the containerdstore.Store) may implement to surface cached media-type metadata for a digest. The transfer endpoint uses it to set a correct manifest Content-Type instead of falling back to application/octet-stream - per "Descriptor and media-type handling": "Do not serve manifest digests with a generic content type unless existing containerd/client behavior proves it is safe."

An empty return value means "unknown"; callers MUST fall back to the kind-based default rather than treating it as an error.

type Option

type Option func(*Server)

Option configures a Server.

func WithByteMetrics added in v0.2.1

func WithByteMetrics(onPeerServeBytes func(kind string, bytes int64)) Option

WithByteMetrics registers a callback for bytes actually transmitted to peer agents. HEAD requests emit no bytes, and Range requests count only the range body sent on the wire.

func WithDescriber

func WithDescriber(d Describer) Option

WithDescriber registers an optional media-type lookup for cached digests. Wire this with the containerdstore.Store so manifest responses carry the correct OCI/Docker media type instead of application/octet-stream.

func WithLogger

func WithLogger(l *slog.Logger) Option

WithLogger plumbs a structured logger.

func WithMaxConcurrentServes added in v0.1.23

func WithMaxConcurrentServes(n int) Option

WithMaxConcurrentServes caps concurrent peer blob-body serves. When the cap is reached, further blob GETs receive 429 Too Many Requests with a Retry-After hint so the requester re-selects another provider instead of queueing behind a saturated seed. That load-shedding is what lets the first finishers complete early and seed the swarm (the cascade). n <= 0 means unlimited. HEAD requests and manifest serves are never capped: they are cheap and are needed for discovery and verification.

func WithMetrics

func WithMetrics(onPeerServe, onPeerMiss func()) Option

WithMetrics registers metric callbacks.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server serves peer-fetch requests from the local content store.

func New

func New(store ifaces.LocalContentStore, opts ...Option) *Server

New builds a Server bound to the given local content store.

func (*Server) Handler

func (s *Server) Handler() http.Handler

Handler returns the HTTP handler. Callers wrap it with h2c via ListenAndServe or use it directly (e.g., httptest in unit tests, where HTTP/1.1 is sufficient because no client uses Range over multiplexed streams).

func (*Server) ListenAndServe

func (s *Server) ListenAndServe(addr string) (func(context.Context) error, error)

ListenAndServe runs the transfer server with h2c support on addr. Returns a function that gracefully shuts the server down.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL