Documentation
¶
Overview ¶
Package transfer is the peer-facing OCI endpoint other Gantry agents pull from. It binds `:5001` over HTTP/2 cleartext (h2c) because NetworkPolicy restricts the port to inter-node traffic and TLS termination would duplicate the libp2p Noise security that Gantry already does for coordination RPCs.
Design (architecture.md the API contract):
- Same OCI URL shape as the mirror: `/v2/<repo>/blobs/<digest>` and `/v2/<repo>/manifests/<digest>`. Tag-shaped manifest requests at this endpoint return **404 unconditionally** - peers must already know the digest (the tag-resolution path runs through the mirror, not here). - Requires `Gantry-Mirrored: 1` request header. Without the header the server returns 400. With the header, the response semantics are: **serve from the local store or return 404**. No DHT lookup, no HRW probe, no `please_pull`, no origin contact. The header is the loop-breaker that prevents two agents from recursing into each other's miss paths. - `Range: bytes=N-M` returns `206 Partial Content` with the correct `Content-Range`. Resumable peer callers use this to continue a live stream from another provider without retransmitting the verified prefix; the contract is preserved for v2 striping. - Metric `p2p_peer_serve_total` is bumped per served body - not `p2p_cache_hit_total`, so cluster scrapes distinguish containerd- facing hits from peer-facing serves.
Index ¶
- Constants
- type Client
- func (c *Client) FetchFromPeer(ctx context.Context, peerAddr string, ref ifaces.OriginRef) (io.ReadCloser, int64, string, error)
- func (c *Client) FetchRangeFromPeer(ctx context.Context, peerAddr string, d digest.Digest, ...) (io.ReadCloser, int64, string, error)
- func (c *Client) HeadFromPeer(ctx context.Context, peerAddr string, ref ifaces.OriginRef) (int64, string, error)
- type ClientOption
- type Describer
- type Option
- func WithByteMetrics(onPeerServeBytes func(kind string, bytes int64)) Option
- func WithDescriber(d Describer) Option
- func WithLogger(l *slog.Logger) Option
- func WithMaxConcurrentRangeServes(n int) Option
- func WithMaxConcurrentServes(n int) Option
- func WithMetrics(onPeerServe, onPeerMiss func()) Option
- func WithRangeSource(rs RangeSource) Option
- type RangeSource
- type Server
Constants ¶
const MirroredHeader = "Gantry-Mirrored"
MirroredHeader is the OCI-extension header peers MUST include on every transfer-endpoint request. It is the loop-breaker described in the design doc.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client implements ifaces.PeerDialer over HTTP/2 cleartext (h2c). Reuse a single Client across all peers - the underlying http.Transport pools per-host connections internally.
func NewClient ¶
func NewClient(opts ...ClientOption) *Client
NewClient builds a Client tuned for peer fetches.
func (*Client) FetchFromPeer ¶
func (c *Client) FetchFromPeer(ctx context.Context, peerAddr string, ref ifaces.OriginRef) (io.ReadCloser, int64, string, error)
FetchFromPeer implements ifaces.PeerDialer.
func (*Client) FetchRangeFromPeer ¶ added in v0.23.0
func (c *Client) FetchRangeFromPeer(ctx context.Context, peerAddr string, d digest.Digest, requested httprange.Range) (io.ReadCloser, int64, string, error)
FetchRangeFromPeer fetches one exact byte range from a peer holding a complete, committed blob. Registry authorization is deliberately not forwarded: digest and range are sufficient at the peer boundary.
type ClientOption ¶
type ClientOption func(*clientOptions)
ClientOption tweaks Client construction.
func WithClientByteMetrics ¶ added in v0.2.1
func WithClientByteMetrics(onBytesRead func(kind string, bytes int64)) ClientOption
WithClientByteMetrics registers a callback for bytes actually read from peer response bodies, including partial failed transfers and retries.
func WithDialTimeout ¶
func WithDialTimeout(d time.Duration) ClientOption
WithDialTimeout sets the TCP dial timeout per peer.
func WithReadIdleTimeout ¶
func WithReadIdleTimeout(d time.Duration) ClientOption
WithReadIdleTimeout configures the h2 ping-based idle stall detector.
func WithRequestTimeout ¶
func WithRequestTimeout(d time.Duration) ClientOption
WithRequestTimeout caps total time per bulk or metadata request. Exact-range requests use a separate, shorter timeout.
type Describer ¶
Describer is an optional capability that callers (typically the containerdstore.Store) may implement to surface cached media-type metadata for a digest. The transfer endpoint uses it to set a correct manifest Content-Type instead of falling back to application/octet-stream - per "Descriptor and media-type handling": "Do not serve manifest digests with a generic content type unless existing containerd/client behavior proves it is safe."
An empty return value means "unknown"; callers MUST fall back to the kind-based default rather than treating it as an error.
type Option ¶
type Option func(*Server)
Option configures a Server.
func WithByteMetrics ¶ added in v0.2.1
WithByteMetrics registers a callback for bytes actually transmitted to peer agents. HEAD requests emit no bytes, and Range requests count only the range body sent on the wire.
func WithDescriber ¶
WithDescriber registers an optional media-type lookup for cached digests. Wire this with the containerdstore.Store so manifest responses carry the correct OCI/Docker media type instead of application/octet-stream.
func WithMaxConcurrentRangeServes ¶ added in v0.23.0
WithMaxConcurrentRangeServes caps concurrent serves of small, explicitly bounded ranges separately from full-blob streams. Artifact Streaming issues these for OverlayBD index reads: they are tiny and latency-critical, so the per-stream bandwidth guard that WithMaxConcurrentServes exists to provide is the wrong control for them. n <= 0 leaves small ranges on the blob cap.
func WithMaxConcurrentServes ¶ added in v0.1.23
WithMaxConcurrentServes caps concurrent peer blob-body serves. When the cap is reached, further blob GETs receive 429 Too Many Requests with a Retry-After hint so the requester re-selects another provider instead of queueing behind a saturated seed. That load-shedding is what lets the first finishers complete early and seed the swarm (the cascade). n <= 0 means unlimited. HEAD requests and manifest serves are never capped: they are cheap and are needed for discovery and verification.
func WithMetrics ¶
func WithMetrics(onPeerServe, onPeerMiss func()) Option
WithMetrics registers metric callbacks.
func WithRangeSource ¶ added in v0.23.0
func WithRangeSource(rs RangeSource) Option
WithRangeSource lets the server answer Range requests for blobs it holds only partially, so cached OverlayBD index reads become peer-servable.
type RangeSource ¶ added in v0.23.0
type RangeSource interface {
OpenRange(ctx context.Context, d digest.Digest, requested httprange.Range) (io.ReadCloser, int64, error)
}
RangeSource serves byte ranges of blobs that are not present in full.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server serves peer-fetch requests from the local content store.
func New ¶
func New(store ifaces.LocalContentStore, opts ...Option) *Server
New builds a Server bound to the given local content store.
func (*Server) Handler ¶
Handler returns the HTTP handler. Callers wrap it with h2c via ListenAndServe or use it directly (e.g., httptest in unit tests, where HTTP/1.1 is sufficient because no client uses Range over multiplexed streams). Handler returns the peer-facing http.Handler.
Wrapping the mux in otelhttp makes this node a link in the requester's trace rather than a dead end: the requesting agent's transfer client injects traceparent, and the propagator here extracts it so the serve span becomes a child of the requester's fetch span. A swarm-wide cascade then renders as one trace with one span per seeding hop.
Source Files