Documentation
¶
Overview ¶
Package fetch is the per-replica fill orchestrator: per-ChunkKey singleflight, pre-header origin retry, per-replica origin concurrency cap, and cross-replica fill via the cluster's internal RPC.
The dedup model is per-replica singleflight + cluster-wide dedup via a rendezvous-hashed coordinator. No disk spool; joiners stream from the leader's in-memory ring buffer.
Pre-header retry: the coordinator may retry origin GETs up to the budget in cfg.Origin.Retry until the first byte is committed to the client response. Once headers are sent retries are not safe and failures become mid-stream aborts.
Index ¶
- type Coordinator
- func (c *Coordinator) FillForPeer(ctx context.Context, k chunk.Key, objectSize int64) (io.ReadCloser, error)
- func (c *Coordinator) GetChunk(ctx context.Context, k chunk.Key, objectSize int64) (io.ReadCloser, error)
- func (c *Coordinator) HeadObject(ctx context.Context, bucket, key string) (origin.ObjectInfo, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Coordinator ¶
type Coordinator struct {
// contains filtered or unexported fields
}
Coordinator orchestrates per-replica chunk fills.
func NewCoordinator ¶
func NewCoordinator( or origin.Origin, cs cachestore.CacheStore, cl *cluster.Cluster, cat *chunkcatalog.Catalog, mc *metadata.Cache, cfg *config.Config, log *slog.Logger, ) *Coordinator
NewCoordinator wires up the fetch coordinator. The log is used for peer-fallback warnings and commit-after-serve failure traces, plus debug-level tracing through every chunk-resolution decision point when the operator enables logging.level: debug. The caller (usually app.Start) injects the app-wide slog.Logger so fetch-path logs are unified with the rest of the runtime's output. Passing nil falls back to slog.Default().
func (*Coordinator) FillForPeer ¶
func (c *Coordinator) FillForPeer(ctx context.Context, k chunk.Key, objectSize int64) (io.ReadCloser, error)
FillForPeer is the path taken by the /internal/fill handler.
The receiver becomes the leader for this fill (or joins an in-flight fill for the same key). Returns a streaming body of the entire chunk.
func (*Coordinator) GetChunk ¶
func (c *Coordinator) GetChunk(ctx context.Context, k chunk.Key, objectSize int64) (io.ReadCloser, error)
GetChunk returns a reader over the chunk's bytes, fulfilling either from CacheStore (hit) or by orchestrating a cluster-wide dedup'd fill (miss).
objectSize is the authoritative size of the object the chunk belongs to (from origin Head). It is used to clamp the cachestore read length and to size the tail chunk correctly on a miss.
On miss:
- If self is the coordinator: run local fill (origin GET via retry, commit to CacheStore, populate buffer for joiners).
- If a peer is the coordinator: send /internal/fill to that peer; stream from peer's response. On 409 Conflict, fall back to local fill.
func (*Coordinator) HeadObject ¶
func (c *Coordinator) HeadObject(ctx context.Context, bucket, key string) (origin.ObjectInfo, error)
HeadObject returns object metadata, satisfying client HEAD requests.
Rejects responses with an empty ETag via origin.MissingETagError. chunk.Path encodes the ETag in its hash input; a stable cache key requires the origin to supply one. Without an ETag, two different versions of the same (bucket, key) would alias to the same chunk.Path and serve stale bytes silently. The negative result is cached at NegativeTTL so we do not re-Head a misconfigured origin on every request.
Source Files