Documentation
¶
Overview ¶
Package fetch is the client download path of the files subsystem (SYN-28): resolve a rootCid through the block-source ladder — local store → peer (SYN-24 seam) → public CARv2 GET with HTTP Range — and expose verified plaintext as a seekable reader.
Downloads are public-read first (SYN-32): for a durable file the object URL is {publicReadBaseUrl}/blob/{spaceId}/{rootCid}, built offline from the cached base; there is no per-file broker round-trip. Every fetched block is cid-verified and persisted into the local sparse CARv2 before use, so streaming, seeking and interrupted downloads all accrete toward a complete local copy.
Index ¶
Constants ¶
const MaxPeerReadLen = 8 << 20
MaxPeerReadLen is the largest single read a peer serves; the FileP2P server rejects anything longer outright. The fetcher coalesces at most maxFetchSpan (4 MiB) per read, so this is generous headroom, and it bounds a peer read's budget no matter what object size the peer itself reported.
Variables ¶
var ErrNotAvailable = fmt.Errorf("filefetch: %w", space.ErrFileNotAvailable)
ErrNotAvailable — the content is not local and cannot be fetched: the file is not durable (P2P arrives with SYN-24) or the network has no public read base. Wraps the public sentinel so SDK consumers match it with errors.Is(err, space.ErrFileNotAvailable).
var ErrRemoteGone = errors.New("filefetch: object not available at public url")
ErrRemoteGone — the object is not (or not yet) readable at its public URL.
Functions ¶
This section is empty.
Types ¶
type BaseURL ¶
BaseURL resolves the network's public read base ("" = none). The SDK wires a persisted-cache provider over the broker Info RPC.
func NewBaseURL ¶
func NewBaseURL(st *store.Store, networkId, override string, info func(ctx context.Context) (string, error)) BaseURL
NewBaseURL builds the standard BaseURL provider: a config override wins outright (private deployments, tests); otherwise the value is read from the store KV (persisted at first resolve, so URL construction works offline from then on) and fetched once via info (the broker Info RPC) when never seen. An empty answer from info is valid ("no public read") and is cached in memory only — the next process retries, so a network that turns public read on is picked up on restart without a TTL scheme.
type CarSource ¶
type CarSource interface {
// ReadRange fetches exactly [off, off+length) (short only at EOF,
// which the callers treat as an error outside the head probe).
ReadRange(ctx context.Context, off, length int64) ([]byte, error)
// ReadProbe fetches the head range and the whole-object size.
ReadProbe(ctx context.Context) (head []byte, total int64, err error)
}
CarSource range-reads one immutable CARv2 object. Both the HTTP public-read reader (remoteCar) and the LAN peer reader (the p2p package's peerCar) implement it, so seed and remoteFetcher are source-agnostic. Exported so the p2p package can supply a peer source.
type File ¶
type File struct {
// contains filtered or unexported fields
}
File is an open verified-plaintext view of one stored file.
type PeerReadBudgeter ¶
type PeerReadBudgeter interface {
// PeerReadBudget returns the round-trip allowance for one read and
// the transfer rate (bytes/s) the peer is expected to sustain; the
// read's budget is roundTrip + length/rate. A zero roundTrip takes
// the LAN default; a zero rate charges nothing per byte.
PeerReadBudget() (roundTrip time.Duration, bytesPerSecond int64)
}
PeerReadBudgeter is the optional interface a CarSource implements to state a read budget other than the LAN default. A read is one round trip plus a transfer, and both differ by orders of magnitude between a LAN and a relayed internet path, so a single fixed figure either expires mid-range on the largest reads or wastes seconds on a stalled 4 KiB probe. Where there is no HTTP fallback an expiry is the whole fetch, so a file that a peer holds in full never arrives.
type PeerSource ¶
type PeerSource interface {
SourceFor(ctx context.Context, spaceId string, root cid.Cid) (src CarSource, ban func(), ok bool)
}
PeerSource selects a peer that holds a file in full — on the local network or, more slowly, across the internet. It returns a CarSource that reads the file's CAR object from that peer, plus a ban hook the fetcher calls when the peer serves INVALID bytes (so a bad peer is not re-selected). ok=false — no peer holds the file / p2p disabled — means the fetch uses the public GET only. Selection is bounded as a whole, not only per candidate: a relayed FileCheck is orders of magnitude slower than a LAN one, so an unbounded sweep would spike latency exactly when no peer has the file.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service is the download orchestrator. One per SDK.
func (*Service) Fetch ¶
func (s *Service) Fetch(ctx context.Context, spaceId string, root cid.Cid, durable bool, ref string) error
Fetch pulls every missing block of (spaceId, root) into the local store (the full-download path behind Pin and tests). No-op when already complete.
func (*Service) Open ¶
func (s *Service) Open(ctx context.Context, spaceId string, root cid.Cid, key []byte, durable bool, ref string) (*File, error)
Open returns a seekable plaintext reader over (spaceId, root), decrypting with the file key. Not-local content is seeded from the public object (header + index → sparse CAR) and blocks stream in on demand; durable=false with no local bytes is ErrNotAvailable. ref is recorded on the local row (refcount for GC).
func (*Service) SetPeer ¶
func (s *Service) SetPeer(p PeerSource)
SetPeer wires the LAN p2p source, consulted before the public GET. Call once during SDK boot; nil leaves fetches public-read only.
Source Files