Documentation
¶
Overview ¶
Package transfer provides minimal, fast blob transfer for tensor-based models.
This package is in x/ because the tensor model storage format is under development. It provides optimized transfer for models with many small blobs (tensor models) rather than few large blobs (typical LLMs).
TODO (jmorganca): Integrate into server/download.go and server/upload.go when stable.
Design Philosophy: This package is intentionally simpler than the main server's download/upload code. Key simplifications for many-small-blob workloads:
- Whole-blob downloads: Each blob downloads as one unit, with HTTP Range resume for blobs >= 64MB on retry; small blobs restart from scratch.
- Whole-blob uploads by default: A single PUT per blob. When the server returns a direct-upload URL the body goes straight to the storage backend; otherwise the body goes to the registry in one shot.
- Multi-part upload fallback: If the server requires it, blobs are split into parts and sent via PATCH with a finalize PUT carrying a composite etag. This is a server-side compatibility path, not the fast path.
- Inline hashing: digests computed during streaming.
- Stall and speed detection (downloads): cancels on no data (stall) or speed below 10% of median.
For large models (multi-GB), use the server's download/upload code which has resumable downloads with JSON state files, async hashing from OS page cache, and per-part speed tracking with rolling median.
Index ¶
Constants ¶
const ( DefaultDownloadConcurrency = 64 DefaultUploadConcurrency = 64 )
Default concurrency limits and settings
Variables ¶
This section is empty.
Functions ¶
Types ¶
type AuthChallenge ¶
AuthChallenge represents a parsed WWW-Authenticate challenge.
type Blob ¶
type Blob struct {
Digest string // sha256:...
Size int64
// From enables cross-repository blob mounting (upload only).
// When set, the upload will first attempt to mount the blob from this source
// repository instead of uploading the data. This is a Docker Registry v2 API
// feature that avoids re-uploading blobs that already exist elsewhere.
//
// Example: From="library/source-model" will add ?mount=<digest>&from=library/source-model
// to the POST /blobs/uploads/ request. If the registry returns 201 Created,
// the blob was mounted successfully and no upload is needed.
//
// See: https://distribution.github.io/distribution/spec/api/#cross-repository-blob-mount
From string
}
Blob represents a content-addressed blob to transfer.
type DownloadOptions ¶
type DownloadOptions struct {
Blobs []Blob // Blobs to download
BaseURL string // Registry base URL
DestDir string // Destination directory for blobs
Repository string // Repository path for blob URLs (e.g., "library/model")
Concurrency int // Max parallel downloads (default DefaultDownloadConcurrency)
BodyConcurrency int // Max simultaneous body-bearing transfers; 0 or negative serializes (capacity 1)
Progress func(completed, total int64) // Progress callback (optional)
Client *http.Client // HTTP client (optional, uses default)
Token string // Auth token (optional)
GetToken func(ctx context.Context, challenge AuthChallenge) (string, error) // Token refresh callback
Logger *slog.Logger // Optional structured logger
UserAgent string // User-Agent header (optional, has default)
StallTimeout time.Duration // Timeout for stall detection (default 10s)
}
DownloadOptions configures a parallel download operation.
type UploadOptions ¶
type UploadOptions struct {
Blobs []Blob // Blobs to upload
BaseURL string // Registry base URL
SrcDir string // Source directory containing blobs
Concurrency int // Max parallel uploads (default DefaultUploadConcurrency)
BodyConcurrency int // Max simultaneous body-bearing transfers; 0 or negative serializes (capacity 1)
Progress func(completed, total int64) // Progress callback (optional)
Client *http.Client // HTTP client (optional, uses default)
Token string // Auth token (optional)
GetToken func(ctx context.Context, challenge AuthChallenge) (string, error) // Token refresh callback
Logger *slog.Logger // Optional structured logger
UserAgent string // User-Agent header (optional, has default)
// Manifest fields (optional) - if set, manifest is pushed after all blobs complete
Manifest []byte // Raw manifest JSON to push
ManifestRef string // Tag or digest for the manifest (e.g., "latest", "sha256:...")
Repository string // Repository path for manifest URL (e.g., "library/model")
}
UploadOptions configures a parallel upload operation.