exporter

package
v0.32.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 6, 2026 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Overview

Package exporter provides typed HTTP helpers for the data-exporter API exposed by a running DataExport. The API has two endpoints: api/v1/block (block volumes served via http.ServeContent with Range support) and api/v1/files (filesystem volumes: trailing- slash paths return a JSON directory listing; other paths stream file bytes).

Index

Constants

View Source 
const DefaultIdleReadTimeout = 2 * time.Minute

DefaultIdleReadTimeout is the conservative default idle window applied to every data-plane response body a Fetcher hands out or consumes. It bounds how long a single Read may block WITHOUT receiving any bytes; it is not an overall transfer deadline, so a slow-but-flowing stream is never aborted. Override per Fetcher with WithIdleReadTimeout (tests use a short window).

Variables

View Source 
var ErrContentRangeMismatch = errors.New("server Content-Range does not match requested range")

ErrContentRangeMismatch is returned when a 206 response's Content-Range header does not cover the byte range the caller requested, so the body must not be trusted at the caller's intended offset.

View Source 
var ErrDataPlaneIdle = errors.New("data-plane read stalled: no bytes within idle timeout")

ErrDataPlaneIdle is reported by a Fetcher-issued response body when no bytes arrive for the configured idle window (see idleReadCloser). The data plane deliberately runs without an overall request deadline — volume transfers are long — so a TCP connection that stops delivering bytes WITHOUT erroring (NAT/LB half-close, wedged exporter pod) would otherwise block a Read (and the whole download) forever. It is wrapped with %w so the chunk/file retry+resume machinery treats a silent stall as an ordinary fetch error.

View Source 
var ErrExpired = errors.New("DataExport expired")

ErrExpired is returned by WaitReady when the DataExport enters the Expired terminal state and can no longer be used for data transfer.

View Source 
var ErrTargetRefMismatch = errors.New("existing DataExport targets a different object")

ErrTargetRefMismatch is returned by EnsureDataExport when a same-named DataExport CR already exists but its Spec.TargetRef names a DIFFERENT object than the request. This can occur after a hash collision or when a DataExport was created manually under the deterministic name. Reusing its endpoint would download the wrong object's bytes. EnsureDataExport refuses instead of silently reusing or deleting it.

View Source 
var ErrTargetUIDMismatch = errors.New("existing DataExport targets a different object UID")

ErrTargetUIDMismatch is returned by EnsureDataExport when an observed DataExport is not stamped for the exact Snapshot CR UID requested by the caller. The targetRef contract does not carry UID, so the client-owned annotation is the authoritative lifecycle discriminator.

View Source 
var ErrTargetUIDRequired = errors.New("snapshot target UID is required")

ErrTargetUIDRequired is returned when a caller attempts to open a DataExport lifecycle without the exact Snapshot CR UID.

Functions

func BlockURL 

func BlockURL(baseURL string) (string, error)

BlockURL returns the block-volume endpoint for a DataExport base URL. The block volume is served at api/v1/block.

func DataExportName 

func DataExportName(namespace, group, resource, kind, leafName string, targetUID types.UID) string

DataExportName derives a deterministic DataExport CR name from the canonical namespaced target identity, including the exact Snapshot CR UID. The readable leaf prefix is normalized only for display; the hash covers the original, unnormalized identity so identities that normalize alike remain distinct. The result is a DNS-1123 label no longer than 63 bytes, which also satisfies Kubernetes object-name limits.

func EnsureDataExport 

func EnsureDataExport(
	ctx context.Context,
	c client.Client,
	namespace,
	group,
	resource,
	kind,
	leafName,
	ttl string,
	opts ...EnsureOption,
) (*deapi.DataExport, error)

EnsureDataExport idempotently creates a DataExport in namespace targeting the snapshot leaf CR identified by {group, kind, leafName, target UID} with the given TTL (empty → "2h"). Returns the DataExport object (newly created or pre-existing).

group and kind must identify a namespaced snapshot CR (e.g. "snapshot.storage.k8s.io" / "VolumeSnapshot" for a CSI VolumeSnapshot leaf, or the domain group / kind for a domain snapshot CR). The controller routes any such targetRef through its kind-agnostic categorySnapshot path.

Pass WithRunOwner to scope ownership to a single download run: the run stamps its ID on any CR it creates and is warned when it adopts a CR another live run owns (see WithRunOwner and inv #10b). Without it, EnsureDataExport keeps its original ownership-agnostic behavior.

func FilesURL 

func FilesURL(baseURL string) (string, error)

FilesURL returns the filesystem root-listing endpoint for a DataExport base URL. The trailing slash instructs the server to return a directory listing.

func ReleaseDataExport 

func ReleaseDataExport(
	ctx context.Context,
	c client.Client,
	log *slog.Logger,
	acquisition *DataExportAcquisition,
) error

ReleaseDataExport deletes only the exact DataExport represented by acquisition. Deterministic name or matching run annotation alone never grants authority. UID, targetRef, target UID annotation, and observed owner must all still match; a foreign owner is never deleted. UID-precondition conflicts and NotFound are idempotent success.

func WaitReady 

func WaitReady(
	ctx context.Context,
	c client.Client,
	log *slog.Logger,
	namespace,
	deName string,
) (*deapi.DataExport, error)

WaitReady polls the DataExport named deName until:

  • its Ready condition is True and Status.URL is populated → returns the DE,
  • it is Ready=False with reason Expired → returns a wrapped ErrExpired,
  • ctx is cancelled or its deadline is exceeded → returns a wrapped ctx.Err() that includes the last observed DataExport status and an inspection hint.

The poll interval is 3 s. A log line is emitted on the first poll and every logEveryN polls (≈15 s) to avoid spamming output while the export initialises. Callers set a deadline via ctx to bound the wait.

Types

type DataExportAcquisition 

type DataExportAcquisition struct {
	// contains filtered or unexported fields
}

DataExportAcquisition is operation-scoped evidence that EnsureDataExport successfully acquired one exact DataExport object. Its fields are private so cleanup callers must obtain it from WithAcquisition rather than infer deletion authority from a deterministic name or run annotation.

func (*DataExportAcquisition) Name 

func (a *DataExportAcquisition) Name() string

Name returns the acquired DataExport name.

func (*DataExportAcquisition) TargetRef 

func (a *DataExportAcquisition) TargetRef() deapi.TargetRefSpec

TargetRef returns the exact targetRef observed when the DataExport was acquired.

func (*DataExportAcquisition) TargetUID 

func (a *DataExportAcquisition) TargetUID() types.UID

TargetUID returns the exact Snapshot CR UID bound to the acquisition.

func (*DataExportAcquisition) UID 

func (a *DataExportAcquisition) UID() types.UID

UID returns the exact UID observed when the DataExport was acquired.

type Doer 

type Doer interface {
	Do(req *http.Request) (*http.Response, error)
}

Doer executes a single HTTP request and returns the response. *http.Client and pkg/libsaferequest.SafeClient both satisfy this interface.

type EnsureOption 

type EnsureOption func(*ensureOptions)

EnsureOption configures optional behavior of EnsureDataExport.

func WithAcquisition 

func WithAcquisition(out **DataExportAcquisition) EnsureOption

WithAcquisition records operation-scoped cleanup evidence when EnsureDataExport successfully returns an exact DataExport. The output remains nil on every pre-acquisition failure, including a targetRef mismatch.

func WithRunOwner 

func WithRunOwner(runID string, log *slog.Logger) EnsureOption

WithRunOwner makes EnsureDataExport stamp runID as the owning run (runOwnerAnnotation) on any DataExport it CREATES, and log an explicit WARN via log when it instead adopts a live CR that a DIFFERENT run already owns. The adopted endpoint is still reused for read-only transfer, but ownership — and therefore the right to delete the CR on release (ReleaseDataExport) — stays with the other run, so neither run tears down the other's in-flight export (inv #10b). runID must be non-empty to take effect; a nil log disables the adoption WARN.

func WithTargetUID 

func WithTargetUID(targetUID types.UID) EnsureOption

WithTargetUID binds the DataExport lifecycle to the exact Snapshot CR incarnation. EnsureDataExport rejects calls that omit it.

func WithTerminatingWaitTimeout 

func WithTerminatingWaitTimeout(d time.Duration) EnsureOption

WithTerminatingWaitTimeout bounds the wait EnsureDataExport performs when it observes the DataExport already TERMINATING (DeletionTimestamp set) and must wait for it to fully vanish before recreating a fresh one. Without this cap the wait is bounded only by ctx, so a caller that passes a deadline-less ctx (e.g. the raw download-run ctx used by the pipeline's stamp-Ensure) would hang the whole run FOREVER on a wedged finalizer or a downed DataExport controller, with no output — a ctx that merely CAN carry a deadline is not enough (code-style §6). The pipeline derives d from the run's ReadinessTimeout. A non-positive d leaves the wait bounded by ctx alone (the pre-existing behavior for callers that do not opt in).

type Export 

type Export struct {
	// contains filtered or unexported fields
}

Export holds a resolved DataExport: a ready HTTP endpoint (Fetcher), the VolumeMode reported by the controller, and the internal base URL. Callers release the underlying DataExport CR via ReleaseDataExport + DataExportName using the leaf name they already have, rather than through this value — releasing by deterministic name also covers the case where OpenExport never returned an Export at all (e.g. cancelled while still waiting for Ready).

func NewExport 

func NewExport(
	namespace,
	deName,
	volumeMode,
	baseURL string,
	fetcher *Fetcher,
	httpClients ...IdleConnectionCloser,
) *Export

NewExport constructs an Export from pre-built components. Any supplied HTTP clients become owned by the Export and are released by CloseIdleConnections. It is intended for testing and alternative transport implementations that bypass the production DataExport lifecycle.

func OpenExport 

func OpenExport(
	ctx context.Context,
	log *slog.Logger,
	c client.Client,
	namespace,
	group,
	resource,
	kind,
	leafName,
	ttl string,
	sc *transport.Client,
	opts ...EnsureOption,
) (*Export, error)

OpenExport creates (or re-uses) a DataExport targeting the snapshot leaf identified by {group, kind, leafName}, waits until it is Ready, and returns an Export ready for data transfer.

An isolated copy of sClient is built for the HTTP Fetcher so that CA injection does not mutate the caller's client.

opts are forwarded verbatim to the inner EnsureDataExport. Callers pass WithRunOwner so that if the deterministic de-<leaf> CR has to be RECREATED here (e.g. it vanished between the pipeline's stamp-Ensure and this inner Ensure), the fresh CR is stamped with this run's ownership rather than left unstamped — closing the per-run ownership gap in the vanish window (inv #10b).

func (*Export) BaseURL 

func (e *Export) BaseURL() string

BaseURL returns the internal base URL of the data-exporter server.

func (*Export) CloseIdleConnections 

func (e *Export) CloseIdleConnections()

CloseIdleConnections releases every HTTP connection pool owned by the Export.

func (*Export) Fetcher 

func (e *Export) Fetcher() *Fetcher

Fetcher returns the HTTP Fetcher wired to the data-exporter endpoint.

func (*Export) VolumeMode 

func (e *Export) VolumeMode() string

VolumeMode returns the volume mode reported by the DataExport controller ("Block" or "Filesystem").

type Fetcher 

type Fetcher struct {
	// contains filtered or unexported fields
}

Fetcher wraps a Doer and exposes typed methods for the data-exporter HTTP API.

func NewFetcher 

func NewFetcher(doer Doer, opts ...FetcherOption) *Fetcher

NewFetcher creates a Fetcher backed by the given Doer. Unless overridden via WithIdleReadTimeout, response bodies carry an idle-read watchdog with DefaultIdleReadTimeout.

func (*Fetcher) GetFile 

func (f *Fetcher) GetFile(ctx context.Context, fileURL string) (io.ReadCloser, error)

GetFile GETs fileURL and returns the response body for streaming. The caller must close the returned ReadCloser.

func (*Fetcher) HeadVolume 

func (f *Fetcher) HeadVolume(ctx context.Context, blockURL string) (int64, error)

HeadVolume issues a HEAD request to blockURL and returns the total content length in bytes.

func (*Fetcher) ListDir 

func (f *Fetcher) ListDir(ctx context.Context, filesURL string, yield func(Item) error) error

ListDir GETs filesURL (which must end with a trailing slash for directory semantics), requests only the inexpensive stat attributes, and calls yield once per stream-decoded directory entry. It retains only the current item, so a flat directory's memory use is independent of its entry count. Source hashes are fetched separately after each regular file's declared size is known because the producer computes hash.md5 synchronously before emitting that listing item.

func (*Fetcher) RangeGet 

func (f *Fetcher) RangeGet(ctx context.Context, blockURL string, start, end int64) (io.ReadCloser, error)

RangeGet issues a GET request with a Range: bytes=start-end header to blockURL and returns the response body. The caller must close the returned ReadCloser. Returns an error unless the server responds with 206 Partial Content AND its Content-Range header confirms the returned body actually covers [start, end]: the block exporter (storage-volume-data-manager images/data-exporter/internal/export_block/ handler.go HandleGetMethod) serves the block device via stdlib http.ServeContent, which always sets Content-Range on a 206 response, so a missing or mismatched header means a misbehaving server/proxy returned bytes from the wrong offset (or the whole object) and must not be trusted at the caller's intended offset.

func (*Fetcher) SourceMD5 

func (f *Fetcher) SourceMD5(ctx context.Context, fileURL string, size int64) (string, error)

SourceMD5 retrieves the producer-computed plaintext MD5 for one regular file through the filesystem exporter's HEAD attribute contract. The producer must read the complete source file before it can send this response header, so the request uses an overall size-derived deadline rather than the transfer body's progress-based idle watchdog.

The budget assumes the source can be hashed at no less than 1 MiB/s, adds one minute of fixed scheduling slack, floors small or unknown sizes at five minutes, and caps untrusted declared sizes at seven days. It is finite for every int64 size. An empty result means an older exporter returned 200 without the optional hash header.

type FetcherOption 

type FetcherOption func(*Fetcher)

FetcherOption customizes a Fetcher at construction time.

func WithIdleReadTimeout 

func WithIdleReadTimeout(d time.Duration) FetcherOption

WithIdleReadTimeout sets the idle-read watchdog window for response bodies this Fetcher issues. A value <= 0 disables the watchdog (bodies are returned unwrapped). When unset, DefaultIdleReadTimeout applies.

func WithSourceHashDoer 

func WithSourceHashDoer(doer Doer) FetcherOption

WithSourceHashDoer sets the transport used for source-hash HEAD requests. Production uses a separate transport because source hashing may legitimately take longer than the ordinary data-plane response-header timeout.

type IdleConnectionCloser 

type IdleConnectionCloser interface {
	CloseIdleConnections()
}

IdleConnectionCloser owns an HTTP connection pool used by an Export.

type Item 

type Item struct {
	Name       string         `json:"name"`
	Type       string         `json:"type"`
	URI        string         `json:"uri"`
	TargetPath string         `json:"targetPath,omitempty"`
	Attributes map[string]any `json:"attributes"`
}

Item is one entry returned by the data-exporter filesystem listing API.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.