Help us shape the future of Go! Take the 2026 Go Developer Survey

storage

package
v0.7.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 25, 2026 License: Apache-2.0 Imports: 46 Imported by: 0

Documentation

Overview

Package storage provides provider selection, immutable storage targets, and single- or multi-copy upload orchestration.

Contexts

ProviderContext identifies one provider but no data set. Its CreateAndAdd operations create a new data set, and its Pull operations fetch pieces for a later create-and-add. DataSetContext identifies one provider and one existing data set; its Commit and Pull operations always use that data set. Neither type changes target after construction.

Both types expose provider-scoped operations such as Store and Download. Download therefore behaves the same on both: it retrieves the requested piece from the configured provider or CDN. Data-set inspection, single or batch deletion, and termination methods are available only on DataSetContext, while standalone data-set creation and recovery are available only on ProviderContext.

Use Service.NewProviderContext or Service.NewDataSetContext when the target ID is already known. Use Service.SelectProviderContext to choose one healthy provider without looking up data sets. Use Service.SelectUploadContexts when preparing a new upload; it may reuse writable data sets whose metadata matches. On known deployments, compact matches take precedence over legacy matches, including empty compact data sets. Within each layout, active pieces take precedence when activity is available, then the lowest ID. Explicitly bound targets and already shared data sets keep their identity.

DataSetRef is the persistent target reference shared by creation results, DataSetContext, and ProviderContext.ForDataSet. Construct it with NewDataSetRef; its zero value is invalid.

DataSetDetails exposes its base data-set fields directly. Reading a promoted field from a zero DataSetDetails does not panic; each field retains its ordinary zero value, including nil pointer and map fields. Check DataSetID and ProviderID when a complete identity is required.

Persistent data-set references, context identities, and commit status and result values use strict lowerCamel JSON field names. Alternate capitalization, unknown or duplicate fields, and incomplete objects are rejected. Submission values are runtime results, not persistence schemas.

Upload batching

The root synapse Client enables commit batching for high-level Upload calls by default. Ready pieces are submitted together once no other upload to the same target is in progress and three seconds pass without a new piece. There is no time limit by default, so a slow upload, including a copy waiting for a busy provider to accept its pull, delays the others for that target. At most four batches are signed and submitted at once; provider confirmation waits do not consume that limit. Configure the root coordinator with synapse.WithUploadBatching, or disable it with synapse.WithoutUploadBatching.

WithUploadIdleWait changes the three-second delay. WithUploadMaxWait ends batch formation at most that long after its first piece is ready. The batch then queues for submission; signing, provider requests, and confirmation are outside this limit. WithoutUploadIdleWait with a zero WithUploadMaxWait queues every piece immediately. With both timers disabled, a batch is submitted only by Service.Flush, the 80-piece legacy limit, a candidate exceeding the message size limit, or a repeated piece CID.

Compatible uploads share a window only when their immutable target matches. Existing data sets match by provider, complete DataSetRef, and exact service URL. New data sets also require the same payee, CDN setting, and data-set metadata. Uploads to the same new target share one data set for the coordinator's lifetime, across windows and sequential uploads, without binding or mutating the source ProviderContext. Different layout cutoffs split windows but still share the same data set. Unbound targets reserve creation space in every window so a terminated shared data set can be replaced without exceeding the message-size limit.

Direct add/pull requests to legacy data sets accept at most 80 pieces. New and compact data sets have no fixed count limit; every request must fit pdp.MaxAddPiecesMessageSize, including signatures and metadata. Low-level methods return errors rather than splitting requests. The root client sets the deployment cutoff automatically. Standalone contexts derive it from WithChainID; WithLegacyPieceStorageIDLimit overrides it, including zero to skip legacy checks. Standalone resolvers use ServiceResolverOptions.LegacyPieceStorageIDLimit and forward it through ContextFactoryOptions.LegacyPieceStorageIDLimit to the caller's factory.

Store, Pull, target resolution, and coordinator admission use the caller context. Successful admission transfers ownership of the piece to the coordinator. Later caller cancellation stops that caller's wait and suppresses later callbacks, but the coordinator continues signing, submission, and confirmation. An Upload context error therefore does not prove that the accepted piece will not commit; reconcile external state before retrying. A terminal result already published when cancellation races takes precedence over the context error.

Each successful provider submission has a five-minute confirmation deadline. WithUploadCommitTimeout changes this limit; WithoutUploadCommitTimeout disables it. Batch formation, queueing, signing, submission, Store, and Pull are outside this deadline. UploadCommitTimeoutError retains the submission handle for continued confirmation with DataSetContext.WaitForCommit or ProviderContext.WaitForCreateAndAdd. A timeout does not cancel the transaction.

Service.Flush waits for uploads that began before the call to reach the coordinator, submits their corresponding windows, and waits for final confirmations. A later compatible upload may join one of those windows, so later uploads are neither guaranteed to be included nor excluded. A waiter that receives a terminal result consumes that piece's failure, so a later Flush does not report it again. Flush still reports failures that no waiter observed. Canceling the Flush context stops only that wait and does not discard those unobserved failures. A completed Flush publishes terminal results before returning. The root [synapse.Client.Close] closes its coordinator without flushing, so call Service.Flush first when a graceful drain is required. Close cannot retract a provider submission that already succeeded.

Standalone users can construct UploadBatcher with NewUploadBatcher and inject it through Options.UploadBatcher or WithUploadBatcher. The caller owns an injected coordinator and must flush or close it. With no injected coordinator, standalone services and contexts retain immediate per-upload commits.

During a multi-copy Service upload, the primary and every secondary participate in batching. Replicas from one Upload call are not guaranteed to share a transaction. Low-level Store, Pull, and commit lifecycle methods keep their direct behavior; only high-level Upload methods use the coordinator.

The Go API intentionally uses an explicitly owned coordinator, functional options, context-aware Flush, and immutable context injection rather than copying the upstream TypeScript SDK's configuration and lifecycle shape.

Signing and payer identity

Storage authorization uses the signer.StorageSigner capability: an EVM address plus 32-byte hash signing. Context constructors accept a nil or typed-nil signer so read-only contexts remain usable; operations that require a signature then return ErrInvalidArgument. WithPayer configures the paying account independently from the signer.

Standalone Service configuration has two additional payer inputs. Options.PayerAddress applies to manager-level helpers and defaults to the configured signer address only when left zero. Set it explicitly when a delegated signer acts for another payer. ServiceResolverOptions.Payer independently determines the payer assigned to contexts created by that resolver. The root synapse Client keeps both values on the root account when synapse.WithStorageSigner configures a delegated signer.

Downloads and deletion

Service.Download accepts exactly one of DownloadOptions.Context, URL, or DataSetID. Downloading by data set ID reads its ownership and CDN metadata. CDN-enabled data sets can be read even when their provider is inactive; provider fallback and writable contexts still require an active PDP product. Data-set and context downloads require PieceCIDv2. All download paths enforce its raw size and verify integrity at the response's actual EOF. Always check the final Read or io.ReadAll error; closing early skips verification.

The root Client configures the data-set download path automatically. Standalone services set Options.DataSetDownloader to their ServiceResolver and provide ServiceResolverOptions.CDNRetriever for the resolver's Payer.

DataSetContext.PieceStatus, DeletePiece, and DeletePieces require PieceCIDv2; other CIDs return ErrInvalidArgument. DeletePieces deduplicates complete PieceCIDv2 values, including their raw sizes. DataSetContext.DeletePieces and DeletePiecesByID have no fixed count limit. They submit one deletion request without splitting or retrying; contract, gas, and transaction limits still apply. Deletion schedules removal rather than immediately erasing the pieces. MaxDeletePiecesBatchSize is deprecated.

Provider selection

Automatic upload selection requires an endorsed primary by default. The primary is selected from providers that are endorsed, FWSS-approved, registry-active, and healthy. Secondary copies use the complete approved, active, healthy pool after excluding the primary. Set UploadOptions.AllowUnendorsedPrimary or SelectUploadContextsOptions.AllowUnendorsedPrimary to true to use the complete approved pool for the primary and skip the endorsement query. Service.SelectProviderContext, explicit provider or data-set contexts, and replacement selection do not query endorsements.

Standalone ServiceResolver users configure the single-method EndorsedProviderSource through ServiceResolverOptions.Endorsements. With that source configured, the zero-value selection policy is strict. Without it, strict selection returns ErrEndorsementsNotConfigured when upload selection runs; setting AllowUnendorsedPrimary continues normally with the approved pool. A nil slice with a nil source error is an empty set, while source errors remain query errors. Standalone callers may provide their own EndorsedProviderSource implementation.

Upload flow

Service.Upload automatically selects targets and performs store, pull, and commit. Copies must be explicitly positive. If fewer targets are available, the upload continues with those targets and reports the requested and actual copy counts through UploadResult. Configure this path with UploadOptions. Primary commits overlap secondary pulls; each ready secondary enters commit independently. Options.PullConcurrency limits secondary copy workflows per upload. Root clients configure it with synapse.WithUploadPullConcurrency. Options.CommitConcurrency limits direct commits through confirmation; batched submissions use the coordinator's own limit and waiting policy.

Secondary callbacks and OnPiecesAdded may overlap across providers. Callbacks must synchronize shared application state and return promptly. No global callback order is guaranteed. OnPiecesConfirmed remains sequential, in target order, after all upload workers finish, and is suppressed after cancellation. After primary storage succeeds, cancellation preserves confirmed copies. With at least one confirmed copy, it returns an UploadResult and nil error; check UploadResult.Complete. With none confirmed, it returns a CommitError. See UploadOptions for callback panic handling.

For an exact preflight and upload, use the same context instances throughout:

  1. Call Service.SelectUploadContexts. A non-empty partial selection is returned with InsufficientUploadContextsError.
  2. Pass the selected contexts to Service.Prepare.
  3. Execute the returned PrepareTransaction, if any.
  4. Pass the same contexts, in the desired primary-to-secondary order, to Service.UploadToContexts.

Cost estimates use costs.MultiContextCosts. Service.CalculateMultiContextCosts returns the estimate directly; Service.Prepare includes it in PrepareResult.Costs. To use a precomputed estimate, set PrepareOptions.Costs without any other preparation options. Otherwise, supply each piece's raw payload size in PrepareOptions.PieceSizes. Every size must be between chain.MinUploadSize and chain.MaxUploadSize. The same plan applies to each context; piece count is derived from the list. Existing contexts require Options.DataSetLeafCountReader. The root client assembles it when the resolved contract topology includes PDPVerifier. Services without that reader return ErrUninitialized instead of estimating existing usage as zero. New contexts and precomputed costs do not need it. Reader results must be non-nil and non-negative; zero means known empty. Read errors retain their original cause and unavailable datasets retain ErrDataSetUnavailable. Invalid successful reader results return ordinary errors, rather than caller argument or unavailable-dataset errors.

UploadToContexts does not select replacements. The first context stores the reader; later contexts pull from it, waiting while a provider's pull queue is full. Configure this path with UploadToContextsOptions. Service.Upload retains automatic replacement for failed secondary copies and replaces a provider whose pull queue is full without waiting. Direct ProviderContext.Upload and DataSetContext.Upload calls store one copy and accept ContextUploadOptions. StorageContext is the sealed, ordered mixed-target view used by selection, preparation, and UploadToContexts; commit lifecycle methods remain on the concrete context whose target determines their meaning.

Contexts carry an immutable ContextIdentity containing payer, chain, and record-keeper identities. Service validates this identity before cost calculation or upload work, preventing contexts from another account or chain from being used accidentally.

Submission recovery

When creation must remain recoverable even if the provider receives the request but its HTTP response is lost, choose and persist a client data-set ID before submitting. Persist the provider ID and context identity beside it, then rebuild the same ProviderContext before looking up the result:

clientDataSetID := types.NewBigInt(123)
providerID := providerContext.ProviderID()
identity := providerContext.ContextIdentity()
// Persist clientDataSetID, providerID, and identity before this call.
_, err := providerContext.CreateDataSet(ctx, &CreateDataSetOptions{
	ClientDataSetID: &clientDataSetID,
})
// After an ambiguous error, rebuild the same provider context and poll:
ref, found, err := freshProviderContext.FindDataSetByClientDataSetID(ctx, clientDataSetID)

A create-and-add request uses CreateAndAddRequest.ClientDataSetID in the same way. Persist the operation kind and piece CIDs as application state too, because finding the data set does not prove that its pieces were added. A false found result means the matching data set is not visible in the current chain state; it is not evidence that the provider rejected the request. Applications own polling and deadline policy.

A nil ClientDataSetID preserves the default random-ID behavior, but an ID generated inside a request cannot be recovered if the provider handle is lost. Client data-set IDs and add-pieces nonces share a payer-scoped on-chain namespace. Never reuse a consumed value, including after data-set deletion. The ID is a correlation key, not an HTTP idempotency key; create POSTs are sent once and are not automatically retried.

ProviderContext.CreateDataSet leaves its receiver unbound. When confirmation must survive a restart, persist StatusURL and ClientDataSetID from the CreateDataSetSubmission received through CreateDataSetOptions.OnSubmitted. A fresh ProviderContext for the same provider can pass those values to ProviderContext.WaitForDataSetCreated. Pass the returned DataSetRef to ProviderContext.ForDataSet to obtain a DataSetContext. Waiting does not require an FWSS reader.

ProviderContext.CreateAndAdd and DataSetContext.Commit are convenience methods that submit once and wait for confirmation. Their OnSubmitted callback receives an independent runtime CommitSubmission after the provider handle is validated and before confirmation starts. For create-and-add, persist StatusURL and the original ClientDataSetID. For add-pieces, persist StatusURL with the target DataSetRef. For explicit recovery control, prefer SubmitCreateAndAdd followed by WaitForCreateAndAdd on ProviderContext, or SubmitCommit followed by WaitForCommit on DataSetContext.

GetCreateAndAddStatus and GetCommitStatus perform one logical status check and return CommitStatePending, CommitStateConfirmed, or CommitStateRejected. A rejected status is returned without an error; WaitForCreateAndAdd and WaitForCommit report the same terminal state as CommitRejectedError.

A failed upload carries its validated submission handle in UploadResult.FailedAttempts or CommitError.FailedAttempts when the SDK registered it before the attempt ended. Cancellation, Close, or a lost response can leave Submission nil even if the provider accepted the submission; nil does not establish that retrying is safe. Handles received after an attempt returns are not added to its result. Resume a create-and-add submission on Service.NewProviderContext for its ProviderID and call WaitForCreateAndAdd with its StatusURL and original ClientDataSetID. For add-pieces, open its DataSet with Service.NewDataSetContext and call WaitForCommit with StatusURL. A batched submission can include other uploads' pieces. Applications that need a durable CID-to-piece-ID mapping must retain their original request order; generic recovery validates only that the provider's confirmed count matches its returned piece IDs.

For service termination, persist the TerminateServiceSubmission returned by SubmitTerminateService or carried by TerminateServiceError, along with the required confirmation policy. Rebuild a Service or DataSetContext on the same chain and record keeper, then pass that submission to WaitForTerminateService. The callback can persist a submission before confirmation starts; recovery after a returned error does not require a callback.

Pulls have no status URL. ProviderContext.SubmitPull and DataSetContext.SubmitPull make one pull request and return the provider's current status without waiting for the pull to finish; the provider identifies the pull by its ExtraData. When the provider's pull queue is full, SubmitPull returns an error matching pdp.ErrPullQueueFull without waiting; resend the same request after the RetryAfter of the wrapped pdp.HTTPError. Pull waits for that delay itself. To resume after a restart, persist the piece CIDs, their source URLs, and ExtraData, then resend the original request unchanged: SubmitPull reports progress, and Pull waits. SubmitPull returns failed statuses without an error; Pull returns an error matching pdp.ErrPullFailed unless every requested piece completes. New ExtraData starts a separate pull, and a failed pull stays failed until retried with new ExtraData. Providers keep pull records for a limited time, so commit soon after every piece completes; the same ExtraData authorizes the commit. A pull through ProviderContext does not create a data set: pass its ExtraData to SubmitCreateAndAdd, which creates one data set per authorization.

Service termination

Service.SubmitTerminateService and DataSetContext.SubmitTerminateService submit once without waiting. Their WaitForTerminateService methods only read status; TerminateService combines submission and waiting. By default the provider relays an immediate termination requiring full payment-account settlement. There is no automatic fallback to direct submission. Set TerminateServiceOptions.SkipProvider to submit through FWSS without provider cooperation; service and payments continue until EndEpoch. Neither path waits until EndEpoch or deletes the remaining data-set state.

TerminateServiceError reports where the call failed:

  • Prepare: this invocation sent no termination authorization or transaction. Fix the cause and submit again.
  • types.ErrTxFailed reports failed direct execution after the requested confirmations. Inspect the data-set state before another attempt.
  • A relayed wait error matching pdp.WaitForTerminateServiceNotFoundError or pdp.WaitForTerminateServiceRejectedError means the provider holds no client request for the data set: it never arrived, or it failed and was discarded. If warmstorage.Service.GetDataSet still reports a zero PDPEndEpoch, submit again or set SkipProvider. The provider reports a request already in progress as Queued instead of starting another.
  • Any other submit or wait error can have an unknown outcome. Resume the retained Submission with WaitForTerminateService instead of submitting again. Accepted records that the provider or node took the submission, not that it executed; Queued records that the provider reported a termination in progress.

OnSubmitted receives the same submission after provider acceptance or direct broadcast, before waiting; it does not indicate confirmation. When a configured FWSS reader reports a nonzero PDPEndEpoch after a provider error, waiting succeeds even if the relay itself failed. A missing data set does not prove termination.

After an uncertain direct broadcast, do not send another direct transaction: resume the submission or track the transaction retained by types.BroadcastError. A zero PDPEndEpoch does not show that the earlier transaction was dropped. A callback cannot close a crash between sending and saving the submission.

Downloads

Context downloads use the PDP and optional CDN clients attached to that context. They enforce the exact raw payload size encoded in PieceCIDv2 and return ErrMaxBytesExceeded if a response is larger. URL-based Service.Download uses the Service HTTP client. Its default client rejects private and reserved network destinations; configure Options.AllowPrivateNetworks only for trusted private infrastructure. URL downloads enforce the raw size in PieceCIDv2; Options.DownloadMaxBytes can impose a smaller limit. PieceCIDv1 URL downloads use only that configured limit.

Custom storage components

StorageContext is sealed and implemented only by ProviderContext and DataSetContext. Custom resolvers may return those built-in contexts but cannot provide their own implementation. Callers may implement UploadResolver, ContextResolver, and ContextSelector to choose and return these contexts. DownloadContext and CDNRetriever accept custom piece retrieval implementations.

A custom ContextSelector must return contexts matching the Service identity and requested exclusions. Complete selections contain exactly the requested number of unique providers. Partial selections contain at least one target and return InsufficientUploadContextsError with counts matching the result.

Example

Example demonstrates uploading a payload via a storage.Service. In practice a Service is obtained from synapse.Client.Storage after calling synapse.New; this example assumes svc is already wired.

package main

import (
	"bytes"
	"context"
	"fmt"
	"log"

	"github.com/ipfs/go-cid"
	"github.com/strahe/synapse-go/storage"
	"github.com/strahe/synapse-go/types"
)

func main() {
	var svc *storage.Service // obtained from synapse.Client.Storage()

	ctx := context.Background()
	payload := bytes.NewReader(bytes.Repeat([]byte("data"), 64))

	res, err := svc.Upload(ctx, payload, &storage.UploadOptions{
		OnProgress: func(uploaded int64) {
			log.Printf("uploaded %d bytes", uploaded)
		},
		OnStored: func(providerID types.BigInt, pieceCID cid.Cid) {
			log.Printf("stored %s on provider %s", pieceCID, providerID)
		},
		OnPiecesConfirmed: func(dataSetID, providerID types.BigInt, pieces []storage.ConfirmedPiece) {
			log.Printf("provider %s confirmed %d piece(s) in dataset %s", providerID, len(pieces), dataSetID)
		},
	})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(res.PieceCID)
}

Index

Examples

Constants

This section is empty.

Variables

View Source
var ErrClosed = lifecycle.ErrClosed

ErrClosed is returned when a method is called after the owning Client has been closed. It aliases the shared closed-client sentinel.

View Source
var ErrDataSetCorrelationConflict = warmstorage.ErrDataSetCorrelationConflict

ErrDataSetCorrelationConflict is a convenience re-export of warmstorage.ErrDataSetCorrelationConflict. Both values match interchangeably with errors.Is.

View Source
var ErrDataSetUnavailable = warmstorage.ErrDataSetUnavailable

ErrDataSetUnavailable is returned when a data set does not exist or is no longer live. It aliases the canonical warmstorage sentinel.

View Source
var ErrEndorsementsNotConfigured = spregistry.ErrEndorsementsNotConfigured

ErrEndorsementsNotConfigured is returned when endorsed-primary selection is required but the resolver has no EndorsedProviderSource. It aliases the spregistry sentinel so callers can match either package's error.

View Source
var ErrInsufficientUploadContexts = errors.New("storage: insufficient upload contexts")

ErrInsufficientUploadContexts is matched by InsufficientUploadContextsError when selection found at least one, but fewer than the requested number of upload targets.

View Source
var ErrInvalidArgument = errors.New("storage: invalid argument")

ErrInvalidArgument is returned, wrapped via fmt.Errorf with %w, when a caller passes an argument that violates a precondition (nil pointer, zero ID, empty required URL, undefined pieceCID, etc.). Match with errors.Is(err, storage.ErrInvalidArgument).

Business / server-invariant errors (no approved providers, duplicate dataset IDs, server returned zero dataSetID, etc.) are intentionally left as plain errors so that errors.Is against ErrInvalidArgument only matches genuine caller-supplied validation failures.

View Source
var ErrInvalidDownloadOptions = errors.New("storage: invalid download options")

ErrInvalidDownloadOptions is returned when DownloadOptions is nil, empty, specifies more than one download source, or sets a zero DataSetID. Service.Download wraps it together with ErrInvalidArgument, so callers may match either.

View Source
var ErrMaxBytesExceeded = errors.New("storage: download exceeded MaxBytes")

ErrMaxBytesExceeded is returned when a download exceeds its byte limit. PieceCIDv2 downloads are bounded by their raw payload size. For URL downloads, a positive Options.DownloadMaxBytes can impose a smaller limit; PieceCIDv1 URL downloads use only that configured limit. The error is returned before streaming when Content-Length exceeds the limit, or from Read.

View Source
var ErrNoEndorsedProvider = errors.New("storage: no eligible endorsed provider")

ErrNoEndorsedProvider is returned when no endorsed provider is both approved, active, and healthy for the requested automatic upload selection.

View Source
var ErrNoHealthyProviders = errors.New("storage: no healthy providers")

ErrNoHealthyProviders is returned when every eligible automatically selected provider fails its PDP health check.

View Source
var ErrPrivateNetwork = safehttp.ErrPrivateNetwork

ErrPrivateNetwork is returned by URL-based downloads and manager-level provider PDP requests when the built-in HTTP client resolves the target to a loopback / link-local / RFC1918 / ULA / multicast / unspecified address and the Service was constructed without Options.AllowPrivateNetworks. It prevents SDK callers from being used as SSRF egress against internal networks.

View Source
var ErrUninitialized = errors.New("storage: service not initialized; use storage.New")

ErrUninitialized is returned when a method is invoked on a zero-value Service (one that was not constructed via New).

View Source
var ErrUnsupportedScheme = errors.New("storage: unsupported URL scheme")

ErrUnsupportedScheme is returned when the URL passed to Service.Download uses a scheme other than http or https.

Functions

This section is empty.

Types

type Allowances

type Allowances struct {
	Service         common.Address
	IsApproved      bool
	RateAllowance   *big.Int
	LockupAllowance *big.Int
	RateUsed        *big.Int
	LockupUsed      *big.Int
	MaxLockupPeriod *big.Int
}

Allowances is the client-specific operator-approval view embedded in StorageInfo. Nil when the caller has no wallet or the lookup failed.

type CDNRetriever added in v0.2.0

type CDNRetriever interface {
	DownloadPiece(context.Context, cid.Cid) (io.ReadCloser, error)
}

CDNRetriever retrieves pieces through an optional CDN for storage contexts.

type CIDMismatchError

type CIDMismatchError struct {
	Expected   cid.Cid
	ComputedV1 cid.Cid
	ComputedV2 cid.Cid
}

CIDMismatchError is returned when the piece CID computed from the downloaded bytes does not match the expected CID. Use errors.AsType to retrieve the computed and expected CIDs for diagnostic purposes.

func (*CIDMismatchError) Error

func (e *CIDMismatchError) Error() string

type CommitError

type CommitError struct {
	ProviderID types.BigInt
	Endpoint   string
	Cause      error
	// PieceCID and Size identify the stored piece.
	PieceCID cid.Cid
	Size     int64
	// FailedAttempts lists every failed attempt, as UploadResult.FailedAttempts
	// would have.
	FailedAttempts []FailedAttempt
}

CommitError is returned when no copy was committed on-chain. The piece may still be stored on providers, and a commit transaction may still confirm. FailedAttempts records failed attempts and any retained submission handles; see FailedAttempt.Submission for recovery limits. ProviderID, Endpoint, and Cause describe the primary provider. When the SDK constructs it, Endpoint is redacted in the same way as StoreError.Endpoint.

func (*CommitError) Error

func (e *CommitError) Error() string

func (*CommitError) Unwrap

func (e *CommitError) Unwrap() error

type CommitKind added in v0.5.0

type CommitKind string

CommitKind identifies whether a submission creates a data set or adds to an existing one.

const (
	// CommitKindCreateAndAdd creates a new data set and adds pieces to it.
	CommitKindCreateAndAdd CommitKind = "create-and-add"
	// CommitKindAddPieces adds pieces to an existing data set.
	CommitKindAddPieces CommitKind = "add-pieces"
)

type CommitRejectedError added in v0.5.0

type CommitRejectedError struct {
	ProviderID types.BigInt
	Status     CommitStatus
}

CommitRejectedError reports a valid terminal rejection with the provider and final status needed for diagnostics.

func (*CommitRejectedError) Error added in v0.5.0

func (e *CommitRejectedError) Error() string

func (*CommitRejectedError) Unwrap added in v0.5.0

func (e *CommitRejectedError) Unwrap() error

Unwrap makes CommitRejectedError match pdp.ErrTxRejected.

type CommitRequest

type CommitRequest struct {
	Pieces []PieceInput
	// ExtraData is an EIP-712 signed add-pieces payload. Leave it nil to let the
	// DataSetContext sign the request. Payloads whose encoded add-pieces calldata
	// exceeds pdp.MaxAddPiecesMessageSize are rejected.
	ExtraData []byte
	// OnSubmitted is invoked with an independent runtime snapshot immediately
	// after the provider returns a valid handle, before confirmation. For restart
	// recovery, persist StatusURL together with the target DataSetRef; do not treat
	// the complete CommitSubmission as a persistence schema. It may be nil.
	// Direct calls do not recover callback panics.
	OnSubmitted func(CommitSubmission)
}

CommitRequest adds pieces to an existing data set.

type CommitResult

type CommitResult struct {
	TransactionID          string         `json:"transactionId"`          // transaction hash from the provider submission
	ConfirmedTransactionID string         `json:"confirmedTransactionId"` // actual on-chain hash, when reported by the provider
	DataSet                DataSetRef     `json:"dataSet"`
	PieceIDs               []types.BigInt `json:"pieceIds"`
	IsNewDataSet           bool           `json:"isNewDataSet"` // true when a new data set was created by this commit
}

CommitResult is returned by a successful create-and-add or commit call.

func (*CommitResult) UnmarshalJSON added in v0.5.0

func (r *CommitResult) UnmarshalJSON(data []byte) error

type CommitState added in v0.5.0

type CommitState string

CommitState is the durable, provider-independent state of a submission.

const (
	// CommitStatePending means the provider has not reported a terminal result.
	CommitStatePending CommitState = "pending"
	// CommitStateConfirmed means all submitted pieces were added successfully.
	CommitStateConfirmed CommitState = "confirmed"
	// CommitStateRejected means the transaction failed or was rejected.
	CommitStateRejected CommitState = "rejected"
)

type CommitStatus added in v0.5.0

type CommitStatus struct {
	Kind                   CommitKind     `json:"kind"`
	State                  CommitState    `json:"state"`
	TransactionID          string         `json:"transactionId"`
	ConfirmedTransactionID string         `json:"confirmedTransactionId"`
	DataSet                *DataSetRef    `json:"dataSet"`
	PieceIDs               []types.BigInt `json:"pieceIds"`
}

CommitStatus is one logical status snapshot for a submission.

func (*CommitStatus) UnmarshalJSON added in v0.5.0

func (s *CommitStatus) UnmarshalJSON(data []byte) error

type CommitSubmission added in v0.5.0

type CommitSubmission struct {
	Kind            CommitKind      `json:"kind"`
	TransactionID   string          `json:"transactionId"`
	StatusURL       string          `json:"statusUrl"`
	ProviderID      types.BigInt    `json:"providerId"`
	Identity        ContextIdentity `json:"identity"`
	DataSet         *DataSetRef     `json:"dataSet"`
	ClientDataSetID *types.BigInt   `json:"clientDataSetId"`
	PieceCIDs       []cid.Cid       `json:"pieceCids"`
}

CommitSubmission describes one successful provider submission. It is a runtime result for callbacks and diagnostics, not a durable recovery record. Persist StatusURL to resume add-pieces. Create-and-add recovery also requires the original ClientDataSetID.

type ConfirmedPiece

type ConfirmedPiece struct {
	PieceID  types.BigInt
	PieceCID cid.Cid
}

ConfirmedPiece carries the on-chain identity reported by an OnPiecesConfirmed callback.

type ContextCostRef

type ContextCostRef struct {
	DataSetID *types.BigInt
	Provider  Provider
	// CurrentDataSetLeafCount is required and non-negative for an existing
	// data set. Zero means known empty. It is ignored for a new data set.
	CurrentDataSetLeafCount *big.Int
	// CurrentLifecycleReserveBalance is required and non-negative for an
	// existing data set. It is ignored for a new data set.
	CurrentLifecycleReserveBalance *big.Int
	// PendingOneTimePayments is the non-negative operation-fee total already
	// waiting to be paid from an existing data set's reserve. Nil defaults to
	// zero. It is ignored for a new data set.
	PendingOneTimePayments *big.Int
	// PDPEndEpoch is required for an existing data set and must point to zero.
	// It is ignored for a new data set.
	PDPEndEpoch *types.Epoch
	WithCDN     bool
}

ContextCostRef references one prospective upload target for multi-context cost aggregation. A nil DataSetID means "a new data set will be created on this provider". Existing data sets require the current leaf count, lifecycle reserve balance, and PDP end epoch. WithCDN determines whether the CDN-fixed lockup is included for this ref.

type ContextFactory

type ContextFactory func(Provider, ContextFactoryOptions) (*ProviderContext, error)

ContextFactory builds an unbound immutable provider context.

type ContextFactoryOptions added in v0.5.0

type ContextFactoryOptions struct {
	// LegacyPieceStorageIDLimit is the first compact data set ID; zero is unknown.
	LegacyPieceStorageIDLimit uint64
	DataSetMetadata           map[string]string
	WithCDN                   bool
}

ContextFactoryOptions configures construction of one provider context.

type ContextIdentity added in v0.5.0

type ContextIdentity struct {
	Payer        common.Address `json:"payer"`
	ChainID      types.ChainID  `json:"chainId"`
	RecordKeeper common.Address `json:"recordKeeper"`
}

ContextIdentity identifies the account and chain configuration used by a storage context for signing and on-chain operations. Its JSON form uses strict lowerCamel field names.

func (*ContextIdentity) UnmarshalJSON added in v0.5.0

func (i *ContextIdentity) UnmarshalJSON(data []byte) error

type ContextOption

type ContextOption func(*contextCore)

ContextOption configures provider and data-set contexts during construction. Nil options are ignored.

func WithCDN

func WithCDN(enabled bool) ContextOption

WithCDN enables CDN services for the data set and CDN-first downloads when a retriever is configured. When true, a "withCDN" metadata marker is added to the EIP-712 dataset-creation message; the contract activates CDN and applies its configured lockup upon seeing it.

func WithCDNRetriever added in v0.2.0

func WithCDNRetriever(r CDNRetriever) ContextOption

WithCDNRetriever injects the optional CDN retriever used by context downloads.

func WithChainID

func WithChainID(chainID types.ChainID) ContextOption

WithChainID sets the EIP-155 chain ID used for EIP-712 domain separation.

func WithDataSetMetadata

func WithDataSetMetadata(metadata map[string]string) ContextOption

WithDataSetMetadata sets the key-value metadata stored with the data set on creation.

func WithDataSetValidator added in v0.3.0

func WithDataSetValidator(v DataSetValidator) ContextOption

WithDataSetValidator injects the validator used before uploading or committing pieces to an existing data set.

func WithFWSSDataSetReader added in v0.3.0

func WithFWSSDataSetReader(r FWSSDataSetReader) ContextOption

WithFWSSDataSetReader injects the reader used to recover caller-identified data-set creation and to reject ended data sets before uploads. The reader must use the context's chain and record keeper.

func WithFWSSTerminator

func WithFWSSTerminator(t FWSSTerminator) ContextOption

WithFWSSTerminator injects the direct transaction dependency used by DataSetContext.TerminateService when SkipProvider is enabled.

func WithLegacyPieceStorageIDLimit added in v0.6.1

func WithLegacyPieceStorageIDLimit(limit uint64) ContextOption

WithLegacyPieceStorageIDLimit overrides the first compact data set ID. Without this option, known ChainIDs supply their deployment's cutoff. Explicit zero skips legacy piece-count checks. The last option wins.

func WithLogger added in v0.2.0

func WithLogger(logger *slog.Logger) ContextOption

WithLogger sets the logger used for internal warnings.

func WithPDPConfigReader

func WithPDPConfigReader(r PDPConfigReader) ContextOption

WithPDPConfigReader injects an optional reader for FWSSView PDPConfig used to estimate the challenge window and proof deadline in DataSetContext.PieceStatus.

func WithPDPVerifierReader

func WithPDPVerifierReader(r PDPVerifierReader) ContextOption

WithPDPVerifierReader injects a reader for PDPVerifier contract state. Required by DataSetContext.GetScheduledRemovals, DataSetContext.PieceStatus, DataSetContext.DeletePiece, and DataSetContext.DeletePieces.

func WithPayer

func WithPayer(payer common.Address) ContextOption

WithPayer sets the EVM address that pays for storage.

func WithPaymentStateReader added in v0.4.0

func WithPaymentStateReader(pay PaymentStateReader, epochs EpochReader, token common.Address) ContextOption

WithPaymentStateReader injects payment readers for provider-relayed termination debt pre-checks.

func WithRecordKeeper

func WithRecordKeeper(addr common.Address) ContextOption

WithRecordKeeper sets the FWSS contract address (record-keeper) used for EIP-712 signing and passed to the PDP provider for Pull and dataset creation.

func WithUploadBatcher added in v0.6.0

func WithUploadBatcher(batcher *UploadBatcher) ContextOption

WithUploadBatcher enables commit batching for high-level Upload calls made through the constructed context. Accepted work uses the batcher's lifecycle rather than the individual Upload context. The caller retains ownership of batcher.

type ContextResolver added in v0.3.0

type ContextResolver interface {
	ResolveProviderContext(context.Context, types.BigInt, NewProviderContextOptions) (*ProviderContext, error)
	ResolveDataSetContext(context.Context, types.BigInt, NewDataSetContextOptions) (*DataSetContext, error)
}

ContextResolver opens explicitly identified provider and data-set targets.

type ContextSelector added in v0.5.0

type ContextSelector interface {
	SelectProviderContext(context.Context, SelectProviderContextOptions) (*ProviderContext, error)
	SelectUploadContexts(context.Context, SelectUploadContextsOptions) (*UploadContextSelection, error)
}

ContextSelector chooses healthy targets for new uploads. Returned contexts must use the Service payer, chain ID, and record keeper, must honor requested provider exclusions, and must not contain duplicate providers.

SelectUploadContexts must set RequestedCopies to opts.Copies. A complete selection contains exactly that many contexts and sets Complete to true. A partial selection contains at least one but fewer than opts.Copies contexts, sets Complete to false, and returns an InsufficientUploadContextsError whose Requested and Available fields match the selection.

type ContextUploadOptions added in v0.6.0

type ContextUploadOptions struct {
	// PieceMetadata is validated, signed, and emitted in the FWSS PieceAdded
	// event. FWSS does not persist it in contract state.
	PieceMetadata map[string]string
	// PieceCID, when defined, is a pre-computed PieceCIDv2 of the payload.
	// When set, the provider client skips inline commP calculation; the server
	// still verifies the uploaded bytes match this value.
	PieceCID cid.Cid
	// OnProgress is invoked after each non-empty Read from the upload reader,
	// with the cumulative bytes sent to the provider so far. It may be nil.
	OnProgress func(bytesUploaded int64)
	// OnStored is invoked once the provider has confirmed storage of the piece.
	// It may be nil.
	OnStored func(providerID types.BigInt, pieceCID cid.Cid)
	// OnPiecesAdded is invoked after the on-chain AddPieces transaction is
	// submitted, carrying the transaction hash and submitted pieces. It may be nil.
	OnPiecesAdded func(txHash string, providerID types.BigInt, pieces []SubmittedPiece)
	// OnPiecesConfirmed is invoked after AddPieces is confirmed, carrying the
	// assigned on-chain IDs. It may be nil.
	OnPiecesConfirmed func(dataSetID, providerID types.BigInt, pieces []ConfirmedPiece)
}

ContextUploadOptions configures ProviderContext.Upload and DataSetContext.Upload. Context uploads store and commit one copy and do not perform provider selection or secondary pulls.

Callbacks may be invoked from internal goroutines. Handlers that share mutable state must be concurrency-safe. Callback panics are handled as described for UploadOptions.

type CopyResult

type CopyResult struct {
	ProviderID   types.BigInt
	DataSetID    types.BigInt
	PieceID      types.BigInt
	Role         CopyRole
	RetrievalURL string // HTTPS retrieval URL for this piece on the provider.
	IsNewDataSet bool
}

CopyResult describes one successfully committed copy.

type CopyRole

type CopyRole string

CopyRole identifies the role a provider plays in a multi-copy upload.

const (
	// CopyRolePrimary is the provider that received the original store.
	CopyRolePrimary CopyRole = "primary"
	// CopyRoleSecondary is a provider that pulled data from the primary.
	CopyRoleSecondary CopyRole = "secondary"
)

type CopyStage

type CopyStage string

CopyStage identifies the pipeline stage at which a provider attempt failed.

const (
	CopyStageStore   CopyStage = "store"
	CopyStagePull    CopyStage = "pull"
	CopyStagePresign CopyStage = "presign"
	CopyStageCommit  CopyStage = "commit"
)

type CreateAndAddRequest added in v0.6.0

type CreateAndAddRequest struct {
	Pieces []PieceInput
	// ExtraData is an EIP-712 signed create-and-add payload. Leave it nil to
	// let the ProviderContext sign the request. Payloads whose encoded
	// add-pieces calldata exceeds pdp.MaxAddPiecesMessageSize are rejected.
	ExtraData []byte
	// ClientDataSetID is the caller-owned uint256 used when creating the data
	// set. Nil generates a random ID. When ExtraData is set, the value must
	// match the ID embedded in its create payload.
	ClientDataSetID *types.BigInt
	// OnSubmitted is invoked with an independent runtime snapshot immediately
	// after the provider returns a valid handle, before confirmation. For restart
	// recovery, persist StatusURL and the original ClientDataSetID; do not treat
	// the complete CommitSubmission as a persistence schema. It may be nil.
	// Direct calls do not recover callback panics.
	OnSubmitted func(CommitSubmission)
}

CreateAndAddRequest creates a data set for a provider and adds pieces to it.

type CreateDataSetOptions added in v0.2.0

type CreateDataSetOptions struct {
	// ClientDataSetID is the caller-owned uint256 included in the create
	// authorization. Nil generates a random ID. Persist a caller-supplied value
	// before calling CreateDataSet when recovery from an ambiguous provider
	// response is required.
	ClientDataSetID *types.BigInt
	// OnSubmitted is invoked after the create transaction is submitted and
	// before waiting for confirmation. It may be nil.
	OnSubmitted func(CreateDataSetSubmission)
}

CreateDataSetOptions configures ProviderContext.CreateDataSet.

type CreateDataSetResult added in v0.2.0

type CreateDataSetResult struct {
	TransactionID          string     `json:"transactionId"`
	ConfirmedTransactionID string     `json:"confirmedTransactionId"`
	DataSet                DataSetRef `json:"dataSet"`
}

CreateDataSetResult is returned after standalone dataset creation confirms.

func (*CreateDataSetResult) UnmarshalJSON added in v0.5.0

func (r *CreateDataSetResult) UnmarshalJSON(data []byte) error

type CreateDataSetSubmission added in v0.2.0

type CreateDataSetSubmission struct {
	ProviderID      types.BigInt `json:"providerId"`
	TransactionID   string       `json:"transactionId"`
	StatusURL       string       `json:"statusUrl"`
	ClientDataSetID types.BigInt `json:"clientDataSetId"`
}

CreateDataSetSubmission describes a submitted create-dataset transaction. It is a runtime result for callbacks and diagnostics, not a durable recovery record. Persist StatusURL and the original ClientDataSetID to resume waiting.

type DataSetCatalog

type DataSetCatalog interface {
	GetApprovedProviderIDs(context.Context, types.ListOptions) ([]types.BigInt, error)
	GetClientDataSets(context.Context, common.Address, types.ListOptions) ([]*warmstorage.DataSetInfo, error)
	GetDataSet(context.Context, types.BigInt) (*warmstorage.DataSetInfo, error)
	GetAllDataSetMetadata(context.Context, types.BigInt) (map[string]string, error)
}

DataSetCatalog provides data-set records to ServiceResolver. The root Client supplies warmstorage.Service.

type DataSetContext added in v0.5.0

type DataSetContext struct {
	// contains filtered or unexported fields
}

DataSetContext represents one immutable provider + data-set target. Commit and Pull operations always target Ref. It is safe for concurrent use.

func NewDataSetContext added in v0.5.0

func NewDataSetContext(provider Provider, client PDPProviderClient, storageSigner signer.StorageSigner, ref DataSetRef, opts ...ContextOption) (*DataSetContext, error)

NewDataSetContext creates an immutable context bound to ref. storageSigner and WithPayer have the same independent signer and payer semantics as NewProviderContext.

func (*DataSetContext) CDNEnabled added in v0.5.0

func (c *DataSetContext) CDNEnabled() bool

CDNEnabled reports whether CDN services are enabled for this context.

func (*DataSetContext) ClientDataSetID added in v0.5.0

func (c *DataSetContext) ClientDataSetID() types.BigInt

ClientDataSetID returns the immutable client-chosen data-set ID.

func (*DataSetContext) Commit added in v0.5.0

Commit adds pieces to the bound data set and waits for confirmation.

func (*DataSetContext) ContextIdentity added in v0.5.0

func (c *DataSetContext) ContextIdentity() ContextIdentity

ContextIdentity returns the immutable payer, chain, and record keeper.

func (*DataSetContext) DataSetID added in v0.5.0

func (c *DataSetContext) DataSetID() types.BigInt

DataSetID returns the immutable on-chain data-set ID.

func (*DataSetContext) DataSetMetadata added in v0.6.0

func (c *DataSetContext) DataSetMetadata() map[string]string

DataSetMetadata returns an independent copy of the metadata associated with this context's data-set selection or creation target.

func (*DataSetContext) DataSetRef added in v0.5.0

func (c *DataSetContext) DataSetRef() (DataSetRef, bool)

DataSetRef returns the immutable target of this context.

func (*DataSetContext) DeletePiece added in v0.5.0

func (c *DataSetContext) DeletePiece(ctx context.Context, pieceCID cid.Cid) (*sdktypes.WriteResult, error)

DeletePiece schedules removal of the first piece matching pieceCID from this context's data set. pieceCID must be a PieceCIDv2; other CIDs return ErrInvalidArgument.

A data set can contain multiple piece IDs with the same piece CID. This method resolves pieceCID with PDPVerifier.findPieceIdsByCid using limit=1 and deletes the first returned piece ID. Prefer DataSetContext.DeletePieceByID when the on-chain piece ID is available.

Missing or non-live data sets return ErrDataSetUnavailable.

The returned WriteResult carries only the transaction hash; there is no on-chain wait.

func (*DataSetContext) DeletePieceByID added in v0.5.0

func (c *DataSetContext) DeletePieceByID(ctx context.Context, pieceID sdktypes.BigInt) (*sdktypes.WriteResult, error)

DeletePieceByID schedules removal of the piece identified by its on-chain piece ID from this context's data set.

Prefer this method when the piece ID is available, because piece CID is not guaranteed to be unique within a data set.

func (*DataSetContext) DeletePieces added in v0.5.1

func (c *DataSetContext) DeletePieces(ctx context.Context, pieceCIDs []cid.Cid) (*sdktypes.WriteResult, error)

DeletePieces schedules removal of the first piece matching each CID in one provider transaction. Duplicate resolved piece IDs are removed while preserving their first occurrence. Every CID must be a PieceCIDv2; otherwise ErrInvalidArgument is returned.

A data set can contain multiple piece IDs with the same piece CID. Use DataSetContext.DeletePiecesByID to remove specific duplicate instances. Missing or non-live data sets return ErrDataSetUnavailable.

func (*DataSetContext) DeletePiecesByID added in v0.5.1

func (c *DataSetContext) DeletePiecesByID(ctx context.Context, pieceIDs []sdktypes.BigInt) (*sdktypes.WriteResult, error)

DeletePiecesByID schedules removal of the identified on-chain piece IDs in one provider transaction. Duplicate IDs are removed while preserving their first occurrence.

func (*DataSetContext) Download added in v0.5.0

func (c *DataSetContext) Download(ctx context.Context, pieceCID cid.Cid) (io.ReadCloser, error)

Download retrieves a piece from CDN when enabled and available, otherwise from the storage provider. It returns ErrMaxBytesExceeded, either immediately or from the reader, if the response exceeds the raw payload size encoded in pieceCID.

func (*DataSetContext) GetCommitStatus added in v0.5.0

func (c *DataSetContext) GetCommitStatus(ctx context.Context, statusURL string) (*CommitStatus, error)

GetCommitStatus checks an add-pieces status URL once. A caller-supplied invalid status URL returns an error matching ErrInvalidArgument and pdp.ErrInvalidStatusURL. An origin mismatch also matches pdp.ErrStatusURLOrigin.

func (*DataSetContext) GetProviderInfo added in v0.5.0

func (c *DataSetContext) GetProviderInfo() Provider

GetProviderInfo returns a copy of the provider configuration.

func (*DataSetContext) GetScheduledRemovals added in v0.5.0

func (c *DataSetContext) GetScheduledRemovals(ctx context.Context) ([]types.BigInt, error)

GetScheduledRemovals returns the list of piece ids that have been scheduled for removal from this data set but have not yet been processed. Missing or non-live data sets return ErrDataSetUnavailable.

func (*DataSetContext) PieceStatus added in v0.5.0

func (c *DataSetContext) PieceStatus(ctx context.Context, pieceCID cid.Cid) (*PieceStatus, error)

PieceStatus returns the current status of pieceCID relative to this context's data set and the challenge schedule. If the piece is not present in the data set, Exists is false and the other fields are zero-valued. pieceCID must be a PieceCIDv2; other CIDs return ErrInvalidArgument. Missing or non-live data sets return ErrDataSetUnavailable.

Chain state and the head epoch are read independently, not at a fixed block. Missing or failed optional PDP configuration leaves estimates unavailable; the challenge time and remaining hours do not require that configuration. Other read failures and caller cancellation return wrapped errors.

func (*DataSetContext) PieceURL added in v0.5.0

func (c *DataSetContext) PieceURL(pieceCID cid.Cid) string

PieceURL returns the HTTPS retrieval URL for the given piece CID on this provider.

func (*DataSetContext) PresignForCommit added in v0.5.0

func (c *DataSetContext) PresignForCommit(ctx context.Context, pieces []PieceInput) ([]byte, error)

PresignForCommit signs an add-pieces payload for the bound data set.

func (*DataSetContext) ProviderID added in v0.5.0

func (c *DataSetContext) ProviderID() types.BigInt

ProviderID returns the provider's numeric ID.

func (*DataSetContext) Pull added in v0.5.0

func (c *DataSetContext) Pull(ctx context.Context, req PullRequest) (*PullResult, error)

Pull asks this provider to fetch pieces for the bound data set and repeats the request until the pull ends, so it also resumes a pull started by DataSetContext.SubmitPull. While the provider's pull queue is full, it waits for the provider's Retry-After and tries again. It succeeds only when every requested piece completes. Otherwise the error matches pdp.ErrPullFailed and the result carries each piece status the provider reported.

func (*DataSetContext) ServiceURL added in v0.5.0

func (c *DataSetContext) ServiceURL() string

ServiceURL returns the base URL of the provider's PDP HTTP API.

func (*DataSetContext) Store added in v0.5.0

func (c *DataSetContext) Store(ctx context.Context, r io.Reader, opts *StoreOptions) (*StoreResult, error)

Store streams data to the provider and waits for it to be parked.

func (*DataSetContext) SubmitCommit added in v0.5.0

func (c *DataSetContext) SubmitCommit(ctx context.Context, req CommitRequest) (*CommitSubmission, error)

SubmitCommit submits one add-pieces transaction and returns without waiting for confirmation.

func (*DataSetContext) SubmitPull added in v0.6.2

func (c *DataSetContext) SubmitPull(ctx context.Context, req PullRequest) (*PullResult, error)

SubmitPull makes one pull request for the bound data set and returns the provider's current status without waiting for the pull to finish. When the provider's pull queue is full, the error matches pdp.ErrPullQueueFull without retrying; read the delay with errors.AsType[*pdp.HTTPError] and resend the same request after RetryAfter. Other transient provider errors are retried. Resending the original request reports the existing pull's progress, because providers identify a pull by its ExtraData. The result lists every piece status the provider reports, and a failed pull is returned as a status rather than an error. ExtraData is required; after every piece completes, the same ExtraData can authorize DataSetContext.SubmitCommit.

func (*DataSetContext) SubmitTerminateService added in v0.7.1

func (c *DataSetContext) SubmitTerminateService(ctx context.Context, opts *TerminateServiceOptions) (*TerminateServiceSubmission, error)

SubmitTerminateService submits this context's termination without waiting. Errors preserve the stage and any recovery handle in TerminateServiceError, except *pdp.ServiceAlreadyTerminatedError when the service already ended.

func (*DataSetContext) TerminateService added in v0.5.0

TerminateService submits once and waits. It does not wait until EndEpoch or remove remaining data-set state. Errors retain recovery facts without a callback.

func (*DataSetContext) Upload added in v0.5.0

Upload stores a single copy and commits it to the bound data set. opts may be nil. When batching is configured, successful admission transfers ownership to the batcher; later caller cancellation stops waiting but not the commit.

func (*DataSetContext) WaitForCommit added in v0.5.0

func (c *DataSetContext) WaitForCommit(ctx context.Context, statusURL string) (*CommitResult, error)

WaitForCommit waits for an add-pieces status URL to confirm or reject. A caller-supplied invalid status URL returns an error matching ErrInvalidArgument and pdp.ErrInvalidStatusURL. An origin mismatch also matches pdp.ErrStatusURLOrigin.

func (*DataSetContext) WaitForTerminateService added in v0.7.1

WaitForTerminateService observes a saved handle for this data set without sending a write request. Errors preserve the latest known handle and receipt.

type DataSetDetails added in v0.5.1

type DataSetDetails = warmstorage.EnhancedDataSetInfo

DataSetDetails aliases warmstorage.EnhancedDataSetInfo, the richer record returned by Service.FindDataSets. Kept as an alias (not a copy) so that callers that already hold *warmstorage.EnhancedDataSetInfo can pass values through without conversion.

type DataSetDetailsCatalog added in v0.3.0

type DataSetDetailsCatalog interface {
	GetClientDataSetsWithDetails(ctx context.Context, payer common.Address, onlyManaged bool) ([]*warmstorage.EnhancedDataSetInfo, error)
}

DataSetDetailsCatalog lists data sets with PDP liveness, FWSS-listener ownership, and metadata. The root Client supplies warmstorage.Service.

type DataSetDownloader added in v0.6.1

type DataSetDownloader interface {
	DownloadDataSet(context.Context, types.BigInt, cid.Cid) (io.ReadCloser, error)
}

DataSetDownloader downloads a piece from a data set identified by ID. The root Client supplies ServiceResolver, which checks ownership and CDN metadata before retrieving a piece, with an active provider as the fallback.

type DataSetFinder

type DataSetFinder interface {
	FindDataSets(ctx context.Context, payer common.Address, onlyManaged bool) ([]*DataSetDetails, error)
}

DataSetFinder lists enriched data sets owned by payer. The root Client supplies a WarmStorage adapter.

type DataSetLeafCountReader added in v0.6.0

type DataSetLeafCountReader interface {
	GetDataSetLeafCount(ctx context.Context, dataSetID sdktypes.BigInt) (*big.Int, error)
}

DataSetLeafCountReader reads existing data-set leaf counts. The root Client supplies a PDPVerifier adapter. A successful result must be non-nil and non-negative; zero means known empty. Missing or non-live data sets return ErrDataSetUnavailable.

type DataSetPDPPaymentTerminatedError added in v0.3.0

type DataSetPDPPaymentTerminatedError struct {
	DataSetID   types.BigInt
	PDPEndEpoch types.Epoch
}

DataSetPDPPaymentTerminatedError is returned when an existing data set's PDP payment rail has an end epoch, so the data set cannot accept writes.

func (*DataSetPDPPaymentTerminatedError) Error added in v0.3.0

type DataSetRef added in v0.5.0

type DataSetRef struct {
	// contains filtered or unexported fields
}

DataSetRef identifies one data set and the provider that owns it. ProviderID and DataSetID must be non-zero. ClientDataSetID is the caller-chosen uint256 used by EIP-712 authorization and may be zero.

func NewDataSetRef added in v0.5.0

func NewDataSetRef(providerID, dataSetID, clientDataSetID types.BigInt) (DataSetRef, error)

NewDataSetRef validates and constructs an immutable data-set target. ClientDataSetID may be explicitly zero.

func (DataSetRef) ClientDataSetID added in v0.5.0

func (r DataSetRef) ClientDataSetID() types.BigInt

ClientDataSetID returns an independent copy of the client-chosen data-set ID.

func (DataSetRef) DataSetID added in v0.5.0

func (r DataSetRef) DataSetID() types.BigInt

DataSetID returns an independent copy of the on-chain data-set ID.

func (DataSetRef) Equal added in v0.5.0

func (r DataSetRef) Equal(other DataSetRef) bool

Equal reports whether both refs identify the same provider and data set.

func (DataSetRef) MarshalJSON added in v0.5.0

func (r DataSetRef) MarshalJSON() ([]byte, error)

MarshalJSON encodes all three identity fields using lowerCamel names. Invalid zero-value refs are rejected instead of being persisted as usable targets.

func (DataSetRef) ProviderID added in v0.5.0

func (r DataSetRef) ProviderID() types.BigInt

ProviderID returns an independent copy of the owning provider ID.

func (*DataSetRef) UnmarshalJSON added in v0.5.0

func (r *DataSetRef) UnmarshalJSON(data []byte) error

UnmarshalJSON requires the exact lowerCamel names and all three non-null identity fields.

type DataSetValidator added in v0.3.0

type DataSetValidator interface {
	ValidateDataSet(ctx context.Context, dataSetID sdktypes.BigInt) error
}

DataSetValidator checks that a data set is live in PDPVerifier and managed by the current FWSS listener. The root Client supplies warmstorage.Service.

type DownloadContext

type DownloadContext interface {
	Download(context.Context, cid.Cid) (io.ReadCloser, error)
}

DownloadContext retrieves pieces from a known storage provider.

type DownloadError

type DownloadError struct {
	URL        string
	StatusCode int   // zero when the failure occurred before receiving a response
	Cause      error // non-nil when the failure has an underlying error (e.g. a network or transport-level error)
}

DownloadError is returned when an HTTP download request fails, either due to a network error or a non-2xx HTTP status code. Use errors.AsType to access the URL and status code. URL, and the URL in any wrapped *url.Error, is redacted in the same way as StoreError.Endpoint; the path is kept as-is.

func (*DownloadError) Error

func (e *DownloadError) Error() string

func (*DownloadError) Unwrap

func (e *DownloadError) Unwrap() error

type DownloadOptions

type DownloadOptions struct {
	Context DownloadContext // delegates to DownloadContext.Download
	URL     string          // direct HTTP or HTTPS URL; validated against pieceCID on read completion
	// DataSetID downloads an owned data set's piece, using its CDN setting.
	// CDN retrieval can succeed even when the provider is inactive.
	DataSetID *types.BigInt
}

DownloadOptions configures a Service.Download call. Exactly one of Context, URL, or DataSetID must be set; other combinations return an error matching both ErrInvalidDownloadOptions and ErrInvalidArgument.

type EndorsedProviderSource added in v0.5.1

type EndorsedProviderSource interface {
	GetEndorsedProviderIDs(context.Context) ([]types.BigInt, error)
}

EndorsedProviderSource supplies the ordered provider IDs eligible to act as automatic upload primaries. The root Client supplies spregistry.Service. Implementations must be safe for concurrent use. A nil slice with a nil error is an empty endorsement set; query and decoding errors must be returned without being converted to an empty set.

type EpochReader added in v0.4.0

type EpochReader interface {
	BlockNumber(ctx context.Context) (uint64, error)
}

EpochReader returns the current chain epoch. The root Client supplies its configured Ethereum client.

type FWSSDataSetReader

type FWSSDataSetReader interface {
	GetDataSet(ctx context.Context, dataSetID sdktypes.BigInt) (*warmstorage.DataSetInfo, error)
	FindDataSetByClientDataSetID(ctx context.Context, payer common.Address, clientDataSetID sdktypes.BigInt) (*warmstorage.DataSetInfo, error)
}

FWSSDataSetReader reads data-set records from the FWSSView contract. Resolvers use it before construction to obtain complete immutable targets; upload paths use it to reject ended existing data sets before sending bytes to a provider; ProviderContext uses it to recover creation by caller-owned client data-set ID. The implementation must target the same chain and record keeper configured on the context. Termination recovery also uses it to reconcile provider observation failures. Satisfied by *warmstorage.Service.

type FWSSTerminationOptions added in v0.6.0

type FWSSTerminationOptions struct {
	// WriteOptions carry WarmStorage write settings. Submission overrides
	// WithWait and WithOnSubmitted so it only broadcasts without notification.
	WriteOptions []warmstorage.WriteOption
}

FWSSTerminationOptions configures a data-set termination request.

type FWSSTerminator

type FWSSTerminator interface {
	TerminateDataSet(ctx context.Context, dataSetID sdktypes.BigInt, opts FWSSTerminationOptions) (*sdktypes.WriteResult, error)
	// WaitForTermination succeeds only after the requested confirmation depth,
	// successful execution, and a termination event matching the configured
	// FWSS address and dataSetID. A returned receipt reached that depth, even
	// alongside an error. Timeout overrides WithWait; confirmations are preserved
	// and submission callbacks are ignored.
	WaitForTermination(ctx context.Context, dataSetID sdktypes.BigInt, hash common.Hash, timeout time.Duration, writeOpts []warmstorage.WriteOption) (*sdktypes.WriteResult, sdktypes.Epoch, error)
}

FWSSTerminator terminates data sets through FWSS. The root Client supplies a WarmStorage adapter. Submission errors before sending must not wrap types.BroadcastError; errors from sending must wrap it. Waiting is read-only.

type FailedAttempt

type FailedAttempt struct {
	ProviderID types.BigInt
	Role       CopyRole
	Stage      CopyStage // pipeline stage where the failure occurred
	Err        error
	Explicit   bool // true when the provider was caller-specified (no auto-retry)
	// Submission is set when the SDK validated and registered the commit
	// handle before this attempt ended. Nil means the handle is unknown, not
	// that the provider rejected the submission; it does not imply retrying is
	// safe. Use StatusURL, and ClientDataSetID for create-and-add, to resume
	// waiting; see Submission recovery in the package documentation.
	Submission *CommitSubmission
}

FailedAttempt records a provider attempt that did not produce a copy.

type FindDataSetsOptions

type FindDataSetsOptions struct {
	// Payer overrides the address configured via Options.PayerAddress.
	// Zero means use the configured default payer.
	Payer common.Address
	// OnlyManaged restricts the returned set to data sets whose
	// record-keeper is the configured FWSS contract.
	OnlyManaged bool
}

FindDataSetsOptions configures Service.FindDataSets. A nil pointer or zero value selects the configured default payer and disables the "only managed" filter.

type GetStorageInfoOptions

type GetStorageInfoOptions struct {
	// Client overrides the default payer. Zero means use the address
	// configured via Options.PayerAddress; if that too is zero the
	// allowances section of the returned StorageInfo will be nil.
	Client common.Address
}

GetStorageInfoOptions configures Service.GetStorageInfo. A nil pointer selects the configured default payer as the client address.

type InsufficientUploadContextsError added in v0.5.0

type InsufficientUploadContextsError struct {
	// Requested is the copy count requested from the selector.
	Requested int
	// Available is the number of usable contexts returned with the error.
	Available int
}

InsufficientUploadContextsError reports a usable partial selection.

func (*InsufficientUploadContextsError) Error added in v0.5.0

func (*InsufficientUploadContextsError) Is added in v0.5.0

Is makes InsufficientUploadContextsError match ErrInsufficientUploadContexts.

type MultiCostCalculator

type MultiCostCalculator interface {
	CalculateMultiContextCosts(ctx context.Context, payer common.Address, pieceSizes []uint64, refs []costs.MultiContextRef, opts *costs.UploadCostOptions) (*costs.MultiContextCosts, error)
}

MultiCostCalculator calculates aggregate upload costs. The root Client supplies costs.Service. Its input contract is defined by costs.Service.CalculateMultiContextCosts: leaf count, lifecycle reserve state, PDP end epoch, and CDN are supplied through refs, not opts.

type MultiCostOptions

type MultiCostOptions struct {
	// EnableCDN forces CDN pricing on every ref (in addition to any
	// per-ref `WithCDN` flag) and governs whether the CDN-fixed lockup
	// is added for new datasets.
	EnableCDN bool
	// ExtraRunwayEpochs is additional runway (epochs) on top of the
	// minimum lockup period. Defaults to 0. Negative values return ErrInvalidArgument.
	ExtraRunwayEpochs int64
	// BufferEpochs is the deposit cushion above current lockup usage to
	// cover transaction latency. Nil uses the cost service default; a pointer
	// to zero disables the buffer. Negative values return ErrInvalidArgument.
	BufferEpochs *int64
}

MultiCostOptions customises the multi-context cost calculation.

type NewDataSetContextOptions added in v0.5.0

type NewDataSetContextOptions struct {
	ProviderID *types.BigInt
	WithCDN    *bool
}

NewDataSetContextOptions configures an explicitly identified data-set context. ProviderID, when set, asserts the data set's provider.

type NewProviderContextOptions added in v0.5.0

type NewProviderContextOptions struct {
	DataSetMetadata map[string]string
	WithCDN         *bool
}

NewProviderContextOptions configures an explicitly identified provider context. WithCDN nil inherits the Service default.

type Options

type Options struct {
	// Resolver selects provider contexts for Upload and supplies replacement
	// candidates when a secondary provider fails. A nil resolver is allowed
	// so the Service can still serve DownloadFromContext / download-by-URL
	// calls; Upload then returns a clean validation error.
	Resolver UploadResolver

	// ContextResolver opens explicitly identified storage targets. When nil and
	// Resolver also implements ContextResolver, New reuses Resolver.
	ContextResolver ContextResolver

	// ContextSelector chooses healthy providers and upload targets. When nil and
	// Resolver also implements ContextSelector, New reuses Resolver.
	ContextSelector ContextSelector

	// DataSetDownloader backs downloads identified by data set ID. The root
	// Client supplies ServiceResolver; nil disables this download mode.
	DataSetDownloader DataSetDownloader

	// HTTPClient is used for URL-based downloads and manager-level provider PDP
	// control requests. nil installs clients sharing a safe transport: downloads
	// use a 24-hour timeout for large transfers, while PDP control requests use
	// [pdp.DefaultHTTPTimeout]. The default transport disables environment-variable
	// proxies to avoid proxy-assisted SSRF bypass. When set, the SDK's built-in
	// SSRF protection is bypassed entirely: the provided client is used unchanged
	// and is responsible for equivalent safeguards, redirect policy, and timeouts.
	// AllowPrivateNetworks has no effect in this case.
	HTTPClient *http.Client

	// Source is the application identifier for dataset namespace isolation.
	// Datasets with different Source values are treated as separate
	// namespaces; reuse only occurs within the same Source.
	Source string

	// DefaultWithCDN is the Client-level CDN default applied when an
	// UploadOptions.WithCDN or context-selection WithCDN field is nil.
	// Ignored when the per-operation field is non-nil.
	DefaultWithCDN bool

	// MaxSecondaryAttempts caps the number of provider candidates tried for
	// each secondary copy slot before giving up. Values <= 0 select the
	// default of 5.
	MaxSecondaryAttempts int

	// PullConcurrency caps concurrent secondary presign/pull/replacement
	// workflows within one upload. Values <= 0 select the default of 4.
	// A value of 1 serializes secondary pulls but still overlaps them with commits.
	PullConcurrency int

	// CommitConcurrency caps concurrent direct commits, including confirmation
	// waits, within one upload. Values <= 0 select the default of 4.
	// With UploadBatcher, the coordinator controls submission concurrency instead.
	CommitConcurrency int

	// AllowPrivateNetworks disables the default SSRF protection applied to
	// URL-based Service.Download calls and manager-level provider PDP requests.
	// When false (the default), the built-in HTTP transport refuses to dial local,
	// private, multicast, unspecified, or otherwise reserved address ranges and
	// returns ErrPrivateNetwork. Set to true only for trusted private download or
	// provider infrastructure. Ignored when HTTPClient is set.
	AllowPrivateNetworks bool

	// DownloadMaxBytes caps the number of bytes a single URL-based
	// Service.Download will return. Zero (the default) adds no configured cap;
	// PieceCIDv2 always caps downloads at its raw payload size. Negative values
	// are invalid. Exceeding the effective limit returns
	// ErrMaxBytesExceeded either eagerly (via Content-Length) or at the terminal
	// Read of the returned reader.
	DownloadMaxBytes int64

	// Logger receives internal warnings. nil disables logging.
	Logger *slog.Logger

	// Lifecycle is checked before service operations that can touch configured
	// backends. Any error returned by CheckClosed is returned without touching
	// those backends. The root synapse Client injects a shared checker whose
	// closed error matches ErrClosed. Nil is allowed for standalone use. A
	// typed-nil value is treated as nil.
	Lifecycle interface{ CheckClosed() error }

	// DataSetFinder backs Service.FindDataSets. Optional; when nil the
	// method returns an ErrUninitialized-wrapped error.
	DataSetFinder DataSetFinder

	// StorageInfoReader backs Service.GetStorageInfo. Optional.
	StorageInfoReader StorageInfoReader

	// DataSetTerminator backs Service.TerminateService with SkipProvider enabled.
	// Optional; the root SDK client assembles the built-in WarmStorage adapter.
	DataSetTerminator FWSSTerminator

	// CostCalculator backs Service.CalculateMultiContextCosts and the
	// cost estimation inside Prepare. Optional; the root SDK client supplies
	// the built-in costs.Service.
	CostCalculator MultiCostCalculator

	// PaymentsFunder backs PrepareTransaction.Execute. Optional.
	PaymentsFunder PaymentsFunder

	// DataSetLeafCountReader backs Service.Prepare for existing-dataset
	// contexts. It is required together with FWSSDataSetReader for that
	// calculation path; missing configuration returns ErrUninitialized.
	// New-dataset and precomputed-cost paths do not need it. The root SDK client
	// supplies its built-in PDPVerifier adapter.
	DataSetLeafCountReader DataSetLeafCountReader

	// FWSSDataSetReader reads existing data-set state for Prepare, reads the
	// on-chain ClientDataSetID during explicit data-set resolution, and checks
	// existing data sets before uploads. A custom context resolver must
	// separately pass the same reader to its contexts to support
	// ProviderContext.FindDataSetByClientDataSetID. When nil, these service-level
	// reads are unavailable.
	FWSSDataSetReader FWSSDataSetReader

	// ProviderResolver resolves provider endpoints for manager-level
	// provider-relayed termination. Optional.
	ProviderResolver ProviderResolver

	// PaymentStateReader and EpochReader back the provider-relayed termination
	// debt pre-check. Optional.
	PaymentStateReader PaymentStateReader
	EpochReader        EpochReader
	PaymentToken       common.Address

	// Signer signs EIP-712 authorizations for manager-level provider-relayed
	// termination. It may differ from PayerAddress when the payer has authorized
	// a delegated signer. It is independent from the signer attached to contexts
	// returned by Resolver. A typed-nil value is treated as nil.
	Signer       signer.StorageSigner
	ChainID      types.ChainID
	RecordKeeper common.Address

	// PayerAddress is the payer/client identity used by manager-level helpers
	// and validated against every StorageContext. It may differ from Signer.
	// When zero, New derives it from a non-nil Signer. Set it explicitly when a
	// delegated Signer authorizes operations for a different payer.
	PayerAddress common.Address

	// UploadBatcher batches commits made by high-level Upload and
	// UploadToContexts calls. Accepted work uses the batcher's lifecycle rather
	// than the individual Upload context. The caller retains ownership and must
	// close the batcher separately.
	UploadBatcher *UploadBatcher
}

Options configures a Service. Unset fields fall back to sensible defaults.

type PDPConfigReader

type PDPConfigReader interface {
	GetPDPConfig(ctx context.Context) (*warmstorage.PDPConfig, error)
}

PDPConfigReader reads proving-period configuration. The root Client supplies warmstorage.Service.

type PDPProviderClient added in v0.2.0

type PDPProviderClient interface {
	UploadPieceStreaming(context.Context, io.Reader, pdp.UploadPieceStreamingOptions) (*pdp.UploadStreamingResult, error)
	DownloadPiece(context.Context, cid.Cid) (io.ReadCloser, int64, error)
	WaitForPieceParked(context.Context, cid.Cid, time.Duration) error
	PullPieces(context.Context, pdp.PullRequest) (*pdp.PullResult, error)
	WaitForPullComplete(context.Context, pdp.PullRequest, time.Duration, func(*pdp.PullResult)) (*pdp.PullResult, error)
	AddPieces(context.Context, types.BigInt, []pdp.AddPieceInput, []byte) (*pdp.AddPiecesResult, error)
	GetAddPiecesStatus(context.Context, string) (*pdp.AddPiecesStatus, error)
	WaitForPiecesAdded(context.Context, string, time.Duration) (*pdp.AddPiecesStatus, error)
	CreateDataSet(context.Context, common.Address, []byte) (*pdp.CreateDataSetResult, error)
	GetDataSetCreationStatus(context.Context, string) (*pdp.CreateDataSetStatus, error)
	WaitForDataSetCreated(context.Context, string, time.Duration) (*pdp.CreateDataSetStatus, error)
	CreateDataSetAndAddPieces(context.Context, common.Address, []pdp.AddPieceInput, []byte) (*pdp.CreateDataSetResult, error)
	GetCreateDataSetAndAddPiecesStatus(context.Context, string) (*pdp.CreateAndAddPiecesStatus, error)
	WaitForCreateDataSetAndAddPieces(context.Context, string, time.Duration) (*pdp.AddPiecesStatus, error)
	SchedulePieceDeletions(ctx context.Context, dataSetID types.BigInt, pieceIDs []types.BigInt, extraData []byte) (common.Hash, error)
	TerminateService(context.Context, pdp.TerminateServiceRequest) (*pdp.TerminateServiceResult, error)
	// Waiting errors return the last status read, if any.
	WaitForTerminateService(context.Context, types.BigInt, time.Duration, func(common.Hash), ...pdp.WaitForTerminateServiceOption) (*pdp.TerminateServiceStatus, error)
}

PDPProviderClient is the provider HTTP API surface required by storage contexts. The root Client supplies pdp.Client.

type PDPProviderSource

type PDPProviderSource interface {
	GetPDPProvider(context.Context, types.BigInt) (*spregistry.PDPProvider, error)
	SelectActivePDPProviders(context.Context, spregistry.ProviderFilter) ([]spregistry.PDPProvider, error)
}

PDPProviderSource provides registry data to ServiceResolver. The root Client supplies spregistry.Service.

type PDPVerifierReader

type PDPVerifierReader interface {
	FindPieceIdsByCid(ctx context.Context, dataSetID sdktypes.BigInt, pieceCID cid.Cid, start, limit uint64) ([]sdktypes.BigInt, error)
	FindPieceIDsByCIDs(ctx context.Context, dataSetID sdktypes.BigInt, pieceCIDs []cid.Cid) ([][]sdktypes.BigInt, error)
	GetScheduledRemovals(ctx context.Context, dataSetID sdktypes.BigInt) ([]sdktypes.BigInt, error)
	GetNextChallengeEpoch(ctx context.Context, dataSetID sdktypes.BigInt) (*big.Int, error)
	BlockNumber(ctx context.Context) (uint64, error)
}

PDPVerifierReader is the read-only PDPVerifier surface required by DataSetContext for piece lifecycle queries (scheduled removals, ID lookup, next challenge epoch) and proving-window calculations. The root Client supplies a PDPVerifier adapter.

type PaymentStateReader added in v0.4.0

type PaymentStateReader interface {
	AccountInfo(ctx context.Context, token, owner common.Address) (*payments.AccountState, error)
}

PaymentStateReader reads payment account state for termination pre-checks. The root Client supplies payments.Service.

type PaymentsFunder

type PaymentsFunder interface {
	FundSync(ctx context.Context, amount *big.Int, opts ...payments.WriteOption) (*sdktypes.WriteResult, error)
}

PaymentsFunder tops up the Payments contract for an upload. The root Client supplies a Payments adapter.

type PieceInput

type PieceInput struct {
	PieceCID      cid.Cid
	PieceMetadata map[string]string // optional key-value metadata emitted in the FWSS PieceAdded event
}

PieceInput describes a single piece being committed on-chain. PieceCID must be a PieceCIDv2 whose raw size is between chain.MinUploadSize and chain.MaxUploadSize; other pieces return ErrInvalidArgument.

type PieceStatus

type PieceStatus struct {
	// Exists reports whether the piece is currently present in the data set.
	Exists bool

	// PieceID is the piece's on-chain numeric id. Only meaningful when
	// Exists is true.
	PieceID types.BigInt

	// DataSetNextChallengeAt is the time of the scheduled challenge epoch,
	// which may be in the past. Zero time indicates unavailable timing.
	DataSetNextChallengeAt time.Time

	// DataSetEstimatedProofDeadline adds the configured window length to the
	// challenge epoch. It need not equal the contract's actual deadline.
	// Zero time indicates unavailable timing.
	DataSetEstimatedProofDeadline time.Time

	// RetrievalURL is the HTTPS URL at which the piece can be retrieved
	// from the provider. Empty when provider info is not available.
	RetrievalURL string

	// InEstimatedChallengeWindow reports whether the head epoch lies in the
	// estimated interval [challenge epoch, estimated deadline). False also
	// covers unavailable estimates and does not establish data-set health.
	InEstimatedChallengeWindow bool

	// HoursUntilNextChallenge is the nonnegative epoch difference in hours.
	// Zero also covers an unavailable challenge schedule.
	HoursUntilNextChallenge float64

	// EstimatedProofDeadlinePassed reports whether the head epoch has reached
	// the estimated deadline. False also covers unavailable estimates.
	EstimatedProofDeadlinePassed bool
}

PieceStatus captures piece membership and data-set scheduling information. Estimated timing does not establish proof history or a missed proof.

type PrepareOptions

type PrepareOptions struct {
	// PieceSizes contains the raw payload size of each piece, copied to every
	// context. Each size must be between chain.MinUploadSize and
	// chain.MaxUploadSize. Required when Costs is nil; must be empty otherwise.
	PieceSizes []uint64
	// Contexts is the exact set of upload targets used for cost calculation.
	// It is required when Costs is nil.
	Contexts []StorageContext
	// Costs short-circuits cost calculation. When set, no other
	// PrepareOptions fields are accepted.
	Costs *costs.MultiContextCosts
	// ExtraRunwayEpochs is additional runway (epochs) above the
	// minimum lockup period passed through to the cost calculator. It is
	// valid only when Costs is nil and must be non-negative.
	ExtraRunwayEpochs int64
	// BufferEpochs is the deposit cushion above current lockup usage
	// used to absorb transaction-latency epochs. Nil uses the cost service
	// default; a pointer to zero disables the buffer. It is valid only when
	// Costs is nil. Negative values return ErrInvalidArgument.
	BufferEpochs *int64
}

PrepareOptions configures Service.Prepare. Costs and context-based cost calculation are mutually exclusive modes.

type PrepareResult

type PrepareResult struct {
	// Costs is the aggregated cost calculation that drove the decision.
	Costs *costs.MultiContextCosts
	// Transaction is non-nil only when funding is required (Ready=false).
	Transaction *PrepareTransaction
}

PrepareResult is the value returned by Service.Prepare.

type PrepareTransaction

type PrepareTransaction struct {
	// DepositAmount is the USDFC amount that will be moved into the
	// payments account. It is a copy; modifying it does not change what
	// Execute sends.
	DepositAmount *big.Int
	// IncludesApproval reports whether the call will also set the FWSS
	// operator to max allowance.
	IncludesApproval bool
	// Execute performs the funding operation. Prepare fixes the deposit
	// amount, approval decision, and max lockup period when it returns; later
	// changes to DepositAmount, PrepareResult.Costs, or PrepareOptions.Costs
	// do not affect Execute. Caller-provided payments.WriteOption values
	// should be limited to write controls such as wait, confirmations, or
	// precheck behavior.
	Execute func(ctx context.Context, opts ...payments.WriteOption) (*types.WriteResult, error)
}

PrepareTransaction is the deferred funding step returned by Prepare when the account is not yet Ready. Execute performs the top-up.

type PricePerTiB

type PricePerTiB struct {
	PerMonth *big.Int
	PerDay   *big.Int
	PerEpoch *big.Int
}

PricePerTiB holds the effective price per TiB for one provisioning profile at three time granularities.

type PricingInfo

type PricingInfo struct {
	NoCDN        PricePerTiB
	WithCDN      PricePerTiB
	TokenAddress common.Address
	TokenSymbol  string
	PriceList    *warmstorage.PriceList
}

PricingInfo is the pricing sub-structure of StorageInfo.

type Provider

type Provider struct {
	ID              types.BigInt   // provider ID from SPRegistry
	ServiceURL      string         // base URL of the provider's PDP HTTP API
	ServiceProvider common.Address // provider's EVM address
	Payee           common.Address // address that receives payments
}

Provider holds the on-chain identity of a storage provider.

type ProviderContext added in v0.5.0

type ProviderContext struct {
	// contains filtered or unexported fields
}

ProviderContext represents one provider without a bound data set. CreateAndAdd operations create a new data set, and Pull operations fetch pieces for a later create-and-add. It is safe for concurrent use; concurrent create operations are independent.

func NewProviderContext added in v0.5.0

func NewProviderContext(provider Provider, client PDPProviderClient, storageSigner signer.StorageSigner, opts ...ContextOption) (*ProviderContext, error)

NewProviderContext creates an immutable context for the given provider and PDP client. CreateAndAdd creates a data set; Pull stages pieces for Commit. provider.ID and provider.ServiceURL are validated here; client must not be nil or typed-nil. storageSigner may be nil or typed-nil; signing prerequisites (such as a non-nil signer plus chain/payer/record-keeper options) are validated by the write paths that need them, e.g. PresignForCommit. WithPayer configures the payer independently from storageSigner.

func (*ProviderContext) CDNEnabled added in v0.5.0

func (c *ProviderContext) CDNEnabled() bool

CDNEnabled reports whether CDN services are enabled for this context.

func (*ProviderContext) ContextIdentity added in v0.5.0

func (c *ProviderContext) ContextIdentity() ContextIdentity

ContextIdentity returns the immutable payer, chain, and record keeper.

func (*ProviderContext) CreateAndAdd added in v0.6.0

func (c *ProviderContext) CreateAndAdd(ctx context.Context, req CreateAndAddRequest) (*CommitResult, error)

CreateAndAdd creates a data set, adds pieces, and waits for confirmation.

func (*ProviderContext) CreateDataSet added in v0.5.0

CreateDataSet creates an empty data set for this provider. The receiver remains unbound; use ProviderContext.ForDataSet with the returned ref to obtain a DataSetContext.

func (*ProviderContext) DataSetMetadata added in v0.6.0

func (c *ProviderContext) DataSetMetadata() map[string]string

DataSetMetadata returns an independent copy of the metadata associated with this context's data-set selection or creation target.

func (*ProviderContext) DataSetRef added in v0.5.0

func (c *ProviderContext) DataSetRef() (DataSetRef, bool)

DataSetRef reports that a ProviderContext is not bound to a data set.

func (*ProviderContext) Download added in v0.5.0

func (c *ProviderContext) Download(ctx context.Context, pieceCID cid.Cid) (io.ReadCloser, error)

Download retrieves a piece from CDN when enabled and available, otherwise from the storage provider. It returns ErrMaxBytesExceeded, either immediately or from the reader, if the response exceeds the raw payload size encoded in pieceCID.

func (*ProviderContext) FindDataSetByClientDataSetID added in v0.6.0

func (c *ProviderContext) FindDataSetByClientDataSetID(
	ctx context.Context,
	clientDataSetID types.BigInt,
) (DataSetRef, bool, error)

FindDataSetByClientDataSetID checks whether a data set created with the caller-owned clientDataSetID is visible on-chain for this provider. A false found value means no matching data set is visible yet; it does not mean the provider rejected the request. Callers choose their own polling policy.

Persist clientDataSetID together with ProviderContext.ProviderID and ProviderContext.ContextIdentity before submitting the create request, then recreate the same context before using this method after a restart. A found create-and-add data set does not prove that its pieces were added.

func (*ProviderContext) ForDataSet added in v0.5.0

func (c *ProviderContext) ForDataSet(ref DataSetRef) (*DataSetContext, error)

ForDataSet returns a new immutable context bound to ref. The receiver is not modified and may continue to create independent data sets.

func (*ProviderContext) GetCreateAndAddStatus added in v0.6.0

func (c *ProviderContext) GetCreateAndAddStatus(
	ctx context.Context,
	statusURL string,
	clientDataSetID types.BigInt,
) (*CommitStatus, error)

GetCreateAndAddStatus checks a create-and-add status URL once. An invalid status URL returns an error matching ErrInvalidArgument and pdp.ErrInvalidStatusURL. An origin mismatch also matches pdp.ErrStatusURLOrigin. ClientDataSetID must be the value used for the original submission; zero is valid only when that original value was zero.

func (*ProviderContext) GetProviderInfo added in v0.5.0

func (c *ProviderContext) GetProviderInfo() Provider

GetProviderInfo returns a copy of the provider configuration.

func (*ProviderContext) PieceURL added in v0.5.0

func (c *ProviderContext) PieceURL(pieceCID cid.Cid) string

PieceURL returns the HTTPS retrieval URL for the given piece CID on this provider.

func (*ProviderContext) PresignForCommit added in v0.5.0

func (c *ProviderContext) PresignForCommit(ctx context.Context, pieces []PieceInput) ([]byte, error)

PresignForCommit signs a create-and-add payload for a new data set.

func (*ProviderContext) ProviderID added in v0.5.0

func (c *ProviderContext) ProviderID() types.BigInt

ProviderID returns the provider's numeric ID.

func (*ProviderContext) Pull added in v0.5.0

Pull asks this provider to fetch pieces for a new data set and repeats the request until the pull ends, so it also resumes a pull started by ProviderContext.SubmitPull. While the provider's pull queue is full, it waits for the provider's Retry-After and tries again. It succeeds only when every requested piece completes. Otherwise the error matches pdp.ErrPullFailed and the result carries each piece status the provider reported.

func (*ProviderContext) ServiceURL added in v0.5.0

func (c *ProviderContext) ServiceURL() string

ServiceURL returns the base URL of the provider's PDP HTTP API.

func (*ProviderContext) Store added in v0.5.0

func (c *ProviderContext) Store(ctx context.Context, r io.Reader, opts *StoreOptions) (*StoreResult, error)

Store streams data to the provider and waits for it to be parked.

func (*ProviderContext) SubmitCreateAndAdd added in v0.6.0

func (c *ProviderContext) SubmitCreateAndAdd(ctx context.Context, req CreateAndAddRequest) (*CommitSubmission, error)

SubmitCreateAndAdd submits one create-and-add transaction and returns without waiting for confirmation.

func (*ProviderContext) SubmitPull added in v0.6.2

func (c *ProviderContext) SubmitPull(ctx context.Context, req PullRequest) (*PullResult, error)

SubmitPull makes one pull request for a new data set and returns the provider's current status without waiting for the pull to finish. When the provider's pull queue is full, the error matches pdp.ErrPullQueueFull without retrying; read the delay with errors.AsType[*pdp.HTTPError] and resend the same request after RetryAfter. Other transient provider errors are retried. Resending the original request reports the existing pull's progress, because providers identify a pull by its ExtraData. The result lists every piece status the provider reports, and a failed pull is returned as a status rather than an error. ExtraData is required and must be a create-and-add authorization, such as one from ProviderContext.PresignForCommit. The pull does not create the data set; after every piece completes, pass the same ExtraData to ProviderContext.SubmitCreateAndAdd.

func (*ProviderContext) Upload added in v0.5.0

Upload stores a single copy and commits it to a new data set. opts may be nil. With batching, it commits to the data set the batcher shares for this provider, data-set metadata, and CDN setting. Successful admission transfers ownership to the batcher; later caller cancellation stops waiting but not the commit.

func (*ProviderContext) WaitForCreateAndAdd added in v0.6.0

func (c *ProviderContext) WaitForCreateAndAdd(
	ctx context.Context,
	statusURL string,
	clientDataSetID types.BigInt,
) (*CommitResult, error)

WaitForCreateAndAdd waits for a create-and-add status URL to confirm or reject. An invalid status URL returns an error matching ErrInvalidArgument and pdp.ErrInvalidStatusURL. An origin mismatch also matches pdp.ErrStatusURLOrigin. ClientDataSetID must be the value used for the original submission; zero is valid only when that original value was zero.

func (*ProviderContext) WaitForDataSetCreated added in v0.5.0

func (c *ProviderContext) WaitForDataSetCreated(
	ctx context.Context,
	statusURL string,
	clientDataSetID types.BigInt,
) (*CreateDataSetResult, error)

WaitForDataSetCreated waits for a previously submitted create-dataset status URL. ClientDataSetID must be the value used for the original submission; zero is valid only when that original value was zero. The receiver remains unbound.

type ProviderResolver added in v0.4.0

type ProviderResolver interface {
	ResolveProvider(ctx context.Context, providerID sdktypes.BigInt) (Provider, error)
}

ProviderResolver resolves storage providers by provider ID. The root Client supplies ServiceResolver.

type PullPieceResult

type PullPieceResult struct {
	PieceCID cid.Cid
	Status   PullStatus
}

PullPieceResult is the per-piece outcome within a PullResult.

type PullRequest

type PullRequest struct {
	// Pieces must be PieceCIDv2 values whose raw sizes are between
	// chain.MinUploadSize and chain.MaxUploadSize.
	Pieces []cid.Cid
	From   func(cid.Cid) string // returns the HTTPS URL for a given piece CID
	// ExtraData is the required EIP-712 signed payload authorising the pull,
	// usually from PresignForCommit. Oversized payloads are rejected before
	// submission.
	ExtraData []byte
	// OnProgress is invoked for each piece status in every provider response.
	// It may be nil. Direct Pull and SubmitPull calls do not recover callback
	// panics.
	OnProgress func(pieceCID cid.Cid, status PullStatus)
}

PullRequest asks a secondary provider to pull pieces from a primary.

type PullResult

type PullResult struct {
	// Status is the provider's overall status. In a SubmitPull result, a
	// complete status does not guarantee that every piece completed; check
	// each entry in Pieces. Pull returns an error in that case.
	Status PullStatus
	// Pieces lists every piece status the provider reports.
	Pieces []PullPieceResult
}

PullResult is the aggregate outcome of a pull operation.

type PullStatus

type PullStatus string

PullStatus is the per-piece or overall status returned by a pull operation.

const (
	PullStatusPending    PullStatus = "pending"
	PullStatusInProgress PullStatus = "inProgress"
	PullStatusRetrying   PullStatus = "retrying"
	PullStatusComplete   PullStatus = "complete"
	PullStatusFailed     PullStatus = "failed"
)

type SelectProviderContextOptions added in v0.5.0

type SelectProviderContextOptions struct {
	ExcludeProviderIDs []types.BigInt
	DataSetMetadata    map[string]string
	WithCDN            *bool
}

SelectProviderContextOptions configures selection of one healthy provider.

type SelectUploadContextsOptions added in v0.5.0

type SelectUploadContextsOptions struct {
	Copies             int
	ExcludeProviderIDs []types.BigInt
	DataSetMetadata    map[string]string
	// AllowUnendorsedPrimary selects the primary from the full approved-provider
	// pool without an endorsement read. The zero value requires endorsement.
	AllowUnendorsedPrimary bool
	WithCDN                *bool
}

SelectUploadContextsOptions configures selection of upload targets.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service orchestrates multi-copy uploads and downloads. Create with New; configure via the Options struct.

func New

func New(opts Options) (*Service, error)

New creates a Service from the given Options.

func (*Service) CalculateMultiContextCosts

func (s *Service) CalculateMultiContextCosts(ctx context.Context, pieceSizes []uint64, refs []ContextCostRef, opts MultiCostOptions, payer common.Address) (*costs.MultiContextCosts, error)

CalculateMultiContextCosts estimates aggregate costs for the given storage targets. pieceSizes contains the raw payload size of each piece, replicated to every target. Each size must be between chain.MinUploadSize and chain.MaxUploadSize. Existing refs require a non-negative leaf count and lifecycle reserve balance, plus a zero PDP end epoch. A zero payer uses the configured default payer.

func (*Service) Download

func (s *Service) Download(ctx context.Context, pieceCID cid.Cid, opts *DownloadOptions) (io.ReadCloser, error)

Download retrieves a piece by URL, through a DownloadContext, or from a data set by ID. URL downloads verify integrity at the response's actual EOF. Check the final Read or io.ReadAll error; closing early skips verification. PieceCIDv2 caps URL downloads at its raw payload size; a positive Options.DownloadMaxBytes can impose a smaller limit. PieceCIDv1 uses only that configured limit. Exceeding a cap returns ErrMaxBytesExceeded before streaming when Content-Length is too large, or from Read.

func (*Service) FindDataSets

func (s *Service) FindDataSets(ctx context.Context, opts *FindDataSetsOptions) ([]*DataSetDetails, error)

FindDataSets returns the enriched list of data sets owned by the caller or by the payer in opts.

func (*Service) Flush added in v0.6.0

func (s *Service) Flush(ctx context.Context) error

Flush waits for uploads started before this call to reach the batcher, submits their windows, and waits for final confirmations. Later compatible uploads may join those windows and are not guaranteed to be included or excluded. Canceling ctx stops only this wait. It is a no-op when batching is not configured.

func (*Service) GetStorageInfo

func (s *Service) GetStorageInfo(ctx context.Context, opts *GetStorageInfoOptions) (*StorageInfo, error)

GetStorageInfo returns the comprehensive chain-wide storage view needed to author an upload.

func (*Service) NewDataSetContext added in v0.5.0

func (s *Service) NewDataSetContext(ctx context.Context, dataSetID types.BigInt, opts NewDataSetContextOptions) (*DataSetContext, error)

NewDataSetContext opens an existing data set owned by the Service payer. The data set may be terminated or otherwise not currently writable.

func (*Service) NewProviderContext added in v0.5.0

func (s *Service) NewProviderContext(ctx context.Context, providerID types.BigInt, opts NewProviderContextOptions) (*ProviderContext, error)

NewProviderContext opens a registered provider without selecting it or checking its current approval, activity, or endpoint health.

func (*Service) Prepare

func (s *Service) Prepare(ctx context.Context, opts *PrepareOptions) (*PrepareResult, error)

Prepare returns the funding transaction needed, if any, to cover one upload of PieceSizes across the supplied contexts. Existing data sets require a DataSetLeafCountReader and FWSSDataSetReader; missing configuration returns ErrUninitialized, and unavailable data sets return ErrDataSetUnavailable. Precomputed Costs bypasses these reads.

func (*Service) SelectProviderContext added in v0.5.0

func (s *Service) SelectProviderContext(ctx context.Context, opts SelectProviderContextOptions) (*ProviderContext, error)

SelectProviderContext selects one approved, active, healthy provider without querying or reusing an existing data set.

func (*Service) SelectUploadContexts added in v0.5.0

func (s *Service) SelectUploadContexts(ctx context.Context, opts SelectUploadContextsOptions) (*UploadContextSelection, error)

SelectUploadContexts selects targets for Prepare and UploadToContexts. A partial result is returned with InsufficientUploadContextsError.

func (*Service) SubmitTerminateService added in v0.7.1

func (s *Service) SubmitTerminateService(ctx context.Context, dataSetID types.BigInt, opts *TerminateServiceOptions) (*TerminateServiceSubmission, error)

SubmitTerminateService submits once without waiting for confirmation. Prepare errors prove this call did not submit. Submit errors preserve a handle with an unknown outcome; inspect TerminateServiceError and its cause. A provider reporting that the service already ended returns *pdp.ServiceAlreadyTerminatedError without a TerminateServiceError.

func (*Service) TerminateService added in v0.4.0

func (s *Service) TerminateService(ctx context.Context, dataSetID types.BigInt, opts *TerminateServiceOptions) (*TerminateServiceResult, error)

TerminateService submits once and waits for an arranged termination. Wait errors retain the submission and observations in TerminateServiceError.

func (*Service) Upload

func (s *Service) Upload(ctx context.Context, r io.Reader, opts *UploadOptions) (*UploadResult, error)

Upload automatically selects targets and runs the multi-copy upload pipeline streaming from r in a single pass. opts must be non-nil. Returns UploadResult whose Complete field indicates whether all requested copies were committed on-chain.

The reader is consumed once by the primary provider; secondary copies are populated via server-to-server Pulls. On success the reader is fully drained; on error it may be only partially consumed.

Timeouts and cancellation: before batching accepts a piece, ctx controls resolution, presign, Store, Pull, and validation. After acceptance, ctx stops this caller's wait and later callbacks, but the batcher continues the commit on its own lifecycle context. A context error therefore does not prove that no transaction will be submitted. Without batching, ctx continues to control the commit wait. The Service itself imposes no upload deadline. Its built-in 24-hour HTTP timeout affects only URL-based downloads; Upload, Pull, and Commit use the StorageContext implementation's HTTP client.

func (*Service) UploadToContexts added in v0.5.0

func (s *Service) UploadToContexts(ctx context.Context, r io.Reader, contexts []StorageContext, opts *UploadToContextsOptions) (*UploadResult, error)

UploadToContexts uploads to the exact contexts supplied by the caller. The first context is primary; later contexts receive provider-to-provider pulls. opts may be nil. When batching is configured, cancellation follows the ownership-transfer contract documented by Service.Upload.

func (*Service) WaitForTerminateService added in v0.7.1

WaitForTerminateService only observes an existing submission. Timeout includes provider resolution and chain reconciliation. No signer is needed. Recovery must use the original chain and record keeper. A missing reader disables chain reconciliation; a missing data set does not establish successful termination.

type ServiceParameters

type ServiceParameters struct {
	EpochsPerMonth int64
	EpochsPerDay   int64
	EpochDuration  int64 // seconds per epoch
	MinUploadSize  int64
	MaxUploadSize  int64
}

ServiceParameters captures the immutable chain/service geometry that StorageInfo callers want to show alongside pricing.

type ServiceResolver

type ServiceResolver struct {
	// contains filtered or unexported fields
}

ServiceResolver opens explicit targets and selects providers and data sets for uploads. With a known cutoff, compact matches precede legacy matches. Within each layout, active pieces precede empty matches when the catalog reports activity. Providers retain their stable order inside each layout group.

func NewServiceResolver

func NewServiceResolver(opts ServiceResolverOptions) (*ServiceResolver, error)

NewServiceResolver constructs a ServiceResolver. Payer, SPRegistry, WarmStorage, and NewContext are required. DataSetValidator, DataSetDetails, and active-piece reads are optional and auto-detected from WarmStorage when available and configured. ProviderPing is optional and defaults to a PDP ping with up to two retries for transient failures.

func (*ServiceResolver) DownloadDataSet added in v0.6.1

func (r *ServiceResolver) DownloadDataSet(ctx context.Context, dataSetID types.BigInt, pieceCID cid.Cid) (io.ReadCloser, error)

DownloadDataSet retrieves a PieceCIDv2 from a data set owned by the resolver's payer. A CDN-enabled data set is tried through CDNRetriever first, regardless of provider activity. Provider retrieval requires an active PDP product.

func (*ServiceResolver) ResolveDataSetContext added in v0.5.0

func (r *ServiceResolver) ResolveDataSetContext(ctx context.Context, dataSetID types.BigInt, opts NewDataSetContextOptions) (*DataSetContext, error)

ResolveDataSetContext opens an existing payer-owned data set without requiring it to be writable.

func (*ServiceResolver) ResolveProvider added in v0.4.0

func (r *ServiceResolver) ResolveProvider(ctx context.Context, providerID types.BigInt) (Provider, error)

ResolveProvider resolves one PDP provider by ID.

func (*ServiceResolver) ResolveProviderContext added in v0.5.0

func (r *ServiceResolver) ResolveProviderContext(ctx context.Context, providerID types.BigInt, opts NewProviderContextOptions) (*ProviderContext, error)

ResolveProviderContext opens a registered provider without selection checks.

func (*ServiceResolver) ResolveUploadContexts

func (r *ServiceResolver) ResolveUploadContexts(ctx context.Context, opts SelectUploadContextsOptions) ([]StorageContext, error)

ResolveUploadContexts selects contexts for an automatic upload.

func (*ServiceResolver) SelectProviderContext added in v0.5.0

func (r *ServiceResolver) SelectProviderContext(ctx context.Context, opts SelectProviderContextOptions) (*ProviderContext, error)

SelectProviderContext selects one approved, active, healthy provider and returns it without a data-set binding.

func (*ServiceResolver) SelectReplacement

SelectReplacement selects one writable provider from the complete approved pool. Endorsement is not required, but endorsed providers remain eligible. ExcludeProviderIDs must include all providers already attempted by the upload.

func (*ServiceResolver) SelectUploadContexts added in v0.5.0

SelectUploadContexts selects writable targets in stable provider order.

type ServiceResolverOptions

type ServiceResolverOptions struct {
	// LegacyPieceStorageIDLimit enables compact-first reuse for this deployment.
	// Zero preserves the existing layout-independent selection order.
	LegacyPieceStorageIDLimit uint64
	// Payer is the EVM address assigned to contexts created by this resolver.
	// Set it to the paying account, not a delegated Storage signer address.
	Payer      common.Address
	SPRegistry PDPProviderSource
	// Endorsements is required only when automatic upload selection uses its
	// default endorsed-primary policy. It may be nil when callers always set
	// AllowUnendorsedPrimary or use explicit provider contexts.
	Endorsements EndorsedProviderSource
	WarmStorage  DataSetCatalog
	// CDNRetriever retrieves pieces for Payer without requiring an active
	// provider. nil limits data-set downloads to active providers.
	CDNRetriever     CDNRetriever
	DataSetValidator DataSetValidator
	DataSetDetails   DataSetDetailsCatalog
	// ProviderPing checks one automatically selected provider endpoint. The
	// resolver may call it concurrently for up to 16 candidates per selection;
	// concurrent selections have independent limits. Each call receives a
	// context whose deadline is no later than eight seconds; a shorter parent
	// deadline takes precedence. Implementations must be safe for concurrent use
	// and honor the context. nil uses [pdp.Client.Ping] with up to two retries for
	// transient failures.
	ProviderPing func(context.Context, string) error
	NewContext   ContextFactory // called per-provider to construct an upload context
}

ServiceResolverOptions configures a ServiceResolver.

type StorageContext added in v0.5.0

type StorageContext interface {
	ContextIdentity() ContextIdentity
	ProviderID() types.BigInt
	GetProviderInfo() Provider
	DataSetRef() (DataSetRef, bool)
	// DataSetMetadata returns an independent copy of the target metadata.
	DataSetMetadata() map[string]string
	ServiceURL() string
	CDNEnabled() bool
	PieceURL(cid.Cid) string
	Store(context.Context, io.Reader, *StoreOptions) (*StoreResult, error)
	Download(context.Context, cid.Cid) (io.ReadCloser, error)
	// contains filtered or unexported methods
}

StorageContext is a sealed immutable upload target implemented by ProviderContext and DataSetContext. Callers may inspect and pass SDK-created values but cannot implement the interface.

type StorageInfo

type StorageInfo struct {
	Pricing           PricingInfo
	Providers         []spregistry.PDPProvider
	ServiceParameters ServiceParameters
	Allowances        *Allowances
}

StorageInfo aggregates the chain-wide storage view a client needs before authoring an upload.

type StorageInfoReader

type StorageInfoReader interface {
	GetStorageInfo(ctx context.Context, client common.Address) (*StorageInfo, error)
}

StorageInfoReader reads the chain-wide StorageInfo view. The root Client supplies an adapter.

type StoreError

type StoreError struct {
	ProviderID types.BigInt
	Endpoint   string
	Cause      error
}

StoreError is returned when the primary store operation fails. When the SDK constructs it, Endpoint is redacted: userinfo is stripped and sensitive query parameter values are masked as "***", so the value is safe to log.

func (*StoreError) Error

func (e *StoreError) Error() string

func (*StoreError) Unwrap

func (e *StoreError) Unwrap() error

type StoreOptions

type StoreOptions struct {
	// PieceCID, when defined, is a pre-computed PieceCIDv2 of the payload.
	// When set, the client skips inline commP calculation; the server still
	// verifies the uploaded bytes match this value.
	PieceCID cid.Cid
	// OnProgress is invoked after each non-empty Read from the reader, with
	// the cumulative bytes sent so far. It may be nil. It can run on the HTTP
	// transport goroutine; a panic aborts the upload and is re-raised with the
	// same value on the goroutine that called Store. The PDP client logs the
	// original stack at Error level when it has a logger.
	OnProgress func(bytesUploaded int64)
}

StoreOptions configures a single provider or data-set context Store call.

type StoreResult

type StoreResult struct {
	PieceCID cid.Cid // PieceCIDv2 of the stored data
	Size     int64   // raw (unpadded) byte count
}

StoreResult is returned by a successful Store call.

type SubmittedPiece

type SubmittedPiece struct {
	PieceCID cid.Cid
}

SubmittedPiece carries the piece identity reported by an OnPiecesAdded callback.

type TerminateServiceDebtError added in v0.4.0

type TerminateServiceDebtError struct {
	Shortfall *big.Int
}

TerminateServiceDebtError is returned before provider-relayed termination when the payer account already has debt.

func (*TerminateServiceDebtError) Error added in v0.4.0

func (e *TerminateServiceDebtError) Error() string

type TerminateServiceError added in v0.7.1

type TerminateServiceError struct {
	Stage      TerminateServiceStage
	Submission *TerminateServiceSubmission
	Receipt    *ethtypes.Receipt
	Cause      error
}

TerminateServiceError preserves the last known termination facts on failure. Prepare failures have no Submission. Submit and wait failures retain a recovery handle when available. Receipt is a confirmed direct receipt whose transaction failed or lacked a matching termination event.

func (*TerminateServiceError) Error added in v0.7.1

func (e *TerminateServiceError) Error() string

func (*TerminateServiceError) Unwrap added in v0.7.1

func (e *TerminateServiceError) Unwrap() error

Unwrap preserves the underlying submission or observation error.

type TerminateServiceOptions added in v0.4.0

type TerminateServiceOptions struct {
	// SkipProvider broadcasts through FWSS. The service and its payments
	// continue until EndEpoch. The default provider path requires settlement.
	SkipProvider bool
	// WriteOptions apply only to direct submission and receipt waiting.
	// WithWait and WithOnSubmitted are overridden; WithConfirmations is kept.
	WriteOptions []warmstorage.WriteOption
	// DirectWaitTimeout bounds direct receipt waiting. A non-positive value
	// selects five minutes. SubmitTerminateService does not wait for a receipt.
	DirectWaitTimeout time.Duration
	// ProviderWaitTimeout bounds provider submission and, for TerminateService,
	// polling and reconciliation together. Zero selects five minutes; negative
	// adds no deadline beyond ctx.
	ProviderWaitTimeout time.Duration
	// PollInterval applies only to provider polling, not submission.
	PollInterval time.Duration
	// OnSubmitted receives an independent handle synchronously after positive
	// acknowledgement and before waiting. It does not indicate confirmation.
	// Errors also preserve recovery information; callback panics propagate.
	OnSubmitted func(TerminateServiceSubmission)
}

TerminateServiceOptions configures termination submission and confirmation.

type TerminateServiceResult added in v0.4.0

type TerminateServiceResult struct {
	// TxHash is the original transaction hash when available.
	TxHash *common.Hash
	// ConfirmedTxHash is the observed on-chain hash after any replacement.
	// Prefer it for receipt lookups when available; otherwise use TxHash.
	ConfirmedTxHash *common.Hash
	DataSetID       types.BigInt
	// EndEpoch is when service and payments end, not a data deletion time.
	EndEpoch types.Epoch
}

TerminateServiceResult records an arranged service termination. It does not wait for EndEpoch or for data-set cleanup. Provider reconciliation can confirm the goal state without proving that the original relay transaction succeeded.

type TerminateServiceStage added in v0.7.1

type TerminateServiceStage string

TerminateServiceStage identifies where a termination call failed. A wait failure does not establish that the original submission was acknowledged.

const (
	// TerminateServiceStagePrepare guarantees this submission did not send a
	// termination authorization or transaction. Preparation can perform reads.
	TerminateServiceStagePrepare TerminateServiceStage = "prepare"
	TerminateServiceStageSubmit  TerminateServiceStage = "submit"
	TerminateServiceStageWait    TerminateServiceStage = "wait"
)

type TerminateServiceSubmission added in v0.7.1

type TerminateServiceSubmission struct {
	DataSetID              types.BigInt `json:"dataSetId"`
	Direct                 bool         `json:"direct"`
	ProviderID             types.BigInt `json:"providerId"`
	Accepted               bool         `json:"accepted"`
	Queued                 bool         `json:"queued"`
	TransactionID          string       `json:"transactionId"`
	ConfirmedTransactionID string       `json:"confirmedTransactionId"`
}

TerminateServiceSubmission identifies a termination for read-only recovery. Restore it with a client for the original chain and record keeper. Hashes identify transactions; Accepted records positive acknowledgement separately.

type TerminateServiceWaitOptions added in v0.7.1

type TerminateServiceWaitOptions struct {
	// Timeout bounds observation, including provider resolution and any chain
	// reconciliation. A non-positive value selects five minutes.
	Timeout time.Duration
	// PollInterval applies only to provider-relayed status polling.
	PollInterval time.Duration
	// WriteOptions apply only to direct receipt waiting. Timeout overrides
	// WithWait, and WithOnSubmitted is ignored. WithConfirmations is preserved.
	WriteOptions []warmstorage.WriteOption
}

TerminateServiceWaitOptions configures read-only termination confirmation.

type UploadBatcher added in v0.6.0

type UploadBatcher struct {
	// contains filtered or unexported fields
}

UploadBatcher coordinates compatible high-level uploads into bounded PDP add-pieces submissions. It is safe for concurrent use.

func NewUploadBatcher added in v0.6.0

func NewUploadBatcher(opts UploadBatcherOptions, options ...UploadBatcherOption) (*UploadBatcher, error)

NewUploadBatcher creates an independent batching coordinator. By default, a batch is submitted three seconds after its last new piece once no other upload to the same target is in progress, with no maximum batch formation wait, and up to four batches may be signed and submitted concurrently. Each successful provider submission has a five-minute confirmation limit; WithUploadCommitTimeout changes it and WithoutUploadCommitTimeout disables it. UploadCommitTimeoutError retains the submission for continued confirmation. When options of the same kind are repeated, the last one takes effect.

func (*UploadBatcher) Close added in v0.6.0

func (b *UploadBatcher) Close() error

Close aborts open and in-flight batches. It is safe to call concurrently and does not wait for external signers or provider requests to return. Call UploadBatcher.Flush first when pending uploads must complete.

func (*UploadBatcher) Flush added in v0.6.0

func (b *UploadBatcher) Flush(ctx context.Context) error

Flush waits for uploads started before this call to reach the batcher, submits their windows, and waits for final confirmations. Later compatible uploads may join those windows and are not guaranteed to be included or excluded. Canceling ctx stops only this wait, does not cancel shared work, and does not prevent a later Flush from reporting batch failures.

type UploadBatcherOption added in v0.6.0

type UploadBatcherOption func(*uploadBatcherConfig)

UploadBatcherOption configures upload batching behavior. Nil options are ignored.

func WithUploadCommitTimeout added in v0.7.1

func WithUploadCommitTimeout(timeout time.Duration) UploadBatcherOption

WithUploadCommitTimeout limits confirmation waiting after a successful provider submission. The default is five minutes. timeout must be positive; it excludes batch formation, queueing, signing, and submission.

func WithUploadIdleWait added in v0.6.0

func WithUploadIdleWait(wait time.Duration) UploadBatcherOption

WithUploadIdleWait sets how long a batch waits without a new piece before it is submitted. The wait starts once no other upload to the same target is in progress; zero submits as soon as that is true.

func WithUploadMaxConcurrentSubmissions added in v0.6.0

func WithUploadMaxConcurrentSubmissions(max int) UploadBatcherOption

WithUploadMaxConcurrentSubmissions limits concurrent signing and provider submissions. Confirmation waits do not consume this limit.

func WithUploadMaxWait added in v0.6.0

func WithUploadMaxWait(wait time.Duration) UploadBatcherOption

WithUploadMaxWait ends batch formation at most wait after its first piece is ready, even while other uploads to the same target are in progress. The batch then queues for submission; signing, provider requests, and confirmation are outside this limit. Zero queues each piece immediately and requires a zero or disabled idle wait. There is no maximum wait by default.

func WithoutUploadCommitTimeout added in v0.7.1

func WithoutUploadCommitTimeout() UploadBatcherOption

WithoutUploadCommitTimeout lets confirmations wait until completion or Close.

func WithoutUploadIdleWait added in v0.6.0

func WithoutUploadIdleWait() UploadBatcherOption

WithoutUploadIdleWait disables inactivity-based submission.

func WithoutUploadMaxWait added in v0.6.0

func WithoutUploadMaxWait() UploadBatcherOption

WithoutUploadMaxWait removes the maximum wait, which is the default.

type UploadBatcherOptions added in v0.6.0

type UploadBatcherOptions struct {
	Identity ContextIdentity
	Signer   signer.StorageSigner
}

UploadBatcherOptions configures the fixed authorization identity of an UploadBatcher. The signer may be an authorized delegated key and need not have the same address as Identity.Payer.

type UploadCommitTimeoutError added in v0.7.1

type UploadCommitTimeoutError struct {
	Submission CommitSubmission
}

UploadCommitTimeoutError reports an upload whose provider submission was not confirmed within the batcher's confirmation deadline. The transaction may still confirm. Submission retains the handle for WaitForCommit or WaitForCreateAndAdd; see FailedAttempt.Submission for recovery limits.

func (*UploadCommitTimeoutError) Error added in v0.7.1

func (e *UploadCommitTimeoutError) Error() string

func (*UploadCommitTimeoutError) Unwrap added in v0.7.1

func (e *UploadCommitTimeoutError) Unwrap() error

Unwrap makes UploadCommitTimeoutError match context.DeadlineExceeded.

type UploadContextSelection added in v0.5.0

type UploadContextSelection struct {
	// Contexts contains unique targets in primary-to-secondary order.
	Contexts []StorageContext
	// RequestedCopies is the requested copy count passed to the selector.
	RequestedCopies int
	// Complete reports whether Contexts contains exactly RequestedCopies targets.
	Complete bool
}

UploadContextSelection contains the targets selected for one upload.

type UploadOptions

type UploadOptions struct {
	// Copies is the number of provider copies to store. It must be positive.
	Copies int
	// PieceMetadata is validated, signed, and emitted in the FWSS PieceAdded
	// event. FWSS does not persist it in contract state.
	PieceMetadata map[string]string
	// DataSetMetadata is stored with the data set on first creation.
	DataSetMetadata map[string]string
	// ExcludeProviderIDs skips these providers during automatic selection and
	// replacement.
	ExcludeProviderIDs []types.BigInt
	// AllowUnendorsedPrimary relaxes automatic primary selection to the complete
	// approved-provider pool and skips the endorsement query. The zero value keeps
	// strict endorsed-primary selection.
	AllowUnendorsedPrimary bool
	// WithCDN is tri-state: nil inherits the Client-level default
	// configured via synapse.WithCDN; non-nil explicitly overrides
	// for this upload. Declare a local variable to take its address:
	//
	//	b := true
	//	opts := &storage.UploadOptions{WithCDN: &b}
	WithCDN *bool
	// PieceCID, when defined, is a pre-computed PieceCIDv2 of the payload.
	// When set, the primary provider client skips inline commP calculation;
	// the server still verifies the uploaded bytes match this value.
	PieceCID cid.Cid
	// OnProgress is invoked after each non-empty Read from the upload
	// reader, with the cumulative bytes sent to the primary provider so
	// far. It may be nil.
	OnProgress func(bytesUploaded int64)
	// OnStored is invoked once the primary provider has confirmed storage of
	// the piece. It may be nil.
	OnStored func(providerID types.BigInt, pieceCID cid.Cid)
	// OnPiecesAdded is invoked after the on-chain AddPieces transaction is
	// submitted for a provider (primary or secondary), carrying the transaction
	// hash and the batch of pieces included in that transaction. During
	// Service.Upload, it may overlap other providers' submission and secondary
	// callbacks. It may be nil.
	OnPiecesAdded func(txHash string, providerID types.BigInt, pieces []SubmittedPiece)
	// OnPiecesConfirmed is invoked after the on-chain AddPieces transaction is
	// confirmed (CommitResult received) for a provider, carrying the assigned
	// on-chain IDs for each piece. During Service.Upload, this callback is
	// invoked sequentially after all commit workers finish. It may be nil.
	OnPiecesConfirmed func(dataSetID, providerID types.BigInt, pieces []ConfirmedPiece)
	// OnCopyComplete is invoked once a secondary provider's SP-to-SP pull
	// completes successfully. It is not fired for the primary (which stores
	// directly). Different providers may invoke it concurrently. It may be nil.
	OnCopyComplete func(providerID types.BigInt, pieceCID cid.Cid)
	// OnCopyFailed is invoked when a secondary provider's SP-to-SP copy
	// attempt fails. Presign failures are not copy attempts and still surface
	// only through FailedAttempts with CopyStagePresign. Primary store/commit
	// failures likewise surface through the Upload return value and
	// FailedAttempts. Different providers may invoke it concurrently. It may be nil.
	OnCopyFailed func(providerID types.BigInt, pieceCID cid.Cid, err error)
	// OnPullProgress is invoked for each piece status update during a
	// secondary-provider pull. Different providers may invoke it concurrently.
	// It may be nil.
	OnPullProgress func(providerID types.BigInt, pieceCID cid.Cid, status PullStatus)
}

UploadOptions configures Service.Upload. Copies must be positive.

Lifecycle callbacks may be invoked from internal orchestration goroutines. Handlers that share mutable state must be concurrency-safe. A callback panic stops the upload as if ctx were canceled and suppresses later callbacks; once the upload has unwound, Upload re-panics with the same value on the calling goroutine. When a logger is configured, the original stack is logged at Error level first. Work already handed to a batcher may still commit.

type UploadResolver

type UploadResolver interface {
	// ResolveUploadContexts selects the initial targets for one automatic upload.
	ResolveUploadContexts(context.Context, SelectUploadContextsOptions) ([]StorageContext, error)
	// SelectReplacement selects one replacement target. ExcludeProviderIDs is the
	// deduplicated union of caller exclusions and every provider already selected
	// or attempted by the upload.
	SelectReplacement(context.Context, SelectProviderContextOptions) (StorageContext, error)
}

UploadResolver selects provider contexts for upload operations and provides replacement candidates when a secondary provider fails. Implementations must be safe for concurrent use by independent uploads. Each call receives an independently owned options value, including its slice, map, and pointer fields, which the implementation may read, modify, or retain without sharing state with the upload pipeline or caller.

type UploadResult

type UploadResult struct {
	PieceCID        cid.Cid // PieceCIDv2 of the stored data
	Size            int64   // raw (unpadded) byte count
	RequestedCopies int
	// Complete is true when all RequestedCopies were committed on-chain.
	// Equivalent to len(Copies) >= RequestedCopies.
	Complete       bool
	Copies         []CopyResult
	FailedAttempts []FailedAttempt
}

UploadResult is returned by a successful Upload call.

Use Complete to determine overall success: it is true when every requested copy was committed on-chain (equivalent to len(Copies) >= RequestedCopies). A non-empty FailedAttempts slice does NOT indicate overall failure — failed attempts may have been resolved by successful retries on other providers.

Example:

result, err := m.Upload(ctx, r, opts)
if err != nil { ... }
if !result.Complete {
    log.Printf("partial upload: %d/%d copies", result.SuccessCount(), result.RequestedCopies)
}

func (*UploadResult) PartialSuccess

func (r *UploadResult) PartialSuccess() bool

PartialSuccess reports whether at least one copy was committed on-chain but fewer than the requested number were obtained. Returns false when Complete is true or when no copies succeeded at all.

func (*UploadResult) PrimaryDataSetID

func (r *UploadResult) PrimaryDataSetID() (types.BigInt, bool)

PrimaryDataSetID returns the DataSetID of the primary copy.

ok is false when no primary copy committed on-chain (even if secondaries did). Callers that need precise provenance should inspect UploadResult.Copies directly.

func (*UploadResult) SuccessCount

func (r *UploadResult) SuccessCount() int

SuccessCount returns the number of copies that were successfully committed on-chain. Equivalent to len(Copies).

func (*UploadResult) SuccessfulProviderIDs

func (r *UploadResult) SuccessfulProviderIDs() []types.BigInt

SuccessfulProviderIDs returns the ProviderID of every copy that committed on-chain, in the order the copies appear in UploadResult.Copies.

type UploadToContextsOptions added in v0.6.0

type UploadToContextsOptions struct {
	// PieceMetadata is validated, signed, and emitted in the FWSS PieceAdded
	// event. FWSS does not persist it in contract state.
	PieceMetadata map[string]string
	// PieceCID, when defined, is a pre-computed PieceCIDv2 of the payload.
	// When set, the primary provider client skips inline commP calculation;
	// the server still verifies the uploaded bytes match this value.
	PieceCID cid.Cid
	// OnProgress is invoked after each non-empty Read from the upload reader,
	// with the cumulative bytes sent to the primary provider so far. It may be nil.
	OnProgress func(bytesUploaded int64)
	// OnStored is invoked once the primary provider has confirmed storage of
	// the piece. It may be nil.
	OnStored func(providerID types.BigInt, pieceCID cid.Cid)
	// OnPiecesAdded is invoked after the on-chain AddPieces transaction is
	// submitted for a provider, carrying the transaction hash and submitted pieces.
	// It may overlap other providers' submission and secondary callbacks.
	// It may be nil.
	OnPiecesAdded func(txHash string, providerID types.BigInt, pieces []SubmittedPiece)
	// OnPiecesConfirmed is invoked after AddPieces is confirmed for a provider,
	// carrying the assigned on-chain IDs. It runs sequentially in target order
	// after all upload workers finish, unless canceled. It may be nil.
	OnPiecesConfirmed func(dataSetID, providerID types.BigInt, pieces []ConfirmedPiece)
	// OnCopyComplete is invoked once a secondary provider's SP-to-SP pull
	// completes successfully. It is not fired for the primary. Different
	// providers may invoke it concurrently. It may be nil.
	OnCopyComplete func(providerID types.BigInt, pieceCID cid.Cid)
	// OnCopyFailed is invoked when a secondary provider's SP-to-SP copy attempt
	// fails. Presign and commit failures are reported through FailedAttempts.
	// Different providers may invoke it concurrently. It may be nil.
	OnCopyFailed func(providerID types.BigInt, pieceCID cid.Cid, err error)
	// OnPullProgress is invoked for each piece status update during a
	// secondary-provider pull. Different providers may invoke it concurrently.
	// It may be nil.
	OnPullProgress func(providerID types.BigInt, pieceCID cid.Cid, status PullStatus)
}

UploadToContextsOptions configures Service.UploadToContexts. The target contexts and their primary-to-secondary order are supplied separately.

Lifecycle callbacks may be invoked from internal orchestration goroutines. Handlers that share mutable state must be concurrency-safe. Callback panics are handled as described for UploadOptions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL