Documentation
¶
Overview ¶
Package storage provides provider selection, immutable storage targets, and single- or multi-copy upload orchestration.
Contexts ¶
ProviderContext identifies one provider but no data set. Its Commit and Pull operations create a new data set. DataSetContext identifies one provider and one existing data set; its Commit and Pull operations always use that data set. Neither type changes target after construction.
Both types expose provider-scoped operations such as Store and Download. Download therefore behaves the same on both: it retrieves the requested piece from the configured provider or CDN. Data-set inspection, deletion, and termination methods are available only on DataSetContext, while standalone data-set creation and recovery are available only on ProviderContext.
Use Service.NewProviderContext or Service.NewDataSetContext when the target ID is already known. Use Service.SelectProviderContext to choose one healthy provider without looking up data sets. Use Service.SelectUploadContexts when preparing a new upload; it may reuse writable data sets whose metadata matches.
DataSetRef is the persistent target reference shared by creation results, DataSetContext, and ProviderContext.ForDataSet. Construct it with NewDataSetRef; its zero value is invalid.
Persistent data-set references, context identities, and create/commit lifecycle values use strict lowerCamel JSON field names. Alternate capitalization, unknown or duplicate fields, and incomplete objects are rejected.
Upload flow ¶
Service.Upload automatically selects targets and performs store, pull, and commit. Copies must be explicitly positive. If fewer targets are available, the upload continues with those targets and reports the requested and actual copy counts through UploadResult.
For an exact preflight and upload, use the same context instances throughout:
- Call Service.SelectUploadContexts. A non-empty partial selection is returned with InsufficientUploadContextsError.
- Pass the selected contexts to Service.Prepare.
- Execute the returned PrepareTransaction, if any.
- Pass the same contexts, in the desired primary-to-secondary order, to Service.UploadToContexts.
UploadToContexts does not select replacements. The first context stores the reader; later contexts pull from it. Service.Upload retains automatic replacement for failed secondary copies.
Contexts carry an immutable ContextIdentity containing payer, chain, and record-keeper identities. Service validates this identity before cost calculation or upload work, preventing contexts from another account or chain from being used accidentally.
Submission recovery ¶
ProviderContext.CreateDataSet leaves its receiver unbound. Persist the CreateDataSetSubmission received through CreateDataSetOptions.OnSubmitted when confirmation must survive a restart. A fresh ProviderContext for the same provider can resume with ProviderContext.WaitForDataSetCreated. Pass the returned DataSetRef to ProviderContext.ForDataSet to obtain a DataSetContext.
ProviderContext.Commit and DataSetContext.Commit are convenience methods that submit once and wait for confirmation. Applications that must survive process restarts can split that lifecycle with SubmitCommit, GetCommitStatus, and WaitForCommit on the same concrete context type. Persist the complete CommitSubmission returned by SubmitCommit before waiting. A fresh context for the same immutable target can resume that submission without signing or submitting another transaction.
GetCommitStatus performs one logical status check and returns CommitStatePending, CommitStateConfirmed, or CommitStateRejected. A rejected status is returned without an error; WaitForCommit reports the same terminal state as CommitRejectedError.
Downloads ¶
Context downloads use the PDP and optional CDN clients attached to that context. They enforce the exact raw payload size encoded in PieceCIDv2 and return ErrMaxBytesExceeded if a response is larger. URL-based Service.Download uses the Service HTTP client. Its default client rejects private and reserved network destinations; configure Options.AllowPrivateNetworks only for trusted private infrastructure. Options.DownloadMaxBytes can cap URL-based downloads.
Stability ¶
During the 0.x phase, public APIs may change between minor releases.
Example ¶
Example demonstrates uploading a payload via a storage.Service. In practice a Service is obtained from synapse.Client.Storage after calling synapse.New; this example assumes svc is already wired.
package main
import (
"bytes"
"context"
"fmt"
"log"
"github.com/ipfs/go-cid"
"github.com/strahe/synapse-go/storage"
"github.com/strahe/synapse-go/types"
)
func main() {
var svc *storage.Service // obtained from synapse.Client.Storage()
ctx := context.Background()
payload := bytes.NewReader(bytes.Repeat([]byte("data"), 64))
res, err := svc.Upload(ctx, payload, &storage.UploadOptions{
OnProgress: func(uploaded int64) {
log.Printf("uploaded %d bytes", uploaded)
},
OnStored: func(providerID types.BigInt, pieceCID cid.Cid) {
log.Printf("stored %s on provider %s", pieceCID, providerID)
},
OnPiecesConfirmed: func(dataSetID, providerID types.BigInt, pieces []storage.ConfirmedPiece) {
log.Printf("provider %s confirmed %d piece(s) in dataset %s", providerID, len(pieces), dataSetID)
},
})
if err != nil {
log.Fatal(err)
}
fmt.Println(res.PieceCID)
}
Output:
Index ¶
- Variables
- type Allowances
- type CDNRetriever
- type CIDMismatchError
- type CommitError
- type CommitKind
- type CommitRejectedError
- type CommitRequest
- type CommitResult
- type CommitState
- type CommitStatus
- type CommitSubmission
- type ConfirmedPiece
- type ContextCostRef
- type ContextFactory
- type ContextFactoryOptions
- type ContextIdentity
- type ContextOption
- func WithCDN(enabled bool) ContextOption
- func WithCDNRetriever(r CDNRetriever) ContextOption
- func WithChainID(chainID types.ChainID) ContextOption
- func WithDataSetMetadata(metadata map[string]string) ContextOption
- func WithDataSetValidator(v DataSetValidator) ContextOption
- func WithFWSSDataSetReader(r FWSSDataSetReader) ContextOption
- func WithFWSSTerminator(t FWSSTerminator) ContextOption
- func WithLogger(logger *slog.Logger) ContextOption
- func WithPDPConfigReader(r PDPConfigReader) ContextOption
- func WithPDPVerifierReader(r PDPVerifierReader) ContextOption
- func WithPayer(payer common.Address) ContextOption
- func WithPaymentStateReader(pay PaymentStateReader, epochs EpochReader, token common.Address) ContextOption
- func WithRecordKeeper(addr common.Address) ContextOption
- type ContextResolver
- type ContextSelector
- type CopyResult
- type CopyRole
- type CopyStage
- type CreateDataSetOptions
- type CreateDataSetResult
- type CreateDataSetSubmission
- type DataSetCatalog
- type DataSetContext
- func (c *DataSetContext) CDNEnabled() bool
- func (c *DataSetContext) ClientDataSetID() types.BigInt
- func (c *DataSetContext) Commit(ctx context.Context, req CommitRequest) (*CommitResult, error)
- func (c *DataSetContext) ContextIdentity() ContextIdentity
- func (c *DataSetContext) DataSetID() types.BigInt
- func (c *DataSetContext) DataSetRef() (DataSetRef, bool)
- func (c *DataSetContext) DeletePiece(ctx context.Context, pieceCID cid.Cid) (*sdktypes.WriteResult, error)
- func (c *DataSetContext) DeletePieceByID(ctx context.Context, pieceID sdktypes.BigInt) (*sdktypes.WriteResult, error)
- func (c *DataSetContext) Download(ctx context.Context, pieceCID cid.Cid) (io.ReadCloser, error)
- func (c *DataSetContext) GetCommitStatus(ctx context.Context, submission CommitSubmission) (*CommitStatus, error)
- func (c *DataSetContext) GetProviderInfo() Provider
- func (c *DataSetContext) GetScheduledRemovals(ctx context.Context) ([]types.BigInt, error)
- func (c *DataSetContext) PieceStatus(ctx context.Context, pieceCID cid.Cid) (*PieceStatus, error)
- func (c *DataSetContext) PieceURL(pieceCID cid.Cid) string
- func (c *DataSetContext) PresignForCommit(ctx context.Context, pieces []PieceInput) ([]byte, error)
- func (c *DataSetContext) ProviderID() types.BigInt
- func (c *DataSetContext) Pull(ctx context.Context, req PullRequest) (*PullResult, error)
- func (c *DataSetContext) ServiceURL() string
- func (c *DataSetContext) Store(ctx context.Context, r io.Reader, opts *StoreOptions) (*StoreResult, error)
- func (c *DataSetContext) SubmitCommit(ctx context.Context, req CommitRequest) (*CommitSubmission, error)
- func (c *DataSetContext) Terminate(ctx context.Context, opts ...warmstorage.WriteOption) (*types.WriteResult, error)
- func (c *DataSetContext) TerminateService(ctx context.Context, opts *TerminateServiceOptions) (*TerminateServiceResult, error)
- func (c *DataSetContext) Upload(ctx context.Context, r io.Reader, opts *UploadOptions) (*UploadResult, error)
- func (c *DataSetContext) WaitForCommit(ctx context.Context, submission CommitSubmission) (*CommitResult, error)
- func (c *DataSetContext) WithCDN() booldeprecated
- type DataSetDetailsCatalog
- type DataSetFinder
- type DataSetInfo
- type DataSetPDPPaymentTerminatedError
- type DataSetRef
- type DataSetSizeReader
- type DataSetValidator
- type DownloadContext
- type DownloadError
- type DownloadOptions
- type EpochReader
- type FWSSDataSetReader
- type FWSSTerminator
- type FailedAttempt
- type FindDataSetsOptions
- type GetStorageInfoOptions
- type InsufficientUploadContextsError
- type MultiContextCosts
- type MultiCostCalculator
- type MultiCostOptions
- type NewDataSetContextOptions
- type NewProviderContextOptions
- type Options
- type PDPConfigReader
- type PDPProviderClient
- type PDPProviderSource
- type PDPVerifierReader
- type PaymentStateReader
- type PaymentsFunder
- type PieceInput
- type PieceStatus
- type PrepareOptions
- type PrepareResult
- type PrepareTransaction
- type PricePerTiB
- type PricingInfo
- type Provider
- type ProviderContext
- func (c *ProviderContext) CDNEnabled() bool
- func (c *ProviderContext) Commit(ctx context.Context, req CommitRequest) (*CommitResult, error)
- func (c *ProviderContext) ContextIdentity() ContextIdentity
- func (c *ProviderContext) CreateDataSet(ctx context.Context, opts *CreateDataSetOptions) (*CreateDataSetResult, error)
- func (c *ProviderContext) DataSetRef() (DataSetRef, bool)
- func (c *ProviderContext) Download(ctx context.Context, pieceCID cid.Cid) (io.ReadCloser, error)
- func (c *ProviderContext) ForDataSet(ref DataSetRef) (*DataSetContext, error)
- func (c *ProviderContext) GetCommitStatus(ctx context.Context, submission CommitSubmission) (*CommitStatus, error)
- func (c *ProviderContext) GetProviderInfo() Provider
- func (c *ProviderContext) PieceURL(pieceCID cid.Cid) string
- func (c *ProviderContext) PresignForCommit(ctx context.Context, pieces []PieceInput) ([]byte, error)
- func (c *ProviderContext) ProviderID() types.BigInt
- func (c *ProviderContext) Pull(ctx context.Context, req PullRequest) (*PullResult, error)
- func (c *ProviderContext) ServiceURL() string
- func (c *ProviderContext) Store(ctx context.Context, r io.Reader, opts *StoreOptions) (*StoreResult, error)
- func (c *ProviderContext) SubmitCommit(ctx context.Context, req CommitRequest) (*CommitSubmission, error)
- func (c *ProviderContext) Upload(ctx context.Context, r io.Reader, opts *UploadOptions) (*UploadResult, error)
- func (c *ProviderContext) WaitForCommit(ctx context.Context, submission CommitSubmission) (*CommitResult, error)
- func (c *ProviderContext) WaitForDataSetCreated(ctx context.Context, submission CreateDataSetSubmission) (*CreateDataSetResult, error)
- func (c *ProviderContext) WithCDN() booldeprecated
- type ProviderResolver
- type PullPieceResult
- type PullRequest
- type PullResult
- type PullStatus
- type SelectProviderContextOptions
- type SelectUploadContextsOptions
- type Service
- func (s *Service) CalculateMultiContextCosts(ctx context.Context, dataSizeBytes uint64, refs []ContextCostRef, ...) (*MultiContextCosts, error)
- func (s *Service) Download(ctx context.Context, pieceCID cid.Cid, opts *DownloadOptions) (io.ReadCloser, error)
- func (s *Service) FindDataSets(ctx context.Context, opts *FindDataSetsOptions) ([]*DataSetInfo, error)
- func (s *Service) GetStorageInfo(ctx context.Context, opts *GetStorageInfoOptions) (*StorageInfo, error)
- func (s *Service) NewDataSetContext(ctx context.Context, dataSetID types.BigInt, opts NewDataSetContextOptions) (*DataSetContext, error)
- func (s *Service) NewProviderContext(ctx context.Context, providerID types.BigInt, opts NewProviderContextOptions) (*ProviderContext, error)
- func (s *Service) Prepare(ctx context.Context, opts *PrepareOptions) (*PrepareResult, error)
- func (s *Service) SelectProviderContext(ctx context.Context, opts SelectProviderContextOptions) (*ProviderContext, error)
- func (s *Service) SelectUploadContexts(ctx context.Context, opts SelectUploadContextsOptions) (*UploadContextSelection, error)
- func (s *Service) TerminateDataSet(ctx context.Context, dataSetID types.BigInt, opts *TerminateDataSetOptions) (*types.WriteResult, error)
- func (s *Service) TerminateService(ctx context.Context, dataSetID types.BigInt, opts *TerminateServiceOptions) (*TerminateServiceResult, error)
- func (s *Service) Upload(ctx context.Context, r io.Reader, opts *UploadOptions) (*UploadResult, error)
- func (s *Service) UploadToContexts(ctx context.Context, r io.Reader, contexts []StorageContext, ...) (*UploadResult, error)
- type ServiceParameters
- type ServiceResolver
- func (r *ServiceResolver) ResolveDataSetContext(ctx context.Context, dataSetID types.BigInt, opts NewDataSetContextOptions) (*DataSetContext, error)
- func (r *ServiceResolver) ResolveProvider(ctx context.Context, providerID types.BigInt) (Provider, error)
- func (r *ServiceResolver) ResolveProviderContext(ctx context.Context, providerID types.BigInt, opts NewProviderContextOptions) (*ProviderContext, error)
- func (r *ServiceResolver) ResolveUploadContexts(ctx context.Context, opts *UploadOptions) ([]StorageContext, bool, error)
- func (r *ServiceResolver) SelectProviderContext(ctx context.Context, opts SelectProviderContextOptions) (*ProviderContext, error)
- func (r *ServiceResolver) SelectReplacement(ctx context.Context, usedProviders map[string]types.BigInt, ...) (StorageContext, error)
- func (r *ServiceResolver) SelectUploadContexts(ctx context.Context, opts SelectUploadContextsOptions) (*UploadContextSelection, error)
- type ServiceResolverOptions
- type StorageContext
- type StorageInfo
- type StorageInfoReader
- type StoreError
- type StoreOptions
- type StoreResult
- type SubmittedPiece
- type TerminateDataSetOptions
- type TerminateServiceDebtError
- type TerminateServiceOptions
- type TerminateServiceResult
- type UploadContextSelection
- type UploadFees
- type UploadLockup
- type UploadOptions
- type UploadResolver
- type UploadResult
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrClosed = lifecycle.ErrClosed
ErrClosed is returned when a method is called after the owning Client has been closed. It aliases the shared closed-client sentinel.
var ErrInsufficientUploadContexts = errors.New("storage: insufficient upload contexts")
ErrInsufficientUploadContexts is matched by InsufficientUploadContextsError when selection found at least one, but fewer than the requested number of upload targets.
var ErrInvalidArgument = errors.New("storage: invalid argument")
ErrInvalidArgument is returned, wrapped via fmt.Errorf with %w, when a caller passes an argument that violates a precondition (nil pointer, zero ID, empty required URL, undefined pieceCID, etc.). Match with errors.Is(err, storage.ErrInvalidArgument).
Business / server-invariant errors (no approved providers, duplicate dataset IDs, server returned zero dataSetID, etc.) are intentionally left as plain errors so that errors.Is against ErrInvalidArgument only matches genuine caller-supplied validation failures.
var ErrInvalidDownloadOptions = errors.New("storage: invalid download options")
ErrInvalidDownloadOptions is returned when DownloadOptions is nil, empty, or specifies more than one download source. Service.Download wraps it together with ErrInvalidArgument, so callers may match either.
var ErrMaxBytesExceeded = errors.New("storage: download exceeded MaxBytes")
ErrMaxBytesExceeded is returned when a download exceeds its byte limit. Context downloads use the raw payload size encoded in PieceCIDv2; URL-based Service.Download calls use a non-zero Options.DownloadMaxBytes. The error is surfaced either eagerly when a trustworthy Content-Length is too large or via the terminal Read on the returned reader.
var ErrNoHealthyProviders = errors.New("storage: no healthy providers")
ErrNoHealthyProviders is returned when every eligible automatically selected provider fails its PDP health check.
var ErrPrivateNetwork = errors.New("storage: private / local network address disallowed")
ErrPrivateNetwork is returned by Service.Download when the target URL resolves to a loopback / link-local / RFC1918 / ULA / multicast / unspecified address and the Service was constructed without Options.AllowPrivateNetworks. It prevents SDK callers from being used as SSRF egress against internal networks.
var ErrUninitialized = errors.New("storage: service not initialized; use storage.New")
ErrUninitialized is returned when a method is invoked on a zero-value Service (one that was not constructed via New).
var ErrUnsupportedScheme = errors.New("storage: unsupported URL scheme")
ErrUnsupportedScheme is returned when the URL passed to Service.Download uses a scheme other than http or https.
Functions ¶
This section is empty.
Types ¶
type Allowances ¶
type Allowances struct {
Service common.Address
IsApproved bool
RateAllowance *big.Int
LockupAllowance *big.Int
RateUsed *big.Int
LockupUsed *big.Int
MaxLockupPeriod *big.Int
}
Allowances is the client-specific operator-approval view embedded in StorageInfo. Nil when the caller has no wallet or the lookup failed.
type CDNRetriever ¶ added in v0.2.0
CDNRetriever provides optional CDN-backed piece retrieval for storage contexts.
type CIDMismatchError ¶
CIDMismatchError is returned when the piece CID computed from the downloaded bytes does not match the expected CID. Use errors.AsType to retrieve the computed and expected CIDs for diagnostic purposes.
func (*CIDMismatchError) Error ¶
func (e *CIDMismatchError) Error() string
type CommitError ¶
CommitError is returned when all on-chain commit attempts fail and no copies are stored. Individual per-provider failures are reported in UploadResult.FailedAttempts.
func (*CommitError) Error ¶
func (e *CommitError) Error() string
func (*CommitError) Unwrap ¶
func (e *CommitError) Unwrap() error
type CommitKind ¶ added in v0.5.0
type CommitKind string
CommitKind identifies whether a submission creates a data set or adds to an existing one.
const ( // CommitKindCreateAndAdd creates a new data set and adds pieces to it. CommitKindCreateAndAdd CommitKind = "create-and-add" // CommitKindAddPieces adds pieces to an existing data set. CommitKindAddPieces CommitKind = "add-pieces" )
type CommitRejectedError ¶ added in v0.5.0
type CommitRejectedError struct {
Submission CommitSubmission
Status CommitStatus
}
CommitRejectedError reports a valid terminal rejection while preserving the submission and final status needed for diagnostics.
func (*CommitRejectedError) Error ¶ added in v0.5.0
func (e *CommitRejectedError) Error() string
func (*CommitRejectedError) Unwrap ¶ added in v0.5.0
func (e *CommitRejectedError) Unwrap() error
Unwrap makes CommitRejectedError match pdp.ErrTxRejected.
type CommitRequest ¶
type CommitRequest struct {
Pieces []PieceInput
ExtraData []byte // EIP-712 signed payload; nil for the primary (create-or-add path)
// OnSubmitted is invoked with the original transaction hash immediately
// after the provider returns a valid submission handle, before confirmation.
// It may be nil. Direct Commit calls do not recover callback panics.
OnSubmitted func(txHash string)
}
CommitRequest triggers on-chain registration of pieces for one provider.
type CommitResult ¶
type CommitResult struct {
TransactionID string `json:"transactionId"` // transaction hash from the provider submission
ConfirmedTransactionID string `json:"confirmedTransactionId"` // actual on-chain hash, when reported by the provider
DataSet DataSetRef `json:"dataSet"`
PieceIDs []types.BigInt `json:"pieceIds"`
IsNewDataSet bool `json:"isNewDataSet"` // true when a new data set was created by this commit
}
CommitResult is returned by a successful Commit call.
func (*CommitResult) UnmarshalJSON ¶ added in v0.5.0
func (r *CommitResult) UnmarshalJSON(data []byte) error
type CommitState ¶ added in v0.5.0
type CommitState string
CommitState is the durable, provider-independent state of a submission.
const ( // CommitStatePending means the provider has not reported a terminal result. CommitStatePending CommitState = "pending" // CommitStateConfirmed means all submitted pieces were added successfully. CommitStateConfirmed CommitState = "confirmed" // CommitStateRejected means the transaction failed or was rejected. CommitStateRejected CommitState = "rejected" )
type CommitStatus ¶ added in v0.5.0
type CommitStatus struct {
Kind CommitKind `json:"kind"`
State CommitState `json:"state"`
TransactionID string `json:"transactionId"`
ConfirmedTransactionID string `json:"confirmedTransactionId"`
DataSet *DataSetRef `json:"dataSet"`
PieceIDs []types.BigInt `json:"pieceIds"`
}
CommitStatus is one logical status snapshot for a submission.
func (*CommitStatus) UnmarshalJSON ¶ added in v0.5.0
func (s *CommitStatus) UnmarshalJSON(data []byte) error
type CommitSubmission ¶ added in v0.5.0
type CommitSubmission struct {
Kind CommitKind `json:"kind"`
TransactionID string `json:"transactionId"`
StatusURL string `json:"statusUrl"`
ProviderID types.BigInt `json:"providerId"`
Identity ContextIdentity `json:"identity"`
DataSet *DataSetRef `json:"dataSet"`
ClientDataSetID *types.BigInt `json:"clientDataSetId"`
PieceCIDs []cid.Cid `json:"pieceCids"`
}
CommitSubmission is a persistable handle returned after one successful provider submission. Persist all fields together and resume it with the same concrete context type. Its JSON form uses strict lowerCamel field names and rejects alternate capitalization.
func (*CommitSubmission) UnmarshalJSON ¶ added in v0.5.0
func (s *CommitSubmission) UnmarshalJSON(data []byte) error
type ConfirmedPiece ¶
ConfirmedPiece carries the on-chain identity reported by an OnPiecesConfirmed callback.
type ContextCostRef ¶
type ContextCostRef struct {
DataSetID *types.BigInt
Provider Provider
CurrentDataSetSizeBytes *big.Int
WithCDN bool
}
ContextCostRef references one prospective upload target for multi-context cost aggregation. A nil DataSetID means "a new data set will be created on this provider". CurrentDataSetSizeBytes is used only when DataSetID is non-nil; zero means the caller does not have the current size handy and wants a zero-size estimate. WithCDN determines whether the CDN-fixed lockup is included for this ref.
type ContextFactory ¶
type ContextFactory func(Provider, ContextFactoryOptions) (*ProviderContext, error)
ContextFactory builds an unbound immutable provider context.
type ContextFactoryOptions ¶ added in v0.5.0
ContextFactoryOptions configures construction of one provider context.
type ContextIdentity ¶ added in v0.5.0
type ContextIdentity struct {
Payer common.Address `json:"payer"`
ChainID types.ChainID `json:"chainId"`
RecordKeeper common.Address `json:"recordKeeper"`
}
ContextIdentity identifies the account and chain configuration used by a storage context for signing and on-chain operations. Its JSON form uses strict lowerCamel field names.
func (*ContextIdentity) UnmarshalJSON ¶ added in v0.5.0
func (i *ContextIdentity) UnmarshalJSON(data []byte) error
type ContextOption ¶
type ContextOption func(*contextCore)
ContextOption configures provider and data-set contexts during construction.
func WithCDN ¶
func WithCDN(enabled bool) ContextOption
WithCDN enables CDN services for the data set and CDN-first downloads when a retriever is configured. When true, a "withCDN" metadata marker is added to the EIP-712 dataset-creation message; the contract activates CDN and applies its configured lockup upon seeing it.
func WithCDNRetriever ¶ added in v0.2.0
func WithCDNRetriever(r CDNRetriever) ContextOption
WithCDNRetriever injects the optional CDN retriever used by context downloads.
func WithChainID ¶
func WithChainID(chainID types.ChainID) ContextOption
WithChainID sets the EIP-155 chain ID used for EIP-712 domain separation.
func WithDataSetMetadata ¶
func WithDataSetMetadata(metadata map[string]string) ContextOption
WithDataSetMetadata sets the key-value metadata stored with the data set on creation.
func WithDataSetValidator ¶ added in v0.3.0
func WithDataSetValidator(v DataSetValidator) ContextOption
WithDataSetValidator injects the validator used before uploading or committing pieces to an existing data set.
func WithFWSSDataSetReader ¶ added in v0.3.0
func WithFWSSDataSetReader(r FWSSDataSetReader) ContextOption
WithFWSSDataSetReader injects the reader used before uploads to reject existing data sets whose PDP payment rail has ended.
func WithFWSSTerminator ¶
func WithFWSSTerminator(t FWSSTerminator) ContextOption
WithFWSSTerminator injects the terminator used by DataSetContext.Terminate.
func WithLogger ¶ added in v0.2.0
func WithLogger(logger *slog.Logger) ContextOption
WithLogger sets the logger used for internal warnings.
func WithPDPConfigReader ¶
func WithPDPConfigReader(r PDPConfigReader) ContextOption
WithPDPConfigReader injects a reader for FWSSView PDPConfig. Required by DataSetContext.PieceStatus for challenge-window math.
func WithPDPVerifierReader ¶
func WithPDPVerifierReader(r PDPVerifierReader) ContextOption
WithPDPVerifierReader injects a reader for PDPVerifier contract state. Required by DataSetContext.GetScheduledRemovals, DataSetContext.PieceStatus, and DataSetContext.DeletePiece.
func WithPayer ¶
func WithPayer(payer common.Address) ContextOption
WithPayer sets the EVM address that pays for storage.
func WithPaymentStateReader ¶ added in v0.4.0
func WithPaymentStateReader(pay PaymentStateReader, epochs EpochReader, token common.Address) ContextOption
WithPaymentStateReader injects payment readers for provider-relayed termination debt pre-checks.
func WithRecordKeeper ¶
func WithRecordKeeper(addr common.Address) ContextOption
WithRecordKeeper sets the FWSS contract address (record-keeper) used for EIP-712 signing and passed to the PDP provider for Pull and dataset creation.
type ContextResolver ¶ added in v0.3.0
type ContextResolver interface {
ResolveProviderContext(context.Context, types.BigInt, NewProviderContextOptions) (*ProviderContext, error)
ResolveDataSetContext(context.Context, types.BigInt, NewDataSetContextOptions) (*DataSetContext, error)
}
ContextResolver opens explicitly identified provider and data-set targets.
type ContextSelector ¶ added in v0.5.0
type ContextSelector interface {
SelectProviderContext(context.Context, SelectProviderContextOptions) (*ProviderContext, error)
SelectUploadContexts(context.Context, SelectUploadContextsOptions) (*UploadContextSelection, error)
}
ContextSelector chooses healthy targets for new uploads.
type CopyResult ¶
type CopyResult struct {
ProviderID types.BigInt
DataSetID types.BigInt
PieceID types.BigInt
Role CopyRole
RetrievalURL string // HTTPS retrieval URL for this piece on the provider.
IsNewDataSet bool
}
CopyResult describes one successfully committed copy.
type CopyRole ¶
type CopyRole string
CopyRole identifies the role a provider plays in a multi-copy upload.
type CopyStage ¶
type CopyStage string
CopyStage identifies the pipeline stage at which a provider attempt failed.
type CreateDataSetOptions ¶ added in v0.2.0
type CreateDataSetOptions struct {
// OnSubmitted is invoked after the create transaction is submitted and
// before waiting for confirmation. It may be nil.
OnSubmitted func(CreateDataSetSubmission)
}
CreateDataSetOptions configures ProviderContext.CreateDataSet.
type CreateDataSetResult ¶ added in v0.2.0
type CreateDataSetResult struct {
TransactionID string `json:"transactionId"`
ConfirmedTransactionID string `json:"confirmedTransactionId"`
DataSet DataSetRef `json:"dataSet"`
}
CreateDataSetResult is returned after standalone dataset creation confirms.
func (*CreateDataSetResult) UnmarshalJSON ¶ added in v0.5.0
func (r *CreateDataSetResult) UnmarshalJSON(data []byte) error
type CreateDataSetSubmission ¶ added in v0.2.0
type CreateDataSetSubmission struct {
ProviderID types.BigInt `json:"providerId"`
TransactionID string `json:"transactionId"`
StatusURL string `json:"statusUrl"`
// ClientDataSetID must be non-nil when resuming a submitted create.
ClientDataSetID *types.BigInt `json:"clientDataSetId"`
}
CreateDataSetSubmission identifies a submitted create-dataset transaction. Persist and restore all fields together using the strict lowerCamel JSON form; incomplete submissions and alternate capitalization are rejected. A zero ProviderID is filled from the ProviderContext used to wait.
func (*CreateDataSetSubmission) UnmarshalJSON ¶ added in v0.5.0
func (s *CreateDataSetSubmission) UnmarshalJSON(data []byte) error
type DataSetCatalog ¶
type DataSetCatalog interface {
GetApprovedProviderIDs(context.Context, types.ListOptions) ([]types.BigInt, error)
GetClientDataSets(context.Context, common.Address, types.ListOptions) ([]*warmstorage.DataSetInfo, error)
GetDataSet(context.Context, types.BigInt) (*warmstorage.DataSetInfo, error)
GetAllDataSetMetadata(context.Context, types.BigInt) (map[string]string, error)
}
DataSetCatalog is the subset of warmstorage.Service used by ServiceResolver.
type DataSetContext ¶ added in v0.5.0
type DataSetContext struct {
// contains filtered or unexported fields
}
DataSetContext represents one immutable provider + data-set target. Commit and Pull operations always target Ref. It is safe for concurrent use.
func NewDataSetContext ¶ added in v0.5.0
func NewDataSetContext(provider Provider, client PDPProviderClient, evmSigner signer.EVMSigner, ref DataSetRef, opts ...ContextOption) (*DataSetContext, error)
NewDataSetContext creates an immutable context bound to ref.
func (*DataSetContext) CDNEnabled ¶ added in v0.5.0
func (c *DataSetContext) CDNEnabled() bool
CDNEnabled reports whether CDN services are enabled for this context.
func (*DataSetContext) ClientDataSetID ¶ added in v0.5.0
func (c *DataSetContext) ClientDataSetID() types.BigInt
ClientDataSetID returns the immutable client-chosen data-set ID.
func (*DataSetContext) Commit ¶ added in v0.5.0
func (c *DataSetContext) Commit(ctx context.Context, req CommitRequest) (*CommitResult, error)
Commit adds pieces to the bound data set and waits for confirmation.
func (*DataSetContext) ContextIdentity ¶ added in v0.5.0
func (c *DataSetContext) ContextIdentity() ContextIdentity
ContextIdentity returns the immutable payer, chain, and record keeper.
func (*DataSetContext) DataSetID ¶ added in v0.5.0
func (c *DataSetContext) DataSetID() types.BigInt
DataSetID returns the immutable on-chain data-set ID.
func (*DataSetContext) DataSetRef ¶ added in v0.5.0
func (c *DataSetContext) DataSetRef() (DataSetRef, bool)
DataSetRef returns the immutable target of this context.
func (*DataSetContext) DeletePiece ¶ added in v0.5.0
func (c *DataSetContext) DeletePiece(ctx context.Context, pieceCID cid.Cid) (*sdktypes.WriteResult, error)
DeletePiece schedules removal of the first piece matching pieceCID from this context's data set.
A data set can contain multiple piece IDs with the same piece CID. This method resolves pieceCID with PDPVerifier.findPieceIdsByCid using limit=1 and deletes the first returned piece ID. Prefer DataSetContext.DeletePieceByID when the on-chain piece ID is available.
The implementation:
- Resolves one on-chain pieceID via PDPVerifier.findPieceIdsByCid.
- Signs an EIP-712 SchedulePieceRemovals message over (clientDataSetID, pieceID) and ABI-encodes the signature as bytes.
- Calls the provider's DELETE /pdp/data-sets/{id}/pieces/{pieceId} endpoint via PDPProviderClient.SchedulePieceDeletion.
The returned WriteResult carries only the transaction hash; there is no on-chain wait.
func (*DataSetContext) DeletePieceByID ¶ added in v0.5.0
func (c *DataSetContext) DeletePieceByID(ctx context.Context, pieceID sdktypes.BigInt) (*sdktypes.WriteResult, error)
DeletePieceByID schedules removal of the piece identified by its on-chain piece ID from this context's data set.
Prefer this method when the piece ID is available, because piece CID is not guaranteed to be unique within a data set.
func (*DataSetContext) Download ¶ added in v0.5.0
func (c *DataSetContext) Download(ctx context.Context, pieceCID cid.Cid) (io.ReadCloser, error)
Download retrieves a piece from CDN when enabled and available, otherwise from the storage provider. It returns ErrMaxBytesExceeded, either immediately or from the reader, if the response exceeds the raw payload size encoded in pieceCID.
func (*DataSetContext) GetCommitStatus ¶ added in v0.5.0
func (c *DataSetContext) GetCommitStatus(ctx context.Context, submission CommitSubmission) (*CommitStatus, error)
GetCommitStatus checks an add-pieces submission once. A caller-supplied status URL outside the provider origin returns an error matching both ErrInvalidArgument and pdp.ErrStatusURLOrigin.
func (*DataSetContext) GetProviderInfo ¶ added in v0.5.0
func (c *DataSetContext) GetProviderInfo() Provider
GetProviderInfo returns a copy of the provider configuration.
func (*DataSetContext) GetScheduledRemovals ¶ added in v0.5.0
GetScheduledRemovals returns the list of piece ids that have been scheduled for removal from this data set but have not yet been processed.
func (*DataSetContext) PieceStatus ¶ added in v0.5.0
func (c *DataSetContext) PieceStatus(ctx context.Context, pieceCID cid.Cid) (*PieceStatus, error)
PieceStatus returns the current status of pieceCID relative to this context's data set and the proving schedule. If the piece is not present in the data set, Exists is false and the other fields are zero-valued.
The call performs up to five concurrent reads: findPieceIdsByCid, getNextChallengeEpoch, BlockNumber, getPDPConfig, and provider info. Any individual failure surfaces as a wrapped error; a nil pdpConfig is tolerated (proof-timing fields remain zero).
func (*DataSetContext) PieceURL ¶ added in v0.5.0
func (c *DataSetContext) PieceURL(pieceCID cid.Cid) string
PieceURL returns the HTTPS retrieval URL for the given piece CID on this provider.
func (*DataSetContext) PresignForCommit ¶ added in v0.5.0
func (c *DataSetContext) PresignForCommit(ctx context.Context, pieces []PieceInput) ([]byte, error)
PresignForCommit signs an add-pieces payload for the bound data set.
func (*DataSetContext) ProviderID ¶ added in v0.5.0
func (c *DataSetContext) ProviderID() types.BigInt
ProviderID returns the provider's numeric ID.
func (*DataSetContext) Pull ¶ added in v0.5.0
func (c *DataSetContext) Pull(ctx context.Context, req PullRequest) (*PullResult, error)
Pull asks this provider to fetch pieces for the bound data set.
func (*DataSetContext) ServiceURL ¶ added in v0.5.0
func (c *DataSetContext) ServiceURL() string
ServiceURL returns the base URL of the provider's PDP HTTP API.
func (*DataSetContext) Store ¶ added in v0.5.0
func (c *DataSetContext) Store(ctx context.Context, r io.Reader, opts *StoreOptions) (*StoreResult, error)
Store streams data to the provider and waits for it to be parked.
func (*DataSetContext) SubmitCommit ¶ added in v0.5.0
func (c *DataSetContext) SubmitCommit(ctx context.Context, req CommitRequest) (*CommitSubmission, error)
SubmitCommit submits one add-pieces transaction and returns without waiting for confirmation.
func (*DataSetContext) Terminate ¶ added in v0.5.0
func (c *DataSetContext) Terminate(ctx context.Context, opts ...warmstorage.WriteOption) (*types.WriteResult, error)
Terminate schedules termination of this context's data set via the FWSS terminateService entry point. On success the provider stops proving the data set and all contained pieces will be removed on-chain.
opts are forwarded to warmstorage.Service.TerminateDataSet (wait / confirmations / etc.).
func (*DataSetContext) TerminateService ¶ added in v0.5.0
func (c *DataSetContext) TerminateService(ctx context.Context, opts *TerminateServiceOptions) (*TerminateServiceResult, error)
TerminateService terminates this context's data set. By default it asks the provider to relay the termination; pass SkipProvider to use the direct FWSS path.
func (*DataSetContext) Upload ¶ added in v0.5.0
func (c *DataSetContext) Upload(ctx context.Context, r io.Reader, opts *UploadOptions) (*UploadResult, error)
Upload stores a single copy and commits it to the bound data set.
func (*DataSetContext) WaitForCommit ¶ added in v0.5.0
func (c *DataSetContext) WaitForCommit(ctx context.Context, submission CommitSubmission) (*CommitResult, error)
WaitForCommit waits for an add-pieces submission to confirm or reject. A caller-supplied status URL outside the provider origin returns an error matching both ErrInvalidArgument and pdp.ErrStatusURLOrigin.
func (*DataSetContext) WithCDN
deprecated
added in
v0.5.0
func (c *DataSetContext) WithCDN() bool
WithCDN reports whether CDN services are enabled for this context.
Deprecated: use CDNEnabled.
type DataSetDetailsCatalog ¶ added in v0.3.0
type DataSetDetailsCatalog interface {
GetClientDataSetsWithDetails(ctx context.Context, payer common.Address, onlyManaged bool) ([]*warmstorage.EnhancedDataSetInfo, error)
}
DataSetDetailsCatalog lists data sets enriched with PDP liveness, FWSS-listener ownership and metadata. Satisfied by *warmstorage.Service.
type DataSetFinder ¶
type DataSetFinder interface {
FindDataSets(ctx context.Context, payer common.Address, onlyManaged bool) ([]*DataSetInfo, error)
}
DataSetFinder lists the enriched data sets owned by `payer`. Satisfied by *warmstorage.Service via GetClientDataSetsWithDetails.
type DataSetInfo ¶
type DataSetInfo = warmstorage.EnhancedDataSetInfo
DataSetInfo aliases warmstorage.EnhancedDataSetInfo, the richer record returned by Service.FindDataSets. Kept as an alias (not a copy) so that callers that already hold *warmstorage.EnhancedDataSetInfo can pass values through without conversion.
type DataSetPDPPaymentTerminatedError ¶ added in v0.3.0
DataSetPDPPaymentTerminatedError is returned when an existing data set's PDP payment rail has an end epoch, so the data set cannot accept writes.
func (*DataSetPDPPaymentTerminatedError) Error ¶ added in v0.3.0
func (e *DataSetPDPPaymentTerminatedError) Error() string
type DataSetRef ¶ added in v0.5.0
type DataSetRef struct {
// contains filtered or unexported fields
}
DataSetRef identifies one data set and the provider that owns it. ProviderID and DataSetID must be non-zero. ClientDataSetID is the caller-chosen uint256 used by EIP-712 authorization and may be zero.
func NewDataSetRef ¶ added in v0.5.0
func NewDataSetRef(providerID, dataSetID, clientDataSetID types.BigInt) (DataSetRef, error)
NewDataSetRef validates and constructs an immutable data-set target. ClientDataSetID may be explicitly zero.
func (DataSetRef) ClientDataSetID ¶ added in v0.5.0
func (r DataSetRef) ClientDataSetID() types.BigInt
ClientDataSetID returns an independent copy of the client-chosen data-set ID.
func (DataSetRef) DataSetID ¶ added in v0.5.0
func (r DataSetRef) DataSetID() types.BigInt
DataSetID returns an independent copy of the on-chain data-set ID.
func (DataSetRef) Equal ¶ added in v0.5.0
func (r DataSetRef) Equal(other DataSetRef) bool
Equal reports whether both refs identify the same provider and data set.
func (DataSetRef) MarshalJSON ¶ added in v0.5.0
func (r DataSetRef) MarshalJSON() ([]byte, error)
MarshalJSON encodes all three identity fields using lowerCamel names. Invalid zero-value refs are rejected instead of being persisted as usable targets.
func (DataSetRef) ProviderID ¶ added in v0.5.0
func (r DataSetRef) ProviderID() types.BigInt
ProviderID returns an independent copy of the owning provider ID.
func (*DataSetRef) UnmarshalJSON ¶ added in v0.5.0
func (r *DataSetRef) UnmarshalJSON(data []byte) error
UnmarshalJSON requires the exact lowerCamel names and all three non-null identity fields.
type DataSetSizeReader ¶
type DataSetSizeReader interface {
GetDataSetSizeBytes(ctx context.Context, dataSetID sdktypes.BigInt) (*big.Int, error)
}
DataSetSizeReader returns the current on-chain size (bytes) of an existing data set, used by Service.Prepare to price lockup accurately for add-pieces scenarios. Satisfied by an adapter around PDPVerifier.getDataSetLeafCount (leafCount * 32).
type DataSetValidator ¶ added in v0.3.0
type DataSetValidator interface {
ValidateDataSet(ctx context.Context, dataSetID sdktypes.BigInt) error
}
DataSetValidator verifies that a data set is live in PDPVerifier and managed by the current FWSS listener. Satisfied by *warmstorage.Service.
type DownloadContext ¶
DownloadContext provides piece retrieval from a known storage provider.
type DownloadError ¶
type DownloadError struct {
URL string
StatusCode int // zero when the failure occurred before receiving a response
Cause error // non-nil when the failure has an underlying error (e.g. a network or transport-level error)
}
DownloadError is returned when an HTTP download request fails, either due to a network error or a non-2xx HTTP status code. Use errors.AsType to access the URL and status code.
func (*DownloadError) Error ¶
func (e *DownloadError) Error() string
func (*DownloadError) Unwrap ¶
func (e *DownloadError) Unwrap() error
type DownloadOptions ¶
type DownloadOptions struct {
Context DownloadContext // when set, delegates to DownloadContext.Download; mutually exclusive with URL
URL string // direct HTTP or HTTPS URL; validated against pieceCID on read completion
}
DownloadOptions configures a Service.Download call. Exactly one of Context or URL must be set; supplying both, or neither, returns an error matching both ErrInvalidDownloadOptions and ErrInvalidArgument.
type EpochReader ¶ added in v0.4.0
EpochReader returns the current chain epoch.
type FWSSDataSetReader ¶
type FWSSDataSetReader interface {
GetDataSet(ctx context.Context, dataSetID sdktypes.BigInt) (*warmstorage.DataSetInfo, error)
}
FWSSDataSetReader reads an existing data set's on-chain record from the FWSSView contract. Resolvers use it before construction to obtain complete immutable targets; upload paths use it to reject ended existing data sets before sending bytes to a provider. Satisfied by *warmstorage.Service (see GetDataSet).
type FWSSTerminator ¶
type FWSSTerminator interface {
TerminateDataSet(ctx context.Context, dataSetID sdktypes.BigInt, opts ...warmstorage.WriteOption) (*sdktypes.WriteResult, error)
}
FWSSTerminator terminates an on-chain data set via FWSS.TerminateService. Satisfied by *warmstorage.Service (see TerminateDataSet).
type FailedAttempt ¶
type FailedAttempt struct {
ProviderID types.BigInt
Role CopyRole
Stage CopyStage // pipeline stage where the failure occurred
Err error
Explicit bool // true when the provider was caller-specified (no auto-retry)
}
FailedAttempt records a provider attempt that did not produce a copy.
type FindDataSetsOptions ¶
type FindDataSetsOptions struct {
// Payer overrides the address configured via Options.PayerAddress.
// Zero means use the configured default payer.
Payer common.Address
// OnlyManaged restricts the returned set to data sets whose
// record-keeper is the configured FWSS contract.
OnlyManaged bool
}
FindDataSetsOptions configures Service.FindDataSets. A nil pointer or zero value selects the configured default payer and disables the "only managed" filter.
type GetStorageInfoOptions ¶
type GetStorageInfoOptions struct {
// Client overrides the default payer. Zero means use the address
// configured via Options.PayerAddress; if that too is zero the
// allowances section of the returned StorageInfo will be nil.
Client common.Address
}
GetStorageInfoOptions configures Service.GetStorageInfo. A nil pointer selects the configured default payer as the client address.
type InsufficientUploadContextsError ¶ added in v0.5.0
InsufficientUploadContextsError reports a usable partial selection.
func (*InsufficientUploadContextsError) Error ¶ added in v0.5.0
func (e *InsufficientUploadContextsError) Error() string
func (*InsufficientUploadContextsError) Is ¶ added in v0.5.0
func (e *InsufficientUploadContextsError) Is(target error) bool
Is makes InsufficientUploadContextsError match ErrInsufficientUploadContexts.
type MultiContextCosts ¶
type MultiContextCosts struct {
RatePerEpoch *big.Int
RatePerMonth *big.Int
Fees UploadFees
Lockup UploadLockup
DepositNeeded *big.Int
RequiredLockupPeriod *big.Int
NeedsFWSSMaxApproval bool
Ready bool
}
MultiContextCosts is the aggregate cost view across N upload targets. It is a flat summary rather than a per-context breakdown.
type MultiCostCalculator ¶
type MultiCostCalculator interface {
CalculateMultiContextCosts(ctx context.Context, payer common.Address, dataSizeBytes *big.Int, refs []ContextCostRef, opts MultiCostOptions) (*MultiContextCosts, error)
}
MultiCostCalculator computes an upload-cost summary for a fan-out across multiple prospective contexts.
type MultiCostOptions ¶
type MultiCostOptions struct {
// EnableCDN forces CDN pricing on every ref (in addition to any
// per-ref `WithCDN` flag) and governs whether the CDN-fixed lockup
// is added for new datasets.
EnableCDN bool
// PieceCount is the number of pieces added per context. Nil or non-positive
// values default to one.
PieceCount *big.Int
// ExtraRunwayEpochs is additional runway (epochs) on top of the
// minimum lockup period. Defaults to 0 when unset.
ExtraRunwayEpochs int64
// BufferEpochs is the deposit cushion above current lockup usage to
// cover transaction latency. Zero uses the cost service default
// (5 epochs).
BufferEpochs int64
}
MultiCostOptions customises the multi-context cost calculation.
type NewDataSetContextOptions ¶ added in v0.5.0
NewDataSetContextOptions configures an explicitly identified data-set context. ProviderID, when set, asserts the data set's provider.
type NewProviderContextOptions ¶ added in v0.5.0
NewProviderContextOptions configures an explicitly identified provider context. WithCDN nil inherits the Service default.
type Options ¶
type Options struct {
// Resolver selects provider contexts for Upload and supplies replacement
// candidates when a secondary provider fails. A nil resolver is allowed
// so the Service can still serve DownloadFromContext / download-by-URL
// calls; Upload then returns a clean validation error.
Resolver UploadResolver
// ContextResolver opens explicitly identified storage targets. When nil and
// Resolver also implements ContextResolver, New reuses Resolver.
ContextResolver ContextResolver
// ContextSelector chooses healthy providers and upload targets. When nil and
// Resolver also implements ContextSelector, New reuses Resolver.
ContextSelector ContextSelector
// HTTPClient is used for URL-based downloads. nil installs a client with
// a 24-hour timeout — long enough for multi-GiB transfers over typical
// storage networks while preventing indefinite hangs. The default client
// also disables environment-variable proxies to avoid proxy-assisted SSRF
// bypass. When set, the SDK's built-in SSRF protection is bypassed
// entirely: the provided Transport is responsible for implementing
// equivalent safeguards (private-network rejection, DNS-rebind close
// window, redirect filtering). AllowPrivateNetworks has no effect in this
// case.
HTTPClient *http.Client
// Source is the application identifier for dataset namespace isolation.
// Datasets with different Source values are treated as separate
// namespaces; reuse only occurs within the same Source.
Source string
// DefaultWithCDN is the Client-level CDN default applied when an
// UploadOptions.WithCDN or context-selection WithCDN field is nil.
// Ignored when the per-operation field is non-nil.
DefaultWithCDN bool
// MaxSecondaryAttempts caps the number of provider candidates tried for
// each secondary copy slot before giving up. Values <= 0 select the
// default of 5.
MaxSecondaryAttempts int
// CommitConcurrency caps the number of concurrent on-chain Commit RPCs
// issued across primary + secondary copies. Values <= 0 select the
// default of 4. The cap only matters when RequestedCopies exceeds it.
CommitConcurrency int
// AllowPrivateNetworks disables the default SSRF protection applied to
// URL-based Service.Download calls. When false (the default), the
// built-in HTTP client refuses to dial local, private, multicast,
// unspecified, or otherwise reserved address ranges and returns
// ErrPrivateNetwork.
// Set to true only when you knowingly need to download from a private
// network (e.g. in-cluster storage). Ignored when HTTPClient is set.
AllowPrivateNetworks bool
// DownloadMaxBytes caps the number of bytes a single URL-based
// Service.Download will return. Zero (the default) disables the cap.
// Exceeding it returns ErrMaxBytesExceeded either eagerly (via
// Content-Length) or at the terminal Read of the returned reader.
DownloadMaxBytes int64
// Logger receives internal warnings. nil disables logging.
Logger *slog.Logger
// Lifecycle is checked before service operations that can touch configured
// backends. Any error returned by CheckClosed is returned without touching
// those backends. The root synapse Client injects a shared checker whose
// closed error matches ErrClosed. Nil is allowed for standalone use. A
// typed-nil value is treated as nil.
Lifecycle interface{ CheckClosed() error }
// DataSetFinder backs Service.FindDataSets. Optional; when nil the
// method returns an ErrUninitialized-wrapped error.
DataSetFinder DataSetFinder
// StorageInfoReader backs Service.GetStorageInfo. Optional.
StorageInfoReader StorageInfoReader
// DataSetTerminator backs Service.TerminateDataSet. Optional.
DataSetTerminator FWSSTerminator
// CostCalculator backs Service.CalculateMultiContextCosts and the
// cost estimation inside Prepare. Optional.
CostCalculator MultiCostCalculator
// PaymentsFunder backs PrepareTransaction.Execute. Optional.
PaymentsFunder PaymentsFunder
// DataSetSizeReader backs the per-dataset size lookup performed by
// Service.Prepare for existing-dataset contexts. Optional; when
// nil, Prepare falls back to zero-size estimates. For
// accurate add-pieces pricing, wire an implementation backed by
// PDPVerifier.getDataSetLeafCount (leafCount * 32 bytes).
DataSetSizeReader DataSetSizeReader
// FWSSDataSetReader is used by explicit data-set resolution to read the
// on-chain ClientDataSetID
// and to equip returned contexts with upload-time ended-dataset
// checks. When nil, those safety nets are skipped.
FWSSDataSetReader FWSSDataSetReader
// ProviderResolver resolves provider endpoints for manager-level
// provider-relayed termination. Optional.
ProviderResolver ProviderResolver
// PaymentStateReader and EpochReader back the provider-relayed termination
// debt pre-check. Optional.
PaymentStateReader PaymentStateReader
EpochReader EpochReader
PaymentToken common.Address
// Signer signs EIP-712 authorizations for manager-level provider-relayed
// termination. It may differ from PayerAddress when the payer has authorized
// a delegated signer. A typed-nil value is treated as nil.
Signer signer.EVMSigner
ChainID types.ChainID
RecordKeeper common.Address
// PayerAddress is the payer/client identity used by manager-level helpers
// and validated against every StorageContext. It may differ from Signer.
// When zero, New derives it from a non-nil Signer.
PayerAddress common.Address
}
Options configures a Service. Unset fields fall back to sensible defaults.
type PDPConfigReader ¶
type PDPConfigReader interface {
GetPDPConfig(ctx context.Context) (*warmstorage.PDPConfig, error)
}
PDPConfigReader returns the proving-period configuration from the FWSSView contract. Satisfied by *warmstorage.Service.
type PDPProviderClient ¶ added in v0.2.0
type PDPProviderClient interface {
UploadPieceStreaming(context.Context, io.Reader, pdp.UploadPieceStreamingOptions) (*pdp.UploadStreamingResult, error)
DownloadPiece(context.Context, cid.Cid) (io.ReadCloser, int64, error)
WaitForPieceParked(context.Context, cid.Cid, time.Duration) error
WaitForPullComplete(context.Context, pdp.PullRequest, time.Duration, func(*pdp.PullResult)) (*pdp.PullResult, error)
AddPieces(context.Context, types.BigInt, []pdp.AddPieceInput, []byte) (*pdp.AddPiecesResult, error)
GetAddPiecesStatus(context.Context, string) (*pdp.AddPiecesStatus, error)
WaitForPiecesAdded(context.Context, string, time.Duration) (*pdp.AddPiecesStatus, error)
CreateDataSet(context.Context, common.Address, []byte) (*pdp.CreateDataSetResult, error)
GetDataSetCreationStatus(context.Context, string) (*pdp.CreateDataSetStatus, error)
WaitForDataSetCreated(context.Context, string, time.Duration) (*pdp.CreateDataSetStatus, error)
CreateDataSetAndAddPieces(context.Context, common.Address, []pdp.AddPieceInput, []byte) (*pdp.CreateDataSetResult, error)
GetCreateDataSetAndAddPiecesStatus(context.Context, string) (*pdp.CreateAndAddPiecesStatus, error)
WaitForCreateDataSetAndAddPieces(context.Context, string, time.Duration) (*pdp.AddPiecesStatus, error)
SchedulePieceDeletion(ctx context.Context, dataSetID, pieceID types.BigInt, extraData []byte) (common.Hash, error)
}
PDPProviderClient is the provider HTTP API surface required by storage contexts. It is injectable for tests and alternate provider clients.
type PDPProviderSource ¶
type PDPProviderSource interface {
GetPDPProvider(context.Context, types.BigInt) (*spregistry.PDPProvider, error)
SelectActivePDPProviders(context.Context, spregistry.ProviderFilter) ([]spregistry.PDPProvider, error)
}
PDPProviderSource is the subset of spregistry.Service used by ServiceResolver.
type PDPVerifierReader ¶
type PDPVerifierReader interface {
FindPieceIdsByCid(ctx context.Context, dataSetID sdktypes.BigInt, pieceCID cid.Cid, start, limit uint64) ([]sdktypes.BigInt, error)
GetScheduledRemovals(ctx context.Context, dataSetID sdktypes.BigInt) ([]sdktypes.BigInt, error)
GetNextChallengeEpoch(ctx context.Context, dataSetID sdktypes.BigInt) (*big.Int, error)
BlockNumber(ctx context.Context) (uint64, error)
}
PDPVerifierReader is the read-only PDPVerifier surface required by DataSetContext for piece lifecycle queries (scheduled removals, id lookup, next challenge epoch) and proving-window calculations. Implementations convert between sdktypes.BigInt / cid.Cid and the abigen-native types (`*big.Int`, `pdpverifier.CidsCid`).
type PaymentStateReader ¶ added in v0.4.0
type PaymentStateReader interface {
AccountInfo(ctx context.Context, token, owner common.Address) (*payments.AccountState, error)
}
PaymentStateReader reads payment account state for termination pre-checks.
type PaymentsFunder ¶
type PaymentsFunder interface {
FundSync(ctx context.Context, amount *big.Int, opts ...payments.WriteOption) (*sdktypes.WriteResult, error)
}
PaymentsFunder tops up the Payments contract for an upload. Narrow view of payments.Service used by PrepareTransaction.Execute.
type PieceInput ¶
type PieceInput struct {
PieceCID cid.Cid
PieceMetadata map[string]string // optional key-value metadata stored with the piece
}
PieceInput describes a single piece being committed on-chain.
type PieceStatus ¶
type PieceStatus struct {
// Exists reports whether the piece is currently present in the data set.
Exists bool
// PieceID is the piece's on-chain numeric id. Only meaningful when
// Exists is true.
PieceID types.BigInt
// DataSetLastProven is the wall-clock time of the most recent
// proof submission for the enclosing data set. Zero time indicates
// "never proven" or "unknown".
DataSetLastProven time.Time
// DataSetNextProofDue is the wall-clock deadline by which the next
// proof must be submitted. Zero time indicates "unknown".
DataSetNextProofDue time.Time
// RetrievalURL is the HTTPS URL at which the piece can be retrieved
// from the provider. Empty when provider info is not available.
RetrievalURL string
// InChallengeWindow reports whether the data set is currently inside
// its challenge window (proof submission is required).
InChallengeWindow bool
// HoursUntilChallengeWindow is the number of hours between now and
// the start of the next challenge window. Zero when inside or past
// the window.
HoursUntilChallengeWindow float64
// IsProofOverdue reports whether the proving deadline has passed
// without a proof being submitted.
IsProofOverdue bool
}
PieceStatus captures the current state of a piece relative to its data set's proving schedule.
PieceID is zero when the piece is not present in the data set (Exists = false); callers should treat (Exists == false) as the "null" TS return.
type PrepareOptions ¶
type PrepareOptions struct {
// DataSize is required when Costs is nil, and must be zero when Costs
// is set. It is the payload size in bytes used for cost calculation.
DataSize uint64
// Contexts is the exact set of upload targets used for cost calculation.
// It is required when Costs is nil.
Contexts []StorageContext
// PieceCount is the number of pieces added to each context. Nil or
// non-positive values default to one.
PieceCount *big.Int
// Costs short-circuits cost calculation. When set, no other
// PrepareOptions fields are accepted.
Costs *MultiContextCosts
// ExtraRunwayEpochs is additional runway (epochs) above the
// minimum lockup period passed through to the cost calculator. It is
// valid only when Costs is nil and must be non-negative.
ExtraRunwayEpochs int64
// BufferEpochs is the deposit cushion above current lockup usage
// used to absorb transaction-latency epochs. Zero falls back to
// the cost service default. It is valid only when Costs is nil and
// must be non-negative.
BufferEpochs int64
}
PrepareOptions configures Service.Prepare. Costs and context-based cost calculation are mutually exclusive modes.
type PrepareResult ¶
type PrepareResult struct {
// Costs is the aggregated cost calculation that drove the decision.
Costs *MultiContextCosts
// Transaction is non-nil only when funding is required (Ready=false).
Transaction *PrepareTransaction
}
PrepareResult is the value returned by Service.Prepare.
type PrepareTransaction ¶
type PrepareTransaction struct {
// DepositAmount is the USDFC amount that will be moved into the
// payments account.
DepositAmount *big.Int
// IncludesApproval reports whether the call will also set the FWSS
// operator to max allowance.
IncludesApproval bool
// Execute performs the funding operation. When approval is required,
// Prepare fixes the approval decision and max lockup period from Costs;
// caller-provided payments.WriteOption values should be limited to write
// controls such as wait, confirmations, or precheck behavior.
Execute func(ctx context.Context, opts ...payments.WriteOption) (*types.WriteResult, error)
}
PrepareTransaction is the deferred funding step returned by Prepare when the account is not yet Ready. Execute performs the top-up.
type PricePerTiB ¶
PricePerTiB holds the effective price per TiB for one provisioning profile at three time granularities.
type PricingInfo ¶
type PricingInfo struct {
NoCDN PricePerTiB
WithCDN PricePerTiB
TokenAddress common.Address
TokenSymbol string
PriceList *warmstorage.PriceList
}
PricingInfo is the pricing sub-structure of StorageInfo.
type Provider ¶
type Provider struct {
ID types.BigInt // provider ID from SPRegistry
ServiceURL string // base URL of the provider's PDP HTTP API
ServiceProvider common.Address // provider's EVM address
Payee common.Address // address that receives payments
}
Provider holds the on-chain identity of a storage provider.
type ProviderContext ¶ added in v0.5.0
type ProviderContext struct {
// contains filtered or unexported fields
}
ProviderContext represents one provider without a bound data set. Commit and Pull operations on a ProviderContext create a new data set. It is safe for concurrent use; concurrent create operations are independent.
func NewProviderContext ¶ added in v0.5.0
func NewProviderContext(provider Provider, client PDPProviderClient, evmSigner signer.EVMSigner, opts ...ContextOption) (*ProviderContext, error)
NewProviderContext creates an immutable context for the given provider and PDP client. Commit and Pull create a new data set. provider.ID, provider.ServiceURL, and client are validated here. Signing prerequisites (such as a non-nil signer plus chain/payer/record-keeper options) are validated by the write paths that need them, e.g. PresignForCommit.
func (*ProviderContext) CDNEnabled ¶ added in v0.5.0
func (c *ProviderContext) CDNEnabled() bool
CDNEnabled reports whether CDN services are enabled for this context.
func (*ProviderContext) Commit ¶ added in v0.5.0
func (c *ProviderContext) Commit(ctx context.Context, req CommitRequest) (*CommitResult, error)
Commit creates a data set, adds pieces, and waits for confirmation.
func (*ProviderContext) ContextIdentity ¶ added in v0.5.0
func (c *ProviderContext) ContextIdentity() ContextIdentity
ContextIdentity returns the immutable payer, chain, and record keeper.
func (*ProviderContext) CreateDataSet ¶ added in v0.5.0
func (c *ProviderContext) CreateDataSet(ctx context.Context, opts *CreateDataSetOptions) (*CreateDataSetResult, error)
CreateDataSet creates an empty data set for this provider. The receiver remains unbound; use ProviderContext.ForDataSet with the returned ref to obtain a DataSetContext.
func (*ProviderContext) DataSetRef ¶ added in v0.5.0
func (c *ProviderContext) DataSetRef() (DataSetRef, bool)
DataSetRef reports that a ProviderContext is not bound to a data set.
func (*ProviderContext) Download ¶ added in v0.5.0
func (c *ProviderContext) Download(ctx context.Context, pieceCID cid.Cid) (io.ReadCloser, error)
Download retrieves a piece from CDN when enabled and available, otherwise from the storage provider. It returns ErrMaxBytesExceeded, either immediately or from the reader, if the response exceeds the raw payload size encoded in pieceCID.
func (*ProviderContext) ForDataSet ¶ added in v0.5.0
func (c *ProviderContext) ForDataSet(ref DataSetRef) (*DataSetContext, error)
ForDataSet returns a new immutable context bound to ref. The receiver is not modified and may continue to create independent data sets.
func (*ProviderContext) GetCommitStatus ¶ added in v0.5.0
func (c *ProviderContext) GetCommitStatus(ctx context.Context, submission CommitSubmission) (*CommitStatus, error)
GetCommitStatus checks a create-and-add submission once. A caller-supplied status URL outside the provider origin returns an error matching both ErrInvalidArgument and pdp.ErrStatusURLOrigin.
func (*ProviderContext) GetProviderInfo ¶ added in v0.5.0
func (c *ProviderContext) GetProviderInfo() Provider
GetProviderInfo returns a copy of the provider configuration.
func (*ProviderContext) PieceURL ¶ added in v0.5.0
func (c *ProviderContext) PieceURL(pieceCID cid.Cid) string
PieceURL returns the HTTPS retrieval URL for the given piece CID on this provider.
func (*ProviderContext) PresignForCommit ¶ added in v0.5.0
func (c *ProviderContext) PresignForCommit(ctx context.Context, pieces []PieceInput) ([]byte, error)
PresignForCommit signs a create-and-add payload for a new data set.
func (*ProviderContext) ProviderID ¶ added in v0.5.0
func (c *ProviderContext) ProviderID() types.BigInt
ProviderID returns the provider's numeric ID.
func (*ProviderContext) Pull ¶ added in v0.5.0
func (c *ProviderContext) Pull(ctx context.Context, req PullRequest) (*PullResult, error)
Pull asks this provider to fetch pieces for a new data set.
func (*ProviderContext) ServiceURL ¶ added in v0.5.0
func (c *ProviderContext) ServiceURL() string
ServiceURL returns the base URL of the provider's PDP HTTP API.
func (*ProviderContext) Store ¶ added in v0.5.0
func (c *ProviderContext) Store(ctx context.Context, r io.Reader, opts *StoreOptions) (*StoreResult, error)
Store streams data to the provider and waits for it to be parked.
func (*ProviderContext) SubmitCommit ¶ added in v0.5.0
func (c *ProviderContext) SubmitCommit(ctx context.Context, req CommitRequest) (*CommitSubmission, error)
SubmitCommit submits one create-and-add transaction and returns without waiting for confirmation.
func (*ProviderContext) Upload ¶ added in v0.5.0
func (c *ProviderContext) Upload(ctx context.Context, r io.Reader, opts *UploadOptions) (*UploadResult, error)
Upload stores a single copy and commits it to a new data set.
func (*ProviderContext) WaitForCommit ¶ added in v0.5.0
func (c *ProviderContext) WaitForCommit(ctx context.Context, submission CommitSubmission) (*CommitResult, error)
WaitForCommit waits for a create-and-add submission to confirm or reject. A caller-supplied status URL outside the provider origin returns an error matching both ErrInvalidArgument and pdp.ErrStatusURLOrigin.
func (*ProviderContext) WaitForDataSetCreated ¶ added in v0.5.0
func (c *ProviderContext) WaitForDataSetCreated(ctx context.Context, submission CreateDataSetSubmission) (*CreateDataSetResult, error)
WaitForDataSetCreated waits for a previously submitted create-dataset transaction. The receiver remains unbound.
func (*ProviderContext) WithCDN
deprecated
added in
v0.5.0
func (c *ProviderContext) WithCDN() bool
WithCDN reports whether CDN services are enabled for this context.
Deprecated: use CDNEnabled.
type ProviderResolver ¶ added in v0.4.0
type ProviderResolver interface {
ResolveProvider(ctx context.Context, providerID sdktypes.BigInt) (Provider, error)
}
ProviderResolver resolves a storage provider by provider ID.
type PullPieceResult ¶
type PullPieceResult struct {
PieceCID cid.Cid
Status PullStatus
}
PullPieceResult is the per-piece outcome within a PullResult.
type PullRequest ¶
type PullRequest struct {
Pieces []cid.Cid
From func(cid.Cid) string // returns the HTTPS URL for a given piece CID
ExtraData []byte // EIP-712 signed payload authorising the pull
// OnProgress is invoked after each piece status update during the pull.
// It may be nil. Direct Pull calls do not recover callback panics.
OnProgress func(pieceCID cid.Cid, status PullStatus)
}
PullRequest asks a secondary provider to pull pieces from a primary.
type PullResult ¶
type PullResult struct {
Status PullStatus
Pieces []PullPieceResult
}
PullResult is the aggregate outcome of a pull operation.
type PullStatus ¶
type PullStatus string
PullStatus is the per-piece or overall status returned by a pull operation.
const ( PullStatusPending PullStatus = "pending" PullStatusInProgress PullStatus = "inProgress" PullStatusRetrying PullStatus = "retrying" PullStatusComplete PullStatus = "complete" PullStatusFailed PullStatus = "failed" )
type SelectProviderContextOptions ¶ added in v0.5.0
type SelectProviderContextOptions struct {
ExcludeProviderIDs []types.BigInt
DataSetMetadata map[string]string
WithCDN *bool
}
SelectProviderContextOptions configures selection of one healthy provider.
type SelectUploadContextsOptions ¶ added in v0.5.0
type SelectUploadContextsOptions struct {
Copies int
ExcludeProviderIDs []types.BigInt
DataSetMetadata map[string]string
WithCDN *bool
}
SelectUploadContextsOptions configures selection of upload targets.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service orchestrates multi-copy uploads and downloads. Create with New; configure via the Options struct.
func (*Service) CalculateMultiContextCosts ¶
func (s *Service) CalculateMultiContextCosts(ctx context.Context, dataSizeBytes uint64, refs []ContextCostRef, opts MultiCostOptions, payer common.Address) (*MultiContextCosts, error)
CalculateMultiContextCosts fans out the cost calculation across the given refs and returns an aggregate result.
func (*Service) Download ¶
func (s *Service) Download(ctx context.Context, pieceCID cid.Cid, opts *DownloadOptions) (io.ReadCloser, error)
Download retrieves a piece by URL or via a DownloadContext. When URL is used, the response body is streamed through a validating reader; the terminal read error from io.ReadAll (or any last Read call that returns io.EOF) carries the integrity check result — callers must not discard it. URL downloads are capped by non-zero Options.DownloadMaxBytes. Exceeding the cap returns ErrMaxBytesExceeded either before streaming starts, when Content-Length is too large, or as the terminal Read error.
func (*Service) FindDataSets ¶
func (s *Service) FindDataSets(ctx context.Context, opts *FindDataSetsOptions) ([]*DataSetInfo, error)
FindDataSets returns the enriched list of data sets owned by the caller or by the payer in opts.
func (*Service) GetStorageInfo ¶
func (s *Service) GetStorageInfo(ctx context.Context, opts *GetStorageInfoOptions) (*StorageInfo, error)
GetStorageInfo returns the comprehensive chain-wide storage view needed to author an upload.
func (*Service) NewDataSetContext ¶ added in v0.5.0
func (s *Service) NewDataSetContext(ctx context.Context, dataSetID types.BigInt, opts NewDataSetContextOptions) (*DataSetContext, error)
NewDataSetContext opens an existing data set owned by the Service payer. The data set may be terminated or otherwise not currently writable.
func (*Service) NewProviderContext ¶ added in v0.5.0
func (s *Service) NewProviderContext(ctx context.Context, providerID types.BigInt, opts NewProviderContextOptions) (*ProviderContext, error)
NewProviderContext opens a registered provider without selecting it or checking its current approval, activity, or endpoint health.
func (*Service) Prepare ¶
func (s *Service) Prepare(ctx context.Context, opts *PrepareOptions) (*PrepareResult, error)
Prepare returns the funding transaction needed, if any, to cover one upload of DataSize bytes across the supplied contexts.
func (*Service) SelectProviderContext ¶ added in v0.5.0
func (s *Service) SelectProviderContext(ctx context.Context, opts SelectProviderContextOptions) (*ProviderContext, error)
SelectProviderContext selects one approved, active, healthy provider without querying or reusing an existing data set.
func (*Service) SelectUploadContexts ¶ added in v0.5.0
func (s *Service) SelectUploadContexts(ctx context.Context, opts SelectUploadContextsOptions) (*UploadContextSelection, error)
SelectUploadContexts selects targets for Prepare and UploadToContexts. A partial result is returned with InsufficientUploadContextsError.
func (*Service) TerminateDataSet ¶
func (s *Service) TerminateDataSet(ctx context.Context, dataSetID types.BigInt, opts *TerminateDataSetOptions) (*types.WriteResult, error)
TerminateDataSet terminates an FWSS-managed data set by ID.
func (*Service) TerminateService ¶ added in v0.4.0
func (s *Service) TerminateService(ctx context.Context, dataSetID types.BigInt, opts *TerminateServiceOptions) (*TerminateServiceResult, error)
TerminateService terminates an FWSS-managed data set by ID. By default it resolves the provider and asks it to relay the termination; pass SkipProvider to use the direct FWSS transaction path.
func (*Service) Upload ¶
func (s *Service) Upload(ctx context.Context, r io.Reader, opts *UploadOptions) (*UploadResult, error)
Upload automatically selects targets and runs the multi-copy upload pipeline streaming from r in a single pass. opts must be non-nil. Returns UploadResult whose Complete field indicates whether all requested copies were committed on-chain.
The reader is consumed once by the primary provider; secondary copies are populated via server-to-server Pulls. On success the reader is fully drained; on error it may be only partially consumed.
Timeouts and cancellation: Upload honours ctx for every step — presign, store, pull and on-chain commit wait. To bound the total upload time (including the blockchain confirmation that populates the returned UploadResult.Copies), wrap ctx with context.WithTimeout; the Service itself does not impose an internal wait deadline. The built-in 24h HTTP timeout on Service only affects URL-based downloads; Upload, Pull, and Commit use the StorageContext implementation's own HTTP client configuration.
func (*Service) UploadToContexts ¶ added in v0.5.0
func (s *Service) UploadToContexts(ctx context.Context, r io.Reader, contexts []StorageContext, opts *UploadOptions) (*UploadResult, error)
UploadToContexts uploads to the exact contexts supplied by the caller. The first context is primary; later contexts receive provider-to-provider pulls.
type ServiceParameters ¶
type ServiceParameters struct {
EpochsPerMonth int64
EpochsPerDay int64
EpochDuration int64 // seconds per epoch
MinUploadSize int64
MaxUploadSize int64
}
ServiceParameters captures the immutable chain/service geometry that StorageInfo callers want to show alongside pricing.
type ServiceResolver ¶
type ServiceResolver struct {
// contains filtered or unexported fields
}
ServiceResolver opens explicit targets and selects providers and data sets for uploads. Automatic selection prefers writable matching data sets with active pieces, then writable empty matches, in stable provider order.
func NewServiceResolver ¶
func NewServiceResolver(opts ServiceResolverOptions) (*ServiceResolver, error)
NewServiceResolver constructs a ServiceResolver. Payer, SPRegistry, WarmStorage, and NewContext are required. DataSetValidator, DataSetDetails, and active-piece reads are optional and auto-detected from WarmStorage when available and configured. ProviderPing is optional and defaults to a PDP ping with up to two retries for transient failures.
func (*ServiceResolver) ResolveDataSetContext ¶ added in v0.5.0
func (r *ServiceResolver) ResolveDataSetContext(ctx context.Context, dataSetID types.BigInt, opts NewDataSetContextOptions) (*DataSetContext, error)
ResolveDataSetContext opens an existing payer-owned data set without requiring it to be writable.
func (*ServiceResolver) ResolveProvider ¶ added in v0.4.0
func (r *ServiceResolver) ResolveProvider(ctx context.Context, providerID types.BigInt) (Provider, error)
ResolveProvider resolves one PDP provider by ID.
func (*ServiceResolver) ResolveProviderContext ¶ added in v0.5.0
func (r *ServiceResolver) ResolveProviderContext(ctx context.Context, providerID types.BigInt, opts NewProviderContextOptions) (*ProviderContext, error)
ResolveProviderContext opens a registered provider without selection checks.
func (*ServiceResolver) ResolveUploadContexts ¶
func (r *ServiceResolver) ResolveUploadContexts(ctx context.Context, opts *UploadOptions) ([]StorageContext, bool, error)
ResolveUploadContexts adapts automatic upload options to context selection.
func (*ServiceResolver) SelectProviderContext ¶ added in v0.5.0
func (r *ServiceResolver) SelectProviderContext(ctx context.Context, opts SelectProviderContextOptions) (*ProviderContext, error)
SelectProviderContext selects one approved, active, healthy provider and returns it without a data-set binding.
func (*ServiceResolver) SelectReplacement ¶
func (r *ServiceResolver) SelectReplacement(ctx context.Context, usedProviders map[string]types.BigInt, opts *UploadOptions) (StorageContext, error)
SelectReplacement selects one writable provider not already used.
func (*ServiceResolver) SelectUploadContexts ¶ added in v0.5.0
func (r *ServiceResolver) SelectUploadContexts(ctx context.Context, opts SelectUploadContextsOptions) (*UploadContextSelection, error)
SelectUploadContexts selects writable targets in stable provider order.
type ServiceResolverOptions ¶
type ServiceResolverOptions struct {
Payer common.Address // EVM address of the paying account
SPRegistry PDPProviderSource
WarmStorage DataSetCatalog
DataSetValidator DataSetValidator
DataSetDetails DataSetDetailsCatalog
// ProviderPing checks one automatically selected provider endpoint. The
// resolver may call it concurrently for up to 16 candidates per selection;
// concurrent selections have independent limits. Each call receives a
// context whose deadline is no later than eight seconds; a shorter parent
// deadline takes precedence. Implementations must be safe for concurrent use
// and honor the context. nil uses [pdp.Client.Ping] with up to two retries for
// transient failures.
ProviderPing func(context.Context, string) error
NewContext ContextFactory // called per-provider to construct an upload context
}
ServiceResolverOptions configures a ServiceResolver.
type StorageContext ¶ added in v0.5.0
type StorageContext interface {
ContextIdentity() ContextIdentity
ProviderID() types.BigInt
GetProviderInfo() Provider
DataSetRef() (DataSetRef, bool)
ServiceURL() string
CDNEnabled() bool
PieceURL(cid.Cid) string
Store(context.Context, io.Reader, *StoreOptions) (*StoreResult, error)
Download(context.Context, cid.Cid) (io.ReadCloser, error)
PresignForCommit(context.Context, []PieceInput) ([]byte, error)
Pull(context.Context, PullRequest) (*PullResult, error)
Commit(context.Context, CommitRequest) (*CommitResult, error)
Upload(context.Context, io.Reader, *UploadOptions) (*UploadResult, error)
}
StorageContext is an immutable provider or data-set upload target.
type StorageInfo ¶
type StorageInfo struct {
Pricing PricingInfo
Providers []spregistry.PDPProvider
ServiceParameters ServiceParameters
Allowances *Allowances
}
StorageInfo aggregates the chain-wide storage view a client needs before authoring an upload.
type StorageInfoReader ¶
type StorageInfoReader interface {
GetStorageInfo(ctx context.Context, client common.Address) (*StorageInfo, error)
}
StorageInfoReader returns the chain-wide StorageInfo view for the given client.
type StoreError ¶
StoreError is returned when the primary store operation fails.
func (*StoreError) Error ¶
func (e *StoreError) Error() string
func (*StoreError) Unwrap ¶
func (e *StoreError) Unwrap() error
type StoreOptions ¶
type StoreOptions struct {
// PieceCID, when defined, is a pre-computed PieceCIDv2 of the payload.
// When set, the client skips inline commP calculation; the server still
// verifies the uploaded bytes match this value.
PieceCID cid.Cid
// OnProgress is invoked after each non-empty Read from the reader, with
// the cumulative bytes sent so far. It may be nil. Direct Store calls do
// not recover callback panics.
OnProgress func(bytesUploaded int64)
}
StoreOptions configures a single provider or data-set context Store call.
type StoreResult ¶
type StoreResult struct {
PieceCID cid.Cid // PieceCIDv2 of the stored data
Size int64 // raw (unpadded) byte count
}
StoreResult is returned by a successful Store call.
type SubmittedPiece ¶
SubmittedPiece carries the piece identity reported by an OnPiecesAdded callback.
type TerminateDataSetOptions ¶
type TerminateDataSetOptions struct {
// WriteOptions are forwarded to warmstorage.TerminateDataSet.
WriteOptions []warmstorage.WriteOption
}
TerminateDataSetOptions configures Service.TerminateDataSet.
type TerminateServiceDebtError ¶ added in v0.4.0
TerminateServiceDebtError is returned before provider-relayed termination when the payer account already has debt.
func (*TerminateServiceDebtError) Error ¶ added in v0.4.0
func (e *TerminateServiceDebtError) Error() string
type TerminateServiceOptions ¶ added in v0.4.0
type TerminateServiceOptions struct {
// SkipProvider uses the direct FWSS transaction path. The new
// TerminateService helper still waits for the receipt so it can return
// EndEpoch. Use DataSetContext.Terminate or Service.TerminateDataSet for the
// legacy hash-only direct write surface.
SkipProvider bool
// WriteOptions are used only when SkipProvider is true.
WriteOptions []warmstorage.WriteOption
// DirectWaitTimeout is used only when SkipProvider is true. Zero selects
// the default wait timeout.
DirectWaitTimeout time.Duration
// ProviderWaitTimeout bounds provider relay submission plus status polling.
// Zero selects the default wait timeout. Negative disables the added timeout
// and leaves cancellation to ctx.
ProviderWaitTimeout time.Duration
// PollInterval is used only for provider-relayed status polling.
PollInterval time.Duration
// OnSubmitted is called when a transaction hash becomes known.
OnSubmitted func(common.Hash)
}
TerminateServiceOptions configures provider-relayed service termination.
type TerminateServiceResult ¶ added in v0.4.0
type TerminateServiceResult struct {
TxHash *common.Hash
DataSetID types.BigInt
EndEpoch types.Epoch
}
TerminateServiceResult is the high-level service termination result.
type UploadContextSelection ¶ added in v0.5.0
type UploadContextSelection struct {
Contexts []StorageContext
RequestedCopies int
Complete bool
}
UploadContextSelection contains the targets selected for one upload.
type UploadFees ¶ added in v0.4.0
UploadFees is the one-time fee breakdown for a prepared upload.
type UploadLockup ¶ added in v0.4.0
type UploadLockup struct {
RateDeltaPerEpoch *big.Int
StreamingLockup *big.Int
LifecycleLockup *big.Int
CDNLockup *big.Int
CacheMissLockup *big.Int
Total *big.Int
}
UploadLockup is the lockup breakdown for a prepared upload.
type UploadOptions ¶
type UploadOptions struct {
// Copies is the number of provider copies to store. Service.Upload requires
// a positive value. Explicit-context upload methods do not accept it.
Copies int
// PieceMetadata is stored with each piece on-chain.
PieceMetadata map[string]string
// DataSetMetadata is stored with the data set on first creation.
DataSetMetadata map[string]string
// ExcludeProviderIDs skips these providers only during auto-selection.
ExcludeProviderIDs []types.BigInt
// WithCDN is tri-state: nil inherits the Client-level default
// configured via synapse.WithCDN; non-nil explicitly overrides
// for this upload. Declare a local variable to take its address:
//
// b := true
// opts := &storage.UploadOptions{WithCDN: &b}
WithCDN *bool
// PieceCID, when defined, is a pre-computed PieceCIDv2 of the payload.
// When set, the primary provider client skips inline commP calculation;
// the server still verifies the uploaded bytes match this value.
PieceCID cid.Cid
// OnProgress is invoked after each non-empty Read from the upload
// reader, with the cumulative bytes sent to the primary provider so
// far. It may be nil.
OnProgress func(bytesUploaded int64)
// OnStored is invoked once the primary provider has confirmed storage of
// the piece. It may be nil.
OnStored func(providerID types.BigInt, pieceCID cid.Cid)
// OnPiecesAdded is invoked after the on-chain AddPieces transaction is
// submitted for a provider (primary or secondary), carrying the transaction
// hash and the batch of pieces included in that transaction. During
// Service.Upload, different providers may invoke this callback
// concurrently when commitConcurrency > 1. It may be nil.
OnPiecesAdded func(txHash string, providerID types.BigInt, pieces []SubmittedPiece)
// OnPiecesConfirmed is invoked after the on-chain AddPieces transaction is
// confirmed (CommitResult received) for a provider, carrying the assigned
// on-chain IDs for each piece. During Service.Upload, this callback is
// invoked sequentially after all commit workers finish. It may be nil.
OnPiecesConfirmed func(dataSetID, providerID types.BigInt, pieces []ConfirmedPiece)
// OnCopyComplete is invoked once a secondary provider's SP-to-SP pull
// completes successfully. It is not fired for the primary (which stores
// directly). It may be nil.
OnCopyComplete func(providerID types.BigInt, pieceCID cid.Cid)
// OnCopyFailed is invoked when a secondary provider's SP-to-SP copy
// attempt fails. Presign failures are not copy attempts and still surface
// only through FailedAttempts with CopyStagePresign. Primary store/commit
// failures likewise surface through the Upload return value and
// FailedAttempts. It may be nil.
OnCopyFailed func(providerID types.BigInt, pieceCID cid.Cid, err error)
// OnPullProgress is invoked for each piece status update during a
// secondary-provider pull. It may be nil.
OnPullProgress func(providerID types.BigInt, pieceCID cid.Cid, status PullStatus)
}
UploadOptions configures upload operations. Service.Upload requires Copies and accepts target-selection fields. Service.UploadToContexts rejects target selection fields. ProviderContext.Upload and DataSetContext.Upload also reject callbacks that only apply to secondary copies.
Some lifecycle callbacks may be invoked from internal orchestration goroutines. Callers that share mutable state across callbacks must keep their handlers concurrency-safe. Service.Upload, ProviderContext.Upload, and DataSetContext.Upload recover and ignore callback panics; when a logger is configured, the first panic per callback name in an upload logs a warning. This recovery does not apply to direct StoreOptions, PullRequest, or CommitRequest hooks.
type UploadResolver ¶
type UploadResolver interface {
ResolveUploadContexts(context.Context, *UploadOptions) ([]StorageContext, bool, error)
SelectReplacement(context.Context, map[string]types.BigInt, *UploadOptions) (StorageContext, error)
}
UploadResolver selects provider contexts for upload operations and provides replacement candidates when a secondary provider fails.
type UploadResult ¶
type UploadResult struct {
PieceCID cid.Cid // PieceCIDv2 of the stored data
Size int64 // raw (unpadded) byte count
RequestedCopies int
// Complete is true when all RequestedCopies were committed on-chain.
// Equivalent to len(Copies) >= RequestedCopies.
Complete bool
Copies []CopyResult
FailedAttempts []FailedAttempt
}
UploadResult is returned by a successful Upload call.
Use Complete to determine overall success: it is true when every requested copy was committed on-chain (equivalent to len(Copies) >= RequestedCopies). A non-empty FailedAttempts slice does NOT indicate overall failure — failed attempts may have been resolved by successful retries on other providers.
Example:
result, err := m.Upload(ctx, r, opts)
if err != nil { ... }
if !result.Complete {
log.Printf("partial upload: %d/%d copies", result.SuccessCount(), result.RequestedCopies)
}
func (*UploadResult) PartialSuccess ¶
func (r *UploadResult) PartialSuccess() bool
PartialSuccess reports whether at least one copy was committed on-chain but fewer than the requested number were obtained. Returns false when Complete is true or when no copies succeeded at all.
func (*UploadResult) PrimaryDataSetID ¶
func (r *UploadResult) PrimaryDataSetID() (types.BigInt, bool)
PrimaryDataSetID returns the DataSetID of the primary copy.
ok is false when no primary copy committed on-chain (even if secondaries did). Callers that need precise provenance should inspect UploadResult.Copies directly.
func (*UploadResult) SuccessCount ¶
func (r *UploadResult) SuccessCount() int
SuccessCount returns the number of copies that were successfully committed on-chain. Equivalent to len(Copies).
func (*UploadResult) SuccessfulProviderIDs ¶
func (r *UploadResult) SuccessfulProviderIDs() []types.BigInt
SuccessfulProviderIDs returns the ProviderID of every copy that committed on-chain, in the order the copies appear in UploadResult.Copies.