provider

package
v0.29.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 13, 2025 License: Apache-2.0, MIT Imports: 21 Imported by: 20

Documentation

Overview

Package provider provides interfaces and tooling for (Re)providers.

This includes methods to provide streams of CIDs (i.e. from pinned merkledags, from blockstores, from single dags etc.). These methods can be used for other purposes, but are usually fed to the Reprovider to announce CIDs.

Index

Constants

View Source 
const (
	// MAGIC: how long we wait before reproviding a key
	DefaultReproviderInterval = time.Hour * 22 // https://github.com/ipfs/kubo/pull/9326

)

Variables

View Source 
var (
	DefaultKeyPrefix = datastore.NewKey("/provider")
)

Functions

This section is empty.

Types

type KeyChanFunc added in v0.9.0 

type KeyChanFunc func(context.Context) (<-chan cid.Cid, error)

KeyChanFunc is function streaming CIDs to pass to content routing

func NewBlockstoreProvider added in v0.9.0 

func NewBlockstoreProvider(bstore blocks.Blockstore) KeyChanFunc

NewBlockstoreProvider returns key provider using bstore.AllKeysChan

func NewBufferedProvider added in v0.29.1 

func NewBufferedProvider(pinsF KeyChanFunc) KeyChanFunc

NewBufferedProvider returns a KeyChanFunc supplying keys from a given KeyChanFunction, but buffering keys in memory if we can read them faster they are consumed. This allows the underlying KeyChanFunc to finish listing pins as soon as possible releasing any resources, locks, at the expense of memory usage.

func NewPinnedProvider added in v0.9.0 

func NewPinnedProvider(onlyRoots bool, pinning pin.Pinner, fetchConfig fetcher.Factory) KeyChanFunc

NewPinnedProvider returns a KeyChanFunc supplying pinned keys. The Provider will block when writing to the channel and there are no readers.

func NewPrioritizedProvider added in v0.19.0 

func NewPrioritizedProvider(priorityCids KeyChanFunc, otherCids KeyChanFunc) KeyChanFunc

type Option added in v0.9.0 

type Option func(system *reprovider) error

Option defines the functional option type that can be used to configure BatchProvidingSystem instances

func Allowlist added in v0.12.0 

func Allowlist(allowlist verifcid.Allowlist) Option

func DatastorePrefix added in v0.9.0 

func DatastorePrefix(k datastore.Key) Option

DatastorePrefix sets a prefix for internal state stored in the Datastore. Defaults to DefaultKeyPrefix.

func KeyProvider added in v0.9.0 

func KeyProvider(fn KeyChanFunc) Option

func MaxBatchSize added in v0.24.1 

func MaxBatchSize(n uint) Option

MaxBatchSize limit how big each batch is. Some content routers like acceleratedDHTClient have sub linear scalling and bigger sizes are thus faster per elements however smaller batch sizes can limit memory usage spike.

func Online added in v0.9.0 

func Online(rsys Provide) Option

Online will enable the router and make it send publishes online. nil can be used to turn the router offline. You can't register multiple providers, if this option is passed multiple times it will error.

func ReproviderInterval added in v0.9.0 

func ReproviderInterval(duration time.Duration) Option

func ThroughputReport added in v0.9.0 

func ThroughputReport(f ThroughputCallback, minimumProvides uint) Option

ThroughputReport will fire the callback synchronously once at least limit multihashes have been advertised, it will then wait until a new set of at least limit multihashes has been advertised. While ThroughputReport is set batches will be at most minimumProvides big. If it returns false it wont ever be called again.

type Provide added in v0.9.0 

type Provide interface {
	Provide(context.Context, cid.Cid, bool) error
}

type ProvideMany added in v0.9.0 

type ProvideMany interface {
	ProvideMany(ctx context.Context, keys []multihash.Multihash) error
}

type Provider 

type Provider interface {
	// Provide takes a cid and makes an attempt to announce it to the network
	Provide(context.Context, cid.Cid, bool) error
}

Provider announces blocks to the network

type Ready added in v0.9.0 

type Ready interface {
	Ready() bool
}

type Reprovider 

type Reprovider interface {
	// Reprovide starts a new reprovide if one isn't running already.
	Reprovide(context.Context) error
}

Reprovider reannounces blocks to the network

type ReproviderStats added in v0.9.0 

type ReproviderStats struct {
	TotalProvides, LastReprovideBatchSize                        uint64
	ReprovideInterval, AvgProvideDuration, LastReprovideDuration time.Duration
	LastRun                                                      time.Time
}

type System 

type System interface {
	Close() error
	Stat() (ReproviderStats, error)
	Provider
	Reprovider
}

System defines the interface for interacting with the value provider system

func New added in v0.9.0 

func New(ds datastore.Batching, opts ...Option) (System, error)

New creates a new System. By default it is offline, that means it will enqueue tasks in ds. To have it publish records in the network use the Online option. If provider casts to ProvideMany the [ProvideMany.ProvideMany] method will be called instead.

If provider casts to Ready, it will wait until [Ready.Ready] is true.

func NewNoopProvider added in v0.9.0 

func NewNoopProvider() System

NewNoopProvider creates a ProviderSystem that does nothing.

type ThroughputCallback added in v0.9.0 

type ThroughputCallback = func(reprovide bool, complete bool, totalKeysProvided uint, totalDuration time.Duration) (continueWatching bool)

Source Files

View all Source files

Directories

Path Synopsis
internal
queue

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.