Documentation
¶
Overview ¶
Package provider provides interfaces and tooling for (Re)providers.
This includes methods to provide streams of CIDs (i.e. from pinned merkledags, from blockstores, from single dags etc.). These methods can be used for other purposes, but are usually fed to the Reprovider to announce CIDs.
Index ¶
- Constants
- Variables
- type KeyChanFunc
- type MultihashProvider
- type Option
- func Allowlist(allowlist verifcid.Allowlist) Option
- func DatastorePrefix(k datastore.Key) Option
- func KeyProvider(fn KeyChanFunc) Option
- func MaxBatchSize(n uint) Option
- func Online(rsys Provide) Option
- func ProvideWorkerCount(n int) Option
- func ReproviderInterval(duration time.Duration) Option
- func ThroughputReport(f ThroughputCallback, minimumProvides uint) Option
- type Provide
- type ProvideMany
- type Provider
- type Ready
- type Reprovider
- type ReproviderStats
- type System
- type ThroughputCallback
Constants ¶
const ( // MAGIC: how long we wait before reproviding a key DefaultReproviderInterval = time.Hour * 22 // https://github.com/ipfs/kubo/pull/9326 )
Variables ¶
var (
DefaultKeyPrefix = datastore.NewKey("/provider")
)
Functions ¶
This section is empty.
Types ¶
type KeyChanFunc ¶ added in v0.9.0
KeyChanFunc is function streaming CIDs to pass to content routing
func NewBufferedProvider ¶ added in v0.29.1
func NewBufferedProvider(pinsF KeyChanFunc) KeyChanFunc
NewBufferedProvider returns a KeyChanFunc supplying keys from a given KeyChanFunction, but buffering keys in memory if we can read them faster they are consumed. This allows the underlying KeyChanFunc to finish listing pins as soon as possible releasing any resources, locks, at the expense of memory usage.
func NewDAGProvider ¶ added in v0.30.0
func NewDAGProvider(root cid.Cid, fetchConfig fetcher.Factory) KeyChanFunc
func NewPrioritizedProvider ¶ added in v0.19.0
func NewPrioritizedProvider(streams ...KeyChanFunc) KeyChanFunc
type MultihashProvider ¶ added in v0.35.0
type MultihashProvider interface {
// StartProviding announces blocks to the network, batching multiple keys for efficiency.
// The force parameter, when true, forces re-providing even if the keys were recently provided.
// When false, the implementation may skip keys that were recently announced based on internal
// rate limiting or caching strategies.
StartProviding(force bool, keys ...mh.Multihash) error
}
MultihashProvider is the interface implementing StartProviding.
type Option ¶ added in v0.9.0
type Option func(system *reprovider) error
Option defines the functional option type that can be used to configure BatchProvidingSystem instances
func DatastorePrefix ¶ added in v0.9.0
DatastorePrefix sets a prefix for internal state stored in the Datastore. Defaults to DefaultKeyPrefix.
func KeyProvider ¶ added in v0.9.0
func KeyProvider(fn KeyChanFunc) Option
func MaxBatchSize ¶ added in v0.24.1
MaxBatchSize limits how big each batch is.
Some content routers like acceleratedDHTClient have sub linear scalling and bigger sizes are thus faster per elements however smaller batch sizes can limit memory usage spike.
func Online ¶ added in v0.9.0
Online will enables the router and makes it send publishes online. A nil value can be used to set the router offline. It is not possible to register multiple providers. If this option is passed multiple times it will error.
func ProvideWorkerCount ¶ added in v0.30.0
ProvideWorkerCount configures the number of concurrent workers that handle the initial provide of newly added CIDs. The maximum rate at which CIDs are advertised is determined by dividing the number of workers by the provide duration. Note that each provide typically opens connections to at least the configured replication factor of peers.
Setting ProvideWorkerCount to 0 enables an unbounded number of workers, which can speed up provide operations under heavy load. Use this option carefully, as it may cause the libp2p node to open a very high number of connections to remote peers.
func ReproviderInterval ¶ added in v0.9.0
func ThroughputReport ¶ added in v0.9.0
func ThroughputReport(f ThroughputCallback, minimumProvides uint) Option
ThroughputReport fires the callback synchronously once at least limit multihashes have been advertised. It will then wait until a new set of at least limit multihashes has been advertised.
While ThroughputReport is set, batches will be at most minimumProvides big. If it returns false it will not be called again.
type ProvideMany ¶ added in v0.9.0
type Provider ¶
type Provider interface {
// Provide takes a cid and makes an attempt to announce it to the network
Provide(context.Context, cid.Cid, bool) error
}
Provider announces blocks to the network
type Reprovider ¶
type Reprovider interface {
// Reprovide starts a new reprovide if one isn't running already.
Reprovide(context.Context) error
}
Reprovider reannounces blocks to the network
type ReproviderStats ¶ added in v0.9.0
type System ¶
type System interface {
// Clear removes all entries from the provide queue. Returns the number of
// CIDs removed from the queue.
Clear() int
Close() error
Stat() (ReproviderStats, error)
SetKeyProvider(kp KeyChanFunc)
Provider
Reprovider
}
System defines the interface for interacting with the value provider system
func New ¶ added in v0.9.0
New creates a new System. By default it is offline, that means it will enqueue tasks in ds. To have it publish records in the network use the Online option. If provider casts to ProvideMany the [ProvideMany.ProvideMany] method will be called instead.
If provider casts to Ready, it will wait until [Ready.Ready] is true.
func NewNoopProvider ¶ added in v0.9.0
func NewNoopProvider() System
NewNoopProvider creates a ProviderSystem that does nothing.
Source Files
Directories