frisbii

package module
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 9, 2023 License: Apache-2.0, MIT Imports: 33 Imported by: 4

README

Frisbii

An experimental minimal IPLD data provider for the IPFS network

Frisbii is currently a simple HTTP server that conforms minimally to the IPFS Trustless Gateway specification, serving IPLD data in CAR format to HTTP clients.

Content routing, when enabled, is delegated to the Interpletary Network Indexer (IPNI) service. On startup, Frisbii can announce its content to IPNI, allowing it to be discovered by clients that query the indexer for that content.

Frisbii does not directly interact with the IPFS DHT, which limits discovery potential for clients which primarily use the DHT. However, with increasing reliance on IPNI for content discovery, this will change over time. In particular, Lassie, a universal IPFS and Filecoin retrieval client, relies on IPNI for content discovery.

Installation

To install the latest version of Frisbii, run:

go install github.com/ipld/frisbii/cmd/frisbii@latest

Alternatively, binaries are available for download on the releases page.

Usage

frisbii --announce=roots --car=/path/to/file.car

Note that for announcements to be successful, Frisbii must be able to determine its public address. You may need to supply a --listen argument with a public address, or use --public-addr to override the address that Frisbii determines for itself.

Full argument list:

  • --car - path to one or more CAR files to serve, this can be a plain path, a glob path to match multiple files, and --car can be supplied multiple times.
  • --announce - announce the given roots to IPNI on startup. Can be roots or none. Defaults to none.
  • --listen - hostname and port to listen on. Defaults to :3747.
  • --public-addr - multiaddr or URL of this server as seen by the indexer and other peers if it is different to the listen address. Defaults address of the server once started (typically the value of --listen).
  • --log-file - path to file to append HTTP request and error logs to. Defaults to stdout.
  • --max-response-duration - maximum duration to spend responding to a request. Defaults to 5m.
  • --max-response-bytes - maximum size of a response from IPNI. Defaults to 100MiB.
  • --verbose - enable verbose logging. Defaults to false. Same as using GOLOG_LOG_LEVEL=debug as an environment variable. GOLOG_LOG_LEVEL can be used for more fine-grained control of log output.
  • --help - show help.
CAR files
  • go-car can be used to author, manipulate and inspect CAR files.
  • Kubo's dag export command can be used to export a CAR file from an IPFS node, although this is limited to complete DAGs.
  • Lassie can be used to export a CAR file from the network.

Both CARv1 and CARv2 formats are usable by Frisbii. However, on startup, Frisbii will need to generate an index in memory for a CARv1. So for faster start-up times it is recommended that you start Frisbii with CARv2 files (using go-car this can be done with car index input.car > output.car).

Using --anounce=roots will announce the roots of all CARs loaded by Frisbii to the indexer. Other blocks are not announced, and will not be discoverable by clients that query the indexer for that content, however they are served by Frisbii when requested directly or as part of a DAG whose root has been advertised.

Library usage

See https://pkg.go.dev/github.com/ipld/frisbii for full documentation.

NewFrisbiiServer() can be used to create a new server given a LinkSystem as a source of IPLD data.

Further Development

The goal of Frisbii is to be maximally minimal according to user need. It is not intended to be a full IPFS node, but rather a simple server that can be used to serve IPLD data. However, the limitations of HTTP as the only transport, the restrictions within the current minimal Trustless Gateway implementation, and reliance on IPNI for content announcement present some challenges for data provision to a wide audience.

Some areas for possible further development (perhaps by you!) include:

  • Support for additional transports, including Graphsync over a minimal DataTransfer implementation to mirror the default transport functionality of Filecoin storage providers, and Bitswap for the widest possible compatibility with existing IPFS clients. Unfortunately both transports introduce considerable complexity.
  • Support for P2P message sending for IPNI, so announcements can be made to multiple indexers via pubsub.
  • DHT support.
  • Support for alternative data storage backends.
  • More advanced announcement options, including all CIDs, or a user-supplied list of CIDs.

License

Apache-2.0/MIT © Protocol Labs

Documentation

Index

Constants

View Source 
const ContextID = "frisbii"

Variables

This section is empty.

Functions

func StreamCar 

func StreamCar(
	ctx context.Context,
	requestLsys linking.LinkSystem,
	out io.Writer,
	request trustlessutils.Request,
) error

StreamCar streams a DAG in CARv1 format to the given writer, using the given selector.

Types

type ErrorLogger 

type ErrorLogger interface {
	LogError(status int, err error)
}

type FrisbiiServer 

type FrisbiiServer struct {
	// contains filtered or unexported fields
}

FrisbiiServer is the main server for the frisbii application, it starts an HTTP server to serve data according to the Trustless Gateway spec and it also provides a mechanism to announce the server to the indexer service.

func NewFrisbiiServer 

func NewFrisbiiServer(
	ctx context.Context,
	logWriter io.Writer,
	lsys linking.LinkSystem,
	address string,
	httpOptions ...HttpOption,
) (*FrisbiiServer, error)

func (*FrisbiiServer) Addr 

func (fs *FrisbiiServer) Addr() net.Addr

func (*FrisbiiServer) Announce 

func (fs *FrisbiiServer) Announce() error

func (*FrisbiiServer) Serve 

func (fs *FrisbiiServer) Serve() error

func (*FrisbiiServer) SetIndexerProvider 

func (fs *FrisbiiServer) SetIndexerProvider(handlerPath string, indexerProvider IndexerProvider) error

type HttpIpfs 

type HttpIpfs struct {
	// contains filtered or unexported fields
}

HttpIpfs is an http.Handler that serves IPLD data via HTTP according to the Trustless Gateway specification.

func NewHttpIpfs 

func NewHttpIpfs(
	ctx context.Context,
	lsys linking.LinkSystem,
	opts ...HttpOption,
) *HttpIpfs

func (*HttpIpfs) ServeHTTP 

func (hi *HttpIpfs) ServeHTTP(res http.ResponseWriter, req *http.Request)

type HttpOption added in v0.1.0 

type HttpOption func(*httpOptions)

func WithMaxResponseBytes added in v0.1.0 

func WithMaxResponseBytes(b int64) HttpOption

WithMaxResponseBytes sets the maximum number of bytes that will be streamed before the connection is closed. This allows a server to limit the amount of data a client can request; and also restricts the ability to serve very large DAGs.

A value of 0 will disable the limitation. This is the default.

func WithMaxResponseDuration added in v0.1.0 

func WithMaxResponseDuration(d time.Duration) HttpOption

WithMaxResponseDuration sets the maximum duration for a response to be streamed before the connection is closed. This allows a server to limit the amount of time a client can hold a connection open; and also restricts the ability to serve very large DAGs.

A value of 0 will disable the limitation. This is the default.

type IndexerProvider 

type IndexerProvider interface {
	GetPublisherHttpFunc() (http.HandlerFunc, error)
	NotifyPut(ctx context.Context, provider *peer.AddrInfo, contextID []byte, md metadata.Metadata) (cid.Cid, error)
}

type LogMiddleware 

type LogMiddleware struct {
	// contains filtered or unexported fields
}

LogMiddlware is a middleware that logs requests to the given io.Writer. it wraps requests in a LoggingResponseWriter that can be used to log standardised messages to the writer.

func NewLogMiddleware 

func NewLogMiddleware(next http.Handler, logWriter io.Writer) *LogMiddleware

func (*LogMiddleware) ServeHTTP 

func (lm *LogMiddleware) ServeHTTP(res http.ResponseWriter, req *http.Request)

type LoggingResponseWriter 

type LoggingResponseWriter struct {
	http.ResponseWriter
	// contains filtered or unexported fields
}

func NewLoggingResponseWriter 

func NewLoggingResponseWriter(w http.ResponseWriter, req *http.Request, logWriter io.Writer) *LoggingResponseWriter

func (*LoggingResponseWriter) Hijack 

func (w *LoggingResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error)

func (*LoggingResponseWriter) Log 

func (w *LoggingResponseWriter) Log(status int, duration time.Duration, bytes int, msg string)

func (*LoggingResponseWriter) LogError 

func (w *LoggingResponseWriter) LogError(status int, err error)

func (*LoggingResponseWriter) Write 

func (w *LoggingResponseWriter) Write(b []byte) (int, error)

func (*LoggingResponseWriter) WriteHeader 

func (w *LoggingResponseWriter) WriteHeader(status int)

type MultiReadableStorage 

type MultiReadableStorage struct {
	// contains filtered or unexported fields
}

MultiReadableStorage manages a list of storage.StreamingReadableStorage stores, providing a unified LinkSystem interface to them.

func NewMultiReadableStorage 

func NewMultiReadableStorage() *MultiReadableStorage

func (*MultiReadableStorage) AddStore 

func (m *MultiReadableStorage) AddStore(store storage.StreamingReadableStorage, roots []cid.Cid)

func (*MultiReadableStorage) Get 

func (m *MultiReadableStorage) Get(ctx context.Context, key string) ([]byte, error)

func (*MultiReadableStorage) GetStream 

func (m *MultiReadableStorage) GetStream(ctx context.Context, key string) (io.ReadCloser, error)

func (*MultiReadableStorage) Has 

func (m *MultiReadableStorage) Has(ctx context.Context, key string) (bool, error)

func (*MultiReadableStorage) RootsLister 

func (m *MultiReadableStorage) RootsLister() provider.MultihashLister

Source Files

View all Source files

Directories

Path Synopsis
cmd
frisbii command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.