storage

package

v0.0.1 Latest Latest Go to latest Published: Jul 15, 2021 License: Apache-2.0 Imports: 12 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/newity/crawler

Links

Open Source Insights

Documentation ¶

Overview ¶

Google cloud Pub/Sub implementation of Storage

Index ¶

type Badger
- func NewBadger(path string) (*Badger, error)
type Nats
- func NewNats(clusterID, clientID, natsURL string, maxPubAcksInflight int) (*Nats, error)
type PubSub
- func NewPubSub(project string, opts ...option.ClientOption) (*PubSub, error)
type Storage

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Badger ¶

type Badger struct {
	// contains filtered or unexported fields
}

func NewBadger ¶

func NewBadger(path string) (*Badger, error)

func (*Badger) Close ¶

func (b *Badger) Close() error

func (*Badger) Delete ¶

func (b *Badger) Delete(key string) error

func (*Badger) Get ¶

func (b *Badger) Get(key string) ([]byte, error)

Get retrieves data from BadgerDB using key

func (*Badger) GetStream ¶

func (b *Badger) GetStream(key string) (<-chan []byte, <-chan error)

func (*Badger) InitChannelsStorage ¶

func (b *Badger) InitChannelsStorage(channels []string) error

func (*Badger) Put ¶

func (b *Badger) Put(key string, value []byte) error

Put saves value by key

type Nats ¶

type Nats struct {
	Connection stan.Conn
	// contains filtered or unexported fields
}

func NewNats ¶

func NewNats(clusterID, clientID, natsURL string, maxPubAcksInflight int) (*Nats, error)

func (*Nats) Close ¶

func (n *Nats) Close() error

Close stops all running goroutines related to topics.

func (*Nats) Delete ¶

func (n *Nats) Delete(key string) error

Detele does not work for Nats.

func (*Nats) Get ¶

func (n *Nats) Get(topic string) ([]byte, error)

Get reads one message from the topic and closes channel.

func (*Nats) GetStream ¶

func (n *Nats) GetStream(topic string) (<-chan []byte, <-chan error)

GetStream reads a stream of messages from topic and writes them to the channel.

func (*Nats) InitChannelsStorage ¶

func (n *Nats) InitChannelsStorage(channels []string) error

func (*Nats) Put ¶

func (n *Nats) Put(topic string, msg []byte) error

Put stores message to topic.

type PubSub ¶

type PubSub struct {
	// contains filtered or unexported fields
}

func NewPubSub ¶

func NewPubSub(project string, opts ...option.ClientOption) (*PubSub, error)

func (*PubSub) Close ¶

func (p *PubSub) Close() error

Close stops all running goroutines related to topics.

func (*PubSub) Delete ¶

func (p *PubSub) Delete(key string) error

Detele deletes topic and subscription specified by key.

func (*PubSub) Get ¶

func (p *PubSub) Get(topic string) ([]byte, error)

Get reads one message from the topic and closes channel.

func (*PubSub) GetStream ¶

func (p *PubSub) GetStream(topic string) (<-chan []byte, <-chan error)

GetStream reads a stream of messages from topic and writes them to the channel.

func (*PubSub) InitChannelsStorage ¶

func (p *PubSub) InitChannelsStorage(channels []string) error

func (*PubSub) Put ¶

func (p *PubSub) Put(topic string, msg []byte) error

Put stores message to topic.

type Storage ¶

type Storage interface {
	// init storage (initial setup of storage and connection create operations)
	InitChannelsStorage(channels []string) error
	// put value by key
	Put(key string, value []byte) error
	// get data from storage by specified key
	Get(key string) ([]byte, error)
	// get channel with some data from storage (for message broker storage implementations)
	GetStream(key string) (<-chan []byte, <-chan error)
	// remove parser.Data from storage
	Delete(key string) error
	// close connection to storage (network connections, file descriptors, goroutines)
	Close() error
}

Storage interface is a contract for storage implementations

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL