spool

package

v1.2.5 Latest Latest Go to latest Published: Apr 7, 2025 License: MIT Imports: 7 Imported by: 1

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/fogfish/stream

Links

Open Source Insights

Documentation ¶

Index ¶

Variables
type File
type FileSystem
type Spool
- func New(reader, writer FileSystem, opt ...opts.Option[Spool]) *Spool

Constants ¶

This section is empty.

Variables ¶

View Source

var (

	// spool is immutable, consumed tasks remains in spool.
	IsImmutable = withMutability(immutable)
	// spool is mutable, consumed tasks are removed
	IsMutable = withMutability(mutable)

	// spool fails on error
	WithStrict = withStrict(strict)
	// spool skips error, continue as warning
	WithSkipError = withStrict(skiperror)
)

Functions ¶

This section is empty.

Types ¶

type File ¶

type File = interface {
	io.Writer
	io.Closer
	Stat() (fs.FileInfo, error)
	Cancel() error
}

File provides I/O access to individual object on the file system.

type FileSystem ¶

type FileSystem interface {
	fs.FS
	Create(path string, attr *struct{}) (File, error)
	Remove(path string) error
}

A FileSystem provides access to a hierarchical file system. The abstraction support I/O to local file system or AWS S3. Use it with https://github.com/fogfish/stream

type Spool ¶

type Spool struct {
	// contains filtered or unexported fields
}

func New ¶

func New(reader, writer FileSystem, opt ...opts.Option[Spool]) *Spool

func (*Spool) ForEach ¶

func (spool *Spool) ForEach(
	ctx context.Context,
	dir string,
	f func(context.Context, string, io.Reader) (io.ReadCloser, error),
) error

Apply the spool function over each file in the reader filesystem, producing results to writer file system.

func (*Spool) ForEachFile ¶

func (spool *Spool) ForEachFile(
	ctx context.Context,
	dir string,
	f func(context.Context, string, []byte) ([]byte, error),
) error

Apply the spool function over each file in the reader filesystem, producing results to writer file system. It is a variant of [ForEach] that used bytes slices.

func (*Spool) Partition ¶ added in v1.2.5

func (spool *Spool) Partition(
	ctx context.Context,
	dir string,
	f func(context.Context, string, io.Reader) (string, error),
) error

Apply the parition function over each file in the reader filesystem, producing results to writer file system.

func (*Spool) PartitionFile ¶ added in v1.2.5

func (spool *Spool) PartitionFile(
	ctx context.Context,
	dir string,
	f func(context.Context, string, []byte) (string, error),
) error

func (*Spool) Write ¶

func (spool *Spool) Write(path string, r io.Reader) error

Write new file to spool

func (*Spool) WriteFile ¶

func (spool *Spool) WriteFile(path string, b []byte) error

Source Files ¶

View all Source files

spool.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL