Documentation
¶
Overview ¶
Package multibuf implements buffer optimized for streaming large chunks of data, multiple reads and optional partial buffering to disk.
Index ¶
Constants ¶
const ( DefaultMemBytes = 1048576 DefaultMaxBytes = -1 // DefaultBufferBytes equivalent of bytes.MinRead used in io.ReadAll. DefaultBufferBytes = 512 )
Default sizes.
Variables ¶
var ( // ErrNoDataReady is an error when getting the reader while the data is not // ready. ErrNoDataReady = errors.New("mailgun/multibuf: no data ready") // ErrReaderHasBeenCalled is an error when getting the reader or writing // while the reader has been called. ErrReaderHasBeenCalled = errors.New("mailgun/multibuf: reader has been called") )
Functions ¶
This section is empty.
Types ¶
type MaxSizeReachedError ¶
type MaxSizeReachedError struct {
MaxSize int64
}
MaxSizeReachedError is returned when the maximum allowed buffer size is reached when reading.
func (*MaxSizeReachedError) Error ¶
func (e *MaxSizeReachedError) Error() string
type MultiReader ¶
type MultiReader interface {
io.Reader
io.Seeker
io.Closer
io.WriterTo
// Size calculates and returns the total size of the reader and not the length remaining.
Size() (int64, error)
}
MultiReader provides Read, Close, Seek and Size methods. In addition to that it supports WriterTo interface to provide efficient writing schemes, as functions like io.Copy use WriterTo when it's available.
func New ¶
func New(input io.Reader, setters ...OptionSetter) (MultiReader, error)
New returns MultiReader that can limit the size of the buffer and persist large buffers to disk. By default, New returns unbound buffer that will read up to 1MB in RAM and will start buffering to disk It supports multiple functional optional arguments:
// Buffer up to 1MB in RAM and limit max buffer size to 20MB multibuf.New(r, multibuf.MemBytes(1024 * 1024), multibuf.MaxBytes(1024 * 1024 * 20))
type OptionSetter ¶
type OptionSetter func(o *options) error
func MaxBytes ¶
func MaxBytes(m int64) OptionSetter
MaxBytes ignored if set to value >=, if request exceeds the specified limit, the reader will return error, by default buffer is not limited, negative values mean no limit.
func MemBytes ¶
func MemBytes(m int64) OptionSetter
MemBytes specifies the largest buffer to hold in RAM before writing to disk, default is 1MB.
func TempDir ¶
func TempDir(dir string) OptionSetter
TempDir sets a custom temp directory to store files.
type WriterOnce ¶
type WriterOnce interface {
// Write implements io.Writer
Write(p []byte) (int, error)
// Reader transfers all data written to this writer to MultiReader. If there was no data written it returns an error
Reader() (MultiReader, error)
// Close WriterOnce owns the data before Reader has been called, so Close will close all the underlying files if Reader has not been called.
Close() error
}
WriterOnce implements write once, read many times writer. Create a WriterOnce and write to it, once Reader() function has been called, the internal data is transferred to MultiReader and this instance of WriterOnce should be no longer used.
func NewWriterOnce ¶
func NewWriterOnce(setters ...OptionSetter) (WriterOnce, error)
NewWriterOnce returns io.ReadWrite compatible object that can limit the size of the buffer and persist large buffers to disk. WriterOnce implements write once, read many times writer. Create a WriterOnce and write to it, once Reader() function has been called, the internal data is transferred to MultiReader and this instance of WriterOnce should be no longer used. By default, NewWriterOnce returns unbound buffer that will allow to write up to 1MB in RAM and will start buffering to disk It supports multiple functional optional arguments:
// Buffer up to 1MB in RAM and limit max buffer size to 20MB multibuf.NewWriterOnce(r, multibuf.MemBytes(1024 * 1024), multibuf.MaxBytes(1024 * 1024 * 20))