Documentation
¶
Index ¶
Constants ¶
const ( // InitialBufferSize is the initial pre-allocated buffer size for in-memory writes InitialBufferSize = 64 * 1024 // 64 KB initial buffer size // MaxInMemorySize is the max number of bytes (currently 1MB) to hold in memory before starting to write to disk MaxInMemorySize = 1024 * 1024 // DefaultMaxRAMUsageFraction is the default fraction of system RAM above which we'll force spooling to disk DefaultMaxRAMUsageFraction = 0.50 )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ReadSeekCloser ¶
type ReadSeekCloser interface {
io.Reader
io.Seeker
ReaderAt
io.Closer
FileName() string
Len() int
}
ReadSeekCloser is an io.Reader + ReaderAt + io.Seeker + io.Closer + Stat
type ReadWriteSeekCloser ¶
type ReadWriteSeekCloser interface {
ReadSeekCloser
io.Writer
}
ReadWriteSeekCloser is an io.Writer + io.Reader + io.Seeker + io.Closer.
func NewSpooledTempFile ¶
func NewSpooledTempFile(filePrefix string, tempDir string, threshold int, fullOnDisk bool, maxRAMUsageFraction float64) ReadWriteSeekCloser
NewSpooledTempFile returns an ReadWriteSeekCloser, with some important constraints:
- You can Write into it, but whenever you call Read or Seek on it, subsequent Write calls will panic.
- If threshold is -1, then the default MaxInMemorySize is used.
- If maxRAMUsageFraction <= 0, we default to DefaultMaxRAMUsageFraction. E.g. 0.5 = 50%.
If the system memory usage is above maxRAMUsageFraction, we skip writing to memory and spool directly on disk to avoid OOM scenarios in high concurrency.
If threshold is less than InitialBufferSize, we default to InitialBufferSize. This can cause a buffer not to spool to disk as expected given the threshold passed in. e.g.: If threshold is 100, it will default to InitialBufferSize (64KB), then 150B are written effectively crossing the passed threshold, but the buffer will not spool to disk as expected. Only when the buffer grows beyond 64KB will it spool to disk.
Source Files