spool

package

v0.1.2 Latest Latest Go to latest Published: Mar 13, 2026 License: MIT Imports: 9 Imported by: 0

Details

View Source

const (
	DefaultMaxFiles = 5
	DefaultMaxAge   = 1 * time.Hour
)

This section is empty.

This section is empty.

type Spool struct {
	Dir      string
	MaxFiles int
	MaxAge   time.Duration
}

Spool manages temporary Arrow IPC files for a session.

func NewSpool(sessionID string) (*Spool, error)

NewSpool creates a new result spool for the given session.

func (s *Spool) Cleanup() error

Cleanup removes files exceeding the max count and max age limits.

func (s *Spool) Destroy() error

Destroy removes the entire spool directory.

func (s *Spool) NewStreamWriter(queryID string) (*StreamWriter, error)

NewStreamWriter creates a streaming IPC writer for the given query. Schema is taken from the first Write call.

func (s *Spool) OpenReader(queryID string) (*ipc.Reader, io.Closer, error)

OpenReader opens a streaming IPC reader for the given query.

func (s *Spool) Path(queryID string) string

Path returns the Arrow IPC file path for a given query ID. Uses filepath.Base to prevent path traversal attacks.

func (s *Spool) Remove(queryID string) error

Remove deletes a single spool file by query ID.

type StreamWriter struct {
	// contains filtered or unexported fields
}

StreamWriter writes Arrow record batches to an IPC streaming file. Batches are flushed to disk immediately — no accumulation in memory.

func (sw *StreamWriter) Close() error

Close flushes and closes the IPC stream and underlying file.

func (sw *StreamWriter) Write(rec arrow.Record) error

Write writes a single record batch to the IPC stream.