Documentation
¶
Index ¶
- Constants
- type Config
- type Spool
- func (s *Spool) Cleanup() (int, error)
- func (s *Spool) Exists(queryID string) bool
- func (s *Spool) IsPinned(queryID string) bool
- func (s *Spool) ListFiles() ([]string, error)
- func (s *Spool) NewStreamWriter(queryID string) (*StreamWriter, error)
- func (s *Spool) OpenReader(queryID string) (*ipc.Reader, io.Closer, error)
- func (s *Spool) Path(queryID string) string
- func (s *Spool) Pin(queryID string) error
- func (s *Spool) Remove(queryID string) error
- func (s *Spool) TotalSize() (int64, error)
- func (s *Spool) Unpin(queryID string) error
- type StreamWriter
Constants ¶
const ( DefaultTTL = 24 * time.Hour DefaultMaxSize = 2 * 1024 * 1024 * 1024 // 2 GB DefaultDir = "" // empty = os.TempDir()/duckdb-kernel )
Default configuration.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶ added in v0.2.0
type Config struct {
Dir string // spool directory (default: /tmp/duckdb-kernel)
TTL time.Duration // max file age (default: 24h)
MaxSize int64 // max total disk usage in bytes (default: 2GB)
}
Config holds spool configuration.
type Spool ¶
type Spool struct {
Dir string
PersistentDir string // optional, set via config or Pin()
TTL time.Duration
MaxSize int64
}
Spool manages Arrow IPC files on disk. Flat directory structure — no session nesting. Files are named {queryID}.arrow. TTL-based cleanup + disk size limit.
Supports two directories:
- Dir (volatile): /tmp/duckdb-kernel/ — TTL cleanup, auto-managed
- PersistentDir (optional): .duckdb-results/ — user-pinned results, no TTL
func NewSpool ¶
NewSpool creates a spool with the given config. Creates the directory if it doesn't exist. Runs initial cleanup (delete expired files, enforce size limit).
func (*Spool) Cleanup ¶
Cleanup deletes expired files and enforces disk size limit. Returns the number of files deleted.
func (*Spool) IsPinned ¶ added in v0.2.0
IsPinned checks if a query result is in the persistent dir.
func (*Spool) ListFiles ¶ added in v0.2.0
ListFiles returns all .arrow file names (without extension) in the spool dir.
func (*Spool) NewStreamWriter ¶ added in v0.1.1
func (s *Spool) NewStreamWriter(queryID string) (*StreamWriter, error)
NewStreamWriter creates a streaming IPC writer for the given query.
func (*Spool) OpenReader ¶ added in v0.1.1
OpenReader opens a streaming IPC reader for the given query. Checks persistent dir first (pinned results), then volatile dir.
func (*Spool) Pin ¶ added in v0.2.0
Pin copies an Arrow file from volatile dir to persistent dir. Idempotent: returns nil if already pinned. Creates persistent dir and .gitignore if needed.
func (*Spool) Remove ¶ added in v0.1.2
Remove deletes a single spool file by query ID (from both dirs).
type StreamWriter ¶ added in v0.1.1
type StreamWriter struct {
// contains filtered or unexported fields
}
StreamWriter writes Arrow record batches to an IPC streaming file.
func (*StreamWriter) Close ¶ added in v0.1.1
func (sw *StreamWriter) Close() error
Close flushes and closes the IPC stream and underlying file.
func (*StreamWriter) Write ¶ added in v0.1.1
func (sw *StreamWriter) Write(rec arrow.RecordBatch) error
Write writes a single record batch to the IPC stream.
Source Files