file

package
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 15, 2026 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package file provides protocol-agnostic file IO adapter bindings for the ports package.

All adapters are stdlib-only (no external dependencies). They implement the ports.SourceAdapter, ports.SinkAdapter, and ports.IOAdapter interfaces and are wired to pipelines via ports.SourcePort.Bind, ports.SinkPort.Bind, and ports.IOPort.Bind.

Sources (use with ports.SourcePort):

  • ScanAdapter — decodes a newline-delimited file (NDJSON, CSV, etc.) line by line
  • WatchAdapter — emits file paths for new files created in a directory

Intermediate (use with ports.IOPort):

  • ReadEachAdapter — reads a complete typed file for each upstream item (enrichment)

Sinks (use with ports.SinkPort):

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DrainWriteAdapter

func DrainWriteAdapter[T any](
	w io.Writer,
	fmt format.Format[T],
	opts DrainWriteAdapterOptions,
) ports.SinkAdapter[T]

DrainWriteAdapter returns a ports.SinkAdapter that encodes each item and writes it as a line to w. Use with ports.SinkPort.Bind:

f, _ := os.Create("results.ndjson")
domain.OEEResults.Bind(ctx, file.DrainWriteAdapter(f, format.JSON(oeeCodec),
    file.DrainWriteAdapterOptions{Path: "results.ndjson"}))

func DrainWriteFileAdapter

func DrainWriteFileAdapter[T any](
	f format.File[T],
	varsFor func(T) map[string]string,
	opts DrainWriteFileAdapterOptions,
) ports.SinkAdapter[T]

DrainWriteFileAdapter returns a ports.SinkAdapter that writes each item as a complete typed file (whole-file overwrite). Use with ports.SinkPort.Bind:

domain.OEEResults.Bind(ctx, file.DrainWriteFileAdapter(resultFile,
    func(oee OEE) map[string]string { return map[string]string{"machineID": oee.MachineID} },
    file.DrainWriteFileAdapterOptions{}))

When the bound ports.SinkPort declares Params, each varsFor result is validated with ports.ValidateParams before the file write; a validation failure is reported to Options.OnError as WriteError wrapping [codex.ValidationErrors] and the item is otherwise skipped (not written).

func ReadAdapter

func ReadAdapter[In, Resp any](
	f format.File[Resp],
	varsFor func(In) map[string]string,
	opts ReadEachAdapterOptions,
) ports.IOAdapter[In, Resp]

ReadAdapter returns a ports.IOAdapter that reads a complete typed file for each In item and emits the file content directly as the response — the 2-type complement of ReadEachAdapter (whose independent file-content type and combine func serve enrichment). Pairs with a ports.FilePattern declared on a ports.IOPort[In, Resp], where the file content IS the port's response type:

calibFile, _ := ports.FileHandle[CalibrationData](domain.Calibration)
domain.Calibration.Bind(ctx, file.ReadAdapter(calibFile,
    func(r SensorReading) map[string]string {
        return map[string]string{"sensorID": r.SensorID}
    },
    file.ReadEachAdapterOptions{}))

When the bound ports.IOPort declares Params, each varsFor result is validated with ports.ValidateParams before the file read; a validation failure is delivered as ReadError wrapping [codex.ValidationErrors].

func ReadEachAdapter

func ReadEachAdapter[In, T, Resp any](
	f format.File[T],
	varsFor func(In) map[string]string,
	combine func(In, T) Resp,
	opts ReadEachAdapterOptions,
) ports.IOAdapter[In, Resp]

ReadEachAdapter returns a ports.IOAdapter that reads a complete typed file for each In item, combining the result. Use with ports.IOPort.Bind:

domain.Calibration.Bind(ctx, file.ReadEachAdapter(calibrationFile,
    func(r SensorReading) map[string]string { return map[string]string{"id": r.SensorID} },
    func(r SensorReading, c CalibrationData) CalibratedReading { return ... },
    file.ReadEachAdapterOptions{}))

When the bound ports.IOPort declares Params, each varsFor result is validated with ports.ValidateParams before the file read; a validation failure is delivered as ReadError wrapping [codex.ValidationErrors].

func ScanAdapter

func ScanAdapter[T any](path string, fmt format.Format[T], opts ScanAdapterOptions) ports.SourceAdapter[T]

ScanAdapter returns a ports.SourceAdapter that reads a file line-by-line, decoding each line. When the file is fully read the adapter exits. Use with ports.SourcePort.Bind:

domain.Readings.Bind(ctx, file.ScanAdapter("readings.ndjson", format.JSON(readingCodec),
    file.ScanAdapterOptions{}))

func WatchAdapter

func WatchAdapter(dir string, interval time.Duration, opts WatchAdapterOptions) ports.SourceAdapter[string]

WatchAdapter returns a ports.SourceAdapter that emits file paths for new files created in dir. Runs until ctx is cancelled. Use with ports.SourcePort.Bind:

domain.NewFiles.Bind(ctx, file.WatchAdapter("/data/incoming", 5*time.Second,
    file.WatchAdapterOptions{}))

Types

type DrainWriteAdapterOptions

type DrainWriteAdapterOptions struct {
	Path      string
	Separator string
	Observer  stats.Observer
	OnError   func(error)
}

DrainWriteAdapterOptions configures DrainWriteAdapter.

type DrainWriteFileAdapterOptions

type DrainWriteFileAdapterOptions struct {
	Observer    stats.Observer
	FileOptions format.FileOptions
	OnError     func(error)
}

DrainWriteFileAdapterOptions configures DrainWriteFileAdapter.

type ReadEachAdapterOptions

type ReadEachAdapterOptions struct {
	Observer    stats.Observer
	FileOptions format.FileOptions
	Buffer      int
}

ReadEachAdapterOptions configures ReadEachAdapter.

type ReadError

type ReadError struct {
	// Err is the underlying error from [format.File.Read].
	Err error
}

ReadError is sent to [Stream.Errors] by [ReadEachStream] when reading or decoding a file fails for an upstream stream item. It wraps the underlying error (typically format.FileReadError or format.FileDecodeError).

func (ReadError) Error

func (e ReadError) Error() string

func (ReadError) LogValue

func (e ReadError) LogValue() slog.Value

LogValue implements slog.LogValuer for structured logging.

func (ReadError) Unwrap

func (e ReadError) Unwrap() error

Unwrap allows errors.Is and errors.As to traverse the underlying error.

type ScanAdapterOptions

type ScanAdapterOptions struct {
	Observer stats.Observer
	Buffer   int
}

ScanAdapterOptions configures ScanAdapter.

type ScanError

type ScanError struct {
	// Path is the file path passed to ScanStream.
	Path string
	// Err is the underlying I/O or decode error.
	Err error
}

ScanError is sent to [Stream.Errors] by [ScanStream] when opening or reading the file fails. When Err wraps gstream.StreamDecodeError, the failure was a codec decode error on a specific line; otherwise it is an I/O error.

func (ScanError) Error

func (e ScanError) Error() string

func (ScanError) LogValue

func (e ScanError) LogValue() slog.Value

LogValue implements slog.LogValuer for structured logging.

func (ScanError) Unwrap

func (e ScanError) Unwrap() error

Unwrap allows errors.Is and errors.As to traverse the underlying error.

type WatchAdapterOptions

type WatchAdapterOptions struct {
	Observer stats.Observer
	Buffer   int
}

WatchAdapterOptions configures WatchAdapter.

type WatchError

type WatchError struct {
	// Dir is the directory being watched.
	Dir string
	// Err is the underlying os.ReadDir error.
	Err error
}

WatchError is sent to [Stream.Errors] by [WatchStream] when a directory read fails during a poll cycle. The stream continues on the next poll interval.

func (WatchError) Error

func (e WatchError) Error() string

func (WatchError) LogValue

func (e WatchError) LogValue() slog.Value

LogValue implements slog.LogValuer for structured logging.

func (WatchError) Unwrap

func (e WatchError) Unwrap() error

Unwrap allows errors.Is and errors.As to traverse the underlying error.

type WriteError

type WriteError struct {
	// Path is the file path, when known. Empty when writing to a non-file writer.
	Path string
	// Err is the underlying encode or write error.
	Err error
}

WriteError is passed to [DrainWriteOptions.OnError] by [DrainWrite] when encoding or writing an item to the writer fails.

func (WriteError) Error

func (e WriteError) Error() string

func (WriteError) LogValue

func (e WriteError) LogValue() slog.Value

LogValue implements slog.LogValuer for structured logging.

func (WriteError) Unwrap

func (e WriteError) Unwrap() error

Unwrap allows errors.Is and errors.As to traverse the underlying error.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL