Documentation
¶
Overview ¶
Package file provides protocol-agnostic file IO adapter bindings for the ports package.
All adapters are stdlib-only (no external dependencies). They implement the ports.SourceAdapter, ports.SinkAdapter, and ports.IOAdapter interfaces and are wired to pipelines via ports.SourcePort.Bind, ports.SinkPort.Bind, and ports.IOPort.Bind.
Sources (use with ports.SourcePort):
- ScanAdapter — decodes a newline-delimited file (NDJSON, CSV, etc.) line by line
- WatchAdapter — emits file paths for new files created in a directory
Intermediate (use with ports.IOPort):
- ReadEachAdapter — reads a complete typed file for each upstream item (enrichment)
Sinks (use with ports.SinkPort):
- DrainWriteAdapter — encodes each item and writes it as a line to an io.Writer
- DrainWriteFileAdapter — writes each item as a complete typed file (whole-file overwrite)
- DrainPatchAdapter — applies each item as an untyped map[string]any partial update (JSON Merge Patch semantics) via ports.File.Patch; map-based formats (JSON/YAML/TOML/format.New) only
- DrainPatchEncodedAdapter — applies each item as a typed partial update via ports.PatchEncoded; same map-based-format restriction
Index ¶
- func DrainPatchAdapter[T any](f ports.File[T], varsFor func(map[string]any) map[string]string, ...) ports.SinkAdapter[map[string]any]
- func DrainPatchEncodedAdapter[T, P any](f ports.File[T], patchCodec codex.Codec[P], varsFor func(P) map[string]string, ...) ports.SinkAdapter[P]
- func DrainWriteAdapter[T any](w io.Writer, fmt format.Format[T], opts DrainWriteAdapterOptions) ports.SinkAdapter[T]
- func DrainWriteFileAdapter[T any](f ports.File[T], varsFor func(T) map[string]string, ...) ports.SinkAdapter[T]
- func ReadAdapter[In, Resp any](f ports.File[Resp], varsFor func(In) map[string]string, ...) ports.IOAdapter[In, Resp]
- func ReadEachAdapter[In, T, Resp any](f ports.File[T], varsFor func(In) map[string]string, combine func(In, T) Resp, ...) ports.IOAdapter[In, Resp]
- func ScanAdapter[T any](path string, fmt format.Format[T], opts ScanAdapterOptions) ports.SourceAdapter[T]
- func WatchAdapter(dir string, interval time.Duration, opts WatchAdapterOptions) ports.SourceAdapter[string]
- type DrainPatchAdapterOptions
- type DrainPatchEncodedAdapterOptions
- type DrainWriteAdapterOptions
- type DrainWriteFileAdapterOptions
- type ReadEachAdapterOptions
- type ReadError
- type ScanAdapterOptions
- type ScanError
- type WatchAdapterOptions
- type WatchError
- type WriteError
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DrainPatchAdapter ¶ added in v0.12.0
func DrainPatchAdapter[T any]( f ports.File[T], varsFor func(map[string]any) map[string]string, opts DrainPatchAdapterOptions, ) ports.SinkAdapter[map[string]any]
DrainPatchAdapter returns a ports.SinkAdapter that applies each item as a partial update (JSON Merge Patch semantics) to an existing typed file via ports.File.Patch, instead of overwriting the whole file like DrainWriteFileAdapter. f is built by hand (not via ports.FilePattern): the stream's item type (map[string]any) is deliberately different from f's own type T, the same way ReadEachAdapter takes an independent content type. Use with ports.SinkPort.Bind:
domain.ConfigUpdates.Bind(ctx, file.DrainPatchAdapter(configFile,
func(_ map[string]any) map[string]string { return nil },
file.DrainPatchAdapterOptions{}))
f's format must be map-based (JSON, YAML, TOML, or format.New); otherwise every item fails with ports.FilePatchNotSupportedError (passed through to Options.OnError unchanged — see below).
When the bound ports.SinkPort declares Params, each varsFor result is validated with ports.ValidateParams before the patch; a validation failure is reported to Options.OnError as WriteError wrapping codex.ValidationErrors and the item is otherwise skipped (not patched). A ports.File.Patch failure (including ports.FilePatchNotSupportedError) is passed to Options.OnError unchanged, mirroring DrainWriteFileAdapter's treatment of ports.File.Write failures.
func DrainPatchEncodedAdapter ¶ added in v0.12.0
func DrainPatchEncodedAdapter[T, P any]( f ports.File[T], patchCodec codex.Codec[P], varsFor func(P) map[string]string, opts DrainPatchEncodedAdapterOptions, ) ports.SinkAdapter[P]
DrainPatchEncodedAdapter returns a ports.SinkAdapter that applies each item as a typed partial update to an existing typed file via ports.PatchEncoded. Unlike DrainPatchAdapter's untyped map[string]any patches, patchCodec fields not present in f's own codec are still persisted — the right choice for intentionally adding new fields to a file. f is built by hand (not via ports.FilePattern): the stream's item type P is deliberately different from f's own type T. Use with ports.SinkPort.Bind:
domain.ConfigUpdates.Bind(ctx, file.DrainPatchEncodedAdapter(configFile, configPatchCodec,
func(_ AppConfigPatch) map[string]string { return nil },
file.DrainPatchEncodedAdapterOptions{}))
f's format must be map-based (JSON, YAML, TOML, or format.New); otherwise every item fails with ports.FilePatchNotSupportedError (passed through to Options.OnError unchanged — see below).
When the bound ports.SinkPort declares Params, each varsFor result is validated with ports.ValidateParams before the patch; a validation failure is reported to Options.OnError as WriteError wrapping codex.ValidationErrors and the item is otherwise skipped (not patched). A ports.PatchEncoded failure (including ports.FilePatchNotSupportedError) is passed to Options.OnError unchanged, mirroring DrainWriteFileAdapter's treatment of ports.File.Write failures.
func DrainWriteAdapter ¶
func DrainWriteAdapter[T any]( w io.Writer, fmt format.Format[T], opts DrainWriteAdapterOptions, ) ports.SinkAdapter[T]
DrainWriteAdapter returns a ports.SinkAdapter that encodes each item and writes it as a line to w. Use with ports.SinkPort.Bind:
f, _ := os.Create("results.ndjson")
domain.OEEResults.Bind(ctx, file.DrainWriteAdapter(f, format.JSON(oeeCodec),
file.DrainWriteAdapterOptions{Path: "results.ndjson"}))
func DrainWriteFileAdapter ¶
func DrainWriteFileAdapter[T any]( f ports.File[T], varsFor func(T) map[string]string, opts DrainWriteFileAdapterOptions, ) ports.SinkAdapter[T]
DrainWriteFileAdapter returns a ports.SinkAdapter that writes each item as a complete typed file (whole-file overwrite). Use with ports.SinkPort.Bind:
domain.OEEResults.Bind(ctx, file.DrainWriteFileAdapter(resultFile,
func(oee OEE) map[string]string { return map[string]string{"machineID": oee.MachineID} },
file.DrainWriteFileAdapterOptions{}))
varsFor may be nil when f declares merge-capable path params (via ports.NewFilePathParam): vars are then derived PER-ITEM from each item's own merge fields automatically via ports.WriteHandle — the same "one struct, one call" convenience [mqtt5.PublishHandle] provides. Pass a non-nil varsFor to keep building the map yourself (e.g. no merge fields declared, or vars come from a field the file's own type doesn't have).
When the bound ports.SinkPort declares Params, each varsFor result is validated with ports.ValidateParams before the file write; a validation failure is reported to Options.OnError as WriteError wrapping codex.ValidationErrors and the item is otherwise skipped (not written).
func ReadAdapter ¶
func ReadAdapter[In, Resp any]( f ports.File[Resp], varsFor func(In) map[string]string, opts ReadEachAdapterOptions, ) ports.IOAdapter[In, Resp]
ReadAdapter returns a ports.IOAdapter that reads a complete typed file for each In item and emits the file content directly as the response — the 2-type complement of ReadEachAdapter (whose independent file-content type and combine func serve enrichment). Pairs with a ports.FilePattern declared on a ports.IOPort[In, Resp], where the file content IS the port's response type:
calibFile, _ := ports.FileHandle[CalibrationData](domain.Calibration)
domain.Calibration.Bind(ctx, file.ReadAdapter(calibFile,
func(r SensorReading) map[string]string {
return map[string]string{"sensorID": r.SensorID}
},
file.ReadEachAdapterOptions{}))
When the bound ports.IOPort declares Params, each varsFor result is validated with ports.ValidateParams before the file read; a validation failure is delivered as ReadError wrapping codex.ValidationErrors.
Reads via ports.File.ReadMerged (inherited from ReadEachAdapter) — merge-capable path params declared via ports.NewFilePathParam are merged into the returned Resp automatically.
func ReadEachAdapter ¶
func ReadEachAdapter[In, T, Resp any]( f ports.File[T], varsFor func(In) map[string]string, combine func(In, T) Resp, opts ReadEachAdapterOptions, ) ports.IOAdapter[In, Resp]
ReadEachAdapter returns a ports.IOAdapter that reads a complete typed file for each In item, combining the result. Use with ports.IOPort.Bind:
domain.Calibration.Bind(ctx, file.ReadEachAdapter(calibrationFile,
func(r SensorReading) map[string]string { return map[string]string{"id": r.SensorID} },
func(r SensorReading, c CalibrationData) CalibratedReading { return ... },
file.ReadEachAdapterOptions{}))
When the bound ports.IOPort declares Params, each varsFor result is validated with ports.ValidateParams before the file read; a validation failure is delivered as ReadError wrapping codex.ValidationErrors.
The file is read via ports.File.ReadMerged — when f declares merge-capable path params (via ports.NewFilePathParam), the vars derived from varsFor(In) are ADDITIONALLY merged into the decoded T (e.g. a path-derived sensorID is populated onto T automatically). Identical to a bare ports.File.Read when f declares no merge fields.
func ScanAdapter ¶
func ScanAdapter[T any](path string, fmt format.Format[T], opts ScanAdapterOptions) ports.SourceAdapter[T]
ScanAdapter returns a ports.SourceAdapter that reads a file line-by-line, decoding each line. When the file is fully read the adapter exits. Use with ports.SourcePort.Bind:
domain.Readings.Bind(ctx, file.ScanAdapter("readings.ndjson", format.JSON(readingCodec),
file.ScanAdapterOptions{}))
func WatchAdapter ¶
func WatchAdapter(dir string, interval time.Duration, opts WatchAdapterOptions) ports.SourceAdapter[string]
WatchAdapter returns a ports.SourceAdapter that emits file paths for new files created in dir. Runs until ctx is cancelled. Use with ports.SourcePort.Bind:
domain.NewFiles.Bind(ctx, file.WatchAdapter("/data/incoming", 5*time.Second,
file.WatchAdapterOptions{}))
Types ¶
type DrainPatchAdapterOptions ¶ added in v0.12.0
type DrainPatchAdapterOptions struct {
Observer stats.Observer
FileOptions ports.FileOptions
OnError func(error)
}
DrainPatchAdapterOptions configures DrainPatchAdapter.
type DrainPatchEncodedAdapterOptions ¶ added in v0.12.0
type DrainPatchEncodedAdapterOptions struct {
Observer stats.Observer
FileOptions ports.FileOptions
OnError func(error)
}
DrainPatchEncodedAdapterOptions configures DrainPatchEncodedAdapter.
type DrainWriteAdapterOptions ¶
type DrainWriteAdapterOptions struct {
Path string
Separator string
Observer stats.Observer
OnError func(error)
}
DrainWriteAdapterOptions configures DrainWriteAdapter.
type DrainWriteFileAdapterOptions ¶
type DrainWriteFileAdapterOptions struct {
Observer stats.Observer
FileOptions ports.FileOptions
OnError func(error)
}
DrainWriteFileAdapterOptions configures DrainWriteFileAdapter.
type ReadEachAdapterOptions ¶
type ReadEachAdapterOptions struct {
Observer stats.Observer
FileOptions ports.FileOptions
Buffer int
}
ReadEachAdapterOptions configures ReadEachAdapter.
type ReadError ¶
type ReadError struct {
// Err is the underlying error from [ports.File.Read].
Err error
}
ReadError is sent to [Stream.Errors] by [ReadEachStream] when reading or decoding a file fails for an upstream stream item. It wraps the underlying error (typically ports.FileReadError or ports.FileDecodeError).
func (ReadError) LogValue ¶
LogValue implements slog.LogValuer for structured logging.
type ScanAdapterOptions ¶
ScanAdapterOptions configures ScanAdapter.
type ScanError ¶
type ScanError struct {
// Path is the file path passed to ScanStream.
Path string
// Err is the underlying I/O or decode error.
Err error
}
ScanError is sent to [Stream.Errors] by [ScanStream] when opening or reading the file fails. When Err wraps gstream.StreamDecodeError, the failure was a codec decode error on a specific line; otherwise it is an I/O error.
func (ScanError) LogValue ¶
LogValue implements slog.LogValuer for structured logging.
type WatchAdapterOptions ¶
WatchAdapterOptions configures WatchAdapter.
type WatchError ¶
type WatchError struct {
// Dir is the directory being watched.
Dir string
// Err is the underlying os.ReadDir error.
Err error
}
WatchError is sent to [Stream.Errors] by [WatchStream] when a directory read fails during a poll cycle. The stream continues on the next poll interval.
func (WatchError) Error ¶
func (e WatchError) Error() string
func (WatchError) LogValue ¶
func (e WatchError) LogValue() slog.Value
LogValue implements slog.LogValuer for structured logging.
type WriteError ¶
type WriteError struct {
// Path is the file path, when known. Empty when writing to a non-file writer.
Path string
// Err is the underlying encode or write error.
Err error
}
WriteError is passed to [DrainWriteOptions.OnError] by [DrainWrite] when encoding or writing an item to the writer fails.
func (WriteError) Error ¶
func (e WriteError) Error() string
func (WriteError) LogValue ¶
func (e WriteError) LogValue() slog.Value
LogValue implements slog.LogValuer for structured logging.