files

package
v2.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 29, 2026 License: AGPL-3.0 Imports: 17 Imported by: 0

Documentation

Overview

Package files provides ergonomic helpers for reading text files: iterating line by line or in fixed-size chunks, streaming chunks asynchronously off large files, slicing a window of lines, and decoding a structured file into a typed value via the encoding package.

Index

Examples

Constants

This section is empty.

Variables

View Source
var (
	// ErrOffsetBeyondEOF is returned when a slice offset lands at or past the end of the input.
	ErrOffsetBeyondEOF = errors.New("offset is at or beyond end of input")
	// ErrNonPositiveChunkSize is returned when a chunk size is zero or negative.
	ErrNonPositiveChunkSize = errors.New("chunk size must be greater than zero")
	// ErrNegativeOffset is returned when a slice offset is negative.
	ErrNegativeOffset = errors.New("offset must not be negative")
	// ErrNegativeCount is returned when a slice count is negative.
	ErrNegativeCount = errors.New("count must not be negative")
)

Functions

func AllChunks

func AllChunks(r io.Reader, n int) ([][]string, error)

AllChunks materializes every chunk of up to n lines. Like AllLines, it is for inputs small enough to hold in memory.

func AllLines

func AllLines(r io.Reader) ([]string, error)

AllLines materializes every line of r. It is a convenience for inputs small enough to hold in memory; prefer Lines for large files. An empty input yields a non-nil, empty slice.

func Chunks

func Chunks(r io.Reader, n int) iter.Seq2[[]string, error]

Chunks yields successive slices of up to n lines; the final chunk may hold fewer than n. n must be greater than zero, otherwise the iterator yields ErrNonPositiveChunkSize once and stops. A read error is surfaced the same way Lines surfaces it, discarding any in-progress partial chunk.

Example
package main

import (
	"fmt"
	"strings"

	"github.com/primandproper/platform-go/v2/files"
)

func main() {
	for chunk, err := range files.Chunks(strings.NewReader("a\nb\nc\n"), 2) {
		if err != nil {
			panic(err)
		}

		fmt.Println(chunk)
	}
}
Output:
[a b]
[c]

func ChunksFile

func ChunksFile(name string, n int) (iter.Seq2[[]string, error], error)

ChunksFile opens name and yields successive slices of up to n lines.

func Decode

func Decode[T any](ctx context.Context, r io.Reader, ct encoding.ContentType) (T, error)

Decode reads all of r and unmarshals it into a T as content type ct — any encoding the encoding package supports (JSON, XML, TOML, YAML, Emoji). It builds a one-off encoder internally.

Example
package main

import (
	"context"
	"fmt"
	"strings"

	"github.com/primandproper/platform-go/v2/encoding"
	"github.com/primandproper/platform-go/v2/files"
)

func main() {
	type config struct {
		Name string `json:"name"`
	}

	cfg, err := files.Decode[config](context.Background(), strings.NewReader(`{"name":"platform"}`), encoding.ContentTypeJSON)
	if err != nil {
		panic(err)
	}

	fmt.Println(cfg.Name)
}
Output:
platform

func DecodeFile

func DecodeFile[T any](ctx context.Context, name string, ct encoding.ContentType) (T, error)

DecodeFile opens name, reads it, and unmarshals it into a T as content type ct. The read is traced via the default Reader, and the encoder traces under the same tracer.

func Lines

func Lines(r io.Reader) iter.Seq2[string, error]

Lines yields each line of r without its trailing newline (handling both \n and \r\n). An unterminated final line is still yielded. A terminal read error is yielded once, as the second value after the last good line; io.EOF is normal completion and is never surfaced.

Lines reads with a bufio.Reader rather than a bufio.Scanner, so there is no 64KB line-length cap.

Example
package main

import (
	"fmt"
	"strings"

	"github.com/primandproper/platform-go/v2/files"
)

func main() {
	for line, err := range files.Lines(strings.NewReader("alpha\nbeta\n")) {
		if err != nil {
			panic(err)
		}

		fmt.Println(line)
	}
}
Output:
alpha
beta

func LinesFile

func LinesFile(name string) (iter.Seq2[string, error], error)

LinesFile opens name and yields each of its lines. The open error is returned up front; read errors are yielded by the iterator. The file is closed when iteration ends or the caller breaks.

func MustAllLines

func MustAllLines(r io.Reader) []string

MustAllLines is like AllLines but panics on error.

func MustChunksFile

func MustChunksFile(name string, n int) iter.Seq2[[]string, error]

MustChunksFile is like ChunksFile but panics on an open error.

func MustDecode

func MustDecode[T any](ctx context.Context, r io.Reader, ct encoding.ContentType) T

MustDecode is like Decode but panics on error.

func MustDecodeFile

func MustDecodeFile[T any](ctx context.Context, name string, ct encoding.ContentType) T

MustDecodeFile is like DecodeFile but panics on error.

func MustLinesFile

func MustLinesFile(name string) iter.Seq2[string, error]

MustLinesFile is like LinesFile but panics on an open error.

func MustSliceLines

func MustSliceLines(r io.Reader, offset, count int) []string

MustSliceLines is like SliceLines but panics on error.

func SliceLines

func SliceLines(r io.Reader, offset, count int) ([]string, error)

SliceLines returns up to count lines after skipping offset lines: "the 10 lines after the first 8" is SliceLines(r, 8, 10). It reads no further than it needs to. If offset lands at or past the end of the input, it returns ErrOffsetBeyondEOF; if fewer than count lines remain, it returns the shorter slice with a nil error.

Example
package main

import (
	"fmt"
	"strings"

	"github.com/primandproper/platform-go/v2/files"
)

func main() {
	lines, err := files.SliceLines(strings.NewReader("0\n1\n2\n3\n4\n"), 1, 2)
	if err != nil {
		panic(err)
	}

	fmt.Println(lines)
}
Output:
[1 2]

func SliceLinesFile

func SliceLinesFile(ctx context.Context, name string, offset, count int) ([]string, error)

SliceLinesFile opens name and returns up to count lines after skipping offset lines.

func StreamChunks

func StreamChunks(ctx context.Context, src io.Reader, n int) (<-chan ChunkResult, error)

StreamChunks reads src in a background goroutine and sends each chunk of up to n lines on the returned channel, which StreamChunks owns and closes. A synchronous error is returned for setup failures (n <= 0), in which case no goroutine is started and the channel is nil. Once streaming, a mid-stream read error or context cancellation is delivered as the final ChunkResult.Err and the channel is then closed; io.EOF is clean completion (the channel simply closes).

func StreamChunksFile

func StreamChunksFile(ctx context.Context, name string, n int) (<-chan ChunkResult, error)

StreamChunksFile opens name and streams chunks of up to n lines on the returned channel.

Types

type ChunkResult

type ChunkResult struct {
	Err   error
	Lines []string
}

ChunkResult is one item streamed by StreamChunks: either a chunk of up to n lines, or a terminal Err. A non-nil Err is always the last value sent before the channel closes.

type Dir

type Dir struct {
	// contains filtered or unexported fields
}

Dir is a handle rooted at a base directory. Its methods take file names relative to that base, so the leading path is supplied once: OpenDir("a/b") then d.StreamChunks(ctx, "stuff.txt", 100) reads a/b/stuff.txt. Chdir navigates freely, including to sibling and parent directories. A Dir is not safe for concurrent Chdir; its read methods are safe to share once the base is stable.

func NewDir

func NewDir(path string, logger logging.Logger, tracerProvider tracing.TracerProvider) (*Dir, error)

NewDir opens a directory handle at path with the given observability dependencies.

func OpenDir

func OpenDir(path string) (*Dir, error)

OpenDir opens a directory handle at path using the default (noop-observability) Reader.

func (*Dir) Chdir

func (d *Dir) Chdir(rel string) error

Chdir navigates to rel (resolved against the current base; an absolute rel replaces it), validating it is a directory before adopting it. It mutates the handle.

func (*Dir) Chunks

func (d *Dir) Chunks(name string, n int) (iter.Seq2[[]string, error], error)

Chunks opens name (relative to the base) and yields chunks of up to n lines.

func (*Dir) Lines

func (d *Dir) Lines(name string) (iter.Seq2[string, error], error)

Lines opens name (relative to the base) and yields each of its lines.

func (*Dir) Path

func (d *Dir) Path() string

Path returns the current base directory, absolute.

func (*Dir) Resolve

func (d *Dir) Resolve(name string) string

Resolve joins name onto the base directory. It is the escape hatch for the generic Decode helpers, which cannot be methods: files.DecodeFile[T](ctx, d.Resolve("config.yaml"), encoding.ContentTypeYAML).

func (*Dir) SliceLines

func (d *Dir) SliceLines(ctx context.Context, name string, offset, count int) ([]string, error)

SliceLines opens name (relative to the base) and returns up to count lines after skipping offset.

func (*Dir) StreamChunks

func (d *Dir) StreamChunks(ctx context.Context, name string, n int) (<-chan ChunkResult, error)

StreamChunks opens name (relative to the base) and streams its chunks of up to n lines.

func (*Dir) Sub

func (d *Dir) Sub(rel string) (*Dir, error)

Sub is the non-mutating form of Chdir: it returns a new *Dir rooted at rel, sharing this Dir's Reader.

type Reader

type Reader interface {
	LinesFile(name string) (iter.Seq2[string, error], error)
	ChunksFile(name string, n int) (iter.Seq2[[]string, error], error)
	SliceLinesFile(ctx context.Context, name string, offset, count int) ([]string, error)
	StreamChunksFile(ctx context.Context, name string, n int) (<-chan ChunkResult, error)
}

Reader reads files by name, with observability around each operation. The line-iterator methods take no context: they are pull-driven, so the consumer's range/break owns cancellation. The methods that read eagerly or stream take a context and open a span.

func NewReader

func NewReader(logger logging.Logger, tracerProvider tracing.TracerProvider) Reader

NewReader builds a new Reader.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL