Documentation
¶
Overview ¶
Package files provides ergonomic helpers for reading text files: iterating line by line or in fixed-size chunks, streaming chunks asynchronously off large files, slicing a window of lines, and decoding a structured file into a typed value via the encoding package.
Index ¶
- Variables
- func AllChunks(r io.Reader, n int) ([][]string, error)
- func AllLines(r io.Reader) ([]string, error)
- func Chunks(r io.Reader, n int) iter.Seq2[[]string, error]
- func ChunksFile(name string, n int) (iter.Seq2[[]string, error], error)
- func Decode[T any](ctx context.Context, r io.Reader, ct encoding.ContentType) (T, error)
- func DecodeFile[T any](ctx context.Context, name string, ct encoding.ContentType) (T, error)
- func Lines(r io.Reader) iter.Seq2[string, error]
- func LinesFile(name string) (iter.Seq2[string, error], error)
- func MustAllLines(r io.Reader) []string
- func MustChunksFile(name string, n int) iter.Seq2[[]string, error]
- func MustDecode[T any](ctx context.Context, r io.Reader, ct encoding.ContentType) T
- func MustDecodeFile[T any](ctx context.Context, name string, ct encoding.ContentType) T
- func MustLinesFile(name string) iter.Seq2[string, error]
- func MustSliceLines(r io.Reader, offset, count int) []string
- func SliceLines(r io.Reader, offset, count int) ([]string, error)
- func SliceLinesFile(ctx context.Context, name string, offset, count int) ([]string, error)
- func StreamChunks(ctx context.Context, src io.Reader, n int) (<-chan ChunkResult, error)
- func StreamChunksFile(ctx context.Context, name string, n int) (<-chan ChunkResult, error)
- type ChunkResult
- type Dir
- func (d *Dir) Chdir(rel string) error
- func (d *Dir) Chunks(name string, n int) (iter.Seq2[[]string, error], error)
- func (d *Dir) Lines(name string) (iter.Seq2[string, error], error)
- func (d *Dir) Path() string
- func (d *Dir) Resolve(name string) string
- func (d *Dir) SliceLines(ctx context.Context, name string, offset, count int) ([]string, error)
- func (d *Dir) StreamChunks(ctx context.Context, name string, n int) (<-chan ChunkResult, error)
- func (d *Dir) Sub(rel string) (*Dir, error)
- type Reader
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrOffsetBeyondEOF is returned when a slice offset lands at or past the end of the input. ErrOffsetBeyondEOF = errors.New("offset is at or beyond end of input") // ErrNonPositiveChunkSize is returned when a chunk size is zero or negative. ErrNonPositiveChunkSize = errors.New("chunk size must be greater than zero") // ErrNegativeOffset is returned when a slice offset is negative. ErrNegativeOffset = errors.New("offset must not be negative") // ErrNegativeCount is returned when a slice count is negative. ErrNegativeCount = errors.New("count must not be negative") )
Functions ¶
func AllChunks ¶
AllChunks materializes every chunk of up to n lines. Like AllLines, it is for inputs small enough to hold in memory.
func AllLines ¶
AllLines materializes every line of r. It is a convenience for inputs small enough to hold in memory; prefer Lines for large files. An empty input yields a non-nil, empty slice.
func Chunks ¶
Chunks yields successive slices of up to n lines; the final chunk may hold fewer than n. n must be greater than zero, otherwise the iterator yields ErrNonPositiveChunkSize once and stops. A read error is surfaced the same way Lines surfaces it, discarding any in-progress partial chunk.
Example ¶
package main
import (
"fmt"
"strings"
"github.com/primandproper/platform-go/v2/files"
)
func main() {
for chunk, err := range files.Chunks(strings.NewReader("a\nb\nc\n"), 2) {
if err != nil {
panic(err)
}
fmt.Println(chunk)
}
}
Output: [a b] [c]
func ChunksFile ¶
ChunksFile opens name and yields successive slices of up to n lines.
func Decode ¶
Decode reads all of r and unmarshals it into a T as content type ct — any encoding the encoding package supports (JSON, XML, TOML, YAML, Emoji). It builds a one-off encoder internally.
Example ¶
package main
import (
"context"
"fmt"
"strings"
"github.com/primandproper/platform-go/v2/encoding"
"github.com/primandproper/platform-go/v2/files"
)
func main() {
type config struct {
Name string `json:"name"`
}
cfg, err := files.Decode[config](context.Background(), strings.NewReader(`{"name":"platform"}`), encoding.ContentTypeJSON)
if err != nil {
panic(err)
}
fmt.Println(cfg.Name)
}
Output: platform
func DecodeFile ¶
DecodeFile opens name, reads it, and unmarshals it into a T as content type ct. The read is traced via the default Reader, and the encoder traces under the same tracer.
func Lines ¶
Lines yields each line of r without its trailing newline (handling both \n and \r\n). An unterminated final line is still yielded. A terminal read error is yielded once, as the second value after the last good line; io.EOF is normal completion and is never surfaced.
Lines reads with a bufio.Reader rather than a bufio.Scanner, so there is no 64KB line-length cap.
Example ¶
package main
import (
"fmt"
"strings"
"github.com/primandproper/platform-go/v2/files"
)
func main() {
for line, err := range files.Lines(strings.NewReader("alpha\nbeta\n")) {
if err != nil {
panic(err)
}
fmt.Println(line)
}
}
Output: alpha beta
func LinesFile ¶
LinesFile opens name and yields each of its lines. The open error is returned up front; read errors are yielded by the iterator. The file is closed when iteration ends or the caller breaks.
func MustAllLines ¶
MustAllLines is like AllLines but panics on error.
func MustChunksFile ¶
MustChunksFile is like ChunksFile but panics on an open error.
func MustDecode ¶
MustDecode is like Decode but panics on error.
func MustDecodeFile ¶
MustDecodeFile is like DecodeFile but panics on error.
func MustLinesFile ¶
MustLinesFile is like LinesFile but panics on an open error.
func MustSliceLines ¶
MustSliceLines is like SliceLines but panics on error.
func SliceLines ¶
SliceLines returns up to count lines after skipping offset lines: "the 10 lines after the first 8" is SliceLines(r, 8, 10). It reads no further than it needs to. If offset lands at or past the end of the input, it returns ErrOffsetBeyondEOF; if fewer than count lines remain, it returns the shorter slice with a nil error.
Example ¶
package main
import (
"fmt"
"strings"
"github.com/primandproper/platform-go/v2/files"
)
func main() {
lines, err := files.SliceLines(strings.NewReader("0\n1\n2\n3\n4\n"), 1, 2)
if err != nil {
panic(err)
}
fmt.Println(lines)
}
Output: [1 2]
func SliceLinesFile ¶
SliceLinesFile opens name and returns up to count lines after skipping offset lines.
func StreamChunks ¶
StreamChunks reads src in a background goroutine and sends each chunk of up to n lines on the returned channel, which StreamChunks owns and closes. A synchronous error is returned for setup failures (n <= 0), in which case no goroutine is started and the channel is nil. Once streaming, a mid-stream read error or context cancellation is delivered as the final ChunkResult.Err and the channel is then closed; io.EOF is clean completion (the channel simply closes).
func StreamChunksFile ¶
StreamChunksFile opens name and streams chunks of up to n lines on the returned channel.
Types ¶
type ChunkResult ¶
ChunkResult is one item streamed by StreamChunks: either a chunk of up to n lines, or a terminal Err. A non-nil Err is always the last value sent before the channel closes.
type Dir ¶
type Dir struct {
// contains filtered or unexported fields
}
Dir is a handle rooted at a base directory. Its methods take file names relative to that base, so the leading path is supplied once: OpenDir("a/b") then d.StreamChunks(ctx, "stuff.txt", 100) reads a/b/stuff.txt. Chdir navigates freely, including to sibling and parent directories. A Dir is not safe for concurrent Chdir; its read methods are safe to share once the base is stable.
func NewDir ¶
func NewDir(path string, logger logging.Logger, tracerProvider tracing.TracerProvider) (*Dir, error)
NewDir opens a directory handle at path with the given observability dependencies.
func OpenDir ¶
OpenDir opens a directory handle at path using the default (noop-observability) Reader.
func (*Dir) Chdir ¶
Chdir navigates to rel (resolved against the current base; an absolute rel replaces it), validating it is a directory before adopting it. It mutates the handle.
func (*Dir) Resolve ¶
Resolve joins name onto the base directory. It is the escape hatch for the generic Decode helpers, which cannot be methods: files.DecodeFile[T](ctx, d.Resolve("config.yaml"), encoding.ContentTypeYAML).
func (*Dir) SliceLines ¶
SliceLines opens name (relative to the base) and returns up to count lines after skipping offset.
func (*Dir) StreamChunks ¶
StreamChunks opens name (relative to the base) and streams its chunks of up to n lines.
type Reader ¶
type Reader interface {
LinesFile(name string) (iter.Seq2[string, error], error)
ChunksFile(name string, n int) (iter.Seq2[[]string, error], error)
SliceLinesFile(ctx context.Context, name string, offset, count int) ([]string, error)
StreamChunksFile(ctx context.Context, name string, n int) (<-chan ChunkResult, error)
}
Reader reads files by name, with observability around each operation. The line-iterator methods take no context: they are pull-driven, so the consumer's range/break owns cancellation. The methods that read eagerly or stream take a context and open a span.