Documentation
¶
Overview ¶
Package files provides ergonomic helpers for reading text files: iterating line by line or in fixed-size chunks, streaming chunks asynchronously off large files, slicing a window of lines, and decoding a structured file into a typed value via the encoding package.
Index ¶
- Variables
- func AllChunks(r io.Reader, n int) ([][]string, error)
- func AllLines(r io.Reader) ([]string, error)
- func Chunks(r io.Reader, n int) iter.Seq2[[]string, error]
- func ChunksFile(name string, n int) (iter.Seq2[[]string, error], error)
- func Decode[T any](ctx context.Context, r io.Reader, ct encoding.ContentType) (T, error)
- func DecodeFile[T any](ctx context.Context, name string, ct encoding.ContentType) (T, error)
- func DecodeFileFS[T any](ctx context.Context, fsys fs.FS, name string, ct encoding.ContentType) (T, error)
- func Lines(r io.Reader) iter.Seq2[string, error]
- func LinesFile(name string) (iter.Seq2[string, error], error)
- func MustAllLines(r io.Reader) []string
- func MustChunksFile(name string, n int) iter.Seq2[[]string, error]
- func MustDecode[T any](ctx context.Context, r io.Reader, ct encoding.ContentType) T
- func MustDecodeFile[T any](ctx context.Context, name string, ct encoding.ContentType) T
- func MustDecodeFileFS[T any](ctx context.Context, fsys fs.FS, name string, ct encoding.ContentType) T
- func MustLinesFile(name string) iter.Seq2[string, error]
- func MustSliceLines(r io.Reader, offset, count int) []string
- func SliceLines(r io.Reader, offset, count int) ([]string, error)
- func SliceLinesFile(ctx context.Context, name string, offset, count int) ([]string, error)
- func StreamChunks(ctx context.Context, src io.Reader, n int) (<-chan ChunkResult, error)
- func StreamChunksFile(ctx context.Context, name string, n int) (<-chan ChunkResult, error)
- type ChunkResult
- type Dir
- func (d *Dir) Chdir(rel string) error
- func (d *Dir) Chunks(name string, n int) (iter.Seq2[[]string, error], error)
- func (d *Dir) Lines(name string) (iter.Seq2[string, error], error)
- func (d *Dir) Path() string
- func (d *Dir) Resolve(name string) string
- func (d *Dir) SliceLines(ctx context.Context, name string, offset, count int) ([]string, error)
- func (d *Dir) StreamChunks(ctx context.Context, name string, n int) (<-chan ChunkResult, error)
- func (d *Dir) Sub(rel string) (*Dir, error)
- type FS
- func (d *FS) Chunks(name string, n int) (iter.Seq2[[]string, error], error)
- func (d *FS) FS() fs.FS
- func (d *FS) Lines(name string) (iter.Seq2[string, error], error)
- func (d *FS) SliceLines(ctx context.Context, name string, offset, count int) ([]string, error)
- func (d *FS) StreamChunks(ctx context.Context, name string, n int) (<-chan ChunkResult, error)
- func (d *FS) Sub(dir string) (*FS, error)
- type Reader
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrOffsetBeyondEOF is returned when a slice offset lands at or past the end of the input. ErrOffsetBeyondEOF = errors.New("offset is at or beyond end of input") // ErrNonPositiveChunkSize is returned when a chunk size is zero or negative. ErrNonPositiveChunkSize = errors.New("chunk size must be greater than zero") // ErrNegativeOffset is returned when a slice offset is negative. ErrNegativeOffset = errors.New("offset must not be negative") // ErrNegativeCount is returned when a slice count is negative. ErrNegativeCount = errors.New("count must not be negative") )
Functions ¶
func AllChunks ¶
AllChunks materializes every chunk of up to n lines. Like AllLines, it is for inputs small enough to hold in memory.
func AllLines ¶
AllLines materializes every line of r. It is a convenience for inputs small enough to hold in memory; prefer Lines for large files. An empty input yields a non-nil, empty slice.
func Chunks ¶
Chunks yields successive slices of up to n lines; the final chunk may hold fewer than n. n must be greater than zero, otherwise the iterator yields ErrNonPositiveChunkSize once and stops. A read error is surfaced the same way Lines surfaces it, discarding any in-progress partial chunk.
Example ¶
package main
import (
"fmt"
"strings"
"github.com/primandproper/platform-go/v5/files"
)
func main() {
for chunk, err := range files.Chunks(strings.NewReader("a\nb\nc\n"), 2) {
if err != nil {
panic(err)
}
fmt.Println(chunk)
}
}
Output: [a b] [c]
func ChunksFile ¶
ChunksFile opens name and yields successive slices of up to n lines.
func Decode ¶
Decode reads all of r and unmarshals it into a T as content type ct — any encoding the encoding package supports (JSON, XML, TOML, YAML, Emoji). It builds a one-off encoder internally.
Example ¶
package main
import (
"context"
"fmt"
"strings"
"github.com/primandproper/platform-go/v5/encoding"
"github.com/primandproper/platform-go/v5/files"
)
func main() {
type config struct {
Name string `json:"name"`
}
cfg, err := files.Decode[config](context.Background(), strings.NewReader(`{"name":"platform"}`), encoding.ContentTypeJSON)
if err != nil {
panic(err)
}
fmt.Println(cfg.Name)
}
Output: platform
func DecodeFile ¶
DecodeFile opens name, reads it, and unmarshals it into a T as content type ct. The read is traced via the default Reader, and the encoder traces under the same tracer.
func DecodeFileFS ¶
func DecodeFileFS[T any](ctx context.Context, fsys fs.FS, name string, ct encoding.ContentType) (T, error)
DecodeFileFS opens name within fsys, reads it, and unmarshals it into a T as content type ct. It is DecodeFile over an arbitrary fs.FS (embed.FS, fstest.MapFS, os.DirFS, …); name must satisfy fs.ValidPath. The read is traced via the default Reader's logger and tracer, and the encoder traces under the same tracer.
func Lines ¶
Lines yields each line of r without its trailing newline (handling both \n and \r\n). An unterminated final line is still yielded. A terminal read error is yielded once, as the second value after the last good line; io.EOF is normal completion and is never surfaced.
Lines reads with a bufio.Reader rather than a bufio.Scanner, so there is no 64KB line-length cap.
Example ¶
package main
import (
"fmt"
"strings"
"github.com/primandproper/platform-go/v5/files"
)
func main() {
for line, err := range files.Lines(strings.NewReader("alpha\nbeta\n")) {
if err != nil {
panic(err)
}
fmt.Println(line)
}
}
Output: alpha beta
func LinesFile ¶
LinesFile opens name and yields each of its lines. The open error is returned up front; read errors are yielded by the iterator. The file is closed when iteration ends or the caller breaks.
func MustAllLines ¶
MustAllLines is like AllLines but panics on error.
func MustChunksFile ¶
MustChunksFile is like ChunksFile but panics on an open error.
func MustDecode ¶
MustDecode is like Decode but panics on error.
func MustDecodeFile ¶
MustDecodeFile is like DecodeFile but panics on error.
func MustDecodeFileFS ¶
func MustDecodeFileFS[T any](ctx context.Context, fsys fs.FS, name string, ct encoding.ContentType) T
MustDecodeFileFS is like DecodeFileFS but panics on error.
func MustLinesFile ¶
MustLinesFile is like LinesFile but panics on an open error.
func MustSliceLines ¶
MustSliceLines is like SliceLines but panics on error.
func SliceLines ¶
SliceLines returns up to count lines after skipping offset lines: "the 10 lines after the first 8" is SliceLines(r, 8, 10). It reads no further than it needs to. If offset lands at or past the end of the input, it returns ErrOffsetBeyondEOF; if fewer than count lines remain, it returns the shorter slice with a nil error.
Example ¶
package main
import (
"fmt"
"strings"
"github.com/primandproper/platform-go/v5/files"
)
func main() {
lines, err := files.SliceLines(strings.NewReader("0\n1\n2\n3\n4\n"), 1, 2)
if err != nil {
panic(err)
}
fmt.Println(lines)
}
Output: [1 2]
func SliceLinesFile ¶
SliceLinesFile opens name and returns up to count lines after skipping offset lines.
func StreamChunks ¶
StreamChunks reads src in a background goroutine and sends each chunk of up to n lines on the returned channel, which StreamChunks owns and closes. A synchronous error is returned for setup failures (n <= 0), in which case no goroutine is started and the channel is nil. Once streaming, a mid-stream read error or context cancellation is delivered as the final ChunkResult.Err and the channel is then closed; io.EOF is clean completion (the channel simply closes).
The caller MUST either drain the channel to completion or cancel ctx to stop early. Abandoning the channel without canceling ctx blocks the background goroutine forever on its next send, leaking the goroutine, its span, and (for StreamChunksFile) the open file handle — cancellation is the only way to unblock a stalled send.
func StreamChunksFile ¶
StreamChunksFile opens name and streams chunks of up to n lines on the returned channel.
Types ¶
type ChunkResult ¶
ChunkResult is one item streamed by StreamChunks: either a chunk of up to n lines, or a terminal Err. A non-nil Err is always the last value sent before the channel closes.
type Dir ¶
type Dir struct {
// contains filtered or unexported fields
}
Dir is a handle rooted at a base directory. Its methods take file names relative to that base, so the leading path is supplied once: OpenDir("a/b") then d.StreamChunks(ctx, "stuff.txt", 100) reads a/b/stuff.txt. Chdir navigates freely, including to sibling and parent directories. A Dir is not safe for concurrent Chdir; its read methods are safe to share once the base is stable.
func NewDir ¶
func NewDir(path string, logger logging.Logger, tracerProvider tracing.TracerProvider) (*Dir, error)
NewDir opens a directory handle at path with the given observability dependencies.
func OpenDir ¶
OpenDir opens a directory handle at path using the default (noop-observability) Reader.
func (*Dir) Chdir ¶
Chdir navigates to rel (resolved against the current base; an absolute rel replaces it), validating it is a directory before adopting it. It mutates the handle.
func (*Dir) Resolve ¶
Resolve joins name onto the base directory. It is the escape hatch for the generic Decode helpers, which cannot be methods: files.DecodeFile[T](ctx, d.Resolve("config.yaml"), encoding.ContentTypeYAML).
func (*Dir) SliceLines ¶
SliceLines opens name (relative to the base) and returns up to count lines after skipping offset.
func (*Dir) StreamChunks ¶
StreamChunks opens name (relative to the base) and streams its chunks of up to n lines.
type FS ¶
type FS struct {
// contains filtered or unexported fields
}
FS is a handle rooted at an fs.FS: the read-only analog of Dir for virtual filesystems such as embed.FS. Its methods take names relative to the root and, because the root is an fs.FS, those names must satisfy fs.ValidPath (slash-separated, unrooted, no "."/".." elements). Sub descends into a subtree via fs.Sub. An FS is safe to share once constructed; Sub returns a new handle rather than mutating this one.
func NewFS ¶
NewFS returns a handle over fsys with the given observability dependencies, mirroring NewDir.
func OpenFS ¶
OpenFS returns a handle over fsys using noop observability, mirroring OpenDir.
Example ¶
ExampleOpenFS reads a name-addressed file from an fs.FS. An embed.FS works identically — pass it to OpenFS (or NewReaderFS) instead of the fstest.MapFS used here for a self-contained example.
package main
import (
"fmt"
"testing/fstest"
"github.com/primandproper/platform-go/v5/files"
)
func main() {
fsys := fstest.MapFS{"greeting.txt": {Data: []byte("hello\nworld\n")}}
lines, err := files.OpenFS(fsys).Lines("greeting.txt")
if err != nil {
panic(err)
}
for line, err := range lines {
if err != nil {
panic(err)
}
fmt.Println(line)
}
}
Output: hello world
func (*FS) FS ¶
FS returns the underlying fs.FS. It is the escape hatch for the generic decode helpers, which cannot be methods: files.DecodeFileFS[T](ctx, d.FS(), "config.yaml", encoding.ContentTypeYAML).
func (*FS) SliceLines ¶
SliceLines opens name (relative to the root) and returns up to count lines after skipping offset.
func (*FS) StreamChunks ¶
StreamChunks opens name (relative to the root) and streams its chunks of up to n lines.
type Reader ¶
type Reader interface {
LinesFile(name string) (iter.Seq2[string, error], error)
ChunksFile(name string, n int) (iter.Seq2[[]string, error], error)
SliceLinesFile(ctx context.Context, name string, offset, count int) ([]string, error)
StreamChunksFile(ctx context.Context, name string, n int) (<-chan ChunkResult, error)
}
Reader reads files by name, with observability around each operation. The line-iterator methods take no context: they are pull-driven, so the consumer's range/break owns cancellation. The methods that read eagerly or stream take a context and open a span.
func NewReader ¶
func NewReader(logger logging.Logger, tracerProvider tracing.TracerProvider) Reader
NewReader builds a Reader that opens files on the OS filesystem, accepting any path os.Open would (including absolute paths and ".."). Use NewReaderFS to read from an embed.FS or other fs.FS.
func NewReaderFS ¶
NewReaderFS builds a Reader that opens files through fsys — an embed.FS, fstest.MapFS, os.DirFS, archive/zip.Reader, or any other fs.FS. Names are interpreted by fsys and so must satisfy fs.ValidPath (slash-separated, unrooted, no "."/".." elements). Every read, slice, stream, and decode behavior is identical to NewReader; only the open is redirected.
Source Files