Documentation
¶
Overview ¶
Package text reads plain text into documents and writes documents to text files.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type FileWriter ¶ added in v0.18.0
type FileWriter struct {
// contains filtered or unexported fields
}
FileWriter calls os.File.Sync before a successful Write returns.
Example ¶
package main
import (
"context"
"fmt"
"os"
"path/filepath"
"github.com/Tangerg/scope/core/document"
"github.com/Tangerg/scope/etl/text"
)
func main() {
directory, err := os.MkdirTemp("", "scope-text-example-")
if err != nil {
panic(err)
}
defer func() {
if cleanupErr := os.RemoveAll(directory); cleanupErr != nil {
panic(cleanupErr)
}
}()
path := filepath.Join(directory, "documents.txt")
writer, err := text.NewFileWriter(text.FileWriterConfig{Path: path})
if err != nil {
panic(err)
}
doc, err := document.NewDocument("Retrieved evidence.", nil)
if err != nil {
panic(err)
}
if writeErr := writer.Write(context.Background(), []*document.Document{doc}); writeErr != nil {
panic(writeErr)
}
contents, err := os.ReadFile(path)
if err != nil {
panic(err)
}
fmt.Print(string(contents))
}
Output: Retrieved evidence.
func NewFileWriter ¶ added in v0.18.0
func NewFileWriter(config FileWriterConfig) (*FileWriter, error)
func (*FileWriter) Write ¶ added in v0.18.0
Write validates and renders every document before touching the destination, so document or formatting failures leave an existing file untouched. A replacing Write also leaves it intact when writing fails: the payload is committed to a sibling file and renamed over the destination. Once writing starts, the payload is committed even if ctx is subsequently canceled. Close errors are joined with earlier I/O failures.
type FileWriterConfig ¶ added in v0.18.0
type FileWriterConfig struct {
// Path is required. Existing files are replaced unless Append is true.
Path string
// DocumentMarkers adds a header naming each document's index within its
// Write call, so appended batches each number from zero.
DocumentMarkers bool
Append bool
// Formatter renders each document. Nil uses [document.TextFormatter],
// which rejects media rather than dropping it from the file.
Formatter document.Formatter
}
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader consumes source once within its configured memory budget. Read admits one attempt on the sequential source. Reopen the input to retry after a failure; concurrent or later attempts return etl.ErrSourceConsumed.
type ReaderConfig ¶
type ReaderConfig struct {
// SourceName identifies the original input; empty means anonymous.
SourceName string
SourceBudget etl.SourceBudget
}
ReaderConfig controls the whole-source memory budget. The zero budget uses etl.DefaultMaxSourceBytes.
Source Files