Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewFile ¶
NewFile creates a sink that writes to a single file. The fs parameter should be a CreateFS filesystem (e.g., from lfs.New or stream.NewFS). Path is passed as-is to fsys.Create() - the client is responsible for proper formatting.
func NewStorage ¶ added in v0.1.17
NewStorage creates a sink that writes to a filesystem directory. The fs parameter should be a CreateFS filesystem (e.g., from lfs.New or stream.NewFS). Document paths are passed as-is to fsys.Create() - the client is responsible for proper formatting.
Types ¶
type File ¶
type File struct {
// contains filtered or unexported fields
}
File writes documents to a single file. All documents are concatenated into one output file with newline separators. The filesystem is provided by the client (e.g., lfs, stream.NewFS).
type FileSystem ¶ added in v0.2.0
type FileSystem struct {
// contains filtered or unexported fields
}
FileSystem writes documents to a filesystem directory. It preserves the document path structure. The filesystem is provided by the client (e.g., lfs, stream.NewFS).
func (*FileSystem) Close ¶ added in v0.2.0
func (s *FileSystem) Close() error
Close implements iosystem.Sink.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer wraps an io.Writer as a Sink. This is useful for integrating with spool.ForEach or other scenarios where you have an io.Writer and want to use it with Pipeline.
Example with spool:
spool.ForEach(ctx, root, func(ctx, path, r, w) error {
source := source.NewReaderSource(path, r)
sink := sink.NewWriterSink(w)
return pipeline.Run(ctx, source, sink)
})
Source Files