README
¶
gio
- gio: Type safe generic
iointerfaces likegio.Reader[int], type safe generic in-memory pipegio.Pipe[string]. Forked from Go stdlib. - gio/pipe: Type safe generic pipeline with a standard input output streams and filters. Enable writing unix-like utilities working on top of native Go types.
- gio/unix: byte stream aware pipeline with a standard input output streams and filters. Works like traditional unix tools.
Example
An equivalent of cat | wc -l using a native Go types and channels under the hood.
out := &StringBuffer{}
stdio := NewStdio[string](
nil,
out,
os.Stderr,
)
err := pipe.NewLine[string]().Run(ctx, stdio, cat, wc)
if err != nil {
log.Fatal(err)
}
fmt.Println(out.String())
// Output: 3
unix.Filter interface
gio/unix Filter and StandardIO interfaces are a basic abstractions for a command to be "plugable" into any compatible endpoint. The https://codeberg.org/gonix/sh integrates all commands with a full shell interpreter mvdan.cc/sh/v3. The definition is quite straightforward.
type StandardIO interface {
Stdin() io.Reader
Stdout() io.Writer
Stderr() io.Writer
}
type Filter interface {
Run(context.Context, StandardIO) error
}
unix.Cmd os/exec wrapper
gio/unix has a *exec.Cmd wrapper allowing any system command to act as a unix.Filter
stdout := bytes.NewBuffer(nil)
stdio := unix.NewStdio(
nil,
stdout,
os.Stderr,
)
cmd := unix.NewCmd(exec.Command("go", "version"))
err := cmd.Run(ctx, stdio)
fmt.Println(out.String())
// Output: go version 1.20 linux/amd64
Shell context
Typical Unix tools needs more input that standard input/output. Shells typically
maintain own context, which must be passed to the tools. unix module defines
ShellContextOption which can be used by the calling shell to pass helper
functions to obtain needed data.
Shell wrappers must implement proper helpers and inject them to builders as a
part of filter builder pattern. This is done to break the dependency between
gio and a shell implementation.
Documentation
¶
Overview ¶
Package gio (generic IO) is a forked variant of the Go standard library io package. It is extended to support operations on slices of any type, denoted as []T.
Generic-aware Reader and Writer interfaces allow for defining tools that work on types other than []byte. While the lack of generic methods in Go (see https://go.googlesource.com/proposal/+/master/design/43651-type-parameters.md and https://go-review.googlesource.com/c/go/+/772441) limits some abstractions, gio enables type-safe implementations for any data stream in the future.
Context-aware helpers ContextReader and ContextWriter wrap context.Context, enabling underlying code to handle signal-like behavior via context cancellation and SIGSTOP/SIGCONT emulation through a wait callback. The actual signal definition or transport mechanism is outside the scope of this package.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Pipe ¶
func Pipe[T any]() (*PipeReader[T], *PipeWriter[T])
Pipe creates a synchronous in-memory pipe. It can be used to connect code expecting an io.Reader with code expecting an io.Writer.
Reads and Writes on the pipe are matched one to one except when multiple Reads are needed to consume a single Write. That is, each Write to the PipeWriter blocks until it has satisfied one or more Reads from the PipeReader that fully consume the written data. The data is copied directly from the Write to the corresponding Read (or Reads); there is no internal buffering.
It is safe to call Read and Write in parallel with each other or with Close. Parallel calls to Read and parallel calls to Write are also safe: the individual calls will be gated sequentially.
Types ¶
type Closer ¶
type Closer interface {
Close() error
}
Closer is the interface that wraps the basic Close method.
The behavior of Close after the first call is undefined. Specific implementations may document their own behavior.
type ContextReader ¶ added in v0.2.0
type ContextReader[T any] struct { // contains filtered or unexported fields }
ContextReader wraps a Reader with context support and an optional wait function which simulates SIGSTOP/SIGCONT unix behavior. The wait function must be bound to the same context and it must stop once context is canceled. Otherwise the deadlock will occur.
func NewContextReader ¶ added in v0.2.0
func NewContextReader[T any](ctx context.Context, r Reader[T], wait func()) ContextReader[T]
NewContextReader returns a ContextReader that checks ctx before every Read. If wait is non-nil, it is called before checking the context.
func (ContextReader[T]) Read ¶ added in v0.2.0
func (r ContextReader[T]) Read(p []T) (n int, err error)
Read implements Reader[T any] interface
type ContextWriter ¶ added in v0.2.0
type ContextWriter[T any] struct { // contains filtered or unexported fields }
ContextWriter wraps a Writer with context support and an optional wait function. The wait function must be bound to the same context and it must stop once context is canceled. Otherwise the deadlock will occur.
func NewContextWriter ¶ added in v0.2.0
func NewContextWriter[T any](ctx context.Context, w Writer[T], wait func()) ContextWriter[T]
NewContextWriter returns a ContextWriter that checks ctx before every Write. If wait is non-nil, it is called before checking the context.
func (ContextWriter[T]) Write ¶ added in v0.2.0
func (w ContextWriter[T]) Write(p []T) (n int, err error)
Write implements the Writer interface. It calls wait if defined, checks for context cancellation, and then writes to the underlying writer.
type PipeReader ¶
type PipeReader[T any] struct { // contains filtered or unexported fields }
A PipeReader is the read half of a pipe.
func (*PipeReader[T]) Close ¶
func (r *PipeReader[T]) Close() error
Close closes the reader; subsequent writes to the write half of the pipe will return the error ErrClosedPipe.
func (*PipeReader[T]) CloseWithError ¶
func (r *PipeReader[T]) CloseWithError(err error) error
CloseWithError closes the reader; subsequent writes to the write half of the pipe will return the error err.
CloseWithError never overwrites the previous error if it exists and always returns nil.
func (*PipeReader[T]) Read ¶
func (r *PipeReader[T]) Read(data []T) (n int, err error)
Read implements the standard Read interface: it reads data from the pipe, blocking until a writer arrives or the write end is closed. If the write end is closed with an error, that error is returned as err; otherwise err is EOF.
type PipeWriter ¶
type PipeWriter[T any] struct { // contains filtered or unexported fields }
A PipeWriter is the write half of a pipe.
func (*PipeWriter[T]) Close ¶
func (w *PipeWriter[T]) Close() error
Close closes the writer; subsequent reads from the read half of the pipe will return no bytes and EOF.
func (*PipeWriter[T]) CloseWithError ¶
func (w *PipeWriter[T]) CloseWithError(err error) error
CloseWithError closes the writer; subsequent reads from the read half of the pipe will return no bytes and the error err, or EOF if err is nil.
CloseWithError never overwrites the previous error if it exists and always returns nil.
func (*PipeWriter[T]) Write ¶
func (w *PipeWriter[T]) Write(data []T) (n int, err error)
Write implements the standard Write interface: it writes data to the pipe, blocking until one or more readers have consumed all the data or the read end is closed. If the read end is closed with an error, that err is returned as err; otherwise err is ErrClosedPipe.
type ReadCloser ¶
ReadCloser is the interface that groups the basic Read and Close methods.
type Reader ¶
Reader is the interface that wraps the basic Read method.
Read reads up to len(p) types into p. It returns the number of types read (0 <= n <= len(p)) and any error encountered. Even if Read returns n < len(p), it may use all of p as scratch space during the call. If some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more.
When Read encounters an error or end-of-file condition after successfully reading n > 0 types, it returns the number of bytes read. It may return the (non-nil) error from the same call or return the error (and n == 0) from a subsequent call. An instance of this general case is that a Reader returning a non-zero number of types at the end of the input stream may return either err == EOF or err == nil. The next Read should return 0, EOF.
Callers should always process the n > 0 bytes returned before considering the error err. Doing so correctly handles I/O errors that happen after reading some types and also both of the allowed EOF behaviors.
Implementations of Read are discouraged from returning a zero byte count with a nil error, except when len(p) == 0. Callers should treat a return of 0 and nil as indicating that nothing happened; in particular it does not indicate EOF.
Implementations must not retain p.
type WriteCloser ¶
WriteCloser is the interface that groups the basic Write and Close methods.
type Writer ¶
Writer is the interface that wraps the basic Write method.
Write writes len(p) types from p to the underlying data stream. It returns the number of types written from p (0 <= n <= len(p)) and any error encountered that caused the write to stop early. Write must return a non-nil error if it returns n < len(p). Write must not modify the slice data, even temporarily.
Implementations must not retain p.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
package pipe implements type-safe unix like resources standard streams and pipelines This can be used to write unix like tools communicating through Go native types which can be connected together like unix allows.
|
package pipe implements type-safe unix like resources standard streams and pipelines This can be used to write unix like tools communicating through Go native types which can be connected together like unix allows. |
|
Package unix implements unix like resources like standard io and provides an universal interface to execute the filters.
|
Package unix implements unix like resources like standard io and provides an universal interface to execute the filters. |