Documentation
¶
Overview ¶
Package script aims to make it easy to write shell-type scripts in Go, for general system administration purposes: reading files, counting lines, matching strings, and so on.
Index ¶
- type Pipe
- func Args() *Pipe
- func Do(req *http.Request) *Pipe
- func Echo(s string) *Pipe
- func Exec(cmdLine string) *Pipe
- func ExecCommand(name string, args ...string) *Pipe
- func File(path string) *Pipe
- func FindFiles(dir string) *Pipe
- func Get(url string) *Pipe
- func IfExists(path string) *Pipe
- func ListFiles(path string) *Pipe
- func NewPipe() *Pipe
- func Post(url string) *Pipe
- func Shell(cmdLine string) *Pipe
- func Slice(s []string) *Pipe
- func Stdin() *Pipe
- func WithContext(ctx context.Context) *Pipe
- func (p *Pipe) AppendFile(path string) (int64, error)
- func (p *Pipe) Basename() *Pipe
- func (p *Pipe) Bytes() ([]byte, error)
- func (p *Pipe) Close() error
- func (p *Pipe) Column(col int) *Pipe
- func (p *Pipe) Concat() *Pipe
- func (p *Pipe) CountLines() (lines int, err error)
- func (p *Pipe) DecodeBase64() *Pipe
- func (p *Pipe) Dirname() *Pipe
- func (p *Pipe) Do(req *http.Request) *Pipe
- func (p *Pipe) EachLine(process func(string, *strings.Builder)) *Pipedeprecated
- func (p *Pipe) Echo(s string) *Pipe
- func (p *Pipe) EncodeBase64() *Pipe
- func (p *Pipe) Error() error
- func (p *Pipe) Exec(cmdLine string) *Pipe
- func (p *Pipe) ExecCommand(name string, args ...string) *Pipe
- func (p *Pipe) ExecForEach(cmdLine string) *Pipe
- func (p *Pipe) ExitStatus() int
- func (p *Pipe) Filter(filter func(io.Reader, io.Writer) error) *Pipe
- func (p *Pipe) FilterLine(filter func(string) string) *Pipe
- func (p *Pipe) FilterScan(filter func(string, io.Writer)) *Pipe
- func (p *Pipe) First(n int) *Pipe
- func (p *Pipe) Freq() *Pipe
- func (p *Pipe) Get(url string) *Pipe
- func (p *Pipe) Hash(hasher hash.Hash) (string, error)
- func (p *Pipe) HashSums(hasher hash.Hash) *Pipe
- func (p *Pipe) JQ(query string) *Pipe
- func (p *Pipe) Join() *Pipe
- func (p *Pipe) Last(n int) *Pipe
- func (p *Pipe) Match(s string) *Pipe
- func (p *Pipe) MatchRegexp(re *regexp.Regexp) *Pipe
- func (p *Pipe) Post(url string) *Pipe
- func (p *Pipe) Read(b []byte) (int, error)
- func (p *Pipe) Reject(s string) *Pipe
- func (p *Pipe) RejectRegexp(re *regexp.Regexp) *Pipe
- func (p *Pipe) Replace(search, replace string) *Pipe
- func (p *Pipe) ReplaceRegexp(re *regexp.Regexp, replace string) *Pipe
- func (p *Pipe) SHA256Sum() (string, error)deprecated
- func (p *Pipe) SHA256Sums() *Pipedeprecated
- func (p *Pipe) SetError(err error)
- func (p *Pipe) Shell(cmdLine string) *Pipe
- func (p *Pipe) Slice() ([]string, error)
- func (p *Pipe) Stdout() (int, error)
- func (p *Pipe) String() (string, error)
- func (p *Pipe) Tee(writers ...io.Writer) *Pipe
- func (p *Pipe) Unique() *Pipe
- func (p *Pipe) Wait() error
- func (p *Pipe) WithContext(ctx context.Context) *Pipe
- func (p *Pipe) WithEnv(env []string) *Pipe
- func (p *Pipe) WithError(err error) *Pipe
- func (p *Pipe) WithHTTPClient(c *http.Client) *Pipe
- func (p *Pipe) WithReader(r io.Reader) *Pipe
- func (p *Pipe) WithStderr(w io.Writer) *Pipe
- func (p *Pipe) WithStdout(w io.Writer) *Pipe
- func (p *Pipe) WriteFile(path string) (int64, error)
- type ReadAutoCloser
Examples ¶
- Args
- Do
- Echo
- ExecCommand (Exit_status_not_zero)
- ExecCommand (Exit_status_zero)
- ExecCommand (Ok)
- File
- FindFiles
- Get
- IfExists (Exec)
- IfExists (False)
- IfExists (NoExec)
- IfExists (True)
- ListFiles
- Pipe.Basename
- Pipe.Bytes
- Pipe.Column
- Pipe.Concat
- Pipe.CountLines
- Pipe.DecodeBase64
- Pipe.Dirname
- Pipe.Do
- Pipe.EachLine
- Pipe.Echo
- Pipe.EncodeBase64
- Pipe.ExecCommand
- Pipe.ExecForEach
- Pipe.ExitStatus
- Pipe.Filter
- Pipe.FilterLine (Stdlib)
- Pipe.FilterLine (User)
- Pipe.FilterScan
- Pipe.First
- Pipe.Freq
- Pipe.Get
- Pipe.Hash
- Pipe.HashSums
- Pipe.JQ
- Pipe.Join
- Pipe.Last
- Pipe.Match
- Pipe.MatchRegexp
- Pipe.Post
- Pipe.Read
- Pipe.Reject
- Pipe.RejectRegexp
- Pipe.Replace
- Pipe.ReplaceRegexp
- Pipe.SHA256Sum
- Pipe.SHA256Sums
- Pipe.Shell
- Pipe.Slice
- Pipe.Stdout
- Pipe.String
- Pipe.Tee (Stdout)
- Pipe.Tee (Writers)
- Pipe.Unique
- Pipe.WithContext
- Pipe.WithStderr
- Shell
- Slice
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Pipe ¶
type Pipe struct {
// Reader is the underlying reader.
Reader ReadAutoCloser
// contains filtered or unexported fields
}
Pipe represents a pipe object with an associated ReadAutoCloser.
func Args ¶ added in v0.7.0
func Args() *Pipe
Args creates a pipe containing the program's command-line arguments from os.Args, excluding the program name, one per line.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Args().Stdout()
// prints command-line arguments
}
Output:
func Do ¶ added in v0.21.0
Do creates a pipe that makes the HTTP request req and produces the response. See Pipe.Do for how the HTTP response status is interpreted.
Example ¶
package main
import (
"fmt"
"log"
"net/http"
"net/http/httptest"
"github.com/bitfield/script"
)
func main() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "some data")
}))
defer ts.Close()
req, err := http.NewRequest(http.MethodGet, ts.URL, http.NoBody)
if err != nil {
log.Println(err)
return
}
script.Do(req).Stdout()
}
Output: some data
func Echo ¶ added in v0.3.0
Echo creates a pipe containing the string s.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("Hello, world!").Stdout()
}
Output: Hello, world!
func Exec ¶ added in v0.5.0
Exec creates a pipe that runs cmdLine as an external command.
Prefer using Shell or ExecCommand instead unless you specifically need shell.Fields-style parsing behaviour.
See Pipe.Exec for details.
func ExecCommand ¶ added in v0.25.0
ExecCommand creates a pipe that runs name as an external command with the supplied args.
See Pipe.ExecCommand for details.
Example (Exit_status_not_zero) ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
p := script.ExecCommand("false")
p.Wait()
fmt.Println(p.ExitStatus())
}
Output: 1
Example (Exit_status_zero) ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
p := script.ExecCommand("echo")
p.Wait()
fmt.Println(p.ExitStatus())
}
Output: 0
Example (Ok) ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.ExecCommand("echo", "Hello, world!").Stdout()
}
Output: Hello, world!
func File ¶
File creates a pipe that reads from the file path.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.File("testdata/hello.txt").Stdout()
}
Output: hello world
func FindFiles ¶ added in v0.16.0
FindFiles creates a pipe listing all the files in the directory dir and its subdirectories recursively, one per line, like Unix find(1). Errors are ignored unless no files are found (in which case the pipe's error status will be set to the last error encountered).
Each line of the output consists of a slash-separated path, starting with the initial directory. For example, if the directory looks like this:
test/
1.txt
2.txt
the pipe's output will be:
test/1.txt test/2.txt
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.FindFiles("testdata/multiple_files_with_subdirectory").Stdout()
}
Output: testdata/multiple_files_with_subdirectory/1.txt testdata/multiple_files_with_subdirectory/2.txt testdata/multiple_files_with_subdirectory/3.tar.zip testdata/multiple_files_with_subdirectory/dir/.hidden testdata/multiple_files_with_subdirectory/dir/1.txt testdata/multiple_files_with_subdirectory/dir/2.txt
func Get ¶ added in v0.21.0
Get creates a pipe that makes an HTTP GET request to url, and produces the response. See Pipe.Do for how the HTTP response status is interpreted.
Example ¶
package main
import (
"fmt"
"net/http"
"net/http/httptest"
"github.com/bitfield/script"
)
func main() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "some data")
}))
defer ts.Close()
script.Get(ts.URL).Stdout()
}
Output: some data
func IfExists ¶ added in v0.14.0
IfExists tests whether path exists.
If the file doesn't exist, the pipe's error status will be set, and if the file does exist, the pipe will have no error status. This can be used to do some operation only if a given file exists, by chaining Pipe.ExecCommand afterwards:
IfExists("/foo/bar").ExecCommand("/usr/bin/something")
Example (Exec) ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.IfExists("./testdata/hello.txt").ExecCommand("echo", "hello").Stdout()
}
Output: hello
Example (False) ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.IfExists("doesntexist").Echo("found it").Stdout()
}
Output:
Example (NoExec) ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.IfExists("doesntexist").ExecCommand("echo", "hello").Stdout()
}
Output:
Example (True) ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.IfExists("./testdata/hello.txt").Echo("found it").Stdout()
}
Output: found it
func ListFiles ¶ added in v0.11.0
ListFiles creates a pipe containing the files or directories specified by path, one per line. path can be a glob expression, as for filepath.Match. For example:
ListFiles("/data/*").Stdout()
ListFiles does not recurse into subdirectories; use FindFiles instead.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.ListFiles("testdata/multiple_files_with_subdirectory").Stdout()
}
Output: testdata/multiple_files_with_subdirectory/1.txt testdata/multiple_files_with_subdirectory/2.txt testdata/multiple_files_with_subdirectory/3.tar.zip testdata/multiple_files_with_subdirectory/dir
func NewPipe ¶
func NewPipe() *Pipe
NewPipe creates a new pipe with an empty reader (use Pipe.WithReader to attach another reader to it).
func Post ¶ added in v0.21.0
Post creates a pipe that makes an HTTP POST request to url, with an empty body, and produces the response. See Pipe.Do for how the HTTP response status is interpreted.
func Shell ¶ added in v0.25.0
Shell creates a pipe that runs cmdLine as a command via the system shell.
See Pipe.Shell for details.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Shell("echo Hello, world!").Stdout()
}
Output: Hello, world!
func Slice ¶ added in v0.11.0
Slice creates a pipe containing each element of s, one per line. If s is empty or nil, then the pipe is empty.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
input := []string{"1", "2", "3"}
script.Slice(input).Stdout()
}
Output: 1 2 3
func WithContext ¶ added in v0.25.0
WithContext creates a pipe with the context ctx.
func (*Pipe) AppendFile ¶ added in v0.4.0
AppendFile appends the contents of the pipe to the file path, creating it if necessary, and returns the number of bytes successfully written, or an error.
func (*Pipe) Basename ¶ added in v0.14.0
Basename reads paths from the pipe, one per line, and removes any leading directory components from each. So, for example, /usr/local/bin/foo would become just foo. This is the complementary operation to Pipe.Dirname.
If any line is empty, Basename will transform it to a single dot. Trailing slashes are removed. The behaviour of Basename is the same as filepath.Base (not by coincidence).
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
input := []string{
"",
"/",
"/root",
"/tmp/example.php",
"/var/tmp/",
"./src/filters",
"C:/Program Files",
}
script.Slice(input).Basename().Stdout()
}
Output: . / root example.php tmp filters Program Files
func (*Pipe) Bytes ¶ added in v0.8.0
Bytes returns the contents of the pipe as a []byte, or an error.
Example ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
data, err := script.Echo("hello").Bytes()
if err != nil {
panic(err)
}
fmt.Printf("%v\n", data)
}
Output: [104 101 108 108 111]
func (*Pipe) Close ¶
Close closes the pipe's associated reader. This is a no-op if the reader is not an io.Closer.
func (*Pipe) Column ¶ added in v0.9.0
Column produces column col of each line of input, where the first column is column 1, and columns are delimited by Unicode whitespace. Lines containing fewer than col columns will be skipped.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
input := []string{
"PID TT STAT TIME COMMAND",
" 1 ?? Ss 873:17.62 /sbin/launchd",
" 50 ?? Ss 13:18.13 /usr/libexec/UserEventAgent (System)",
" 51 ?? Ss 22:56.75 /usr/sbin/syslogd",
}
script.Slice(input).Column(1).Stdout()
}
Output: PID 1 50 51
func (*Pipe) Concat ¶ added in v0.8.0
Concat reads paths from the pipe, one per line, and produces the contents of all the corresponding files in sequence. If there are any errors (for example, non-existent files), these will be ignored, execution will continue, and the pipe's error status will not be set.
This makes it convenient to write programs that take a list of paths on the command line. For example:
script.Args().Concat().Stdout()
The list of paths could also come from a file:
script.File("filelist.txt").Concat()
Or from the output of a command:
script.ExecCommand("ls /var/app/config/").Concat().Stdout()
Each input file will be closed once it has been fully read. If any of the files can't be opened or read, Concat will simply skip these and carry on, without setting the pipe's error status. This mimics the behaviour of Unix cat(1).
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
input := []string{
"testdata/test.txt",
"testdata/doesntexist.txt",
"testdata/hello.txt",
}
script.Slice(input).Concat().Stdout()
}
Output: This is the first line in the file. Hello, world. This is another line in the file. hello world
func (*Pipe) CountLines ¶
CountLines returns the number of lines of input, or an error.
Example ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
n, err := script.Echo("a\nb\nc\n").CountLines()
if err != nil {
panic(err)
}
fmt.Println(n)
}
Output: 3
func (*Pipe) DecodeBase64 ¶ added in v0.23.0
DecodeBase64 produces the string represented by the base64 encoded input.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("SGVsbG8sIHdvcmxkIQ==").DecodeBase64().Stdout()
}
Output: Hello, world!
func (*Pipe) Dirname ¶ added in v0.14.0
Dirname reads paths from the pipe, one per line, and produces only the parent directories of each path. For example, /usr/local/bin/foo would become just /usr/local/bin. This is the complementary operation to Pipe.Basename.
If a line is empty, Dirname will transform it to a single dot. Trailing slashes are removed, unless Dirname returns the root folder. Otherwise, the behaviour of Dirname is the same as filepath.Dir (not by coincidence).
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
input := []string{
"",
"/",
"/root",
"/tmp/example.php",
"/var/tmp/",
"./src/filters",
"C:/Program Files",
}
script.Slice(input).Dirname().Stdout()
}
Output: . / / /tmp /var ./src C:
func (*Pipe) Do ¶ added in v0.21.0
Do performs the HTTP request req using the pipe's configured HTTP client, as set by Pipe.WithHTTPClient, or http.DefaultClient otherwise. The response body is streamed concurrently to the pipe's output. If the response status is anything other than HTTP 200-299, the pipe's error status is set.
Context ¶
The request does not inherit the pipe's context (if any was set by Pipe.WithContext). If you want the request to be cancellable, you will need to set your own context on it.
Example ¶
package main
import (
"fmt"
"io"
"log"
"net/http"
"net/http/httptest"
"strings"
"github.com/bitfield/script"
)
func main() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
data, err := io.ReadAll(r.Body)
if err != nil {
log.Fatal(err)
}
fmt.Fprintf(w, "You said: %s", data)
}))
defer ts.Close()
req, err := http.NewRequest(http.MethodGet, ts.URL, strings.NewReader("hello"))
if err != nil {
log.Println(err)
return
}
script.NewPipe().Do(req).Stdout()
}
Output: You said: hello
func (*Pipe) EachLine
deprecated
added in
v0.3.0
EachLine calls the function process on each line of input, passing it the line as a string, and a *strings.Builder to write its output to.
Deprecated: use Pipe.FilterLine or Pipe.FilterScan instead, which run concurrently and don't do unnecessary reads on the input.
Example ¶
package main
import (
"fmt"
"strings"
"github.com/bitfield/script"
)
func main() {
script.File("testdata/test.txt").EachLine(func(line string, out *strings.Builder) {
fmt.Fprintln(out, ">", line)
}).Stdout()
}
Output: > This is the first line in the file. > Hello, world. > This is another line in the file.
func (*Pipe) Echo ¶ added in v0.2.0
Echo sets the pipe's reader to one that produces the string s, detaching any existing reader without draining or closing it.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.NewPipe().Echo("Hello, world!").Stdout()
}
Output: Hello, world!
func (*Pipe) EncodeBase64 ¶ added in v0.23.0
EncodeBase64 produces the base64 encoding of the input.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("Hello, world!").EncodeBase64().Stdout()
}
Output: SGVsbG8sIHdvcmxkIQ==
func (*Pipe) Error ¶
Error returns any error present on the pipe, or nil otherwise. Error is not a sink and does not wait until the pipe reaches completion. To wait for completion before returning the error, see Pipe.Wait.
func (*Pipe) Exec ¶ added in v0.5.0
Exec runs cmdLine as an external command.
The command receives the contents of the pipe as its standard input, and the command's combined output (interleaving standard output and standard error) is sent to the pipe.
Argument parsing ¶
cmdLine is split into a program name and arguments using shell.Fields-style parsing. To pass the program name and arguments as separate parameters, use Pipe.ExecCommand instead.
Because shell.Fields handles splitting, quoting, and variable expansion in a way that's slightly inconsistent with shells on various platforms, Exec will at some point be deprecated in favour of Pipe.ExecCommand (if you are constructing the command line from individual arguments) or Pipe.Shell (if the command line is a string).
See Pipe.ExecCommand for details of error handling, context, and environment variables.
func (*Pipe) ExecCommand ¶ added in v0.25.0
ExecCommand runs name as an external command with the supplied args.
The command receives the contents of the pipe as its standard input, and the command's combined output (interleaving standard output and standard error) is sent to the pipe.
Argument parsing ¶
The arguments are not parsed or expanded in any way; name and args are passed directly to the operating system, exactly as supplied, with no shell involved. This is the same API as Go's standard os/exec.Command. See also Pipe.Shell which passes the command line through the system shell for variable expansion, etc.
Environment ¶
The command inherits the current process's environment, or the pipe's environment if one was previously set using Pipe.WithEnv.
Context ¶
The command inherits the pipe's context (if any was set by Pipe.WithContext), and will be cancelled if the context is cancelled or times out.
Error handling ¶
If the command had a non-zero exit status, the pipe's error status will also be set to the string "exit status X", where X is the integer exit status. Even in the event of a non-zero exit status, the command's output will still be available in the pipe. This is often helpful for debugging. However, because Pipe.String is a no-op if the pipe's error status is set, if you want output you will need to reset the error status before calling Pipe.String.
If the command writes to its standard error stream, this will also go to the pipe, along with its standard output. However, the standard error text can instead be redirected to a supplied writer, using Pipe.WithStderr.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("Hello, world!").ExecCommand("tr", "a-z", "A-Z").Stdout()
}
Output: HELLO, WORLD!
func (*Pipe) ExecForEach ¶ added in v0.15.0
ExecForEach runs cmdLine for each line of input.
cmdLine is rendered as a Go template for each line of input, running the resulting command, and produces the combined output of all these commands in sequence. See Pipe.ExecCommand for details on error handling and environment variables.
This is mostly useful for substituting data into commands using Go template syntax. For example:
ListFiles("*").ExecForEach("touch {{.}}").Wait()
Environment ¶
The command inherits the current process's environment, optionally modified by Pipe.WithEnv.
Context ¶
The command inherits the pipe's context (if any was set by Pipe.WithContext), and will be cancelled if the context is cancelled or times out.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("a\nb\nc\n").ExecForEach("echo {{.}}").Stdout()
}
Output: a b c
func (*Pipe) ExitStatus ¶ added in v0.5.0
ExitStatus returns the integer exit status of a previous command (for example run by Pipe.ExecCommand). This will be zero unless the pipe's error status is set and the error matches the pattern “exit status %d”.
Example ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
p := script.ExecCommand("echo")
fmt.Println(p.ExitStatus())
}
Output: 0
func (*Pipe) Filter ¶ added in v0.19.0
Filter sends the contents of the pipe to the function filter and produces the result. filter takes an io.Reader to read its input from and an io.Writer to write its output to, and returns an error, which will be set on the pipe.
filter runs concurrently, so its goroutine will not exit until the pipe has been fully read. Use Pipe.Wait to wait for all concurrent filters to complete. These goroutines are not automatically cancelled by the pipe's context, if one was set by Pipe.WithContext.
Example ¶
package main
import (
"fmt"
"io"
"github.com/bitfield/script"
)
func main() {
script.Echo("hello world").Filter(func(r io.Reader, w io.Writer) error {
n, err := io.Copy(w, r)
fmt.Fprintf(w, "\nfiltered %d bytes\n", n)
return err
}).Stdout()
}
Output: hello world filtered 11 bytes
func (*Pipe) FilterLine ¶ added in v0.19.0
FilterLine sends the contents of the pipe to the function filter, a line at a time, and produces the result. filter takes each line as a string and returns a string as its output. See Pipe.Filter for concurrency handling.
Example (Stdlib) ¶
package main
import (
"strings"
"github.com/bitfield/script"
)
func main() {
script.Echo("a\nb\nc").FilterLine(strings.ToUpper).Stdout()
}
Output: A B C
Example (User) ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("a\nb\nc").FilterLine(func(line string) string {
return "> " + line
}).Stdout()
}
Output: > a > b > c
func (*Pipe) FilterScan ¶ added in v0.19.0
FilterScan sends the contents of the pipe to the function filter, a line at a time, and produces the result. filter takes each line as a string and an io.Writer to write its output to. See Pipe.Filter for concurrency handling.
Example ¶
package main
import (
"fmt"
"io"
"github.com/bitfield/script"
)
func main() {
script.Echo("a\nb\nc").FilterScan(func(line string, w io.Writer) {
fmt.Fprintf(w, "scanned line: %q\n", line)
}).Stdout()
}
Output: scanned line: "a" scanned line: "b" scanned line: "c"
func (*Pipe) First ¶ added in v0.9.0
First produces only the first n lines of the pipe's contents, or all the lines if there are less than n. If n is zero or negative, there is no output at all. When n lines have been produced, First stops reading its input and sends EOF to its output.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("a\nb\nc\n").First(2).Stdout()
}
Output: a b
func (*Pipe) Freq ¶ added in v0.9.0
Freq produces a freqency count of unique lines.
Freq waits for the pipe to complete and then produces only the unique lines from the pipe's contents, each prefixed with a frequency count, in descending numerical order (most frequent lines first). Lines with equal frequency will be sorted alphabetically.
For example, we could take a common shell pipeline like this:
sort input.txt |uniq -c |sort -rn
and replace it with:
File("input.txt").Freq().Stdout()
Or to get only the ten most common lines:
File("input.txt").Freq().First(10).Stdout()
Like Unix uniq(1), Freq right-justifies its count values in a column for readability, padding with spaces if necessary.
See also Pipe.Unique, which is concurrent, does not sort its input, and suppresses only consecutive duplicate lines, without producing a frequency count.
Example ¶
package main
import (
"strings"
"github.com/bitfield/script"
)
func main() {
input := strings.Join([]string{
"apple",
"orange",
"banana",
"banana",
"apple",
"orange",
"kumquat",
"apple",
"orange",
"apple",
"banana",
"banana",
"apple",
"apple",
"orange",
"apple",
"apple",
"apple",
"apple",
}, "\n")
script.Echo(input).Freq().Stdout()
}
Output: 10 apple 4 banana 4 orange 1 kumquat
func (*Pipe) Get ¶ added in v0.21.0
Get makes an HTTP GET request to url, sending the contents of the pipe as the request body, and produces the server's response. See Pipe.Do for how the HTTP response status is interpreted.
Context ¶
The request inherits the pipe's context (if any was set by Pipe.WithContext), and will be cancelled if the context is cancelled or times out.
Example ¶
package main
import (
"fmt"
"io"
"log"
"net/http"
"net/http/httptest"
"github.com/bitfield/script"
)
func main() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
data, err := io.ReadAll(r.Body)
if err != nil {
log.Fatal(err)
}
fmt.Fprintf(w, "You said: %s", data)
}))
defer ts.Close()
script.Echo("hello").Get(ts.URL).Stdout()
}
Output: You said: hello
func (*Pipe) Hash ¶ added in v0.24.0
Hash returns the hex-encoded hash of the entire contents of the pipe based on the provided hasher, or an error. To perform hashing on files, see Pipe.HashSums.
Example ¶
package main
import (
"crypto/sha512"
"fmt"
"github.com/bitfield/script"
)
func main() {
sum, err := script.Echo("hello world").Hash(sha512.New())
if err != nil {
panic(err)
}
fmt.Println(sum)
}
Output: 309ecc489c12d6eb4cc40f50c902f2b4d0ed77ee511a7c7a9bcd3ca86d4cd86f989dd35bc5ff499670da34255b45b0cfd830e81f605dcf7dc5542e93ae9cd76f
func (*Pipe) HashSums ¶ added in v0.24.0
HashSums reads paths from the pipe, one per line, and produces the hex-encoded hash of each corresponding file based on the provided hasher, one per line. Any files that cannot be opened or read will be ignored. To perform hashing on the contents of the pipe, see Pipe.Hash.
Example ¶
package main
import (
"crypto/sha256"
"github.com/bitfield/script"
)
func main() {
script.ListFiles("testdata/multiple_files").HashSums(sha256.New()).Stdout()
}
Output: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
func (*Pipe) JQ ¶ added in v0.20.0
JQ executes query on the pipe's contents (presumed to be valid JSON or JSONLines data), applying the query to each newline-delimited input value and producing results until the first error is encountered. An invalid query or value will set the appropriate error on the pipe.
The exact dialect of JQ supported is that provided by github.com/itchyny/gojq, whose documentation explains the differences between it and standard JQ.
Example ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
kernel := "Darwin"
arch := "x86_64"
query := fmt.Sprintf(".assets[] | select(.name | endswith(\"%s_%s.tar.gz\")).browser_download_url", kernel, arch)
script.File("testdata/releases.json").JQ(query).Stdout()
}
Output: "https://github.com/mbarley333/blackjack/releases/download/v0.3.3/blackjack_0.3.3_Darwin_x86_64.tar.gz"
func (*Pipe) Join ¶ added in v0.7.0
Join joins all the lines in the pipe's contents into a single space-separated string, which will always end with a newline.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("hello\nworld\n").Join().Stdout()
}
Output: hello world
func (*Pipe) Last ¶ added in v0.12.0
Last produces only the last n lines of the pipe's contents, or all the lines if there are less than n. If n is zero or negative, there is no output at all.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("a\nb\nc\n").Last(2).Stdout()
}
Output: b c
func (*Pipe) Match ¶ added in v0.2.0
Match produces only the input lines that contain the string s.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("a\nb\nc\n").Match("b").Stdout()
}
Output: b
func (*Pipe) MatchRegexp ¶ added in v0.3.0
MatchRegexp produces only the input lines that match the compiled regexp re.
Example ¶
package main
import (
"regexp"
"github.com/bitfield/script"
)
func main() {
re := regexp.MustCompile("w.*d")
script.Echo("hello\nworld\n").MatchRegexp(re).Stdout()
}
Output: world
func (*Pipe) Post ¶ added in v0.21.0
Post makes an HTTP POST request to url, using the contents of the pipe as the request body, and produces the server's response. See Pipe.Do for how the HTTP response status is interpreted.
Context ¶
The request inherits the pipe's context (if any was set by Pipe.WithContext), and will be cancelled if the context is cancelled or times out.
Example ¶
package main
import (
"fmt"
"io"
"log"
"net/http"
"net/http/httptest"
"github.com/bitfield/script"
)
func main() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
data, err := io.ReadAll(r.Body)
if err != nil {
log.Fatal(err)
}
fmt.Fprintf(w, "You said: %s", data)
}))
defer ts.Close()
script.Echo("hello").Post(ts.URL).Stdout()
}
Output: You said: hello
func (*Pipe) Read ¶ added in v0.8.1
Read reads up to len(b) bytes from the pipe into b. It returns the number of bytes read and any error encountered. At end of file, or on a nil pipe, Read returns 0, io.EOF.
Example ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
buf := make([]byte, 12)
n, err := script.Echo("hello world\n").Read(buf)
if err != nil {
panic(err)
}
fmt.Println(n)
}
Output: 12
func (*Pipe) Reject ¶ added in v0.2.0
Reject produces only lines that do not contain the string s.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("a\nb\nc\n").Reject("b").Stdout()
}
Output: a c
func (*Pipe) RejectRegexp ¶ added in v0.3.0
RejectRegexp produces only lines that don't match the compiled regexp re.
Example ¶
package main
import (
"regexp"
"github.com/bitfield/script"
)
func main() {
re := regexp.MustCompile("w.*d")
script.Echo("hello\nworld\n").RejectRegexp(re).Stdout()
}
Output: hello
func (*Pipe) Replace ¶ added in v0.10.0
Replace replaces all occurrences of the string search with the string replace.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("a\nb\nc\n").Replace("b", "replacement").Stdout()
}
Output: a replacement c
func (*Pipe) ReplaceRegexp ¶ added in v0.10.0
ReplaceRegexp replaces all matches of the compiled regexp re with the string replace. $x variables in the replace string are interpreted as by [regexp#Regexp.Expand]; for example, $1 represents the text of the first submatch.
Example ¶
package main
import (
"regexp"
"github.com/bitfield/script"
)
func main() {
re := regexp.MustCompile("w.*d")
script.Echo("hello\nworld\n").ReplaceRegexp(re, "replacement").Stdout()
}
Output: hello replacement
func (*Pipe) SHA256Sum
deprecated
added in
v0.17.0
SHA256Sum returns the hex-encoded SHA-256 hash of the entire contents of the pipe, or an error.
Deprecated: SHA256Sum has been deprecated by Pipe.Hash. To get the SHA-256 hash for the contents of the pipe, call `Hash(sha256.new())`
Example ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
sum, err := script.Echo("hello world").SHA256Sum()
if err != nil {
panic(err)
}
fmt.Println(sum)
}
Output: b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9
func (*Pipe) SHA256Sums
deprecated
added in
v0.17.0
SHA256Sums reads paths from the pipe, one per line, and produces the hex-encoded SHA-256 hash of each corresponding file, one per line. Any files that cannot be opened or read will be ignored.
Deprecated: SHA256Sums has been deprecated by Pipe.HashSums. To get the SHA-256 hash for each file path in the pipe, call `HashSums(sha256.new())`
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("testdata/test.txt").SHA256Sums().Stdout()
}
Output: a562c9c95e2ff3403e7ffcd8508c6b54d47d5f251387758d3e63dbaaa8296341
func (*Pipe) Shell ¶ added in v0.25.0
Shell runs cmdLine as a command via the operating system's standard shell.
cmdLine will be passed to the shell process for expansion and execution ("sh -c" on Unix-like systems, "cmd /C" on Windows). The command will receive the contents of the pipe as input, and the command's combined output will be sent to the pipe.
Argument parsing ¶
cmdLine is passed to the shell completely unmodified as a single argument; Shell performs no parsing of cmdLine at all, so the shell alone is responsible for interpreting quoting, variable expansion, and other syntax, exactly as it would on an interactive command line. Shell is preferred over Pipe.Exec because it eliminates inconsistencies in quoting and expansion.
Note that variable syntax differs by platform: Unix shells expand variables written as $VAR, while cmd.exe on Windows expands variables written as %VAR%.
See Pipe.ExecCommand for details on error handling, context, and environment variables set via Pipe.WithEnv.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("Hello, world!").Shell("tr a-z A-Z").Stdout()
}
Output: HELLO, WORLD!
func (*Pipe) Slice ¶ added in v0.18.0
Slice returns the pipe's contents as a slice of strings, one element per line, or an error.
An empty pipe will produce an empty slice. A pipe containing a single empty line (that is, a single \n character) will produce a slice containing the empty string as its single element.
Example ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
s, err := script.Echo("a\nb\nc\n").Slice()
if err != nil {
panic(err)
}
fmt.Println(s)
}
Output: [a b c]
func (*Pipe) Stdout ¶ added in v0.6.0
Stdout copies the pipe's contents to its configured standard output (using Pipe.WithStdout), or to os.Stdout otherwise, and returns the number of bytes successfully written, together with any error.
Example ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
n, err := script.Echo("a\nb\nc\n").Stdout()
if err != nil {
panic(err)
}
fmt.Println(n)
}
Output: a b c 6
func (*Pipe) String ¶
String returns the pipe's contents as a string, together with any error.
Example ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
s, err := script.Echo("hello\nworld").String()
if err != nil {
panic(err)
}
fmt.Println(s)
}
Output: hello world
func (*Pipe) Tee ¶ added in v0.22.0
Tee copies the pipe's contents to each of the supplied writers, like Unix tee(1). If no writers are supplied, the default is the pipe's standard output.
Example (Stdout) ¶
package main
import (
"fmt"
"github.com/bitfield/script"
)
func main() {
s, err := script.Echo("hello\n").Tee().String()
if err != nil {
panic(err)
}
fmt.Println(s)
}
Output: hello hello
Example (Writers) ¶
package main
import (
"bytes"
"fmt"
"github.com/bitfield/script"
)
func main() {
buf1, buf2 := new(bytes.Buffer), new(bytes.Buffer)
s, err := script.Echo("hello\n").Tee(buf1, buf2).String()
if err != nil {
panic(err)
}
fmt.Print(s)
fmt.Print(buf1.String())
fmt.Print(buf2.String())
}
Output: hello hello hello
func (*Pipe) Unique ¶ added in v0.25.1
Unique suppresses consecutive duplicate lines.
Non-consecutive duplicates (for example A, B, A) are retained.
See also Pipe.Freq, which waits for the pipe to complete and then produces frequency counts of unique lines.
Example ¶
package main
import (
"github.com/bitfield/script"
)
func main() {
script.Echo("orange\napple\napple\napple\nbanana\n").Unique().Stdout()
}
Output: orange apple banana
func (*Pipe) Wait ¶ added in v0.19.0
Wait reads the pipe to completion and returns any error present on the pipe, or nil otherwise. This is mostly useful for waiting until concurrent filters have completed (see Pipe.Filter).
func (*Pipe) WithContext ¶ added in v0.25.0
WithContext sets the context inherited by subsequent Pipe.ExecCommand, Pipe.ExecForEach, Pipe.Get, Pipe.Post, and Pipe.Shell commands.
Example ¶
package main
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"time"
"github.com/bitfield/script"
)
func main() {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
time.Sleep(20 * time.Millisecond)
}))
defer ts.Close()
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Millisecond)
defer cancel()
_, err := script.NewPipe().WithContext(ctx).Get(ts.URL).String()
if err != nil {
fmt.Println("cancelled")
}
}
Output: cancelled
func (*Pipe) WithEnv ¶ added in v0.23.0
WithEnv sets the environment inherited by subsequent Pipe.ExecCommand and Pipe.ExecForEach, and Pipe.Shell commands to the string slice env, using the same format as os/exec.Cmd.Env. An empty slice unsets all existing environment variables.
func (*Pipe) WithHTTPClient ¶ added in v0.21.0
WithHTTPClient sets the HTTP client c for use with subsequent requests via Pipe.Do, Pipe.Get, or Pipe.Post. For example, to make a request using a client with a timeout:
NewPipe().WithHTTPClient(&http.Client{
Timeout: 10 * time.Second,
}).Get("https://example.com").Stdout()
func (*Pipe) WithReader ¶
WithReader sets the pipe's input reader to r. Once r has been completely read, it will be closed if necessary.
func (*Pipe) WithStderr ¶ added in v0.22.0
WithStderr sets the standard error output writer for commands.
The standard error output of commands run by Pipe.ExecCommand, Pipe.ExecForEach, and Pipe.Shell commands will go to w, instead of the pipe as they would otherwise.
Example ¶
package main
import (
"bytes"
"fmt"
"strings"
"github.com/bitfield/script"
)
func main() {
buf := new(bytes.Buffer)
script.NewPipe().WithStderr(buf).ExecCommand("go").Wait()
fmt.Println(strings.Contains(buf.String(), "Usage"))
}
Output: true
func (*Pipe) WithStdout ¶ added in v0.18.2
WithStdout sets the pipe's standard output to the writer w, instead of the default os.Stdout.
type ReadAutoCloser ¶ added in v0.8.0
type ReadAutoCloser struct {
// contains filtered or unexported fields
}
ReadAutoCloser wraps an io.ReadCloser so that it will be automatically closed once it has been fully read.
func NewReadAutoCloser ¶ added in v0.8.0
func NewReadAutoCloser(r io.Reader) ReadAutoCloser
NewReadAutoCloser returns a ReadAutoCloser wrapping the reader r.
func (ReadAutoCloser) Close ¶ added in v0.8.0
func (ra ReadAutoCloser) Close() error
Close closes ra's reader, returning any resulting error.
func (ReadAutoCloser) Read ¶ added in v0.8.0
func (ra ReadAutoCloser) Read(b []byte) (n int, err error)
Read reads up to len(b) bytes from ra's reader into b. It returns the number of bytes read and any error encountered. At end of file, Read returns 0, io.EOF. If end-of-file is reached, the reader will be closed.

