shellrunner

shellrunner

Package shellrunner lets one script a command line shell as if a human were running it.

The package separates the problem of orchestrating shell execution from the problem of generating a shell command and parsing the shell's response to said command.

What's a shell?

A shell is any program that offers a prompt and accepts commands from stdIn, e.g bash or mysql.

A command is entered, and the resulting execution prints to stdOut and stdErr. When the command completes, the prompt appears again. Execution ends when stdIn closes (e.g. user hits Ctrl-d) or when some special command is entered (e.g. quit or exit).

The purpose of a shell, as opposed to a single-purpose program that doesn't offer a prompt (e.g. ls, sed, kubectl, ...) , is to allow for state that endures over multiple commands.

The state can have things like environment variables, caches, database connections, etc. State can be expensive to build; a shell lets a user pay to build it once, then run many commands in its context. State established by one command can impact behavior of subsequent commands.

Requirements and assumptions

Logic between commands.

The output of command n must be able to influence the choice of and arguments to command n+1.

That's the whole point of scripting.

Four outcomes.

The outcome of asking a shell to run a command is one of the following:

• crash - shell exits with non-zero status.
• exit - shell exits with zero status.
  If this happens unintentionally, it's treated as a crash.
• timeout - shell fails to become ready in a given time period after issuing a command.
  This is treated as a crash.
• ready - shell runs the command and is ready to accept another command.

Command generation and parsing live together

The package is structured such that all a Go author need do is implement the Commander interface, then pass instances of the implementation to the Run method of a Runner. When a Run call returns, the Commander instance can be consulted. A commander can offer any number of methods yielding validated data acquired from the shell.

This design embraces the notion that the code that parses a command's output should live close to the code that generates the actual command line. The parser should know all the command line arguments and flags used.

The Commander can be tested for its ability to compose a command line, and it's ability to parse output that the shell is expected to generate in response to that command line.

Unreliable prompts, unreliable newlines

A human knows a shell has completed command n and is awaiting command n+1 because they see a prompt following the output of command n. Usually, but not always, the prompt is on a new line.

But when running a shell as a subprocess, the shell can see that stdIn is not a tty, and in general will not issue a prompt to either stdOut or stdErr. Sometimes prompts are undesirable, as they contaminate piped output streams. Sometimes command output can accidentally contain the prompt string, making it useless asa sentinel.

Likewise, a shell isn't obligated to provide a newline after command output completes. Newlines added to output can be undesirable (e.g. base64 -d, echo -n).

For these reasons, a shell runner cannot depend on prompts and newlines to unambiguously distinguish the data from commands n-1, n and n+1 on stdOut and stdErr.

This leads to the use of sentinels.

Sentinels

stdOut

A Runner demands the existence of (at least) one sentinel command for stdOut.

Such a command

• does very little,
• does it quickly,
• and has a deterministic, newline terminated output value on stdOut.

Example:

$ echo "rumpelstiltskin"
rumpelstiltskin

Commands that print a program's version, a help message, and/or a copyright message are good candidates for Sentinel on the stdOut stream.

stdErr

Likewise, a Runner needs a sentinel command for stdErr.

This command differs from the stdOut sentinel only in that its output goes to stdErr. Usually a shell will complain to stdErr if it sees a command it doesn't recognize. An unrecognized command is a good stdErr sentinel.

Example:

$ rumpelstiltskin
rumpelstiltskin: command not found

Example

See also example_test.go

In Start, the shell subprocess is started, and given the stdOut and stdErr sentinels to establish that they behave as expected. If they don't, it will be impossible to know when a given command has finished, so Start will fail early to indicate this.

If Start succeeds without error, every command given to the subprocess via Run will be automatically followed by the two sentinels.

The two output streams from the subprocess (stdOut and stdErr) will be scanned for the two deterministic sentinel output values. Once these two output values have been found, the preceding output (both streams) will be delivered to the command for processing.

The shell will self-terminate on crash or exit, and will be actively terminated on command timeout.

Your job as a framework user is implement Commander instances in Go, and write a main that feeds them into Runner.

shell, _ := NewRunner(&Parameters{
	Path: "/some/path/to/shellProgram",
	Args: []string{
		"--enableThis",
		"--setFoo", "4",
	},
	ExitCommand: "exit",
    OutSentinel: MakeOutSentinel(),
    ErrSentinel: MakeErrSentinel(),
})
err := shell.Start(5 * time.Second)
if err != nil { /* handle it */ }
c := NewCommander("someCommand")
err := shell.Run(c, 5 * time.Second)
if err != nil { /* handle it */ }
value := c.GetSomeValueParsedFromCommandOutput()
c := NewCommander("someOtherCommand " + value)
err := shell.Run(c, 3 * time.Second)
if err != nil { /* handle it */ }
// ... etc.
err := shell.Stop()
if err != nil { /* handle it */ }