Documentation
¶
Overview ¶
Package executor runs the steps of a GitHub Actions job inside a sandbox. It is a port of the parts of the official runner (actions/runner, C#, MIT) that CI-1 needs: file commands, shells, contexts and the steps runner. The package does no sandboxing and no networking: it runs local processes and reports to a Reporter.
Index ¶
- Constants
- Variables
- func FixUpScriptContents(shellType, contents string) string
- func FormatArguments(format, scriptPath string) (string, error)
- func NewState(contextsJSON, matrixJSON, jobKey string, r RunnerInfo, repoFallback string) (*State, Layout, error)
- func ParseArguments(s string) []string
- func ParseShellOption(shell string) (command, args string)
- func PrependedPath(list []string, orig string) string
- func ResolveWorkingDirectory(workspace, dir string) string
- func ScriptArgumentsFormat(shellType string) string
- func ScriptFileExtension(shellType string) string
- type CancelReason
- type Canceller
- type Conclusion
- type Config
- type FileCommands
- type Layout
- type Lookup
- type NotFoundError
- type Pair
- type Reporter
- type Result
- type RunnerInfo
- type Script
- type ShellPlan
- type Sink
- type State
- func (s *State) AddPath(p string)
- func (s *State) Event() expr.Value
- func (s *State) EventJSON() string
- func (s *State) GitHub(k string) string
- func (s *State) JobEnv() *ordered
- func (s *State) Layout() Layout
- func (s *State) Output(id, name string) string
- func (s *State) Prepend() []string
- func (s *State) RuntimeEnv() *ordered
- func (s *State) Secrets() []string
- func (s *State) ServerContexts() map[string]expr.Value
- func (s *State) SetEnv(k, v string)
- func (s *State) SetGitHub(k, v string)
- func (s *State) SetOutput(id, name, value string)
- func (s *State) SetStatus(status string)
- func (s *State) SetStepResult(id, outcome, conclusion string)
- func (s *State) Status() string
- func (s *State) Values(env *ordered) map[string]expr.Value
- func (s *State) WithLayout(l Layout)
Constants ¶
const ( SetUpJobName = "Set up job" CompleteJobName = "Complete job" )
Names of the implicit steps (JobExtension.cs:70 and :15).
const ( DefaultTermAfter = 7500 * time.Millisecond DefaultKillAfter = 2500 * time.Millisecond // DefaultWaitDelay is how long the executor waits for the output pipes to // close after the shell exited, before it gives up on background children // that still hold them: ProcessInvoker.ProcessExitedHandler waits 5 s // (ProcessInvoker.cs:491). DefaultWaitDelay = 5 * time.Second )
Cancellation timings of the official runner (Runner.Sdk/ProcessInvoker.cs:32): SIGINT at once, SIGTERM after 7.5 s, kill 2.5 s later.
const DefaultTimeoutMinutes = 360
DefaultTimeoutMinutes is the timeout of a job without timeout-minutes.
const DefaultToolCache = "/opt/hostedtoolcache"
DefaultToolCache is RUNNER_TOOL_CACHE on the ermite image.
const MaxOutputBytes = 1 << 20
MaxOutputBytes is the limit of ermite on the job outputs, names and values together (1 MiB).
const ProcessTrackingID = "RUNNER_TRACKING_ID"
ProcessTrackingID is the variable that marks every process of a job so that "Complete job" can find the orphans (Constants.ProcessTrackingId).
Variables ¶
var ErrInvalidShell error = msgError("Invalid shell option. Shell must be a valid built-in (bash, sh, cmd, powershell, pwsh) or a format string containing '{0}'")
ErrInvalidShell is the error of a shell whose argument format has no {0} (ScriptHandler.cs:245).
Functions ¶
func FixUpScriptContents ¶
FixUpScriptContents is ScriptHandlerHelpers.FixUpScriptContents (:51). The switch is case-sensitive, unlike the two lookups above.
func FormatArguments ¶
FormatArguments is string.Format(argFormat, path) of .NET with one argument: {0} is the path, {{ and }} are braces; anything else is a format error. The script path has its double quotes escaped first (ScriptHandler.cs:252).
func NewState ¶
func NewState(contextsJSON, matrixJSON, jobKey string, r RunnerInfo, repoFallback string) (*State, Layout, error)
NewState builds the contexts of a job from the JSON the server sent (the "contexts" field of the job) and the matrix JSON, with the context initialisation of JobRunner.cs:142-177 and ExecutionContext.InitializeJob (ExecutionContext.cs:1001-1056).
func ParseArguments ¶
ParseArguments splits an argument string into arguments the way .NET does for Process.Start on Unix (Process.Unix.cs ParseArgumentsIntoList): spaces and tabs separate; double quotes group; backslashes only count before a quote.
func ParseShellOption ¶
ParseShellOption is ParseShellOptionString (:70): "COMMAND [..ARGS]" split at the first space.
func PrependedPath ¶
PrependedPath returns the PATH value that results from the AddPath calls recorded in list (in call order) in front of orig, like Handler.AddPrependPathToEnvironment (Handler.cs:205) with PathUtil.PrependPath.
func ResolveWorkingDirectory ¶
ResolveWorkingDirectory is Path.Combine(workspace, workingDirectory ?? "") (ScriptHandler.cs:168): an absolute directory replaces the workspace.
func ScriptArgumentsFormat ¶
ScriptArgumentsFormat is GetScriptArgumentsFormat (:33): the argument format of a built-in shell, "" for any other command.
func ScriptFileExtension ¶
ScriptFileExtension is GetScriptFileExtension (:42): ".sh", ".py", ... or "".
Types ¶
type CancelReason ¶
type CancelReason int
CancelReason says why a job is stopped.
const ( // CancelUser is a cancellation by the owner or by fail-fast: the job ends // cancelled. CancelUser CancelReason = iota + 1 // CancelTimeout is the job timeout: the job ends timed_out. CancelTimeout // CancelShutdown is the runner stopping: the job ends cancelled with // RunnerShutdown set, and the steps that did not start are skipped. CancelShutdown )
Cancel reasons.
type Canceller ¶
type Canceller struct {
// contains filtered or unexported fields
}
Canceller stops a job. The first Cancel wins; later calls are ignored (the server only sends the first reason too).
func NewCanceller ¶
func NewCanceller() *Canceller
NewCanceller returns a Canceller that has not fired.
func (*Canceller) Cancel ¶
func (c *Canceller) Cancel(reason CancelReason)
Cancel stops the job for reason.
func (*Canceller) Done ¶
func (c *Canceller) Done() <-chan struct{}
Done is closed when the job is cancelled.
func (*Canceller) Reason ¶
func (c *Canceller) Reason() CancelReason
Reason returns why the job was cancelled; zero before Done is closed.
type Conclusion ¶
type Conclusion string
Conclusion is how a step or a job ended, with the names of the protocol (CiConclusion in ermite-api).
const ( ConclusionSuccess Conclusion = "success" ConclusionFailure Conclusion = "failure" ConclusionCancelled Conclusion = "cancelled" ConclusionSkipped Conclusion = "skipped" ConclusionTimedOut Conclusion = "timed_out" // jobs only )
Conclusions.
type Config ¶
type Config struct {
// WorkflowPath and WorkflowYAML are the workflow file, which the executor
// parses itself; JobKey selects the job.
WorkflowPath string
WorkflowYAML []byte
JobKey string
// JobName is the display name of the job, for "Complete job name: ...".
// Empty means the job key.
JobName string
// ContextsJSON and MatrixJSON are the fields of the job message.
ContextsJSON string
MatrixJSON string
// Repo is the repository name used when the github context lacks one.
Repo string
// TimeoutMinutes is the job timeout; 0 means DefaultTimeoutMinutes. It is
// used for the message and, without Deadline, for the timer.
TimeoutMinutes int
// Deadline is the time left of the job timeout when Run is called, when the
// timeout started counting earlier (at the receipt of the job by the host);
// 0 means TimeoutMinutes from now.
Deadline time.Duration
Runner RunnerInfo
RunnerVersion string
// Layout overrides the directory layout (tests); nil means the hosted layout
// under Home.
Layout *Layout
// Home is the home directory of the user of the steps; empty means
// /home/runner.
Home string
// ToolCache is RUNNER_TOOL_CACHE; empty means DefaultToolCache. The official
// runner reads it from its own environment (HostContext.cs:411).
ToolCache string
// BaseEnv is the environment steps inherit; nil means os.Environ().
BaseEnv []string
// Debug turns ##[debug] lines on (ACTIONS_STEP_DEBUG).
Debug bool
Reporter Reporter
// Log receives the runner's own diagnostics (not the job log); the zero value
// discards them.
Log zerowrap.Logger
// Credential, when set, is the user the steps run as.
Credential *syscall.Credential
// Cancellation timings and test hooks, see procSpec.
TermAfter, KillAfter, WaitDelay time.Duration
Signal func(pid int, sig syscall.Signal)
// Now returns the current time; nil means time.Now.
Now func() time.Time
}
Config describes the job to run.
type FileCommands ¶
type FileCommands struct {
// contains filtered or unexported fields
}
FileCommands manages the per-step files, like FileCommandManager.
The directory belongs to the user of the steps, and the executor reads the files as root after each step, so every access goes through an os.Root opened before the first step ran: replacing the directory or a file by a symbolic link cannot make the executor read or write anything outside it. Files are opened with O_NOFOLLOW and must be regular files.
func NewFileCommands ¶
func NewFileCommands(dir string) (*FileCommands, error)
NewFileCommands uses dir (RUNNER_TEMP/_runner_file_commands), creating it. Call Close when the job is over.
func (*FileCommands) Init ¶
func (f *FileCommands) Init() (map[string]string, error)
Init creates fresh, empty files for the next step (after deleting the ones of the previous step) and returns the environment variables that name them (InitializeFiles, FileCommandManager.cs:42).
func (*FileCommands) Process ¶
func (f *FileCommands) Process(sink Sink)
Process reads the files of the step that just ran and applies them to sink. A command that fails does not stop the others (ProcessFiles, :75).
func (*FileCommands) SetOwner ¶
func (f *FileCommands) SetOwner(uid, gid int)
SetOwner gives the files of the next steps to uid:gid.
type Layout ¶
type Layout struct {
Home string // HOME of the runner user
// WorkRoot is the _work directory: <Home>/work.
WorkRoot string
// Temp is RUNNER_TEMP: <WorkRoot>/_temp.
Temp string
// ToolCache is RUNNER_TOOL_CACHE.
ToolCache string
// RunnerWorkspace is RUNNER_WORKSPACE: <WorkRoot>/<repo>.
RunnerWorkspace string
// Workspace is GITHUB_WORKSPACE: <WorkRoot>/<repo>/<repo>.
Workspace string
}
Layout is the directory layout of a job, like a hosted runner's (/home/runner/work/<repo>/<repo>, RUNNER_TEMP, RUNNER_TOOL_CACHE).
type Lookup ¶
type Lookup struct {
// Path is the PATH of the executor process (not of the step).
Path string
// Prepend is Global.PrependPath in the order of the AddPath calls.
Prepend []string
}
Lookup finds executables like WhichUtil.Which (Runner.Sdk/Util/WhichUtil.cs): in PATH with the directories added by GITHUB_PATH in front.
type NotFoundError ¶
type NotFoundError struct{ Command string }
NotFoundError is the FileNotFoundException of Which(require: true).
func (*NotFoundError) Error ¶
func (e *NotFoundError) Error() string
type Pair ¶
type Pair struct{ Key, Value string }
Pair is one name and value read from an environment-style file.
func ParseEnvFile ¶
ParseEnvFile parses the contents of GITHUB_ENV, GITHUB_OUTPUT or GITHUB_STATE: EnvFileKeyValuePairs.GetEnumerator (FileCommandManager.cs:364).
Lines are NAME=VALUE (split at the first "=") or heredocs NAME<<DELIMITER up to a line equal to DELIMITER; the form whose marker comes first wins. Empty lines between entries are skipped. Only "\n" ends a line (on Linux). The pairs read before an error are returned with it, because the original applies them as it enumerates.
type Reporter ¶
type Reporter interface {
// StepStarted announces a step that begins to run, with its display name.
// A step skipped by its condition gets StepStarted and StepCompleted too.
StepStarted(number int, name string, at time.Time)
// Log is one line of the job log, without its line terminator.
Log(number int, at time.Time, line string)
// StepCompleted ends a step.
StepCompleted(number int, name string, conclusion Conclusion, at time.Time)
// JobCompleted is the last call: the result of the job.
JobCompleted(Result)
}
Reporter receives what happens in a job. The executor calls it from several goroutines but never concurrently: calls are serialised.
Steps are numbered like GitHub's log: "Set up job" is 1, the steps of the workflow follow, and "Complete job" is last. Every log line belongs to the step that is running.
type Result ¶
type Result struct {
Conclusion Conclusion
// Outputs are the job outputs, evaluated at the end of the job. Empty values
// are left out, like the official runner does.
Outputs map[string]string
// Error is a job-level error (a timeout, a workflow that cannot be run);
// step errors are in the log.
Error string
// RunnerShutdown is true when the job was cut short because the runner was
// told to stop (see ErrShutdown).
RunnerShutdown bool
}
Result is the outcome of a job.
type RunnerInfo ¶
type RunnerInfo struct {
Name string
// Environment is "self-hosted" (the default) or "github-hosted".
Environment string
// Arch is "X64" or "ARM64"; empty means the architecture of this process.
Arch string
}
RunnerInfo describes the runner for the runner context.
type Script ¶
type Script struct {
// Path is the script file.
Path string
// Argv is the command: the shell and its arguments with the script path.
Argv []string
}
Script is a script written to the temp directory, ready to run.
func WriteScript ¶
WriteScript writes the script of a step to the directory of root (RUNNER_TEMP) under a random name with the extension of the shell, and builds the command line (ScriptHandler.RunAsync :247-285). The file has no BOM and is not executable: the shell reads it. The directory belongs to the user of the steps, so the file is created through root, exclusively and without following links (see createExcl).
type ShellPlan ¶
type ShellPlan struct {
// Command is the shell command: "sh" for the default shell (whose path is
// bash when found), else the first word of the shell option.
Command string
// CommandPath is the executable that is started, and shown on the
// "shell:" line of the log.
CommandPath string
// ArgFormat is the argument format with {0} for the script path.
ArgFormat string
}
ShellPlan is how a step's script is run: the shell chosen by the step, the job defaults, or the runner's default.
func PlanShell ¶
PlanShell chooses the shell like ScriptHandler.RunAsync (:170-234) and PrintActionDetails (:65-117). shell is the step's `shell` or, when empty, defaults.run.shell (the caller does that merge, :171-180).
With no shell: bash if it is in PATH, else sh, with the arguments of "sh" (-e {0}). "sh" is only the name used for the arguments and the extension.
type Sink ¶
type Sink interface {
// SetEnv sets an environment variable of the job (GITHUB_ENV).
SetEnv(name, value string)
// SetOutput sets an output of the step (GITHUB_OUTPUT).
SetOutput(name, value string)
// AddPath prepends a directory to PATH for the next steps (GITHUB_PATH). The
// caller keeps the list like Global.PrependPath: the last added is first
// in PATH, and a repeated entry moves to the end of the list.
AddPath(path string)
// SaveState keeps a value for the step's own post phase (GITHUB_STATE).
SaveState(name, value string)
// Error writes an error issue to the log (##[error]).
Error(msg string)
// Debug writes a ##[debug] line when step debugging is on.
Debug(msg string)
// CommandFailed marks the step as failed (context.CommandResult = Failed).
CommandFailed()
}
Sink receives the effects of the file commands of a step.
type State ¶
type State struct {
// contains filtered or unexported fields
}
State holds the contexts of a job as the official runner does (ExecutionContext.ExpressionValues): the contexts the server computed, and github, runner, job, env and steps, which the runner owns.
func (*State) AddPath ¶
AddPath records a GITHUB_PATH entry: a repeated entry moves to the end (AddPathFileCommand, FileCommandManager.cs:152).
func (*State) EventJSON ¶
EventJSON returns github.event as JSON for event.json, or "" without event.
func (*State) JobEnv ¶
func (s *State) JobEnv() *ordered
JobEnv returns a copy of the job environment.
func (*State) RuntimeEnv ¶
func (s *State) RuntimeEnv() *ordered
RuntimeEnv is the environment that the contexts add to every step: the allow-listed keys of github as GITHUB_<KEY> (strings and booleans only, as in GitHubContext.GetRuntimeEnvironmentVariables), and every key of runner as RUNNER_<KEY> (RunnerContext.cs:11). Numbers and objects are skipped: the server must send GitHub's string forms ("run_id": "123").
func (*State) Secrets ¶
Secrets returns the values that must never leave the job: the token and the base64 of "x-access-token:<token>" (ExecutionContext.cs:1026-1028); newMasker adds their other encodings.
func (*State) ServerContexts ¶
ServerContexts returns the contexts that the server has when it evaluates the name of a job: github as received (without the keys the runner adds), needs, strategy, matrix, vars and inputs.
func (*State) SetEnv ¶
SetEnv sets a variable of the job environment (Global.EnvironmentVariables and the env context).
func (*State) SetOutput ¶
SetOutput sets an output of step id (StepsContext.SetOutput). Steps with a generated id keep no outputs.
func (*State) SetStepResult ¶
SetStepResult records outcome and conclusion of a completed step (ExecutionContext.UpdateGlobalStepsContext).
func (*State) Values ¶
Values returns the contexts for expression evaluation. stepEnv, when not nil, is the env context of a step (job environment with the step's own variables); otherwise the env context is the job environment.
func (*State) WithLayout ¶
WithLayout replaces the directory layout (tests and non-default homes). It must be called before the first step.