Documentation
¶
Overview ¶
Package tree is how a file tool reaches the step's directory: directly, or through the image's own userland in the step's container.
Index ¶
- Constants
- Variables
- func ResolvePath(dir, rel string) (string, error)
- func ResolveWritePath(dir, rel string) (string, error)
- type Container
- func (c Container) Execute(ctx context.Context, script string) (stdout, stderr string, code int, err error)
- func (c Container) ListDir(ctx context.Context, p string) ([]Entry, error)
- func (c Container) OpenRange(ctx context.Context, p string, through int) (io.ReadCloser, error)
- func (c Container) ReadBytes(ctx context.Context, p string, limit int64) ([]byte, error)
- func (c Container) Resolve(ctx context.Context, rel string) (string, error)
- func (c Container) ResolveWrite(ctx context.Context, rel string) (string, error)
- func (c Container) Root() string
- func (c Container) Search(ctx context.Context, base string, opts SearchOpts) (SearchResult, error)
- func (c Container) Stat(ctx context.Context, p string) (Stat, error)
- func (c Container) WriteFile(ctx context.Context, p string, data []byte, appendTo bool) error
- type Entry
- type Host
- func (h Host) ListDir(_ context.Context, path string) ([]Entry, error)
- func (h Host) OpenRange(_ context.Context, path string, _ int) (io.ReadCloser, error)
- func (h Host) ReadBytes(_ context.Context, path string, limit int64) ([]byte, error)
- func (h Host) Resolve(_ context.Context, rel string) (string, error)
- func (h Host) ResolveWrite(_ context.Context, rel string) (string, error)
- func (h Host) Root() string
- func (h Host) Search(_ context.Context, base string, opts SearchOpts) (SearchResult, error)
- func (h Host) Stat(_ context.Context, path string) (Stat, error)
- func (h Host) WriteFile(_ context.Context, path string, data []byte, appendTo bool) error
- type Lost
- type Match
- type SearchOpts
- type SearchResult
- type Stat
- type Tree
Constants ¶
const ( // MaxSearchLineBytes bounds a single returned matching line. A minified // bundle or a generated table can carry lines of tens of KB, any one of // which would blow the whole result budget on its own. 500 bytes keeps // nearly every real source line whole — the previous 200 cut most // matching lines mid-statement, which mattered for edit_file old_strings // copied from results. MaxSearchLineBytes = 500 // MaxSearchContentBudgetBytes bounds the TOTAL bytes of kept content // matches (line text plus SearchMatchOverheadBytes each). This is what // lets both the line bound and the result ceiling be generous: short // typical lines can fill all maxSearchContentResults slots, while // maximum-length lines saturate the byte budget after ~45 matches — // either way the result fits inline, by construction. MaxSearchContentBudgetBytes = 28_000 // SearchMatchOverheadBytes is the per-match allowance for the path, line // number, and JSON scaffolding around the line text. SearchMatchOverheadBytes = 120 // MaxSearchFileBytes skips files too large to be worth scanning line by // line. A match inside a 2MB file is nearly always a generated artifact. MaxSearchFileBytes = 2 << 20 // MaxSearchFilesScanned bounds the walk itself, so a search rooted at a // huge tree terminates in bounded time rather than stat-ing every inode. MaxSearchFilesScanned = 20_000 )
const MaxScanBytes = 10 << 20 // 10 MiB
MaxScanBytes bounds the largest single line readFileRange will pull into memory before deciding what to keep. It's well above maxReadFileBytes (the return budget) so a spilled file's long line — base64/minified/`jq -c` output, which frequently has no newlines at all — is still readable (byte-truncated to the budget) rather than failing the scan with a cryptic "token too long"; a line even larger than this degrades to truncated=true, still not a hard error. Matches internal/shell's SpillMaxBytes.
const SpillDirName = ".steps-agent-out"
SpillDirName is the fixed name of the subdirectory a run_shell/custom tool's oversized output is spilled into (see newToolOutputSpillDir). config.artifactNamePattern requires a declared input/output name to start with a letter or digit, so this leading '.' can never collide with a real one.
Variables ¶
var ErrNoAccess = errors.New("the step's container could not be reached")
ErrNoAccess is a tree operation that could not run at all — a container that will not start, a worker that went away. Distinct from a tool's own failure, since the conversation cannot recover from it by trying a different path.
Functions ¶
func ResolvePath ¶
ResolvePath resolves rel (as given by the model) against dir and rejects any result that escapes dir — lexically (a crafted "../../etc/passwd" style path) and, once a target actually exists, by symlink (see rejectSymlinkEscape).
rel may be absolute: an oversized run_shell/custom-tool output spills to a file under the working directory and hands the model that file's absolute path (shell.SpillPointerMessage), and the working directory itself is an absolute path the model is told in its own system message — both are spellings of a location already inside dir, not an escape attempt. IsAbs used to reject both outright, which is why maxReadFileBytes' stated purpose ("a spilled tool output can be read back whole in one call") never actually worked: the containment check below is what makes an absolute path safe, so restricting rel to a relative spelling was never load-bearing — it only blocked the legitimate case along with the escaping one.
The lexical half is config.ConfinedPath, the same predicate load asks of context_paths:, so a path load accepts is one this accepts.
func ResolveWritePath ¶
ResolveWritePath resolves rel like ResolvePath, then closes a gap ResolvePath alone leaves open for a brand-new file: EvalSymlinks fails with ENOENT on a nonexistent leaf regardless of whether an ancestor directory is a symlink (rejectSymlinkEscape then treats that as "nothing to leak"), so a target file that doesn't exist yet would otherwise skip the escape check entirely even when its parent directory is a symlink planted to point outside dir.
When the target does not exist yet it creates missing parent directories (mkdir -p), but only after verifying the closest EXISTING ancestor is within the workspace.
Types ¶
type Container ¶
Container runs each file operation as a POSIX sh script in the step's container, through the same shell.Runner its run_shell uses. Nothing is mounted and nothing is pushed: the scripts use only the commands treeCommands names — which probeUserland checks for before a token is spent — plus printf, a shell builtin everywhere. That is why they work on a foreign machine's image without steps having to put anything there.
func (Container) Execute ¶
func (c Container) Execute(ctx context.Context, script string) (stdout, stderr string, code int, err error)
Execute runs one script in the container. Only a failure to run at all is an error, and it is filed with Lost.
func (Container) ListDir ¶
ListDir emits one tab-separated row per entry. find does the walking so a name with spaces survives, and the type comes from a test rather than from parsing ls, whose long format is not the same program to program.
func (Container) OpenRange ¶
OpenRange asks sed for the lines the caller will actually keep, which is the one place the container can do less work than the host rather than more: a range deep in a large file crosses the wire as the range, not as the file.
func (Container) Resolve ¶
Resolve confines rel to the container's tree. The lexical half is the same predicate the host asks; the symlink half has to be asked of the container, because a link resolves against ITS root — where /etc/passwd is the image's, not this machine's.
func (Container) ResolveWrite ¶
ResolveWrite closes the gap resolve leaves for a file that does not exist yet: a missing leaf has no link to follow, so a parent directory planted as a symlink out of the tree would never be looked at. The closest ancestor that does exist is checked instead, and only then are the parents created.
func (Container) Search ¶
func (c Container) Search(ctx context.Context, base string, opts SearchOpts) (SearchResult, error)
Search runs the walk as two commands and keeps every judgement on this side. find only lists paths — a format no two implementations can disagree about — so the prune list, the glob, the size limit, the head limit, the per-line cap, the byte budget and files_scanned are all still decided by the same Go that decides them for a host agent. grep only reports matching lines. Nothing about which results the model sees is delegated to the image.
func (Container) Stat ¶
Stat runs a script that prints a kind letter and a byte size, or exits 1 for a path that is not there. Size comes from wc -c rather than from ls, whose long-format columns differ between busybox and GNU in exactly the fields a parser would need.
type Host ¶
type Host struct{ Dir string }
Host reads and writes the step directory directly. It is the behaviour these tools had before a container could hold them, kept whole so an agent with no image: is byte-for-byte unaffected.
func (Host) OpenRange ¶
OpenRange hands back the file itself and ignores through: the scanner the caller runs is what finds the lines, and on this side there is nothing cheaper to do than let it pass over them. The container implementation has a reason to honour it, and the caller cannot tell the difference because both start at line one.
func (Host) ResolveWrite ¶
ResolveWrite satisfies Tree.
func (Host) Search ¶
func (h Host) Search(_ context.Context, base string, opts SearchOpts) (SearchResult, error)
Search satisfies Tree.
type Lost ¶
type Lost struct {
// contains filtered or unexported fields
}
Lost records the first infrastructure failure a tool hit, so the conversation stops instead of arguing with a machine that is gone.
Every tool failure is ordinarily data: a nonzero exit, a missing file, a refused pattern all come back to the model because the model is who can react to them. A worker that was reclaimed is not that. Handed back as `{"error": ...}` it looks like an ordinary setback, and the model will reasonably try the call again, and again, until max_turns — spending the whole step's budget on a host that no longer exists.
type SearchOpts ¶
type SearchOpts struct {
Re *regexp.Regexp // nil for a filename-only search
Glob string // "" matches every file
Mode string
HeadLimit int
// pattern and caseInsensitive are the model's own arguments, kept because a container tree cannot hand a compiled *regexp.Regexp to grep — it renders the pattern to POSIX ERE instead, and folding is a grep flag rather than part of the expression.
Pattern string
CaseInsensitive bool
}
SearchOpts is one resolved search_files call.
type SearchResult ¶
type SearchResult struct {
Files []string
Matches []Match
Counts []map[string]any
Total int
FilesScanned int
BudgetHit bool
// contains filtered or unexported fields
}
SearchResult accumulates a walk. total counts every match found, not just the ones kept, so the model is told the true scale and can narrow instead of assuming it saw everything — the same total/truncated contract execListDir already exposes.
type Stat ¶
Stat is the part of os.FileInfo the file tools actually ask for. No mode: an edit keeps a file's own by writing THROUGH it (see Host.writeFile and Container.writeFile), so nothing has to carry one across.
type Tree ¶
type Tree interface {
// Root is the directory paths resolve against, in this tree's own spelling. For a container on a worker that is the worker's path, which is also what the model is told its working directory is.
Root() string
// Resolve confines rel to root and answers the absolute path in this tree's spelling.
Resolve(ctx context.Context, rel string) (string, error)
// ResolveWrite is resolve for a path that need not exist yet: the closest existing ancestor is what gets checked against root, since a missing leaf has no link to follow. Missing parents are created by whichever caller goes on to write.
ResolveWrite(ctx context.Context, rel string) (string, error)
// Stat answers os.ErrNotExist for a missing path, like os.Stat.
Stat(ctx context.Context, path string) (Stat, error)
// ReadBytes is the first limit bytes of a file.
ReadBytes(ctx context.Context, path string, limit int64) ([]byte, error)
// OpenRange is a reader over the file's first through lines, or over the whole file when through is zero. Both implementations start at line one, so the caller's scanner counts the same lines either way.
OpenRange(ctx context.Context, path string, through int) (io.ReadCloser, error)
// WriteFile replaces path's contents, keeping the file's mode when it already exists.
WriteFile(ctx context.Context, path string, data []byte, appendTo bool) error
// ListDir is one directory's entries, unsorted, as the tool reports them.
ListDir(ctx context.Context, path string) ([]Entry, error)
// Search walks base under opts and returns what the model is shown.
Search(ctx context.Context, base string, opts SearchOpts) (SearchResult, error)
}
Tree is the IO half of the file tools. Every cap, glob, prune and result shape stays in the callers, so the two implementations differ only in how bytes are fetched and never in what the model is told — which is the whole promise, since an agent that moves to a worker must not start answering differently.
func ForStep ¶
func ForStep(ctx context.Context, runner shell.Runner, image, dir string, specs []config.ToolSpec, lost *Lost) (Tree, string, error)
ForStep answers the tree a step's file tools should use and the working directory the model should be told about. With no image: both are this process's own; with one, the file tools move into the container so that they and run_shell address a single copy of the tree, and the directory named is the CONTAINER's — every path handed back to the model now comes from there, and for a placed step this machine's spelling of it names a path on the wrong host.