tree

package
v0.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 25, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package tree is how a file tool reaches the step's directory: directly, or through the image's own userland in the step's container.

Index

Constants

View Source
const (
	// MaxSearchLineBytes bounds a single returned matching line. A minified
	// bundle or a generated table can carry lines of tens of KB, any one of
	// which would blow the whole result budget on its own. 500 bytes keeps
	// nearly every real source line whole — the previous 200 cut most
	// matching lines mid-statement, which mattered for edit_file old_strings
	// copied from results.
	MaxSearchLineBytes = 500

	// MaxSearchContentBudgetBytes bounds the TOTAL bytes of kept content
	// matches (line text plus SearchMatchOverheadBytes each). This is what
	// lets both the line bound and the result ceiling be generous: short
	// typical lines can fill all maxSearchContentResults slots, while
	// maximum-length lines saturate the byte budget after ~45 matches —
	// either way the result fits inline, by construction.
	MaxSearchContentBudgetBytes = 28_000

	// SearchMatchOverheadBytes is the per-match allowance for the path, line
	// number, and JSON scaffolding around the line text.
	SearchMatchOverheadBytes = 120

	// MaxSearchFileBytes skips files too large to be worth scanning line by
	// line. A match inside a 2MB file is nearly always a generated artifact.
	MaxSearchFileBytes = 2 << 20

	// MaxSearchFilesScanned bounds the walk itself, so a search rooted at a
	// huge tree terminates in bounded time rather than stat-ing every inode.
	MaxSearchFilesScanned = 20_000
)
View Source
const MaxScanBytes = 10 << 20 // 10 MiB

MaxScanBytes bounds the largest single line readFileRange will pull into memory before deciding what to keep. It's well above maxReadFileBytes (the return budget) so a spilled file's long line — base64/minified/`jq -c` output, which frequently has no newlines at all — is still readable (byte-truncated to the budget) rather than failing the scan with a cryptic "token too long"; a line even larger than this degrades to truncated=true, still not a hard error. Matches internal/shell's SpillMaxBytes.

View Source
const SpillDirName = ".steps-agent-out"

SpillDirName is the fixed name of the subdirectory a run_shell/custom tool's oversized output is spilled into (see newToolOutputSpillDir). config.artifactNamePattern requires a declared input/output name to start with a letter or digit, so this leading '.' can never collide with a real one.

Variables

View Source
var ErrNoAccess = errors.New("the step's container could not be reached")

ErrNoAccess is a tree operation that could not run at all — a container that will not start, a worker that went away. Distinct from a tool's own failure, since the conversation cannot recover from it by trying a different path.

Functions

func ResolvePath

func ResolvePath(dir, rel string) (string, error)

ResolvePath resolves rel (as given by the model) against dir and rejects any result that escapes dir — lexically (a crafted "../../etc/passwd" style path) and, once a target actually exists, by symlink (see rejectSymlinkEscape).

rel may be absolute: an oversized run_shell/custom-tool output spills to a file under the working directory and hands the model that file's absolute path (shell.SpillPointerMessage), and the working directory itself is an absolute path the model is told in its own system message — both are spellings of a location already inside dir, not an escape attempt. IsAbs used to reject both outright, which is why maxReadFileBytes' stated purpose ("a spilled tool output can be read back whole in one call") never actually worked: the containment check below is what makes an absolute path safe, so restricting rel to a relative spelling was never load-bearing — it only blocked the legitimate case along with the escaping one.

The lexical half is config.ConfinedPath, the same predicate load asks of context_paths:, so a path load accepts is one this accepts.

func ResolveWritePath

func ResolveWritePath(dir, rel string) (string, error)

ResolveWritePath resolves rel like ResolvePath, then closes a gap ResolvePath alone leaves open for a brand-new file: EvalSymlinks fails with ENOENT on a nonexistent leaf regardless of whether an ancestor directory is a symlink (rejectSymlinkEscape then treats that as "nothing to leak"), so a target file that doesn't exist yet would otherwise skip the escape check entirely even when its parent directory is a symlink planted to point outside dir.

When the target does not exist yet it creates missing parent directories (mkdir -p), but only after verifying the closest EXISTING ancestor is within the workspace.

Types

type Container

type Container struct {
	Runner shell.Runner

	Lost *Lost
	// contains filtered or unexported fields
}

Container runs each file operation as a POSIX sh script in the step's container, through the same shell.Runner its run_shell uses. Nothing is mounted and nothing is pushed: the scripts use only the commands treeCommands names — which probeUserland checks for before a token is spent — plus printf, a shell builtin everywhere. That is why they work on a foreign machine's image without steps having to put anything there.

func (Container) Execute

func (c Container) Execute(ctx context.Context, script string) (stdout, stderr string, code int, err error)

Execute runs one script in the container. Only a failure to run at all is an error, and it is filed with Lost.

func (Container) ListDir

func (c Container) ListDir(ctx context.Context, p string) ([]Entry, error)

ListDir emits one tab-separated row per entry. find does the walking so a name with spaces survives, and the type comes from a test rather than from parsing ls, whose long format is not the same program to program.

func (Container) OpenRange

func (c Container) OpenRange(ctx context.Context, p string, through int) (io.ReadCloser, error)

OpenRange asks sed for the lines the caller will actually keep, which is the one place the container can do less work than the host rather than more: a range deep in a large file crosses the wire as the range, not as the file.

func (Container) ReadBytes

func (c Container) ReadBytes(ctx context.Context, p string, limit int64) ([]byte, error)

ReadBytes satisfies Tree.

func (Container) Resolve

func (c Container) Resolve(ctx context.Context, rel string) (string, error)

Resolve confines rel to the container's tree. The lexical half is the same predicate the host asks; the symlink half has to be asked of the container, because a link resolves against ITS root — where /etc/passwd is the image's, not this machine's.

func (Container) ResolveWrite

func (c Container) ResolveWrite(ctx context.Context, rel string) (string, error)

ResolveWrite closes the gap resolve leaves for a file that does not exist yet: a missing leaf has no link to follow, so a parent directory planted as a symlink out of the tree would never be looked at. The closest ancestor that does exist is checked instead, and only then are the parents created.

func (Container) Root

func (c Container) Root() string

Root satisfies Tree.

func (Container) Search

func (c Container) Search(ctx context.Context, base string, opts SearchOpts) (SearchResult, error)

Search runs the walk as two commands and keeps every judgement on this side. find only lists paths — a format no two implementations can disagree about — so the prune list, the glob, the size limit, the head limit, the per-line cap, the byte budget and files_scanned are all still decided by the same Go that decides them for a host agent. grep only reports matching lines. Nothing about which results the model sees is delegated to the image.

func (Container) Stat

func (c Container) Stat(ctx context.Context, p string) (Stat, error)

Stat runs a script that prints a kind letter and a byte size, or exits 1 for a path that is not there. Size comes from wc -c rather than from ls, whose long-format columns differ between busybox and GNU in exactly the fields a parser would need.

func (Container) WriteFile

func (c Container) WriteFile(ctx context.Context, p string, data []byte, appendTo bool) error

WriteFile stages the content and finishes with `cat staged > target`, never a move. The redirection writes THROUGH the existing inode, so the file keeps its mode; a rename would replace it and quietly strip the executable bit off a checked-in script.

type Entry

type Entry struct {
	Name  string
	IsDir bool
	Size  int64
}

Entry is one list_dir row.

type Host

type Host struct{ Dir string }

Host reads and writes the step directory directly. It is the behaviour these tools had before a container could hold them, kept whole so an agent with no image: is byte-for-byte unaffected.

func (Host) ListDir

func (h Host) ListDir(_ context.Context, path string) ([]Entry, error)

ListDir satisfies Tree.

func (Host) OpenRange

func (h Host) OpenRange(_ context.Context, path string, _ int) (io.ReadCloser, error)

OpenRange hands back the file itself and ignores through: the scanner the caller runs is what finds the lines, and on this side there is nothing cheaper to do than let it pass over them. The container implementation has a reason to honour it, and the caller cannot tell the difference because both start at line one.

func (Host) ReadBytes

func (h Host) ReadBytes(_ context.Context, path string, limit int64) ([]byte, error)

ReadBytes satisfies Tree.

func (Host) Resolve

func (h Host) Resolve(_ context.Context, rel string) (string, error)

Resolve satisfies Tree.

func (Host) ResolveWrite

func (h Host) ResolveWrite(_ context.Context, rel string) (string, error)

ResolveWrite satisfies Tree.

func (Host) Root

func (h Host) Root() string

Root satisfies Tree.

func (Host) Search

func (h Host) Search(_ context.Context, base string, opts SearchOpts) (SearchResult, error)

Search satisfies Tree.

func (Host) Stat

func (h Host) Stat(_ context.Context, path string) (Stat, error)

Stat satisfies Tree.

func (Host) WriteFile

func (h Host) WriteFile(_ context.Context, path string, data []byte, appendTo bool) error

WriteFile passes defaultToolFileMode for a file that does not exist yet; an existing one keeps the mode it has, since the OS ignores the permission argument once the inode is there. That is what stops an edit to a checked-in script from stripping its executable bit.

type Lost

type Lost struct {
	// contains filtered or unexported fields
}

Lost records the first infrastructure failure a tool hit, so the conversation stops instead of arguing with a machine that is gone.

Every tool failure is ordinarily data: a nonzero exit, a missing file, a refused pattern all come back to the model because the model is who can react to them. A worker that was reclaimed is not that. Handed back as `{"error": ...}` it looks like an ordinary setback, and the model will reasonably try the call again, and again, until max_turns — spending the whole step's budget on a host that no longer exists.

func (*Lost) Note

func (l *Lost) Note(err error)

Note records err if it is the kind of failure that ends the step, and ignores it otherwise.

func (*Lost) Taken

func (l *Lost) Taken() error

Taken is the recorded failure, if there was one.

type Match

type Match struct {
	Path string
	Line int
	Text string
}

Match is one matching line in content mode.

type SearchOpts

type SearchOpts struct {
	Re        *regexp.Regexp // nil for a filename-only search
	Glob      string         // "" matches every file
	Mode      string
	HeadLimit int
	// pattern and caseInsensitive are the model's own arguments, kept because a container tree cannot hand a compiled *regexp.Regexp to grep — it renders the pattern to POSIX ERE instead, and folding is a grep flag rather than part of the expression.
	Pattern         string
	CaseInsensitive bool
}

SearchOpts is one resolved search_files call.

type SearchResult

type SearchResult struct {
	Files        []string
	Matches      []Match
	Counts       []map[string]any
	Total        int
	FilesScanned int
	BudgetHit    bool
	// contains filtered or unexported fields
}

SearchResult accumulates a walk. total counts every match found, not just the ones kept, so the model is told the true scale and can narrow instead of assuming it saw everything — the same total/truncated contract execListDir already exposes.

type Stat

type Stat struct {
	Size  int64
	IsDir bool
}

Stat is the part of os.FileInfo the file tools actually ask for. No mode: an edit keeps a file's own by writing THROUGH it (see Host.writeFile and Container.writeFile), so nothing has to carry one across.

type Tree

type Tree interface {
	// Root is the directory paths resolve against, in this tree's own spelling. For a container on a worker that is the worker's path, which is also what the model is told its working directory is.
	Root() string
	// Resolve confines rel to root and answers the absolute path in this tree's spelling.
	Resolve(ctx context.Context, rel string) (string, error)
	// ResolveWrite is resolve for a path that need not exist yet: the closest existing ancestor is what gets checked against root, since a missing leaf has no link to follow. Missing parents are created by whichever caller goes on to write.
	ResolveWrite(ctx context.Context, rel string) (string, error)
	// Stat answers os.ErrNotExist for a missing path, like os.Stat.
	Stat(ctx context.Context, path string) (Stat, error)
	// ReadBytes is the first limit bytes of a file.
	ReadBytes(ctx context.Context, path string, limit int64) ([]byte, error)
	// OpenRange is a reader over the file's first through lines, or over the whole file when through is zero. Both implementations start at line one, so the caller's scanner counts the same lines either way.
	OpenRange(ctx context.Context, path string, through int) (io.ReadCloser, error)
	// WriteFile replaces path's contents, keeping the file's mode when it already exists.
	WriteFile(ctx context.Context, path string, data []byte, appendTo bool) error
	// ListDir is one directory's entries, unsorted, as the tool reports them.
	ListDir(ctx context.Context, path string) ([]Entry, error)
	// Search walks base under opts and returns what the model is shown.
	Search(ctx context.Context, base string, opts SearchOpts) (SearchResult, error)
}

Tree is the IO half of the file tools. Every cap, glob, prune and result shape stays in the callers, so the two implementations differ only in how bytes are fetched and never in what the model is told — which is the whole promise, since an agent that moves to a worker must not start answering differently.

func ForStep

func ForStep(ctx context.Context, runner shell.Runner, image, dir string, specs []config.ToolSpec, lost *Lost) (Tree, string, error)

ForStep answers the tree a step's file tools should use and the working directory the model should be told about. With no image: both are this process's own; with one, the file tools move into the container so that they and run_shell address a single copy of the tree, and the directory named is the CONTAINER's — every path handed back to the model now comes from there, and for a placed step this machine's spelling of it names a path on the wrong host.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL