testutil

package
v0.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 19, 2026 License: MIT Imports: 18 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ArtifactRoot string

ArtifactRoot is the absolute path to the artifact output directory. Must be set in TestMain before any tests run.

View Source 
var ArtifactTimestamp = time.Now().Format("2006-01-02T15-04-05")

ArtifactTimestamp is the timestamp subdirectory for this test run.

Functions

func ArtifactRunDir 

func ArtifactRunDir() string

ArtifactRunDir returns the directory for the current test run.

func AssertCheckpointAdvanced 

func AssertCheckpointAdvanced(t *testing.T, s *RepoState)

AssertCheckpointAdvanced asserts the checkpoint branch moved forward.

func AssertCheckpointExists 

func AssertCheckpointExists(t *testing.T, dir string, checkpointID string)

AssertCheckpointExists asserts that the checkpoint ID is mentioned on the checkpoint branch and that its metadata.json exists in the tree.

func AssertCheckpointFilesTouched 

func AssertCheckpointFilesTouched(t *testing.T, dir string, checkpointID string, expected []string)

AssertCheckpointFilesTouched asserts the checkpoint metadata lists exactly the expected files in files_touched (order-independent).

func AssertCheckpointFilesTouchedContains 

func AssertCheckpointFilesTouchedContains(t *testing.T, dir string, checkpointID string, file string)

AssertCheckpointFilesTouchedContains asserts files_touched contains a specific file path (subset check, unlike the exact-match AssertCheckpointFilesTouched).

func AssertCheckpointHasSingleSession 

func AssertCheckpointHasSingleSession(t *testing.T, dir string, checkpointID string)

AssertCheckpointHasSingleSession asserts checkpoint metadata has exactly one session.

func AssertCheckpointIDFormat 

func AssertCheckpointIDFormat(t *testing.T, checkpointID string)

AssertCheckpointIDFormat asserts the checkpoint ID is 12 lowercase hex chars.

func AssertCheckpointInLastN 

func AssertCheckpointInLastN(t *testing.T, dir string, checkpointID string, n int)

AssertCheckpointInLastN asserts the given checkpoint ID appears in at least n commits on the checkpoint branch (e.g. initial + catchup). Uses --grep to find matching commits regardless of position, so extra commits from multi-commit agent turns don't cause false failures.

func AssertCheckpointMetadataComplete 

func AssertCheckpointMetadataComplete(t *testing.T, dir string, checkpointID string)

AssertCheckpointMetadataComplete asserts essential fields in checkpoint metadata are populated.

func AssertCheckpointNotAdvanced 

func AssertCheckpointNotAdvanced(t *testing.T, s *RepoState)

AssertCheckpointNotAdvanced asserts the checkpoint branch has NOT moved.

func AssertCommitLinkedToCheckpoint 

func AssertCommitLinkedToCheckpoint(t *testing.T, dir string, ref string)

AssertCommitLinkedToCheckpoint asserts the trailer exists AND the checkpoint data exists on the checkpoint branch.

func AssertDistinctSessions 

func AssertDistinctSessions(t *testing.T, dir string, checkpointIDs []string)

AssertDistinctSessions asserts session metadata across checkpoints has unique session IDs.

func AssertFileExists 

func AssertFileExists(t *testing.T, dir string, glob string)

AssertFileExists asserts that at least one file matches the glob pattern relative to dir.

func AssertHasCheckpointTrailer 

func AssertHasCheckpointTrailer(t *testing.T, dir string, ref string) string

AssertHasCheckpointTrailer asserts the commit has an Entire-Checkpoint trailer, validates its format, and returns its value.

func AssertHasShadowBranches 

func AssertHasShadowBranches(t *testing.T, dir string)

AssertHasShadowBranches asserts that at least one shadow branch (entire/*) exists, excluding entire/checkpoints/*. Use this when the shadow branch is expected to persist (e.g., session is still idle).

func AssertNewCommits 

func AssertNewCommits(t *testing.T, s *RepoState, atLeast int)

AssertNewCommits polls until at least `atLeast` new commits exist since setup, or fails after 20 seconds. Polling handles the race where an interactive agent's prompt pattern appears before its git commit lands on disk.

func AssertNoCheckpointTrailer 

func AssertNoCheckpointTrailer(t *testing.T, dir string, ref string)

AssertNoCheckpointTrailer asserts the commit does NOT have an Entire-Checkpoint trailer.

func AssertNoShadowBranches 

func AssertNoShadowBranches(t *testing.T, dir string)

AssertNoShadowBranches asserts that no shadow branches (entire/*) remain, excluding entire/checkpoints/*. Shadow branches should be cleaned up after commits condense session data to the metadata branch.

func CaptureArtifacts 

func CaptureArtifacts(t *testing.T, s *RepoState)

CaptureArtifacts captures git state, checkpoint metadata, entire logs, and console output to the artifact directory.

func CheckpointIDs 

func CheckpointIDs(t *testing.T, dir string) []string

CheckpointIDs lists all checkpoint IDs from the tree at the tip of the checkpoint branch. It parses the two-level directory structure ({prefix}/{suffix}/metadata.json) and returns the concatenated IDs.

func CheckpointPath 

func CheckpointPath(id string) string

func ForEachAgent 

func ForEachAgent(t *testing.T, timeout time.Duration, fn func(t *testing.T, s *RepoState, ctx context.Context))

ForEachAgent runs fn as a parallel subtest for every registered agent. It handles repo setup, concurrency gating, context timeout, and cleanup. The timeout is scaled by each agent's TimeoutMultiplier.

If RunPrompt detects a transient API error (e.g. rate limit, token refresh failure), it panics with errScenarioRestart. ForEachAgent recovers from the panic and restarts the entire scenario with a fresh repository, up to maxScenarioRestarts times. This avoids stale CLI session state from the failed attempt poisoning the retry.

func GetCheckpointTrailer 

func GetCheckpointTrailer(t *testing.T, dir string, ref string) string

GetCheckpointTrailer extracts the Entire-Checkpoint trailer value from a code commit. Returns the trimmed trailer value, or an empty string if the trailer is not present.

func Git 

func Git(t *testing.T, dir string, args ...string)

Git runs a git command in the given directory and fails the test if it returns a non-zero exit code.

func GitOutput 

func GitOutput(t *testing.T, dir string, args ...string) string

GitOutput runs a git command in the given directory, returns its trimmed stdout, and fails the test on error.

func GitOutputErr 

func GitOutputErr(dir string, args ...string) (string, error)

GitOutputErr runs a git command and returns (output, error) without failing the test. For commands expected to fail.

func NewCheckpointCommits 

func NewCheckpointCommits(t *testing.T, s *RepoState) []string

NewCheckpointCommits returns the SHAs of commits added to the entire/checkpoints/v1 branch since the test was set up, oldest first.

func PatchSettings 

func PatchSettings(t *testing.T, dir string, extra map[string]any)

PatchSettings merges extra keys into .entire/settings.json.

func SetRunDir 

func SetRunDir(dir string)

SetRunDir overrides the artifact run directory (e.g. from E2E_ARTIFACT_DIR).

func SetupBareRemote 

func SetupBareRemote(t *testing.T, s *RepoState) string

SetupBareRemote creates a bare git repo, adds it as "origin", and pushes the initial commit. Returns the bare repo path.

func ValidateCheckpointDeep 

func ValidateCheckpointDeep(t *testing.T, dir string, v DeepCheckpointValidation)

ValidateCheckpointDeep performs comprehensive validation of checkpoint metadata on the checkpoint branch, including transcript JSONL validity, content hash verification, and prompt content checking.

func WaitForCheckpoint 

func WaitForCheckpoint(t *testing.T, s *RepoState, timeout time.Duration)

WaitForCheckpoint polls until the checkpoint branch advances from its initial state, or fails the test after timeout. Use this before any assertions that depend on the checkpoint branch (post-commit hook is async).

func WaitForCheckpointAdvanceFrom 

func WaitForCheckpointAdvanceFrom(t *testing.T, dir string, fromRef string, timeout time.Duration)

WaitForCheckpointAdvanceFrom polls until the checkpoint branch advances from the given ref, or fails the test after timeout. Use this when waiting for a second (or subsequent) checkpoint after recording the branch position.

func WaitForFileExists 

func WaitForFileExists(t *testing.T, dir string, glob string, timeout time.Duration)

WaitForFileExists polls until at least one file matches the glob pattern relative to dir, or fails the test after timeout. Handles the race where an interactive agent's prompt pattern appears before file writes land on disk.

func WaitForSessionIdle 

func WaitForSessionIdle(t *testing.T, dir string, timeout time.Duration)

WaitForSessionIdle polls the session state files in .git/entire-sessions/ until no session has phase "active", or fails the test after timeout. This handles the race where an agent's prompt pattern appears in the TUI before the turn-end hook has completed (transitioning ACTIVE → IDLE).

Types

type Attribution 

type Attribution struct {
	CalculatedAt    time.Time `json:"calculated_at"`
	AgentLines      int       `json:"agent_lines"`
	HumanAdded      int       `json:"human_added"`
	HumanModified   int       `json:"human_modified"`
	HumanRemoved    int       `json:"human_removed"`
	TotalCommitted  int       `json:"total_committed"`
	AgentPercentage float64   `json:"agent_percentage"`
}

type CheckpointMetadata 

type CheckpointMetadata struct {
	CLIVersion       string       `json:"cli_version"`
	CheckpointID     string       `json:"checkpoint_id"`
	Strategy         string       `json:"strategy"`
	Branch           string       `json:"branch"`
	CheckpointsCount int          `json:"checkpoints_count"`
	FilesTouched     []string     `json:"files_touched"`
	Sessions         []SessionRef `json:"sessions"`
	TokenUsage       TokenUsage   `json:"token_usage"`
}

func ReadCheckpointMetadata 

func ReadCheckpointMetadata(t *testing.T, dir string, checkpointID string) CheckpointMetadata

ReadCheckpointMetadata reads the checkpoint-level metadata.json from the tip of the checkpoint branch for the given checkpoint ID.

type DeepCheckpointValidation 

type DeepCheckpointValidation struct {
	CheckpointID              string
	Strategy                  string
	FilesTouched              []string
	ExpectedPrompts           []string
	ExpectedTranscriptContent []string
}

DeepCheckpointValidation contains expected values for comprehensive checkpoint validation.

type RepoState 

type RepoState struct {
	Agent            agents.Agent
	Dir              string
	ArtifactDir      string
	HeadBefore       string
	CheckpointBefore string
	ConsoleLog       *os.File
	// contains filtered or unexported fields
}

RepoState holds the working state for a single test's cloned repository.

func SetupRepo 

func SetupRepo(t *testing.T, agent agents.Agent) *RepoState

SetupRepo creates a fresh git repository in a temporary directory, seeds it with an initial commit, and runs `entire enable` for the given agent. Artifact capture is registered as a cleanup function.

When E2E_KEEP_REPOS is set, the temporary directory is not cleaned up so it can be inspected after the test. A symlink in the artifact dir points to the preserved repo.

func (*RepoState) Git 

func (s *RepoState) Git(t *testing.T, args ...string)

Git runs a git command in the repo and logs it to ConsoleLog.

func (*RepoState) IsExternalAgent added in v0.5.1 

func (s *RepoState) IsExternalAgent() bool

IsExternalAgent returns true if the agent implements the ExternalAgent interface and reports itself as external.

func (*RepoState) RunPrompt 

func (s *RepoState) RunPrompt(t *testing.T, ctx context.Context, prompt string, opts ...agents.Option) (agents.Output, error)

RunPrompt runs an agent prompt, logs the command and output to ConsoleLog, and returns the result. If the agent reports a transient API error, it panics with errScenarioRestart to trigger a full scenario restart in ForEachAgent (see runScenario).

func (*RepoState) Send 

func (s *RepoState) Send(t *testing.T, session agents.Session, input string)

Send sends input to an interactive session and logs it to ConsoleLog. Fails the test on error.

func (*RepoState) StartSession 

func (s *RepoState) StartSession(t *testing.T, ctx context.Context) agents.Session

StartSession starts an interactive session and registers it for pane capture in artifacts. Returns nil if the agent does not support interactive mode. The session is closed automatically during test cleanup.

func (*RepoState) WaitFor 

func (s *RepoState) WaitFor(t *testing.T, session agents.Session, pattern string, timeout time.Duration)

WaitFor waits for a pattern in the interactive session's pane and logs the pane content to ConsoleLog after the wait completes (success or failure).

type SessionMetadata 

type SessionMetadata struct {
	CLIVersion         string      `json:"cli_version"`
	CheckpointID       string      `json:"checkpoint_id"`
	SessionID          string      `json:"session_id"`
	Strategy           string      `json:"strategy"`
	CreatedAt          time.Time   `json:"created_at"`
	Branch             string      `json:"branch"`
	Agent              string      `json:"agent"`
	CheckpointsCount   int         `json:"checkpoints_count"`
	FilesTouched       []string    `json:"files_touched"`
	TokenUsage         TokenUsage  `json:"token_usage"`
	InitialAttribution Attribution `json:"initial_attribution"`
	TranscriptPath     string      `json:"transcript_path"`
}

func ReadSessionMetadata 

func ReadSessionMetadata(t *testing.T, dir string, checkpointID string, sessionIndex int) SessionMetadata

ReadSessionMetadata reads a session's metadata.json from the tip of the checkpoint branch for the given checkpoint ID and session index.

func WaitForSessionMetadata 

func WaitForSessionMetadata(t *testing.T, dir string, checkpointID string, sessionIndex int, timeout time.Duration) SessionMetadata

WaitForSessionMetadata polls until session metadata exists on the checkpoint branch for the given checkpoint ID and session index, then returns it. This handles the race where the checkpoint branch advances before session metadata is fully committed.

type SessionRef 

type SessionRef struct {
	Metadata    string `json:"metadata"`
	Transcript  string `json:"transcript"`
	Context     string `json:"context"`
	ContentHash string `json:"content_hash"`
	Prompt      string `json:"prompt"`
}

type TokenUsage 

type TokenUsage struct {
	InputTokens         int `json:"input_tokens"`
	CacheCreationTokens int `json:"cache_creation_tokens"`
	CacheReadTokens     int `json:"cache_read_tokens"`
	OutputTokens        int `json:"output_tokens"`
	APICallCount        int `json:"api_call_count"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.