testutil

package
v0.6.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 18, 2026 License: MIT Imports: 30 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ArtifactRoot string

ArtifactRoot is the absolute path to the artifact output directory. Must be set in TestMain before any tests run.

View Source 
var ArtifactTimestamp = time.Now().Format("2006-01-02T15-04-05")

ArtifactTimestamp is the timestamp subdirectory for this test run.

Functions

func ApplySuiteCheckpointsMode added in v0.6.0 

func ApplySuiteCheckpointsMode(t *testing.T, dir string)

ApplySuiteCheckpointsMode configures an arbitrary repo for the current suite-wide E2E checkpoints mode. Useful for repos created outside SetupRepo, such as fresh clones in remote-resume scenarios.

func ArtifactRunDir 

func ArtifactRunDir() string

ArtifactRunDir returns the directory for the current test run.

func AssertCheckpointAdvanced 

func AssertCheckpointAdvanced(t *testing.T, s *RepoState)

AssertCheckpointAdvanced asserts the checkpoint branch moved forward.

func AssertCheckpointExists 

func AssertCheckpointExists(t *testing.T, dir string, checkpointID string)

AssertCheckpointExists asserts that the checkpoint ID is mentioned on the checkpoint branch and that its metadata.json exists in the tree.

func AssertCheckpointFilesTouched 

func AssertCheckpointFilesTouched(t *testing.T, dir string, checkpointID string, expected []string)

AssertCheckpointFilesTouched asserts the checkpoint metadata lists exactly the expected files in files_touched (order-independent).

func AssertCheckpointFilesTouchedContains 

func AssertCheckpointFilesTouchedContains(t *testing.T, dir string, checkpointID string, file string)

AssertCheckpointFilesTouchedContains asserts files_touched contains a specific file path (subset check, unlike the exact-match AssertCheckpointFilesTouched).

func AssertCheckpointHasSingleSession 

func AssertCheckpointHasSingleSession(t *testing.T, dir string, checkpointID string)

AssertCheckpointHasSingleSession asserts checkpoint metadata has exactly one session.

func AssertCheckpointIDFormat 

func AssertCheckpointIDFormat(t *testing.T, checkpointID string)

AssertCheckpointIDFormat asserts the checkpoint ID is 12 lowercase hex chars.

func AssertCheckpointInLastN 

func AssertCheckpointInLastN(t *testing.T, dir string, checkpointID string, n int)

AssertCheckpointInLastN asserts the given checkpoint ID appears in at least n commits on the checkpoint branch (e.g. initial + catchup). Uses --grep to find matching commits regardless of position, so extra commits from multi-commit agent turns don't cause false failures.

func AssertCheckpointMetadataComplete 

func AssertCheckpointMetadataComplete(t *testing.T, dir string, checkpointID string)

AssertCheckpointMetadataComplete asserts essential fields in checkpoint metadata are populated.

func AssertCheckpointNotAdvanced 

func AssertCheckpointNotAdvanced(t *testing.T, s *RepoState)

AssertCheckpointNotAdvanced asserts the checkpoint branch has NOT moved.

func AssertCommitLinkedToCheckpoint 

func AssertCommitLinkedToCheckpoint(t *testing.T, dir string, ref string)

AssertCommitLinkedToCheckpoint asserts the trailer exists AND the checkpoint data exists on the checkpoint branch.

func AssertConsoleLogDoesNotContain added in v0.5.6 

func AssertConsoleLogDoesNotContain(t *testing.T, s *RepoState, forbidden ...string)

AssertConsoleLogDoesNotContain asserts that the current test's console.log does not contain any of the forbidden substrings.

func AssertDistinctSessions 

func AssertDistinctSessions(t *testing.T, dir string, checkpointIDs []string)

AssertDistinctSessions asserts session metadata across checkpoints has unique session IDs.

func AssertFileExists 

func AssertFileExists(t *testing.T, dir string, glob string)

AssertFileExists asserts that at least one file matches the glob pattern relative to dir.

func AssertHasCheckpointTrailer 

func AssertHasCheckpointTrailer(t *testing.T, dir string, ref string) string

AssertHasCheckpointTrailer asserts the commit has an Entire-Checkpoint trailer, validates its format, and returns its value.

func AssertHasShadowBranches 

func AssertHasShadowBranches(t *testing.T, dir string)

AssertHasShadowBranches asserts that at least one shadow branch (entire/*) exists, excluding entire/checkpoints/*. Use this when the shadow branch is expected to persist (e.g., session is still idle).

func AssertNewCommits 

func AssertNewCommits(t *testing.T, s *RepoState, atLeast int)

AssertNewCommits polls until at least `atLeast` new commits exist since setup, or fails after 20 seconds. Polling handles the race where an interactive agent's prompt pattern appears before its git commit lands on disk.

func AssertNewCommitsWithTimeout added in v0.5.2 

func AssertNewCommitsWithTimeout(t *testing.T, s *RepoState, atLeast int, timeout time.Duration)

AssertNewCommitsWithTimeout is like AssertNewCommits but with a configurable timeout. Use this when WaitFor may settle on stale pane content (e.g. multi-turn interactive tests where the previous turn's prompt is still visible), giving the agent more time to complete its commit.

func AssertNoCheckpointTrailer 

func AssertNoCheckpointTrailer(t *testing.T, dir string, ref string)

AssertNoCheckpointTrailer asserts the commit does NOT have an Entire-Checkpoint trailer.

func CaptureArtifacts 

func CaptureArtifacts(t *testing.T, s *RepoState)

CaptureArtifacts captures git state, checkpoint metadata, entire logs, and console output to the artifact directory.

func CheckpointIDs 

func CheckpointIDs(t *testing.T, dir string) []string

CheckpointIDs lists all checkpoint IDs from the tree at the tip of the checkpoint branch. It parses the two-level directory structure ({prefix}/{suffix}/metadata.json) and returns the concatenated IDs.

func CheckpointPath 

func CheckpointPath(id string) string

func CheckpointVerifyRef added in v0.6.0 

func CheckpointVerifyRef() string

CheckpointVerifyRef returns the exact local metadata ref name tests should use for presence checks such as rev-parse --verify.

func CheckpointsMode added in v0.6.0 

func CheckpointsMode() string

CheckpointsMode returns the active E2E checkpoints mode from E2E_CHECKPOINTS_MODE, defaulting to "legacy".

func CloneAndEnableEntire added in v0.6.0 

func CloneAndEnableEntire(t *testing.T, bareDir string, agentName string) string

CloneAndEnableEntire clones a bare remote into a fresh temp dir, configures a test git identity, enables Entire for the given agent, applies the active suite checkpoints mode, and commits the enable changes if needed.

func CommitIfDirty added in v0.6.0 

func CommitIfDirty(t *testing.T, dir string, message string)

CommitIfDirty stages all changes and creates a commit only when the worktree or index is non-empty. This keeps cloned-repo E2E tests robust when `entire enable` is already idempotent and produces no repo changes.

func CurrentCheckpointRef added in v0.6.0 

func CurrentCheckpointRef(t *testing.T, dir string) string

CurrentCheckpointRef returns the current hash of the checkpoint ref used for reads in the active suite mode. It fails if the ref does not exist.

func EmptyDir added in v0.5.2 

func EmptyDir(t *testing.T) string

EmptyDir returns the path to an empty temporary directory, cleaned up when the test finishes. Useful as a cross-platform replacement for /dev/null in git config paths like core.hooksPath (git on Windows cannot open NUL).

func EnableCheckpointsV2 added in v0.6.0 

func EnableCheckpointsV2(t *testing.T, dir string)

EnableCheckpointsV2 switches an E2E repo into v2 dual-write mode while preserving existing strategy_options written during setup.

func EnableCheckpointsVersion2 added in v0.6.0 

func EnableCheckpointsVersion2(t *testing.T, dir string)

EnableCheckpointsVersion2 switches an E2E repo into strict v2-only mode.

func ForEachAgent 

func ForEachAgent(t *testing.T, timeout time.Duration, fn func(t *testing.T, s *RepoState, ctx context.Context))

ForEachAgent runs fn as a parallel subtest for every registered agent. It handles repo setup, concurrency gating, context timeout, and cleanup. The timeout is scaled by each agent's TimeoutMultiplier.

If RunPrompt detects a transient API error (e.g. rate limit, token refresh failure), it panics with errScenarioRestart. ForEachAgent recovers from the panic and restarts the entire scenario with a fresh repository, up to maxScenarioRestarts times. This avoids stale CLI session state from the failed attempt poisoning the retry.

func ForEachNamedAgent added in v0.6.0 

func ForEachNamedAgent(t *testing.T, timeout time.Duration, names []string, fn func(t *testing.T, s *RepoState, ctx context.Context))

ForEachNamedAgent runs fn as a parallel subtest for each registered agent whose name matches one of the provided names.

func GetCheckpointTrailer 

func GetCheckpointTrailer(t *testing.T, dir string, ref string) string

GetCheckpointTrailer extracts the Entire-Checkpoint trailer value from a code commit. Returns the trimmed trailer value, or an empty string if the trailer is not present.

func Git 

func Git(t *testing.T, dir string, args ...string)

Git runs a git command in the given directory and fails the test if it returns a non-zero exit code.

func GitOutput 

func GitOutput(t *testing.T, dir string, args ...string) string

GitOutput runs a git command in the given directory, returns its trimmed stdout, and fails the test on error.

func GitOutputErr 

func GitOutputErr(dir string, args ...string) (string, error)

GitOutputErr runs a git command and returns (output, error) without failing the test. For commands expected to fail.

func NewCheckpointCommits 

func NewCheckpointCommits(t *testing.T, s *RepoState) []string

NewCheckpointCommits returns the SHAs of commits added to the entire/checkpoints/v1 branch since the test was set up, oldest first.

func PatchSettings 

func PatchSettings(t *testing.T, dir string, extra map[string]any)

PatchSettings merges extra keys into .entire/settings.json.

func PushCheckpointRefs added in v0.6.0 

func PushCheckpointRefs(t *testing.T, dir string)

PushCheckpointRefs pushes the checkpoint refs used by the active suite mode to the origin remote. Remote-resume tests use this instead of hardcoding legacy v1 branch pushes.

func RestoredSessionTranscriptPath added in v0.6.0 

func RestoredSessionTranscriptPath(t *testing.T, repoDir string, meta SessionMetadata) (string, bool)

RestoredSessionTranscriptPath returns the path where resume should restore the transcript for agents that use file-backed restored sessions.

func SetRunDir 

func SetRunDir(dir string)

SetRunDir overrides the artifact run directory (e.g. from E2E_ARTIFACT_DIR).

func SetupBareRemote 

func SetupBareRemote(t *testing.T, s *RepoState) string

SetupBareRemote creates a bare git repo, adds it as "origin", and pushes the initial commit. Returns the bare repo path.

func ShadowBranches added in v0.6.0 

func ShadowBranches(t *testing.T, dir string) []string

ShadowBranches returns all shadow branches (entire/*) excluding entire/checkpoints/*.

func ValidateCheckpointDeep 

func ValidateCheckpointDeep(t *testing.T, dir string, v DeepCheckpointValidation)

ValidateCheckpointDeep performs comprehensive validation of checkpoint metadata on the checkpoint branch, including transcript JSONL validity, content hash verification, and prompt content checking.

func WaitForCheckpoint 

func WaitForCheckpoint(t *testing.T, s *RepoState, timeout time.Duration)

WaitForCheckpoint polls until the checkpoint branch advances from its initial state, or fails the test after timeout. Use this before any assertions that depend on the checkpoint branch (post-commit hook is async).

func WaitForCheckpointAdvanceFrom 

func WaitForCheckpointAdvanceFrom(t *testing.T, dir string, fromRef string, timeout time.Duration)

WaitForCheckpointAdvanceFrom polls until the checkpoint branch advances from the given ref, or fails the test after timeout. Use this when waiting for a second (or subsequent) checkpoint after recording the branch position.

func WaitForCheckpointExists added in v0.5.2 

func WaitForCheckpointExists(t *testing.T, dir string, checkpointID string, timeout time.Duration)

WaitForCheckpointExists polls until the checkpoint ID appears on the checkpoint branch and its metadata.json is readable, or fails after timeout.

func WaitForFileExists 

func WaitForFileExists(t *testing.T, dir string, glob string, timeout time.Duration)

WaitForFileExists polls until at least one file matches the glob pattern relative to dir, or fails the test after timeout. Handles the race where an interactive agent's prompt pattern appears before file writes land on disk.

func WaitForNoShadowBranches added in v0.5.2 

func WaitForNoShadowBranches(t *testing.T, dir string, timeout time.Duration)

WaitForNoShadowBranches polls until all shadow branches are cleaned up or the timeout expires. Shadow branch cleanup can lag slightly behind checkpoint condensation (carry-forward creates intermediate branches that are deleted asynchronously).

func WaitForSessionIdle 

func WaitForSessionIdle(t *testing.T, dir string, timeout time.Duration)

WaitForSessionIdle polls the session state files in .git/entire-sessions/ until no session has phase "active", or fails the test after timeout. This handles the race where an agent's prompt pattern appears in the TUI before the turn-end hook has completed (transitioning ACTIVE → IDLE).

Types

type Attribution 

type Attribution struct {
	CalculatedAt    time.Time `json:"calculated_at"`
	AgentLines      int       `json:"agent_lines"`
	HumanAdded      int       `json:"human_added"`
	HumanModified   int       `json:"human_modified"`
	HumanRemoved    int       `json:"human_removed"`
	TotalCommitted  int       `json:"total_committed"`
	AgentPercentage float64   `json:"agent_percentage"`
}

type CheckpointMetadata 

type CheckpointMetadata struct {
	CLIVersion       string       `json:"cli_version"`
	CheckpointID     string       `json:"checkpoint_id"`
	Strategy         string       `json:"strategy"`
	Branch           string       `json:"branch"`
	CheckpointsCount int          `json:"checkpoints_count"`
	FilesTouched     []string     `json:"files_touched"`
	Sessions         []SessionRef `json:"sessions"`
	TokenUsage       TokenUsage   `json:"token_usage"`
}

func ReadCheckpointMetadata 

func ReadCheckpointMetadata(t *testing.T, dir string, checkpointID string) CheckpointMetadata

ReadCheckpointMetadata reads the checkpoint-level metadata.json from the tip of the checkpoint branch for the given checkpoint ID.

type DeepCheckpointValidation 

type DeepCheckpointValidation struct {
	CheckpointID              string
	Strategy                  string
	FilesTouched              []string
	ExpectedPrompts           []string
	ExpectedTranscriptContent []string
}

DeepCheckpointValidation contains expected values for comprehensive checkpoint validation.

type RepoState 

type RepoState struct {
	Agent            agents.Agent
	Dir              string
	ArtifactDir      string
	HeadBefore       string
	CheckpointBefore string
	ConsoleLog       *os.File
	// contains filtered or unexported fields
}

RepoState holds the working state for a single test's cloned repository.

func SetupRepo 

func SetupRepo(t *testing.T, agent agents.Agent) *RepoState

SetupRepo creates a fresh git repository in a temporary directory, seeds it with an initial commit, and runs `entire enable` for the given agent. Artifact capture is registered as a cleanup function.

When E2E_KEEP_REPOS is set, the temporary directory is not cleaned up so it can be inspected after the test. A symlink in the artifact dir points to the preserved repo.

func (*RepoState) Git 

func (s *RepoState) Git(t *testing.T, args ...string)

Git runs a git command in the repo and logs it to ConsoleLog.

func (*RepoState) IsExternalAgent added in v0.5.1 

func (s *RepoState) IsExternalAgent() bool

IsExternalAgent returns true if the agent implements the ExternalAgent interface and reports itself as external.

func (*RepoState) RunPrompt 

func (s *RepoState) RunPrompt(t *testing.T, ctx context.Context, prompt string, opts ...agents.Option) (agents.Output, error)

RunPrompt runs an agent prompt, logs the command and output to ConsoleLog, and returns the result. If the agent reports a transient API error, it panics with errScenarioRestart to trigger a full scenario restart in ForEachAgent (see runScenario).

func (*RepoState) Send 

func (s *RepoState) Send(t *testing.T, session agents.Session, input string)

Send sends input to an interactive session and logs it to ConsoleLog. Fails the test on error.

func (*RepoState) StartSession 

func (s *RepoState) StartSession(t *testing.T, ctx context.Context) agents.Session

StartSession starts an interactive session and registers it for pane capture in artifacts. Returns nil if the agent does not support interactive mode. The session is closed automatically during test cleanup.

func (*RepoState) WaitFor 

func (s *RepoState) WaitFor(t *testing.T, session agents.Session, pattern string, timeout time.Duration)

WaitFor waits for a pattern in the interactive session's pane and logs the pane content to ConsoleLog after the wait completes (success or failure).

type SessionMetadata 

type SessionMetadata struct {
	CLIVersion         string      `json:"cli_version"`
	CheckpointID       string      `json:"checkpoint_id"`
	SessionID          string      `json:"session_id"`
	Strategy           string      `json:"strategy"`
	CreatedAt          time.Time   `json:"created_at"`
	Branch             string      `json:"branch"`
	Agent              string      `json:"agent"`
	Model              string      `json:"model"`
	CheckpointsCount   int         `json:"checkpoints_count"`
	FilesTouched       []string    `json:"files_touched"`
	TokenUsage         TokenUsage  `json:"token_usage"`
	InitialAttribution Attribution `json:"initial_attribution"`
	TranscriptPath     string      `json:"transcript_path"`

	// CheckpointTranscriptStart is the transcript.jsonl line offset where this
	// checkpoint's contributions begin. For the first checkpoint in a session
	// it's 0; for subsequent checkpoints it must point past prior content so
	// consumers (explain, attribution, summaries) can scope the transcript
	// correctly. omitempty in the on-wire form, so we read 0 when absent.
	CheckpointTranscriptStart int `json:"checkpoint_transcript_start,omitempty"`
}

func ReadSessionMetadata 

func ReadSessionMetadata(t *testing.T, dir string, checkpointID string, sessionIndex int) SessionMetadata

ReadSessionMetadata reads a session's metadata.json from the tip of the checkpoint branch for the given checkpoint ID and session index.

func WaitForSessionMetadata 

func WaitForSessionMetadata(t *testing.T, dir string, checkpointID string, sessionIndex int, timeout time.Duration) SessionMetadata

WaitForSessionMetadata polls until session metadata exists on the checkpoint branch for the given checkpoint ID and session index, then returns it. This handles the race where the checkpoint branch advances before session metadata is fully committed.

type SessionRef 

type SessionRef struct {
	Metadata    string `json:"metadata"`
	Transcript  string `json:"transcript"`
	Context     string `json:"context"`
	ContentHash string `json:"content_hash"`
	Prompt      string `json:"prompt"`
}

type TokenUsage 

type TokenUsage struct {
	InputTokens         int `json:"input_tokens"`
	CacheCreationTokens int `json:"cache_creation_tokens"`
	CacheReadTokens     int `json:"cache_read_tokens"`
	OutputTokens        int `json:"output_tokens"`
	APICallCount        int `json:"api_call_count"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.