explore

package
v0.2.0-beta.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 4, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Overview

Package explore implements autonomous AI-driven exploratory testing of mobile applications on top of the platform-neutral device driver surface.

The package is peripheral, like internal/aiengine: it depends on internal/device (Driver, TreeNode) and internal/model (command keywords for flow export), and is wired together only in internal/cli. It must never be imported by internal/engine, internal/device, or internal/capability.

The workflow is deterministic at the strategic level and AI-driven at the tactical level: an exploration session observes the current screen, researches it into a UI map, plans prioritized scenarios, executes them step by step through the driver, verifies outcomes, reports findings, and exports passing runs as runnable flow YAML. Learned per-screen recipes and operator-authored hints persist across sessions.

Index

Constants

View Source
const MaskedText = "***"

MaskedText replaces recorded input that may have landed in a secure field. Recorders write it, replay refuses to execute it, and export parameterizes it — a typed secret never survives in any artifact.

Variables

View Source
var ErrBudgetExhausted = errors.New("explore: test budget exhausted")

ErrBudgetExhausted reports that the session test budget was spent.

View Source
var ErrDeviceUnreachable = errors.New("explore: device unreachable")

ErrDeviceUnreachable is returned (wrapped) by a tool handler when the driver's transport is gone -- the runner process died, the socket refuses. It ends the loop as an error: every further action would fail the same way, and a model told "tool failed" spends its whole budget on waits and retries against a dead endpoint (seen live, 2026-08-28).

View Source
var ErrNoAIProvider = errors.New("explore: no AI provider configured")

ErrNoAIProvider reports that exploration cannot start because no AI provider is configured. Exploration is AI-driven and fails closed.

View Source
var ErrScreenUnobservable = errors.New("explore: screen cannot be observed")

ErrScreenUnobservable is returned (wrapped) by a tool handler when the driver answered the observation with a failure it says will repeat -- the app under test left the foreground, say. It ends the loop for the same reason ErrDeviceUnreachable does, but the device is fine and the cause must not be reported as an unreachable one. The tester has no tool that restores an app to the foreground: seen live 2026-08-31, a run spent ten of fifteen steps on waits, taps and swipes that all failed with the same message.

View Source
var ErrStopRequested = errors.New("explore: stop requested")

ErrStopRequested is returned by a tool handler to end the loop cleanly.

View Source
var ErrUnreadableReply = errors.New("unreadable model reply")

ErrUnreadableReply marks a failure to decode what a model sent, as opposed to a failure to reach it at all. Callers that fold both into one message -- the outcome judge writes the reason into the report -- need to tell an operator which of the two happened.

Functions

func ControlLabel

func ControlLabel(node device.TreeNode) string

ControlLabel names a row for a model-facing table. iOS answers with an element's accessibility VALUE in text and its NAME in accessibilityText, and ElementLabel prefers text -- so a switch read "1" and the month strip of a calendar read "No events" thirty-one times, one name between every day on screen. Android puts a widget's name in text and has no such split, so no Android row moves. Nothing matches on this: selectors, locators, and exported flows go on reading ElementLabel.

func DecodeReply

func DecodeReply(text string, target any) error

func ElementBounds

func ElementBounds(node device.TreeNode) (device.Bounds, bool)

ElementBounds returns bounds a caller can act on: parsed, and enclosing a real area. A box of zero width and height parses cleanly and centers on the screen corner, so reporting it as usable sent every consumer somewhere the element is not -- a tap to (0,0), an exported point locator aimed there. Measured on iOS 26.2: 12 of 18 rows on one captured screen were such boxes.

func ElementLabel

func ElementLabel(node device.TreeNode) string

ElementLabel returns the human-visible label of a node across both dialects: Android carries it in text (or a hint), iOS in accessibilityText or title. A table that reads only the Android keys shows every iOS row as "-", which left a live researcher with nothing but geometry to name the screen's controls by.

func ElementRole

func ElementRole(node device.TreeNode) string

ElementRole names what kind of control a node is, in the vocabulary a model can use: the Android class without its package, the iOS element type by name, or the raw type value when neither applies.

func IsCheckable

func IsCheckable(node device.TreeNode) bool

IsCheckable reports whether this element has an on/off state at all. Android names the widget in class; iOS carries a numeric element type.

func IsSecureTextInput

func IsSecureTextInput(node device.TreeNode) bool

IsSecureTextInput reports whether this element masks what is typed into it: an iOS secure text field or an Android password input. What lands in such a field is a secret by declaration and must never reach recordings.

func IsTextInput

func IsTextInput(node device.TreeNode) bool

IsTextInput reports whether typed text can land in this element once it holds keyboard focus. Android names the widget in class; iOS carries a numeric element type.

func PointLocator

func PointLocator(point device.Point) string

PointLocator answers the point locator value for a screen coordinate in the "x,y" form the engine's tapOn point accepts: two base-10 integers. A half-pixel center floors rather than rounds, so the point stays inside the element's own bounds and never lands one pixel past an edge of the screen.

func PriorityRank

func PriorityRank(p Priority) int

PriorityRank returns a sortable rank, lower is more urgent. Unknown priorities rank after all known ones.

func RowState

func RowState(node device.TreeNode) string

RowState says how a row is set, for a model-facing table: whether a control with an on/off state is on, and whether the platform marks this row as the selected one. Empty for a row with neither, which is nearly all of them -- iOS reports checked and selected false for every element on screen, and marking those would call every button an unselected switch.

The selected mark carries the calendar's own day: of the 125 nodes of a Calendar day view captured on iOS 26.2, exactly one is flagged, and two sessions filed a defect whose evidence was that no fact said which day the screen was showing.

func RowValue

func RowValue(node device.TreeNode) string

RowValue answers the value a row carries when that is not already its name. iOS splits the two -- the name in accessibilityText, the value in text -- and naming a row by its name would otherwise drop the value entirely: a calendar day is picked on "1 event" against "No events", which is nowhere else on the screen. Android puts the name in text and has no split, so this is empty there. A text input is left out: the table says what it HOLDS, and a secure one says nothing.

func ScreenKeyWords

func ScreenKeyWords(key string) string

ScreenKeyWords renders the readable part of a screen key: the salient labels Key() slugified, with the digest dropped and the dashes turned back into spaces. A key that is nothing but a digest has no readable part and answers empty.

A salient label that slugifies to eight hex characters loses that one word. Nothing keys on this -- it names a screen for a model and decides whether two keys mean the same screen, never which file to read.

func StepLine

func StepLine(step StepRecord) string

StepLine renders one executed step as a single line: the action, its text or direction, the status, whether the screen changed, and any note or error.

The supervisor prompt and the written session log share this rendering, so an operator reading the artifact sees the run exactly as the supervisor did.

func StepLines

func StepLines(steps []StepRecord) []string

StepLines renders every step of a run. An empty run says so rather than answering with nothing, because a caller that prints the lines would otherwise show a heading with no body.

func Truncate

func Truncate(value string, max int) string

Truncate shortens a value to max characters, marking the cut with an ellipsis. It counts runes rather than bytes: a byte offset lands inside a multi-byte character for any label the device shows in a non-Latin script, and %q then escapes the broken tail into \xNN, so the corruption reaches the artifact looking like data.

Every consumer of device text shortens it somewhere -- the element table the model reads, the step log, the session report, a failure cause -- and each one grew its own copy of this until they were four. One is enough.

func UnfencedJSON

func UnfencedJSON(text string) string

UnfencedJSON strips the markdown code fence real models wrap around a JSON-only reply even when told not to (proven live with gpt-4o on 2026-08-11). Bare replies pass through untouched; the decode after it stays strict, so tolerance stops at the wrapper.

Types

type Action

type Action struct {
	Kind ActionKind
	// Target locates the element the action addresses, nil for
	// screen-level actions (back, swipe by direction, launch).
	Target *Locator
	// Text is input text, key name, link, or assertion text by kind.
	Text string
	// Direction is up/down/left/right for swipe and scroll kinds.
	Direction string
	// Masked marks an input whose real text was withheld because it may
	// have landed in a secure field; Text then carries MaskedText. The
	// flag, not the text, is the signal — a user can type a literal "***".
	Masked bool
}

Action is one concrete device interaction or check.

type ActionKind

type ActionKind string

ActionKind names one device action the tester can take. Kinds map onto flow command keywords at export time.

const (
	ActionTap       ActionKind = "tap"
	ActionLongPress ActionKind = "longPress"
	ActionInput     ActionKind = "input"
	ActionErase     ActionKind = "erase"
	ActionSwipe     ActionKind = "swipe"
	ActionScroll    ActionKind = "scroll"
	ActionBack      ActionKind = "back"
	ActionPressKey  ActionKind = "pressKey"
	ActionHideKeys  ActionKind = "hideKeyboard"
	ActionOpenLink  ActionKind = "openLink"
	ActionLaunch    ActionKind = "launchApp"
	ActionStop      ActionKind = "stopApp"
	ActionWait      ActionKind = "wait"
	ActionVerify    ActionKind = "verify"
)

Tester action kinds.

type Analyst

type Analyst interface {
	Report(ctx context.Context, report *SessionReport) (string, error)
}

Analyst renders a session report: findings clustered by root cause, product defects separated from automation problems.

type ChatRequest

type ChatRequest struct {
	Messages []Message
	Tools    []ToolSpec
	// ForceTool requires the model to call some tool this turn.
	ForceTool bool
	// MaxTokens caps the reply when positive.
	MaxTokens int
}

ChatRequest is one model invocation.

type ChatResponse

type ChatResponse struct {
	Message Message
	Usage   Usage
}

ChatResponse is the model reply plus usage accounting.

func ChatJSON

func ChatJSON(ctx context.Context, llm LLM, request ChatRequest, target any) (ChatResponse, error)

ChatJSON asks the model once, and asks again when the reply does not decode -- carrying the rejected reply and the error, since the model is the only one who can see what it cut off. A transport failure is returned as it is: that is a provider to wait on, not a reply to correct. A reply that decodes into the wrong answer is the model's answer and is returned too; only unreadable ones are worth a second call.

type Config

type Config struct {
	// AppID is the application under exploration (bundle id or package).
	AppID string
	// Platform is the selected execution platform token name
	// (android, ios, ios-physical, web).
	Platform string
	// StateDir is the per-app persistent directory holding knowledge,
	// learned recipes, plans, and research caches across sessions.
	StateDir string
	// OutputDir receives per-session artifacts: reports, exported flows,
	// screenshots, recordings.
	OutputDir string
	// MaxTests bounds how many scenarios the session may execute.
	// Zero means the caller must supply a positive budget; the runner
	// refuses a zero budget rather than defaulting to unlimited.
	MaxTests int
	// MaxStepsPerTest bounds the tester tool loop for one scenario.
	MaxStepsPerTest int
	// Styles are planning style names rotated across iterations.
	Styles []string
	// PilotEnabled turns on the supervisor conversation. When disabled,
	// scenario verdicts fall back to expected-outcome matching alone.
	PilotEnabled bool
	// RecordVideo starts a screen recording for the session when the
	// platform supports it.
	RecordVideo bool
	// SessionName tags entities the tester creates so reports and
	// cleanup can identify them.
	SessionName string
	// Clock supplies time; nil means real time.
	Clock func() time.Time
}

Config carries the resolved settings for one exploration session. internal/cli parses flags and environment into this struct.

func (Config) Now

func (c Config) Now() time.Time

Now returns the configured clock time, or wall time when unset.

type Crew

type Crew struct {
	Observer   Observer
	Researcher Researcher
	Planner    Planner
	Tester     Tester
	Navigator  Navigator
	Analyst    Analyst
	Exporter   Exporter
}

Crew bundles the role implementations for one exploration session.

type ExperienceStore

type ExperienceStore interface {
	// Index lists entry titles for a screen, cheap enough to inject
	// into every conversation.
	Index(ctx context.Context, screen ScreenSignature) ([]string, error)
	// Get fetches one entry body by title.
	Get(ctx context.Context, screen ScreenSignature, title string) (string, error)
	// Record appends or replaces an entry for a screen. Secrets must
	// be redacted before writing.
	Record(ctx context.Context, screen ScreenSignature, entry MemoryEntry) error
}

ExperienceStore persists machine-learned per-screen recipes: what worked, what failed, working locator solutions. Injected into agent conversations as a table of contents; bodies are fetched on demand.

type Exporter

type Exporter interface {
	ExportFlow(result *TestResult, appID string) ([]byte, error)
}

Exporter turns a finished run into a runnable flow YAML document. Implementations must round-trip the output through the flow parser before returning it.

type FlatElement

type FlatElement struct {
	EIDX int
	Node device.TreeNode
	// Path is the child-index chain from the root, e.g. "0/2/1".
	Path string
	// Depth is the nesting level, root = 0.
	Depth int
}

FlatElement is one interactive element of the flattened accessibility tree. EIDX is a machine-assigned index that joins the tree dump, the element table shown to models, and screenshot annotations.

func FlattenScreen

func FlattenScreen(root device.TreeNode) ([]FlatElement, error)

FlattenScreen lists the elements of a screen tree that agents interact with, assigning each a stable EIDX in document order. The same tree always yields the same indexes, so research maps and tester tools agree on element identity within one observation.

type KnowledgeStore

type KnowledgeStore interface {
	// Match returns the hint bodies that apply to a screen.
	Match(ctx context.Context, screen ScreenSignature) ([]string, error)
}

KnowledgeStore serves operator-authored hints: credentials pointers, form rules, navigation quirks. Matched to screens by pattern.

type LLM

type LLM interface {
	Chat(ctx context.Context, request ChatRequest) (ChatResponse, error)
}

LLM is the narrow chat seam this package needs. The langchaingo-backed implementation lives in internal/aiengine so the provider dependency stays quarantined there.

type Locator

type Locator struct {
	Kind  LocatorKind
	Value string
	// Index disambiguates when Value matches several elements.
	Index int
	// Label names the element for a human reader when Value cannot: a point
	// or a tree path says where, never what. Nothing matches on it and the
	// exporter never writes it -- an element reached by coordinate is reached
	// by coordinate on replay too.
	Label string
}

Locator is one way to find an element on a screen.

type LocatorKind

type LocatorKind string

LocatorKind orders the locator ladder: stable identifiers first, visible text second, tree path third, grid point last.

const (
	LocatorID    LocatorKind = "id"
	LocatorText  LocatorKind = "text"
	LocatorPath  LocatorKind = "path"
	LocatorPoint LocatorKind = "point"
)

Locator kinds, from most to least stable.

type LoopResult

type LoopResult struct {
	// Messages is the full conversation including tool turns.
	Messages []Message
	Usage    Usage
	// Stopped reports that a handler requested the stop.
	Stopped bool
	// Exhausted reports that the iteration bound was hit before the
	// model finished.
	Exhausted bool
}

LoopResult is the outcome of a bounded tool loop.

func RunToolLoop

func RunToolLoop(ctx context.Context, llm LLM, messages []Message, box ToolBox, maxIterations int) (LoopResult, error)

RunToolLoop drives one bounded agent conversation: invoke the model, execute requested tools, feed results back, and repeat until the model answers without tool calls, a handler requests a stop, the bound is hit, or the context ends. The input slice is never mutated.

type MappedElement

type MappedElement struct {
	EIDX     int
	Role     string
	Label    string
	Locators []Locator
	// Notes carries validation rules, data hints, or vision findings.
	Notes string
}

MappedElement is one interactive element in a researched UI map.

type MemoryEntry

type MemoryEntry struct {
	Title string
	Body  string
}

MemoryEntry is one titled note in a per-screen memory file.

type Message

type Message struct {
	Role       Role
	Text       string
	ImagePNG   []byte
	ToolCalls  []ToolCall
	ToolCallID string
}

Message is one turn in an agent conversation. Text and ImagePNG may both be set (multimodal user turns). Assistant turns may carry tool calls; tool turns answer exactly one call by ID.

type MissReason

type MissReason string

MissReason says why an expected outcome was not met. It is meaningful only on an unmet check, and its zero value is a product defect: a judge that cannot classify a miss must not talk the report out of reporting one.

const (
	// MissDefect: the app was expected to produce the outcome and did not.
	MissDefect MissReason = ""
	// MissUnpromised: the app has no such feature, or the screen cannot
	// express the expectation at all. Scenario wording, not a defect.
	MissUnpromised MissReason = "unpromised"
	// MissUnjudged: no verdict was reached, because the judge model failed,
	// answered unreadably, or said the facts it was given cannot decide the
	// question (an outcome about colour or shape, which no text table
	// carries). An automation problem, not a product one.
	MissUnjudged MissReason = "unjudged"
)

Reasons an expected outcome went unmet.

type ModelSet

type ModelSet struct {
	Worker  LLM
	Manager LLM
	Vision  LLM
}

ModelSet groups the role tiers. Cheap fast workers consume screen dumps; a smarter manager reads only short summaries; a vision-capable model reads screenshots. Any field may hold the same underlying model.

type Navigator interface {
	// EnsureReady prepares the app for exploration from any state.
	EnsureReady(ctx context.Context) (*ScreenState, error)
	// Reach tries to bring the app to the screen named by key,
	// using learned recipes first. The steps it took come back with the
	// screen: they are the prefix an exported flow needs to replay from
	// the same place, and only the caller of Reach knows the run they
	// belong to.
	Reach(ctx context.Context, key string) (*ScreenState, []StepRecord, error)
}

Navigator brings the device to a usable starting state: app launched, foregrounded, past login when knowledge covers it.

type Observer

type Observer interface {
	Observe(ctx context.Context) (*ScreenState, error)
}

Observer captures the current screen: settle, hierarchy, screenshot, flattening, and signature computation.

type OutcomeCheck

type OutcomeCheck struct {
	Expected string
	Met      bool
	Evidence string
	// Missed classifies an unmet outcome so the report can separate a
	// product defect from a planning artifact and from a missing verdict.
	Missed MissReason
	// Driver marks a check_visible probe the run made along the way. It is
	// evidence for the judge, never the scenario's verdict, so the report
	// never files one as a finding.
	Driver bool
}

OutcomeCheck is one expected outcome with its verification result.

type Plan

type Plan struct {
	AppID     string
	CreatedAt time.Time
	Scenarios []Scenario
}

Plan is the growing collection of scenarios for one exploration target.

func (*Plan) Pending

func (p *Plan) Pending() []Scenario

Pending returns the scenarios still waiting to run, ordered by priority rank then insertion order.

type PlanRequest

type PlanRequest struct {
	Map *UIMap
	// Style selects the planning style for this iteration.
	Style string
	// Existing lists scenario names already planned or executed, for
	// dedup.
	Existing []string
	// Unmet lists expected outcomes earlier runs of this session looked for
	// and did not find. Naming them keeps the planner from writing the same
	// expectation every round: mmx78 planned the same invented day view
	// three times.
	//
	// The claim is deliberately weak. The earlier field carried only the
	// outcomes a judge ruled the app never offers, which is a judgement no
	// session ever made -- every report says "0 with an expectation the app
	// never promised" while filing defects of exactly that kind. "A run
	// looked and did not find it" needs nobody's opinion.
	Unmet []string
	// Focus optionally narrows planning to one feature or region.
	Focus string
	// Budget caps how many scenarios to emit.
	Budget int
}

PlanRequest carries what planning needs for one iteration.

type Planner

type Planner interface {
	PlanNext(ctx context.Context, request PlanRequest) ([]Scenario, error)
}

Planner emits prioritized scenarios for a researched screen.

type Priority

type Priority string

Priority ranks a scenario. The tester runs higher priorities first.

const (
	PriorityCritical  Priority = "critical"
	PriorityImportant Priority = "important"
	PriorityHigh      Priority = "high"
	PriorityNormal    Priority = "normal"
	PriorityLow       Priority = "low"
)

Scenario priorities, highest first.

type Researcher

type Researcher interface {
	Research(ctx context.Context, state *ScreenState) (*UIMap, error)
}

Researcher turns an observed screen into a validated UI map. Results are cached by screen signature in the state directory.

type Role

type Role string

Role identifies the author of a chat message.

const (
	RoleSystem    Role = "system"
	RoleUser      Role = "user"
	RoleAssistant Role = "assistant"
	RoleTool      Role = "tool"
)

Chat roles.

type Scenario

type Scenario struct {
	Name     string
	Priority Priority
	// Style names the planning style that produced this scenario.
	Style string
	// StartScreen is the screen key the scenario begins on.
	StartScreen string
	Steps       []string
	Expected    []string
	Status      ScenarioStatus
}

Scenario is one planned test: a business-focused goal with atomic steps as guidance and atomic expected outcomes for verification. Steps guide, outcomes verify; the tester adapts steps to the live app but never weakens outcomes.

type ScenarioStatus

type ScenarioStatus string

ScenarioStatus tracks a scenario through the session.

const (
	ScenarioPending ScenarioStatus = "pending"
	ScenarioRunning ScenarioStatus = "running"
	ScenarioPassed  ScenarioStatus = "passed"
	ScenarioFailed  ScenarioStatus = "failed"
	ScenarioSkipped ScenarioStatus = "skipped"
)

Scenario lifecycle states.

type ScreenSignature

type ScreenSignature struct {
	// AppID scopes the signature to the application.
	AppID string
	// Salient holds a few stable prominent labels (screen title, tab
	// name) that make the signature human-readable.
	Salient []string
	// TreeDigest is a short hex digest of the normalized flattened
	// accessibility tree.
	TreeDigest string
}

ScreenSignature identifies one distinct screen state of the app under exploration. It plays the role a URL plays for a web page: memory files, research caches, and learned recipes are keyed by it. The digest must be normalized against animation and volatile text so that the same logical screen yields the same signature.

func ComputeSignature

func ComputeSignature(appID string, root device.TreeNode) ScreenSignature

ComputeSignature derives the screen signature for a tree: a digest over normalized structure and text, plus a few salient labels. Digit runs collapse so counters and timestamps do not split one logical screen into many signatures.

func (ScreenSignature) Key

func (s ScreenSignature) Key() string

Key returns a filesystem-safe slug identifying this screen, built from the salient labels and a digest prefix.

func (ScreenSignature) NamesTheSameScreen

func (s ScreenSignature) NamesTheSameScreen(key string) bool

NamesTheSameScreen reports whether key names this screen. The whole key matches, and so do the labels alone: the digest covers the WHOLE tree, so a different day highlighted or a different scroll offset renames the screen, and a check on the digest can only ever match a screen nothing has touched. mmx69 measured that -- four reaches spent their whole turn budget on a screen the app was already showing.

This is a check, not a store key. Do NOT use it to look a recipe up: internal/explore/CLAUDE.md records what a prefix match on the digest cost the store, which is one screen's recipe served to another.

func (ScreenSignature) Same

func (s ScreenSignature) Same(other ScreenSignature) bool

Same reports whether two signatures identify the same screen state.

type ScreenState

type ScreenState struct {
	Signature     ScreenSignature
	Hierarchy     device.TreeNode
	Elements      []FlatElement
	ScreenshotPNG []byte
	CapturedAt    time.Time
	// DialogActive reports that a modal surface (alert, sheet, dialog)
	// dominates the screen.
	DialogActive bool
	// Viewport is the screen the device reported when this state was
	// captured. A zero one means nobody measured, not a screen of no size.
	Viewport device.Bounds
}

ScreenState is one full observation of the device screen.

func (*ScreenState) FullTree

func (state *ScreenState) FullTree() (*hierarchy.Element, error)

FullTree normalizes the captured hierarchy and prunes nothing. Only a caller asking what could ever match wants this -- deciding whether a generalized selector stays unambiguous, where a row one scroll away is a second match waiting to happen. Anything deciding what to touch or what is on screen wants VisibleTree.

func (*ScreenState) VisibleTree

func (state *ScreenState) VisibleTree() (*hierarchy.Element, error)

VisibleTree normalizes the captured hierarchy and prunes it the way the engine does before matching (internal/engine/lookup.go). Everything that selects an element by name has to see the same screen the exported flow will be replayed against; matching the raw tree reaches elements with no area, whose centre is the screen corner, and elements past the screen edge, whose centre is off it.

An unmeasured viewport prunes nothing: a caller holding a hand-built state has said nothing about screen size, and treating that as a screen of zero size would hide every element instead.

type Section

type Section struct {
	Name     string
	Notes    string
	Elements []MappedElement
	// Trigger records the action that revealed a hidden section
	// (expanded row, opened sheet), empty for always-visible sections.
	Trigger string
}

Section groups related elements of one screen region.

type SessionReport

type SessionReport struct {
	AppID    string
	Platform string
	Results  []TestResult
	// Markdown is the rendered analyst report.
	Markdown string
	Usage    Usage
	Started  time.Time
	Finished time.Time
}

SessionReport aggregates a whole exploration session.

func RunSession

func RunSession(ctx context.Context, config Config, crew Crew) (*SessionReport, error)

RunSession drives the deterministic strategic loop: observe, research, plan in the current style, execute pending scenarios by priority, and report. Tactical decisions live inside the role implementations. The session stops when the test budget is spent, planning dries up across a full style rotation, or the context ends.

type StepRecord

type StepRecord struct {
	Index   int
	Action  Action
	Status  StepStatus
	Note    string
	Before  ScreenSignature
	After   ScreenSignature
	At      time.Time
	ErrText string
	// TargetMiss marks a step that failed because the screen has no element
	// the agent's target names. The device answered; the agent aimed at
	// nothing, so a report must not read this as broken equipment.
	TargetMiss bool
}

StepRecord is one executed step of a scenario run, with the screen signatures around it so state transitions are auditable.

type StepStatus

type StepStatus string

StepStatus reports how one executed step ended.

const (
	StepOK       StepStatus = "ok"
	StepFailed   StepStatus = "failed"
	StepRecov    StepStatus = "recovered"
	StepNoChange StepStatus = "no-change"
)

Step outcomes.

type TargetMissError

type TargetMissError struct {
	Reason string
}

TargetMissError is the error a target that resolves to no element returns. The message is the whole error: the agent reads it as a tool result, so it carries no wrapper text.

func (TargetMissError) Error

func (e TargetMissError) Error() string

type TestResult

type TestResult struct {
	Scenario Scenario
	Status   TestStatus
	// Prelude is what the navigator did to bring the app to the scenario's
	// start screen, before the run's own first step. A relaunch does not
	// land on that screen -- an app restores its last view -- so a flow
	// exported without this walk begins somewhere the recording never was.
	Prelude  []StepRecord
	Steps    []StepRecord
	Outcomes []OutcomeCheck
	Notes    []string
	Started  time.Time
	Finished time.Time
	// Verdict is the supervisor summary when the pilot ran, or the
	// outcome-matching summary otherwise.
	Verdict string
}

TestResult is the full record of one scenario execution.

type TestStatus

type TestStatus string

TestStatus is the final verdict for one scenario run.

const (
	TestPassed  TestStatus = "passed"
	TestFailed  TestStatus = "failed"
	TestStopped TestStatus = "stopped"
)

Scenario run verdicts.

type Tester

type Tester interface {
	RunScenario(ctx context.Context, scenario Scenario, start *ScreenState) (*TestResult, error)
}

Tester executes one scenario against the live device, adapting steps while never weakening expected outcomes.

type ToolBox

type ToolBox struct {
	Specs    []ToolSpec
	Handlers map[string]ToolHandler
}

ToolBox pairs tool declarations with their handlers.

type ToolCall

type ToolCall struct {
	ID        string
	Name      string
	Arguments json.RawMessage
}

ToolCall is a model-requested invocation of a registered tool.

type ToolHandler

type ToolHandler func(ctx context.Context, args json.RawMessage) (string, error)

ToolHandler executes one tool call and returns the text shown to the model. Returning ErrStopRequested (possibly wrapped) ends the loop with Stopped set; any other error is reported to the model as a tool failure and the loop continues.

type ToolSpec

type ToolSpec struct {
	Name        string
	Description string
	Schema      json.RawMessage
}

ToolSpec declares a tool the model may call. Schema is a JSON Schema object for the arguments.

type UIMap

type UIMap struct {
	Screen    ScreenSignature
	Sections  []Section
	CreatedAt time.Time
	// Markdown is the rendered map given to planning and testing
	// conversations.
	Markdown string
}

UIMap is the researched, validated map of one screen.

type Usage

type Usage struct {
	InputTokens  int
	OutputTokens int
}

Usage reports token spend for one invocation.

Directories

Path Synopsis
Package export turns finished exploration runs into runnable two-document flow YAML, validated through the flow parser before it is returned.
Package export turns finished exploration runs into runnable two-document flow YAML, validated through the flow parser before it is returned.
Package memory implements the filesystem stores of the per-app explore state directory: learned per-screen recipes, operator-authored hints, saved plans, and cached research maps.
Package memory implements the filesystem stores of the per-app explore state directory: learned per-screen recipes, operator-authored hints, saved plans, and cached research maps.
Package planning turns researched UI maps into prioritized, deduplicated test scenarios through one model conversation per iteration.
Package planning turns researched UI maps into prioritized, deduplicated test scenarios through one model conversation per iteration.
Package report renders a finished exploration session into a markdown summary and aggregates token spend across the session.
Package report renders a finished exploration session into a markdown summary and aggregates token spend across the session.
Package research implements the Observer and Researcher exploration roles: capturing one settled screen of the app through the device driver and turning that capture into a validated UI map for planning and testing conversations.
Package research implements the Observer and Researcher exploration roles: capturing one settled screen of the app through the device driver and turning that capture into a validated UI map for planning and testing conversations.
Package run implements the execution roles of exploration mode: the Tester tool loop, the Pilot supervisor conversation, and the Navigator that brings the app to a usable screen.
Package run implements the execution roles of exploration mode: the Tester tool loop, the Pilot supervisor conversation, and the Navigator that brings the app to a usable screen.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL