agent

package
v0.0.0-...-fad84cb Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 13, 2026 License: Apache-2.0 Imports: 27 Imported by: 0

Documentation

Overview

Package agent implements LLM-driven test failure analysis using the GitHub Copilot SDK. It wraps the Copilot CLI process and provides a structured session interface for analysis workflows.

Index

Constants

View Source
const (
	// CopilotAuthModeLoggedIn uses the developer's GitHub CLI session.
	CopilotAuthModeLoggedIn = "logged-in"
	// CopilotAuthModeToken reads a GitHub token from a file.
	CopilotAuthModeToken = "token"
	// CopilotAuthModeBYOK uses an Azure Entra token against a model endpoint.
	CopilotAuthModeBYOK = "byok"
)
View Source
const FirstChainQuestion = "Why did this test fail?"

FirstChainQuestion is the required question for the first link in the causal chain.

Variables

This section is empty.

Functions

func BuildInitialPrompt

func BuildInitialPrompt(manifest, testError, testOutput, siblingTests, dataDir string, worktreePaths map[string]string) string

BuildInitialPrompt creates the initial user prompt for an analysis run, including the manifest, test logs, and available source code worktrees.

func BuildReviewPrompt

func BuildReviewPrompt(rendered string) string

BuildReviewPrompt constructs the prompt sent to the agent during review rounds, asking it to review and re-emit the analysis.

func BuildSummaryPrompt

func BuildSummaryPrompt(testAnalyses []string) string

BuildSummaryPrompt creates the user prompt for the job-level summary step.

func BuildSystemMessageConfig

func BuildSystemMessageConfig() (*copilot.SystemMessageConfig, error)

BuildSystemMessageConfig assembles a SystemMessageConfig in "customize" mode.

Section strategy (from design review):

  • SectionIdentity: replace with our domain identity
  • SectionTone: replace with our analysis tone
  • SectionCodeChangeRules: remove (agent doesn't write code)
  • SectionToolInstructions: keep (SDK manages tool descriptions)
  • SectionToolEfficiency: keep
  • SectionSafety: keep
  • SectionCustomInstructions: append domain content (references, exemplars, output schema)

func NewKustoTool

func NewKustoTool(client KustoClient) copilot.Tool

NewKustoTool creates a kusto_query Copilot SDK tool backed by the given client.

func RenderMarkdown

func RenderMarkdown(chain *HydratedChain, testName string) string

RenderMarkdown produces a low-fidelity markdown document from a hydrated analysis chain. This is used to show the agent the full rendered output — including query result tables — so it can review narrative coherence, evidence quality, and depth before finalizing.

func TableToMarkdown

func TableToMarkdown(t *tabular.Table) string

TableToMarkdown renders a tabular.Table as a markdown table. If the table is nil or has no columns, the string "(no results)" is returned.

func Validate

func Validate(chain *HydratedChain) error

Validate checks hydration-specific requirements on a hydrated chain: - Every kusto proof has a non-empty share URI (generated during hydration) This is a lightweight post-hydration check; structural validation (non-empty summary, claims, proof items, etc.) is performed earlier by ValidateDraft.

func ValidateDraft

func ValidateDraft(ctx context.Context, client KustoClient, draft *DraftChain, vc *ValidationContext) string

ValidateDraft checks a DraftChain for structural problems and executes every KQL snippet against the provided Kusto client. It validates log proof line ranges against actual log contents, code proof line ranges against actual source files, and discovery paths against the data directory. It returns a human-readable feedback string describing all issues found, suitable for sending back to the agent as a correction prompt. If everything is valid, the returned string is empty.

Types

type ADXKustoClient

type ADXKustoClient struct {
	// contains filtered or unexported fields
}

ADXKustoClient wraps an Azure Data Explorer client to implement KustoClient. It executes queries against a specific database and formats results as markdown.

func NewADXKustoClient

func NewADXKustoClient(credential azcore.TokenCredential, clusterURI, database string) (*ADXKustoClient, error)

NewADXKustoClient creates a KustoClient that queries a specific ADX cluster and database. The client is created using the provided credential and cluster URI. The caller is responsible for calling Close when done.

func (*ADXKustoClient) Close

func (c *ADXKustoClient) Close() error

Close releases the underlying Kusto client resources.

func (*ADXKustoClient) Query

func (c *ADXKustoClient) Query(ctx context.Context, kqlQuery string) (*tabular.Table, error)

Query executes a KQL query and returns the result as a tabular.Table.

type AgentConfig

type AgentConfig struct {
	// AuthMode is one of "logged-in", "token", or "byok".
	AuthMode string

	// GitHubTokenFile is the path to a file containing a GitHub token (token mode).
	GitHubTokenFile string

	// ModelEndpoint is the Azure AI Foundry endpoint URL (byok mode).
	ModelEndpoint string

	// ModelDeployment is the model deployment name (byok mode).
	ModelDeployment string

	// AzureCredential is used to acquire Entra tokens for BYOK sessions.
	AzureCredential azcore.TokenCredential

	// Model overrides the default model for Copilot sessions.
	Model string

	// MaxRounds is the maximum number of tool-call rounds per session.
	MaxRounds int

	// Verbosity is the log verbosity level from the CLI. When >= 5,
	// the Copilot CLI subprocess is started with --log-level=debug and
	// all session events are traced.
	Verbosity int
}

AgentConfig configures the agent's auth and model settings. This is the superset of configuration options — callers populate only the fields relevant to their auth mode.

type AnalyzeOptions

type AnalyzeOptions struct {
	// Manifest is the raw manifest.json content.
	Manifest []byte

	// TestName is the name of the failed test (used for rendering).
	TestName string

	// TestError is the content of test_logs/error.log, or empty.
	TestError string

	// TestOutput is the content of test_logs/output.log, or empty.
	TestOutput string

	// SiblingTests is the content of sibling_tests.json, or empty.
	SiblingTests string

	// DataDir is the root of the structured data directory.
	DataDir string

	// WorktreePaths maps repository names to local filesystem paths.
	WorktreePaths map[string]string

	// KustoCluster is the Kusto cluster URI for hydration share links.
	KustoCluster string

	// KustoDatabase is the Kusto database name.
	KustoDatabase string

	// MaxValidationRounds is the maximum number of parse/validate correction
	// rounds per validate-draft cycle. Zero defaults to 10.
	MaxValidationRounds int

	// ReviewRounds is the number of review passes. Zero defaults to 3.
	ReviewRounds int
}

AnalyzeOptions configures a single analysis run.

type AnalyzeResult

type AnalyzeResult struct {
	// HydratedChain is the fully validated and hydrated causal chain.
	HydratedChain *HydratedChain

	// DraftChain is the last validated draft before the final hydration.
	DraftChain *DraftChain
}

AnalyzeResult contains the output of a successful analysis.

func Analyze

func Analyze(ctx context.Context, logger logr.Logger, session *Session, kustoClient KustoClient, opts AnalyzeOptions) (*AnalyzeResult, error)

Analyze runs the full agentic analysis loop: initial prompt, validate-draft, hydrate, and review rounds. It requires an already-created Session and KustoClient. The caller is responsible for session lifecycle (create, save conversation, delete/disconnect) and Kusto client lifecycle (create, close).

The function sends the initial prompt, validates and corrects the agent's output, hydrates proof items with real query results and code excerpts, then runs review rounds where the agent sees its rendered output and can refine it.

type CachingKustoClient

type CachingKustoClient struct {
	// contains filtered or unexported fields
}

CachingKustoClient wraps a KustoClient and caches successful query results in memory, keyed by the KQL query string. This avoids re-running identical queries across validation and hydration rounds. Only successful results are cached; errors are always retried against the underlying client.

func NewCachingKustoClient

func NewCachingKustoClient(delegate KustoClient) *CachingKustoClient

NewCachingKustoClient wraps the given client with an in-memory query cache.

func (*CachingKustoClient) Query

func (c *CachingKustoClient) Query(ctx context.Context, kqlQuery string) (*tabular.Table, error)

Query returns a cached result for the given KQL if available, otherwise delegates to the underlying client and caches a successful result.

type ChainLink struct {
	Question string      `json:"question"`
	Answer   string      `json:"answer"`
	Notes    string      `json:"notes,omitempty"`
	Proof    []ProofItem `json:"proof"`
}

ChainLink is one link in the causal why-chain. Each link poses a "why?" question and provides an answer backed by proof. The first link's question is always "Why did this test fail?"; subsequent questions follow naturally from the previous answer.

type CopilotClient

type CopilotClient struct {
	// contains filtered or unexported fields
}

CopilotClient wraps a copilot.Client that manages the Copilot CLI process. One instance is created per process lifetime.

func NewCopilotClient

func NewCopilotClient(cfg *AgentConfig) (*CopilotClient, error)

NewCopilotClient creates a CopilotClient configured for the given auth mode. The underlying CLI process is started lazily on first session creation (AutoStart defaults to true).

func (*CopilotClient) CreateSession

func (c *CopilotClient) CreateSession(ctx context.Context, logger logr.Logger, cfg SessionConfig) (*Session, error)

CreateSession creates a new Copilot session for an analysis run.

func (*CopilotClient) Stop

func (c *CopilotClient) Stop() error

Stop shuts down the Copilot CLI process and releases all resources.

type DiscoveryItem

type DiscoveryItem struct {
	// Label is a human-readable description of what this discovery item establishes.
	Label string `json:"label"`

	// KQL is an agent-authored Kusto query whose results establish provenance
	// for constants used in proof queries.
	KQL string `json:"kql"`
}

DiscoveryItem is an agent-authored KQL query with a label, used to establish provenance for constants in proof queries that are not already covered by the pre-gathered data directory (which is embedded automatically during hydration).

type DraftChain

type DraftChain struct {
	RootCause   string          `json:"root_cause"`
	Summary     string          `json:"summary"`
	Notes       string          `json:"notes,omitempty"`
	Discovery   []DiscoveryItem `json:"discovery,omitempty"`
	Chain       []ChainLink     `json:"chain"`
	Suggestions []string        `json:"suggestions,omitempty"`
}

DraftChain is the structured output the agent must produce as its final message. This is the pre-hydration format — KQL queries have no share URIs or result tables yet.

func ParseDraftChain

func ParseDraftChain(output string) (*DraftChain, error)

ParseDraftChain parses the agent's final output as a DraftChain.

func ValidateDraftLoop

func ValidateDraftLoop(
	ctx context.Context,
	logger logr.Logger,
	session *Session,
	kustoClient KustoClient,
	vc *ValidationContext,
	output string,
	maxRounds int,
) (*DraftChain, string, error)

ValidateDraftLoop parses and validates the agent's output, sending correction feedback for up to maxRounds iterations. It returns the validated draft chain and the raw output string (which may have been updated by agent corrections).

type HydratedChain

type HydratedChain struct {
	RootCause   string              `json:"root_cause"`
	Summary     string              `json:"summary"`
	Notes       string              `json:"notes,omitempty"`
	Discovery   []HydratedDiscovery `json:"discovery,omitempty"`
	Chain       []HydratedLink      `json:"chain"`
	Suggestions []string            `json:"suggestions,omitempty"`
}

HydratedChain extends DraftChain with query results and share URIs.

type HydratedDiscovery

type HydratedDiscovery struct {
	Directory string         `json:"directory,omitempty"`
	Label     string         `json:"label"`
	KQL       string         `json:"kql"`
	ShareURI  string         `json:"share_uri"`
	Table     *tabular.Table `json:"table,omitempty"`
}

HydratedDiscovery is a single leaf query directory from a discovery path, hydrated with its README summary, KQL, share URI, and parsed query results.

type HydratedLink struct {
	Question string              `json:"question"`
	Answer   string              `json:"answer"`
	Notes    string              `json:"notes,omitempty"`
	Proof    []HydratedProofItem `json:"proof"`
}

HydratedLink is a chain link with hydrated proof items.

type HydratedProofItem

type HydratedProofItem struct {
	ProofItem

	// Kusto proof fields populated during hydration
	ShareURI string         `json:"share_uri,omitempty"`
	Table    *tabular.Table `json:"table,omitempty"`

	// Code proof field populated during hydration by reading the worktree
	CodeExcerpt string `json:"code_excerpt,omitempty"`

	// Log proof field populated during hydration by extracting lines from test logs
	LogExcerpt string `json:"log_excerpt,omitempty"`
}

HydratedProofItem extends ProofItem with fields populated during hydration.

type Hydrator

type Hydrator struct {
	// contains filtered or unexported fields
}

Hydrator takes a draft chain produced by the agent and populates share URIs and result tables for all Kusto proofs by re-running the queries deterministically.

func NewHydrator

func NewHydrator(kustoClient KustoClient, kustoEndpoint, kustoDatabase string, worktreePaths map[string]string, testError, testOutput, dataDir string) *Hydrator

NewHydrator creates a Hydrator with the given Kusto client, cluster details, worktree paths for resolving code proof excerpts, test log contents for resolving log proof excerpts, and data directory for resolving discovery paths.

func (*Hydrator) Hydrate

func (h *Hydrator) Hydrate(ctx context.Context, draft *DraftChain) (*HydratedChain, error)

Hydrate takes a DraftChain and produces a HydratedChain by re-running all KQL queries and generating share URIs.

type KustoClient

type KustoClient interface {
	// Query executes a KQL query and returns the result as a tabular.Table.
	Query(ctx context.Context, kql string) (*tabular.Table, error)
}

KustoClient is the interface for executing KQL queries against Azure Data Explorer.

type ProofItem

type ProofItem struct {
	Type string `json:"type"` // "kusto", "code", "log"

	// Kusto proof fields
	KQL  string `json:"kql,omitempty"`
	Note string `json:"note,omitempty"`

	// Code proof fields
	Repo  string `json:"repo,omitempty"`
	File  string `json:"file,omitempty"`
	Lines [2]int `json:"lines,omitempty"` // 1-indexed, inclusive; used by code and log proofs

	// Log proof fields
	Source string `json:"source,omitempty"` // "error" or "output"
}

ProofItem is evidence supporting a chain link claim.

type Session

type Session struct {
	// contains filtered or unexported fields
}

Session wraps a copilot.Session for a single analysis run. It snapshots the full conversation history after every successful SendAndWait so that the conversation can be saved even if the CLI process is no longer available (e.g. after ctrl-C kills the subprocess).

func (*Session) Delete

func (s *Session) Delete(ctx context.Context) error

Delete permanently removes all session data from disk.

func (*Session) Disconnect

func (s *Session) Disconnect() error

Disconnect releases in-memory session resources. Session state is preserved on disk and can be resumed later.

func (*Session) SaveConversation

func (s *Session) SaveConversation(path string)

SaveConversation writes the most recent conversation snapshot to a JSON file at the given path. Because messages are snapshotted after every successful turn, this works even after the CLI subprocess has exited. This is best-effort: errors are logged but not returned.

func (*Session) SendAndWait

func (s *Session) SendAndWait(ctx context.Context, prompt string) (string, error)

SendAndWait sends a prompt to the session and blocks until the agent is idle. If ctx is cancelled, the in-flight work is aborted. Returns the final assistant message content.

func (*Session) SessionID

func (s *Session) SessionID() string

SessionID returns the unique identifier for this session.

type SessionConfig

type SessionConfig struct {
	// WorkingDirectory is the workspace root for the Copilot session.
	// Tool operations (read_file, grep, bash, glob) are relative to this directory.
	WorkingDirectory string

	// SystemMessage configures system prompt customization.
	SystemMessage *copilot.SystemMessageConfig

	// Tools are custom tools (e.g. kusto_query) registered on this session.
	Tools []copilot.Tool

	// Model overrides the default model for this session.
	Model string
}

SessionConfig configures a new analysis session.

type ValidationContext

type ValidationContext struct {
	// ValidRepos is the set of repository names that have source code worktrees available.
	ValidRepos map[string]bool
	// WorktreePaths maps repository names to local filesystem paths for their git worktrees.
	WorktreePaths map[string]string
	// DataDir is the root of the gathered data directory (for discovery path validation).
	DataDir string
	// TestError is the contents of the test error.log file.
	TestError string
	// TestOutput is the contents of the test output.log file.
	TestOutput string
}

ValidationContext holds the data needed to validate a DraftChain beyond structural checks — file system paths, log contents, and worktree locations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL