Documentation
¶
Overview ¶
Package agent implements LLM-driven test failure analysis using the GitHub Copilot SDK. It wraps the Copilot CLI process and provides a structured session interface for analysis workflows.
Index ¶
- Constants
- func BuildInitialPrompt(manifest, testError, testOutput, siblingTests, dataDir string, ...) string
- func BuildReviewPrompt(rendered string) string
- func BuildSummaryPrompt(testAnalyses []string) string
- func BuildSystemMessageConfig() (*copilot.SystemMessageConfig, error)
- func NewKustoTool(client KustoClient) copilot.Tool
- func RenderMarkdown(chain *HydratedChain, testName string) string
- func TableToMarkdown(t *tabular.Table) string
- func Validate(chain *HydratedChain) error
- func ValidateDraft(ctx context.Context, client KustoClient, draft *DraftChain, ...) string
- type ADXKustoClient
- type AgentConfig
- type AnalyzeOptions
- type AnalyzeResult
- type CachingKustoClient
- type ChainLink
- type CopilotClient
- type DiscoveryItem
- type DraftChain
- type HydratedChain
- type HydratedDiscovery
- type HydratedLink
- type HydratedProofItem
- type Hydrator
- type KustoClient
- type ProofItem
- type Session
- type SessionConfig
- type ValidationContext
Constants ¶
const ( // CopilotAuthModeLoggedIn uses the developer's GitHub CLI session. CopilotAuthModeLoggedIn = "logged-in" // CopilotAuthModeToken reads a GitHub token from a file. CopilotAuthModeToken = "token" // CopilotAuthModeBYOK uses an Azure Entra token against a model endpoint. CopilotAuthModeBYOK = "byok" )
const FirstChainQuestion = "Why did this test fail?"
FirstChainQuestion is the required question for the first link in the causal chain.
Variables ¶
This section is empty.
Functions ¶
func BuildInitialPrompt ¶
func BuildInitialPrompt(manifest, testError, testOutput, siblingTests, dataDir string, worktreePaths map[string]string) string
BuildInitialPrompt creates the initial user prompt for an analysis run, including the manifest, test logs, and available source code worktrees.
func BuildReviewPrompt ¶
BuildReviewPrompt constructs the prompt sent to the agent during review rounds, asking it to review and re-emit the analysis.
func BuildSummaryPrompt ¶
BuildSummaryPrompt creates the user prompt for the job-level summary step.
func BuildSystemMessageConfig ¶
func BuildSystemMessageConfig() (*copilot.SystemMessageConfig, error)
BuildSystemMessageConfig assembles a SystemMessageConfig in "customize" mode.
Section strategy (from design review):
- SectionIdentity: replace with our domain identity
- SectionTone: replace with our analysis tone
- SectionCodeChangeRules: remove (agent doesn't write code)
- SectionToolInstructions: keep (SDK manages tool descriptions)
- SectionToolEfficiency: keep
- SectionSafety: keep
- SectionCustomInstructions: append domain content (references, exemplars, output schema)
func NewKustoTool ¶
func NewKustoTool(client KustoClient) copilot.Tool
NewKustoTool creates a kusto_query Copilot SDK tool backed by the given client.
func RenderMarkdown ¶
func RenderMarkdown(chain *HydratedChain, testName string) string
RenderMarkdown produces a low-fidelity markdown document from a hydrated analysis chain. This is used to show the agent the full rendered output — including query result tables — so it can review narrative coherence, evidence quality, and depth before finalizing.
func TableToMarkdown ¶
TableToMarkdown renders a tabular.Table as a markdown table. If the table is nil or has no columns, the string "(no results)" is returned.
func Validate ¶
func Validate(chain *HydratedChain) error
Validate checks hydration-specific requirements on a hydrated chain: - Every kusto proof has a non-empty share URI (generated during hydration) This is a lightweight post-hydration check; structural validation (non-empty summary, claims, proof items, etc.) is performed earlier by ValidateDraft.
func ValidateDraft ¶
func ValidateDraft(ctx context.Context, client KustoClient, draft *DraftChain, vc *ValidationContext) string
ValidateDraft checks a DraftChain for structural problems and executes every KQL snippet against the provided Kusto client. It validates log proof line ranges against actual log contents, code proof line ranges against actual source files, and discovery paths against the data directory. It returns a human-readable feedback string describing all issues found, suitable for sending back to the agent as a correction prompt. If everything is valid, the returned string is empty.
Types ¶
type ADXKustoClient ¶
type ADXKustoClient struct {
// contains filtered or unexported fields
}
ADXKustoClient wraps an Azure Data Explorer client to implement KustoClient. It executes queries against a specific database and formats results as markdown.
func NewADXKustoClient ¶
func NewADXKustoClient(credential azcore.TokenCredential, clusterURI, database string) (*ADXKustoClient, error)
NewADXKustoClient creates a KustoClient that queries a specific ADX cluster and database. The client is created using the provided credential and cluster URI. The caller is responsible for calling Close when done.
func (*ADXKustoClient) Close ¶
func (c *ADXKustoClient) Close() error
Close releases the underlying Kusto client resources.
type AgentConfig ¶
type AgentConfig struct {
// AuthMode is one of "logged-in", "token", or "byok".
AuthMode string
// GitHubTokenFile is the path to a file containing a GitHub token (token mode).
GitHubTokenFile string
// ModelEndpoint is the Azure AI Foundry endpoint URL (byok mode).
ModelEndpoint string
// ModelDeployment is the model deployment name (byok mode).
ModelDeployment string
// AzureCredential is used to acquire Entra tokens for BYOK sessions.
AzureCredential azcore.TokenCredential
// Model overrides the default model for Copilot sessions.
Model string
// MaxRounds is the maximum number of tool-call rounds per session.
MaxRounds int
// Verbosity is the log verbosity level from the CLI. When >= 5,
// the Copilot CLI subprocess is started with --log-level=debug and
// all session events are traced.
Verbosity int
}
AgentConfig configures the agent's auth and model settings. This is the superset of configuration options — callers populate only the fields relevant to their auth mode.
type AnalyzeOptions ¶
type AnalyzeOptions struct {
// Manifest is the raw manifest.json content.
Manifest []byte
// TestName is the name of the failed test (used for rendering).
TestName string
// TestError is the content of test_logs/error.log, or empty.
TestError string
// TestOutput is the content of test_logs/output.log, or empty.
TestOutput string
// SiblingTests is the content of sibling_tests.json, or empty.
SiblingTests string
// DataDir is the root of the structured data directory.
DataDir string
// WorktreePaths maps repository names to local filesystem paths.
WorktreePaths map[string]string
// KustoCluster is the Kusto cluster URI for hydration share links.
KustoCluster string
// KustoDatabase is the Kusto database name.
KustoDatabase string
// MaxValidationRounds is the maximum number of parse/validate correction
// rounds per validate-draft cycle. Zero defaults to 10.
MaxValidationRounds int
// ReviewRounds is the number of review passes. Zero defaults to 3.
ReviewRounds int
}
AnalyzeOptions configures a single analysis run.
type AnalyzeResult ¶
type AnalyzeResult struct {
// HydratedChain is the fully validated and hydrated causal chain.
HydratedChain *HydratedChain
// DraftChain is the last validated draft before the final hydration.
DraftChain *DraftChain
}
AnalyzeResult contains the output of a successful analysis.
func Analyze ¶
func Analyze(ctx context.Context, logger logr.Logger, session *Session, kustoClient KustoClient, opts AnalyzeOptions) (*AnalyzeResult, error)
Analyze runs the full agentic analysis loop: initial prompt, validate-draft, hydrate, and review rounds. It requires an already-created Session and KustoClient. The caller is responsible for session lifecycle (create, save conversation, delete/disconnect) and Kusto client lifecycle (create, close).
The function sends the initial prompt, validates and corrects the agent's output, hydrates proof items with real query results and code excerpts, then runs review rounds where the agent sees its rendered output and can refine it.
type CachingKustoClient ¶
type CachingKustoClient struct {
// contains filtered or unexported fields
}
CachingKustoClient wraps a KustoClient and caches successful query results in memory, keyed by the KQL query string. This avoids re-running identical queries across validation and hydration rounds. Only successful results are cached; errors are always retried against the underlying client.
func NewCachingKustoClient ¶
func NewCachingKustoClient(delegate KustoClient) *CachingKustoClient
NewCachingKustoClient wraps the given client with an in-memory query cache.
type ChainLink ¶
type ChainLink struct {
Question string `json:"question"`
Answer string `json:"answer"`
Notes string `json:"notes,omitempty"`
Proof []ProofItem `json:"proof"`
}
ChainLink is one link in the causal why-chain. Each link poses a "why?" question and provides an answer backed by proof. The first link's question is always "Why did this test fail?"; subsequent questions follow naturally from the previous answer.
type CopilotClient ¶
type CopilotClient struct {
// contains filtered or unexported fields
}
CopilotClient wraps a copilot.Client that manages the Copilot CLI process. One instance is created per process lifetime.
func NewCopilotClient ¶
func NewCopilotClient(cfg *AgentConfig) (*CopilotClient, error)
NewCopilotClient creates a CopilotClient configured for the given auth mode. The underlying CLI process is started lazily on first session creation (AutoStart defaults to true).
func (*CopilotClient) CreateSession ¶
func (c *CopilotClient) CreateSession(ctx context.Context, logger logr.Logger, cfg SessionConfig) (*Session, error)
CreateSession creates a new Copilot session for an analysis run.
func (*CopilotClient) Stop ¶
func (c *CopilotClient) Stop() error
Stop shuts down the Copilot CLI process and releases all resources.
type DiscoveryItem ¶
type DiscoveryItem struct {
// Label is a human-readable description of what this discovery item establishes.
Label string `json:"label"`
// KQL is an agent-authored Kusto query whose results establish provenance
// for constants used in proof queries.
KQL string `json:"kql"`
}
DiscoveryItem is an agent-authored KQL query with a label, used to establish provenance for constants in proof queries that are not already covered by the pre-gathered data directory (which is embedded automatically during hydration).
type DraftChain ¶
type DraftChain struct {
RootCause string `json:"root_cause"`
Summary string `json:"summary"`
Notes string `json:"notes,omitempty"`
Discovery []DiscoveryItem `json:"discovery,omitempty"`
Chain []ChainLink `json:"chain"`
Suggestions []string `json:"suggestions,omitempty"`
}
DraftChain is the structured output the agent must produce as its final message. This is the pre-hydration format — KQL queries have no share URIs or result tables yet.
func ParseDraftChain ¶
func ParseDraftChain(output string) (*DraftChain, error)
ParseDraftChain parses the agent's final output as a DraftChain.
func ValidateDraftLoop ¶
func ValidateDraftLoop( ctx context.Context, logger logr.Logger, session *Session, kustoClient KustoClient, vc *ValidationContext, output string, maxRounds int, ) (*DraftChain, string, error)
ValidateDraftLoop parses and validates the agent's output, sending correction feedback for up to maxRounds iterations. It returns the validated draft chain and the raw output string (which may have been updated by agent corrections).
type HydratedChain ¶
type HydratedChain struct {
RootCause string `json:"root_cause"`
Summary string `json:"summary"`
Notes string `json:"notes,omitempty"`
Discovery []HydratedDiscovery `json:"discovery,omitempty"`
Chain []HydratedLink `json:"chain"`
Suggestions []string `json:"suggestions,omitempty"`
}
HydratedChain extends DraftChain with query results and share URIs.
type HydratedDiscovery ¶
type HydratedDiscovery struct {
Directory string `json:"directory,omitempty"`
Label string `json:"label"`
KQL string `json:"kql"`
Table *tabular.Table `json:"table,omitempty"`
}
HydratedDiscovery is a single leaf query directory from a discovery path, hydrated with its README summary, KQL, share URI, and parsed query results.
type HydratedLink ¶
type HydratedLink struct {
Question string `json:"question"`
Answer string `json:"answer"`
Notes string `json:"notes,omitempty"`
Proof []HydratedProofItem `json:"proof"`
}
HydratedLink is a chain link with hydrated proof items.
type HydratedProofItem ¶
type HydratedProofItem struct {
ProofItem
// Kusto proof fields populated during hydration
Table *tabular.Table `json:"table,omitempty"`
// Code proof field populated during hydration by reading the worktree
CodeExcerpt string `json:"code_excerpt,omitempty"`
// Log proof field populated during hydration by extracting lines from test logs
LogExcerpt string `json:"log_excerpt,omitempty"`
}
HydratedProofItem extends ProofItem with fields populated during hydration.
type Hydrator ¶
type Hydrator struct {
// contains filtered or unexported fields
}
Hydrator takes a draft chain produced by the agent and populates share URIs and result tables for all Kusto proofs by re-running the queries deterministically.
func NewHydrator ¶
func NewHydrator(kustoClient KustoClient, kustoEndpoint, kustoDatabase string, worktreePaths map[string]string, testError, testOutput, dataDir string) *Hydrator
NewHydrator creates a Hydrator with the given Kusto client, cluster details, worktree paths for resolving code proof excerpts, test log contents for resolving log proof excerpts, and data directory for resolving discovery paths.
func (*Hydrator) Hydrate ¶
func (h *Hydrator) Hydrate(ctx context.Context, draft *DraftChain) (*HydratedChain, error)
Hydrate takes a DraftChain and produces a HydratedChain by re-running all KQL queries and generating share URIs.
type KustoClient ¶
type KustoClient interface {
// Query executes a KQL query and returns the result as a tabular.Table.
Query(ctx context.Context, kql string) (*tabular.Table, error)
}
KustoClient is the interface for executing KQL queries against Azure Data Explorer.
type ProofItem ¶
type ProofItem struct {
Type string `json:"type"` // "kusto", "code", "log"
// Kusto proof fields
KQL string `json:"kql,omitempty"`
Note string `json:"note,omitempty"`
// Code proof fields
Repo string `json:"repo,omitempty"`
File string `json:"file,omitempty"`
Lines [2]int `json:"lines,omitempty"` // 1-indexed, inclusive; used by code and log proofs
// Log proof fields
Source string `json:"source,omitempty"` // "error" or "output"
}
ProofItem is evidence supporting a chain link claim.
type Session ¶
type Session struct {
// contains filtered or unexported fields
}
Session wraps a copilot.Session for a single analysis run. It snapshots the full conversation history after every successful SendAndWait so that the conversation can be saved even if the CLI process is no longer available (e.g. after ctrl-C kills the subprocess).
func (*Session) Disconnect ¶
Disconnect releases in-memory session resources. Session state is preserved on disk and can be resumed later.
func (*Session) SaveConversation ¶
SaveConversation writes the most recent conversation snapshot to a JSON file at the given path. Because messages are snapshotted after every successful turn, this works even after the CLI subprocess has exited. This is best-effort: errors are logged but not returned.
func (*Session) SendAndWait ¶
SendAndWait sends a prompt to the session and blocks until the agent is idle. If ctx is cancelled, the in-flight work is aborted. Returns the final assistant message content.
type SessionConfig ¶
type SessionConfig struct {
// WorkingDirectory is the workspace root for the Copilot session.
// Tool operations (read_file, grep, bash, glob) are relative to this directory.
WorkingDirectory string
// SystemMessage configures system prompt customization.
SystemMessage *copilot.SystemMessageConfig
// Tools are custom tools (e.g. kusto_query) registered on this session.
Tools []copilot.Tool
// Model overrides the default model for this session.
Model string
}
SessionConfig configures a new analysis session.
type ValidationContext ¶
type ValidationContext struct {
// ValidRepos is the set of repository names that have source code worktrees available.
ValidRepos map[string]bool
// WorktreePaths maps repository names to local filesystem paths for their git worktrees.
WorktreePaths map[string]string
// DataDir is the root of the gathered data directory (for discovery path validation).
DataDir string
// TestError is the contents of the test error.log file.
TestError string
// TestOutput is the contents of the test output.log file.
TestOutput string
}
ValidationContext holds the data needed to validate a DraftChain beyond structural checks — file system paths, log contents, and worktree locations.