contract

package
v1.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 25, 2026 License: MIT Imports: 23 Imported by: 0

Documentation

Overview

Package contract wires Sentinel into the Forge dashboard's contract path. It registers the `sentinel` contributor and answers its intents from the engine. This is the only dashboard surface Sentinel has: the React shell reads it through @forge-go/dashboard-plugin-sentinel.

Index

Constants

View Source 
const ContributorName = "sentinel"

ContributorName is the join key between this contract and packages/plugin-sentinel's `extension` field. A mismatch hides the React plugin with no error anywhere, which is what an uninstalled extension looks like.

Variables

This section is empty.

Functions

func Register 

func Register(d *dispatcher.Dispatcher, reg dashcontract.Registry, wreg dashcontract.WardenRegistry, deps Deps) error

Register loads and validates the embedded manifest, registers the contributor, and binds every intent's handler.

Types

type BaselineRef 

type BaselineRef struct {
	ID       string  `json:"id"`
	Name     string  `json:"name"`
	PassRate float64 `json:"passRate"`
}

BaselineRef names a baseline with the pass rate a trend line draws.

type BaselineResultView 

type BaselineResultView struct {
	CaseID          string             `json:"caseId"`
	CaseName        string             `json:"caseName"`
	Score           float64            `json:"score"`
	Status          string             `json:"status"`
	DimensionScores map[string]float64 `json:"dimensionScores"`
}

BaselineResultView is one case as the baseline recorded it.

type BaselineView 

type BaselineView struct {
	ID              string             `json:"id"`
	SuiteID         string             `json:"suiteId"`
	SuiteName       string             `json:"suiteName"`
	RunID           string             `json:"runId"`
	Name            string             `json:"name"`
	PassRate        float64            `json:"passRate"`
	AvgScore        float64            `json:"avgScore"`
	DimensionScores map[string]float64 `json:"dimensionScores"`
	CaseCount       int                `json:"caseCount"`
	IsCurrent       bool               `json:"isCurrent"`
	CreatedAt       string             `json:"createdAt"`
}

BaselineView is a saved baseline.

type CaseNameView 

type CaseNameView struct {
	CaseID   string `json:"caseId"`
	CaseName string `json:"caseName"`
}

CaseNameView names a case.

type CasePair 

type CasePair struct {
	CaseID   string     `json:"caseId"`
	CaseName string     `json:"caseName"`
	A        *ResultRow `json:"a,omitempty"`
	B        *ResultRow `json:"b,omitempty"`
}

CasePair is one case's result in each run; either side may be missing.

type CaseView 

type CaseView struct {
	ID           string             `json:"id"`
	SuiteID      string             `json:"suiteId"`
	Name         string             `json:"name"`
	Input        string             `json:"input"`
	Expected     string             `json:"expected,omitempty"`
	ScenarioType string             `json:"scenarioType"`
	Tags         []string           `json:"tags"`
	Scorers      []ScorerConfigView `json:"scorers"`
	Context      map[string]any     `json:"context"`
	Metadata     map[string]any     `json:"metadata"`
	RedTeam      *RedTeamRef        `json:"redTeam,omitempty"`
	CreatedAt    string             `json:"createdAt"`
	UpdatedAt    string             `json:"updatedAt"`
}

CaseView is a test case. A red-team case's not_contains substring is the system prompt it looks for, so it is redacted to its length.

type ConfigView 

type ConfigView struct {
	DefaultModel        string       `json:"defaultModel"`
	Temperature         float64      `json:"temperature"`
	PassThreshold       float64      `json:"passThreshold"`
	RegressionThreshold float64      `json:"regressionThreshold"`
	Concurrency         int          `json:"concurrency"`
	Targets             []TargetView `json:"targets"`
	Scorers             []ScorerView `json:"scorers"`
}

ConfigView is the engine's effective configuration and what it can run.

type Deps 

type Deps struct {
	// Engine is required.
	Engine *engine.Engine
	// DashboardAppID is the app served when the principal carries no
	// app_id claim. Empty means such requests are refused.
	DashboardAppID string
	// Logger receives every error a handler maps to INTERNAL. Optional.
	Logger forge.Logger
}

Deps bundles what the handlers need.

type MetricDelta 

type MetricDelta struct {
	Metric string  `json:"metric"`
	A      float64 `json:"a"`
	B      float64 `json:"b"`
	Delta  float64 `json:"delta"`
}

MetricDelta is one aggregate metric for A, for B, and B minus A.

type PromptVersionView 

type PromptVersionView struct {
	ID             string   `json:"id"`
	SuiteID        string   `json:"suiteId"`
	Version        int      `json:"version"`
	SystemPrompt   string   `json:"systemPrompt"`
	Changelog      string   `json:"changelog,omitempty"`
	IsCurrent      bool     `json:"isCurrent"`
	RunCount       int      `json:"runCount"`
	LatestPassRate *float64 `json:"latestPassRate,omitempty"`
	CreatedAt      string   `json:"createdAt"`
}

PromptVersionView is one version of a suite's prompt. RunCount and LatestPassRate come from the runs that recorded this version, not from the never-written columns on the version row.

type RedTeamRef 

type RedTeamRef struct {
	AttackType string `json:"attackType"`
}

RedTeamRef marks a red-team case or result.

type RedTeamReport 

type RedTeamReport struct {
	JudgedBy []string       `json:"judgedBy"`
	ByType   []RedTeamTally `json:"byType"`
	Total    int            `json:"total"`
	Bypassed int            `json:"bypassed"`
	Unscored int            `json:"unscored"`
}

RedTeamReport is built when read from a run's results and its cases. JudgedBy is the sorted, deduplicated union of the scorers the run recorded and the scorers carried by every red-team case that has a result in the run (the engine runs a case's own scorers on top of the run's). Three of the five generators attach no scorer of their own, so the rate means only what those scorers can detect. JudgedBy and ByType are never null.

type RedTeamTally 

type RedTeamTally struct {
	AttackType string `json:"attackType"`
	Total      int    `json:"total"`
	Bypassed   int    `json:"bypassed"`
	Unscored   int    `json:"unscored"`
}

RedTeamTally is one attack type's outcome in a run. A bypass is a result that failed its scoring; unscored counts results that errored.

type Redaction 

type Redaction struct {
	Key    string `json:"key"`
	Length int    `json:"length"`
}

Redaction says a config value was withheld and how long it was.

type RegressedCaseView 

type RegressedCaseView struct {
	CaseID   string  `json:"caseId"`
	CaseName string  `json:"caseName"`
	OldScore float64 `json:"oldScore"`
	NewScore float64 `json:"newScore"`
	Delta    float64 `json:"delta"`
}

RegressedCaseView is a case that fell beyond the threshold.

type RegressionSummary 

type RegressionSummary struct {
	RunID      string      `json:"runId"`
	SuiteID    string      `json:"suiteId"`
	SuiteName  string      `json:"suiteName"`
	CreatedAt  string      `json:"createdAt"`
	Baseline   BaselineRef `json:"baseline"`
	WorstDelta float64     `json:"worstDelta"`
}

RegressionSummary is one recently regressed run.

type RegressionView 

type RegressionView struct {
	State             string              `json:"state"`
	Reason            string              `json:"reason,omitempty"`
	Baseline          *BaselineRef        `json:"baseline,omitempty"`
	Threshold         *float64            `json:"threshold,omitempty"`
	ThresholdSource   string              `json:"thresholdSource,omitempty"`
	HasRegression     bool                `json:"hasRegression"`
	WorstDelta        *float64            `json:"worstDelta,omitempty"`
	PassRateDelta     float64             `json:"passRateDelta"`
	AvgScoreDelta     float64             `json:"avgScoreDelta"`
	DimensionDeltas   map[string]float64  `json:"dimensionDeltas"`
	RegressedCases    []RegressedCaseView `json:"regressedCases"`
	MissingCases      []CaseNameView      `json:"missingCases"`
	NewCases          []CaseNameView      `json:"newCases"`
	MissingDimensions []string            `json:"missingDimensions"`
}

RegressionView is an explicit state machine, so no page has to infer whether a missing baseline means "passed":

  • running: the run has not finished; no verdict yet.
  • notComparable: Reason is runFailed, runCancelled or otherSuite.
  • noBaseline: the suite has no current baseline to compare against.
  • compared: the remaining fields are set.

type ResultRow 

type ResultRow struct {
	ID              string             `json:"id"`
	CaseID          string             `json:"caseId"`
	CaseName        string             `json:"caseName"`
	Status          string             `json:"status"`
	Score           float64            `json:"score"`
	LatencyMs       int                `json:"latencyMs"`
	TokensUsed      int                `json:"tokensUsed"`
	Cost            float64            `json:"cost"`
	DimensionScores map[string]float64 `json:"dimensionScores"`
	RedTeam         *RedTeamRef        `json:"redTeam,omitempty"`
	// Scorers is each scorer's verdict on the case, in the order the engine
	// recorded them: case scorers that could not be built first, then the
	// run's scorers and the case's own as they ran.
	// Reasons and details stay on results.detail: a reason can quote the
	// output, and a red-team output is only shown when asked for.
	Scorers []ScorerVerdict `json:"scorers"`
	// Error carries the target's own error text for this case (what
	// target.Call returned), or the scorers' when one could not judge it,
	// stored as written. It comes from the code under test and the judges,
	// never from the store.
	Error string `json:"error,omitempty"`
}

ResultRow is one case's result without its output, which can be large.

type ResultView 

type ResultView struct {
	ResultRow
	Output        string             `json:"output"`
	OutputLength  int                `json:"outputLength"`
	ScorerResults []ScorerResultView `json:"scorerResults"`
	RunTrace      *TraceView         `json:"runTrace,omitempty"`
}

ResultView is one case's full result, output included. The output may be hostile content (a jailbreak the model complied with); the client renders it as text and collapses red-team outputs.

type RunSettingsView 

type RunSettingsView struct {
	PassThreshold       *float64 `json:"passThreshold,omitempty"`
	RegressionThreshold *float64 `json:"regressionThreshold,omitempty"`
	Concurrency         *int     `json:"concurrency,omitempty"`
	Target              string   `json:"target,omitempty"`
	Scorers             []string `json:"scorers,omitempty"`
	Model               string   `json:"model,omitempty"`
	PromptVersionID     string   `json:"promptVersionId,omitempty"`
}

RunSettingsView is what a run recorded when it started. A field is absent when the run predates recording, which means "not recorded", never today's configuration.

type RunView 

type RunView struct {
	ID              string             `json:"id"`
	SuiteID         string             `json:"suiteId"`
	SuiteName       string             `json:"suiteName"`
	Model           string             `json:"model"`
	Temperature     float64            `json:"temperature"`
	State           string             `json:"state"`
	TotalCases      int                `json:"totalCases"`
	CompletedCases  int                `json:"completedCases"`
	Passed          int                `json:"passed"`
	Failed          int                `json:"failed"`
	Errored         int                `json:"errored"`
	PassRate        float64            `json:"passRate"`
	AvgScore        float64            `json:"avgScore"`
	AvgLatencyMs    int                `json:"avgLatencyMs"`
	TotalTokens     int                `json:"totalTokens"`
	TotalCost       float64            `json:"totalCost"`
	DimensionScores map[string]float64 `json:"dimensionScores"`
	Settings        RunSettingsView    `json:"settings"`
	// Error is why the run failed, as the engine recorded it. The engine
	// writes a generic message here for a store failure; it never carries
	// store error text.
	Error          string  `json:"error,omitempty"`
	CreatedAt      string  `json:"createdAt"`
	CompletedAt    *string `json:"completedAt,omitempty"`
	LastProgressAt *string `json:"lastProgressAt,omitempty"`
}

RunView is a run. CompletedCases and Errored always come from the stored results, which are the only place those numbers live. While a run is running its counters come from the results stored so far.

type ScorerConfigView 

type ScorerConfigView struct {
	Name     string         `json:"name"`
	Config   map[string]any `json:"config"`
	Redacted *Redaction     `json:"redacted,omitempty"`
}

ScorerConfigView is a case's scorer config as the client may see it.

type ScorerResultView 

type ScorerResultView struct {
	ScorerName string         `json:"scorerName"`
	Score      float64        `json:"score"`
	Passed     bool           `json:"passed"`
	Reason     string         `json:"reason"`
	Dimension  string         `json:"dimension,omitempty"`
	Details    map[string]any `json:"details,omitempty"`
	// Errored is true when the scorer could not judge the case. Passed is
	// false then too, but the output was not failed.
	Errored bool `json:"errored,omitempty"`
}

ScorerResultView is one scorer's verdict on a case.

type ScorerVerdict 

type ScorerVerdict struct {
	Name   string `json:"name"`
	Passed bool   `json:"passed"`
	// Errored is true when the scorer could not judge the case, which is
	// not the same as failing it.
	Errored bool `json:"errored,omitempty"`
}

ScorerVerdict is one scorer's pass or fail on a result.

type ScorerView 

type ScorerView struct {
	Name           string `json:"name"`
	Description    string `json:"description"`
	Dimension      string `json:"dimension,omitempty"`
	UsesLLM        bool   `json:"usesLlm"`
	RequiresConfig bool   `json:"requiresConfig"`
}

ScorerView is a registered scorer. RequiresConfig scorers cannot be picked for a run without configuration; UsesLLM scorers cost money.

type StepView 

type StepView struct {
	Index      int    `json:"index"`
	Type       string `json:"type"`
	Output     string `json:"output"`
	TokensUsed int    `json:"tokensUsed"`
}

StepView is one reasoning step.

type SuiteView 

type SuiteView struct {
	ID                   string       `json:"id"`
	Name                 string       `json:"name"`
	Description          string       `json:"description"`
	Model                string       `json:"model"`
	Temperature          float64      `json:"temperature"`
	PersonaRef           string       `json:"personaRef,omitempty"`
	SystemPrompt         string       `json:"systemPrompt"`
	PromptSource         string       `json:"promptSource"`
	CurrentPromptVersion *VersionRef  `json:"currentPromptVersion,omitempty"`
	CurrentBaseline      *BaselineRef `json:"currentBaseline,omitempty"`
	CaseCount            int64        `json:"caseCount"`
	CreatedAt            string       `json:"createdAt"`
	UpdatedAt            string       `json:"updatedAt"`
}

SuiteView is a suite as the dashboard shows it. SystemPrompt is the suite's own prompt; PromptSource says which prompt runs actually send: "version" when a prompt version is current, otherwise "suite".

type TargetView 

type TargetView struct {
	Name        string `json:"name"`
	Description string `json:"description"`
}

TargetView is a registered target.

type ToolView 

type ToolView struct {
	ToolName  string `json:"toolName"`
	Arguments string `json:"arguments"`
	Result    string `json:"result"`
	Error     string `json:"error,omitempty"`
}

ToolView is one tool call.

type TraceView 

type TraceView struct {
	Steps     []StepView `json:"steps"`
	ToolCalls []ToolView `json:"toolCalls"`
}

TraceView is an agent run's trace.

type TrendPoint 

type TrendPoint struct {
	RunID           string             `json:"runId"`
	CreatedAt       string             `json:"createdAt"`
	PassRate        float64            `json:"passRate"`
	AvgScore        float64            `json:"avgScore"`
	DimensionScores map[string]float64 `json:"dimensionScores"`
	TotalCost       float64            `json:"totalCost"`
	Settings        RunSettingsView    `json:"settings"`
}

TrendPoint is one completed run on a suite's trend line.

type VersionRef 

type VersionRef struct {
	ID      string `json:"id"`
	Version int    `json:"version"`
}

VersionRef names a prompt version.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.