sentinel

package module
v1.10.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 4, 2026 License: MIT Imports: 25 Imported by: 0

Documentation

Overview

Package sentinel connects the Sentinel engine to Cortex lifecycle hooks. Automatic run evaluation is not implemented.

Index

Constants

View Source 
const AssuranceCorpusRevision = "assurance-corpus-v1"

AssuranceCorpusRevision identifies the fixed case and ablation roster.

Variables

This section is empty.

Functions

func EngineOption 

func EngineOption(c vessel.Vessel, opts ...Option) engine.Option

EngineOption returns a cortex engine.Option that auto-discovers a Sentinel engine from the DI container and registers the Sentinel plugin with Cortex. If Sentinel is not available, this is a no-op.

func NewAgentClient 

func NewAgentClient(eng *engine.Engine) target.AgentClient

NewAgentClient creates a Sentinel AgentClient backed by a Cortex engine. Agent lookup is scope-guarded (scope travels on ctx), so no app identifier is needed to resolve the agent to run.

Types

type AssuranceArm 

type AssuranceArm struct {
	ID           string `json:"id"`
	Verification bool   `json:"verification"`
	Reflection   bool   `json:"reflection"`
	Memory       bool   `json:"memory"`
	Team         bool   `json:"team"`
}

AssuranceArm declares exactly one change from the verification arm. Prepared digests and authenticated host bindings must establish that the flags were used.

func FixedAssuranceArms 

func FixedAssuranceArms() []AssuranceArm

FixedAssuranceArms includes the unchecked baseline and three controlled ablations.

type AssuranceArmReport 

type AssuranceArmReport struct {
	Arm     AssuranceArm     `json:"arm"`
	Metrics AssuranceMetrics `json:"metrics"`
}

AssuranceArmReport keeps every planned trial, including missing observations.

type AssuranceAuthorizer 

type AssuranceAuthorizer interface {
	BenchmarkAuthorizer
	AuthorizeAssurance(context.Context, AssuranceManifest, string, []AssuranceResultRef) error
	AuthorizeAssuranceObservation(context.Context, AssuranceManifest, AssuranceResultRef, AssuranceObservation) error
}

AssuranceAuthorizer authenticates preregistration, exact scope, runtime arm settings, independent assessment, source ledger and current evidence grants. It checks all references again before disclosure, including earlier reads. Benchmark authority additionally authenticates each ablation's exact job map.

type AssuranceCase 

type AssuranceCase struct {
	ID       string `json:"id"`
	Domain   string `json:"domain"`
	Input    string `json:"input"`
	Required string `json:"required"`
	Security bool   `json:"security"`
	Recovery bool   `json:"recovery"`
}

AssuranceCase fixes an input and independent acceptance requirement. Security and recovery cases require complete observation counters before qualification.

func FixedAssuranceCases 

func FixedAssuranceCases() []AssuranceCase

FixedAssuranceCases returns a detached, versioned regression corpus. It is deliberately public fixture material, never an uncontaminated live holdout.

type AssuranceComparison 

type AssuranceComparison struct {
	ArmID  string          `json:"arm_id"`
	Roster BenchmarkRoster `json:"roster"`
}

AssuranceComparison binds an ablation to the existing recorded comparison. Host authorization verifies this roster's case/arm mapping to the manifest.

type AssuranceComparisonReport 

type AssuranceComparisonReport struct {
	ArmID      string                         `json:"arm_id"`
	Comparison *comparison.RecordedComparison `json:"comparison"`
}

AssuranceComparisonReport uses Sentinel's comparison without changing its completeness, required checks, confidence interval or unknown-cost semantics.

type AssuranceCorpusRunner 

type AssuranceCorpusRunner struct {
	// contains filtered or unexported fields
}

AssuranceCorpusRunner reads at most 60 observations and four frozen comparisons. It never calls a model, changes an experiment or dispatches external effects.

func NewAssuranceCorpusRunner 

func NewAssuranceCorpusRunner(repository evidence.Repository, store comparison.RecordedStore, authority AssuranceAuthorizer, manifest AssuranceManifest) (*AssuranceCorpusRunner, error)

NewAssuranceCorpusRunner freezes a complete case/arm roster without reading data.

func (*AssuranceCorpusRunner) Digest 

func (r *AssuranceCorpusRunner) Digest() string

Digest returns the immutable manifest identity to bind captured observations.

func (*AssuranceCorpusRunner) Report 

func (r *AssuranceCorpusRunner) Report(ctx context.Context, refs []AssuranceResultRef) (*AssuranceReport, error)

Report hydrates only explicitly named evidence under a 30-second deadline.

type AssuranceCoverage 

type AssuranceCoverage struct {
	Claims       int `json:"claims"`
	Verification int `json:"verification"`
	Abstention   int `json:"abstention"`
	Recovery     int `json:"recovery"`
	Security     int `json:"security"`
	Latency      int `json:"latency"`
	Cost         int `json:"cost"`
}

AssuranceCoverage counts measured fields separately from planned trials.

type AssuranceManifest 

type AssuranceManifest struct {
	SchemaVersion       int                   `json:"schema_version"`
	CorpusRevision      string                `json:"corpus_revision"`
	Scope               cortex.Scope          `json:"scope"`
	Mode                string                `json:"mode"`
	ModelRevision       string                `json:"model_revision"`
	EnvironmentRevision string                `json:"environment_revision"`
	Producer            string                `json:"producer"`
	FrozenAt            time.Time             `json:"frozen_at"`
	Deadline            time.Time             `json:"deadline"`
	Slots               []AssuranceSlot       `json:"slots"`
	Comparisons         []AssuranceComparison `json:"comparisons,omitempty"`
}

AssuranceManifest must be frozen before any observation is produced. Mode is deterministic_fixture or installed_measurement; neither means live-qualified. The public corpus cannot serve as an unexposed optimizer adoption holdout.

type AssuranceMetrics 

type AssuranceMetrics struct {
	Planned                 int                   `json:"planned"`
	Calibration             CalibrationStatistics `json:"calibration"`
	Coverage                AssuranceCoverage     `json:"coverage"`
	FalseSuccess            AssuranceRate         `json:"false_success"`
	UnresolvedSuccessClaims int                   `json:"unresolved_success_claims"`
	VerifiedTaskSuccess     AssuranceRate         `json:"verified_task_success"`
	Abstention              AssuranceRate         `json:"abstention"`
	SelectiveAccuracy       AssuranceRate         `json:"selective_accuracy"`
	Recovery                AssuranceRate         `json:"recovery"`
	RequiredSecurityTrials  int                   `json:"required_security_trials"`
	SecurityObserved        int                   `json:"security_observed"`
	SecurityStatus          string                `json:"security_status"`
	UnauthorizedOperations  int                   `json:"unauthorized_operations"`
	SecretLeaks             int                   `json:"secret_leaks"`
	ArtifactSubstitutions   int                   `json:"artifact_substitutions"`
	DuplicateEffects        int                   `json:"duplicate_effects"`
	UnsafePublications      int                   `json:"unsafe_publications"`
	P50Millis               *int64                `json:"p50_ms"`
	P95Millis               *int64                `json:"p95_ms"`
	KnownCostMicros         map[string]int64      `json:"known_cost_micros"`
	TotalCostMicros         *int64                `json:"total_cost_micros"`
	CostPerVerifiedMicros   *float64              `json:"cost_per_verified_micros"`
}

AssuranceMetrics preserves all planned slots and partial cost subtotals. Latency percentiles use nearest rank over observed durations only. Cost totals and cost per verified result require complete accounting in a single currency.

type AssuranceObservation 

type AssuranceObservation struct {
	SchemaVersion  int                `json:"schema_version"`
	ManifestDigest string             `json:"manifest_digest"`
	CaseID         string             `json:"case_id"`
	ArmID          string             `json:"arm_id"`
	PreparedDigest string             `json:"prepared_digest"`
	SourceDigest   string             `json:"source_digest"`
	ObservedAt     time.Time          `json:"observed_at"`
	Telemetry      AssuranceTelemetry `json:"telemetry"`
}

AssuranceObservation is retained, independently assessed telemetry for one planned slot. SourceDigest identifies the underlying run, verifier and usage ledger authenticated by the host, not a model's self-assessment.

type AssuranceRate 

type AssuranceRate struct {
	Numerator   int      `json:"numerator"`
	Denominator int      `json:"denominator"`
	Value       *float64 `json:"value"`
}

AssuranceRate retains its observed numerator and denominator. A zero denominator has no estimate; callers must inspect coverage for missing data.

type AssuranceReport 

type AssuranceReport struct {
	SchemaVersion      int                         `json:"schema_version"`
	ManifestDigest     string                      `json:"manifest_digest"`
	Mode               string                      `json:"mode"`
	Arms               []AssuranceArmReport        `json:"arms"`
	Comparisons        []AssuranceComparisonReport `json:"comparisons"`
	MissingComparisons []string                    `json:"missing_comparisons"`
	Evidence           []AssuranceResultRef        `json:"evidence"`
}

AssuranceReport is evidence review, never an activation or rollout decision.

type AssuranceResultRef 

type AssuranceResultRef struct {
	CaseID   string       `json:"case_id"`
	ArmID    string       `json:"arm_id"`
	Artifact evidence.Ref `json:"artifact"`
}

AssuranceResultRef is an explicit slot mapping. Omitted slots remain missing.

type AssuranceSlot 

type AssuranceSlot struct {
	CaseID         string `json:"case_id"`
	ArmID          string `json:"arm_id"`
	PreparedDigest string `json:"prepared_digest"`
}

AssuranceSlot binds each fixed case/arm pair to the actual prepared runtime.

type AssuranceTelemetry 

type AssuranceTelemetry struct {
	Outcome                CalibrationOutcome `json:"outcome"`
	ClaimedSuccess         *bool              `json:"claimed_success,omitempty"`
	Verified               *bool              `json:"verified,omitempty"`
	Abstained              *bool              `json:"abstained,omitempty"`
	Recovered              *bool              `json:"recovered,omitempty"`
	UnauthorizedOperations *int               `json:"unauthorized_operations,omitempty"`
	SecretLeaks            *int               `json:"secret_leaks,omitempty"`
	ArtifactSubstitutions  *int               `json:"artifact_substitutions,omitempty"`
	DuplicateEffects       *int               `json:"duplicate_effects,omitempty"`
	UnsafePublications     *int               `json:"unsafe_publications,omitempty"`
	LatencyMillis          *int64             `json:"latency_ms,omitempty"`
	CostMicros             *int64             `json:"cost_micros,omitempty"`
	Currency               string             `json:"currency,omitempty"`
	Prediction             *float64           `json:"prediction,omitempty"`
}

AssuranceTelemetry is independently authenticated observation data. Nil fields remain unknown. CostMicros includes execution, evaluation and recovery charges. Verified is the system's claim, while Outcome is the independent assessor's result, so a failed outcome with Verified=true remains a measurable false success.

type BenchmarkAuthorizer 

type BenchmarkAuthorizer interface {
	AuthorizeBenchmark(context.Context, BenchmarkRoster, string) error
}

BenchmarkAuthorizer approves current cross-system scope mapping and disclosure. The digest identifies the complete frozen roster, not a model-selected subset.

type BenchmarkGroup 

type BenchmarkGroup struct {
	Domain    string                      `json:"domain"`
	Plan      comparison.RecordedPlanRef  `json:"plan"`
	Baseline  []comparison.RecordedJobRef `json:"baseline"`
	Candidate []comparison.RecordedJobRef `json:"candidate"`
}

BenchmarkGroup freezes one domain's exact plan and full evaluation job roster. Missing and cancelled trials belong in these jobs, never in an omitted batch.

type BenchmarkRoster 

type BenchmarkRoster struct {
	Scope        cortex.Scope     `json:"scope"`
	AppID        string           `json:"app_id"`
	TenantID     string           `json:"tenant_id"`
	Revision     string           `json:"revision"`
	Seed         uint64           `json:"seed"`
	MinimumCases int              `json:"minimum_cases"`
	Groups       []BenchmarkGroup `json:"groups"`
}

BenchmarkRoster is host configuration frozen before reading scores. Domains describe the corpus; this adapter does not claim that a host's corpus is live-qualified.

type CalibrationAuthorizer 

type CalibrationAuthorizer interface {
	AuthorizeCalibration(context.Context, engine.OperationalCalibrationRequest, CalibrationRegistration) error
}

CalibrationAuthorizer checks the current run's authority, deployment pins, comparable deadline constraints and current cohort revocation/contamination. It runs before and after evidence reads. Returning nil attests those facts.

type CalibrationBin 

type CalibrationBin struct {
	Lower          float64 `json:"lower"`
	Upper          float64 `json:"upper"`
	Count          int     `json:"count"`
	MeanPrediction float64 `json:"mean_prediction"`
	SuccessRate    float64 `json:"success_rate"`
}

CalibrationBin retains prediction coverage alongside observed success.

type CalibrationCohort 

type CalibrationCohort struct {
	SchemaVersion     int                          `json:"schema_version"`
	Revision          string                       `json:"revision"`
	EvaluatorRevision string                       `json:"evaluator_revision"`
	Binding           cognitive.CalibrationBinding `json:"binding"`
	DiscoveryItems    []string                     `json:"discovery_items"`
	HoldoutItems      []string                     `json:"holdout_items"`
	WindowStart       time.Time                    `json:"window_start"`
	WindowEnd         time.Time                    `json:"window_end"`
	ExpiresAt         time.Time                    `json:"expires_at"`
	Trials            []CalibrationTrial           `json:"trials"`
}

CalibrationCohort retains the frozen dataset roster and every trial outcome. This first method permits one trial per distinct item. Repeated or correlated samples need a separately versioned cluster-aware estimator.

type CalibrationMeasurement 

type CalibrationMeasurement struct {
	Profile    cognitive.CalibrationProfile `json:"profile"`
	Statistics CalibrationStatistics        `json:"statistics"`
}

CalibrationMeasurement binds the absolute profile and diagnostic statistics. Paired candidate deltas remain owned by Sentinel's comparison package.

func MeasureCalibration 

func MeasureCalibration(c CalibrationCohort, ref evidence.Ref) (CalibrationMeasurement, error)

MeasureCalibration computes a reproducible profile from authenticated cohort bytes. This pure function does not authenticate the producer or publish evidence.

type CalibrationOutcome 

type CalibrationOutcome string

CalibrationOutcome partitions every frozen holdout trial, including absent results. 

const (
	CalibrationSuccess     CalibrationOutcome = "success"
	CalibrationFailure     CalibrationOutcome = "failure"
	CalibrationMissing     CalibrationOutcome = "missing"
	CalibrationGraderError CalibrationOutcome = "grader_error"
	CalibrationCancelled   CalibrationOutcome = "cancelled"
)

type CalibrationRegistration 

type CalibrationRegistration struct {
	AgentID  id.AgentID
	Model    string
	Binding  cognitive.CalibrationBinding
	Profile  evidence.Ref
	Producer string
}

CalibrationRegistration is host-owned routing to one immutable measured profile. Task class and deployment pins cannot be selected by the model.

type CalibrationStatistics 

type CalibrationStatistics struct {
	Successes       int                           `json:"successes"`
	Failures        int                           `json:"failures"`
	Missing         int                           `json:"missing"`
	GraderErrors    int                           `json:"grader_errors"`
	Cancelled       int                           `json:"cancelled"`
	Estimate        float64                       `json:"estimate"`
	Interval        cognitive.CalibrationInterval `json:"interval"`
	PredictedTrials int                           `json:"predicted_trials"`
	Brier           *float64                      `json:"brier,omitempty"`
	Bins            []CalibrationBin              `json:"bins"`
}

CalibrationStatistics uses all planned holdout trials as the task-success denominator. Brier and bins use only resolved trials with prior predictions; PredictedTrials makes that narrower denominator explicit.

type CalibrationTrial 

type CalibrationTrial struct {
	ItemDigest string             `json:"item_digest"`
	Outcome    CalibrationOutcome `json:"outcome"`
	Prediction *float64           `json:"prediction,omitempty"`
}

CalibrationTrial is one independent holdout item. Prediction, when present, must be captured before its outcome by the trusted evaluator. It is measured for calibration only and never establishes execution authority.

type EvidenceCalibrationProvider 

type EvidenceCalibrationProvider struct {
	// contains filtered or unexported fields
}

EvidenceCalibrationProvider hydrates and recomputes registered cohort evidence. It has no dynamic discovery or score-based cohort selection.

func NewEvidenceCalibrationProvider 

func NewEvidenceCalibrationProvider(repository evidence.Repository, authority CalibrationAuthorizer, registrations []CalibrationRegistration) (*EvidenceCalibrationProvider, error)

NewEvidenceCalibrationProvider freezes a bounded host roster. Registering two cohorts for the same runtime pins is rejected instead of choosing the best score.

func (*EvidenceCalibrationProvider) ObserveCalibration 

func (p *EvidenceCalibrationProvider) ObserveCalibration(ctx context.Context, request engine.OperationalCalibrationRequest) (engine.OperationalCalibrationObservation, error)

ObserveCalibration returns measured origin only after current authorization, canonical artifact validation and exact reproduction of the stored statistics.

type FrozenBenchmark 

type FrozenBenchmark struct {
	// contains filtered or unexported fields
}

FrozenBenchmark compares retained evaluations with Sentinel's existing paired bootstrap math. It cannot execute tools, approve effects or activate a policy.

func NewFrozenBenchmark 

func NewFrozenBenchmark(store comparison.RecordedStore, authority BenchmarkAuthorizer, roster BenchmarkRoster) (*FrozenBenchmark, error)

NewFrozenBenchmark detaches a bounded roster of at most 32 plans and 256 jobs.

func (*FrozenBenchmark) Compare 

func (b *FrozenBenchmark) Compare(ctx context.Context) (*comparison.RecordedComparison, error)

Compare authenticates exact scope and pinned requests, then retains Sentinel's incompleteness, required failures and unknown cost without inventing zeroes.

type Option 

type Option func(*pluginOptions)

Option configures the Plugin.

func WithAutoEval 

func WithAutoEval(enabled bool) Option

WithAutoEval enables or disables automatic evaluation after each Cortex agent run completes.

type Plugin 

type Plugin struct {
	// contains filtered or unexported fields
}

Plugin integrates Sentinel evaluation capabilities into Cortex. It implements Cortex's plugin.Extension interface and lifecycle hooks to automatically evaluate agent runs when configured.

func New 

func New(eng *sentinelengine.Engine, opts ...Option) *Plugin

New creates a new Sentinel Cortex plugin.

func (*Plugin) Engine 

func (p *Plugin) Engine() *sentinelengine.Engine

Engine returns the underlying Sentinel engine.

func (*Plugin) Name 

func (p *Plugin) Name() string

Name returns the plugin name.

func (*Plugin) OnRunCompleted 

func (p *Plugin) OnRunCompleted(_ context.Context, _ id.AgentID, _ id.AgentRunID, _ string, _ time.Duration) error

OnRunCompleted is called after every Cortex agent run. When auto-eval is enabled and a matching evaluation suite exists, it triggers a Sentinel evaluation asynchronously.

func (*Plugin) OnShutdown 

func (p *Plugin) OnShutdown(_ context.Context) error

OnShutdown is called during graceful shutdown.

Source Files

View all Source files

Directories

Path Synopsis
Package optimization adapts captured Cortex executions to Sentinel scoring.
 Package optimization adapts captured Cortex executions to Sentinel scoring.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.