Documentation
¶
Overview ¶
Package core holds the shared data types of the gaslit engine. It is a leaf package: every other internal package imports it, and it imports nothing internal. The module root re-exports these types under the public `gaslit` package via type aliases.
Index ¶
- Constants
- func Clamp01(v float64) float64
- func DefaultNLIProbe(question, r1, r2 string) string
- func ExtractJSONObject(raw string) (string, error)
- func Tokenize(s string) map[string]bool
- type AgentConfig
- type AggregationFn
- type Answer
- type AnswerSchema
- type AttributionFn
- type BudgetError
- type CallOption
- func WithBudgetCall(b CostBudget) CallOption
- func WithClaimCall(s string) CallOption
- func WithConfidenceCall(c float64) CallOption
- func WithConsistencyScorerCall(s ConsistencyScorer) CallOption
- func WithGrillCountCall(n int) CallOption
- func WithGrillStrategyCall(s GrillStrategy) CallOption
- func WithHooksCall(h Hooks) CallOption
- func WithMaxRoundsCall(n int) CallOption
- func WithMetadataCall(m map[string]any) CallOption
- func WithOptionLabelsCall(l []string) CallOption
- func WithProtocolCall(p Protocol) CallOption
- func WithThresholdsCall(t Thresholds) CallOption
- func WithVerboseCall(v bool) CallOption
- type CallOptions
- type Classifier
- type Config
- type ConsistencyScorer
- type CostBudget
- type DiagnosticResult
- type DriftMetric
- type DriftVerdict
- type Event
- type EventType
- type FullDebateSummary
- type GrillStrategy
- type Hooks
- type InstrumentationStats
- type JSONError
- type MCQSchema
- type OpenEndedSchema
- type Option
- func WithAgentConfig(name string, ac AgentConfig) Option
- func WithAggregation(fn AggregationFn) Option
- func WithAnswerSchema(s AnswerSchema) Option
- func WithAttribution(fn AttributionFn) Option
- func WithBudget(b CostBudget) Option
- func WithClassifier(cl Classifier) Option
- func WithConcurrency(n int) Option
- func WithConsistencyModel(m model.BaseModel[*schema.Message]) Option
- func WithConsistencyScorer(s ConsistencyScorer) Option
- func WithDriftMetric(m DriftMetric) Option
- func WithExtraTools(tools ...tool.BaseTool) Option
- func WithGrillCount(n int) Option
- func WithGrillStrategy(s GrillStrategy) Option
- func WithHooks(h Hooks) Option
- func WithLogger(l *slog.Logger) Option
- func WithMaxRetries(n int) Option
- func WithMaxRounds(n int) Option
- func WithOptionLabels(labels []string) Option
- func WithPacingDelay(d time.Duration) Option
- func WithPromptTemplates(t PromptTemplates) Option
- func WithProtocol(p Protocol) Option
- func WithRecorder(r Recorder) Option
- func WithRedact(fn func(string) string) Option
- func WithRequestTimeout(d time.Duration) Option
- func WithThresholds(t Thresholds) Option
- func WithVerbose(v bool) Option
- type Phase
- type PromptResult
- type PromptTemplates
- type Protocol
- type QuadrantState
- type Recorder
- type StreamEvent
- type Thresholds
Constants ¶
const DefaultJSONInstruction = `` /* 571-byte string literal not displayed */
DefaultJSONInstruction is the output contract agents must follow for multiple-choice answers. Custom AnswerSchema implementations may provide their own instructions instead.
const OpenEndedInstruction = `` /* 223-byte string literal not displayed */
OpenEndedInstruction is the output contract for free-form (option-less) questions.
Variables ¶
This section is empty.
Functions ¶
func DefaultNLIProbe ¶
DefaultNLIProbe is the built-in entailment probe builder. It is used as the LLM scorer's fallback when no PromptTemplates.NLIProbe is wired.
func ExtractJSONObject ¶
ExtractJSONObject finds the outermost balanced JSON object in raw output.
Types ¶
type AgentConfig ¶
type AgentConfig struct {
// Instruction, when non-empty, is a per-agent system instruction that is
// prepended to every prompt sent to this agent.
Instruction string
// Weight is the influence weight used by weighted aggregation functions.
// Zero means the default weight of 1.0.
Weight float64
// SkipPhases lists the phases this agent is excluded from.
SkipPhases map[Phase]bool
}
AgentConfig customizes a single registered panelist, keyed by agent name.
type AggregationFn ¶
type AggregationFn func(after map[string]*Answer, labels []string, weights map[string]float64) string
AggregationFn derives the winning option from the panel's final positions. It returns the winning option label, or "" when there is no clear winner.
type Answer ¶
type Answer struct {
Answer string `json:"answer"`
Confidences map[string]float64 `json:"confidences,omitempty"`
Reasoning string `json:"reasoning"`
Challenge *challengeRequest `json:"challenge,omitempty"`
}
Answer is a single agent's position on a question: its pick, per-option confidences (absent for open-ended questions), reasoning, and an optional nomination of another panelist for cross-examination.
type AnswerSchema ¶
type AnswerSchema interface {
// Instruction is the output-contract text appended to prompts.
Instruction() string
// Parse extracts an Answer from raw model output. It must tolerate leading
// prose, markdown fences, and trailing chatter.
Parse(raw string) (*Answer, error)
}
AnswerSchema defines the JSON contract agents must follow when answering, and how raw model output is parsed into an Answer.
type AttributionFn ¶
type AttributionFn func(res *PromptResult, labels []string, verdict *DriftVerdict)
AttributionFn determines who drove the primary shift. It mutates the verdict with PrimaryDriver and DriverImpact.
type BudgetError ¶
BudgetError is returned when a CostBudget is exhausted.
func (*BudgetError) Error ¶
func (e *BudgetError) Error() string
type CallOption ¶
type CallOption func(*CallOptions)
CallOption customizes a single engine call (Debate, Baseline, Grill, CCI, ...) without mutating the engine's construction-time configuration.
func WithBudgetCall ¶
func WithBudgetCall(b CostBudget) CallOption
func WithClaimCall ¶
func WithClaimCall(s string) CallOption
func WithConfidenceCall ¶
func WithConfidenceCall(c float64) CallOption
func WithConsistencyScorerCall ¶
func WithConsistencyScorerCall(s ConsistencyScorer) CallOption
WithConsistencyScorerCall overrides the consistency scorer for a single call (e.g. Rescore) without rebuilding the engine.
func WithGrillCountCall ¶
func WithGrillCountCall(n int) CallOption
func WithGrillStrategyCall ¶
func WithGrillStrategyCall(s GrillStrategy) CallOption
func WithHooksCall ¶
func WithHooksCall(h Hooks) CallOption
func WithMaxRoundsCall ¶
func WithMaxRoundsCall(n int) CallOption
func WithMetadataCall ¶
func WithMetadataCall(m map[string]any) CallOption
func WithOptionLabelsCall ¶
func WithOptionLabelsCall(l []string) CallOption
func WithProtocolCall ¶
func WithProtocolCall(p Protocol) CallOption
func WithThresholdsCall ¶
func WithThresholdsCall(t Thresholds) CallOption
func WithVerboseCall ¶
func WithVerboseCall(v bool) CallOption
type CallOptions ¶
type CallOptions struct {
Verbose bool
MaxRounds int
OptionLabels []string
Thresholds Thresholds
GrillStrategy GrillStrategy
GrillCount int
Protocol Protocol
Metadata map[string]any
Confidence *float64
Claim string
Hooks *Hooks
Budget CostBudget
ConsistencyScorer ConsistencyScorer
}
CallOptions is the resolved per-call configuration.
type Classifier ¶
type Classifier func(confScore, consistencyScore float64) (QuadrantState, string)
Classifier maps confidence and consistency scores onto the 2D quadrant grid.
type Config ¶
type Config struct {
// SupervisorModel is the model that powers the moderator (and, when
// ConsistencyModel is unset, the default LLM consistency scorer). Required.
SupervisorModel model.BaseModel[*schema.Message]
// ConsistencyModel powers the default LLM consistency scorer. When nil it
// falls back to SupervisorModel. Use a cheap model here and reserve
// SupervisorModel for the moderator.
ConsistencyModel model.BaseModel[*schema.Message]
// Agents is the panel of debaters. At least one is required. Agent names
// (from Agent.Name) must be unique.
Agents []adk.Agent
// Logger receives all verbose/progress output. Default slog.Default().
Logger *slog.Logger
// PacingDelay throttles each model invocation by sleeping before the call.
// Zero disables pacing. Default 0.
PacingDelay time.Duration
// RequestTimeout bounds a single model invocation. Zero disables. Default 0.
RequestTimeout time.Duration
// MaxRetries is how many corrective re-prompts to attempt when an agent
// returns unparseable output. Default 2.
MaxRetries int
// Concurrency is the number of agents queried in parallel during the
// baseline and evaluation phases. Default 1 (sequential).
Concurrency int
// MaxRounds is the number of (cross-examine → evaluate) rounds run by
// Debate. Default 1.
MaxRounds int
// Thresholds drives drift detection and quadrant classification. Zero
// Thresholds resolves to the defaults.
Thresholds Thresholds
// OptionLabels overrides the auto-generated A, B, C... labels used for
// multiple-choice questions. Leave nil to auto-generate.
OptionLabels []string
// AnswerSchema defines the JSON output contract agents must follow. Default
// is the MCQ schema. Open-ended questions (no options) always use the
// open-ended schema regardless of this field.
AnswerSchema AnswerSchema
// Protocol selects the cross-examination orchestration strategy.
Protocol Protocol
// GrillStrategy selects which agents get grilling diagnostics.
GrillStrategy GrillStrategy
// GrillCount is the number of agents grilled by GrillTopN. Default 1.
GrillCount int
// AggregationFn derives the winning option from the final positions.
AggregationFn AggregationFn
// DriftMetric computes the distribution shift between an agent's before and
// after positions. Only used for multiple-choice questions.
DriftMetric DriftMetric
// AttributionFn determines who drove the primary shift.
AttributionFn AttributionFn
// Classifier maps confidence/consistency onto the quadrant grid.
Classifier Classifier
// ConsistencyScorer evaluates whether two arguments from the same agent
// contradict each other. Defaults to an LLM scorer over ConsistencyModel.
ConsistencyScorer ConsistencyScorer
// PromptTemplates overrides any or all of the built-in prompts.
PromptTemplates PromptTemplates
// AgentOverrides holds per-agent customization keyed by agent name.
AgentOverrides map[string]AgentConfig
// ExtraTools are additional tools given to the moderator alongside the
// built-in cross_examine_agent tool.
ExtraTools []tool.BaseTool
// Hooks are observability callbacks fired at pipeline milestones.
Hooks Hooks
// Recorder receives a JSON-serializable Event stream for persistence.
Recorder Recorder
// Redact, when non-nil, is applied to every outgoing prompt before it is
// sent to any model.
Redact func(string) string
// Budget caps total work per debate.
Budget CostBudget
// Verbose enables default progress logging when a call does not override it.
Verbose bool
}
Config is the construction-time configuration for a Gaslit engine. The zero value is not usable directly; build one via Preset* helpers and/or the Option funcs. The engine applies defaults for every unset field.
func PresetDefault ¶
func PresetDefault() Config
PresetDefault returns the recommended all-purpose configuration.
func PresetLenient ¶
func PresetLenient() Config
PresetLenient returns a configuration that ignores small shifts and grades agents leniently.
func PresetStrict ¶
func PresetStrict() Config
PresetStrict returns a configuration that flags smaller shifts and demands higher confidence/consistency before an agent is graded PEAK.
type ConsistencyScorer ¶
type ConsistencyScorer interface {
Score(ctx context.Context, question, r1, r2 string) (float64, error)
}
ConsistencyScorer evaluates whether two arguments from the same agent contradict each other. Scores range 0.0 (contradicts itself) to 1.0 (clean).
type CostBudget ¶
type CostBudget struct {
// MaxCalls limits the total number of model invocations.
MaxCalls int
// MaxTokens is an approximate token budget (characters / 4) consumed by
// all outgoing prompt content.
MaxTokens int
// Deadline bounds the total wall-clock time of a debate.
Deadline time.Duration
}
CostBudget caps how much work a single debate (or debate batch) may perform. A zero value for any field means "unlimited" for that field.
type DiagnosticResult ¶
type DiagnosticResult struct {
AgentName string `json:"agent_name"`
ConfidenceScore float64 `json:"confidence_score"` // X-axis (0.0 -> 1.0)
ConsistencyScore float64 `json:"consistency_score"` // Y-axis (0.0 -> 1.0)
Quadrant QuadrantState `json:"quadrant"`
InitialReasonR1 string `json:"r1_initial_reasoning"`
CounterReasonR2 string `json:"r2_counter_reasoning"`
Recommendation string `json:"recommendation"`
}
DiagnosticResult is the outcome of grilling a single agent: where it lands on the confidence/consistency grid and what action is recommended.
type DriftMetric ¶
DriftMetric computes the distribution shift between an agent's before and after confidence distributions over the given option labels.
type DriftVerdict ¶
type DriftVerdict struct {
// HasDrift reports whether any agent flipped or significantly eroded.
HasDrift bool `json:"has_drift"`
// DriftedAgents lists the names of agents that drifted.
DriftedAgents []string `json:"drifted_agents"`
// PrimaryDriver is the agent (or the supervisor) responsible for the shift.
PrimaryDriver string `json:"primary_driver"`
// DriverImpact is a human-readable description of who swayed whom.
DriverImpact string `json:"driver_impact"`
// AgentShifts maps each agent name to its measured shift.
AgentShifts map[string]float64 `json:"agent_shifts"`
}
DriftVerdict describes whether agents changed their positions over the course of a debate, which agents moved, and who drove the shift.
type Event ¶
type Event struct {
Time time.Time `json:"time"`
Type EventType `json:"type"`
Agent string `json:"agent,omitempty"`
Data json.RawMessage `json:"data,omitempty"`
}
Event is a JSON-serializable record of a pipeline milestone, suitable for persistence or replay.
type EventType ¶
type EventType string
EventType identifies a recorded pipeline event.
const ( EventPhaseStart EventType = "phase_start" EventPhaseEnd EventType = "phase_end" EventBaseline EventType = "baseline" EventCrossExam EventType = "cross_exam" EventEval EventType = "evaluation" EventDrift EventType = "drift" EventDiagnostic EventType = "diagnostic" EventSummary EventType = "summary" EventDone EventType = "done" EventError EventType = "error" )
type FullDebateSummary ¶
type FullDebateSummary struct {
Question string `json:"question"`
Options []string `json:"options"`
OptionLabels []string `json:"option_labels"`
PromptResult *PromptResult `json:"prompt_result"`
DriftVerdict DriftVerdict `json:"drift_verdict"`
Diagnostics map[string]*DiagnosticResult `json:"diagnostics,omitempty"` // Targeted grilling results
Winner string `json:"winner"` // Aggregated winning option ("" if none)
OverallOutcome string `json:"overall_outcome"`
Metadata map[string]any `json:"metadata,omitempty"`
}
FullDebateSummary is the complete result of a Debate run: the question and options, pre/post positions, drift verdict, targeted grilling diagnostics, and the aggregated winner.
func ReclassifySummary ¶
func ReclassifySummary(s *FullDebateSummary, thr Thresholds) *FullDebateSummary
ReclassifySummary recomputes every diagnostic's quadrant and recommendation from its stored confidence and consistency scores. It makes no model calls, so it is the cheapest way to iterate on thresholds or quadrant-tallying logic against a saved FullDebateSummary.
type GrillStrategy ¶
type GrillStrategy int
GrillStrategy selects which panelists receive a grilling diagnostic.
const ( // GrillNone skips grilling entirely. GrillNone GrillStrategy = iota // GrillSingleSuspect grills the single most drifted/influential agent. GrillSingleSuspect // GrillInfluencer grills the primary driver when it is a panelist, // otherwise the most drifted agent. GrillInfluencer // GrillAllDrifted grills every agent flagged as drifted. GrillAllDrifted // GrillTopN grills the GrillCount most drifted agents. GrillTopN )
func (GrillStrategy) String ¶
func (s GrillStrategy) String() string
type Hooks ¶
type Hooks struct {
OnPhaseStart func(ctx context.Context, phase Phase)
OnPhaseEnd func(ctx context.Context, phase Phase)
OnBaseline func(ctx context.Context, agent string, ans *Answer)
OnCrossExam func(ctx context.Context, target, prompt string)
OnEval func(ctx context.Context, agent string, ans *Answer)
OnDrift func(ctx context.Context, verdict *DriftVerdict)
OnDiagnostic func(ctx context.Context, diag *DiagnosticResult)
}
Hooks are observability callbacks fired at pipeline milestones. Nil fields are skipped. All hooks receive the debate's context.
func MergeHooks ¶
MergeHooks combines a base set with overrides; non-nil override fields take precedence.
type InstrumentationStats ¶
type InstrumentationStats struct {
TotalCalls int64
Tokens int64
AgentCalls map[string]int64
PhaseDuration map[string][]time.Duration
}
InstrumentationStats is a point-in-time snapshot of engine usage.
type JSONError ¶
type JSONError struct {
// contains filtered or unexported fields
}
JSONError is a sentinel for malformed JSON extraction.
type MCQSchema ¶
type MCQSchema struct{}
MCQSchema is the default schema for multiple-choice questions: it requires an option label and a confidence for every option.
func (*MCQSchema) Instruction ¶
type OpenEndedSchema ¶
type OpenEndedSchema struct{}
OpenEndedSchema is the schema for free-form questions: agents return their answer text with no option labels or confidences.
func (*OpenEndedSchema) Instruction ¶
func (s *OpenEndedSchema) Instruction() string
type Option ¶
type Option func(*Config)
Option mutates a Config before the engine is constructed. All fields left unset still receive their defaults during engine construction.
func WithAgentConfig ¶
func WithAgentConfig(name string, ac AgentConfig) Option
func WithAggregation ¶
func WithAggregation(fn AggregationFn) Option
func WithAnswerSchema ¶
func WithAnswerSchema(s AnswerSchema) Option
func WithAttribution ¶
func WithAttribution(fn AttributionFn) Option
func WithBudget ¶
func WithBudget(b CostBudget) Option
func WithClassifier ¶
func WithClassifier(cl Classifier) Option
func WithConcurrency ¶
func WithConsistencyModel ¶
WithConsistencyModel points the default LLM consistency scorer at a separate model, decoupling it from the moderator's SupervisorModel.
func WithConsistencyScorer ¶
func WithConsistencyScorer(s ConsistencyScorer) Option
WithConsistencyScorer swaps the consistency check to a custom scorer.
func WithDriftMetric ¶
func WithDriftMetric(m DriftMetric) Option
func WithExtraTools ¶
func WithGrillCount ¶
func WithGrillStrategy ¶
func WithGrillStrategy(s GrillStrategy) Option
func WithLogger ¶
func WithMaxRetries ¶
func WithMaxRounds ¶
func WithOptionLabels ¶
func WithPacingDelay ¶
func WithPromptTemplates ¶
func WithPromptTemplates(t PromptTemplates) Option
func WithProtocol ¶
func WithRecorder ¶
func WithRedact ¶
func WithRequestTimeout ¶
func WithThresholds ¶
func WithThresholds(t Thresholds) Option
func WithVerbose ¶
type Phase ¶
type Phase int
Phase identifies a pipeline stage. It is used by hooks, instrumentation, and per-agent phase skipping.
type PromptResult ¶
PromptResult captures each agent's position before and after the debate.
type PromptTemplates ¶
type PromptTemplates struct {
// SupervisorSystem is the moderator system prompt. %s is replaced with the
// comma-joined panelist names.
SupervisorSystem string
// CrossExamineLead is formatted with (challenge prompt, schema
// instruction) when the moderator or a protocol challenges an agent.
CrossExamineLead string
// FinalEvalPrompt precedes the schema instruction when re-collecting final
// positions.
FinalEvalPrompt string
// InitialPrompt builds the baseline question prompt.
InitialPrompt func(question string, options []string, labels []string, instr string) string
// DebateTrigger builds the moderator's cross-examination briefing.
DebateTrigger func(question string, names []string, before map[string]*Answer) string
// GrillProbe builds the adversarial inversion probe.
GrillProbe func(question, claim string) string
// NLIProbe builds the consistency-evaluation prompt.
NLIProbe func(question, r1, r2 string) string
// ChallengePrompt builds a direct peer challenge used by the non-supervised
// protocols.
ChallengePrompt func(from, claim, reasoning string) string
// ConfidenceProbe asks an agent to self-report confidence in a claim.
ConfidenceProbe func(claim string) string
}
PromptTemplates lets callers override any or all of the prompts the engine builds. Function fields default to the built-in builders; string fields default to the built-in templates. Zero fields are resolved by New.
func (PromptTemplates) Resolve ¶
func (t PromptTemplates) Resolve() PromptTemplates
Resolve fills zero fields with the built-in defaults.
type Protocol ¶
type Protocol int
Protocol is the cross-examination orchestration strategy.
const ( // ProtocolSupervised has the moderator agent run the cross-examination. ProtocolSupervised Protocol = iota // ProtocolAdversarialPairs pairs panelists and runs a direct challenge // exchange within each pair. ProtocolAdversarialPairs // ProtocolRoundRobin has every panelist challenge every other panelist. ProtocolRoundRobin // ProtocolDevilsAdvocate confronts each panelist with the strongest // opposing argument available in the panel. ProtocolDevilsAdvocate )
type QuadrantState ¶
type QuadrantState string
QuadrantState is the 2D classification of an agent based on its confidence (x-axis) and semantic consistency (y-axis) under cross-examination.
const ( // QuadrantPeak indicates high confidence and high consistency. QuadrantPeak QuadrantState = "PEAK (Grounded Knowledge)" // QuadrantHallucinating indicates high confidence with fabricated contradictions. QuadrantHallucinating QuadrantState = "HALLUCINATING / MALICIOUS" // QuadrantKBDeficit indicates consistent but low-confidence answers. QuadrantKBDeficit QuadrantState = "KB_DEFICIT (Incomplete Info)" // QuadrantWrong indicates low confidence and inconsistent internal logic. QuadrantWrong QuadrantState = "WRONG / UNCERTAIN" )
func ClassifyWith ¶
func ClassifyWith(confScore, consistencyScore float64, thr Thresholds) (QuadrantState, string)
ClassifyWith maps (confidence, consistency) onto the quadrant grid.
func ThresholdClassifier ¶
func ThresholdClassifier(confScore, consistencyScore float64) (QuadrantState, string)
ThresholdClassifier is the default classifier, using the configured confidence/consistency thresholds.
type StreamEvent ¶
type StreamEvent struct {
Type EventType `json:"type"`
Phase Phase `json:"phase,omitempty"`
Agent string `json:"agent,omitempty"`
Prompt string `json:"prompt,omitempty"`
Answer *Answer `json:"answer,omitempty"`
Verdict *DriftVerdict `json:"verdict,omitempty"`
Diagnostic *DiagnosticResult `json:"diagnostic,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
Error error `json:"-"`
}
StreamEvent is a single milestone emitted by the streaming API.
type Thresholds ¶
type Thresholds struct {
// DriftSignificance is the minimum shift (TVD or lexical distance) or
// confidence erosion required to flag an agent as drifted. Default 0.15.
DriftSignificance float64
// Confidence is the x-axis (confidence) threshold for quadrant
// classification. Default 0.55.
Confidence float64
// Consistency is the y-axis (consistency) threshold for quadrant
// classification. Default 0.60.
Consistency float64
}
Thresholds are the tunable cut-offs used across drift detection and quadrant classification. All fields are floats in the range [0.0, 1.0].
func (Thresholds) Resolve ¶
func (t Thresholds) Resolve() Thresholds
Resolve fills zero fields with the default values.