challenge

package
v0.84.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 26, 2026 License: MIT Imports: 15 Imported by: 0

README

Challenge definition and grading contract

pkg/challenge owns the immutable, schema-versioned definition of a guided SQL challenge and the deterministic primitives that grade a browser-runtime observation. It does not execute SQL or own attempts, authorization or completion records.

Asset and route

A published revision is a project asset at challenges/<challengeId>/<revision>.json (AssetPath). The existing query page opens it in guided mode at the project-scoped route suffix challenge/<challengeId>/v/<revision> (GuidedRouteSuffix); the full address is /project/<projectRef>/challenge/<id>/v/<revision>.

Conventional fixture paths are challenges/<challengeId>/fixtures/<fixtureId>.db. Paths are not identity: each fixture is pinned by SHA-256 and byte length in the manifest.

Golden wire examples

Purpose Path
Write challenge: SQL final-state oracle + group-min invariant + Check-definition part testdata/challenges/deduplicate-customers/1.json
Read challenge: typed result oracle + closed multi-select part testdata/challenges/retry-evidence/1.json
Pristine-run observations for both challenges testdata/goldens/*.json
Language-neutral grading regression vectors testdata/grading-vectors.json
Language-neutral bounded-feedback vectors testdata/feedback-vectors.json
Fixture bytes (committed, pinned) testdata/fixtures/<challengeId>/*.db
Deterministic generator testdata/generate_goldens.py

The manifests decode as the Go Manifest type and are validated by Open. Unknown fields, duplicate JSON keys, unsupported versions/dialect/engine, range violations, broken fixture references and closed-option violations are refused with a stable challenge.Error code that reports Recoverable(). Open returns no value on refusal, so no attempt or completion can be derived from a refused definition.

Grading

  • CompareTypedRows is the single typed comparison used by both result and final-state oracles. It compares a positional column set exactly, ignores column aliases unless requireColumnNames is set, preserves duplicate multiplicity and NULL, and keeps SQLite storage classes distinct per typed cell (INTEGER 2 differs from REAL 2.0) unless the manifest declares a non-negative numericTolerance (an explicit "0" permits numeric equality but no difference). Column metadata (columns[].type) describes a column and is validated for supported shape, but it is never compared with observed metadata or with a cell's storage class; heterogeneous results and all-NULL columns are therefore legitimate. Expected oracle rows and observed rows use the same local shape/value rules.
  • Result oracles compare typed rows as an unordered multiset by default; ordered comparison happens only when ordering: "ordered" is declared. A failed unordered comparison carries complete missing/unexpected counts and at most three examples, each bounded to 240 Unicode code points plus an ellipsis marker when truncated; ordered comparison keeps its first differing cell.
  • Final-state oracles compare every declared table exactly and then evaluate named invariants. The observed table set must match the declared set exactly, so an undeclared or newly created table cannot pass unnoticed. The only invariant in challenge-grading-v1 is group-min: one surviving row per declared group whose retention column equals the expected lowest value. The retention cell of every expected row must be a non-null numeric value, checked by cell value rather than by declared column type; a numeric column declaration alone never makes a TEXT cell numeric. This is a challenge-only declarative rule, not a production Check accepting arbitrary learner SQL.
  • Structured answers use closed field and option IDs and exact set/value equality. Unknown fields/options, duplicates and wrong cardinality are reported against the part ID without AI judgement.
  • Check-definition answers select one declared saved query plus a supported typed expectation (row-count over the result set). Their outcomes are evaluated on the declared broken, repaired and mutated fixtures. Observation recordsets pass the same challenge-local column and row shape/value checks, so a mixed-result row count is valid without weakening shared pkg/apicontract validators. No SQL text executes inside this package.
  • GradeAttempt derives completion only from the manifest's required parts: every required part must be pass with completed execution status.

Compatibility decisions

  • Typed values, recordsets, columns and strict JSON decoding are reused from pkg/apicontract; Check expectations reuse detective.Validator. No parallel Check/Work ownership model is introduced.
  • pkg/recordsetcompare is not reused for grading because it requires execution receipts and key-based row identity. Challenge grading is receipt-free and positional; forcing it through that API would either fabricate receipts or weaken its guarantees. The reusable comparator here is CompareTypedRows.
  • The manifest is public content, so expected rows and correct structured selections are visible in the asset. Browser-hidden fixtures are inspectable by design and must not be described as secret or tamper-proof.
  • The engine is pinned exactly (SupportedEngine); a revision graded on another engine is refused. Changed data or grading semantics require a new revision.

Regenerating fixtures and goldens

cd pkg/challenge
python3 testdata/generate_goldens.py
go test ./pkg/challenge

The generator deserializes the committed fixture bytes into a fresh in-memory database for every answer and every run, so each observation is produced from pristine bytes. It records the generator's Python/SQLite version in the golden; the manifests pin the browser engine (@sqlite.org/sqlite-wasm 3.54.1-build1).

Documentation

Overview

Package challenge owns the immutable, schema-versioned definition of a guided SQL challenge and its deterministic grading primitives.

A challenge definition is a project asset (see AssetPath) whose revision pins the visible schema, the fixture-suite byte hashes, the answer parts, the result/final-state oracles, the grading semantics and the execution engine. Opening a definition verifies every pin before any execution is allowed; there is no partially trusted manifest.

Grading here is deliberately passive: the browser runtime executes the learner's SQL against a disposable fixture and hands the resulting typed rows or final table state back to this package. Nothing in this package opens a database, executes SQL, performs authorization, calls a model or records completion.

Index

Constants

View Source 
const (
	ProblemUnknownQuery           = "unknown-query"
	ProblemUnsupportedExpectation = "unsupported-expectation"
	ProblemExpectationMismatch    = "expectation-mismatch"
	ProblemMissingObservation     = "missing-observation"
	ProblemCaseOutcome            = "case-outcome"
)

Stable check-definition problem codes.

View Source 
const (
	OrderUnordered = "unordered"
	OrderOrdered   = "ordered"
)

Row ordering modes. Unordered comparison is the default contract for result sets; ordered comparison is asserted only when the manifest declares it.

View Source 
const (
	ProblemColumnCount = "column-count"
	ProblemColumnName  = "column-name"
	// ProblemColumnType is reserved. Column metadata types describe a column,
	// not a graded cell; SQLite storage class is compared per typed cell.
	ProblemColumnType     = "column-type"
	ProblemRowShape       = "row-shape"
	ProblemRowValueType   = "row-value-type"
	ProblemRowOrder       = "row-order"
	ProblemRowContent     = "row-content"
	ProblemInvalidOptions = "invalid-options"
	ProblemInvariant      = "invariant"
)

Stable comparison problem codes. They are wire values used in feedback.

View Source 
const (
	OutcomePass    = "pass"
	OutcomeFail    = "fail"
	OutcomeUnknown = "unknown"

	StatusCompleted = "completed"
	StatusError     = "error"
)

Outcome and execution-status vocabulary shared with Detective results.

View Source 
const (
	ProblemSQLBytes       = "sql-bytes"
	ProblemSQLEmpty       = "sql-empty"
	ProblemExecutionError = "execution-error"
	ProblemObservation    = "observation-missing"
	ProblemOracleInvalid  = "invalid-oracle"
	ProblemFinalState     = "final-state"
)

SQL grading problem codes.

View Source 
const (
	SchemaVersion  = 1
	GradingVersion = "challenge-grading-v1"
)

SchemaVersion and GradingVersion are part of the wire contract. A manifest is graded only by the exact semantics it pins: changing a predicate without changing these values is a contract violation, and changing them requires a new revision.

View Source 
const (
	MaxDefinitionBytes = 4 << 20
	MaxFixtureBytes    = 10 << 20
	MaxSQLBytes        = 64 << 10
	MaxResultRows      = 1000
	MaxResultColumns   = 40
)

Limits are the approved execution limits. A manifest cannot relax them, and definition values are validated against them while the asset is opened.

View Source 
const (
	AnswerSQL             = "sql"
	AnswerStructured      = "structured"
	AnswerCheckDefinition = "check-definition"
)

Answer kinds. Each part has exactly one kind and one matching answer body.

View Source 
const (
	CardinalitySingle = "single"
	CardinalityMulti  = "multi"
)

Field cardinalities for closed structured answers.

View Source 
const (
	DifficultyIntro        = "intro"
	DifficultyIntermediate = "intermediate"
	DifficultyAdvanced     = "advanced"
)

Difficulty values exposed by discovery.

View Source 
const (
	OracleResult     = "result"
	OracleFinalState = "final-state"
)

Oracle kinds for a SQL answer part. Exactly one oracle is attached to each SQL case.

View Source 
const (
	ProblemUnknownField      = "unknown-field"
	ProblemUnknownOption     = "unknown-option"
	ProblemDuplicateOption   = "duplicate-option"
	ProblemMissingSelection  = "missing-selection"
	ProblemSingleCardinality = "single-cardinality"
	ProblemSelectionCount    = "selection-cardinality"
	ProblemSelectionMismatch = "selection-mismatch"
)

Stable structured-grading problem codes.

View Source 
const InvariantGroupMin = "group-min"

Invariant kinds are deliberately small and deterministic. A future rule needs a new grading version instead of an open-ended predicate language.

Variables

View Source 
var SupportedEngine = Engine{
	Package:       "@sqlite.org/sqlite-wasm",
	Version:       "3.54.1-build1",
	SQLiteVersion: "3.54.1",
}

SupportedEngine is the browser challenge runtime pinned by the approved execution foundation. A revision declaring any other engine is refused rather than silently graded by different SQLite behaviour.

Functions

func AssetPath 

func AssetPath(challengeID string, revision int) (string, error)

AssetPath is the project-relative path of a published challenge revision.

func FixtureAssetPath 

func FixtureAssetPath(challengeID, fixtureID string) (string, error)

FixtureAssetPath is the conventional project-relative path of a fixture inside the challenge asset directory. Fixture identity is always the pinned SHA-256, not this path.

func GuidedRouteSuffix 

func GuidedRouteSuffix(challengeID string, revision int) (string, error)

GuidedRouteSuffix is the project-scoped route suffix that opens the existing query page in guided mode: /project/<projectRef>/challenge/<id>/v/<revision>. Core deliberately does not parse the project reference; the caller owns its exact spelling and pins the returned revision.

Types

type AttemptOutcome 

type AttemptOutcome struct {
	Complete      bool          `json:"complete"`
	Parts         []PartOutcome `json:"parts"`
	BlockingParts []string      `json:"blockingParts,omitempty"`
}

AttemptOutcome is the only place completion is derived. It is true only when every required part passed with a completed execution.

func GradeAttempt 

func GradeAttempt(manifest Manifest, outcomes []PartOutcome) (AttemptOutcome, error)

GradeAttempt validates that the provided outcomes correspond exactly to the manifest's parts and derives completion. It never accepts a caller-supplied completion boolean.

type CheckCase 

type CheckCase struct {
	Fixture         string `json:"fixture"`
	ExpectedOutcome string `json:"expectedOutcome"`
}

CheckCase pins one fixture to the outcome a correct Check must produce.

type CheckCaseResult 

type CheckCaseResult struct {
	Fixture         string `json:"fixture"`
	ExpectedOutcome string `json:"expectedOutcome"`
	ObservedOutcome string `json:"observedOutcome,omitempty"`
	Outcome         string `json:"outcome"`
	Problem         string `json:"problem,omitempty"`
	Message         string `json:"message,omitempty"`
}

CheckCaseResult reports one fixture's expected versus observed outcome.

type CheckDefinitionAnswer 

type CheckDefinitionAnswer struct {
	QueryID     string              `json:"queryId"`
	Expectation detective.Validator `json:"expectation"`
}

CheckDefinitionAnswer is a learner's saved-query selection plus expectation.

type CheckDefinitionPart 

type CheckDefinitionPart struct {
	QueryIDs    []string            `json:"queryIds"`
	Expectation detective.Validator `json:"expectation"`
	Cases       []CheckCase         `json:"cases"`
}

CheckDefinitionPart asks the learner to select one saved training query and one typed supported expectation. Outcomes are validated deterministically on broken, repaired and mutated fixtures; no model judgement is involved.

type CheckDefinitionResult 

type CheckDefinitionResult struct {
	Outcome         string            `json:"outcome"`
	ExecutionStatus string            `json:"executionStatus"`
	Problem         string            `json:"problem,omitempty"`
	Message         string            `json:"message,omitempty"`
	Cases           []CheckCaseResult `json:"cases"`
}

CheckDefinitionResult is the outcome of a Check-definition answer.

func EvaluateCheckDefinition 

func EvaluateCheckDefinition(part CheckDefinitionPart, answer CheckDefinitionAnswer, observations map[string]Observation) CheckDefinitionResult

EvaluateCheckDefinition grades a submitted Check definition. Observation rows must come from the disposable runtime executing the selected saved query against each pristine fixture.

func (CheckDefinitionResult) AsOutcome 

func (r CheckDefinitionResult) AsOutcome(part Part) PartOutcome

AsOutcome projects the result onto the attempt contract.

type Code 

type Code string

Code is a stable refusal code for a definition or answer problem. 

const (
	// CodeInvalidManifest covers structural, range and reference violations.
	CodeInvalidManifest Code = "invalid_manifest"
	// CodeUnsupportedSchema refuses a schema version this package cannot grade.
	CodeUnsupportedSchema Code = "unsupported_schema"
	// CodeUnsupportedGradingVersion refuses changed predicate semantics.
	CodeUnsupportedGradingVersion Code = "unsupported_grading_version"
	// CodeUnsupportedDialect refuses a dialect the browser runtime cannot host.
	CodeUnsupportedDialect Code = "unsupported_dialect"
	// CodeUnsupportedEngine refuses a revision pinned to another engine.
	CodeUnsupportedEngine Code = "unsupported_engine"
	// CodeFixtureMissing refuses a pin without bytes.
	CodeFixtureMissing Code = "fixture_missing"
	// CodeFixtureUnknown refuses bytes that no revision pins.
	CodeFixtureUnknown Code = "fixture_unknown"
	// CodeFixtureBytes refuses a byte length that differs from its pin.
	CodeFixtureBytes Code = "fixture_bytes"
	// CodeFixtureHash refuses bytes whose SHA-256 differs from the pin.
	CodeFixtureHash Code = "fixture_hash"
	// CodeInvalidAnswer covers an answer that cannot be graded at all.
	CodeInvalidAnswer Code = "invalid_answer"
)

func ErrorCode 

func ErrorCode(err error) Code

ErrorCode returns the stable refusal code, or "" for an unclassified error.

type Engine 

type Engine struct {
	Package       string `json:"package"`
	Version       string `json:"version"`
	SQLiteVersion string `json:"sqliteVersion"`
}

Engine is the exact execution engine a revision is graded on. The fields are pinned in the manifest and must match the runtime that opens it.

type Error 

type Error struct {
	Code    Code   `json:"code"`
	Field   string `json:"field,omitempty"`
	Message string `json:"message"`
}

Error is a definition or answer refusal. Definition refusals are recoverable: reloading a supported revision or its original bytes resolves them, and no completion may be recorded from a refused open.

func (*Error) Error 

func (e *Error) Error() string

func (*Error) Recoverable 

func (e *Error) Recoverable() bool

Recoverable reports whether retrying with a supported revision or the pinned bytes can succeed. Every refusal produced here is recoverable.

type FieldOutcome 

type FieldOutcome struct {
	FieldID string `json:"fieldId"`
	Outcome string `json:"outcome"`
	Problem string `json:"problem,omitempty"`
	Message string `json:"message,omitempty"`
}

FieldOutcome reports one field's deterministic result.

type FinalStateOracle 

type FinalStateOracle struct {
	Tables     []TableExpectation `json:"tables"`
	Invariants []Invariant        `json:"invariants,omitempty"`
}

FinalStateOracle declares expected table contents plus named invariants that explain a violation. Exact row comparison already covers expected survivors and unrelated rows; invariants add the violated retention rule to feedback.

func (FinalStateOracle) Validate 

func (f FinalStateOracle) Validate() error

type FinalStateResult 

type FinalStateResult struct {
	Pass       bool              `json:"pass"`
	Problem    string            `json:"problem,omitempty"`
	Message    string            `json:"message,omitempty"`
	Tables     []TableResult     `json:"tables"`
	Invariants []InvariantResult `json:"invariants,omitempty"`
}

FinalStateResult is the deterministic outcome of a write answer. Every declared table and every invariant must pass.

func EvaluateFinalState 

func EvaluateFinalState(oracle FinalStateOracle, actual []TableState) FinalStateResult

EvaluateFinalState compares observed tables with the declared expected contents and evaluates each named invariant. It never guesses an expected value: the manifest's expected table is the single answer key.

func (FinalStateResult) ViolatedInvariant 

func (r FinalStateResult) ViolatedInvariant() string

ViolatedInvariant returns the first failed invariant ID, if any.

type Fixture 

type Fixture struct {
	ID     string `json:"id"`
	Label  string `json:"label"`
	SHA256 string `json:"sha256"`
	Bytes  int    `json:"bytes"`
	Runs   int    `json:"runs"`
}

Fixture is one immutable SHA-256/byte-length pin. Runs is the number of times the submitted SQL is applied (2 proves idempotency).

type FixtureSuite 

type FixtureSuite struct {
	ID       string    `json:"id"`
	Fixtures []Fixture `json:"fixtures"`
}

FixtureSuite pins every byte the graded answer is executed against.

type Hint 

type Hint struct {
	ID   string `json:"id"`
	Text string `json:"text"`
}

Hint is one progressive hint; order is the array order.

type Invariant 

type Invariant struct {
	ID          string   `json:"id"`
	Kind        string   `json:"kind"`
	Table       string   `json:"table"`
	GroupBy     []string `json:"groupBy"`
	Column      string   `json:"column"`
	Description string   `json:"description"`
}

Invariant names a retention rule the final state must satisfy. GroupMin requires exactly one surviving row per declared group and requires that row's Column to equal the lowest value in the expected table.

func (Invariant) Validate 

func (i Invariant) Validate(tables map[string]TableExpectation) error

type InvariantResult 

type InvariantResult struct {
	ID      string `json:"id"`
	Pass    bool   `json:"pass"`
	Problem string `json:"problem,omitempty"`
	Message string `json:"message,omitempty"`
}

InvariantResult reports one named retention invariant.

type Manifest 

type Manifest struct {
	SchemaVersion      int                 `json:"schemaVersion"`
	GradingVersion     string              `json:"gradingVersion"`
	ChallengeID        string              `json:"challengeId"`
	Revision           int                 `json:"revision"`
	Title              string              `json:"title"`
	Objective          string              `json:"objective"`
	Difficulty         string              `json:"difficulty"`
	Dialect            string              `json:"dialect"`
	AllowedOperation   string              `json:"allowedOperation"`
	Timing             TimingMode          `json:"timing"`
	Engine             Engine              `json:"engine"`
	FixtureSuite       FixtureSuite        `json:"fixtureSuite"`
	VisibleSchema      VisibleSchema       `json:"visibleSchema"`
	Parts              []Part              `json:"parts"`
	Hints              []Hint              `json:"hints"`
	ReferenceSolutions []ReferenceSolution `json:"referenceSolutions"`
}

Manifest is one immutable published challenge revision.

func LoadManifest 

func LoadManifest(data []byte) (Manifest, error)

LoadManifest strictly decodes and validates manifest bytes. Unknown fields, duplicate keys, unsupported versions, range violations and broken internal references are refused before execution.

func (Manifest) Validate 

func (m Manifest) Validate() error

Validate checks every structural, range and internal reference rule. It is intentionally independent of fixture bytes; Open verifies the pins.

type Observation 

type Observation struct {
	Rows      *apicontract.Recordset `json:"rows,omitempty"`
	Tables    []TableState           `json:"tables,omitempty"`
	ErrorCode string                 `json:"errorCode,omitempty"`
}

Observation is one pristine-fixture run supplied by the disposable runtime. It is not learner input and carries no authorization claim.

type Opened 

type Opened struct {
	Manifest         Manifest
	DefinitionDigest string
	// contains filtered or unexported fields
}

Opened is a manifest whose schema, references and fixture pins have all been verified. It holds its own fixture snapshots; later mutation of caller buffers cannot change what an attempt executes.

func Open 

func Open(manifestJSON []byte, fixtures map[string][]byte) (*Opened, error)

Open verifies the manifest and every pinned fixture byte before returning a definition. A refused open returns no Opened value, so no attempt or completion can be created from it.

func (*Opened) Fixture 

func (o *Opened) Fixture(id string) ([]byte, bool)

Fixture returns a defensive copy of one verified fixture snapshot.

func (*Opened) FixtureBytesEqual 

func (o *Opened) FixtureBytesEqual(id string, other []byte) bool

FixtureBytesEqual is used by inspection receipts that need to prove two opens used identical pristine bytes.

func (*Opened) FixtureIDs 

func (o *Opened) FixtureIDs() []string

FixtureIDs returns the pinned fixture IDs in deterministic order.

func (*Opened) HasFixture 

func (o *Opened) HasFixture(id string) bool

HasFixture reports whether a fixture ID is part of the verified suite.

func (*Opened) RequireEngine 

func (o *Opened) RequireEngine(engine Engine) error

RequireEngine refuses a runtime whose exact engine differs from the manifest pin. Grading semantics and engine version travel together.

type Oracle 

type Oracle struct {
	Kind       string
	Result     *ResultOracle
	FinalState *FinalStateOracle
}

Oracle is a closed union. It names either the typed rows a read answer must return or the final table state a write answer must leave behind.

func (Oracle) MarshalJSON 

func (o Oracle) MarshalJSON() ([]byte, error)

func (*Oracle) UnmarshalJSON 

func (o *Oracle) UnmarshalJSON(data []byte) error

func (Oracle) Validate 

func (o Oracle) Validate() error

Validate checks the oracle shape and its internal answer semantics.

type Part 

type Part struct {
	ID              string               `json:"id"`
	Title           string               `json:"title"`
	Required        bool                 `json:"required"`
	AnswerKind      string               `json:"answerKind"`
	SQL             *SQLPart             `json:"sql,omitempty"`
	Structured      *StructuredPart      `json:"structured,omitempty"`
	CheckDefinition *CheckDefinitionPart `json:"checkDefinition,omitempty"`
}

Part is one named, required-by-default answer required to complete a challenge.

func (Part) Validate 

func (p Part) Validate() error

type PartOutcome 

type PartOutcome struct {
	PartID          string `json:"partId"`
	AnswerKind      string `json:"answerKind"`
	Required        bool   `json:"required"`
	Outcome         string `json:"outcome"`
	ExecutionStatus string `json:"executionStatus"`
	Problem         string `json:"problem,omitempty"`
	Message         string `json:"message,omitempty"`
}

PartOutcome is the normalized, untrusted-input-resistant result of one part.

type ReferenceAnswer 

type ReferenceAnswer struct {
	SQL             string                 `json:"sql,omitempty"`
	Structured      *StructuredAnswer      `json:"structured,omitempty"`
	CheckDefinition *CheckDefinitionAnswer `json:"checkDefinition,omitempty"`
}

ReferenceAnswer is a closed union matching the part's answer kind.

type ReferenceSolution 

type ReferenceSolution struct {
	PartID string          `json:"partId"`
	Answer ReferenceAnswer `json:"answer"`
}

ReferenceSolution declares an assisted answer. Consuming UI must tag it as assistance; a reference answer never awards completion by itself.

type ResultOracle 

type ResultOracle struct {
	Columns            []apicontract.Column       `json:"columns"`
	Rows               [][]apicontract.TypedValue `json:"rows"`
	Ordering           string                     `json:"ordering"`
	RequireColumnNames bool                       `json:"requireColumnNames,omitempty"`
	NumericTolerance   string                     `json:"numericTolerance,omitempty"`
}

ResultOracle declares the typed recordset a read answer must produce.

func (ResultOracle) Validate 

func (r ResultOracle) Validate() error

type RowDiff 

type RowDiff struct {
	Match      bool     `json:"match"`
	Problem    string   `json:"problem,omitempty"`
	Message    string   `json:"message,omitempty"`
	Missing    int      `json:"missing,omitempty"`
	Unexpected int      `json:"unexpected,omitempty"`
	Examples   []string `json:"examples,omitempty"`
}

RowDiff is a bounded, deterministic description of why two typed row sets differ. Missing and Unexpected count whole rows in an unordered comparison; ordered comparisons report the first differing position instead.

func CompareTypedRows 

func CompareTypedRows(expectedColumns, actualColumns []apicontract.Column, expectedRows, actualRows [][]apicontract.TypedValue, options RowOptions) RowDiff

CompareTypedRows compares a declared expected recordset with an observed one. Column metadata is validated for supported shape, but a column's declared type is descriptive and is not compared with the observed metadata or the cells in it: SQLite storage class participates in equality per typed cell. Values are matched by declared column position; column count is exact; column aliases are ignored unless RequireColumnNames is set. Storage classes remain distinct (INTEGER 2 differs from REAL 2.0) unless an explicit non-negative numeric tolerance is declared.

type RowOptions 

type RowOptions struct {
	Ordering           string `json:"ordering"`
	RequireColumnNames bool   `json:"requireColumnNames,omitempty"`
	NumericTolerance   string `json:"numericTolerance,omitempty"`
}

RowOptions controls typed row comparison.

type SQLCase 

type SQLCase struct {
	Fixture string `json:"fixture"`
	Oracle  Oracle `json:"oracle"`
}

SQLCase pins one fixture to one oracle.

type SQLCaseResult 

type SQLCaseResult struct {
	Fixture         string   `json:"fixture"`
	Outcome         string   `json:"outcome"`
	ExecutionStatus string   `json:"executionStatus"`
	Problem         string   `json:"problem,omitempty"`
	InvariantID     string   `json:"invariantId,omitempty"`
	Message         string   `json:"message,omitempty"`
	Missing         int      `json:"missing,omitempty"`
	Unexpected      int      `json:"unexpected,omitempty"`
	Examples        []string `json:"examples,omitempty"`
}

SQLCaseResult reports one fixture's graded outcome. Missing, Unexpected and Examples are present only for a same-shape unordered row-content mismatch; they carry bounded, deterministic feedback and never affect the verdict.

type SQLPart 

type SQLPart struct {
	Cases []SQLCase `json:"cases"`
}

SQLPart declares the fixture cases and their expected oracles.

type SQLPartResult 

type SQLPartResult struct {
	Outcome         string          `json:"outcome"`
	ExecutionStatus string          `json:"executionStatus"`
	Problem         string          `json:"problem,omitempty"`
	Message         string          `json:"message,omitempty"`
	Cases           []SQLCaseResult `json:"cases"`
}

SQLPartResult reports every fixture case for one SQL answer.

func GradeSQLPart 

func GradeSQLPart(part SQLPart, sql string, observations map[string]Observation) SQLPartResult

GradeSQLPart grades one SQL text against every declared case. The runtime executes it on independent, pristine fixture copies; this function only compares typed evidence.

func (SQLPartResult) AsOutcome 

func (r SQLPartResult) AsOutcome(part Part) PartOutcome

AsOutcome projects the result onto the attempt contract.

type StructuredAnswer 

type StructuredAnswer struct {
	Selections map[string][]string `json:"selections"`
}

StructuredAnswer is both the expected answer and a learner submission.

type StructuredField 

type StructuredField struct {
	ID            string             `json:"id"`
	Label         string             `json:"label"`
	Cardinality   string             `json:"cardinality"`
	Required      bool               `json:"required"`
	MinSelections int                `json:"minSelections,omitempty"`
	MaxSelections int                `json:"maxSelections,omitempty"`
	Options       []StructuredOption `json:"options"`
}

StructuredField declares one closed field. Single fields always require exactly one option; multi fields declare an inclusive min/max selection count.

type StructuredOption 

type StructuredOption struct {
	ID    string `json:"id"`
	Label string `json:"label"`
}

StructuredOption is one closed option ID and its learner-facing label.

type StructuredPart 

type StructuredPart struct {
	Fields   []StructuredField `json:"fields"`
	Expected StructuredAnswer  `json:"expected"`
}

StructuredPart is a closed selection answer: every field and option is declared, and grading is exact value/set equality. No free text, model or fuzzy matching is involved.

func (StructuredPart) Validate 

func (s StructuredPart) Validate() error

Validate checks the closed field/option space and the expected answer.

type StructuredResult 

type StructuredResult struct {
	Outcome         string         `json:"outcome"`
	ExecutionStatus string         `json:"executionStatus"`
	Problem         string         `json:"problem,omitempty"`
	Message         string         `json:"message,omitempty"`
	Fields          []FieldOutcome `json:"fields"`
}

StructuredResult is the outcome of a closed-selection answer.

func EvaluateStructured 

func EvaluateStructured(part StructuredPart, answer StructuredAnswer) StructuredResult

EvaluateStructured grades a closed selection without AI judgement.

func (StructuredResult) AsOutcome 

func (r StructuredResult) AsOutcome(part Part) PartOutcome

AsOutcome projects the result onto the attempt contract.

type TableExpectation 

type TableExpectation struct {
	Name    string                     `json:"name"`
	Columns []apicontract.Column       `json:"columns"`
	Rows    [][]apicontract.TypedValue `json:"rows"`
}

TableExpectation is the exact expected final content of one table.

func (TableExpectation) Validate 

func (t TableExpectation) Validate() error

type TableResult 

type TableResult struct {
	Table   string `json:"table"`
	Pass    bool   `json:"pass"`
	Problem string `json:"problem,omitempty"`
	Message string `json:"message,omitempty"`
}

TableResult reports one declared table's exact final-state comparison.

type TableState 

type TableState struct {
	Name    string                     `json:"name"`
	Columns []apicontract.Column       `json:"columns"`
	Rows    [][]apicontract.TypedValue `json:"rows"`
}

TableState is an observed final table supplied by the disposable runtime.

type TimingMode 

type TimingMode string

TimingMode is the attempt timing policy pinned by a revision. 

const (
	TimingOff      TimingMode = "off"
	TimingOptional TimingMode = "optional"
	TimingRequired TimingMode = "required"
)

type VisibleColumn 

type VisibleColumn struct {
	Name       string `json:"name"`
	Type       string `json:"type"`
	PrimaryKey bool   `json:"primaryKey,omitempty"`
	NotNull    bool   `json:"notNull,omitempty"`
	Collation  string `json:"collation,omitempty"`
}

VisibleColumn is one visible column, including the SQLite collation that participates in equality.

type VisibleSchema 

type VisibleSchema struct {
	Tables []VisibleTable `json:"tables"`
}

VisibleSchema is the schema shown to the learner.

type VisibleTable 

type VisibleTable struct {
	Name    string          `json:"name"`
	Columns []VisibleColumn `json:"columns"`
}

VisibleTable is one table in the visible schema.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.