Documentation
¶
Overview ¶
Package aigen is the AI schema-generation layer: it turns a natural-language description into a VALID Appximo schema.
ARCHITECTURE (AI-F2-S4, decided with real data): the plain VALIDATOR-GUIDED LOOP. The model generates a candidate schema; schema.ValidateReport (the engine's own validator) is the trusted external oracle that returns actionable errors; the loop feeds them back and the model self-corrects until valid (Huang et al. ICLR 2024: self-correction without an external oracle does not work). The real measurement (AI-F2-S3) showed this reaches ~90% first-try / 100% convergence / ~$0.006 per schema with the CHEAP model — the democratization thesis, confirmed on real data.
ARCHIVED / EXPERIMENTAL — constrained decoding (AI-F1-S1 structured-outputs envelope, AI-F2-S2 array-IR). The intent was to fix p_struct=1 at the decoder, but the real Anthropic strict-outputs subset REJECTS the Appximo grammar (every object needs additionalProperties:false → the schema's open maps can't be expressed; a hard 16-union-param cap → the field grammar with ~17 optionals exceeds it), so both modes silently fall back to plain. They are kept opt-in (OutputSchema/IROutputSchema, Options.ArrayIR) for measurement (pkg/aigen/eval) and because the array-IR transforms (ir.go) are the structured representation the VISUAL EDITOR will reuse — not for production decoding. See docs/AI_SCHEMA_GENERATION.md.
It is a NEW, isolated layer: it imports pkg/schema (the validator) but the engine core imports nothing here. The model is reached over raw net/http (the standard /v1/messages endpoint) — no new dependency, CGO-free, like OAuth/MFA.
Index ¶
- Constants
- Variables
- func HasResources(raw []byte) bool
- func IROutputSchema() map[string]any
- func IRToMap(ir map[string]any) map[string]any
- func MapToIR(doc map[string]any) map[string]any
- func OutputSchema() map[string]any
- func Spec() string
- func SupportsStructuredOutput(model string) bool
- func TranslateMapPathToIR(mapPath string, ir map[string]any) string
- type AnthropicClient
- type Attempt
- type Completion
- type Message
- type ModelClient
- type Options
- type Pricing
- type Request
- type Result
- type Usage
Constants ¶
const ( ModelHaiku = "claude-haiku-4-5" // the cheap model — the thesis ModelSonnet = "claude-sonnet-4-6" // mid tier — the comparison ModelOpus = "claude-opus-4-8" // the most capable — the upper bound )
Model IDs (verified against the Anthropic model catalog). The thesis under test is "a CHEAP model is enough for schema generation", so Haiku is the default; the more capable models are here for the economic comparison. The model is a pure parameter (the ModelClient seam) — swapping it never touches the loop, so the layer is not tied to one model (the SOTA moves constantly).
const ( SchemaURL = "https://appximo.com/schema/v1" SchemaVersion = "1" )
SchemaURL / SchemaVersion are the two required top-level constants the envelope pins (a frequent source of model error, eliminated by construction).
const DefaultMaxIterations = 3
DefaultMaxIterations bounds the generate→validate→correct loop. AI-F1-S1 lowered it from 5 to 3: with the structural envelope guaranteed by constrained decoding, the loop only spends rounds on SEMANTIC corrections, which converge faster — so the budget no longer needs to absorb structural-mistake rounds.
const DefaultModel = ModelHaiku
DefaultModel is the model the loop uses unless overridden. Haiku, on purpose: the whole point is to show the economy holds with the cheapest model.
const GrammarCore = `` /* 11728-byte string literal not displayed */
GrammarCore is the COMPACT Appximo schema grammar for an LLM — the closed sets (types, ops, actions), the strict-key rule, naming, relations, and one canonical worked example. It is deliberately condensed from docs/SCHEMA_REFERENCE.md (~2000 lines) to exactly the facts the validator is strict about.
It is the SINGLE SOURCE shared by two consumers (JSON-EDITOR-S3):
- the internal generation loop (systemPrompt below — the ai-generate command)
- `appximo spec` (Spec, spec.go) — the printable pack for an EXTERNAL agent (Claude Code / Cursor / any LLM with the user's own subscription)
Change the grammar here and both stay in sync by construction; prompt_test.go additionally validates every embedded example against the real validator.
const SpecExampleAdvanced = `` /* 3439-byte string literal not displayed */
SpecExampleAdvanced is the advanced worked example printed by `appximo spec` — kept as a standalone const so spec_test.go validates it against the real validator (the spec can never teach an invalid shape).
Variables ¶
var ErrNoAPIKey = fmt.Errorf("aigen: ANTHROPIC_API_KEY is not set (export it to use ai-generate; never hardcode it)")
ErrNoAPIKey is returned by NewAnthropicClient when ANTHROPIC_API_KEY is unset, so the CLI can print a clear message instead of failing obscurely mid-request.
Functions ¶
func HasResources ¶
HasResources reports whether raw schema bytes carry at least one resource. The generator uses it as a guard: structured outputs forcing an EMPTY resources map would otherwise "validate" (the engine accepts zero resources) and the loop would wrongly converge to a useless schema — so an empty result disables the structured path and regenerates plainly.
func IROutputSchema ¶
IROutputSchema returns the JSON Schema the decoder is constrained to when the model generates in ARRAY-IR form (AI-F2-S2).
ARCHIVED / EXPERIMENTAL (AI-F2-S4): the real Anthropic strict-outputs subset REJECTS this schema — it caps union-typed (nullable) parameters at 16 and the Appximo field grammar has ~17 optional keys, so the request errors ("too many parameters with union types") → the generator falls back to the plain validator-guided loop (the default). Kept opt-in for measurement and as the record of the boundary; the transforms in ir.go are reused by the visual editor.
Unlike OutputSchema (which can only pin the ENVELOPE because the map form is not expressible in the strict subset), this schema constrains the structure IN DEPTH: every arbitrary-keyed map is an array of objects with a FIXED item schema, which the strict subset CAN express.
Every construction here is inside the structured-outputs strict subset:
- every object is additionalProperties:false with EVERY property in `required` (the subset forbids optional properties; an optional is emulated as a nullable required key — type ["T","null"] — so the model emits an absent option as explicit null, which IRToMap collapses back to absent);
- fixed value sets are `enum` (the 9 field types, on_delete/on_update, format, relation/hook/action kinds, the events) — so a wrong value is impossible at decode time, not merely correctable;
- arrays carry a single fixed `items` schema;
- the two top-level constants are pinned with `const` (as in OutputSchema).
It uses NONE of the disallowed keys (patternProperties / propertyNames / additionalProperties-as-schema / oneOf / not / pattern / min*/max* / multipleOf / minItems / ...) — a test (TestIROutputSchemaIsStrictSubset) walks the whole tree and asserts it. The deep structural error class (wrong type, unknown deep key) can therefore no longer occur in IR generation, leaving only the semantic, cross-reference class for the validator + loop.
CAVEATS (documented, not bugs): a few values are genuinely polymorphic in the grammar and use a multi-type union rather than a single type — `resources` in a role ("*" string OR an array), state_machine `initial` (string OR array), and a field `default` (string/number/bool). A union is still strict-subset-shaped (it is just `type: [...]`); these stay the validator's job for the precise rule.
func IRToMap ¶
IRToMap is the inverse: the array-IR back to the map form the engine consumes. Optional keys whose value is nil are DROPPED (strict structured outputs emulates an optional as a nullable required key, so a real model emits absent options as explicit null — collapsing them to absent restores the canonical map; MapToIR never emits nil, so this is invisible to the round-trip).
func MapToIR ¶
MapToIR converts a decoded Appximo schema (map form) to the array-IR. It is total and deterministic: arbitrary-keyed maps become arrays sorted by their explicit key; every other value passes through unchanged.
func OutputSchema ¶
OutputSchema returns the JSON Schema passed to Anthropic structured outputs (output_config.format) so the model's response is DECODE-CONSTRAINED to it.
ARCHIVED / EXPERIMENTAL (AI-F2-S4): the real Anthropic strict-outputs subset REJECTS this envelope — it requires additionalProperties:false on EVERY object, so the deliberately-open `resources`/`rbac` here cannot be expressed, and the request errors → the generator falls back to the plain validator-guided loop (the default). Kept opt-in for measurement (pkg/aigen/eval) and as the documented record of the boundary; the plain loop is the product path. See docs/AI_SCHEMA_GENERATION.md.
Why only the ENVELOPE. The strict-outputs subset is narrow: every object must declare additionalProperties:false, and it offers no patternProperties / propertyNames / additionalProperties-as-schema. The Appximo schema is an arbitrary-keyed map (resources keyed by resource name, fields by field name, roles by role name) — that shape is simply not expressible in the subset. So this schema constrains the part that IS expressible — the top-level envelope — and deliberately leaves resources/rbac as open objects.
What the decoder therefore GUARANTEES by construction (these error classes can no longer occur, so the loop never spends an iteration on them):
- the output is well-formed JSON (no fences, no prose, no truncated garbage);
- $schema is exactly the v1 const, version is exactly "1";
- name is present and a string; resources is present;
- no UNKNOWN top-level key (additionalProperties:false on the root).
What still falls to schema.ValidateReport + the correction loop (the deep, map-keyed structure the subset cannot reach, plus all cross-reference semantics): field types, strict field/resource keys, enums, relations/FKs to existing targets, RBAC fields, state machines, defaults. The model keeps emitting the canonical MAP form, so the validator's error paths (resources.X.fields.Y...) stay coherent for in-context correction.
NOTE: resources/rbac are typed "object" with no additionalProperties here, so the model can fill them with arbitrary keys. If a stricter live interpretation forced them empty, the generator detects an empty-resources result and falls back to plain generation (HasResources / the loop's guard) — never a silent convergence to an empty schema.
func SupportsStructuredOutput ¶
SupportsStructuredOutput reports whether the model can use structured outputs.
func TranslateMapPathToIR ¶
TranslateMapPathToIR rewrites a validator map path into the equivalent IR path, using ir (the IR document the model produced) to resolve each named segment to its array index. It is best-effort and total: a segment it cannot resolve is emitted verbatim (never a crash), so an unrecognized path degrades to the map path rather than failing the correction round.
Types ¶
type AnthropicClient ¶
type AnthropicClient struct {
// contains filtered or unexported fields
}
AnthropicClient is the real ModelClient: a raw net/http call to the standard /v1/messages endpoint. No SDK dependency — the request/response shapes are stable and small.
func NewAnthropicClient ¶
func NewAnthropicClient(model string) (*AnthropicClient, error)
NewAnthropicClient builds a client for the given model, reading the API key from ANTHROPIC_API_KEY (the only supported source — never a literal). The base URL can be overridden with ANTHROPIC_BASE_URL (for a proxy/gateway or tests). Returns ErrNoAPIKey when the key is absent.
func (*AnthropicClient) Complete ¶
func (c *AnthropicClient) Complete(ctx context.Context, req Request) (Completion, error)
Complete sends one request and returns the concatenated text content + usage. The stable system prompt is sent as a cache_control block (prompt caching), so repeated correction rounds re-read it cheaply. A structured request carries req.OutputSchema as output_config.format. A safety refusal is surfaced as Completion.Refused (not an error) so the loop can stop cleanly.
func (*AnthropicClient) Model ¶
func (c *AnthropicClient) Model() string
Model returns the model id this client targets.
type Attempt ¶
type Attempt struct {
Iteration int `json:"iteration"`
Raw string `json:"-"`
Valid bool `json:"valid"`
Structured bool `json:"structured"`
StructuralCount int `json:"structural_errors"`
SemanticCount int `json:"semantic_errors"`
Errors []schema.StructuredError `json:"errors,omitempty"`
Usage Usage `json:"usage"`
}
Attempt records one round of the loop: whether it validated, the errors split by layer (structural vs semantic — the metric that shows the envelope removed the structural class), and the tokens that round consumed.
type Completion ¶
Completion is one model response. Refused is set when the model declined the request for safety (stop_reason "refusal") — that is NOT a schema error (the loop must not try to correct it), it is a model-level decline.
type Message ¶
type Message struct {
Role string `json:"role"` // "user" | "assistant"
Content string `json:"content"`
}
Message is one turn in the generate/correct conversation.
type ModelClient ¶
type ModelClient interface {
Complete(ctx context.Context, req Request) (Completion, error)
}
ModelClient is the seam the loop depends on. The real implementation calls the Anthropic API; tests inject a deterministic stub, so the loop is provable with NO network and NO API key. The seam is also where a future cheap→expensive CASCADE wrapper would live (try Haiku, escalate to Opus on non-convergence) — a ModelClient that delegates to others; the loop is unaware.
type Options ¶
type Options struct {
// MaxIterations caps the correction rounds (default DefaultMaxIterations).
MaxIterations int
// Model prices the run (set to the ModelClient's model id for an accurate cost).
Model string
// NoStructured disables structured outputs (forces the plain validator-guided
// loop). AI-F2-S4 made plain the PRODUCT DEFAULT — the real measurement
// (AI-F2-S3) showed it reaches ~90% first-try / 100% convergence / ~$0.006 per
// schema with the cheap model, and that structured decoding does not engage on
// the real API. The CLI passes NoStructured:true by default; structured/array-IR
// are EXPERIMENTAL opt-ins. (The generator also falls back to plain automatically
// on a structured error or an empty-resources result.)
NoStructured bool
// ArrayIR generates in the array-IR form (AI-F2-S2): the decoder is constrained
// to IROutputSchema and the loop transforms IR→map before validating. EXPERIMENTAL
// / archived: it does NOT engage on the real Anthropic API (the strict-outputs
// subset caps union params at 16; the field grammar has ~17 optionals), so it
// silently falls back to plain — kept for measurement (ai-eval) and because the IR
// transforms (ir.go) are reused by the visual editor, not for production decoding.
ArrayIR bool
}
Options configures a Generate run.
type Pricing ¶
Pricing is a model's USD price per 1,000,000 tokens (input / output). Cache reads are billed at 0.1x input and cache writes at 1.25x input (Anthropic prompt-caching economics), applied in Usage.CostUSD.
func PricingFor ¶
PricingFor returns the published pricing for a model id (zero value + false when the model is unknown).
type Request ¶
type Request struct {
System string
Messages []Message
OutputSchema map[string]any // nil → plain generation (no structured outputs)
}
Request is one model call. OutputSchema (when non-nil) engages structured outputs so the response is decode-constrained to that JSON Schema.
type Result ¶
type Result struct {
Schema json.RawMessage `json:"schema"`
Valid bool `json:"valid"`
Converged bool `json:"converged"`
FirstTry bool `json:"first_try"`
Iterations int `json:"iterations"`
// Structured reports whether the final/converged generation used structured
// outputs (false when it fell back to plain generation).
Structured bool `json:"structured"`
// ArrayIR reports whether the final/converged generation used the array-IR form
// (false when it fell back, or when never requested).
ArrayIR bool `json:"array_ir"`
Attempts []Attempt `json:"attempts"`
Usage Usage `json:"usage"`
Model string `json:"model"`
CostUSD float64 `json:"cost_usd"`
// Refused is set when the model declined for safety — not a schema failure.
Refused bool `json:"refused,omitempty"`
RefusalText string `json:"refusal_text,omitempty"`
// RemainingErrors are the final schema's validation errors when it did not
// converge (empty on success).
RemainingErrors []schema.StructuredError `json:"remaining_errors,omitempty"`
}
Result is the outcome of a Generate run — the artifact plus the economic instrumentation that validates the democratization thesis.
func Generate ¶
func Generate(ctx context.Context, client ModelClient, description string, opts Options) (*Result, error)
Generate runs the re-architected loop: structured generation (envelope guaranteed by decoding) → schema.ValidateReport (the trusted oracle, full structural+semantic as defense-in-depth) → feed back the remaining errors → repeat until valid or the budget is exhausted. It is pure orchestration over the injected ModelClient and the engine's own validator.
type Usage ¶
type Usage struct {
InputTokens int `json:"input_tokens"`
OutputTokens int `json:"output_tokens"`
CacheCreationTokens int `json:"cache_creation_tokens,omitempty"`
CacheReadTokens int `json:"cache_read_tokens,omitempty"`
}
Usage is the token accounting for one or more model calls, including the prompt-cache split (the stable system prompt is cached, so repeated iterations re-read it at 0.1x instead of paying full input each round).
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package eval is the scientific measurement instrument for the AI schema- generation layer (AI-F2-S1): a stratified NL→schema gold test set + a paired ablation harness + statistics with the rigor the research demands.
|
Package eval is the scientific measurement instrument for the AI schema- generation layer (AI-F2-S1): a stratified NL→schema gold test set + a paired ablation harness + statistics with the rigor the research demands. |