schema

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 8, 2026 License: MPL-2.0 Imports: 14 Imported by: 0

Documentation

Overview

Package schema is the key catalogue's declaration vocabulary and its value-validation engine (#49, schema-model ADR as amended by the flat-model ADR). It is a pure library: no store, no authorization, no clock — a declaration in, a verdict out — so #50's value saves and #51's publish pipeline consume exactly the rules this package's fixtures pin.

Two authorities live here and nowhere else:

  • CompileClassified() is the DECLARATION authority. Saving a declaration blocks on well-formedness and classification compatibility: a value may be wrong, a rule may not be meaningless (ADR § Validation timing). Every refusal names what it refused, because a rule that appears to enforce something and does not is worse than no rule at all.
  • Compiled.Validate() is the VALUE authority: one declaration against one string, with the ADR's fixed lexical semantics and its error-disclosure rules. Everything Hikyo delivers is a string on the wire, so a type is a parse-and-reject rule, never a storage format.

The contract layer carries shape only (a type name is a string, a bound is a number); every rule below is enforced here, because an internal caller — a future import, a job, the isolation harness — never passes through request validation.

Index

Constants

View Source
const (
	// MaxKeyGroupsPerProject bounds the coupling vocabulary. A group is a
	// co-publish unit, not a tag: hundreds of them means something else was
	// wanted.
	MaxKeyGroupsPerProject = 100
	// MaxKeyNameBytes is the ops-catalogue key-name bound (domain model
	// § Canonical key grammar). Bytes, not code points: the grammar is ASCII.
	MaxKeyNameBytes = 128
	// MaxDescriptionBytes bounds the free-text description, which may hold a
	// URL — there is no separate docs field.
	MaxDescriptionBytes = 4096
	// MaxEnumMembers, MaxPatternBytes, MaxAnyOfAlternatives are the ADR's
	// declaration-size bounds. `any_of` validation is linear in the number of
	// alternatives and every alternative's failure is enumerated, so the
	// alternative count bounds both the work and the error payload.
	MaxEnumMembers       = 256
	MaxEnumMemberBytes   = 512
	MaxPatternBytes      = 512
	MaxAnyOfAlternatives = 8
	// The JSON Schema declaration bounds. Bytes bound the document, depth
	// bounds nesting, and the subschema count bounds the applicator graph —
	// none of the three implies the others, so all three are enforced. Values
	// are the ops-spec § 8 "Schema & validation" bounds verbatim (≤ 64 KiB
	// declaration, `$ref` nesting ≤ 32) — the operations spec owns the
	// enumeration (schema-model ADR § profile).
	//
	MaxJSONSchemaBytes = 65536
	MaxJSONSchemaDepth = 32
	// MaxEvaluationWork is the ADR's STEP CAP, enforced where it is decidable:
	// at declaration. It IS a step cap and not merely a size bound, because the
	// profile admits no keyword that costs more than LINEAR time in the
	// instance per subschema — `uniqueItems` and `contains`, the two that do,
	// are excluded by name (internal/schema/jsonschema.go carries the per-
	// keyword audit). With every superlinear keyword out, worst-case evaluation
	// is bounded by subschemas × validated instance bytes: one product, checked
	// once, before anything can be evaluated against it.
	//
	// The product's first factor is the EXPANDED evaluation-path count, not the
	// declared subschema count: `$ref` reuse expands, so a document declaring a
	// few dozen subschemas can drive millions of evaluations. See
	// profileWalk.checkWorkBudget.
	//
	// MaxJSONSchemaSubschemas is DERIVED from this budget rather than invented
	// beside it: two independently chosen numbers is how a "bound" stops
	// bounding the thing it names. It bounds the graph STRUCTURALLY, which is
	// what makes the expansion computable over it.
	MaxValidatedInstanceBytes = MaxValueBytes
	MaxEvaluationWork         = 16 << 20
	MaxJSONSchemaSubschemas   = MaxEvaluationWork / MaxValidatedInstanceBytes
	// MaxValueBytes is the evaluation budget's instance-size half: the largest
	// value the engine will validate at all. Beyond it the verdict is a loud
	// budget failure, never "assume valid".
	MaxValueBytes = 65536
	// MaxVerdictErrors and MaxVerdictErrorBytes cap what one verdict may
	// report. Error multiplicity leaks structure for a secret key and is a
	// response-amplification lever for any key — but the leak is closed by the
	// disclosure rule (schema locations only, never instance paths, locked #12),
	// so the ops-spec § 8 values (≤ 100 errors / 64 KiB per verdict) govern the
	// count and size, and the byte cap bounds the amplification.
	MaxVerdictErrors     = 100
	MaxVerdictErrorBytes = 65536
	// EvaluationDeadline is the per-validation wall-clock budget, the ops-spec
	// § 8 value (100 ms deadline per value). RE2 is linear so the primitives
	// cannot exceed it; it exists for the JSON Schema leg, whose cost the profile
	// bounds statically and this bounds dynamically. Exceeding it fails the
	// operation loud.
	EvaluationDeadline = 100 * time.Millisecond
	// MaxConcurrentJSONSchemaEvaluations bounds how many JSON Schema
	// evaluations may be in flight at once, ACROSS THE PROCESS.
	//
	// It exists because the wall-clock budget abandons the wait, not the work:
	// the library call is synchronous and cannot be cancelled, so an overrun
	// leaves a goroutine running that nothing is waiting for. Without a
	// ceiling, a caller who can trigger overruns can accumulate them, and the
	// deadline that was meant to bound the damage becomes the thing that
	// multiplies it. A slot is released when the EVALUATION finishes, never
	// when its waiter gives up, so abandoned work still counts against the
	// bound. Admission never queues at all: a full set of slots is an immediate
	// refusal, because a queue of waiters is the unbounded work this ceiling
	// exists to stop. Concrete value is this slice's; the ops spec owns it.
	MaxConcurrentJSONSchemaEvaluations = 8
)

Bounds. Declaration and validation are attacker-triggerable work (threat model § Availability), so every one of them is a named constant with a loud named refusal rather than a number buried in a comparison. The concrete values are this slice's, chosen for the ops spec's Pi-4 floor and recorded as disposition items — the ops spec owns them once it exists.

Variables

View Source
var ErrInvalidDeclaration = errors.New("invalid declaration")

ErrInvalidDeclaration classifies every declaration-time refusal, so the service layer maps one sentinel rather than matching messages.

Functions

func Canonical

func Canonical(d Declaration) ([]byte, error)

Canonical renders the declaration's byte-stable storage form: normalized (members and schemes trimmed and lower-cased where the semantics are case-insensitive, the JSON Schema re-encoded from its parsed form) and deterministically ordered, because Go's encoder emits struct fields in declaration order and object keys sorted.

Byte stability is load-bearing three times over: it is the schema export's requirement (ADR § Authority), it makes canonical-form deduplication of identical declarations a byte comparison, and it makes "did a value-dependent rule change?" — the reveal gate's input — a byte diff rather than a field-by-field walk that a new field could silently escape.

func CheckDescription

func CheckDescription(what, s string) error

CheckDescription bounds the free-text description and refuses the two byte classes no Hikyo-held string may carry.

func CheckGroupName

func CheckGroupName(name string) error

CheckGroupName bounds a key group's name. A group is a project-level label on a coupling, so it takes the free-text entity rules rather than the delivery grammar — nothing is ever delivered under a group name.

func CheckGroupPresence

func CheckGroupPresence(a, b PresenceRules) error

CheckGroupPresence is the same conflict across two members of one key group. A group's all-or-none resolved presence means either every member resolves to `set` in an environment or none do, so one member required where another is forbidden can never hold — statically, before any value exists.

func CheckKeyName

func CheckKeyName(name string) error

CheckKeyName enforces the canonical grammar and the name bound. Names are unique among live keys per project — that half is the store's constraint, because uniqueness is a property of the set, not of the string.

func CheckPersonalRequired

func CheckPersonalRequired(personal Presence, rules PresenceRules, grouped bool) error

CheckPersonalRequired validates `personal_required_in`, the environments where each developer is expected to hold a personal value (personal-values ADR). Its mode is `none` or `explicit` only: `all` would bind environments created later to a requirement no one can see coming. A key cannot be personally required where it is also `required_in` (the shared value must exist) or `forbidden_in` (no value may exist), and a key in a group cannot carry it, because a group's presence is all-or-none.

func CheckPresence

func CheckPresence(required, forbidden Presence) error

CheckPresence enforces well-formedness and the statically decidable conflict: a key both required and forbidden in the same environment — including via `mode: all` on either side — is rejected at DECLARATION rather than discovered at publish, because a rule that can never be satisfied is a broken rule, not a value that happens to be wrong.

func Normalize

func Normalize(value string) string

Normalize is the write-time trim, and the only place it lives: leading and trailing whitespace is removed on every write path, for every type, using Unicode whitespace semantics. It runs BEFORE validation, so a whitespace-only value becomes empty and is then refused by `allow_empty` rather than stored.

It is separate from Validate because its RESULT is plaintext: the write path needs the normalized bytes, and nothing else does.

func TypeExpression

func TypeExpression(types []Type) string

TypeExpression is the compact textual form used by the key catalogue's table output and by import presence reads. A declaration with one primitive is that primitive; a union preserves its canonical alternative order.

func ValueDependentChange

func ValueDependentChange(before, after Declaration) bool

ValueDependentChange reports whether the move from `before` to `after` changes a rule that must be EVALUATED against an existing value — type, min/max, minLength/maxLength, pattern, enum members, url schemes, allow_empty, any_of alternatives, and the JSON Schema (ADR § Changing a value-dependent rule on a secret key is a disclosure).

The whole Declaration is value-dependent by construction: metadata (description, deprecated, deprecation_note, folder path) and presence rules live outside it, on the key row. So the answer is a canonical-byte comparison — which is also why a new constraint field cannot escape the reveal gate by being forgotten in a field list.

An unrenderable declaration is reported as a change: refusing to evaluate is the fail-closed direction for a gate.

Types

type Classification

type Classification string

Classification is the key's `secret | config` classification (#7). It is an INPUT to validation because it decides what a failure may say: a `config` value is readable under ordinary environment read, a `secret` one is not, and the ADR's error-disclosure rule turns on exactly that.

const (
	Config Classification = "config"
	Secret Classification = "secret"
)

func (Classification) Valid

func (c Classification) Valid() bool

Valid reports whether c is one of the two declared classifications. There is no third, and no default: a caller holding an unrecognised string has a bug, not a preference.

type Compiled

type Compiled struct {
	// contains filtered or unexported fields
}

Compiled is one key's ready-to-run rule set. It is immutable and safe for concurrent use: compiled regexps and compiled JSON Schemas are, and the engine writes nothing back.

func CompileClassified

func CompileClassified(classification Classification, d Declaration) (*Compiled, error)

CompileClassified is the persisted-declaration authority. It normalizes the wire declaration once, enforces the key classification against that exact form, and builds the validators from it. Boundary callers must use this constructor so a declaration cannot reach storage or application with its classification check omitted or reordered.

Compilation is per schema revision, not per validation: patterns and JSON Schemas are compiled once here and reused by every Validate (ADR § Bounds: "compiled once per schema revision and cached, never recompiled per validation, so a fetch storm cannot be amplified into CPU").

func (*Compiled) Canonical

func (c *Compiled) Canonical() ([]byte, error)

Canonical renders the byte-stable storage form of the same normalized declaration that the compiled validators use.

func (*Compiled) Declaration

func (c *Compiled) Declaration() Declaration

Declaration returns the normalized declaration this artifact was built from — the canonical form, so a caller storing it stores what validation used.

func (*Compiled) Validate

func (c *Compiled) Validate(value string) Verdict

Validate runs the declaration against one value. Failure disclosure uses the classification fixed by CompileClassified, so callers cannot compile under one classification and validate under another.

type Declaration

type Declaration struct {
	Rule  *Rule  `json:"rule,omitempty"`
	AnyOf []Rule `json:"any_of,omitempty"`
}

Declaration is a key's value-dependent rule set: exactly one primitive Rule, or an `any_of` union of at least two of them. Named `any_of` and never `oneOf`: JSON Schema — embedded verbatim for `json` keys — defines `oneOf` as exactly-one, and two meanings for one word inside one product is a trap for operators and implementers alike. Overlapping alternatives are fine and never an error.

func ParseDeclaration

func ParseDeclaration(raw []byte) (Declaration, error)

ParseDeclaration reads a stored declaration. Unknown fields are refused rather than dropped: a declaration written by a newer version and silently truncated by an older one would deliver a key under rules nobody declared.

type Failure

type Failure struct {
	// Alternative indexes the failing `any_of` alternative, or -1 for a
	// single-rule declaration. A total `any_of` failure enumerates every
	// alternative's own failure — a bare "matched none of 2 alternatives" is
	// useless to an operator.
	Alternative int `json:"alternative"`
	// Keyword is the schema location that failed: the declared constraint's
	// name, or `json_schema` with the failing keyword path.
	Keyword string `json:"keyword"`
	// Message is schema-derived prose, safe to show for any classification.
	Message string `json:"message"`
	// InstancePath is the JSON pointer into the value; `config` keys only.
	InstancePath string `json:"instance_path,omitempty"`
}

Failure is one rule violation. It carries the SCHEMA LOCATION that failed and never an instance-derived pointer, because for a `secret`-classified key the instance is the thing being protected: a JSON value of `{"AKIA…credential…":"x"}` failing `additionalProperties:false` would, under a naive implementation, put the plaintext in the error message, the log and the audit record.

InstancePath is populated ONLY for a `config`-classified key, which is readable under ordinary environment read anyway. It is not "redacted afterwards": for a secret key the instance-derived text never enters the Failure at all, so there is no path by which a later refactor could leak it.

type Presence

type Presence struct {
	Mode         PresenceMode `json:"mode"`
	Environments []string     `json:"environment_ids,omitempty"`
}

Presence is one rule: `required_in` or `forbidden_in`.

func (Presence) Covers

func (p Presence) Covers(envID string) bool

Covers answers whether this rule applies to one environment. `all` is symbolic and therefore covers every environment including ones created after the declaration was written — which is the whole reason the mode is stored rather than expanded into ids.

It is the value model's question (#50): "is this key forbidden here?" gates a write, and "is it required here?" gates a clear. Both are decided against the mode, never against a materialized id list.

type PresenceMode

type PresenceMode string

PresenceMode is the rule's shape. It is a mode plus an explicit set rather than a bare id list because `all` must be SYMBOLIC: had it been expanded into the ids existing at declaration time, a newly created environment would silently be exempt from a rule the operator wrote as "always".

const (
	// PresenceNone is the default for both rules.
	PresenceNone PresenceMode = "none"
	// PresenceAll covers environments created later, by construction.
	PresenceAll PresenceMode = "all"
	// PresenceExplicit carries the environment-id set.
	PresenceExplicit PresenceMode = "explicit"
)

type PresenceRules

type PresenceRules struct {
	Required  Presence `json:"required_in"`
	Forbidden Presence `json:"forbidden_in"`
}

PresenceRules is a key's pair of rules, which is the unit the group check compares.

func DefaultPresenceRules

func DefaultPresenceRules() PresenceRules

DefaultPresenceRules is the declaration default: neither required nor forbidden anywhere.

type Rule

type Rule struct {
	Type Type `json:"type"`

	// string
	MinLength  *int   `json:"min_length,omitempty"`
	MaxLength  *int   `json:"max_length,omitempty"`
	Pattern    string `json:"pattern,omitempty"`
	AllowEmpty bool   `json:"allow_empty,omitempty"`

	// integer
	Min *int64 `json:"min,omitempty"`
	Max *int64 `json:"max,omitempty"`

	// enum
	Members []string `json:"members,omitempty"`

	// url
	Schemes []string `json:"schemes,omitempty"`

	// json
	JSONSchema json.RawMessage `json:"json_schema,omitempty"`
}

Rule is one primitive type declaration with its constraints. Fields belong to exactly one type; a constraint set on the wrong type is a refusal, not an ignored field, because a silently ignored `pattern` on an integer key is the "appears to enforce something and does not" failure the ADR rejects everywhere.

type Type

type Type string

Type is one of the six primitives. `float`/`number`, `email`, `duration`, `ip`, `port` and `date` are deliberately absent — each is `string` + pattern or `integer` + range, and every type costs a validator, a UI affordance and a documentation entry (ADR § Declaration vocabulary).

const (
	TypeString  Type = "string"
	TypeInteger Type = "integer"
	TypeBoolean Type = "boolean"
	TypeEnum    Type = "enum"
	TypeURL     Type = "url"
	TypeJSON    Type = "json"
)

type Verdict

type Verdict struct {
	Valid bool `json:"valid"`
	// Errors is empty exactly when Valid.
	Errors []Failure `json:"errors,omitempty"`
}

Verdict is the engine's answer.

It carries NO VALUE and no fragment of one, by construction. A verdict is the thing that gets stored on a draft, rendered in a matrix cell, logged and audited; a plaintext field on it would put a `secret` value in every one of those places, and no amount of care at each call site is a substitute for the value not being there. The write-time normalization the caller needs is Normalize, which the WRITE path calls and the reporting path never does.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL