Documentation
¶
Index ¶
- Constants
- Variables
- func CheckoutCommit(fsys FS, dir string) (string, error)
- func HasErrors(findings []Finding) bool
- func ParseYAML(data []byte) (*Node, *SyntaxError)
- func PrototypeNames() []string
- func SchemaCommit() string
- func SchemaJSON() []byte
- func SchemaSource() string
- func SortFindings(findings []Finding)
- type Binding
- type Checker
- type Concept
- type ConceptRef
- type Entity
- type Entry
- type FS
- type File
- type Finding
- type Graph
- type HCLReader
- type Kind
- type Measure
- type Model
- type ModelReader
- type ModelRef
- type Node
- type OSFS
- type Profile
- type Property
- type RefTarget
- type Resolver
- type Severity
- type Source
- type SyntaxError
- type Validator
- type Value
Constants ¶
const ( RuleSchema = "schema" RuleNoFiles = "no-meaning-files" RuleSubdirectoryFile = "subdirectory-meaning-file" RuleDuplicateConcept = "duplicate-concept" RuleDuplicateSource = "duplicate-source" RuleDuplicateValue = "duplicate-value" RuleAmbiguousWord = "ambiguous-word" RuleReferenceSyntax = "reference-syntax" RuleUnknownConcept = "unknown-concept" RuleUnresolved = "unresolved-graph" RulePinMismatch = "pin-mismatch" RuleSelfPin = "self-pin" RuleTargetKind = "target-kind" RuleExtendsKind = "extends-kind" RuleExtendsCycle = "extends-cycle" RuleNarrowing = "narrowing" RuleUnit = "unit" RuleMeasureInput = "measure-input" RuleMeasureDimension = "measure-dimension" RuleRatio = "ratio-aggregation" RuleSource = "undeclared-source" RuleEntityBindings = "entity-bindings" RuleModels = "models" RuleBindingModel = "binding-model" RuleBindingRole = "binding-role" RuleUniversal = "universal" RuleSymlink = "symlink-meaning-file" RuleUnreadableDir = "unreadable-directory" RuleFileTooLarge = "file-too-large" )
Rule identifiers of findings. They are stable: scripts may match on them.
const ( // MaxFileBytes bounds the size of a meaning file and of a model file. MaxFileBytes = 8 << 20 // MaxYAMLDepth bounds the nesting of collections in a meaning file. MaxYAMLDepth = 64 )
The YAML subset.
A meaning file is read as a plainly defined subset of YAML 1.2, in which it agrees with the reference checker's YAML library (the `yaml` 2.x package of npm) on every file it accepts, and which it enforces before anything is interpreted. Anything outside the subset is refused with a finding of its own that says what to write instead; it is never interpreted.
The subset:
- UTF-8 text, optionally starting with a byte order mark (which may not be followed by an indented first line or a block sequence: the reference parser counts it as a column). No NUL, no other control character, no U+0085, U+2028, U+2029 or U+FEFF inside the text. Lines end in LF or CRLF (no lone CR). Tabs only where they cannot change what is read: see "Tabs" below.
- One document, optionally started by a "---" line. No directives (%YAML, %TAG) and no other document markers.
- Block mappings and sequences (a sequence may sit at the indent of its key), flow mappings and sequences (which may span lines, without comments), and scalars: plain (which may continue on following, more indented lines), single-quoted and double-quoted (on one line; the JSON escapes, \/ and \u surrogate pairs included), and literal and folded block scalars (| and >, with an optional - but no indentation indicator and no keep chomping, +: the reference parser and other readers disagree about the last line break of a file that ends in the scalar). A blank line inside or at the end of a block scalar may not hold more spaces than the text is indented by.
- No anchors, aliases, merge keys (<<) or tags of any kind, and no explicit keys (?).
- String keys only: a key that YAML reads as null, a boolean or a number must be quoted. No repeated key in one mapping.
- Numbers are decimal: an integer within the range a double holds exactly (up to 2^53), or a finite float. No 0x or 0o numbers, no .inf or .nan.
- At most MaxFileBytes of text and MaxYAMLDepth levels of nesting.
const ( RuleYAML = "yaml" RuleYAMLEncoding = "yaml-encoding" RuleYAMLCharacter = "yaml-character" RuleYAMLLineEnding = "yaml-line-ending" RuleYAMLTab = "yaml-tab" RuleYAMLDirective = "yaml-directive" RuleYAMLDocuments = "yaml-documents" RuleYAMLAnchor = "yaml-anchor" RuleYAMLTag = "yaml-tag" RuleYAMLKey = "yaml-key" RuleYAMLDuplicate = "yaml-duplicate-key" RuleYAMLNumber = "yaml-number" RuleYAMLEscape = "yaml-escape" RuleYAMLUnsupported = "yaml-unsupported" RuleYAMLLimit = "yaml-limit" )
Rule identifiers of the YAML findings.
const FileSuffix = ".meaning.yaml"
FileSuffix ends the name of every meaning file.
const MaxAmbiguousWords = 1000
MaxAmbiguousWords bounds how many words that name two concepts the universal profile lists: when graphs of thousands of concepts share one word, the pairs would number in the millions. The first MaxAmbiguousWords are listed, and a finding of the same rule says that the others are not.
Variables ¶
var ErrTooLarge = errors.New("the file is larger than what is read of it")
ErrTooLarge is the error of FS.ReadFileMax for a file of more than the bytes asked for.
Functions ¶
func CheckoutCommit ¶
CheckoutCommit returns the commit a git checkout in dir is at: the commit id that HEAD holds, or that the branch HEAD names points at. It reads plain files only (.git/HEAD, the branch's ref file or .git/packed-refs); it starts no process and uses no network, and it follows a linked worktree's .git file to its git directory. HEAD, the ref files and packed-refs are read only when they are regular files that are not symbolic links, whose names are spelled exactly as asked (a case-insensitive file system does not make MAIN the branch main), and only up to a documented size (1 KiB for a ref, 8 MiB for packed-refs); otherwise the commit cannot be read, and the error says so.
It tells which commit the checkout was made at, and nothing more: it does not look at the working tree, so files that were edited, added or deleted since the checkout are not noticed, and it does not check the objects of the repository. An error says why the commit cannot be read, for example that dir is not a git checkout.
func ParseYAML ¶
func ParseYAML(data []byte) (*Node, *SyntaxError)
ParseYAML reads one document of the subset described above. An empty file is a Null node.
func PrototypeNames ¶
func PrototypeNames() []string
PrototypeNames lists the names that the reference checker refuses in a model (see the findings of HCLReader), sorted.
func SchemaCommit ¶
func SchemaCommit() string
SchemaCommit is the commit of github.com/meaninggraph/core the embedded schema was taken from.
func SchemaJSON ¶
func SchemaJSON() []byte
SchemaJSON returns meaning.schema.json of github.com/meaninggraph/core, the schema of format meaning/draft-1, as it is embedded. SchemaSource says which commit it was taken from; scripts/check-schema-drift.sh compares both with a checkout of core.
func SchemaSource ¶
func SchemaSource() string
SchemaSource records where SchemaJSON comes from: repository, commit, path and sha256 of the file, one "key: value" per line.
func SortFindings ¶
func SortFindings(findings []Finding)
SortFindings orders findings by file, line, rule, severity and message, so that the same files always give the same list.
Types ¶
type Checker ¶
type Checker struct {
// Schema validates each file; the embedded schema when nil.
Schema Validator
// Resolve reads other graphs; when nil none is available.
Resolve Resolver
// Models reads the models bindings name; HCLReader when nil.
Models ModelReader
// Profile selects extra rules.
Profile Profile
}
Checker checks a Graph. The zero value checks against the embedded schema, with no other graph available.
func (Checker) Check ¶
Check validates every file of g against the schema and checks what the schema cannot: references resolve, extends joins compatible kinds without a cycle, values-of and units-of name entities, measures and ratios are consistent, and ids and words are unique. The findings are sorted.
type Concept ¶
type Concept struct {
ID, Kind string
Of, Extends, ValuesOf, UnitsOf string
Unit, Source string
Labels map[string]string
Synonyms map[string][]string
Values []Value
Measure *Measure
Bindings []Binding
Line int
// contains filtered or unexported fields
}
Concept is a concept of a meaning file, decoded leniently: a field of the wrong type is left empty, because the schema check reports it.
type ConceptRef ¶
type ConceptRef struct {
// Repo is {host}/{org}/{repo}, empty for a bare id.
Repo string
ID string
// Pin is the ?ref= value, empty when there is none.
Pin string
}
ConceptRef is a parsed reference to a concept: a bare id (same graph), or meaning://{host}/{org}/{repo}/{id} with an optional ?ref= pin.
func ParseConceptRef ¶
func ParseConceptRef(ref string) (ConceptRef, bool)
ParseConceptRef parses a concept reference, as the format defines it.
type Entity ¶
type Entity struct {
// Key is the names of the properties that form the entity's key.
Key []string
Properties map[string]Property
}
Entity is an entity of a ModelSpec module.
type FS ¶
type FS interface {
Stat(name string) (fs.FileInfo, error)
ReadFile(name string) ([]byte, error)
// ReadFileMax reads a file of at most max bytes without reading more of a
// larger one, which is the error ErrTooLarge.
ReadFileMax(name string, max int64) ([]byte, error)
ReadDir(name string) ([]fs.DirEntry, error)
}
FS is the file system the library reads meaning files, and the models they name, through. Paths are the host's own (filepath) paths. The library never writes, never touches the network and never starts a process.
type File ¶
type File struct {
// Path is the path as it was given.
Path string
// Root is the parsed document; nil when it could not be parsed.
Root *Node
// ParseErr is why Root is nil.
ParseErr *SyntaxError
ID, Name, License string
// Models maps a module short name to the path of its source, relative to
// the file.
Models map[string]string
Sources []Source
Concepts []*Concept
}
File is one meaning file.
type Finding ¶
type Finding struct {
// File is the path of the file the finding is about, as it was given.
File string `json:"file"`
// Line is the 1-based line, or 0 when it is not known.
Line int `json:"line,omitempty"`
// Rule is a stable identifier of the rule that was broken.
Rule string `json:"rule"`
Severity Severity `json:"severity"`
Message string `json:"message"`
}
Finding is one thing a check found.
type Graph ¶
type Graph struct {
// Address is the graph's own address, {host}/{org}/{repo} (for example
// github.com/meaninggraph/core), or empty when it is not known. Set it,
// before the graph is checked or handed to GraphResolver, for every graph
// whose files refer to the graph by its address: a meaning:// reference to
// the address from inside the graph is a reference to itself, and without
// the address it is a reference to another graph. Neither LoadDir nor
// LoadFiles sets it.
Address string
// Dir is the directory the files were read from by LoadDir, else empty.
Dir string
Files []*File
// Concepts maps a concept id to its first declaration.
Concepts map[string]*Entry
// contains filtered or unexported fields
}
Graph is a set of meaning files read as one repository of concepts, as the format defines it: a file is only packaging, and a concept id is unique across all of them.
type HCLReader ¶
type HCLReader struct{}
HCLReader reads a ModelSpec module from its HCL source. It is a port of the parser of the Node reference checker (scripts/lib/modelspec.mjs of github.com/meaninggraph/core), and accepts exactly what ModelSpec v0 HCL allows there: named blocks, and attributes whose values are strings, numbers, booleans or lists of those. Expressions, interpolation and map-style values are refused rather than guessed.
It is a stand-in. Once the ModelSpec library (github.com/modelspec-org/cli, package pkg/modelspec) is released, a ModelReader built on it replaces this one; see ModelReader.
type Model ¶
Model is what a meaning file's bindings need to know about a ModelSpec module: its entities, their keys and their properties.
type ModelReader ¶
ModelReader reads the model of a module from the file a meaning file's `models` names. The single implementation today is HCLReader, which reads the subset of ModelSpec HCL that the Node reference checker reads.
ModelReader is the seam for the ModelSpec library: once github.com/modelspec-org/cli (package pkg/modelspec) has a release, a ModelReader that calls it replaces HCLReader and nothing else in this package changes.
type ModelRef ¶
type ModelRef struct {
// Repo is {host}/{org}/{repo} for a model of another repository, empty
// for one of the same repository.
Repo, Module, Name, Pin string
}
ModelRef is a parsed modelspec:// reference to an entity.
func ParseModelRef ¶
ParseModelRef parses modelspec:///{module}.{Entity} (same repository) or modelspec://{host}/{org}/{repo}/{module}.{Entity}.
type Node ¶
type Node struct {
Kind Kind
Line int
// Text is the value of a scalar: the string itself, or the canonical
// text of a number or a boolean.
Text string
// Keys are the keys of a Map in document order.
Keys []string
// Fields are the values of a Map by key.
Fields map[string]*Node
// Items are the elements of a Seq.
Items []*Node
}
Node is a parsed YAML value with the line it starts on. Scalars are resolved the way the Node reference checker's YAML library resolves them (YAML 1.2 core schema): only true/false are booleans, and yes/no/on/off and dates are strings. See ParseYAML for the subset of YAML that is read.
func (*Node) At ¶
At returns the line of the value at a JSON pointer's tokens, or of the deepest node on the way to it that exists.
type OSFS ¶
type OSFS struct{}
OSFS reads the host's file system.
func (OSFS) ReadFileMax ¶
ReadFileMax reads a file of the host's file system, at most max bytes of it.
type Profile ¶
type Profile string
Profile selects extra rules on top of the format's.
const ( // ProfileDefault applies the rules of the format only. ProfileDefault Profile = "" // ProfileUniversal also applies the rules of a repository of universal // concepts, such as github.com/meaninggraph/core: a LICENSE file, CC0-1.0 // on every file, no models and no bindings, no meaning file outside the // root, and one word names one concept. ProfileUniversal Profile = "universal" )
type Property ¶
type Property struct {
// Type is the ModelSpec type ("string", "int", ...), empty for a
// reference to another entity.
Type string
// Reference says the property is a reference to an entity, in the sense of
// the reference checker: its `entity` attribute is set.
Reference bool
// Entity is the name of the entity a reference property points at. It is
// empty when the property is no reference, or when the attribute is not a
// string (and so names no entity).
Entity string
}
Property is a property of an entity.
func (Property) IsReference ¶
IsReference reports whether the property is a reference to an entity.
type RefTarget ¶
type RefTarget struct {
// Ref is the full name of the ref (refs/heads/main, refs/tags/v1, ...).
Ref string
// ID is what the ref file or packed-refs holds, or the commit an annotated
// tag in packed-refs peels to.
ID string
// MaybeTagObject says that ID may be that of a tag object, not of the commit:
// a loose refs/tags file holds the id of the tag object when the tag is
// annotated, and so does a packed one when packed-refs does not say that it is
// fully peeled and gives no peeled line for it. Telling the two apart means
// reading the object, which this does not do.
MaybeTagObject bool
}
RefTarget is a ref of a git checkout that a pin names, and what it points at.
func RefTargets ¶
RefTargets finds the refs that the branch or tag name names in the checkout in dir: a branch, a tag, or the branch of the remote called origin, each as a ref file or in packed-refs. Like CheckoutCommit it reads plain files, bounded, not through symbolic links, and with the exact spelling of the name. A name that is no valid branch or tag name (empty, a trailing slash, an empty, "." or ".." segment, a character outside the grammar) is an error, and so is a ref that cannot be read that way.
type Resolver ¶
Resolver gives the graph a meaning:// reference names: repo is {host}/{org}/{repo} and pin is the ?ref= value (empty when there is none). An error says why the graph is not available. Nothing in this package fetches anything: a Resolver reads what the caller has at hand.
func GraphResolver ¶
GraphResolver resolves references from graphs the caller already has, keyed by address. It follows the reference checker's rules for a repository it reads from git: a reference to another repository must carry a ?ref= pin, and a repository that cannot be read is reported as such. A supplied graph must itself be valid: a file that is not YAML in the subset, or that breaks the schema, makes the graph unreadable. The pin cannot be verified against a directory here; see VerifyPin.
A graph's own address (Graph.Address) must be set for references to meaning://<its address>/... made from inside that graph to resolve to itself; GraphResolver does not set it. The resolver is not safe for concurrent use.
type Severity ¶
type Severity string
Severity says how serious a finding is. Only errors make a check fail.
const ( // Error means the files are not valid meaning files; the check fails. Error Severity = "error" // Warning means something is likely wrong but the Node reference checker // accepts it; the check does not fail. Warning Severity = "warning" // Info states a fact the reader should know, such as which directory // stood in for another graph. Info Severity = "info" )
The severities.
type SyntaxError ¶
SyntaxError is a problem with the YAML of a file: Rule says which rule of the subset (or of YAML itself) it breaks, Line where, 0 when not known.
func (*SyntaxError) Error ¶
func (e *SyntaxError) Error() string
type Validator ¶
Validator checks a value against the meaning-file schema. *jsonschema.Schema is one; tests supply others.
func DefaultSchema ¶
func DefaultSchema() Validator
DefaultSchema returns the schema embedded in the binary, compiled once.