Documentation
¶
Overview ¶
Package specoracle keeps a local, gitignored cache of the OpenAPI blocks JFrog embeds on each docs.jfrog.com endpoint page and compares them with the specs this project authors under api/. JFrog's definitions are proprietary, so the cache is a test oracle only: it is fetched on demand, never committed, and the committed artefact is the parity baseline, which lists only paths, field names and status codes (decision record 0001).
Index ¶
- Constants
- Variables
- func ExtractBlock(page string) ([]byte, bool)
- func FamilySlug(index string) string
- func NormalisePath(p string) string
- func PageSlug(pageURL string) string
- func ParseIndex(body string) [][2]string
- type Baseline
- type ContentRecord
- type Diff
- type FamilySummary
- type FetchOptions
- type Inventory
- type Manifest
- type ManifestEntry
- type OpRecord
- type Operation
- type Oracle
- type Param
- type ParamRecord
- type Report
- type ScaffoldOptions
- type ScaffoldResult
- type SchemaField
- type SchemaRecord
Constants ¶
const DefaultUserAgent = "artifactweave-specoracle (https://github.com/weaveplatform/artifactweave)"
DefaultUserAgent identifies the fetcher to docs.jfrog.com so a polite operator can find us if the crawl misbehaves.
const ManifestFile = "manifest.json"
ManifestFile is the name of the manifest written into the cache directory. Its presence is how TestSpecParity decides whether a cache exists.
const Placeholder = "TODO(author)"
Placeholder is written into every description a scaffolded fragment carries. TestOpenAPIFieldDescriptions rejects it once the fragment loses its x-draft marker, so an author cannot ship a scaffold by accident.
Variables ¶
var ( // ErrBadStatus is wrapped around unexpected HTTP statuses. ErrBadStatus = errors.New("unexpected HTTP status") )
Sentinel errors, wrapped with context where they are returned.
Functions ¶
func ExtractBlock ¶
ExtractBlock finds the OpenAPI document embedded in an endpoint page. The pages carry it under an "OpenAPI definition" heading as a fenced json block; older renderings inline the JSON. Both are handled, and HTML entities are decoded because some pages escape angle brackets inside descriptions. The returned bytes are compacted JSON so cache hashes are stable across whitespace changes on the site.
func FamilySlug ¶
FamilySlug turns an index URL into a directory name, dropping the host, the "reference" segment and the llms.txt file name: https://docs.jfrog.com/artifactory/reference/repositories-api/llms.txt becomes artifactory-repositories-api.
func NormalisePath ¶
NormalisePath replaces every {name} with {} so differently named path parameters still match.
func ParseIndex ¶
ParseIndex extracts (title, url) pairs from an llms.txt body.
Types ¶
type Baseline ¶
type Baseline struct {
Missing []string `json:"missing"`
Divergent map[string][]string `json:"divergent"`
}
Baseline is the committed list of accepted gaps. It names only operation keys and finding strings (paths, field names, status codes), never JFrog's text, so it can live in the repository while the cache cannot.
func ReadBaseline ¶
ReadBaseline loads a baseline file. A missing file is an empty baseline, which makes every documented operation a regression: that is the right failure mode for a repository that forgot to commit the file.
type ContentRecord ¶
ContentRecord names the schema behind one media type.
type Diff ¶
Diff splits the report against a baseline into regressions (new gaps) and improvements (baseline entries that no longer apply and can be removed).
type FamilySummary ¶
FamilySummary is one row of Summary.
type FetchOptions ¶
type FetchOptions struct {
// Indexes are llms.txt URLs; each lists endpoint pages as "- [Title](url): text".
Indexes []string
// CacheDir receives <index-slug>/<page-slug>.json files and the manifest.
CacheDir string
// Client is used for every request; nil means http.DefaultClient with a timeout.
Client *http.Client
// Concurrency bounds parallel page fetches (default 4).
Concurrency int
// Delay is the pause before each page request (default 200 ms).
Delay time.Duration
// UserAgent overrides DefaultUserAgent.
UserAgent string
// Retries is the number of attempts on 429 and 5xx responses (default 3).
Retries int
}
FetchOptions configures Fetch. Zero values pick polite defaults.
type Inventory ¶
type Inventory struct {
GeneratedAt time.Time `json:"generated_at"`
Pages int `json:"pages"`
Schemas []SchemaRecord `json:"schemas"`
Operations []OpRecord `json:"operations"`
Errors []string `json:"errors,omitempty"`
}
Inventory is what `specoracle inventory` writes.
func BuildInventory ¶
BuildInventory extracts schemas and operations from every cached block.
func (Inventory) WriteMarkdown ¶
WriteMarkdown writes the human-readable inventory: one operations table and one fields table per family, no prose from the source.
type Manifest ¶
type Manifest struct {
FetchedAt time.Time `json:"fetched_at"`
UserAgent string `json:"user_agent"`
Entries []ManifestEntry `json:"entries"`
}
Manifest is written as manifest.json and lists what the cache holds.
func Fetch ¶
func Fetch(ctx context.Context, opts FetchOptions) (Manifest, error)
Fetch downloads every endpoint page listed by the indexes, extracts the embedded OpenAPI block and writes it into CacheDir. Pages without a block and transport failures are recorded in the manifest rather than aborting, because the docs site changes and a single bad page must not hide the rest.
type ManifestEntry ¶
type ManifestEntry struct {
Index string `json:"index"`
Family string `json:"family"`
Title string `json:"title"`
URL string `json:"url"`
File string `json:"file,omitempty"`
Status int `json:"status,omitempty"`
SHA256 string `json:"sha256,omitempty"`
Skipped string `json:"skipped,omitempty"`
Error string `json:"error,omitempty"`
}
ManifestEntry records one page of one index.
type OpRecord ¶
type OpRecord struct {
Key string `json:"key"`
Family string `json:"family"`
Tag string `json:"tag,omitempty"`
ID string `json:"id,omitempty"`
Method string `json:"method"`
Path string `json:"path"`
Source string `json:"source"`
Params []ParamRecord `json:"params,omitempty"`
Request []ContentRecord `json:"request,omitempty"`
Responses map[string][]ContentRecord `json:"responses"`
Deprecated bool `json:"deprecated,omitempty"`
}
OpRecord is one documented operation.
type Operation ¶
type Operation struct {
// Key is "METHOD /full/path" with every path parameter name replaced by
// {}, because clients never see parameter names.
Key string
Family string
Source string
ID string
Method string
Path string
Params []Param
RequestMedia []string
RequestProps []string
// Responses maps a status code to the property paths of its JSON body.
Responses map[string][]string
Deprecated bool
}
Operation is one method on one full path, flattened to what matters for wire compatibility. Property paths use "a.b" for nested objects and "a[].b" for array items.
type Oracle ¶
Oracle is a set of operations plus the parse problems met while loading.
type ParamRecord ¶
type ParamRecord struct {
Name string `json:"name"`
In string `json:"in"`
Required bool `json:"required,omitempty"`
Type string `json:"type,omitempty"`
Enum []string `json:"enum,omitempty"`
Default any `json:"default,omitempty"`
}
ParamRecord is an operation parameter with the facts clients depend on.
type Report ¶
type Report struct {
// Missing lists operation keys JFrog documents that we do not define.
Missing []string
// Divergent maps an operation key to the findings against our definition.
Divergent map[string][]string
// Matched lists keys we define with no findings.
Matched []string
// Extra lists keys we define that JFrog does not document (probes, stubs).
Extra []string
// Families records the family of every JFrog key, for the summary.
Families map[string]string
// Errors carries parse problems from either side.
Errors []string
}
Report is the outcome of comparing JFrog's operations with ours.
func Compare ¶
Compare checks every documented operation against ours. Descriptions are never compared: only what a client can observe on the wire.
func (Report) Summary ¶
func (r Report) Summary() []FamilySummary
Summary counts outcomes per documentation family, sorted by family.
func (Report) WriteBaseline ¶
WriteBaseline records the report's current gaps as the accepted baseline.
type ScaffoldOptions ¶
type ScaffoldOptions struct {
// CacheDir is the oracle cache.
CacheDir string
// Family is the cache directory (index slug) to scaffold.
Family string
// ServiceBase is our service prefix, for example "/artifactory". Only
// operations whose full path starts with it are emitted; the rest are
// listed in Result.Skipped so the author can scaffold them elsewhere.
ServiceBase string
// Tag is written on every operation; defaults to the family's last word.
Tag string
// SkipSchemas are component names the service already defines (the shared
// error envelope, for instance); references are kept but the schema is not
// emitted again.
SkipSchemas []string
}
ScaffoldOptions selects what to scaffold and how to name it.
type ScaffoldResult ¶
type ScaffoldResult struct {
YAML []byte
Operations int
Schemas int
// Skipped lists operation keys outside ServiceBase.
Skipped []string
// Renamed maps original schema names to the family-prefixed names used
// because another family defines the same name with a different shape.
Renamed map[string]string
}
ScaffoldResult reports what was written and what the author must resolve.
func Scaffold ¶
func Scaffold(opts ScaffoldOptions) (ScaffoldResult, error)
Scaffold converts every block of one family into an OpenAPI 3.0.3 fragment (paths plus components) with placeholder descriptions. Structure is a fact about the wire protocol; prose is not, so none of JFrog's is carried over.
type SchemaField ¶
type SchemaField struct {
Name string `json:"name"`
Type string `json:"type,omitempty"`
Format string `json:"format,omitempty"`
Ref string `json:"ref,omitempty"`
Items string `json:"items,omitempty"`
Enum []string `json:"enum,omitempty"`
Required bool `json:"required,omitempty"`
ReadOnly bool `json:"read_only,omitempty"`
// WriteOnly marks fields JFrog accepts but never returns, such as passwords.
WriteOnly bool `json:"write_only,omitempty"`
Nullable bool `json:"nullable,omitempty"`
Default any `json:"default,omitempty"`
}
SchemaField is one property of a named schema.
type SchemaRecord ¶
type SchemaRecord struct {
Family string `json:"family"`
Name string `json:"name"`
Type string `json:"type,omitempty"`
Composed []string `json:"composed,omitempty"`
Fields []SchemaField `json:"fields,omitempty"`
Variants int `json:"variants"`
UsedBy int `json:"used_by"`
}
SchemaRecord is a named component schema within one family. Variants counts the distinct shapes the same name has across that family's blocks; anything above one means authors must look at both pages before trusting the fields.