Documentation
¶
Index ¶
- Constants
- func PruneScanStore(ctx context.Context, path string) error
- func Render(w io.Writer, data []byte) error
- func RenderDatabase(w io.Writer, path string) error
- func RenderDatabaseContext(ctx context.Context, w io.Writer, path string) error
- type Access
- type CollectOptions
- type CollectSummary
- type DBSummary
- type DirectAccess
- type EnrichmentMetric
- type EnrichmentPlan
- type EnrichmentSummary
- type EnrichmentWindow
- type GrafanaAccess
- type InventoryAnalysis
- type InventoryLabel
- type InventorySnapshot
- type LabelScanOptions
- type LabelScanSummary
- type ScanOptions
- type ScanStore
- func (s *ScanStore) Close() error
- func (s *ScanStore) EnrichmentSummary(ctx context.Context) (EnrichmentSummary, error)
- func (s *ScanStore) Initialize(ctx context.Context, seed io.Reader) error
- func (s *ScanStore) LabelSummary(ctx context.Context) (LabelScanSummary, error)
- func (s *ScanStore) MigrateLabels(ctx context.Context) error
- func (s *ScanStore) PlanEnrichment(ctx context.Context, reader io.Reader) error
- func (s *ScanStore) PlanLabels(ctx context.Context, o LabelScanOptions) error
- func (s *ScanStore) PlanNamespaceRecovery(ctx context.Context) (int, error)
- func (s *ScanStore) RefreshCatalog(ctx context.Context, seed io.Reader) error
- func (s *ScanStore) RefreshCatalogRetryEmpty(ctx context.Context, seed io.Reader) error
- func (s *ScanStore) RepairPartition(ctx context.Context, parentID string) (int, error)
- func (s *ScanStore) Scan(ctx context.Context, o ScanOptions) error
- func (s *ScanStore) Summary(ctx context.Context) (DBSummary, error)
Constants ¶
const DatabaseReportMaxBytes = 16 << 20
DatabaseReportMaxBytes bounds CI output and its subsequent in-memory embedding.
Variables ¶
This section is empty.
Functions ¶
func PruneScanStore ¶
PruneScanStore compacts an existing, completed scan artifact without removing observations, attempts, errors or provenance. Stop scanners and close their stores and all readers first. This is explicit maintenance, never scan cleanup. The exclusive connection retains its lock between batches and through VACUUM, preventing a new scanner from claiming work after the durable-state check.
func Render ¶
Render writes a standalone report from a schemaVersion 1 evidence archive. Analysis, fallback tables and SVG charts are generated offline in Go. Optional CDN-hosted ECharts adds a treemap; no evidence is fetched by the browser. Failed or missing query evidence is displayed as unavailable, not as zero.
func RenderDatabase ¶
RenderDatabase writes a standalone, offline report from a schema-1 scan database. It opens SQLite read-only and reads one consistent snapshot, including during a live scan. Accepted observations and bounded full-run label summaries are used; raw responses and physical-series inventories are never embedded.
func RenderDatabaseContext ¶
RenderDatabaseContext bounds snapshot queries, CPU loops and HTML writes by ctx. Oversized reports are refused, never truncated. RenderDatabase remains available for offline rendering without the CI size limits.
Types ¶
type Access ¶
Access maps the logical Azure Resource Manager and workspace Prometheus URLs the tool reasons about (and records as evidence) to the URL actually requested and the token scope for that request. Evidence and databases only contain logical URLs, so they are independent of the access mode.
type CollectOptions ¶
type CollectOptions struct {
Workspaces []string
Start, End time.Time
Metrics []string
InventoryMetrics []string // Exact names; at most three workspace/metric pairs, two 12h snapshots each.
InventorySelection map[string][]string // Optional workspace-scoped inventory names.
Selection map[string][]string
Context json.RawMessage
RunName, ProwURL string
Output string
Credential azcore.TokenCredential
// Transport is optional and primarily useful for offline tests. Redirects are
// always disabled, including when a custom transport is supplied.
Transport http.RoundTripper
// Access selects how requests reach ARM and the workspaces; nil is DirectAccess.
Access Access
}
CollectOptions describes an explicitly bounded, Azure public-cloud collection. Metrics are matched independently against each workspace's discovered catalog. Selection optionally adds exact names keyed by workspace name or ARM ID.
func (CollectOptions) Validate ¶
func (o CollectOptions) Validate() error
Validate rejects unsafe or unbounded plans before acquiring tokens or writing.
type CollectSummary ¶
type CollectSummary struct {
Requests, Failed, PromQL, Platform, Selected int
}
CollectSummary counts attempted requests, not inferred ingestion or scan cost.
func AddInventories ¶
func AddInventories(ctx context.Context, input []byte, o CollectOptions) (CollectSummary, error)
AddInventories supplements a schema-1 artifact without repeating its requests. Output must be a new path. Source metadata and raw records are preserved; the supplied credential and transport are used only for the six-or-fewer new GETs. Workspace IDs and snapshot times come from input, not the options.
func Collect ¶
func Collect(ctx context.Context, o CollectOptions) (CollectSummary, error)
Collect makes serial GETs without retries. It creates Output exclusively and replaces it with a schema-1 checkpoint after each response. Partial evidence remains renderable on query failures or cancellation; any failure returns an error.
type DBSummary ¶
type DBSummary struct {
State string `json:"state"`
PlanningState string `json:"planningState"`
Workspaces int `json:"workspaces"`
Metrics int `json:"metrics"`
EmptyCatalogs int `json:"emptyCatalogs"`
Queries map[string]int `json:"queries"`
Blocked map[string]int `json:"blocked"`
RankingTotal int `json:"rankingTotal"`
RankingSucceeded int `json:"rankingSucceeded"`
Attempts int `json:"attempts"`
Observations int `json:"observations"`
CoverageComplete bool `json:"coverageComplete"`
SchedulingComplete bool `json:"schedulingComplete"`
NextWake float64 `json:"nextWake"`
}
DBSummary distinguishes finished scheduling from complete ranking coverage. NextWake is a Unix timestamp, zero when no delayed or leased work remains.
type DirectAccess ¶
type DirectAccess struct{}
DirectAccess talks to Azure Resource Manager and the workspace Prometheus endpoints directly with the caller's identity.
type EnrichmentMetric ¶
type EnrichmentPlan ¶
type EnrichmentPlan struct {
Windows []EnrichmentWindow `json:"windows"`
Metrics []EnrichmentMetric `json:"metrics"`
}
EnrichmentPlan is a bounded cross product, not arbitrary PromQL or endpoints.
type EnrichmentSummary ¶
type EnrichmentWindow ¶
type GrafanaAccess ¶
type GrafanaAccess struct {
// contains filtered or unexported fields
}
GrafanaAccess sends requests through the datasource resource routes of an Azure Managed Grafana instance. Grafana authenticates downstream with its own identity, so the caller needs only Grafana access, not workspace or ARM permissions. Datasources are discovered from the instance:
- ARM requests use the single Azure Monitor datasource;
- Prometheus requests use the Prometheus datasource whose URL is the workspace's query endpoint.
func NewGrafanaAccess ¶
func NewGrafanaAccess(ctx context.Context, endpoint string, credential azcore.TokenCredential, transport http.RoundTripper) (*GrafanaAccess, error)
NewGrafanaAccess discovers datasources from the Grafana instance at endpoint.
type InventoryAnalysis ¶
type InventoryAnalysis struct {
Before InventorySnapshot `json:"before"`
End InventorySnapshot `json:"end"`
Common int `json:"common"`
Added int `json:"added"`
Removed int `json:"removed"`
}
InventoryAnalysis describes retained sampled series, not received-series quota accounting. Added/removed compare two rolling (t-12h,t] sets, not cluster births.
func AnalyzeInventory ¶
func AnalyzeInventory(beforeBody, endBody []byte) (InventoryAnalysis, error)
AnalyzeInventory compares full physical identities with AMW case-insensitivity. Metric identity belongs to the enclosing inventory: count_over_time drops __name__, and no synthetic metric aliases are introduced into labelsets.
type InventoryLabel ¶
type InventorySnapshot ¶
type InventorySnapshot struct {
Series int `json:"series"`
WithoutReplica int `json:"withoutReplica"`
Labels map[string]InventoryLabel `json:"labels"`
}
type LabelScanOptions ¶
LabelScanOptions bounds the cumulative physical attempts, including retries and interrupted requests. Increasing a limit resumes the same immutable plan.
type LabelScanSummary ¶
type LabelScanSummary struct {
Metrics int `json:"metrics"`
SkippedZero int `json:"skippedZero"`
Complete int `json:"complete"`
Unverified int `json:"unverified"`
Mismatched int `json:"mismatched"`
Requests int `json:"requests"`
ChargedBytes int64 `json:"chargedBytes"`
CoverageComplete bool `json:"coverageComplete"`
Reconciled int `json:"sourceReconciled"`
SourceReconciliationComplete bool `json:"sourceReconciliationComplete"`
}
type ScanOptions ¶
type ScanOptions struct {
Workers int
Credential azcore.TokenCredential
Transport http.RoundTripper
// Access selects how requests reach the workspaces; nil is DirectAccess.
Access Access
// Kinds selects logical enrichment roles through enrichment_query links.
// Only their exact requests are executed/recovered, without partition planning
// or run completion updates. Empty preserves normal catalog scanning.
// inventory_labels selects the separate full-run label ledger exclusively.
Kinds []string
// NamespaceOnly executes only immutable namespace inventory children, without
// legacy partition planning. The durable total attempt budget includes retries.
NamespaceOnly bool
}
ScanOptions affects execution only; windows, catalogs and exact requests come from the database. Workers defaults to four, with one initial request per workspace, rising to two after ten consecutive successes.
type ScanStore ¶
type ScanStore struct {
// contains filtered or unexported fields
}
ScanStore owns one SQLite connection; caller goroutines queue for it. Independent stores/processes serialize writes through SQLite's immediate transactions, while WAL permits concurrent read snapshots. All persisted times are Unix seconds (REAL), except attempt duration_ms. Schema version is PRAGMA user_version.
func OpenScanStore ¶
OpenScanStore opens or creates a schema-1 scan database, with WAL, full sync, foreign keys and a busy timeout on every connection. Close after use.
func (*ScanStore) EnrichmentSummary ¶
func (s *ScanStore) EnrichmentSummary(ctx context.Context) (EnrichmentSummary, error)
func (*ScanStore) Initialize ¶
Initialize registers schema-1 evidence durably before importing and planning in one transaction. Once registration completes, Initialize(ctx, nil) or Scan can recover without the input. An interrupted upload still needs the input. A supplied seed must match the registered byte fingerprint. Temporary compressed seed chunks are removed atomically with plan publication; successful raw bodies are not retained afterwards. Context is provenance, not verified truth.
func (*ScanStore) LabelSummary ¶
func (s *ScanStore) LabelSummary(ctx context.Context) (LabelScanSummary, error)
func (*ScanStore) MigrateLabels ¶
MigrateLabels is offline and restartable. The original raw table remains intact until every normalized row and frozen source group has been validated. Call only on an isolated SQLite backup; no cloud request is made here.
func (*ScanStore) PlanEnrichment ¶
PlanEnrichment atomically stores an immutable plan and all requests. A nil reader resumes the stored plan; a conflicting replacement is never appended.
func (*ScanStore) PlanLabels ¶
func (s *ScanStore) PlanLabels(ctx context.Context, o LabelScanOptions) error
PlanLabels selects every catalog pair except confidently complete empty run results. The window is exactly (run.start,run.end], never a 12h inventory. Full labels use the same case-folded identity as the ranking dictionaries.
func (*ScanStore) PlanNamespaceRecovery ¶
PlanNamespaceRecovery replaces all unfinished additive ranking roots once, in one transaction. Inventory is a hint, not proof of completeness: complements always cover unseen, empty and absent labels. No requests are made here.
func (*ScanStore) RefreshCatalog ¶
RefreshCatalog merges discovery into the original fixed-window run. It never makes requests or changes accepted results. Stop scanners before refreshing.
func (*ScanStore) RefreshCatalogRetryEmpty ¶
RefreshCatalogRetryEmpty also reopens directly measured empty ranking queries. Partition and enrichment plans remain immutable; use a new DB to remeasure them.
func (*ScanStore) RepairPartition ¶
RepairPartition durably plans ONE explicitly selected hard-limit query. It makes no HTTP requests and never resets a hard cap to pending. Stop older scanners first. Scan also plans hard limits automatically. A second call for the same parent is a no-op, even after inventory changes. Cluster, namespace, instance-host and job splits preserve inherited predicates. Every root has a total descendant quota, not just a sibling cap.
func (*ScanStore) Scan ¶
func (s *ScanStore) Scan(ctx context.Context, o ScanOptions) error
Scan resumes persisted work until every query has succeeded or is blocked. Blocked coverage is not a scheduling failure: inspect Summary afterwards. Cancellation releases this process's leases; crashes are recovered at expiry.
Source Files
¶
- access.go
- active.go
- artifact.go
- collect.go
- contribution.go
- inventory.go
- render.go
- scheduler.go
- sqlite.go
- sqlite_bootstrap.go
- sqlite_budget.go
- sqlite_enrichment.go
- sqlite_events.go
- sqlite_import.go
- sqlite_label_detail.go
- sqlite_label_scan.go
- sqlite_maintenance.go
- sqlite_namespace_inventory.go
- sqlite_partition.go
- sqlite_refresh.go
- sqlite_render.go
- sqlite_sample_detail.go
- sqlite_stream.go