store

package
v0.18.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 18, 2026 License: MIT Imports: 33 Imported by: 0

Documentation

Overview

RDS IAM database authentication for the Postgres backend. When DISCO_PG_IAM_AUTH is set, the scanner authenticates to Postgres with a short-lived IAM token instead of a password, so the scanner task needs no DB password provisioned. Tokens (~15 min TTL) gate only the handshake, so a fresh one is minted before every physical connection via the pgx BeforeConnect hook.

Postgres backend for *Store: single struct, driver-branched dialect bits; the SQLite path is untouched. The backend is single-tenant — a plain pool against one schema. Module consumers needing multi-tenancy (disco-saas) inject a per-connection hook via WithAfterConnect (below) to pin search_path and set the app.* GUCs their row-level-security layer needs.

Package store is the SQLite persistence layer (modernc.org/sqlite, CGO-free) for resources, relationships, hierarchy closure, and scan lifecycle. See store/CLAUDE.md for table shape and edge kinds.

Index

Constants

View Source 
const (
	DirOut  = "out"
	DirIn   = "in"
	DirBoth = "both"
)

Direction constants for GraphWalkOpts.

View Source 
const (
	RelContains          = "contains"
	RelAttachedTo        = "attached-to"
	RelUses              = "uses"
	RelRoutesTo          = "routes-to"
	RelPeer              = "peer"
	RelAssumes           = "assumes"
	RelCrossAccountTrust = "cross-account-trust" // AWS IAM role trust → foreign account/role/user
	RelCrossSubRBAC      = "cross-sub-rbac"      // Azure RBAC assignment → resource in different sub
	RelCrossProjectIAM   = "cross-project-iam"   // GCP IAM binding → SA in different project
	RelOrgIAM            = "org-iam"             // GCP org/folder IAM binding → SA in some project, org-wide blast radius
	RelBoundedBy         = "bounded-by"          // IAM principal → permission-boundary policy (AWS) or analogue
)

Relationship kind constants.

Variables

View Source 
var ErrNoPath = errors.New("no path between resources")

ErrNoPath signals GraphPath found no route to the destination within the configured depth/filter constraints. cmd layer maps this to a nonzero exit code so shell pipelines can branch on reachability.

Functions

func ResourceID 

func ResourceID(provider, accountID, nativeID string) string

ResourceID computes a stable deterministic ID for a resource. type is deliberately excluded — a resource's identity is (provider, account, native_id); its type is a versioned attribute (a type change supersedes, it does not fork).

func TargetSchemaVersion 

func TargetSchemaVersion() (int, error)

TargetSchemaVersion returns the highest migration version embedded in the binary — the schema state a fully-migrated DB will land at after Open() applies any pending migrations. Used by read-only callers (cmd/helpers.go's openDB) to detect a stale on-disk schema and reject with a clear hint.

func ToRFC3339 

func ToRFC3339(s string) string

ToRFC3339 is the exported alias for callers outside the package (cmd/summary.go) that need to project a SQLite-flavoured timestamp.

Types

type CheckRun 

type CheckRun struct {
	ID             string   `db:"id"`
	StartedAt      string   `db:"started_at"`
	FinishedAt     *string  `db:"finished_at"`
	RulesPathsJSON string   `db:"rules_paths"`
	RulesPaths     []string `db:"-"`
	PacksJSON      string   `db:"packs"`
	Packs          []string `db:"-"`
	SeverityFilter *string  `db:"severity_filter"`
	ResourceCount  *int     `db:"resource_count"`
	FindingCount   *int     `db:"finding_count"`
	// WorkspaceID is the per-workspace RLS discriminator. Omitted from the read
	// projection (checkRunColumns) like Resource.WorkspaceID — the single-tenant
	// store never reads it and the disco-saas RLS layer filters by workspace — so it stays
	// nil. Kept as a field so the type still documents the column.
	WorkspaceID *string `db:"workspace_id"`
}

CheckRun records one invocation of `disco check --persist`. ID shape mirrors scans.id (32-hex-char crypto/rand). Timestamps are RFC3339 UTC.

type Checkpoint 

type Checkpoint struct {
	ScanID    string
	Provider  string
	Service   string
	Scope     string
	LastToken string
	UpdatedAt time.Time
}

Checkpoint is a per-(scan, provider, service, scope) progress marker. Scanners save one after each successfully-upserted page; on resume, `LastToken` feeds back to the upstream SDK as a continuation cursor. The token is opaque to the store — this schema is the foundation for a future incremental-scan feature (see ROADMAP.md).

type Driver 

type Driver string

Driver names a supported backend for WrapTx. 

const (
	DriverSQLite   Driver = Driver(driverSQLite)
	DriverPostgres Driver = Driver(driverPostgres)
)

type FindingFilter 

type FindingFilter struct {
	CheckRunID string
	FindingID  string
	Category   string
	Providers  []string
	Type       string
	ResourceID string
	Since      string
	Limit      uint64
	Offset     uint64
}

FindingFilter shapes ListFindings queries. Empty fields skip the clause. Since is RFC3339; the SQL JOIN compares against check_runs.started_at.

type GraphAllOpts 

type GraphAllOpts struct {
	// IncludeManaged: when false (default), provider-managed resources only
	// appear if they have at least one edge to a customer-managed resource;
	// orphan managed resources (e.g. unused AWS-managed policies, built-in
	// Azure role definitions) drop out. When true, every resource is
	// included regardless of edges.
	IncludeManaged bool
	ExcludeTypes   []string
	ExcludeRegions []string
	MaxNodes       int
	MaxEdges       int
}

GraphAllOpts configures GraphAll. No traversal knobs (depth/kinds/direction) since GraphAll is store-dump style — every relationship in the DB is a candidate edge. Filter knobs mirror GraphWalk so the cmd layer reuses flag plumbing.

type GraphEdge 

type GraphEdge struct {
	FromID string `json:"fromId"`
	ToID   string `json:"toId"`
	Kind   string `json:"kind"`
}

GraphEdge is a directed relationship between two resource IDs.

type GraphNode 

type GraphNode struct {
	Resource Resource `json:"resource"`
	Depth    int      `json:"depth"`
}

GraphNode is a resource plus the BFS depth at which it was first reached.

type GraphPathOpts 

type GraphPathOpts struct {
	MaxDepth       int      // 0 = unlimited (capped internally to 64 to bound runtime)
	Kinds          []string // empty = all
	Direction      string   // out/in/both, default both
	IncludeManaged bool
	ExcludeTypes   []string
	ExcludeRegions []string
}

GraphPathOpts configures GraphPath. Direction defaults to "both" (treats the graph as undirected for reachability) since "can A reach B" usually wants the answer regardless of edge orientation.

type GraphResult 

type GraphResult struct {
	SeedID          string      `json:"seedId"`
	Nodes           []GraphNode `json:"nodes"`
	Edges           []GraphEdge `json:"edges"`
	TruncatedNodes  int         `json:"truncatedNodes,omitempty"`
	TruncatedEdges  int         `json:"truncatedEdges,omitempty"`
	ExcludedTypes   int         `json:"excludedTypes,omitempty"`   // dropped by ExcludeTypes
	ExcludedRegions int         `json:"excludedRegions,omitempty"` // dropped by ExcludeRegions
}

GraphResult is GraphWalk's output: seed ID plus ordered nodes and edges. Truncated{Nodes,Edges} count candidates dropped by the caller's MaxNodes/MaxEdges caps; zero when no cap or no truncation.

type GraphWalkOpts 

type GraphWalkOpts struct {
	MaxDepth  int      // inclusive; 0 = seed only
	Kinds     []string // empty = all relationship kinds
	Direction string   // "out", "in", "both" (default: "both")
	// IncludeManaged=false treats provider-managed resources as terminal: they
	// appear as edge endpoints when reached from a non-managed node, but BFS
	// does not expand through them. The seed itself is never filtered.
	IncludeManaged bool
	// ExcludeTypes drops nodes whose Type matches any pattern. Patterns are
	// either literal ("aws:iam:role") or suffix-glob ("aws:iam:*"). The seed
	// is never excluded by this filter.
	ExcludeTypes []string
	// ExcludeRegions drops nodes whose Region matches any entry exactly.
	// Resources without a region (global services) are never matched.
	// The seed is never excluded by this filter.
	ExcludeRegions []string
	// MaxNodes / MaxEdges cap the result size. BFS halts adding new nodes /
	// edges once the cap is hit; the cumulative drop count is returned in
	// GraphResult.Truncated{Nodes,Edges}. 0 means unlimited.
	MaxNodes int
	MaxEdges int
}

GraphWalkOpts configures GraphWalk traversal.

type PGOption added in v0.9.0 

type PGOption func(*pgConfig)

PGOption configures an OpenPostgres call.

func WithAfterConnect added in v0.9.0 

func WithAfterConnect(h func(context.Context, *pgconn.PgConn) error) PGOption

WithAfterConnect registers a hook that runs once per physical connection, before any handle reaches database/sql. It is the multi-tenant extension point: disco-saas passes a hook that SETs search_path to a per-tenant schema and set_config's app.tenant_id / app.workspace_id for its RLS policies. The CLI passes no options and gets a plain single-tenant pool.

Composes with RDS IAM auth (pgx's separate BeforeConnect phase) — both can be active at once.

type RelEdge added in v0.10.0 

type RelEdge struct {
	FromID    string
	ToID      string
	Kind      string
	Direction string
	Attrs     *string
}

RelEdge is one relationship to upsert in a batch via UpsertRelationships. Empty Direction defaults to "directed". Mirrors UpsertRelationship's args.

type Relationship 

type Relationship struct {
	ID           int64   `db:"id"`
	FromID       string  `db:"from_id"`
	ToID         string  `db:"to_id"`
	Kind         string  `db:"kind"`
	Direction    string  `db:"direction"`
	Attributes   *string `db:"attributes"` // JSON
	DiscoveredAt string  `db:"discovered_at"`
	// WorkspaceID is the per-workspace RLS discriminator. Omitted from the read
	// projection (relationshipColumns), like Resource.WorkspaceID — the
	// single-tenant store never reads it and disco-saas's RLS layer filters by
	// workspace — so it stays nil. (No TenantID field: the single-tenant schema
	// has no tenant_id column; the disco-saas overlay column simply isn't selected.)
	WorkspaceID *string `db:"workspace_id" json:"-"`
}

Relationship represents a directed edge between two resources.

type Resource 

type Resource struct {
	ID                string  `db:"id"                  json:"id"`
	Provider          string  `db:"provider"            json:"provider"`
	AccountID         string  `db:"account_id"          json:"accountId"`
	AccountName       *string `db:"account_name"        json:"accountName"`
	Type              string  `db:"type"                json:"type"`
	NativeID          string  `db:"native_id"           json:"nativeId"`
	Name              *string `db:"name"                json:"name"`
	Region            *string `db:"region"              json:"region"`
	Zone              *string `db:"zone"                json:"zone"`
	Status            *string `db:"status"              json:"status"`
	TagsJSON          *string `db:"tags"                json:"-"` // surfaced as `tags` via MarshalJSON
	AttributesJSON    string  `db:"attributes"          json:"-"` // surfaced as `attributes` via MarshalJSON
	CreatedAt         *string `db:"created_at"          json:"createdAt"`
	DiscoveredAt      string  `db:"discovered_at"       json:"discoveredAt"`
	DiscoveredBy      string  `db:"discovered_by"       json:"discoveredBy"`
	ManagedByProvider bool    `db:"managed_by_provider" json:"managedByProvider"`
	WorkspaceID       *string `db:"workspace_id"        json:"-"` // per-workspace RLS discriminator; nil when the disco-saas app.workspace_id GUC was unset

}

Resource represents a discovered cloud resource.

Wire shape ≠ storage shape: AttributesJSON / TagsJSON live as JSON strings in the DB but MarshalJSON / UnmarshalJSON surface them as nested `attributes` / `tags` objects on the wire, with camelCase keys matching policy.Finding and coverage.Row.

Contract: every key documented under `disco check --help` is always present on the wire (never dropped by `,omitempty`). Optional pointer fields render as `null` when unset; `tags` always emits as an object (possibly empty); `managed_by_provider` always emits its bool. Drift here is the F6 paper-cut from focus-group/SUMMARY.md — fix at this struct, not in each command's renderer.

func (Resource) MarshalJSON 

func (r Resource) MarshalJSON() ([]byte, error)

MarshalJSON emits Resource with camelCase keys and nested `attributes` / `tags` rather than stringified JSON blobs. Empty / missing / malformed blobs render as `{}` so consumers can always traverse `input.attributes.X` without a presence check.

func (*Resource) Tags 

func (r *Resource) Tags() (map[string]string, error)

Tags unmarshals the resource's JSON tags into a string map.

func (*Resource) UnmarshalAttributes 

func (r *Resource) UnmarshalAttributes(v any) error

UnmarshalAttributes unmarshals the resource's JSON attributes into v.

func (*Resource) UnmarshalJSON 

func (r *Resource) UnmarshalJSON(data []byte) error

UnmarshalJSON reverses MarshalJSON: nested `attributes` / `tags` objects fold back into AttributesJSON / TagsJSON strings so []Resource round-trips byte-stably through encode → decode.

type ResourceFilter 

type ResourceFilter struct {
	Providers    []string
	AccountID    string
	Types        []string
	ExcludeTypes []string
	Regions      []string
	Status       string
	// DiscoveredBy, when set, restricts results to rows inserted by the named
	// scan id (matches `discovered_by`). The verified_by axis is selected
	// separately via ScanAs (see the --scan-as flag).
	DiscoveredBy string
	// DiscoveredSince filters rows whose discovered_at >= this RFC3339
	// timestamp. Stored timestamps sort lexicographically same as
	// chronologically, so plain string comparison works. Pairs with
	// DiscoveredBefore for half-open `[since, before)` queries.
	DiscoveredSince string
	// DiscoveredBefore filters rows whose discovered_at < this RFC3339
	// timestamp (strict). Used for "stale" hygiene queries and as the upper
	// half of a half-open interval with DiscoveredSince.
	DiscoveredBefore string
	// CreatedBefore filters rows whose created_at < this RFC3339 timestamp.
	// Anchored on the resource's intrinsic CreateDate (lifted from the SDK at
	// scan time), NOT discovered_at. Rows with NULL created_at are excluded;
	// not every scanner lifts the SDK timestamp yet (see EBS volume precedent
	// in commit 8e61c52).
	CreatedBefore string
	// CreatedSince filters rows whose created_at >= this RFC3339 timestamp.
	// Pairs with CreatedBefore for half-open interval queries on intrinsic
	// age. Same NULL caveat as CreatedBefore.
	CreatedSince string
	TagKey       string
	TagValue     string
	Limit        uint64
	Offset       uint64
	// ID, when set, restricts the result to a single row by primary key.
	// Mirrors a `WHERE id = ?` short-circuit.
	ID string
	// IncludeManaged when false hides provider-managed resources (built-in
	// roles, AWS-owned prefix lists, etc.). Defaults false at the SQL layer.
	IncludeManaged bool
	// SkipGlobals, when true, excludes rows whose region = "global". Used by
	// `disco resources --exclude-global-region` and friends to opt out of the
	// default "include globals when filtering by --regions" behaviour.
	SkipGlobals bool
}

ResourceFilter defines optional filters for ListResources.

type ResourceVersion 

type ResourceVersion struct {
	Resource
	VerifiedAt        *string `db:"verified_at"          json:"verifiedAt"`
	VerifiedBy        *string `db:"verified_by"          json:"verifiedBy"`
	RootID            string  `db:"root_id"              json:"rootId"`
	PreviousVersionID *string `db:"previous_version_id"  json:"previousVersionId"`
	SupersededBy      *string `db:"superseded_by"        json:"supersededBy"`
	VersionRowID      string  `db:"version_row_id"       json:"versionRowId"`
	DeletedAt         *string `db:"deleted_at"           json:"deletedAt"`
	DeletedBy         *string `db:"deleted_by"           json:"deletedBy"`
}

ResourceVersion is the wire shape that carries verification and version-chain metadata. Embeds Resource, so new Resource fields cascade here automatically — no parallel edit needed.

Identity model:

  • VersionRowID is the per-row UUIDv7 PK in the resources table.
  • RootID is the deterministic ResourceID hash shared across every row in this resource's version chain. Resource.ID (embedded) also carries this hash — the resourceSelectColumns hook aliases `root_id AS id` on read so Resource projections stay consistent.
  • PreviousVersionID points at the immediate predecessor in the chain (NULL on the root row).
  • SupersededBy points at the successor that replaced this version (NULL on the current row of every chain).
  • DeletedAt / DeletedBy carry the archival tombstone (see Store.ArchiveResource); NULL on a live row. A tombstone is the current row (SupersededBy NULL) with DeletedAt set.

type Scan 

type Scan struct {
	ID            string   `db:"id"`
	StartedAt     string   `db:"started_at"`
	FinishedAt    *string  `db:"finished_at"`
	Status        string   `db:"status"`
	Providers     []string `db:"-"` // stored as JSON
	ProvidersJSON string   `db:"providers"`
	ScopeJSON     string   `db:"scope"`
	Error         *string  `db:"error"`
	// ErrorsJSON is the structured per-service failure array, JSON-encoded.
	// SQLite stores it as TEXT, PG as JSONB; both round-trip through this
	// string field. Default '[]' so SELECT * never NULL-scans.
	ErrorsJSON    *string `db:"errors"`
	ResourceCount *int    `db:"resource_count"`
	MetaJSON      *string `db:"meta"`
	// WorkspaceID is the per-workspace RLS discriminator. Omitted from the read
	// projection (scanColumns) like Resource.WorkspaceID — the single-tenant store
	// never reads it and disco-saas's RLS layer filters per-workspace, so it stays nil.
	// Kept as a field so the type still documents the column.
	WorkspaceID *string `db:"workspace_id" json:"-"`
}

Scan represents a single discovery run.

func (Scan) MarshalJSON 

func (s Scan) MarshalJSON() ([]byte, error)

MarshalJSON renders a Scan with camelCase keys, parsed providers / scope / meta objects, and RFC3339 timestamps. The SQLite `datetime('now')` shape (`YYYY-MM-DD HH:MM:SS`) is normalised so consumers can use a single `time.Parse(time.RFC3339, ...)` regardless of row source.

type ScanDiff 

type ScanDiff struct {
	FromScanID string     `json:"fromScanId"` // "A"
	ToScanID   string     `json:"toScanId"`   // "B"
	Added      []Resource `json:"added"`      // first seen in scan B
	Stale      []Resource `json:"stale"`      // last seen in scan A, not refreshed by B
}

ScanDiff summarizes the resource delta between two scan runs A and B, assuming B ran after A over the same scope.

Limitations: the current schema stores only the *latest* state of each resource, not per-scan snapshots. This means:

  • Added is precise: resources whose discovered_by == B.ID were first observed in scan B.
  • Stale is approximate: resources whose verified_by == A.ID have not been re-verified by B (or later). They may be deleted in the cloud, or simply outside B's scope.
  • Updated (attribute drift) cannot be computed without historical snapshots — not implemented.

type ScanError 

type ScanError struct {
	Provider string // "aws", "azure", "gcp"
	Service  string // e.g. "ec2", "iam", "resolve:resolveBackupVaults"
	Scope    string // accountID[/region] or subscriptionID or projectID
	Message  string // err.Error()
}

ScanError is a per-service / per-resolver failure captured during a scan. Errors never abort the scan; they are accumulated and rendered as a grouped block at the end so the user sees each failure exactly once.

type ScanErrorEntry 

type ScanErrorEntry struct {
	Service string `json:"service"`
	Region  string `json:"region"`
	Code    string `json:"code"`
	Message string `json:"message"`
}

ScanErrorEntry is one structured failure row appended to scans.errors. service / region narrow the failure scope for UI grouping/filtering; code mirrors the AWS API error code (or a synthesised "transient" / "auth" / "throttle" for non-AWS providers); message is human-readable but terse.

type ScanWarning 

type ScanWarning struct {
	Provider string // "aws", "azure", "gcp"
	Service  string // e.g. "kms:ListKeys", "compute"
	Scope    string // accountID[/region] or subscriptionID or projectID
	Message  string // err.Error()
}

ScanWarning is a non-fatal skip captured during a scan — typically an access-denied error on a single service or region. The scan orchestrator collects warnings in memory and renders them as a grouped block after the scan, instead of interleaving with progress output.

type ServiceStatus added in v0.14.0 

type ServiceStatus uint8

ServiceStatus is the terminal state of a per-service scan, surfaced as a uniform "(<scope>: <state>)" suffix on the progress line (see cmd/scan.go::serviceStatusSuffix). ServiceDisabled: account/subscription/ project hasn't enabled the service but could → "(<tenant>: disabled)". ServiceUnavailable: service not deployed in this AWS region, nothing the user can do → "(region: unavailable)". ServiceNotEntitled: service exists but the tenant can't self-enable it — a support-tier gate (Trusted Advisor API needs Business/Enterprise AWS Support), a service closed to new customers (Migration Hub), or an account AWS hasn't made eligible (CloudSearch) → "(<tenant>: not entitled)". Distinct from ServiceDisabled because there's no toggle the user controls. ServiceBillingDisabled: the project/account has billing disabled (self-enableable — associate a billing account) → "(<tenant>: billing disabled)". Sits beside ServiceDisabled (both self-enableable preconditions) rather than the warnings block. 

const (
	ServiceOK ServiceStatus = iota
	ServiceDisabled
	ServiceUnavailable
	ServiceNotEntitled
	ServiceBillingDisabled
)

type Store 

type Store struct {
	OnServiceComplete func(service, scope string, total, newCount, changed, errCount int, status ServiceStatus) // after each service scan; scope = AWS region (or "global"), Azure subscription ID, GCP project ID; errCount>0 surfaces "(with errors)", status surfaces "(<tenant>: disabled)" / "(region: unavailable)" / "(<tenant>: not entitled)" / "(<tenant>: billing disabled)"
	OnResolveStart    func(provider string)                                                                     // just before phase-2 resolvers run
	OnResolveComplete func(provider string, edges int)                                                          // after all resolvers finish
	OnWarn            func(ScanWarning)                                                                         // skip-worthy error handled (transient, access-denied)
	OnError           func(ScanError)                                                                           // service or resolver failure; never aborts the scan
	// contains filtered or unexported fields
}

func Open 

func Open(path string) (*Store, error)

Open opens (or creates) the SQLite database at path and applies any pending migrations. It creates the parent directory if it does not exist.

func OpenPostgres 

func OpenPostgres(ctx context.Context, dsn string, opts ...PGOption) (*Store, error)

OpenPostgres opens a Postgres-backed Store at dsn. With no options it's the single-tenant PG backend the CLI uses; pass WithAfterConnect to layer multi-tenancy onto every connection.

The same *Store satisfies every read/write method the SQLite path supports; dialect differences live in dialect.go, rebind helpers, and the migration runner.

func OpenReadOnly 

func OpenReadOnly(path string) (*Store, error)

OpenReadOnly opens the SQLite database at path with SQLITE_OPEN_READONLY. Skips migrate (schema is whatever's on disk) and skips chmod (caller owns the file). Any write path will fail at SQLite layer with "attempt to write a readonly database" — structural enforcement of the auditor / pipeline-handoff contract.

func WrapTx 

func WrapTx(tx *sqlx.Tx, drv Driver) *Store

WrapTx returns a *Store that runs queries against tx instead of a connection pool. The returned store does NOT own the transaction — caller must Commit or Rollback. Close() is a no-op.

Intended for read-only use from a multi-tenant request path, where the caller has already issued `SET LOCAL search_path = tenant_<hex>, public` and `SET LOCAL app.tenant_id = '<uuid>'` on the tx. Write methods that call s.db.Begin* directly (UpsertResources, UpsertRelationships, RecordHierarchyBatch, etc.) will panic on a nil pool — intentional; do not invoke them on this code path.

drv must match the dialect of the tx's driver: store.DriverPostgres for a pgx-backed tx, store.DriverSQLite for SQLite.

func (*Store) AppendScanError 

func (s *Store) AppendScanError(id string, e ScanErrorEntry) error

AppendScanError appends one structured entry to scans.errors. PG path uses jsonb_insert; the SQLite fallback JSON-encodes the array once and rewrites. Both are concurrency-safe under the existing per-scan write pattern (one scanner mutates one scan).

func (*Store) ArchiveResource added in v0.15.0 

func (s *Store) ArchiveResource(rootID, deletedBy string) (bool, error)

ArchiveResource marks the current version of the resource chain identified by rootID as an archival tombstone, and reports whether a row was archived.

It appends a new current version row that carries deleted_at (now) and deleted_by, copying the prior row's payload and superseding it (previous_version_id -> old row, old row superseded_by -> the tombstone), so the pre-archive state stays in the version chain and Store.GetResourceVersions still returns it. verified_at / verified_by are carried forward unchanged: they record the last scan that actually saw the resource, which archiving does not alter.

deletedBy attributes the tombstone — a caller identifier (user id) for a manual archive, or a scan id for an automated coverage reaper.

The archive is soft and reversible. A later scan that re-sees the resource resurrects it automatically (UpsertResources clears deleted_at on its verify-only path and starts a fresh non-deleted row on a version split); Store.RestoreResource lifts the tombstone directly.

It reports false with a nil error when rootID has no live current row — the chain does not exist, or its current row is already a tombstone. The operation is therefore idempotent: a second call archives nothing.

func (*Store) BeginRelBuffer added in v0.10.0 

func (s *Store) BeginRelBuffer() *Store

BeginRelBuffer returns a shallow copy of the Store whose UpsertRelationship calls accumulate into an in-memory buffer instead of each running its own autocommit transaction. Call FlushRelBuffer on the returned store to write every buffered edge in one transaction via UpsertRelationships.

Phase-2 resolvers emit ~1k+ edges per scan; on SQLite (MaxOpenConns=1) every autocommit INSERT serialises on the single writer, each paying prepare + WAL-frame + lock overhead. Buffering per resolver collapses that to one tx per resolver. Each call returns an independent buffer, so concurrent resolvers (Azure runs them in an errgroup) each get their own — safe to use one buffered store per resolver goroutine. activeCounter is preserved, so the ReportResolveComplete edge tally is unaffected.

func (*Store) Close 

func (s *Store) Close() error

Close closes the underlying database connection. No-op for tx-bound Stores produced by WrapTx — the caller owns the transaction lifecycle.

For a writable SQLite store it first ends the WAL session cleanly: checkpoint the WAL tail into the main file, then switch journal_mode to DELETE so SQLite deterministically removes the -wal/-shm sidecars (the next Open re-enables WAL). Without this, SQLite's best-effort last-connection auto-delete — which needs an exclusive lock and is silently skipped if a reader lingers — is the source of orphaned sidecars. Errors are ignored: a held lock leaves the WAL in place (safely replayed on next open) rather than failing the command.

func (*Store) CompleteScan 

func (s *Store) CompleteScan(id string, resourceCount int) error

CompleteScan marks a scan as completed and records the resource count.

func (*Store) CountManaged 

func (s *Store) CountManaged() (int, error)

CountManaged returns the count of provider-managed rows in resources — the population a customer-only query (IncludeManaged=false) hides. Used by `disco check` to print the excluded-managed count alongside the evaluated count so SecEng/Compliance personas don't misread the small denominator as "no resources".

func (*Store) CountResourcesByScan 

func (s *Store) CountResourcesByScan(scanID string) (int, error)

CountResourcesByScan returns the number of resources recorded under a scan ID.

func (*Store) CreateScan 

func (s *Store) CreateScan(providers []string, scope map[string]any) (string, error)

CreateScan inserts a new scan record with status "running" and returns its ID (freshly minted, 32-hex). Use CreateScanWithID when the caller (e.g. an external orchestrator) needs to assign the id ahead of time so its audit trail / resources / scans share a single identifier.

func (*Store) CreateScanWithID 

func (s *Store) CreateScanWithID(id string, providers []string, scope map[string]any) (string, error)

CreateScanWithID inserts a new scan record with the supplied id. An empty id falls back to freshly-minted 32-hex (CreateScan's legacy behaviour).

func (*Store) CurrentSchemaVersion 

func (s *Store) CurrentSchemaVersion() (int, error)

CurrentSchemaVersion reads the highest applied migration from the on-disk schema_migrations table. Returns 0 when the table is missing or empty (a fresh-but-empty DB or a pre-migration-tracking checkout). Read-only callers pair this with TargetSchemaVersion to detect stale schemas and reject before issuing queries that would surface as cryptic SQLite errors.

func (*Store) DB 

func (s *Store) DB() *sqlx.DB

DB returns the underlying sqlx.DB for use in packages that need direct access. Returns nil for tx-bound Stores produced by WrapTx.

func (*Store) DeleteCheckRun 

func (s *Store) DeleteCheckRun(id string) error

DeleteCheckRun removes the run and (via FK CASCADE) every finding row owned by it. Used by future retention-pruning paths.

func (*Store) DeleteScanCheckpoints 

func (s *Store) DeleteScanCheckpoints(scanID string) error

DeleteScanCheckpoints removes every checkpoint for a scan. Called once the scan completes so the table stays bounded — abandoned scan rows live until explicit cleanup or the next resume of the same scan_id.

func (*Store) DescendantsOf 

func (s *Store) DescendantsOf(parentID string, f ResourceFilter) ([]Resource, error)

DescendantsOf returns all resources descended from parentID (any depth). Caller passes a deterministic ResourceID hash; hierarchy_closure stores hashes, so the join key on r is resourceIDColumn() (`root_id`).

func (*Store) DiffScans 

func (s *Store) DiffScans(fromScanID, toScanID string) (*ScanDiff, error)

DiffScans returns the resource delta between two scan runs. See ScanDiff for semantics and limitations.

func (*Store) FailScan 

func (s *Store) FailScan(id string, scanErr string) error

FailScan marks a scan as failed with an error message.

func (*Store) FlushRelBuffer added in v0.10.0 

func (s *Store) FlushRelBuffer() error

FlushRelBuffer writes all edges buffered since BeginRelBuffer in a single transaction and clears the buffer. No-op (nil) on a store without a buffer.

func (*Store) GetCheckRun 

func (s *Store) GetCheckRun(id string) (*CheckRun, error)

GetCheckRun retrieves one check_run by ID. Decodes RulesPaths + Packs.

func (*Store) GetCheckpoint 

func (s *Store) GetCheckpoint(scanID, provider, service, scope string) (lastToken string, ok bool, err error)

GetCheckpoint returns the latest token for the (scan, provider, service, scope) tuple, or ("", false, nil) when no row exists. Resume logic should fall back to a fresh scan when ok=false.

func (*Store) GetResource 

func (s *Store) GetResource(id string) (*Resource, error)

GetResource retrieves a single resource by its caller-facing ID (the deterministic ResourceID hash). Resolves to the current version row of the chain whose root_id matches.

func (*Store) GetResourceVersions 

func (s *Store) GetResourceVersions(rootID string) ([]ResourceVersion, error)

GetResourceVersions walks the full version chain for a resource and returns every row in chronological order (root first). Surfaced via `disco history`.

func (*Store) GetScan 

func (s *Store) GetScan(id string) (*Scan, error)

GetScan retrieves a scan by ID.

func (*Store) GraphAll 

func (s *Store) GraphAll(opts GraphAllOpts) (*GraphResult, error)

GraphAll returns the whole resource graph in one shot — every customer resource plus every provider-managed resource that has at least one edge touching a customer resource. Edges are kept only when both endpoints survive filtering. SeedID is empty since there is no traversal seed.

Prefer GraphWalk / GraphPath for seeded queries; GraphAll's working set scales with the full resource + relationship table.

func (*Store) GraphPath 

func (s *Store) GraphPath(fromID, toID string, opts GraphPathOpts) (*GraphResult, error)

GraphPath returns the shortest edge sequence from fromID to toID using BFS across the relationships graph in the requested direction. Returns ErrNoPath if no route exists within the constraints. Nodes are returned in path order with Depth = distance from from. Edges are in path order.

func (*Store) GraphWalk 

func (s *Store) GraphWalk(seedID string, opts GraphWalkOpts) (*GraphResult, error)

GraphWalk does a bounded BFS starting at seedID, following edges in the requested direction(s) filtered by kind, and returns the reachable subgraph.

func (*Store) InsertResourcesIfAbsent added in v0.14.0 

func (s *Store) InsertResourcesIfAbsent(resources []*Resource) (inserted int, err error)

InsertResourcesIfAbsent inserts each resource only when no current-version row already holds its natural key, otherwise leaves the existing row untouched. Unlike UpsertResources it never runs the verify or version-split paths, so a populated row is never reduced back to a placeholder's attributes.

This is the reference-discovery primitive: a resolver that sees an edge into an account/subscription/project outside the current scan scope inserts an empty-attribute row at that resource's real self-node natural key, giving the edge a stable FK target (the deterministic root_id is identical whether the row is the empty placeholder or later fully populated). When that target is itself scanned — this run or a future one — its own scanner calls UpsertResources, finds the placeholder as the current version, and version-splits it to the populated form. The placeholder is preserved in history; the edge keeps resolving across the split.

Returns the count of rows actually inserted (placeholders whose natural key was already occupied don't count).

func (*Store) LatestCompleteScan 

func (s *Store) LatestCompleteScan(provider string) (*Scan, error)

LatestCompleteScan returns the most-recent scan with status "complete" or "partial" that names the given provider (or any provider when provider=""). Returns sql.ErrNoRows when no such scan exists. Used by `disco scan --if-older-than` to skip cron-driven re-scans when a recent run already covered the same provider scope.

func (*Store) LatestIncompleteScan 

func (s *Store) LatestIncompleteScan() (*Scan, error)

LatestIncompleteScan returns the most-recent scan whose status is "running" or "partial" — the candidate for `disco scan --resume` to pick up. Returns (nil, nil) when no such scan exists. Status "failed" is excluded — those require manual triage rather than blind resume.

func (*Store) ListCheckRuns 

func (s *Store) ListCheckRuns() ([]CheckRun, error)

ListCheckRuns returns every check_run, newest first. Tie-break on rowid DESC so two runs created within the same SQLite-second order deterministically (mirrors ListScans).

func (*Store) ListCheckpoints 

func (s *Store) ListCheckpoints(scanID string) ([]Checkpoint, error)

ListCheckpoints returns every checkpoint for a scan, ordered by service then scope so callers iterate deterministically. Used by the --resume CLI to summarise pending work, and by a future incremental scanner that pre-fetches all checkpoints in one round-trip.

func (*Store) ListFindings 

func (s *Store) ListFindings(f FindingFilter) ([]StoredFinding, error)

ListFindings returns findings matching the filter. Sorted by severity rank (critical → low) then finding_id then resource_id for stable reporting.

func (*Store) ListRelationships 

func (s *Store) ListRelationships() ([]Relationship, error)

ListRelationships returns every edge in the store. Used by GraphAll for the unfiltered "complete graph" subcommand; per-seed walks should prefer RelationshipsFrom / RelationshipsTo so they don't pull the whole table.

func (*Store) ListResources 

func (s *Store) ListResources(f ResourceFilter) ([]Resource, error)

ListResources returns resources matching the given filters.

func (*Store) ListScans 

func (s *Store) ListScans() ([]Scan, error)

ListScans returns all scans ordered by start time descending. Providers is decoded per row so callers can render the slice without a follow-up GetScan fan-out.

func (*Store) PartialScan 

func (s *Store) PartialScan(id string, resourceCount int, scanErr string) error

PartialScan marks a scan as partially completed: at least one provider succeeded while others failed. The error message should summarize which providers failed and why.

func (*Store) PersistCheckRun 

func (s *Store) PersistCheckRun(rulesPaths, packs []string, severityFilter string, resourceCount int, findings []StoredFinding) (string, error)

PersistCheckRun writes one check_runs row plus N findings rows in a single transaction. Returns the generated check_run ID. Caller pre- builds []StoredFinding (CheckRunID is filled in by this method) — keeps the store package free of a policy import which would cycle.

func (*Store) RecordHierarchy 

func (s *Store) RecordHierarchy(childID, parentID string) error

RecordHierarchy writes both halves of a parent/child relationship in one transaction:

  • the child's self-entry in hierarchy_closure (depth 0) plus one closure row per ancestor of parent, extending depth+1, so the transitive closure stays O(1) for "all descendants of X" (org→folder→project produces nine rows: each node's self entry plus org→folder, org→project, folder→project);
  • a depth-1 `parent → child contains` row in `relationships` so GraphWalk (relationships-only) sees the hierarchy edge — this unification is what makes Azure/GCP hierarchy visible in `disco graph`, since those providers record hierarchy via closure only.

Missing endpoints (a pair referencing a resource not in `resources`, e.g. an unscanned Azure RG) skip the relationship row and emit a ScanWarning via ReportWarning — operators see the drift, callers stay simple. Closure rows always go down regardless.

func (*Store) RecordHierarchyBatch 

func (s *Store) RecordHierarchyBatch(pairs [][2]string) error

RecordHierarchyBatch is the multi-pair form of RecordHierarchy. Pairs are `[2]string{childID, parentID}`. Missing-endpoint pairs collect into one ScanWarning at the end, so one warning per call surfaces drift without spamming busy scans.

func (*Store) RelationshipsFrom 

func (s *Store) RelationshipsFrom(fromID string, kinds ...string) ([]Relationship, error)

RelationshipsFrom returns all edges originating from a resource.

func (*Store) RelationshipsTo 

func (s *Store) RelationshipsTo(toID string, kinds ...string) ([]Relationship, error)

RelationshipsTo returns all edges pointing to a resource.

func (*Store) ReportError 

func (s *Store) ReportError(e ScanError)

ReportError fires OnError if set. Providers call this when a service or resolver fails — the error is accumulated and rendered as a grouped block at the end of the scan, never aborting other in-flight work.

func (*Store) ReportResolveComplete 

func (s *Store) ReportResolveComplete(provider string, edges int)

ReportResolveComplete fires OnResolveComplete with the supplied edge count. Call this immediately after all resolvers finish.

func (*Store) ReportResolveStart 

func (s *Store) ReportResolveStart(provider string)

ReportResolveStart fires OnResolveStart (if set). Call this immediately before the phase-2 resolver loop begins.

func (*Store) ReportService 

func (s *Store) ReportService(service, scope string, total, newCount, changed, errCount int, status ServiceStatus)

ReportService invokes OnServiceComplete if set, called after each service scan function returns. scope is the per-call dimension that would otherwise duplicate the line in multi-region / multi-account scans (AWS region or "global", Azure subscription ID, GCP project ID). total = resources seen this scan; newCount = resources discovered for the first time (never previously in the DB); changed = existing resources whose attributes or tags changed this scan (a version split); errCount = errors encountered scanning this service (>0 surfaces as a "(with errors)" suffix on the progress line). status = ServiceDisabled (not enabled in this account/project/subscription), ServiceNotEntitled (exists but the tenant can't self-enable it), or ServiceUnavailable (not deployed in this AWS region); each surfaces as a "(<scope>: <state>)" suffix, mutually exclusive with errCount>0 since a skipped service emits no errors.

func (*Store) ReportWarning 

func (s *Store) ReportWarning(w ScanWarning)

ReportWarning fires OnWarn if set. Providers call this from skip-handling helpers (skipIfAccessDenied, skipIfDenied) in place of log.Printf so that warnings can be collected and rendered as a single grouped block rather than interleaving with aligned progress output.

func (*Store) ResolveResource 

func (s *Store) ResolveResource(arg, provider, rtype, account string) (*Resource, error)

ResolveResource finds a resource by its 32-hex ID, an 8+-char hex prefix of an ID, an exact native_id/name, or a substring of native_id/name. Disambiguation flags (provider/rtype/account) narrow the result set; ambiguity surfaces as a multi-candidate error.

Two-pass matcher: exact native_id/name first, then prefix-on-id and substring-on-native_id/name, so paste-the-short-ID workflows ("the ticket says 8895a0bd") work the same as full IDs. F12 fix from focus-group/SUMMARY.md.

func (*Store) RestoreResource added in v0.15.0 

func (s *Store) RestoreResource(rootID string) (bool, error)

RestoreResource lifts the archival tombstone on the current version of the resource chain identified by rootID, clearing deleted_at / deleted_by in place, and reports whether a tombstone was lifted.

The clear is done in place on the current row (matching how a scan re-sight resurrects via UpsertResources' verify-only path) rather than as another version split, so restoring leaves no extra chain entry.

It reports false with a nil error when rootID has no archived current row — the chain does not exist, or its current row is already live. The operation is therefore idempotent.

func (*Store) ReversedContainsEdges 

func (s *Store) ReversedContainsEdges() ([]Relationship, error)

ReversedContainsEdges returns `contains` rows where `to_id` is an ancestor of `from_id` per hierarchy_closure — i.e. edges pointing child→parent instead of the canonical parent→child. Invariant: always empty; non-empty means a scanner regressed and emits reversed direction. Used by the store-side test guarding against drift; CI fails on any non-empty return.

func (*Store) SaveCheckpoint 

func (s *Store) SaveCheckpoint(scanID, provider, service, scope, lastToken string) error

SaveCheckpoint upserts a checkpoint row. lastToken may be empty (page returned no continuation token); callers persist that so resume logic distinguishes "completed" from "never started". Updates `updated_at` on every call so stale checkpoints can be detected.

func (*Store) UpsertRelationship 

func (s *Store) UpsertRelationship(fromID, toID, kind, direction string, attrs *string) error

UpsertRelationship inserts a relationship, ignoring conflicts (idempotent). On a buffered store (BeginRelBuffer) the edge is accumulated and written later by FlushRelBuffer in a single transaction instead of its own autocommit.

func (*Store) UpsertRelationships added in v0.10.0 

func (s *Store) UpsertRelationships(edges []RelEdge) error

UpsertRelationships upserts many relationships in a single transaction with a reused prepared statement — the batch form of UpsertRelationship. Idempotent (same ON CONFLICT). Empty input is a no-op. Callers that buffer via BeginRelBuffer reach this through FlushRelBuffer; it is also callable directly.

func (*Store) UpsertResource 

func (s *Store) UpsertResource(r *Resource) (int, error)

UpsertResource inserts or replaces a single resource. Delegates to UpsertResources, whose version-chain logic lives in resources_upsert.go.

func (*Store) UpsertResources 

func (s *Store) UpsertResources(resources []*Resource) (inserted int, err error)

func (*Store) WithRelCounter 

func (s *Store) WithRelCounter(c *atomic.Int64) *Store

WithRelCounter returns a shallow copy of the Store with activeCounter set. UpsertRelationship increments the counter on each call. Providers create a local atomic.Int64, pass it here, then read it for ReportResolveComplete. Safe to shallow-copy because the struct contains no embedded sync/atomic values.

func (*Store) WithUpsertCounters added in v0.14.0 

func (s *Store) WithUpsertCounters(newC, changedC *atomic.Int64) *Store

WithUpsertCounters returns a shallow copy of the Store with the new/changed upsert counters bound. UpsertResources bumps newC on each first-discovery and changedC on each version split. Providers create two local atomic.Int64 around a per-service scan dispatch, pass them here, then read them for ReportService — attributing the split per (service, scope) without threading a second count through every scanner signature. Mirrors WithRelCounter.

type StoredFinding 

type StoredFinding struct {
	RowID       int64   `db:"id"`
	CheckRunID  string  `db:"check_run_id"`
	FindingID   string  `db:"finding_id"`
	ResourceID  string  `db:"resource_id"`
	Severity    string  `db:"severity"`
	Message     string  `db:"message"`
	Provider    *string `db:"provider"`
	Type        *string `db:"type"`
	Name        *string `db:"name"`
	Region      *string `db:"region"`
	Category    *string `db:"category"`
	Remediation *string `db:"remediation"`
	RefURL      *string `db:"ref_url"`
	TagsJSON    *string `db:"tags"`
	// WorkspaceID: per-workspace RLS discriminator, omitted from the read
	// projection (findingColumns) like the other shared-table reads; stays nil.
	WorkspaceID *string `db:"workspace_id"`
}

StoredFinding is the on-disk shape of a policy.Finding. Pointer fields allow NULL in the DB so optional Rego output keys round-trip cleanly.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.