Documentation
¶
Overview ¶
Package storage contains the GSEER persistence layer. The v0.0.1 implementation is PostgreSQL-only (ADR-0016: no embedded DB).
M1: package skeleton only (this doc.go). M3 lands the pgx pool, golang-migrate runner (one transaction per migration), and the migration set v1 (assets, services, observations, helper views, data_sources, source_checkpoints, ingestion_dead_letter, scopes, collection_jobs, suppressions, schema_migrations). M4 lands the repository types (Asset, Service, Observation with provenance) and the search_index union view.
The storage package is read-side for search (the union view is queried by the planner) and write-side for ingestion (v0.0.2+). No storage code imports any egress package.
Index ¶
Constants ¶
const FixtureSourceName = "fixture"
FixtureSourceName is the data_sources.name of the labeled synthetic fixture source (D-M3-1: the fixture source is itself a registry row).
const ProvenanceFixture = "fixture"
ProvenanceFixture is the observations.provenance value for every row written by the fixture loader; it can never be promoted to real (QS-6).
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Asset ¶
type Asset struct {
ID int64 `json:"id"`
Kind AssetKind `json:"kind"`
Value string `json:"value"`
// Typed columns mirroring the kind (only the kind's column is set).
IP *string `json:"ip,omitempty"`
Domain *string `json:"domain,omitempty"`
Hostname *string `json:"hostname,omitempty"`
ASN int `json:"asn,omitempty"`
ASNOrg string `json:"asn_org,omitempty"`
GeoCountry string `json:"geo_country,omitempty"`
GeoRegion string `json:"geo_region,omitempty"`
GeoCity string `json:"geo_city,omitempty"`
// Sources that have observed this asset (search_index projection).
Sources []string `json:"sources,omitempty"`
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
Stale bool `json:"stale"`
}
Asset is a searched entity: an IP, a domain, or a hostname with its metadata (SPEC-05 §2 row 1). Uniqueness: (kind, value).
type AssetKind ¶
type AssetKind string
AssetKind enumerates the searchable asset types (SPEC-05 §2 row 1).
type DataSource ¶
type DataSource struct {
ID int64 `json:"id"`
Name string `json:"name"`
Connector string `json:"connector"` // "fixture", "ct", "rdap", ...
TermsRef string `json:"terms_ref,omitempty"`
// Config is the connector-specific configuration (JSONB in the DB).
Config map[string]any `json:"config,omitempty"`
// Enabled is always false for v0.0.1 runtime sources: nothing is
// collected in the Developer Preview (SPEC-09 §1).
Enabled bool `json:"enabled"`
CreatedAt time.Time `json:"created_at"`
}
DataSource is a registered data source (SPEC-05 §2 row 5). The v0.0.1 fixture source is itself a DataSource row whose TermsRef documents the labeled-synthetic nature (owner decision D-M3-1).
type Fact ¶
type Fact struct {
Asset Asset
Service *Service // nil = asset-level fact (no service row)
Observation Observation
}
Fact is one atomic ingest unit: the asset the fact is about, the optional service observed on that asset (nil = asset-level fact, no service row), and the provenance-bearing observation that ties them to a source (SPEC-05 §2/§5).
type FactResult ¶
type FactResult struct {
AssetID, ServiceID, ObservationID int64 // ServiceID 0 when Service==nil
AssetInserted, ServiceInserted, ObservationInserted bool
Suppressed bool
}
FactResult reports what one IngestFact applied: the resolved row ids (ServiceID is 0 when Fact.Service was nil) and which rows were freshly inserted (via the (xmax = 0) RETURNING heuristic). Re-ingesting an identical fact yields all inserted=false with identical ids. Suppressed=true means the fact matched an enabled suppression rule and NOTHING was written (M13); all ids are then zero.
type IngestStats ¶
type IngestStats struct {
AssetsInserted, AssetsUpdated int
ServicesInserted, ServicesUpdated int
ObservationsInserted, ObservationsDeduped int
Suppressed int
}
IngestStats aggregates an IngestFacts run. "Updated" counts rows whose identity already existed and was refreshed (idempotent replay); observations that collapsed onto an already-stored row count as deduped. Suppressed counts facts refused by the suppression gate before any write (M13) — they are neither inserted nor updated and carry no observation row.
type Observation ¶
type Observation struct {
ID int64 `json:"id"`
SourceID int64 `json:"source_id"`
AssetID int64 `json:"asset_id"`
ServiceID *int64 `json:"service_id,omitempty"`
ObservedAt time.Time `json:"observed_at"`
PayloadHash string `json:"payload_hash"`
SourceCursor string `json:"source_cursor"`
// Provenance labels the record: "fixture" for the synthetic dataset
// (QS-6, AC-9.7.x); source categories from v0.0.2 onwards.
Provenance string `json:"provenance"`
Evidence map[string]any `json:"evidence,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
Observation is one provenance-bearing fact (SPEC-05 §2 row 3, §5). It carries the L1 delivery identity (SourceID + SourceCursor) and the L2 fact identity (SourceID, AssetID, ServiceID, ObservedAt, PayloadHash). In v0.0.1 only the fixture loader writes observations (M4); real collectors arrive in v0.0.2.
type Service ¶
type Service struct {
ID int64 `json:"id"`
AssetID int64 `json:"asset_id"`
Port int `json:"port"`
Transport string `json:"transport"` // tcp | udp
Protocol string `json:"protocol"`
Name string `json:"name,omitempty"`
Product string `json:"product,omitempty"`
Version string `json:"version,omitempty"`
FirstSeen time.Time `json:"first_seen"`
LastSeen time.Time `json:"last_seen"`
Stale bool `json:"stale"`
}
Service is an observed service on an asset (SPEC-05 §2 row 2). Uniqueness: (asset_id, port, transport).
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package migrations embeds the GSEER SQL migration files so the compiled binary is self-contained (no runtime file-path lookup).
|
Package migrations embeds the GSEER SQL migration files so the compiled binary is self-contained (no runtime file-path lookup). |
|
Package postgres is the PostgreSQL adapter under internal/storage.
|
Package postgres is the PostgreSQL adapter under internal/storage. |