sqlite

package
v0.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 10, 2026 License: AGPL-3.0 Imports: 18 Imported by: 0

Documentation

Index

Constants

View Source 
const RetractLimit = 100_000

RetractLimit bounds one retraction run (same value as the PG store).

Variables

View Source 
var ErrSourceNotFound = fmt.Errorf("data source not registered")

ErrSourceNotFound — registry lookup miss (same sentinel as PG).

Functions

func BuildDSN 

func BuildDSN(path string) string

BuildDSN turns a filesystem path into the driver DSN. The path is NOT required to exist or be absolute (relative paths stay relative — the CLI resolves to absolute at setup time; opening keeps whatever the caller passes).

func ForceVersion 

func ForceVersion(ctx context.Context, db *sql.DB, version int64, dirty bool) error

ForceVersion re-points the schema_migrations row (the recovery path for DIRTY after a manual fix — same semantics as the PG force).

func FormatTS 

func FormatTS(t time.Time) string

FormatTS renders t in the canonical storage form (UTC, microseconds).

func IsSuppressedError 

func IsSuppressedError(err error) bool

IsSuppressedError reports whether err is a suppression refusal (Go gate or GSUP1 backstop trigger).

func MigrationHead 

func MigrationHead() (int, error)

MigrationHead returns the embedded head migration version.

func ParseTS 

func ParseTS(s string) (time.Time, error)

ParseTS parses any accepted timestamp shape into time.Time (UTC).

Types

type DataSourceRow 

type DataSourceRow struct {
	ID         int64
	Name       string
	Connector  string
	TermsRef   string
	ConfigJSON []byte
	Enabled    bool
	CreatedAt  time.Time
}

DataSourceRow is the registry row (same field set as the postgres package's type).

type DeadLetterRecord 

type DeadLetterRecord struct {
	ID       int64
	Payload  []byte
	Reason   string
	Status   string
	Failure  string
	Resolved string
}

DeadLetterRecord is one dead-letter row.

type Fact 

type Fact = storage.Fact

Type aliases — identical to the postgres package's, so callers' code compiles against either backend.

type FactResult 

type FactResult = storage.FactResult

Type aliases — identical to the postgres package's, so callers' code compiles against either backend.

type IndexStats 

type IndexStats struct {
	Assets       int64
	Services     int64
	Observations int64
	FixtureObs   int64
	LastObserved *time.Time
}

IndexStats is the index snapshot (searchlite.Stats parity source).

type IngestStats 

type IngestStats = storage.IngestStats

Type aliases — identical to the postgres package's, so callers' code compiles against either backend.

type MigrationState 

type MigrationState string

MigrationState is the database's migration position (the same vocabulary as the PostgreSQL CLI surface). 

const (
	StateAtHead        MigrationState = "AT_HEAD"
	StateBehind        MigrationState = "BEHIND"
	StateAhead         MigrationState = "AHEAD"
	StateDirty         MigrationState = "DIRTY"
	StateUninitialized MigrationState = "UNINITIALIZED"
)

type MigrationStatus 

type MigrationStatus struct {
	Head    int
	Version int64
	Dirty   bool
	State   MigrationState
}

MigrationStatus is a point-in-time snapshot.

func GetMigrationStatus 

func GetMigrationStatus(ctx context.Context, db *sql.DB) (MigrationStatus, error)

GetMigrationStatus reports the database's position without writing.

func Migrate 

func Migrate(ctx context.Context, db *sql.DB) (MigrationStatus, error)

Migrate applies every pending embedded migration inside its own transaction. Idempotent: AT_HEAD is a no-op.

type OpenMode 

type OpenMode string

OpenMode describes what Open did to the database file. 

const (
	// OpenCreated — the file did not exist and was created 0600.
	OpenCreated OpenMode = "created"
	// OpenExisting — the file existed and was opened as-is.
	OpenExisting OpenMode = "existing"
)

type OpenResult 

type OpenResult struct {
	DB *sql.DB
	// Mode is created|existing.
	Mode OpenMode
	// PermNote is non-empty when an existing file has group/other read
	// or write bits set (the operator's choice, surfaced — not fixed).
	PermNote string
	// Path is the (unchanged) path that was opened.
	Path string
}

OpenResult reports the opened database plus the permission decisions.

func Open 

func Open(path string) (*OpenResult, error)

Open opens (and if needed creates) the SQLite database at path with the SQ-04 permission policy and the SQ-05 pool settings, registers the gseer_* scalar functions, and returns the handle.

type PurgeStats 

type PurgeStats struct {
	ObservationsRemoved int64
	ServicesRemoved     int64
	AssetsRemoved       int64
}

PurgeStats reports one PurgeFixtures run (same field set as PG).

type Querier 

type Querier interface {
	ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error)
	QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error)
	QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row
}

Querier is the tx-or-pool surface the cores run against.

type RetractStats 

type RetractStats struct {
	AssetsRetracted     int
	ServicesRemoved     int
	ObservationsRemoved int
	AuditRows           int
}

RetractStats reports one RetractSuppressed run (same field set as the postgres package's type).

type Store 

type Store struct {
	DB *sql.DB
	// contains filtered or unexported fields
}

Store is the personal-mode storage backend.

func NewStore 

func NewStore(db *sql.DB) *Store

NewStore wraps an opened DB (Open → Migrate → NewStore).

func (*Store) AddDeadLetter 

func (s *Store) AddDeadLetter(ctx context.Context, sourceID int64, payload map[string]any, reason string) error

AddDeadLetter records one failed batch (audit + retry surface).

func (*Store) AddSuppression 

func (s *Store) AddSuppression(ctx context.Context, in SuppressInput) (suppress.Rule, bool, error)

AddSuppression inserts a rule (value canonicalized first) and returns it with its database id. existed=true when an identical (kind, value) rule was already present — the existing row is returned unchanged (idempotent re-adds are not audit events).

func (*Store) Close 

func (s *Store) Close() error

Close closes the underlying database handle.

func (*Store) DeadLetterStats 

func (s *Store) DeadLetterStats(ctx context.Context) (map[int64]int, error)

DeadLetterStats counts dead letters per source.

func (*Store) DiscardDeadLetters 

func (s *Store) DiscardDeadLetters(ctx context.Context, sourceID int64, limit int) (int64, error)

DiscardDeadLetters drops up to limit dead letters of one source and returns how many were removed.

func (*Store) EnsureFixtureSource 

func (s *Store) EnsureFixtureSource(ctx context.Context) (int64, error)

EnsureFixtureSource upserts the fixture source registry row.

func (*Store) EnsureSource 

func (s *Store) EnsureSource(ctx context.Context, name, connector, termsRef string, cfg map[string]any) (int64, error)

EnsureSource registers a data source (idempotent by name).

func (*Store) FailDeadLetter 

func (s *Store) FailDeadLetter(ctx context.Context, id int64, failure string) error

FailDeadLetter marks one dead letter failed.

func (*Store) FinishCollectionJob 

func (s *Store) FinishCollectionJob(ctx context.Context, jobID int64, status string, stats map[string]any, errMsg string) error

FinishCollectionJob closes a collection_jobs row.

func (*Store) GetCheckpoint 

func (s *Store) GetCheckpoint(ctx context.Context, sourceID int64, key string) (string, error)

GetCheckpoint reads one source checkpoint ("" when unset).

func (*Store) GetSource 

func (s *Store) GetSource(ctx context.Context, name string) (DataSourceRow, error)

GetSource resolves one registered source by name.

func (*Store) IndexSeedValues 

func (s *Store) IndexSeedValues(ctx context.Context, kind string, limit int) ([]string, error)

IndexSeedValues returns up to limit asset values of the given kind, freshest first (seed_mode "index" — no third-party target list is ever consulted).

func (*Store) IngestFact 

func (s *Store) IngestFact(ctx context.Context, f *Fact) (FactResult, error)

IngestFact applies one Fact atomically with the full suppression gate (source exclusion first, then asset rules; a suppressed fact writes nothing and lands one ingest-block audit row).

func (*Store) IngestFacts 

func (s *Store) IngestFacts(ctx context.Context, facts []Fact) (IngestStats, error)

IngestFacts applies facts one at a time (each fact commits separately; idempotent re-runs) with ONE compiled rule set for the whole batch.

func (*Store) InsertObservation 

func (s *Store) InsertObservation(ctx context.Context, o *storage.Observation) (int64, bool, error)

func (*Store) ListSources 

func (s *Store) ListSources(ctx context.Context) ([]DataSourceRow, error)

ListSources returns every registered source.

func (*Store) ListSuppressions 

func (s *Store) ListSuppressions(ctx context.Context) ([]suppress.Rule, error)

ListSuppressions returns every rule (same contract as LoadSuppressions; the CLI reads through this name).

func (*Store) LoadSuppressions 

func (s *Store) LoadSuppressions(ctx context.Context) ([]suppress.Rule, error)

LoadSuppressions reads every rule row (enabled, disabled and expired — the Matcher applies the filters so a rule flip is visible in one place).

func (*Store) PendingDeadLetters 

func (s *Store) PendingDeadLetters(ctx context.Context, sourceID int64, limit int) ([]DeadLetterRecord, error)

PendingDeadLetters returns up to limit pending dead letters of one source.

func (*Store) PurgeFixtures 

func (s *Store) PurgeFixtures(ctx context.Context) (PurgeStats, error)

PurgeFixtures deletes every fixture-tagged observation/service/asset in ONE scoped transaction (rows not tagged provenance='fixture' are never touched) — the QS-6 purge parity path.

func (*Store) RemoveSuppression 

func (s *Store) RemoveSuppression(ctx context.Context, id int64) (bool, error)

RemoveSuppression deletes a rule. The audit history survives the rule's removal (suppression_id goes NULL via ON DELETE SET NULL).

func (*Store) ResolveDeadLetter 

func (s *Store) ResolveDeadLetter(ctx context.Context, id int64, note string) error

ResolveDeadLetter marks one dead letter resolved.

func (*Store) RetractSuppressed 

func (s *Store) RetractSuppressed(ctx context.Context, rule suppress.Rule) (RetractStats, error)

RetractSuppressed removes the already-indexed records matched by rule and writes one retroactive-retract audit row per retracted asset (plus one summary row for source retractions). The whole retraction is ONE transaction.

Scope by rule kind:

  • ip/cidr: kind='ip' assets by exact / containment match (netip in Go — the personal ip column stores canonical text, so containment is decided on parsed values, never on text renderings);
  • domain: kind='domain'|'hostname' assets by exact name;
  • domain_suffix: those kinds by strict-label suffix;
  • source: every observation of that source, then orphaned services, then the source name stripped from assets.sources[], then assets left with no observation, no service and no source name.

func (*Store) SetCheckpoint 

func (s *Store) SetCheckpoint(ctx context.Context, sourceID int64, key, cursor string) error

SetCheckpoint upserts one source checkpoint.

func (*Store) SetFingerprintEngine 

func (s *Store) SetFingerprintEngine(reg *fprule.Registry)

SetFingerprintEngine attaches the fingerprint enrichment registry (same hook as the PG store; optional).

func (*Store) SetSourceEnabled 

func (s *Store) SetSourceEnabled(ctx context.Context, name string, enabled bool) error

SetSourceEnabled flips a source's enabled flag (the enable/disable gate the runner checks before collecting).

func (*Store) StartCollectionJob 

func (s *Store) StartCollectionJob(ctx context.Context, sourceID int64, kind string) (int64, error)

StartCollectionJob opens a collection_jobs row (status running).

func (*Store) Stats 

func (s *Store) Stats(ctx context.Context) (IndexStats, error)

Stats snapshots the index counts (one round-trip, read-only).

func (*Store) SuppressionApplications 

func (s *Store) SuppressionApplications(ctx context.Context, limit int) ([]SuppressionApplication, error)

suppressionApplicationsList reads the audit surface (the CLI's suppress audit/list path).

func (*Store) UpsertAsset 

func (s *Store) UpsertAsset(ctx context.Context, a *storage.Asset) (int64, bool, error)

func (*Store) UpsertService 

func (s *Store) UpsertService(ctx context.Context, sv *storage.Service) (int64, bool, error)

type SuppressInput 

type SuppressInput struct {
	Kind      suppress.Kind
	Value     string // raw (any case, trailing dot, non-punycode); normalized here
	Reason    string
	CreatedBy string
	ExpiresAt *time.Time
}

SuppressInput is one new suppression rule (same shape as PG).

type SuppressionApplication 

type SuppressionApplication struct {
	ID            int64
	SuppressionID int64
	Action        string
	AssetID       int64
	Detail        string
	AppliedAt     string
}

SuppressionApplication is one audit row (denormalized detail).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.