capture

package
v1.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 2, 2026 License: Apache-2.0 Imports: 38 Imported by: 0

Documentation

Overview

Package capture sends the whole catalog and stores the responses as a snapshot set: restore of the state and a check of its fingerprint, prepare, read-only requests (in parallel, within rate limits) and a search for any of them that changed data, guards, another restore, and the sequences and requests that change data, one at a time, each with the changes it made to the state.

Index

Constants

View Source
const CommandTimeout = 10 * time.Minute

CommandTimeout bounds one command step.

Variables

View Source
var (
	ErrConfig = errors.New("configuration")
	ErrEnv    = errors.New("environment")
)

Error kinds, mapped to exit codes by the command line.

Functions

func ByMethod

func ByMethod(cat *catalog.Catalog, only string) []string

ByMethod lists the selected requests (only: a glob over IDs, or empty) that change data only by their method (catalog.Request.MutatingByMethod).

func Choose

func Choose(runs []Throughput) int

Choose picks the smallest setting whose throughput is within 10% of the best one: more requests at once only where they pay, since every extra one is a chance for an effect of the order.

func LoadTokens

func LoadTokens(ctx context.Context, p *config.Project) (map[string]string, error)

loadTokens reads the token source, then the identities' own sources. LoadTokens reads the tokens of every identity.

func MigrateState

func MigrateState(ctx context.Context, p *config.Project, meta *snapshot.Meta, out, log io.Writer) (map[string]map[string]string, error)

MigrateState proves that the state now is the one a baseline whose fingerprint has an older format (meta.StateFormat) was captured from, and returns the fingerprint in the current format for the baseline. The working state is restored from the pristine copy (which must have been seeded in the current format) and fingerprinted; the items whose hash differs from the baseline's are fingerprinted again as the baseline's format made them (state.Formatter), and each must then have the baseline's hash, or be missing from both. Anything else is other data.

func Probe

func Probe(ctx context.Context, p *config.Project, cat *catalog.Catalog, opt ProbeOptions) (map[string][]Throughput, error)

Probe measures, for each service, how long its first read-only requests take at concurrency 1, 2, 4 … up to Max, on the restored and prepared state, after one unmeasured pass that warms the application's caches. The services are measured one after another and the responses are dropped. Requests that take a value from another response or upload files are left out. The state is left as the requests leave it: the capture that follows starts with a restore.

func Queries

func Queries(ctx context.Context, p *config.Project, log io.Writer) (query func(ctx context.Context, name, sql string) ([]string, error), close func(), err error)

Queries opens the database components of a project for read-only queries on their pristine copies; close releases them.

func RunLength

func RunLength(statuses []string) string

RunLength writes a list of statuses as runs: "200×60, 429×1".

func Seed

func Seed(ctx context.Context, p *config.Project, opt SeedOptions) error

Seed runs the seed commands, which load the data into the working state, then makes that state the pristine copy of every database component and records its fingerprint.

func Seeded

func Seeded(ctx context.Context, p *config.Project) (bool, error)

Seeded reports whether every database component has a pristine copy on the server its DSN names now.

func Wait

func Wait()

Wait waits until every previous capture moved aside is removed.

Types

type Options

type Options struct {
	Tag                         string
	Only                        string // glob over request IDs
	Force                       bool   // replace an existing capture with this tag
	AllowRemote                 bool   // allow a target outside allow_hosts
	AllowRemoteMutations        bool   // and send requests that change data to it
	AllowMutatingWithoutRestore bool
	Restore                     string        // once (default): before the run and before the requests that change data; each: also before every sequence
	Baseline                    *snapshot.Set // the state must match the one this baseline was captured from
	SkipFingerprintCheck        bool
	Out                         io.Writer // progress
	Log                         io.Writer // output of hook commands
	Env                         []string  // added to the environment of the project's commands (bisect: the commit)
	// Concurrency replaces the configured concurrency of the services it
	// names (the decision of concurrency: auto); it applies to the
	// read-only requests only, as every concurrency does.
	Concurrency map[string]int
}

Options of one capture.

type ProbeOptions

type ProbeOptions struct {
	Services    []string // canonical names
	Requests    int      // read-only requests sent at each setting
	Max         int      // the highest setting tried
	AllowRemote bool
	Out, Log    io.Writer
	Env         []string
}

ProbeOptions say which services the throughput probe measures.

type SeedOptions

type SeedOptions struct {
	// SkipCommands copies the working state as it is, without the seed:
	// commands: the data was loaded another way (a cached dump in CI).
	SkipCommands bool
	Out, Log     io.Writer
}

SeedOptions configure driftgate seed.

type Summary

type Summary struct {
	Requests     int
	Failed       []string                 // requests whose transport failed
	Personal     map[string][]pii.Finding // public requests whose body held personal data or credentials (stored as private)
	Unscanned    map[string]string        // public requests whose body could not be scanned, and why (stored as private)
	Tracked      bool                     // side effects were recorded
	Writers      map[string][]string      // read-only requests that changed the state → items
	Unattributed []string                 // items read-only requests changed that no single request explains
	Time         Time                     // where the time went
}

Summary of a finished capture.

func Run

func Run(ctx context.Context, p *config.Project, cat *catalog.Catalog, opt Options) (_ *Summary, err error)

Run captures the catalog into the project's store under opt.Tag.

type Throughput

type Throughput struct {
	Concurrency int
	Requests    int
	Duration    time.Duration
}

Throughput is how long the probe requests took at one setting.

type Time

type Time struct {
	Total, Fingerprint, Restore time.Duration
	Fingerprints, Reused        int // fingerprints taken, and those a check of the write counters made unnecessary
}

Time is where a capture spent its time.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL