diagnosticexport

package
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 23, 2026 License: AGPL-3.0 Imports: 32 Imported by: 0

Documentation

Index

Constants

View Source 
const StageCanary = "canary"

StageCanary is the only stage this exporter may run in while RustFS is a release candidate.

Variables

View Source 
var ErrRemoteObjectExists = errors.New("remote object already exists")

Functions

func AcceptedStages 

func AcceptedStages() []string

AcceptedStages is the closed set of deployment stages the fleet accepts for the diagnostic exporter, and it is deliberately one declaration rather than two equal literals.

The exporter refuses to start outside this set, and the observer refuses to grade a status reported from outside it. Those are the same policy read by two processes, and they were two independent string comparisons: promoting the exporter would have relaxed its own validator while leaving the observer refusing the very status the promotion produced, turning a successful promotion into a fail-closed observer on every serving host.

Promotion is therefore one edit here, and TestAcceptedStagesAreTheOnlyStageGate fails if a second literal reappears anywhere.

func ListBundleBatch 

func ListBundleBatch(ctx context.Context, config Config, limit int) ([]string, int, int64, error)

ListBundleBatch returns the oldest bounded slice of the durable spool using the source inode modification time. Ordering by filename is not a global FIFO because the instance name precedes the capture timestamp. Invalid bundles remain visible and sort first so corruption cannot hide behind healthy traffic.

func ListBundles 

func ListBundles(config Config) ([]string, error)

func RemoveBundle 

func RemoveBundle(config Config, bundle Bundle) error

RemoveBundle removes exactly the source inode verified by ReadBundle. A replacement or symlink race fails closed and remains in the WAL.

func StageAccepted 

func StageAccepted(stage string) bool

StageAccepted reports whether a stage may run and be graded.

Types

type Bundle 

type Bundle struct {
	Name       string
	Content    []byte
	SHA256     string
	Manifest   workerdiagnostics.Manifest
	CapturedAt time.Time
	ObjectKey  string
	// contains filtered or unexported fields
}

func ReadBundle 

func ReadBundle(ctx context.Context, config Config, name string) (Bundle, error)

type Config 

type Config struct {
	SchemaVersion         int      `yaml:"schema_version" json:"schema_version"`
	DeploymentStage       string   `yaml:"deployment_stage" json:"deployment_stage"`
	SourceDirectory       string   `yaml:"source_directory" json:"source_directory"`
	SourceOwner           string   `yaml:"source_owner" json:"source_owner"`
	StateDirectory        string   `yaml:"state_directory" json:"state_directory"`
	Endpoint              string   `yaml:"endpoint" json:"endpoint"`
	Region                string   `yaml:"region" json:"region"`
	Bucket                string   `yaml:"bucket" json:"bucket"`
	Prefix                string   `yaml:"prefix" json:"prefix"`
	Repositories          []string `yaml:"repositories" json:"repositories"`
	AccountScopes         []string `yaml:"account_scopes" json:"account_scopes"`
	Pools                 []string `yaml:"pools" json:"pools"`
	Trusts                []string `yaml:"trusts" json:"trusts"`
	Platform              string   `yaml:"platform" json:"platform"`
	Architecture          string   `yaml:"architecture" json:"architecture"`
	PathStyle             bool     `yaml:"path_style" json:"path_style"`
	CAFile                string   `yaml:"ca_file" json:"ca_file"`
	AccessKeyFile         string   `yaml:"access_key_file" json:"access_key_file"`
	SecretKeyFile         string   `yaml:"secret_key_file" json:"secret_key_file"`
	RequestTimeoutSeconds int      `yaml:"request_timeout_seconds" json:"request_timeout_seconds"`
	SourceRetentionHours  int      `yaml:"source_retention_hours" json:"source_retention_hours"`
	MaxBundleBytes        int64    `yaml:"max_bundle_bytes" json:"max_bundle_bytes"`
	MaxDecompressedBytes  int64    `yaml:"max_decompressed_bytes" json:"max_decompressed_bytes"`
}

Config is deliberately narrow while RustFS remains a canary backend. It contains no secret values; systemd credential paths point at the per-unit credential mount.

func LoadConfig 

func LoadConfig(filename string) (Config, error)

func ParseConfig 

func ParseConfig(content []byte) (Config, error)

ParseConfig validates committed deployment configuration without applying the runtime ownership checks enforced by LoadConfig.

func (Config) AllowsPool 

func (c Config) AllowsPool(pool string) bool

AllowsPool reports whether a manifest pool is in the reviewed, strictly sorted allowlist. Invalid or unvalidated lists fail closed.

func (Config) AllowsRepository 

func (c Config) AllowsRepository(repository string) bool

func (Config) AllowsRepositoryAccount 

func (c Config) AllowsRepositoryAccount(repository string) bool

func (Config) AllowsTrust 

func (c Config) AllowsTrust(trust string) bool

AllowsTrust reports whether a diagnostic manifest belongs to a reviewed trust domain. Invalid or unvalidated lists fail closed.

func (Config) Validate 

func (c Config) Validate() error

type ExportError 

type ExportError struct {
	Code   string
	Failed int
}

func (ExportError) Error 

func (e ExportError) Error() string

type ExportRecord 

type ExportRecord struct {
	SHA256         string `json:"sha256"`
	ObjectKey      string `json:"object_key"`
	Bytes          int64  `json:"bytes"`
	ExportedAt     string `json:"exported_at"`
	LastVerifiedAt string `json:"last_verified_at"`
}

type Exporter 

type Exporter struct {
	Config Config
	Store  ObjectStore
	State  StateStore
	Now    func() time.Time
}

func (Exporter) Run 

func (e Exporter) Run(ctx context.Context) (Summary, error)

type ObjectStore 

type ObjectStore interface {
	Head(context.Context, string, string) (RemoteObject, error)
	Put(context.Context, string, Bundle) error
}

type Progress 

type Progress struct {
	SchemaVersion   int    `json:"schema_version"`
	ObservedAt      string `json:"observed_at"`
	LastProgressAt  string `json:"last_progress_at,omitempty"`
	LastFullSyncAt  string `json:"last_full_sync_at,omitempty"`
	ScannedBundles  int    `json:"scanned_bundles"`
	ExportedBundles int    `json:"exported_bundles"`
	DeletedBundles  int    `json:"deleted_bundles"`
	FailedBundles   int    `json:"failed_bundles"`
	BacklogBundles  int    `json:"backlog_bundles"`
	BacklogBytes    int64  `json:"backlog_bytes"`
}

func (Progress) Validate 

func (p Progress) Validate() error

type RemoteObject 

type RemoteObject struct {
	Exists        bool
	Bytes         int64
	SHA256        string
	SchemaVersion string
}

type S3Store 

type S3Store struct {
	// contains filtered or unexported fields
}

func NewS3Store 

func NewS3Store(config Config) (*S3Store, error)

func (*S3Store) Head 

func (s *S3Store) Head(ctx context.Context, bucket, key string) (RemoteObject, error)

func (*S3Store) Put 

func (s *S3Store) Put(ctx context.Context, bucket string, bundle Bundle) error

type State 

type State struct {
	SchemaVersion int                     `json:"schema_version"`
	Status        Status                  `json:"status"`
	Exports       map[string]ExportRecord `json:"exports"`
}

type StateStore 

type StateStore struct {
	Directory string
}

func (StateStore) Load 

func (s StateStore) Load() (State, error)

func (StateStore) LoadProgress 

func (s StateStore) LoadProgress() (Progress, error)

func (StateStore) Save 

func (s StateStore) Save(state State) error

func (StateStore) SaveProgress 

func (s StateStore) SaveProgress(progress Progress) error

type Status 

type Status struct {
	SchemaVersion       int    `json:"schema_version"`
	DeploymentStage     string `json:"deployment_stage"`
	ObservedAt          string `json:"observed_at"`
	LastSuccessAt       string `json:"last_success_at,omitempty"`
	LastErrorCode       string `json:"last_error_code,omitempty"`
	ConsecutiveFailures int    `json:"consecutive_failures"`
	SourceBundles       int    `json:"source_bundles"`
	ExportedBundles     int    `json:"exported_bundles"`
	PendingBundles      int    `json:"pending_bundles"`
	SourceBytes         int64  `json:"source_bytes"`
	ExportedBytes       int64  `json:"exported_bytes"`
	LastProgressAt      string `json:"-"`
	LastFullSyncAt      string `json:"-"`
	ScannedBundles      int    `json:"-"`
	DeletedBundles      int    `json:"-"`
}

func ReadStatus 

func ReadStatus(directory string) (Status, error)

func (Status) LastFullSyncTime 

func (s Status) LastFullSyncTime() (time.Time, error)

func (Status) LastProgressTime 

func (s Status) LastProgressTime() (time.Time, error)

func (Status) LastSuccessTime 

func (s Status) LastSuccessTime() (time.Time, error)

func (Status) ObservedTime 

func (s Status) ObservedTime() (time.Time, error)

type Summary 

type Summary struct {
	SourceBundles   int   `json:"source_bundles"`
	ExportedBundles int   `json:"exported_bundles"`
	PendingBundles  int   `json:"pending_bundles"`
	SourceBytes     int64 `json:"source_bytes"`
	ExportedBytes   int64 `json:"exported_bytes"`
	ScannedBundles  int   `json:"scanned_bundles"`
	DeletedBundles  int   `json:"deleted_bundles"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.