loader

package
v1.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 15, 2026 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Overview

Package loader provides rules loading via parser-kit (multi-source fallback/merge).

Index

Constants

View Source
const (
	ModeOnlyLocal                  = "ONLY_LOCAL"
	ModeOnlyRemote                 = "ONLY_REMOTE"
	ModeRemoteFirst                = "REMOTE_FIRST"
	ModeRemoteFirstAllowRemoteFail = "REMOTE_FIRST_ALLOW_REMOTE_FAILED"
	ModeLocalFirst                 = "LOCAL_FIRST"
	ModeLocalFirstAllowRemoteFail  = "LOCAL_FIRST_ALLOW_REMOTE_FAILED"
)

Merge/environment modes. These mirror the historical MODE values so behavior is preserved; the values are matched case-insensitively after trimming.

Variables

This section is empty.

Functions

func BuildLoadOptions

func BuildLoadOptions(_ *cmd.Config, appMode string) *parserkit.LoadOptions[define.AllowListUser]

BuildLoadOptions builds parser-kit LoadOptions from warden config and app mode.

parser-kit v3 keeps only the decoding concerns here. Timeouts, retries and TLS belong to the source that has them, so they moved to RemoteOptions, and the former AllowEmptyFile is now AllowMissing() on the file source.

func BuildSources

func BuildSources(rulesFile, dataDir, configURL, auth, appMode string, remoteOpts RemoteOptions) []parserkit.Source

BuildSources builds parser-kit sources for the given mode (priority order). When dataDir is non-empty, all *.json files in that directory are added as file sources (sorted by name).

parser-kit v3 dropped Source.Type, so local and remote fetchers are collected separately and the priorities are assigned per mode afterwards. The resulting order and priority numbers are identical to those the Type-based swap produced.

Types

type LoadResult added in v0.14.0

type LoadResult struct {
	// Attempts records actual source calls for this load only. It is diagnostic,
	// not required-source coverage or source-success freshness evidence.
	Attempts    []SourceAttempt
	Manifests   []SourceManifest
	SourceSetID string
	// Users is the resolved rule set. It is only meaningful when Err is nil.
	Users []define.AllowListUser
	// Source records where the resolved rule set came from.
	Source Source
	// Version is a short content hash of the resolved rule set (empty when Err != nil).
	Version string
	// LoadedAt is when the result was produced.
	LoadedAt time.Time
	// Degraded is true when the mode allowed a remote failure and the result was
	// served from local/last-known-good instead of the preferred remote source.
	Degraded bool
	// DegradedReason is a short, non-sensitive reason code (e.g. "remote_failed").
	DegradedReason string
	// Err is the combined root cause when loading failed. It never carries secrets.
	Err error
}

LoadResult is the outcome of a load attempt. It separates "what was read" from "how the mode decided to use it" so callers (and the background refresher) can reason about degraded operation without inspecting the concrete error.

type RemoteOptions added in v1.4.0

type RemoteOptions struct {
	HTTPClient *http.Client

	// Timeout bounds one remote request, retries included. Zero leaves the
	// request bounded only by the caller's context, which is what a zero
	// LoadOptions.HTTPTimeout did in v1.
	Timeout time.Duration

	// InsecureSkipVerify disables TLS verification for remote sources.
	InsecureSkipVerify bool
	RequireHTTPS       bool

	// MaxIdleConns is the idle-connection pool size of the client this
	// package builds (HTTP_MAX_IDLE_CONNS). Zero selects the default.
	MaxIdleConns int
}

RemoteOptions carries the per-source remote settings that parser-kit v1 kept on LoadOptions. Retries are fixed by define, so only the two configurable values live here.

func BuildRemoteOptions added in v1.4.0

func BuildRemoteOptions(cfg *cmd.Config) RemoteOptions

BuildRemoteOptions builds the remote source settings from warden config.

The values are read exactly as BuildLoadOptions read them before the upgrade: cfg.HTTPTimeout seconds when a config is present -- including zero, which means "no bound of our own" -- and define.DEFAULT_TIMEOUT when it is nil. RulesLoader.httpTimeout is deliberately not reused: it substitutes DEFAULT_TIMEOUT for a non-positive cfg.HTTPTimeout, which is right for the encrypted-remote path it serves but would change the timeout parser-kit applies.

type RulesLoader

type RulesLoader struct {
	// contains filtered or unexported fields
}

RulesLoader wraps parser-kit DataLoader and exposes FromFile/Load by (rulesFile, configURL, auth).

func NewRulesLoader

func NewRulesLoader(cfg *cmd.Config, appMode string) (*RulesLoader, error)

NewRulesLoader creates a RulesLoader using cfg and appMode.

func NewRulesLoaderWithHTTPClient added in v1.6.0

func NewRulesLoaderWithHTTPClient(cfg *cmd.Config, appMode string, client *http.Client) (*RulesLoader, error)

NewRulesLoaderWithHTTPClient supplies a caller-owned remote client. It is useful for embedded callers with their own network policy and deterministic tests. Server startup always uses NewRulesLoader and the restricted default transport.

func (*RulesLoader) FromFile

func (r *RulesLoader) FromFile(ctx context.Context, path string) ([]define.AllowListUser, error)

FromFile loads rules from a local file.

The fetcher is built exactly as BuildSources builds a file source, so a missing file is tolerated in ONLY_LOCAL mode and an empty file fails, as parser-kit v1's FromFile did through AllowEmptyFile and its JSON decode.

func (*RulesLoader) Load

func (r *RulesLoader) Load(ctx context.Context, rulesFile, dataDir, configURL, auth string) ([]define.AllowListUser, error)

Load loads rules from sources built from (rulesFile, dataDir, configURL, auth) and r.appMode. It preserves the historical signature and error semantics by delegating to LoadWithResult.

func (*RulesLoader) LoadWithResult added in v0.14.0

func (r *RulesLoader) LoadWithResult(ctx context.Context, rulesFile, dataDir, configURL, auth string) (result LoadResult)

LoadWithResult loads rules and returns a structured LoadResult that separates the read source from the mode decision. Remote failures no longer short-circuit before the mode policy is applied: modes that tolerate remote failure fall back to the local rule set and mark the result Degraded, while strict modes surface the root cause. This function never returns partial/unvalidated data with a nil Err.

type Source added in v0.14.0

type Source string

Source identifies where a successfully loaded rule set originated.

const (
	// SourceRemote indicates the rules came from the remote configuration endpoint.
	SourceRemote Source = "remote"
	// SourceLocal indicates the rules came from local file(s)/directory.
	SourceLocal Source = "local"
	// SourceMerged indicates the rules came from a merge of remote and local.
	SourceMerged Source = "merged"
	// SourceRedis indicates the rules were restored from the shared Redis cache during
	// process bootstrap or recovery. Legacy arrays lack original provenance;
	// versioned snapshots retain the original source separately.
	SourceRedis Source = "redis"
	// SourceNone indicates no rules were loaded.
	SourceNone Source = "none"
)

type SourceAttempt added in v1.5.0

type SourceAttempt struct {
	StartedAt        time.Time
	FinishedAt       time.Time
	SourceID         string
	SourceOrdinal    string
	ManifestRevision string
	Kind             Source
	Transport        string
	Outcome          string
	Reason           string
	Rows             int
	WinningRows      int
	Attempt          int
	SelectedInput    bool
}

SourceAttempt describes an actual source invocation, not a configured source. IDs are resource digests; SourceOrdinal retains the per-family position. HTTP retries remain one invocation. SelectedInput refers to loader resolution, not subsequent cache validation or publication. WinningRows counts rows surviving the loader's existing merges. An empty or fully shadowed input can be selected with zero winning rows.

type SourceDescriptor added in v1.5.0

type SourceDescriptor struct {
	ID      string
	Ordinal string
	Kind    Source
}

SourceDescriptor identifies a configured resource without retaining its address.

type SourceManifest added in v1.5.0

type SourceManifest struct {
	ObservedAt  time.Time
	Revision    string
	SourceSetID string
	Discovery   string
	Sources     []SourceDescriptor
	Sequence    int
}

SourceManifest records one actual source-building pass, including discovery. Repeated fallback discovery may produce a different revision in the same load.

func CloneManifests added in v1.5.0

func CloneManifests(input []SourceManifest) []SourceManifest

CloneManifests detaches the nested source lists for immutable consumers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL