Documentation
¶
Overview ¶
Package storage provides a generic layered YAML store engine.
Both internal/config and internal/project compose a Store[T] with their own schema types. The store handles file discovery (static paths or walk-up), per-file loading with migrations, N-way merge with provenance tracking, and scoped writes with atomic I/O.
Index ¶
- Variables
- func GenerateDefaultsYAML[T Schema]() string
- func ResolveProjectRoot() (string, error)
- func ValidateDirectories() error
- type Field
- type FieldKind
- type FieldSet
- type KindFunc
- type LayerInfo
- type Migration
- type NormalizeOption
- type Option
- func WithCacheDir() Option
- func WithConfigDir() Option
- func WithDataDir() Option
- func WithDefaultFilename(name string) Option
- func WithDefaults(yaml string) Option
- func WithDefaultsFromStruct[T Schema]() Option
- func WithDirs(dirs ...string) Option
- func WithDotDefault() Option
- func WithFilenames(names ...string) Option
- func WithLock() Option
- func WithMigrations(fns ...Migration) Option
- func WithPaths(dirs ...string) Option
- func WithStateDir() Option
- func WithWalkUp() Option
- type Schema
- type Store
- func (s *Store[T]) Delete(path string) (bool, error)
- func (s *Store[T]) Get() *Tdeprecated
- func (s *Store[T]) Layers() []LayerInfo
- func (s *Store[T]) MarkForWrite(path string)
- func (s *Store[T]) Provenance(path string) (LayerInfo, bool)
- func (s *Store[T]) ProvenanceMap() map[string]string
- func (s *Store[T]) Read() *T
- func (s *Store[T]) Refresh() error
- func (s *Store[T]) Set(fn func(*T)) error
- func (s *Store[T]) Write(opts ...WriteOption) error
- func (s *Store[T]) WriteTo(path string) error
- type WriteOption
Constants ¶
This section is empty.
Variables ¶
var ErrNotInProject = errors.New("storage: CWD is not within a registered project")
ErrNotInProject is returned when CWD is not within a registered project's directory tree. Walk-up discovery falls back to home-level configs only.
var ErrRegistryNotFound = errors.New("storage: project registry not found")
ErrRegistryNotFound is returned when the project registry file cannot be located during walk-up discovery. Discovery continues with explicit paths.
Functions ¶
func GenerateDefaultsYAML ¶ added in v0.6.0
GenerateDefaultsYAML walks the struct tags of T and produces a YAML string containing all fields that have a non-empty `default` tag. The output is suitable for WithDefaults — it provides the same base-layer behavior as a handwritten YAML constant, but derived from the struct definition.
Type coercion ensures YAML types match the Go field type:
- KindBool → YAML bool (true/false)
- KindInt → YAML int
- KindStringSlice → YAML sequence (comma-separated tag → []string)
- KindDuration → YAML string (e.g. "30s")
- KindText → YAML string
func ResolveProjectRoot ¶
ResolveProjectRoot resolves the project root for the current working directory by reading the project registry and finding the deepest registered root that contains CWD. Returns ErrRegistryNotFound if the registry does not exist, and ErrNotInProject if CWD is not within any registered project.
func ValidateDirectories ¶ added in v0.3.2
func ValidateDirectories() error
ValidateDirectories resolves all four XDG-style directories and returns an error if any two resolve to the same path. This catches misconfiguration (e.g. CLAWKER_DATA_DIR accidentally pointing at the config directory).
Types ¶
type Field ¶ added in v0.6.0
type Field interface {
Path() string // Dotted YAML path (e.g. "build.image").
Kind() FieldKind // Data type classification.
Label() string // Short human-readable name (from `label` tag or derived from YAML key).
Description() string // Help text (from `desc` tag).
Default() string // Default value hint (from `default` tag), may be empty.
Required() bool // Whether this field must have a value (from `required:"true"` tag).
MergeTag() string // Merge strategy hint (from `merge` tag): "union", "overwrite", or "".
}
Field describes a single configuration field's schema metadata. Concrete implementations are created by NewField or NormalizeFields.
type FieldKind ¶ added in v0.6.0
type FieldKind int
FieldKind classifies a configuration field's data type for schema consumers (TUI editors, doc generators, CLI help).
const ( KindText FieldKind = iota // string KindBool // bool or *bool KindSelect // constrained string (options set by consumer) KindInt // int, int64 KindStringSlice // []string KindDuration // time.Duration KindMap // map[string]string (only — other map types must register via KindFunc) KindStructSlice // []struct (non-string slice of structs) // KindLast is the boundary for storage-defined kinds. Consumer packages // define domain-specific kinds starting here: // // const KindMyType storage.FieldKind = storage.KindLast + 1 KindLast )
type FieldSet ¶ added in v0.6.0
type FieldSet interface {
All() []Field // All fields in discovery order.
Get(path string) Field // Lookup by dotted path; returns nil if not found.
Group(prefix string) []Field // Fields whose path starts with prefix + ".".
Len() int // Number of fields.
}
FieldSet is an ordered, indexed collection of Field values.
func NewFieldSet ¶ added in v0.6.0
NewFieldSet creates a FieldSet from a slice of Field values. The slice order is preserved by [FieldSet.All].
func NormalizeFields ¶ added in v0.6.0
func NormalizeFields[T any](v T, opts ...NormalizeOption) FieldSet
NormalizeFields reflects over v's exported struct fields and produces a FieldSet containing schema metadata derived from struct tags:
- `yaml:"name"` — dotted path key (falls back to lowercased field name)
- `label:"Display Name"` — human-readable label (falls back to YAML key)
- `desc:"Help text"` — field description
- `default:"value"` — default value hint
- `required:"true"` — marks load-bearing fields that must have a value
Go type → FieldKind mapping:
- string → KindText
- bool, *bool → KindBool
- int, int64 → KindInt
- []string → KindStringSlice
- time.Duration → KindDuration
- struct, *struct → recursed (not a leaf field)
- map[string]string → KindMap
- []struct → KindStructSlice
- unrecognized types → KindFunc (if registered) → panic
NormalizeFields does NOT extract runtime values — only schema metadata. Panics if v is not a struct or pointer-to-struct, or if any exported field has an unsupported type and no KindFunc claims it.
type KindFunc ¶ added in v0.6.0
KindFunc classifies a reflect.Type that the normalizer does not recognize. Return (kind, true) to claim the type. Return (0, false) to fall through to the default panic.
type LayerInfo ¶
type LayerInfo struct {
Filename string // which filename matched (e.g., "clawker.yaml")
Path string // resolved absolute path
Data map[string]any // raw YAML data from this file only (read-only copy)
}
LayerInfo describes a discovered configuration layer.
type Migration ¶
Migration is a caller-provided function that inspects a raw YAML map and optionally transforms it. Returns true if the map was modified (triggers an atomic re-save of the source file).
type NormalizeOption ¶ added in v0.6.0
type NormalizeOption func(*normalizeOpts)
NormalizeOption configures NormalizeFields behavior.
func WithKindFunc ¶ added in v0.6.0
func WithKindFunc(fn KindFunc) NormalizeOption
WithKindFunc registers a classifier for domain-specific types. When NormalizeFields encounters a Go type it does not recognize, it calls fn before panicking. This lets consumer packages define custom FieldKind values (starting at KindLast) without modifying the storage package.
type Option ¶
type Option func(*options)
Option configures store construction via NewStore.
func WithCacheDir ¶
func WithCacheDir() Option
WithCacheDir adds the resolved cache directory to the explicit path list. Resolution: CLAWKER_CACHE_DIR > XDG_CACHE_HOME > ~/.cache/clawker
func WithConfigDir ¶
func WithConfigDir() Option
WithConfigDir adds the resolved config directory to the explicit path list. Resolution: CLAWKER_CONFIG_DIR > XDG_CONFIG_HOME > ~/.config/clawker
func WithDataDir ¶
func WithDataDir() Option
WithDataDir adds the resolved data directory to the explicit path list. Resolution: CLAWKER_DATA_DIR > XDG_DATA_HOME > ~/.local/share/clawker
func WithDefaultFilename ¶ added in v0.6.0
WithDefaultFilename sets the filename used when writing to a directory with no existing file layers. Without this, filenames[0] is used, which may be a local override variant rather than the main config file.
func WithDefaults ¶
WithDefaults provides a YAML string as the lowest-priority base layer. The string is parsed and merged before any discovered files. The same constant can be used for scaffolding (clawker init) and defaults.
func WithDefaultsFromStruct ¶ added in v0.6.0
WithDefaultsFromStruct generates a defaults YAML string from the `default` struct tags of T and registers it as the lowest-priority base layer. This is equivalent to WithDefaults(GenerateDefaultsYAML[T]()).
func WithDirs ¶ added in v0.2.4
WithDirs adds directories to be probed with dual placement discovery. Each directory uses the same dual-placement logic as walk-up: if a .clawker/ subdirectory exists, it probes .clawker/{filename} (dir form); otherwise it probes .{filename} (flat dotfile form). Both .yaml and .yml extensions are accepted. No registry required. Directories are probed in the order given (first = highest priority). Priority: walk-up > dirs > explicit paths (WithPaths/WithConfigDir/etc.).
func WithDotDefault ¶ added in v0.6.0
func WithDotDefault() Option
WithDotDefault enables dual-placement dot-prefix logic in defaultWritePath. When the store has no file layers and falls back to CWD, the filename is written as .{filename} (or .clawker/{filename} if .clawker/ exists) instead of the raw filename. Use this for stores discovered via walk-up where files are dot-prefixed by convention.
func WithFilenames ¶
WithFilenames sets the ordered list of filenames to discover. All filenames must share the same schema type T. At each walk-up level the first filename in the list takes merge precedence when discovered at the same depth.
func WithLock ¶
func WithLock() Option
WithLock enables flock-based advisory locking for Write operations. Use for stores that need cross-process mutual exclusion (e.g. registry).
func WithMigrations ¶
WithMigrations registers precondition-based migration functions. Each migration runs independently on every discovered file's raw map. Migrations that return true trigger an atomic re-save of that file.
func WithPaths ¶
WithPaths adds explicit directories to the discovery path list. Files are probed as {dir}/{filename} for each configured filename.
func WithStateDir ¶
func WithStateDir() Option
WithStateDir adds the resolved state directory to the explicit path list. Resolution: CLAWKER_STATE_DIR > XDG_STATE_HOME > ~/.local/state/clawker
func WithWalkUp ¶
func WithWalkUp() Option
WithWalkUp enables bounded walk-up discovery from CWD to the registered project root. The store resolves both CWD and project root internally: CWD via os.Getwd(), project root by reading the registry at dataDir(). At each level the store checks for .clawker/{filename} (dir form) first, then .{filename} (flat dotfile form). Walk-up never proceeds past the project root. If CWD is not within a registered project, walk-up is skipped and discovery falls back to explicit paths only.
type Schema ¶ added in v0.6.0
type Schema interface {
Fields() FieldSet
}
Schema is the contract that configuration types implement to expose their field metadata. Store is constrained to Schema, making field descriptions a compile-time requirement for all stored types.
type Store ¶
type Store[T Schema] struct { // contains filtered or unexported fields }
func NewFromString ¶
NewFromString constructs a store with an explicit YAML string as the virtual layer, merged on top of defaults. File discovery, migrations, and all other options work normally.
The virtual layer (defaults + raw string) is the lowest-priority data source. Discovered file layers override it. Fields that remain from the virtual layer (not overridden by files) are marked dirty since they have never been persisted.
With no options, the store has no file discovery — useful for seeding a new config that will be written via Write(ToPath(...)).
func (*Store[T]) Delete ¶ added in v0.6.0
Delete removes a dotted field path from the node tree (e.g. "agent.editor") and republishes the snapshot. This allows a field to be "unset" so that lower-priority layers can show through on next load.
Empty parent maps are NOT pruned — the tree retains the structure. Returns true if the key was found and deleted, false if it wasn't in the tree.
func (*Store[T]) Layers ¶
Layers returns information about the discovered configuration layers. Layers are ordered from highest priority (index 0) to lowest. No lock needed — layers are immutable after construction.
func (*Store[T]) MarkForWrite ¶ added in v0.6.0
MarkForWrite adds a dotted field path to the write set so the next Write includes it regardless of whether Set detected a change.
Use this when persisting a value to a specific layer file where the merged result is already identical (e.g. writing the current winning value to a lower-priority layer). Normal Set-based dirty tracking won't catch this because the merged tree didn't change.
func (*Store[T]) Provenance ¶ added in v0.6.0
Provenance returns the layer that provided the winning value for the given dotted field path (e.g. "build.image", "security.firewall.enable"). Returns the LayerInfo and true if provenance is known, or zero value and false for fields that came from defaults or have no provenance record.
No lock needed — provenance is immutable after construction.
func (*Store[T]) ProvenanceMap ¶ added in v0.6.0
ProvenanceMap returns a mapping of dotted field paths to their source layer paths. Virtual layer fields (defaults) have an empty path.
No lock needed — provenance is immutable after construction.
func (*Store[T]) Read ¶ added in v0.3.2
func (s *Store[T]) Read() *T
Read returns the current immutable snapshot. The returned pointer is safe to hold, inspect, and pass around — it will never be mutated by the store. Set publishes new snapshots via atomic swap; existing readers are unaffected.
Lock-free: uses atomic.Pointer.Load.
func (*Store[T]) Refresh ¶ added in v0.6.0
Refresh re-discovers layer files, re-reads them from disk, and re-merges into a fresh snapshot. This picks up external modifications to existing files and newly created files found via discovery that weren't written by this store.
Note: Write() already remerges and injects new layers for files it writes, so Refresh is only needed for external changes (e.g. another process modified a config file).
func (*Store[T]) Set ¶
Set applies a mutation function to a deep copy of the current value, syncs the change into the node tree, and atomically publishes the new snapshot. Changes are not persisted until Write is called.
The copy-on-write approach means existing Read callers holding the old snapshot are unaffected — they see consistent (stale) data.
After fn runs, the mutated copy is serialized back into the tree using structToMap (which ignores omitempty tags). This ensures that explicit zero-value assignments (e.g. setting a bool to false) are captured in the tree for persistence.
func (*Store[T]) Write ¶
func (s *Store[T]) Write(opts ...WriteOption) error
Write persists dirty fields to disk, then refreshes layer data from the written files so that subsequent Layers() calls return current values.
Only fields mutated since the last Write (via Set or Delete) are written. Set fields are merged into the target file; deleted fields are removed from it. This ensures per-field precision in multi-layer configurations.
Without options, each dirty field is routed to the layer it originated from (via provenance). Fields without provenance route to the highest-priority layer.
With ToPath, all dirty fields are written to the given absolute path. With ToLayer, all dirty fields are written to the specified layer.
Write sequence per target: read existing file → merge set fields → remove deleted fields → atomic write (temp+rename). If locking is enabled (WithLock), each file write is wrapped in a cross-process flock.
After a successful write, dirty tracking is cleared and layer data is refreshed from disk.
type WriteOption ¶ added in v0.6.0
type WriteOption struct {
// contains filtered or unexported fields
}
WriteOption configures how Write persists data.
func ToLayer ¶ added in v0.6.0
func ToLayer(idx int) WriteOption
ToLayer targets Write to a specific discovered layer by index. Layer indices correspond to Layers() ordering (0 = highest priority).
func ToPath ¶ added in v0.6.0
func ToPath(path string) WriteOption
ToPath targets Write to an explicit absolute filesystem path. Use this when writing to a new file or a known path outside the discovered layer set.
Source Files