dd

package
v0.3.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 8, 2025 License: MIT Imports: 10 Imported by: 24

README

dd - Dynamic Data

Convert between Go structs and maps with ease

The dd package provides bidirectional data binding between Go structs and map[string]any, enabling dynamic data handling for configuration, persistence, and API marshaling. Since maps are a foundational data structure, this facilitates seamless integration with any network protocol, object store, database, or file format that works with key-value data.

Quick Start

import "github.com/michaelquigley/df/dd"

// struct → map
user := User{Name: "John", Age: 30}
data, _ := dd.Unbind(user)
// data: map[string]any{"name": "John", "age": 30}

// map → struct  
userData := map[string]any{"name": "Alice", "age": 25}
user, _ := dd.New[User](userData)
// user: User{Name: "Alice", Age: 25}

Key Features

  • 🔄 Bidirectional Binding: Seamlessly convert structs ↔ maps
  • 🏷️ Struct Tags: Control field mapping with df tags
  • ⚡ Type Coercion: Automatic type conversion (strings→numbers, etc.)
  • 🗺️ Typed Maps: Full support for map[K]V with any comparable key type
  • 📁 File I/O: Direct JSON/YAML binding with BindFromJSON(), UnbindToYAML()
  • 🔗 Object References: Pointer[T] type with cycle-safe linking
  • 🎭 Dynamic Types: Runtime type discrimination via Dynamic interface
  • ✅ Validation: Required fields and custom validation rules

Core Functions

  • dd.New[T](data) - Type-safe struct creation from map
  • dd.Bind(target, data) - Bind data to existing struct
  • dd.Unbind(struct) - Convert struct to map
  • dd.Merge(base, override) - Deep merge two data maps

Common Patterns

Struct Tags for Control

type User struct {
    Name  string `dd:"+required"`           // required field
    Email string `dd:"email_address"`       // custom field name
    Token string `dd:"-"`                   // excluded from binding
    Age   int    `dd:",default=18"`         // default value
}

File Persistence

// Load config from JSON
config, _ := dd.BindFromJSON[AppConfig]("config.json")

// Save to YAML
dd.UnbindToYAML(config, "config.yaml")

Dynamic Types

// Handle different object types at runtime
data := map[string]any{
    "type": "user",
    "name": "John",
}
obj, _ := dd.New[dd.Dynamic](data)  // Creates appropriate type

Typed Maps

// Maps with typed keys and values
type ServerConfig struct {
    Servers map[int]Server  // int keys from JSON strings
    Cache   map[string]CacheConfig
}

// JSON: {"servers": {"1": {...}, "2": {...}}}
config, _ := dd.New[ServerConfig](data)
server := config.Servers[1]  // Direct typed access

Examples

See examples/ for progressive tutorials from basic binding to advanced object references and dynamic types.

Part of the df framework - dynamic foundation for Go applications

Documentation

Index

Constants

View Source 
const (
	TypeKey = "type" // discriminator key for Dynamic types
	RefKey  = "$ref" // reference key for Pointer types
)

Magic string constants for special keys to avoid typos

Variables

This section is empty.

Functions

func Bind 

func Bind(target interface{}, data map[string]any, opts ...*Options) error

Bind populates the exported fields of target (a pointer to a struct) from the given data map. Keys are matched using either a struct tag `dd:"name,+required"` (where name overrides the key and the optional "+required" flag enforces presence), `dd:"-"` to skip a field, or, when no tag is provided, a best-effort snake_case conversion of the field name.

Use Bind when you need to control how the prototype object is allocated. Use New when you just want to allocate a new object to bind off the heap.

supported kinds: - primitives: string, bool, all int/uint sizes, float32/64, time.Duration, time.Time (from RFC3339 strings) - pointers to the above - structs and pointers to structs (recursively bound from map[string]any) - slices of the above (slice items are bound from []interface{}) - maps with comparable key types and any supported value type (map keys from JSON/YAML are coerced from strings)

interface types are not supported and will return an error if encountered, except for fields of type Dynamic which are resolved using Options.DynamicBinders.

opts are optional; pass nil or omit to use defaults.

func BindFromJSON 

func BindFromJSON(target interface{}, path string, opts ...*Options) error

BindFromJSON reads JSON from the specified file path and binds it to the target struct.

func BindFromYAML 

func BindFromYAML(target interface{}, path string, opts ...*Options) error

BindFromYAML reads YAML from the specified file path and binds it to the target struct.

func Inspect 

func Inspect(source interface{}, opts ...*InspectOptions) (string, error)

Inspect returns a human-readable representation of a struct's resolved state. designed for configuration debugging and validation. secret fields marked with `dd:",+secret"` are hidden unless ShowSecrets is true.

the output format is a clean, indented pseudo-data structure optimized for readability rather than parseability.

supported types: - primitives: string, bool, all int/uint sizes, float32/64, time.Duration - pointers to the above (nil pointers shown as "<nil>") - structs and pointers to structs (recursively inspected) - slices of the above (shown as numbered lists) - Dynamic interface implementations (shown with their type) - Pointer[T] references (shown with resolved state)

opts are optional; pass nil or omit to use defaults. 

func Link(targets ...interface{}) error

Link resolves all pointer references in the target objects by building a registry of all Identifiable objects and then resolving Pointer fields to their target objects. objects are namespaced by their concrete type to prevent Id clashes between different types.

func Merge 

func Merge(target interface{}, data map[string]any, opts ...*Options) error

Merge populates the exported fields of an existing target struct from the given data map, preserving any existing field values that are not present in the data. This allows binding partial data to pre-initialized structs with default values.

uses the same field mapping rules as Bind: struct tags, snake_case conversion, etc.

supported kinds are the same as Bind.

opts are optional; pass nil or omit to use defaults.

func MergeFromJSON 

func MergeFromJSON(target interface{}, path string, opts ...*Options) error

MergeFromJSON reads JSON from the specified file path and merges it with the target struct.

func MergeFromYAML 

func MergeFromYAML(target interface{}, path string, opts ...*Options) error

MergeFromYAML reads YAML from the specified file path and merges it with the target struct.

func MustInspect 

func MustInspect(source interface{}, opts ...*InspectOptions) string

MustInspect returns a human-readable representation of a struct's resolved state, panicking if an error occurs. see Inspect for full documentation.

func New 

func New[T any](data map[string]any, opts ...*Options) (*T, error)

New creates and populates a new instance of type T from the given data map. Unlike Bind, which requires a pre-allocated target pointer, New automatically allocates the object and returns a pointer to the populated struct.

Use Bind instead of New when you need to control where and how the target object is instantiated. New just allocates a fresh target off the heap.

Example usage: 

type Person struct {
    Name string
    Age  int
}

data := map[string]any{"name": "John", "age": 30}
person, err := New[Person](data)
if err != nil {
    // handle error
}
// person is now *Person with Name="John" and Age=30

supported kinds and field mapping rules are the same as Bind.

opts are optional; pass nil or omit to use defaults.

func NewFromJSON 

func NewFromJSON[T any](path string, opts ...*Options) (*T, error)

NewFromJSON reads JSON from the specified file path and returns a new instance of type T.

func NewFromYAML 

func NewFromYAML[T any](path string, opts ...*Options) (*T, error)

NewFromYAML reads YAML from the specified file path and returns a new instance of type T.

func Unbind 

func Unbind(source interface{}, opts ...*Options) (map[string]any, error)

Unbind converts a struct (or pointer to struct) into a map[string]any honoring the same `df` tags used by Bind: - `dd:"name"` overrides the key name - `dd:"-"` skips the field - when no tag is provided, the key defaults to snake_case of the field name

pointers to values: if nil, the key is omitted; otherwise the pointed value is emitted. slices, structs, maps, and nested pointers are handled recursively. time.Duration values are emitted as strings using Duration.String() (e.g., "30s"). time.Time values are emitted as RFC3339 strings (e.g., "2024-03-15T14:30:45Z"). map keys are converted to strings for JSON/YAML compatibility. Interface fields are not supported, except for fields of type `Dynamic` (and slices of `Dynamic`), which are converted via their ToMap() method which now returns (map[string]any, error).

opts are optional; pass nil or omit to use defaults.

func UnbindToJSON 

func UnbindToJSON(source interface{}, path string) error

UnbindToJSON converts a struct to map using Unbind, then writes it as JSON to the specified file path.

func UnbindToYAML 

func UnbindToYAML(source interface{}, path string) error

UnbindToYAML converts a struct to map using Unbind, then writes it as YAML to the specified file path.

Types

type BindingError 

type BindingError struct {
	Path  string
	Field string
	Key   string
	Cause error
}

BindingError represents struct field binding errors

func (*BindingError) Error 

func (e *BindingError) Error() string

func (*BindingError) Unwrap 

func (e *BindingError) Unwrap() error

type ConversionError 

type ConversionError struct {
	Path    string
	Value   string
	Type    string
	Message string
	Cause   error
}

ConversionError represents data conversion failures

func (*ConversionError) Error 

func (e *ConversionError) Error() string

func (*ConversionError) Unwrap 

func (e *ConversionError) Unwrap() error

type Converter 

type Converter interface {
	// FromRaw converts a raw value (from the data map) to the target type.
	// the input can be any type that appears in the data map (string, int, bool, etc.).
	FromRaw(raw interface{}) (interface{}, error)

	// ToRaw converts a typed value back to a raw value for serialization.
	// the output should be a type that can be marshaled (string, int, bool, etc.).
	ToRaw(value interface{}) (interface{}, error)
}

Converter defines a bidirectional type conversion interface for custom field types. it allows users to define how their custom types should be converted to/from the raw data.

type DdTag 

type DdTag struct {
	Name       string // external field name override, empty means use default
	Required   bool   // true if field is required during binding
	Secret     bool   // true if field contains sensitive data
	Skip       bool   // true if field should be skipped entirely
	MatchValue string // expected value that must match during binding, empty means no constraint
	HasMatch   bool   // true if a match constraint is specified
}

DdTag holds the parsed values from a `df` struct tag.

type Dynamic 

type Dynamic interface {
	Type() string
	ToMap() (map[string]any, error)
}

Dynamic fields can be used when the concrete type of a field is selected dynamically through the `type` data provided in the incoming `map` that will be passed to `Bind`. A polymorphic field type.

type FileError 

type FileError struct {
	Path      string
	Operation string
	Cause     error
}

FileError represents file I/O operation errors

func (*FileError) Error 

func (e *FileError) Error() string

func (*FileError) IsNotFound 

func (e *FileError) IsNotFound() bool

IsNotFound checks if the FileError represents a file not found error.

func (*FileError) Unwrap 

func (e *FileError) Unwrap() error

type Identifiable 

type Identifiable interface {
	GetId() string
}

Identifiable objects can participate in pointer references by providing a unique Id.

type IndexError 

type IndexError struct {
	Index int
	Cause error
}

IndexError represents errors with array/slice indexing

func (*IndexError) Error 

func (e *IndexError) Error() string

func (*IndexError) Unwrap 

func (e *IndexError) Unwrap() error

type InspectOptions 

type InspectOptions struct {
	// MaxDepth limits recursion depth to prevent infinite loops.
	MaxDepth int
	// Indent sets the indentation string (defaults to "  ").
	Indent string
	// ShowSecrets includes secret fields in output when true.
	ShowSecrets bool
}

InspectOptions configures inspection behavior.

type Linker 

type Linker struct {
	// contains filtered or unexported fields
}

Linker encapsulates the linking process, providing enhanced state management and advanced features.

func NewLinker 

func NewLinker(opts ...LinkerOptions) *Linker

NewLinker creates a new Linker with optional options. If no options are provided, default options are used.

func (*Linker) ClearCache 

func (l *Linker) ClearCache()

ClearCache clears the internal registry cache if caching is enabled. 

func (l *Linker) Link(targets ...interface{}) error

Link resolves all pointer references in the target objects by building a registry of all Identifiable objects and then resolving Pointer fields to their target objects. objects are namespaced by their concrete type to prevent Id clashes between different types.

func (*Linker) Register 

func (l *Linker) Register(targets ...interface{}) error

Register performs phase 1 of linking: collecting all Identifiable objects. This can be used for multi-stage linking where you want to register objects from multiple sources before resolving references.

func (*Linker) ResolveReferences 

func (l *Linker) ResolveReferences(target interface{}) error

ResolveReferences performs phase 2 of linking: resolving all pointer references using the collected registry. This can be used after collecting from multiple sources.

type LinkerOptions 

type LinkerOptions struct {
	// EnableCaching enables registry caching for repeated linking operations
	EnableCaching bool
	// AllowPartialResolution allows linking to succeed even if some references can't be resolved
	AllowPartialResolution bool
}

LinkerOptions configures the behavior of a Linker instance.

type Marshaler 

type Marshaler interface {
	MarshalDd() (map[string]any, error)
}

Marshaler allows a type to define its own marshalling logic to a map[string]any.

type Options 

type Options struct {
	// DynamicBinders maps a discriminator string (found under the "type" key in the input map) to a function that
	// consumes the full map and returns a concrete value implementing the Dynamic interface.
	DynamicBinders map[string]func(map[string]any) (Dynamic, error)

	// FieldDynamicBinders allows specifying binder sets per field path. The key is the structured path of the field as
	// used internally by Bind, e.g.: "Root.Items" for a slice field, "Root.Nested.Field" for nested fields.
	// any array indices in the path are ignored for matching purposes.
	// when present for a field, this map takes precedence over DynamicBinders.
	FieldDynamicBinders map[string]map[string]func(map[string]any) (Dynamic, error)

	// Converters maps Go types to custom converters for type conversion.
	// the key is the reflect.Type of the target field, and the value is a Converter
	// that handles bidirectional conversion between raw data and the target type.
	Converters map[reflect.Type]Converter
}

Options configures binding behavior.

type Pointer 

type Pointer[T Identifiable] struct {
	Ref      string `dd:"$ref"`
	Resolved T      // internal resolved reference (exported for reflection)
}

Pointer represents a reference to an object of type T that implements Identifiable. During binding, the reference is stored as a string. During linking, it's resolved to the actual object.

func (*Pointer[T]) IsResolved 

func (p *Pointer[T]) IsResolved() bool

IsResolved returns true if the pointer has been resolved to an actual object.

func (*Pointer[T]) Resolve 

func (p *Pointer[T]) Resolve() T

Resolve returns the resolved object, or the zero value of T if not yet resolved.

type PointerError 

type PointerError struct {
	Path      string
	Reference string
	Message   string
	Cause     error
}

PointerError represents pointer resolution errors

func (*PointerError) Error 

func (e *PointerError) Error() string

func (*PointerError) Unwrap 

func (e *PointerError) Unwrap() error

type RequiredFieldError 

type RequiredFieldError struct {
	Path  string
	Field string
}

RequiredFieldError represents missing required field errors

func (*RequiredFieldError) Error 

func (e *RequiredFieldError) Error() string

type TypeMismatchError 

type TypeMismatchError struct {
	Path     string
	Expected string
	Actual   string
}

TypeMismatchError represents type conversion errors

func (*TypeMismatchError) Error 

func (e *TypeMismatchError) Error() string

type UnbindingError 

type UnbindingError struct {
	Path  string
	Field string
	Key   string
	Cause error
}

UnbindingError represents struct field unbinding errors

func (*UnbindingError) Error 

func (e *UnbindingError) Error() string

func (*UnbindingError) Unwrap 

func (e *UnbindingError) Unwrap() error

type Unmarshaler 

type Unmarshaler interface {
	UnmarshalDd(data map[string]any) error
}

Unmarshaler allows a type to define its own unmarshalling logic from a map[string]any.

type UnsupportedError 

type UnsupportedError struct {
	Path      string
	Operation string
	Type      string
}

UnsupportedError represents unsupported operation errors

func (*UnsupportedError) Error 

func (e *UnsupportedError) Error() string

type ValidationError 

type ValidationError struct {
	Field   string
	Message string
}

ValidationError represents errors in input validation

func (*ValidationError) Error 

func (e *ValidationError) Error() string

type ValueMismatchError 

type ValueMismatchError struct {
	Path     string
	Field    string
	Expected string
	Actual   string
}

ValueMismatchError represents errors when a field value doesn't match the expected constraint

func (*ValueMismatchError) Error 

func (e *ValueMismatchError) Error() string

Source Files

View all Source files

Directories

Path Synopsis
examples
dd_01_hello_world command
dd_02_struct_tags command
dd_03_type_coercion command
dd_04_file_io command
dd_05_nested_structs command
dd_06_validation command
dd_07_converters command
dd_08_marshalers command
dd_09_dynamic_types command
dd_10_field_binders command
dd_11_pointers command
dd_12_advanced_linking command
dd_13_typed_maps command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.