dd

package
v0.3.12 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 18, 2026 License: MIT Imports: 11 Imported by: 24

README

dd - Dynamic Data

Convert between Go structs and maps with ease

The dd package provides bidirectional data binding between Go structs and map[string]any, enabling dynamic data handling for configuration, persistence, and API marshaling. Since maps are a foundational data structure, this facilitates seamless integration with any network protocol, object store, database, or file format that works with key-value data.

Quick Start

import "github.com/michaelquigley/df/dd"

// struct → map
user := User{Name: "John", Age: 30}
data, _ := dd.Unbind(user)
// data: map[string]any{"name": "John", "age": 30}

// map → struct  
userData := map[string]any{"name": "Alice", "age": 25}
user, _ := dd.New[User](userData)
// user: User{Name: "Alice", Age: 25}

Key Features

  • 🔄 Bidirectional Binding: Seamlessly convert structs ↔ maps
  • 🏷️ Struct Tags: Control field mapping with df tags
  • ⚡ Type Coercion: Automatic type conversion (strings→numbers, etc.)
  • 🗺️ Typed Maps: Full support for map[K]V with any comparable key type
  • 📁 File I/O: Direct JSON/YAML binding with BindFromJSON(), UnbindToYAML()
  • 🔗 Object References: Pointer[T] type with cycle-safe linking
  • 🎭 Dynamic Types: Runtime type discrimination via Dynamic interface
  • ✅ Validation: Required fields and custom validation rules

Core Functions

  • dd.New[T](data) - Type-safe struct creation from map
  • dd.Bind(target, data) - Bind data to existing struct
  • dd.Unbind(struct) - Convert struct to map
  • dd.Merge(base, override) - Deep merge two data maps

Common Patterns

Struct Tags for Control

type User struct {
    Name  string `dd:"+required"`           // required field
    Email string `dd:"email_address"`       // custom field name
    Token string `dd:"-"`                   // excluded from binding
    Age   int    `dd:",default=18"`         // default value
}

File Persistence

// Load config from JSON
config, _ := dd.BindFromJSON[AppConfig]("config.json")

// Save to YAML
dd.UnbindToYAML(config, "config.yaml")

Dynamic Types

// Handle different object types at runtime
data := map[string]any{
    "type": "user",
    "name": "John",
}
obj, _ := dd.New[dd.Dynamic](data)  // Creates appropriate type

Typed Maps

// Maps with typed keys and values
type ServerConfig struct {
    Servers map[int]Server  // int keys from JSON strings
    Cache   map[string]CacheConfig
}

// JSON: {"servers": {"1": {...}, "2": {...}}}
config, _ := dd.New[ServerConfig](data)
server := config.Servers[1]  // Direct typed access

Examples

See examples/ for progressive tutorials from basic binding to advanced object references and dynamic types.

Part of the df framework - dynamic foundation for Go applications

Documentation

Index

Constants

View Source 
const (
	TypeKey = "type" // discriminator key for Dynamic types
	RefKey  = "$ref" // reference key for Pointer types
)

Magic string constants for special keys to avoid typos

Variables

This section is empty.

Functions

func Bind 

func Bind(target interface{}, data map[string]any, opts ...*Options) error

Bind populates the exported fields of target (a pointer to a struct) from the given data map. Keys are matched using either a struct tag `dd:"name,+required"` (where name overrides the key and the optional "+required" flag enforces presence), `dd:"-"` to skip a field, or, when no tag is provided, a best-effort snake_case conversion of the field name.

Use Bind when you need to control how the prototype object is allocated. Use New when you just want to allocate a new object to bind off the heap.

supported kinds: - primitives: string, bool, all int/uint sizes, float32/64, time.Duration, time.Time (from RFC3339 strings) - pointers to the above - structs and pointers to structs (recursively bound from map[string]any) - slices of the above (slice items are bound from []interface{}) - maps with comparable key types and any supported value type (map keys from JSON/YAML are coerced from strings)

interface types are not supported and will return an error if encountered, except for fields of type Dynamic which are resolved using Options.DynamicBinders.

opts are optional; pass nil or omit to use defaults.

func BindJSON added in v0.3.6 

func BindJSON(target interface{}, data []byte, opts ...*Options) error

BindJSON parses JSON data and binds it to the target struct.

func BindJSONFile added in v0.3.6 

func BindJSONFile(target interface{}, path string, opts ...*Options) error

BindJSONFile reads JSON from the specified file path and binds it to the target struct.

func BindJSONReader added in v0.3.6 

func BindJSONReader(target interface{}, r io.Reader, opts ...*Options) error

BindJSONReader reads JSON from an io.Reader and binds it to the target struct.

func BindYAML added in v0.3.6 

func BindYAML(target interface{}, data []byte, opts ...*Options) error

BindYAML parses YAML data and binds it to the target struct.

func BindYAMLFile added in v0.3.6 

func BindYAMLFile(target interface{}, path string, opts ...*Options) error

BindYAMLFile reads YAML from the specified file path and binds it to the target struct.

func BindYAMLReader added in v0.3.6 

func BindYAMLReader(target interface{}, r io.Reader, opts ...*Options) error

BindYAMLReader reads YAML from an io.Reader and binds it to the target struct.

func Inspect 

func Inspect(source interface{}, opts ...*InspectOptions) (string, error)

Inspect returns a human-readable representation of a struct's resolved state. designed for configuration debugging and validation. secret fields marked with `dd:",+secret"` are hidden unless ShowSecrets is true.

the output format is a clean, indented pseudo-data structure optimized for readability rather than parseability.

supported types: - primitives: string, bool, all int/uint sizes, float32/64, time.Duration - pointers to the above (nil pointers shown as "<nil>") - structs and pointers to structs (recursively inspected) - slices of the above (shown as numbered lists) - Dynamic interface implementations (shown with their type) - Pointer[T] references (shown with resolved state)

opts are optional; pass nil or omit to use defaults. 

func Link(targets ...interface{}) error

Link resolves all pointer references in the target objects by building a registry of all Identifiable objects and then resolving Pointer fields to their target objects. objects are namespaced by their concrete type to prevent Id clashes between different types.

func Merge 

func Merge(target interface{}, data map[string]any, opts ...*Options) error

Merge populates the exported fields of an existing target struct from the given data map, preserving any existing field values that are not present in the data. This allows binding partial data to pre-initialized structs with default values.

uses the same field mapping rules as Bind: struct tags, snake_case conversion, etc.

supported kinds are the same as Bind.

opts are optional; pass nil or omit to use defaults.

func MergeJSON added in v0.3.6 

func MergeJSON(target interface{}, data []byte, opts ...*Options) error

MergeJSON parses JSON data and merges it with the target struct.

func MergeJSONFile added in v0.3.6 

func MergeJSONFile(target interface{}, path string, opts ...*Options) error

MergeJSONFile reads JSON from the specified file path and merges it with the target struct.

func MergeJSONReader added in v0.3.6 

func MergeJSONReader(target interface{}, r io.Reader, opts ...*Options) error

MergeJSONReader reads JSON from an io.Reader and merges it with the target struct.

func MergeYAML added in v0.3.6 

func MergeYAML(target interface{}, data []byte, opts ...*Options) error

MergeYAML parses YAML data and merges it with the target struct.

func MergeYAMLFile added in v0.3.6 

func MergeYAMLFile(target interface{}, path string, opts ...*Options) error

MergeYAMLFile reads YAML from the specified file path and merges it with the target struct.

func MergeYAMLReader added in v0.3.6 

func MergeYAMLReader(target interface{}, r io.Reader, opts ...*Options) error

MergeYAMLReader reads YAML from an io.Reader and merges it with the target struct.

func MustInspect 

func MustInspect(source interface{}, opts ...*InspectOptions) string

MustInspect returns a human-readable representation of a struct's resolved state, panicking if an error occurs. see Inspect for full documentation.

func New 

func New[T any](data map[string]any, opts ...*Options) (*T, error)

New creates and populates a new instance of type T from the given data map. Unlike Bind, which requires a pre-allocated target pointer, New automatically allocates the object and returns a pointer to the populated struct.

Use Bind instead of New when you need to control where and how the target object is instantiated. New just allocates a fresh target off the heap.

Example usage: 

type Person struct {
    Name string
    Age  int
}

data := map[string]any{"name": "John", "age": 30}
person, err := New[Person](data)
if err != nil {
    // handle error
}
// person is now *Person with Name="John" and Age=30

supported kinds and field mapping rules are the same as Bind.

opts are optional; pass nil or omit to use defaults.

func NewJSON added in v0.3.6 

func NewJSON[T any](data []byte, opts ...*Options) (*T, error)

NewJSON parses JSON data and returns a new instance of type T.

func NewJSONFile added in v0.3.6 

func NewJSONFile[T any](path string, opts ...*Options) (*T, error)

NewJSONFile reads JSON from the specified file path and returns a new instance of type T.

func NewJSONReader added in v0.3.6 

func NewJSONReader[T any](r io.Reader, opts ...*Options) (*T, error)

NewJSONReader reads JSON from an io.Reader and returns a new instance of type T.

func NewYAML added in v0.3.6 

func NewYAML[T any](data []byte, opts ...*Options) (*T, error)

NewYAML parses YAML data and returns a new instance of type T.

func NewYAMLFile added in v0.3.6 

func NewYAMLFile[T any](path string, opts ...*Options) (*T, error)

NewYAMLFile reads YAML from the specified file path and returns a new instance of type T.

func NewYAMLReader added in v0.3.6 

func NewYAMLReader[T any](r io.Reader, opts ...*Options) (*T, error)

NewYAMLReader reads YAML from an io.Reader and returns a new instance of type T.

func Unbind 

func Unbind(source interface{}, opts ...*Options) (map[string]any, error)

Unbind converts a struct (or pointer to struct) into a map[string]any honoring the same `dd` tags used by Bind: - `dd:"name"` overrides the key name - `dd:"-"` skips the field - `dd:",+omitempty"` omits the field if it has a zero value - when no tag is provided, the key defaults to snake_case of the field name

pointers to values: if nil, the key is omitted; otherwise the pointed value is emitted. slices, structs, maps, and nested pointers are handled recursively. time.Duration values are emitted as strings using Duration.String() (e.g., "30s"). time.Time values are emitted as RFC3339 strings (e.g., "2024-03-15T14:30:45Z"). map keys are converted to strings for JSON/YAML compatibility. Interface fields are not supported, except for fields of type `Dynamic` (and slices of `Dynamic`), which are converted via their ToMap() method which now returns (map[string]any, error).

opts are optional; pass nil or omit to use defaults.

func UnbindJSON added in v0.3.6 

func UnbindJSON(source interface{}, opts ...*Options) ([]byte, error)

UnbindJSON converts a struct to JSON bytes.

func UnbindJSONFile added in v0.3.6 

func UnbindJSONFile(source interface{}, path string, opts ...*Options) error

UnbindJSONFile converts a struct to JSON and writes it to the specified file path.

func UnbindJSONWriter added in v0.3.6 

func UnbindJSONWriter(source interface{}, w io.Writer, opts ...*Options) error

UnbindJSONWriter converts a struct to JSON and writes it to an io.Writer.

func UnbindYAML added in v0.3.6 

func UnbindYAML(source interface{}, opts ...*Options) ([]byte, error)

UnbindYAML converts a struct to YAML bytes.

func UnbindYAMLFile added in v0.3.6 

func UnbindYAMLFile(source interface{}, path string, opts ...*Options) error

UnbindYAMLFile converts a struct to YAML and writes it to the specified file path.

func UnbindYAMLWriter added in v0.3.6 

func UnbindYAMLWriter(source interface{}, w io.Writer, opts ...*Options) error

UnbindYAMLWriter converts a struct to YAML and writes it to an io.Writer.

Types

type BindingError 

type BindingError struct {
	Path  string
	Field string
	Key   string
	Cause error
}

BindingError represents struct field binding errors

func (*BindingError) Error 

func (e *BindingError) Error() string

func (*BindingError) Unwrap 

func (e *BindingError) Unwrap() error

type ConversionError 

type ConversionError struct {
	Path    string
	Value   string
	Type    string
	Message string
	Cause   error
}

ConversionError represents data conversion failures

func (*ConversionError) Error 

func (e *ConversionError) Error() string

func (*ConversionError) Unwrap 

func (e *ConversionError) Unwrap() error

type Converter 

type Converter interface {
	// FromRaw converts a raw value (from the data map) to the target type.
	// the input can be any type that appears in the data map (string, int, bool, etc.).
	FromRaw(raw interface{}) (interface{}, error)

	// ToRaw converts a typed value back to a raw value for serialization.
	// the output should be a type that can be marshaled (string, int, bool, etc.).
	ToRaw(value interface{}) (interface{}, error)
}

Converter defines a bidirectional type conversion interface for custom field types. it allows users to define how their custom types should be converted to/from the raw data.

type DdTag 

type DdTag struct {
	Name       string // external field name override, empty means use default
	Required   bool   // true if field is required during binding
	Secret     bool   // true if field contains sensitive data
	Skip       bool   // true if field should be skipped entirely
	MatchValue string // expected value that must match during binding, empty means no constraint
	HasMatch   bool   // true if a match constraint is specified
	Extra      bool   // true if field should capture unmatched keys
	OmitEmpty  bool   // true if field should be omitted when zero during unbinding
}

DdTag holds the parsed values from a `dd` struct tag.

type Dynamic 

type Dynamic interface {
	Type() string
	ToMap() (map[string]any, error)
}

Dynamic fields can be used when the concrete type of a field is selected dynamically through the `type` data provided in the incoming `map` that will be passed to `Bind`. A polymorphic field type.

type FileError 

type FileError struct {
	Path      string
	Operation string
	Cause     error
}

FileError represents file I/O operation errors

func (*FileError) Error 

func (e *FileError) Error() string

func (*FileError) IsNotFound 

func (e *FileError) IsNotFound() bool

IsNotFound checks if the FileError represents a file not found error.

func (*FileError) Unwrap 

func (e *FileError) Unwrap() error

type Identifiable 

type Identifiable interface {
	GetId() string
}

Identifiable objects can participate in pointer references by providing a unique Id.

type IndexError 

type IndexError struct {
	Index int
	Cause error
}

IndexError represents errors with array/slice indexing

func (*IndexError) Error 

func (e *IndexError) Error() string

func (*IndexError) Unwrap 

func (e *IndexError) Unwrap() error

type InspectOptions 

type InspectOptions struct {
	// MaxDepth limits recursion depth to prevent infinite loops.
	MaxDepth int
	// Indent sets the indentation string (defaults to "  ").
	Indent string
	// ShowSecrets includes secret fields in output when true.
	ShowSecrets bool
}

InspectOptions configures inspection behavior.

type Linker 

type Linker struct {
	// contains filtered or unexported fields
}

Linker encapsulates the linking process, providing enhanced state management and advanced features.

func NewLinker 

func NewLinker(opts ...LinkerOptions) *Linker

NewLinker creates a new Linker with optional options. If no options are provided, default options are used.

func (*Linker) ClearCache 

func (l *Linker) ClearCache()

ClearCache clears the internal registry cache if caching is enabled. 

func (l *Linker) Link(targets ...interface{}) error

Link resolves all pointer references in the target objects by building a registry of all Identifiable objects and then resolving Pointer fields to their target objects. objects are namespaced by their concrete type to prevent Id clashes between different types.

func (*Linker) Register 

func (l *Linker) Register(targets ...interface{}) error

Register performs phase 1 of linking: collecting all Identifiable objects. This can be used for multi-stage linking where you want to register objects from multiple sources before resolving references.

func (*Linker) ResolveReferences 

func (l *Linker) ResolveReferences(target interface{}) error

ResolveReferences performs phase 2 of linking: resolving all pointer references using the collected registry. This can be used after collecting from multiple sources.

type LinkerOptions 

type LinkerOptions struct {
	// EnableCaching enables registry caching for repeated linking operations
	EnableCaching bool
	// AllowPartialResolution allows linking to succeed even if some references can't be resolved
	AllowPartialResolution bool
}

LinkerOptions configures the behavior of a Linker instance.

type Marshaler 

type Marshaler interface {
	MarshalDd() (map[string]any, error)
}

Marshaler allows a type to define its own marshalling logic to a map[string]any.

type MultipleExtraFieldsError added in v0.3.8 

type MultipleExtraFieldsError struct {
	Path string
}

MultipleExtraFieldsError represents the error when a struct has more than one +extra field

func (*MultipleExtraFieldsError) Error added in v0.3.8 

func (e *MultipleExtraFieldsError) Error() string

type Options 

type Options struct {
	// DynamicBinders maps a discriminator string (found under the "type" key in the input map) to a function that
	// consumes the full map and returns a concrete value implementing the Dynamic interface.
	DynamicBinders map[string]func(map[string]any) (Dynamic, error)

	// FieldDynamicBinders allows specifying binder sets per field path. The key is the structured path of the field as
	// used internally by Bind, e.g.: "Root.Items" for a slice field, "Root.Nested.Field" for nested fields.
	// any array indices in the path are ignored for matching purposes.
	// when present for a field, this map takes precedence over DynamicBinders.
	FieldDynamicBinders map[string]map[string]func(map[string]any) (Dynamic, error)

	// Converters maps Go types to custom converters for type conversion.
	// the key is the reflect.Type of the target field, and the value is a Converter
	// that handles bidirectional conversion between raw data and the target type.
	Converters map[reflect.Type]Converter
}

Options configures binding behavior.

type Pointer 

type Pointer[T Identifiable] struct {
	Ref      string `dd:"$ref"`
	Resolved T      // internal resolved reference (exported for reflection)
}

Pointer represents a reference to an object of type T that implements Identifiable. During binding, the reference is stored as a string. During linking, it's resolved to the actual object.

func (*Pointer[T]) IsResolved 

func (p *Pointer[T]) IsResolved() bool

IsResolved returns true if the pointer has been resolved to an actual object.

func (*Pointer[T]) Resolve 

func (p *Pointer[T]) Resolve() T

Resolve returns the resolved object, or the zero value of T if not yet resolved.

type PointerError 

type PointerError struct {
	Path      string
	Reference string
	Message   string
	Cause     error
}

PointerError represents pointer resolution errors

func (*PointerError) Error 

func (e *PointerError) Error() string

func (*PointerError) Unwrap 

func (e *PointerError) Unwrap() error

type RequiredFieldError 

type RequiredFieldError struct {
	Path  string
	Field string
}

RequiredFieldError represents missing required field errors

func (*RequiredFieldError) Error 

func (e *RequiredFieldError) Error() string

type TypeMismatchError 

type TypeMismatchError struct {
	Path     string
	Expected string
	Actual   string
}

TypeMismatchError represents type conversion errors

func (*TypeMismatchError) Error 

func (e *TypeMismatchError) Error() string

type UnbindingError 

type UnbindingError struct {
	Path  string
	Field string
	Key   string
	Cause error
}

UnbindingError represents struct field unbinding errors

func (*UnbindingError) Error 

func (e *UnbindingError) Error() string

func (*UnbindingError) Unwrap 

func (e *UnbindingError) Unwrap() error

type Unmarshaler 

type Unmarshaler interface {
	UnmarshalDd(data map[string]any) error
}

Unmarshaler allows a type to define its own unmarshalling logic from a map[string]any.

type UnsupportedError 

type UnsupportedError struct {
	Path      string
	Operation string
	Type      string
}

UnsupportedError represents unsupported operation errors

func (*UnsupportedError) Error 

func (e *UnsupportedError) Error() string

type ValidationError 

type ValidationError struct {
	Field   string
	Message string
}

ValidationError represents errors in input validation

func (*ValidationError) Error 

func (e *ValidationError) Error() string

type ValueMismatchError 

type ValueMismatchError struct {
	Path     string
	Field    string
	Expected string
	Actual   string
}

ValueMismatchError represents errors when a field value doesn't match the expected constraint

func (*ValueMismatchError) Error 

func (e *ValueMismatchError) Error() string

Source Files

View all Source files

Directories

Path Synopsis
examples
dd_01_hello_world command
dd_02_struct_tags command
dd_03_type_coercion command
dd_04_io command
dd_05_nested_structs command
dd_06_validation command
dd_07_converters command
dd_08_marshalers command
dd_09_dynamic_types command
dd_10_field_binders command
dd_11_pointers command
dd_12_advanced_linking command
dd_13_typed_maps command
dd_14_extra_fields command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.