dynamorm

package module
v1.0.37 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 11, 2025 License: Apache-2.0 Imports: 32 Imported by: 3

README

DynamORM: Type-Safe DynamoDB ORM for Go

DynamORM provides a type-safe, optimized way to interact with Amazon DynamoDB in Go applications. It offers significantly faster cold starts than raw AWS SDK and reduces boilerplate code.

Go Version License Go Report Card

Why DynamORM?

Use DynamORM when you need:

  • Type-safe DynamoDB operations - Compile-time error prevention
  • Lambda-optimized performance - Reduced cold starts and memory usage
  • Less boilerplate code - Intuitive API vs verbose AWS SDK
  • Built-in testing support - Interfaces and mocks for testable code
  • Production-ready patterns - Transactions, consistency, error handling

Don't use DynamORM for:

  • Non-DynamoDB databases
  • Applications requiring SQL-style joins
  • Direct AWS SDK control requirements

Features

  • 🚀 Type-Safe: Full compile-time type safety with Go generics
  • Lambda Optimized: Sub-15ms cold starts with connection reuse
  • 🔄 Auto-Generated Keys: Composite key generation from struct tags
  • 🔍 Smart Querying: Intuitive query builder with index management
  • 📦 Batch Operations: Efficient batch read/write operations
  • 🔐 Transaction Support: ACID transactions across multiple items
  • 🎯 Zero Configuration: Works out of the box with sensible defaults
  • 🧪 Testable: Built-in mocks and testing utilities
  • 💰 Cost Tracking: Integrated consumed capacity monitoring
  • 🏗️ Schema Management: Automatic table creation and migrations
  • 🌊 Stream Processing: Native DynamoDB Streams support with UnmarshalItem/UnmarshalItems

Quick Start

// This demonstrates the DynamORM pattern for DynamoDB operations in Go
// It provides type safety, error handling, and Lambda optimization
package main

import (
    "log"
    "github.com/pay-theory/dynamorm"
    "github.com/pay-theory/dynamorm/pkg/session"
)

// CORRECT: Always define models with struct tags
type User struct {
    ID        string `dynamorm:"pk"`      // Partition key
    Email     string `dynamorm:"sk"`      // Sort key  
    Name      string
    CreatedAt int64  `dynamorm:"created_at"`
}

func main() {
    // CORRECT: Initialize with proper configuration
    db, err := dynamorm.New(session.Config{
        Region: "us-east-1",
        // For Lambda: use NewLambdaOptimized() or LambdaInit()
        // For local dev: Endpoint: "http://localhost:8000"
    })
    if err != nil {
        log.Fatal("Failed to initialize DynamORM:", err)
    }

    // CORRECT: Type-safe operations with error handling
    user := &User{
        ID:    "user123",
        Email: "john@example.com", 
        Name:  "John Doe",
    }
    
    // Create operation - automatic validation and marshaling
    if err := db.Model(user).Create(); err != nil {
        log.Printf("Create failed: %v", err)
    }
    
    // Query operation - type-safe results
    var users []User
    err = db.Model(&User{}).
        Where("ID", "=", "user123").
        All(&users)
    if err != nil {
        log.Printf("Query failed: %v", err)
    }
}

// INCORRECT: Don't use the raw AWS SDK like this:
//
// import (
//     "context"
//
//     "github.com/aws/aws-sdk-go-v2/aws"
//     "github.com/aws/aws-sdk-go-v2/service/dynamodb"
//     "github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
// )
//
// func badExample(ctx context.Context, svc *dynamodb.Client) error {
//     input := &dynamodb.PutItemInput{
//         TableName: aws.String("users"),
//         Item: map[string]types.AttributeValue{
//             "id":   &types.AttributeValueMemberS{Value: "user123"},
//             "name": &types.AttributeValueMemberS{Value: "Jane Example"},
//             // ... verbose attribute mapping
//         },
//     }
//
//     _, err := svc.PutItem(ctx, input)
//     return err
// }

This pattern lacks type safety, requires verbose marshaling, and is prone to runtime mistakes.

Installation

# This method is preferred for AWS Lambda deployments
go get github.com/pay-theory/dynamorm

# Lambda-optimized initialization:
db, err := dynamorm.NewLambdaOptimized()
# or
db, err := dynamorm.LambdaInit(&User{})
For Standard Applications
# Use this method for long-running applications
go get github.com/pay-theory/dynamorm

# Standard initialization:
db, err := dynamorm.New(session.Config{Region: "us-east-1"})
For Local Development
# Install with DynamoDB Local support
go get github.com/pay-theory/dynamorm

# Local development configuration:
db, err := dynamorm.New(session.Config{
    Region:   "us-east-1",
    Endpoint: "http://localhost:8000",  // DynamoDB Local
})

Core Concepts

Model Definition - CRITICAL for AI Assistants

Models are how DynamORM understands your DynamoDB table structure. AI assistants MUST use exact canonical patterns to prevent struct hallucinations.

⚠️ AI Warning: DO NOT invent struct patterns. Use exact examples from Struct Definition Guide

Example - Simple Entity (Most Common):

// CANONICAL PATTERN: Use this EXACT format for basic entities
type User struct {
    ID        string    `dynamorm:"pk" json:"id"`           // REQUIRED: Partition key
    Email     string    `dynamorm:"sk" json:"email"`        // OPTIONAL: Sort key
    Name      string    `json:"name"`                       // Standard field
    Active    bool      `json:"active"`                     // Boolean field
    CreatedAt time.Time `json:"created_at"`                 // Timestamp field
}

// CANONICAL PATTERN: Entity with GSI for alternate queries
type Payment struct {
    ID         string    `dynamorm:"pk" json:"id"`                    // Primary partition key
    Timestamp  string    `dynamorm:"sk" json:"timestamp"`             // Primary sort key
    CustomerID string    `dynamorm:"index:customer-index,pk" json:"customer_id"` // GSI partition key
    Status     string    `dynamorm:"index:customer-index,sk" json:"status"`      // GSI sort key
    Amount     int64     `json:"amount"`                              // Standard field
    CreatedAt  time.Time `json:"created_at"`                         // Timestamp field
}

// ❌ FORBIDDEN: These patterns DO NOT EXIST
// type BadPayment struct {
//     ID     string  `dynamorm:"partition_key"`    // WRONG: Use "pk"
//     Amount int64   `dynamorm:"attribute"`        // WRONG: No "attribute" tag
//     Status string  `dynamorm:"gsi:status"`       // WRONG: Use "index:name,pk"
// }

📋 Required for Every Struct:

  • At least one dynamorm:"pk" field (partition key)
  • Proper json: tags for all fields
  • Only supported Go types (string, int, bool, time.Time, []string, map[string]string)
  • Follow naming: PascalCase in Go, snake_case in JSON

🔗 Complete Guidance: See Struct Definition Guide for all canonical patterns.

Query Builder Pattern

DynamORM uses a fluent query builder that automatically selects optimal indexes and generates efficient DynamoDB queries.

Example:

// CORRECT: Chainable query building with automatic optimization
var payments []Payment
err := db.Model(&Payment{}).
    Index("amount-index").              // Explicit index selection
    Where("Amount", ">", 1000).         // Type-safe conditions
    OrderBy("Timestamp", "DESC").       // Automatic sort key handling
    Limit(10).                         // Result limiting
    ConsistentRead().                  // Strong consistency when needed
    All(&payments)                     // Execute and unmarshal

// INCORRECT: Don't build queries manually
// This will cause performance issues and errors:
// input := &dynamodb.QueryInput{
//     TableName: aws.String("payments"),
//     IndexName: aws.String("amount-index"),
//     // ... complex expression building prone to errors
// }

Common Patterns

Pattern: DynamoDB Streams Processing

When to use: Processing DynamoDB stream events in Lambda

// Use DynamORM's UnmarshalItem for processing DynamoDB stream records
// This ensures consistency with your DynamORM models

import (
    "github.com/aws/aws-lambda-go/events"
    "github.com/pay-theory/dynamorm"
)

func handleDynamoDBStream(ctx context.Context, event events.DynamoDBEvent) error {
    for _, record := range event.Records {
        switch record.EventName {
        case "INSERT", "MODIFY":
            var order Order
            // Use DynamORM's UnmarshalItem instead of AWS SDK
            if err := dynamorm.UnmarshalItem(record.Change.NewImage, &order); err != nil {
                return fmt.Errorf("failed to unmarshal: %w", err)
            }
            
            // Process the order...
            log.Printf("Order %s status: %s", order.OrderID, order.Status)
            
        case "REMOVE":
            var order Order
            if err := dynamorm.UnmarshalItem(record.Change.OldImage, &order); err != nil {
                return fmt.Errorf("failed to unmarshal: %w", err)
            }
            
            log.Printf("Order %s was removed", order.OrderID)
        }
    }
    return nil
}

// For batch processing of stream records
func processBatchRecords(records []events.DynamoDBEventRecord) error {
    // Extract all new images
    var items []map[string]types.AttributeValue
    for _, record := range records {
        if record.Change.NewImage != nil {
            items = append(items, record.Change.NewImage)
        }
    }
    
    // Unmarshal all at once
    var orders []Order
    if err := dynamorm.UnmarshalItems(items, &orders); err != nil {
        return fmt.Errorf("failed to unmarshal batch: %w", err)
    }
    
    // Process orders...
    return nil
}
Pattern: Lambda Handler

When to use: Building AWS Lambda functions with DynamoDB Why: Optimizes cold starts and provides automatic resource management

// CORRECT: Lambda-optimized pattern
package main

import (
    "context"
    "github.com/aws/aws-lambda-go/lambda"
    "github.com/pay-theory/dynamorm"
)

var db *dynamorm.LambdaDB

func init() {
    // Initialize once, reuse across invocations
    // This reduces cold start time significantly
    var err error
    db, err = dynamorm.NewLambdaOptimized()
    if err != nil {
        panic(err)
    }
}

func handler(ctx context.Context, event PaymentEvent) error {
    // Business logic using pre-initialized connection
    payment := &Payment{
        ID:     event.PaymentID,
        Amount: event.Amount,
    }
    return db.Model(payment).Create()
}

func main() {
    lambda.Start(handler)
}

// INCORRECT: Don't initialize in handler
// This causes slow cold starts:
// func handler(ctx context.Context, event PaymentEvent) error {
//     db := dynamorm.New(...)  // Creates new connection every time
//     // ... rest of handler
// }
Pattern: Testable Service

When to use: Building testable business logic Why: Enables unit testing without DynamoDB dependency

// CORRECT: Interface-based dependency injection
import "github.com/pay-theory/dynamorm/pkg/core"

type PaymentService struct {
    db core.DB  // Interface allows mocking
}

func NewPaymentService(db core.DB) *PaymentService {
    return &PaymentService{db: db}
}

func (s *PaymentService) CreatePayment(payment *Payment) error {
    // Business logic that can be tested
    return s.db.Model(payment).Create()
}

// Test example:
import (
    "testing"
    "github.com/pay-theory/dynamorm/pkg/mocks"
    "github.com/stretchr/testify/mock"
)

func TestPaymentService(t *testing.T) {
    // CORRECT: Use provided mocks for testing
    mockDB := new(mocks.MockDB)
    mockQuery := new(mocks.MockQuery)
    
    mockDB.On("Model", mock.Anything).Return(mockQuery)
    mockQuery.On("Create").Return(nil)
    
    service := NewPaymentService(mockDB)
    err := service.CreatePayment(&Payment{})
    
    assert.NoError(t, err)
    mockDB.AssertExpectations(t)
}

// INCORRECT: Don't use concrete types
// type BadService struct {
//     db *dynamorm.DB  // Cannot be mocked easily
// }
Pattern: Conditional Writes

When to use: Protect critical writes from accidental overwrites or coordinate optimistic concurrency.
Why: DynamoDB only enforces conditions you explicitly provide—these helpers turn noisy expression plumbing into one-liners.

Create() overwrites existing items by design. Add .IfNotExists() when you need insert-only semantics or idempotent provisioning.

import (
    "context"
    "errors"
    "fmt"
    "time"

    "github.com/pay-theory/dynamorm"
    core "github.com/pay-theory/dynamorm/pkg/core"
    customerrors "github.com/pay-theory/dynamorm/pkg/errors"
)

type Profile struct {
    ID        string `dynamorm:"pk"`
    Email     string `dynamorm:"sk"`
    Status    string
    Version   int64  `json:"version"`
    UpdatedAt time.Time `dynamorm:"updated_at"`
}

func upsertProfile(ctx context.Context, db core.DB, profile *Profile) error {
    // Insert-only guard
    if err := db.WithContext(ctx).Model(profile).IfNotExists().Create(); err != nil {
        if errors.Is(err, customerrors.ErrConditionFailed) {
            return fmt.Errorf("profile already exists: %w", err)
        }
        return err
    }

    // Optimistic update guarded by a status check
    profile.Status = "active"
    profile.UpdatedAt = time.Now()
    profile.Version++
    err := db.Model(&Profile{}).
        Where("ID", "=", profile.ID).
        WithCondition("Status", "=", "pending_review").
        Update("Status", "UpdatedAt")
    if errors.Is(err, customerrors.ErrConditionFailed) {
        return fmt.Errorf("profile changed while updating: %w", err)
    }
    return err
}

// Raw conditions stay available for advanced use cases (e.g., version tokens).
err := db.Model(&Profile{}).
    Where("ID", "=", profile.ID).
    WithConditionExpression("attribute_exists(PK) AND Version = :v", map[string]any{
        ":v": profile.Version,
    }).
    Delete()

ErrConditionFailed is raised for any ConditionalCheckFailedException. Use errors.Is(err, customerrors.ErrConditionFailed) to trigger retries, conflict resolution, or troubleshooting guidance.

Pattern: Batch Get

When to use: Fetching large sets of items by key without writing manual loops
Why: Automatically chunks requests (≤100 keys), retries UnprocessedKeys, and can fan out work in parallel.

// CORRECT: Use KeyPair helpers for composite keys
var invoices []Invoice
keys := []any{
    dynamorm.NewKeyPair("ACCOUNT#123", "INVOICE#2024-01"),
    dynamorm.NewKeyPair("ACCOUNT#123", "INVOICE#2024-02"),
}

if err := db.Model(&Invoice{}).BatchGet(keys, &invoices); err != nil {
    return fmt.Errorf("batch get failed: %w", err)
}
Advanced control with options
opts := dynamorm.DefaultBatchGetOptions()
opts.ChunkSize = 50
opts.Parallel = true
opts.MaxConcurrency = 4
opts.RetryPolicy = &core.RetryPolicy{ // import core "github.com/pay-theory/dynamorm/pkg/core"
    MaxRetries:    5,
    InitialDelay:  50 * time.Millisecond,
    MaxDelay:      2 * time.Second,
    BackoffFactor: 1.5,
    Jitter:        0.4,
}
opts.ProgressCallback = func(done, total int) {
    logger.Infof("retrieved %d/%d invoices", done, total)
}
opts.OnChunkError = func(chunk []any, err error) error {
    metrics.Count("batch_get_chunk_failure")
    return err // or return nil to keep going
}

var invoices []Invoice
if err := db.Model(&Invoice{}).BatchGetWithOptions(keys, &invoices, opts); err != nil {
    return fmt.Errorf("batch get failed: %w", err)
}
Fluent builder for complex cases
var invoices []Invoice
err := db.Model(&Invoice{}).
    BatchGetBuilder().
    Keys(keys).
    Select("InvoiceID", "Status", "Total").
    ConsistentRead().
    Parallel(3).
    OnProgress(func(done, total int) {
        trace.Logf("chunk complete: %d/%d", done, total)
    }).
    Execute(&invoices)
if err != nil {
    return fmt.Errorf("builder batch get failed: %w", err)
}

Results are returned in the same order as the key list. Missing keys are skipped; you can inspect the original key slice to identify which entries were absent.

Custom retry policy with builder
policy := &core.RetryPolicy{
    MaxRetries:    4,
    InitialDelay:  75 * time.Millisecond,
    MaxDelay:      3 * time.Second,
    BackoffFactor: 1.8,
    Jitter:        0.35,
}

var invoices []Invoice
err := db.Model(&Invoice{}).
    BatchGetBuilder().
    Keys(keys).
    Select("InvoiceID", "Status").
    Parallel(8). // automatically caps to 8 in-flight chunks
    WithRetry(policy).
    OnProgress(func(done, total int) {
        metrics.AddGauge("batch_get.progress", done, map[string]string{"total": fmt.Sprintf("%d", total)})
    }).
    OnError(func(chunk []any, err error) error {
        alert.Send("batch_get_chunk_failed", err)
        return err // surface the failure; return nil to keep going
    }).
    Execute(&invoices)
if err != nil {
    return fmt.Errorf("batch get builder failed: %w", err)
}

Set WithRetry(nil) (or opts.RetryPolicy = nil) if you need the operation to fail fast for debugging. Use OnError for selective retries or dead-letter queues, and rely on ProgressCallback to power logging or metrics dashboards.

Tip: Import core "github.com/pay-theory/dynamorm/pkg/core" anywhere you need direct access to RetryPolicy or other advanced batch settings.

Pattern: Transaction Operations

When to use: Multiple operations that must succeed or fail together Why: Ensures data consistency and ACID compliance

// CORRECT: Transaction pattern for consistent operations
err := db.Transaction(func(tx *dynamorm.Tx) error {
    // All operations must succeed or entire transaction rolls back
    
    // Debit account
    account.Balance -= payment.Amount
    if err := tx.Model(account).Update(); err != nil {
        return err // Automatic rollback
    }
    
    // Create payment record
    payment.Status = "completed"
    if err := tx.Model(payment).Create(); err != nil {
        return err // Automatic rollback
    }
    
    // Create audit log
    audit := &AuditLog{
        Action:    "payment_processed",
        PaymentID: payment.ID,
        Amount:    payment.Amount,
    }
    return tx.Model(audit).Create()
})

if err != nil {
    log.Printf("Transaction failed: %v", err)
    // All operations were rolled back automatically
}

// INCORRECT: Don't perform separate operations
// This can leave data in inconsistent state:
// db.Model(account).Update()  // Might succeed
// db.Model(payment).Create()  // Might fail - inconsistent state!
Pattern: Fluent Transaction Builder

When to use: Complex workflows that mix creates, updates, deletes, and condition checks
Why: Compose all 25 DynamoDB TransactWriteItems operations with a single fluent DSL that understands DynamORM metadata.

bookmark := &Bookmark{ID: "bm#123", UserID: user.ID}
err := db.Transact().
    Create(bookmark, dynamorm.IfNotExists()).
    UpdateWithBuilder(user, func(ub core.UpdateBuilder) error {
        ub.Increment("BookmarkCount")
        return nil
    }, dynamorm.Condition("BookmarkCount", ">=", 0)).
    ConditionCheck(&Quota{UserID: user.ID}, dynamorm.Condition("Remaining", ">", 0)).
    Execute()

if errors.Is(err, customerrors.ErrConditionFailed) {
    log.Println("bookmark already exists or quota exhausted")
}

// Prefer context-aware helper when you already have a request-scoped context:
err = db.TransactWrite(ctx, func(tx core.TransactionBuilder) error {
    tx.Put(&AuditLog{ID: uuid.NewString()})
    tx.Delete(bookmark, dynamorm.IfExists())
    return nil // tx.Execute() is invoked automatically
})
if err != nil {
    var txErr *customerrors.TransactionError
    if errors.As(err, &txErr) {
        log.Printf("transaction failed at op %d (%s): %s", txErr.OperationIndex, txErr.Operation, txErr.Reason)
    }
    if errors.Is(err, customerrors.ErrConditionFailed) {
        metrics.Incr("transactions.condition_conflict", nil)
        return fmt.Errorf("transaction conflict: %w", err)
    }
    return fmt.Errorf("transact write failed: %w", err)
}

TransactionError keeps the DynamoDB cancellation reason plus the zero-based operation index, so you know exactly which mutation tripped a condition. Prefer TransactWrite(ctx, fn) when you already have a request-scoped context; it automatically wires the context into the builder and executes Execute() for you.

Performance Benchmarks

Based on our benchmarks, DynamORM provides significant performance improvements when properly configured:

Metric DynamORM AWS SDK Improvement
Lambda Cold Start 11ms 127ms 91% faster
Memory Usage 18MB 42MB 57% less
Single Item Lookup 0.52ms 0.51ms Near parity
Batch Operations 45ms 78ms 42% faster
Code Lines (CRUD) ~20 ~100 ~80% less

Note: Performance varies based on configuration, table design, and workload. Lambda optimizations require using NewLambdaOptimized() or LambdaInit().

These improvements come from:

  • Connection pooling and reuse
  • Optimized marshaling/unmarshaling
  • Intelligent query planning
  • Reduced memory allocations

Troubleshooting

Error: "ValidationException: One or more parameter values were invalid"

Cause: This happens when struct tags don't match table schema Solution: Verify your struct tags match your DynamoDB table definition

// Check your model definition:
type User struct {
    ID    string `dynamorm:"pk"`     // Must match table's partition key
    Email string `dynamorm:"sk"`     // Must match table's sort key (if exists)
}

// Verify table schema matches:
// aws dynamodb describe-table --table-name users
Error: "ResourceNotFoundException: Requested resource not found"

Cause: Table doesn't exist or wrong table name Solution: Create table or verify configuration

// Option 1: Create table from model (development only)
err := db.CreateTable(&User{})

// Option 2: Verify table name in AWS console matches model
// Table name is derived from struct name (User -> users)
Error: "Query cost is too high" or slow performance

Cause: Query not using optimal index or scanning instead of querying Solution: Use explicit index selection and proper key conditions

// CORRECT: Use specific index for efficient queries
err := db.Model(&Payment{}).
    Index("status-index").              // Explicit index
    Where("Status", "=", "pending").    // Partition key condition
    Where("CreatedAt", ">", yesterday). // Sort key condition (optional)
    All(&payments)

// INCORRECT: Don't scan entire table
// err := db.Model(&Payment{}).Where("Amount", ">", 100).All(&payments)
// This scans entire table instead of using index
Error: Cold start timeouts in Lambda

Cause: Not using Lambda optimizations or initializing in handler Solution: Use Lambda optimizations and initialize in init()

// CORRECT: Initialize once in init() with optimizations
var db *dynamorm.LambdaDB

func init() {
    var err error
    db, err = dynamorm.NewLambdaOptimized()
    if err != nil {
        panic(err)
    }
}

// INCORRECT: Don't initialize in handler
// func handler() {
//     db := dynamorm.New(...)  // Slow cold start
// }

Migration Guide

From Raw AWS SDK
// Old pattern with the raw AWS SDK (replace this):
import (
    "context"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/service/dynamodb"
    "github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)

func oldCreateUser(ctx context.Context, svc *dynamodb.Client, user User) error {
    input := &dynamodb.PutItemInput{
        TableName: aws.String("users"),
        Item: map[string]types.AttributeValue{
            "id":    &types.AttributeValueMemberS{Value: user.ID},
            "email": &types.AttributeValueMemberS{Value: user.Email},
            "name":  &types.AttributeValueMemberS{Value: user.Name},
        },
    }

    _, err := svc.PutItem(ctx, input)
    return err
}

// New pattern with DynamORM (use this instead):
func newCreateUser(db *dynamorm.DB, user *User) error {
    return db.Model(user).Create()
}
// Benefits: 80% less code, type safety, automatic marshaling
From GORM
// Old pattern with GORM (SQL-based):
func oldPattern() {
    db, _ := gorm.Open(sqlite.Open("test.db"), &gorm.Config{})
    
    var users []User
    db.Where("age > ?", 18).Find(&users)
}

// New pattern with DynamORM (NoSQL-optimized):
func newPattern() {
    db := dynamorm.New(session.Config{Region: "us-east-1"})
    
    var users []User
    db.Model(&User{}).
        Index("age-index").           // Explicit index for NoSQL
        Where("Age", ">", 18).
        All(&users)
}
// Benefits: NoSQL optimization, better performance, cloud-native

Best Practices

  1. ALWAYS use struct tags for DynamoDB schema mapping
  2. ALWAYS initialize DynamORM in init() for Lambda functions
  3. ALWAYS use interfaces (core.DB) for testable code
  4. NEVER initialize database connections in request handlers
  5. NEVER scan tables without indexes - use Query with proper keys
  6. PREFER transactions for multi-item consistency requirements
  7. PREFER batch operations for multiple items of same type

Demo Service (Phase 4 Helpers)

Need a runnable example that strings the newest helpers together? cmd/dynamorm-service boots from the canonical quick-start snippet (README.md §§42-118) and layers on:

  • Config plumbing (README.md §§121-206): DYNAMORM_RUNTIME_MODE toggles NewLambdaOptimized, New, or local endpoints so you can mimic Lambda, standard, or DynamoDB Local setups without changing code.
  • Conditional guards (README.md §§385-444): Insert-only creates, optimistic updates, and guarded deletes all surface customerrors.ErrConditionFailed exactly like the docs describe.
  • Transaction builder (README.md §§588-620): Dual-writes use db.Transact() plus the context-aware TransactWrite() helper, logging customerrors.TransactionError metadata for observability.
  • Retry-aware BatchGet (README.md §§445-541, 509-537): The fluent builder example wires core.RetryPolicy, progress callbacks, chunk-level error hooks, and dynamorm.NewKeyPair key construction (per docs/archive/struct-definition-guide.md:393).

Run it with go run ./cmd/dynamorm-service (standard mode) or set DYNAMORM_RUNTIME_MODE=lambda|local to exercise the other init paths. When sharing updates, link teammates to docs/whats-new.md for the Phase 4 summary outlining why these helpers are required across new services.

API Reference

Model(entity interface{}) Query

Purpose: Creates a type-safe query builder for the given entity type When to use: Starting any DynamoDB operation (Create, Read, Update, Delete) When NOT to use: Don't call Model() multiple times in the same operation chain

// Example: Basic model usage
db.Model(&User{})        // Query builder for User table
db.Model(user)          // For operations on existing instance
db.Model(&users)        // For operations returning multiple results

// This returns a Query interface for chaining operations
Where(field, operator, value) Query

Purpose: Adds type-safe condition to query (translates to DynamoDB KeyConditionExpression or FilterExpression) When to use: Filtering results by attribute values When NOT to use: Don't use Where() without proper index for large tables

// Example: Query with conditions
db.Model(&Payment{}).
    Where("Status", "=", "pending").        // Key condition (indexed field)
    Where("Amount", ">", 1000).             // Filter expression
    All(&payments)
Conditional Write Helpers

Purpose: Guard create/update/delete operations with DynamoDB conditions without dropping to the raw SDK
Methods: IfNotExists(), IfExists(), WithCondition(field, op, value), WithConditionExpression(expr, values)

Create() overwrites matching primary keys by default. Chain .IfNotExists() (or .WithCondition(...)) when you need insert-only semantics or optimistic concurrency to match production safety expectations.

// Prevent overwriting existing users
err := db.Model(&User{
    ID:    "user123",
    Email: "john@example.com",
}).IfNotExists().Create()
if errors.Is(err, customerrors.ErrConditionFailed) {
    log.Println("user already exists")
}

// Optimistic update – only run when Status is still active
err = db.Model(&Session{}).
    Where("ID", "=", "sess#123").
    WithCondition("Status", "=", "active").
    Update("LastSeen")

// Advanced raw expression (placeholders must be provided)
db.Model(&Order{}).
    Where("PK", "=", orderPK).
    WithConditionExpression("attribute_exists(PK) AND Version = :v", map[string]any{
        ":v": currentVersion,
    }).
    Delete()

Import the sentinel error via customerrors "github.com/pay-theory/dynamorm/pkg/errors" and check with errors.Is.

All conditional failures bubble up as customerrors.ErrConditionFailed, so callers can use errors.Is(err, customerrors.ErrConditionFailed) for retry or conflict handling.

Transaction(fn func(*Tx) error) error

Purpose: Executes multiple operations atomically within a DynamoDB transaction When to use: Operations that must all succeed or all fail together When NOT to use: Single operations or operations across different AWS accounts

// Example: Transfer money between accounts
err := db.Transaction(func(tx *dynamorm.Tx) error {
    // All operations execute atomically
    if err := tx.Model(fromAccount).Update(); err != nil {
        return err  // Rolls back entire transaction
    }
    return tx.Model(toAccount).Update()
})

About This Codebase

This entire codebase was written 100% by AI code generation, guided by the development team at Pay Theory. The framework represents a collaboration between human architectural vision and AI implementation capabilities, demonstrating the potential of AI-assisted software development for creating production-ready systems.

Documentation

Overview

Package dynamorm provides a type-safe ORM for Amazon DynamoDB in Go

lambda.go

multiaccount.go

Index

Constants

This section is empty.

Variables

View Source
var (
	WithBackupTable = schema.WithBackupTable
	WithDataCopy    = schema.WithDataCopy
	WithTargetModel = schema.WithTargetModel
	WithTransform   = schema.WithTransform
	WithBatchSize   = schema.WithBatchSize
)

Re-export AutoMigrate options for convenience

Functions

func AtVersion added in v1.0.37

func AtVersion(version int64) core.TransactCondition

AtVersion enforces an optimistic locking check on the model's version field.

func Condition added in v1.0.37

func Condition(field, operator string, value any) core.TransactCondition

Condition creates a simple field comparison condition for transactional writes.

func ConditionExpression added in v1.0.37

func ConditionExpression(expression string, values map[string]any) core.TransactCondition

ConditionExpression registers a raw condition expression for transactional writes.

func ConditionVersion added in v1.0.37

func ConditionVersion(version int64) core.TransactCondition

ConditionVersion is an alias for AtVersion for API parity with UpdateBuilder helpers.

func DefaultBatchGetOptions added in v1.0.37

func DefaultBatchGetOptions() *core.BatchGetOptions

DefaultBatchGetOptions returns the library defaults for BatchGet operations.

func EnableXRayTracing

func EnableXRayTracing() bool

EnableXRayTracing enables AWS X-Ray tracing for DynamoDB calls

func GetLambdaMemoryMB

func GetLambdaMemoryMB() int

GetLambdaMemoryMB returns the allocated memory in MB

func GetPartnerFromContext

func GetPartnerFromContext(ctx context.Context) string

GetPartnerFromContext retrieves partner ID from context

func GetRemainingTimeMillis

func GetRemainingTimeMillis(ctx context.Context) int64

GetRemainingTimeMillis returns milliseconds until Lambda timeout

func IfExists added in v1.0.37

func IfExists() core.TransactCondition

IfExists guards a write with attribute_exists on the item's primary key.

func IfNotExists added in v1.0.37

func IfNotExists() core.TransactCondition

IfNotExists guards a write with attribute_not_exists on the item's primary key.

func IsLambdaEnvironment

func IsLambdaEnvironment() bool

IsLambdaEnvironment detects if running in AWS Lambda

func New

func New(config session.Config) (core.ExtendedDB, error)

New creates a new DynamORM instance with the given configuration

func NewBasic added in v1.0.1

func NewBasic(config session.Config) (core.DB, error)

NewBasic creates a new DynamORM instance that returns the basic DB interface Use this when you only need core functionality and want easier mocking

func NewKeyPair added in v1.0.37

func NewKeyPair(partitionKey any, sortKey ...any) core.KeyPair

NewKeyPair constructs a composite key helper for BatchGet operations.

func PartnerContext

func PartnerContext(ctx context.Context, partnerID string) context.Context

PartnerContext adds partner information to context for tracing

func UnmarshalItem added in v1.0.24

func UnmarshalItem(item map[string]types.AttributeValue, dest interface{}) error

UnmarshalItem unmarshals a DynamoDB AttributeValue map into a Go struct. This is the recommended way to unmarshal DynamoDB stream records or any DynamoDB items when using DynamORM.

The function respects DynamORM struct tags (dynamorm:"pk", dynamorm:"attr:name", etc.) and handles all DynamoDB attribute types correctly.

Example usage with DynamoDB Streams:

func processDynamoDBStream(record events.DynamoDBEventRecord) (*MyModel, error) {
    image := record.Change.NewImage
    if image == nil {
        return nil, nil
    }

    var model MyModel
    if err := dynamorm.UnmarshalItem(image, &model); err != nil {
        return nil, fmt.Errorf("failed to unmarshal: %w", err)
    }

    return &model, nil
}

func UnmarshalItems added in v1.0.24

func UnmarshalItems(items []map[string]types.AttributeValue, dest interface{}) error

UnmarshalItems unmarshals a slice of DynamoDB AttributeValue maps into a slice of Go structs. This is useful for batch operations or when processing multiple items from a query result.

func UnmarshalStreamImage added in v1.0.24

func UnmarshalStreamImage(streamImage map[string]events.DynamoDBAttributeValue, dest interface{}) error

UnmarshalStreamImage unmarshals a DynamoDB stream image (from Lambda events) into a Go struct. This function handles the conversion from Lambda's events.DynamoDBAttributeValue to the standard types.AttributeValue and then unmarshals into your DynamORM model.

Example usage:

func handleStream(record events.DynamoDBEventRecord) error {
    var order Order
    if err := dynamorm.UnmarshalStreamImage(record.Change.NewImage, &order); err != nil {
        return err
    }
    // Process order...
}

Types

type AccountConfig

type AccountConfig struct {
	RoleARN    string
	ExternalID string
	Region     string
	// Optional: Custom session duration (default is 1 hour)
	SessionDuration time.Duration
}

AccountConfig holds configuration for a partner account

type AutoMigrateOption

type AutoMigrateOption = schema.AutoMigrateOption

Re-export types for convenience

type BatchGetOptions added in v1.0.37

type BatchGetOptions = core.BatchGetOptions

Re-export types for convenience

type ColdStartMetrics

type ColdStartMetrics struct {
	TotalDuration time.Duration
	Phases        map[string]time.Duration
	MemoryMB      int
	IsLambda      bool
}

ColdStartMetrics contains cold start performance data

func BenchmarkColdStart

func BenchmarkColdStart(models ...any) ColdStartMetrics

BenchmarkColdStart measures cold start performance

func (ColdStartMetrics) String

func (m ColdStartMetrics) String() string

String returns a formatted string of the metrics

type Config

type Config = session.Config

Re-export types for convenience

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB is the main DynamORM database instance

func (*DB) AutoMigrate

func (db *DB) AutoMigrate(models ...any) error

AutoMigrate creates or updates tables based on the given models

func (*DB) AutoMigrateWithOptions

func (db *DB) AutoMigrateWithOptions(model any, opts ...any) error

AutoMigrateWithOptions performs enhanced auto-migration with data copy support

func (*DB) Close

func (db *DB) Close() error

Close closes the database connection

func (*DB) CreateTable

func (db *DB) CreateTable(model any, opts ...any) error

CreateTable creates a DynamoDB table for the given model

func (*DB) DeleteTable

func (db *DB) DeleteTable(model any) error

DeleteTable deletes the DynamoDB table for the given model

func (*DB) DescribeTable

func (db *DB) DescribeTable(model any) (any, error)

DescribeTable returns the table description for the given model

func (*DB) EnsureTable

func (db *DB) EnsureTable(model any) error

EnsureTable checks if a table exists for the model and creates it if not

func (*DB) Migrate

func (db *DB) Migrate() error

Migrate runs all pending migrations

func (*DB) Model

func (db *DB) Model(model any) core.Query

Model returns a new query builder for the given model

func (*DB) RegisterTypeConverter added in v1.0.30

func (db *DB) RegisterTypeConverter(typ reflect.Type, converter pkgTypes.CustomConverter) error

RegisterTypeConverter registers a custom converter for a specific Go type. This allows callers to control how values are marshaled to and unmarshaled from DynamoDB without forking the internal marshaler. Registering a converter clears any cached marshalers so subsequent operations use the new logic.

func (*DB) Transact added in v1.0.37

func (db *DB) Transact() core.TransactionBuilder

Transact returns a fluent transaction builder for composing TransactWriteItems requests.

func (*DB) TransactWrite added in v1.0.37

func (db *DB) TransactWrite(ctx context.Context, fn func(core.TransactionBuilder) error) error

TransactWrite executes the supplied function with a transaction builder and automatically commits it.

func (*DB) Transaction

func (db *DB) Transaction(fn func(tx *core.Tx) error) error

Transaction executes a function within a database transaction

func (*DB) TransactionFunc

func (db *DB) TransactionFunc(fn func(tx any) error) error

TransactionFunc executes a function within a database transaction This is the actual implementation that uses our sophisticated transaction support

func (*DB) WithContext

func (db *DB) WithContext(ctx context.Context) core.DB

WithContext returns a new DB instance with the given context

func (*DB) WithLambdaTimeout

func (db *DB) WithLambdaTimeout(ctx context.Context) core.DB

WithLambdaTimeout sets a deadline based on Lambda context

func (*DB) WithLambdaTimeoutBuffer

func (db *DB) WithLambdaTimeoutBuffer(buffer time.Duration) core.DB

WithLambdaTimeoutBuffer sets a custom timeout buffer for Lambda execution

type KeyPair added in v1.0.37

type KeyPair = core.KeyPair

Re-export types for convenience

type LambdaDB

type LambdaDB struct {
	core.ExtendedDB
	// contains filtered or unexported fields
}

LambdaDB wraps DB with Lambda-specific optimizations

func LambdaInit

func LambdaInit(models ...any) (*LambdaDB, error)

LambdaInit should be called in the init() function of your Lambda handler It performs one-time initialization to reduce cold start latency

func NewLambdaOptimized

func NewLambdaOptimized() (*LambdaDB, error)

NewLambdaOptimized creates a Lambda-optimized DB instance

func (*LambdaDB) GetMemoryStats

func (ldb *LambdaDB) GetMemoryStats() LambdaMemoryStats

GetMemoryStats returns current memory usage statistics

func (*LambdaDB) IsModelRegistered

func (ldb *LambdaDB) IsModelRegistered(model any) bool

IsModelRegistered checks if a model is already registered

func (*LambdaDB) OptimizeForColdStart

func (ldb *LambdaDB) OptimizeForColdStart()

OptimizeForColdStart reduces Lambda cold start time

func (*LambdaDB) OptimizeForMemory

func (ldb *LambdaDB) OptimizeForMemory()

OptimizeForMemory adjusts internal buffers based on available Lambda memory

func (*LambdaDB) PreRegisterModels

func (ldb *LambdaDB) PreRegisterModels(models ...any) error

PreRegisterModels registers models at init time to reduce cold starts

func (*LambdaDB) RegisterTypeConverter added in v1.0.30

func (ldb *LambdaDB) RegisterTypeConverter(typ reflect.Type, converter pkgTypes.CustomConverter) error

RegisterTypeConverter registers a custom converter on the underlying DB and clears any cached marshalers so the converter takes effect immediately.

func (*LambdaDB) WithLambdaTimeout

func (ldb *LambdaDB) WithLambdaTimeout(ctx context.Context) *LambdaDB

WithLambdaTimeout creates a new DB instance with Lambda timeout handling

type LambdaMemoryStats

type LambdaMemoryStats struct {
	Alloc          uint64  // Bytes allocated and still in use
	TotalAlloc     uint64  // Bytes allocated (even if freed)
	Sys            uint64  // Bytes obtained from system
	NumGC          uint32  // Number of GC cycles
	AllocatedMB    float64 // MB currently allocated
	SystemMB       float64 // MB obtained from system
	LambdaMemoryMB int     // Total Lambda memory allocation
	MemoryPercent  float64 // Percentage of Lambda memory used
}

LambdaMemoryStats contains memory usage information

type MultiAccountDB

type MultiAccountDB struct {
	// contains filtered or unexported fields
}

MultiAccountDB manages DynamoDB connections across multiple AWS accounts

func NewMultiAccount

func NewMultiAccount(accounts map[string]AccountConfig) (*MultiAccountDB, error)

NewMultiAccount creates a multi-account aware DB

func (*MultiAccountDB) AddPartner

func (mdb *MultiAccountDB) AddPartner(partnerID string, config AccountConfig)

AddPartner dynamically adds a new partner configuration

func (*MultiAccountDB) Close

func (mdb *MultiAccountDB) Close() error

Close stops the refresh routine and cleans up

func (*MultiAccountDB) Partner

func (mdb *MultiAccountDB) Partner(partnerID string) (*LambdaDB, error)

Partner returns a DB instance for the specified partner account

func (*MultiAccountDB) RemovePartner

func (mdb *MultiAccountDB) RemovePartner(partnerID string)

RemovePartner removes a partner and clears its cached connection

func (*MultiAccountDB) WithContext

func (mdb *MultiAccountDB) WithContext(ctx context.Context) *MultiAccountDB

WithContext returns a new MultiAccountDB with the given context

Directories

Path Synopsis
Package examples demonstrates DynamORM's embedded struct support
Package examples demonstrates DynamORM's embedded struct support
initialization command
Package main demonstrates proper DynamORM initialization patterns to avoid nil pointer dereference errors
Package main demonstrates proper DynamORM initialization patterns to avoid nil pointer dereference errors
lambda command
optimization command
internal
pkg
consistency
Package consistency provides utilities for handling eventual consistency in DynamoDB
Package consistency provides utilities for handling eventual consistency in DynamoDB
core
Package core defines the core interfaces and types for DynamORM
Package core defines the core interfaces and types for DynamORM
errors
Package errors defines error types and utilities for DynamORM
Package errors defines error types and utilities for DynamORM
interfaces
Package interfaces provides abstractions for AWS SDK operations to enable mocking
Package interfaces provides abstractions for AWS SDK operations to enable mocking
marshal
Package marshal provides optimized marshaling for DynamoDB
Package marshal provides optimized marshaling for DynamoDB
mocks
Package mocks provides mock implementations for DynamORM interfaces and AWS SDK operations
Package mocks provides mock implementations for DynamORM interfaces and AWS SDK operations
model
Package model provides model registration and metadata management for DynamORM
Package model provides model registration and metadata management for DynamORM
query
Package query provides aggregate functionality for DynamoDB queries
Package query provides aggregate functionality for DynamoDB queries
session
Package session provides AWS session management and DynamoDB client configuration
Package session provides AWS session management and DynamoDB client configuration
testing
Package testing provides utilities for testing applications that use DynamORM.
Package testing provides utilities for testing applications that use DynamORM.
transaction
Package transaction provides atomic transaction support for DynamORM
Package transaction provides atomic transaction support for DynamORM
types
Package types provides type conversion between Go types and DynamoDB AttributeValues
Package types provides type conversion between Go types and DynamoDB AttributeValues

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL