telemetry

package module
v0.0.0-...-a36e201 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 20, 2026 License: Apache-2.0 Imports: 37 Imported by: 0

README

Durable Execution Telemetry Exporter

Observability plugin for the AWS Durable Execution SDK for Go. Emits execution snapshot records to configurable destinations as durable workflows run, complete, or fail. Records capture execution metadata, timing, operation details, and content fields (inputs, outputs, errors) for diagnostics and monitoring.

This module is built on the experimental plugin API (durable.Plugin, durable.PluginFactory, durable.WithPlugins), so its versions stay below v1. Any release may change the telemetry package API and the record schema. Upgrade this module together with the SDK. See Versioning in the root README.

Installation

go get github.com/aws/aws-durable-execution-sdk-go/telemetry

Quick Start

package main

import (
    "github.com/aws/aws-durable-execution-sdk-go/durable"
    "github.com/aws/aws-durable-execution-sdk-go/telemetry"
)

func handler(ctx durable.Context, event any) (any, error) {
    // ... your durable workflow ...
    return nil, nil
}

func main() {
    // TelemetryPlugin is a durable.PluginFactory: the handler calls its
    // CreatePlugin once per invocation.
    durable.Start(handler, durable.WithPlugins(telemetry.New(telemetry.Config{})))
}

With a zero-value Config, the plugin emits a single JSON record to stdout when the execution completes (succeeded or failed). Lambda captures stdout to the function's CloudWatch log group automatically, so this requires zero additional IAM permissions.

Invocation lifetime and concurrency

TelemetryPlugin is a factory shared by the execution environment. Each CreatePlugin call creates a separate record, operation index, sampling decision, mutex, and on-change dispatch state. Hooks and log enrichment for one invocation use only that invocation's state. Concurrent operation hooks within an invocation remain synchronized.

The factory initializes execution metadata, input, and sampling before returning the hooks. A resumed invocation retains the execution ARN and original execution start timestamp. Its request ID identifies the new invocation. Sampling uses the execution ARN, so the decision stays the same across resumes.

At invocation start, the plugin initializes its operation buffer from the full checkpointed snapshot. This includes unchanged nested operations in completed child contexts whose bodies do not run again. The root execution operation is excluded from the operations array. OperationDetail and the SDK's durable.WithPluginChildOperationsDepth still limit the operations reported. Subsequent hooks update the buffer by operation ID.

Exporters and content transforms remain shared. InputTransform, OutputTransform, and each OperationOverride.Result may be called concurrently by different invocations. Callers must make these functions and their captured state safe for concurrent use. Within one invocation, content transforms remain serialized.

Transforms run only for sampled-in invocations. InputTransform and OutputTransform run at most once per invocation. A Result transform runs once for an operation's result in an invocation, and again only if that result text changes. Each new invocation replays checkpointed results, so it calls the Result transform again. Transforms run while the plugin holds its per-invocation lock. So a transform should return quickly and must not block.

Each invocation waits for its own in-flight on-change dispatches before its final emission. It does not wait for another invocation's dispatches. Shared exporters can receive overlapping Export and Flush calls from different invocations. This change does not add a shared scheduler or establish a complete exporter concurrency contract. Invocation-local records alone do not make an exporter or its dependencies concurrency-safe. Exporter concurrency and scheduling remain separate follow-up work.

Configuration

telemetry.Config controls all plugin behavior:

type Config struct {
    EmitMode        EmitMode
    OperationDetail OperationDetail
    SamplingRate    float64
    Exporters       []Exporter
    Content         ContentConfig
}
EmitMode

Controls when a record is sent to exporters.

Value Behavior
EmitOnComplete Emit only when the execution reaches a terminal state (succeeded or failed). This is the default.
EmitAlways Emit after every invocation regardless of outcome.
EmitOnChange Emit a running snapshot on every operation start/end, plus the terminal record.
OperationDetail

Controls which operations appear in the record's Operations array.

Value Behavior
OperationDetailTopLevel Only top-level operations (those without a ParentID). This is the default.
OperationDetailFullTree Every operation, including children of contexts.
SamplingRate

Fraction of executions (0.0 to 1.0) that emit records. Defaults to 1.0 (every execution). The decision is per-execution and deterministic, derived from an FNV-1a hash of the execution ARN.

plugin := telemetry.New(telemetry.Config{
    SamplingRate: 0.1, // 10% of executions
})
ContentConfig

Controls what content appears in emitted records.

type ContentConfig struct {
    ExcludeInput    bool
    InputTransform  func(any) any
    ExcludeOutput   bool
    OutputTransform func(any) any
    Operations      ContentOperations
}

type ContentOperations struct {
    Overrides     []OperationOverride
    ExcludeErrors bool
}

type OperationOverride struct {
    OperationName string
    Exclude       bool
    Result        func(any) any
}

The zero value includes the execution input and output unchanged, omits operation results, and includes operation and execution errors. Each field changes one thing, so set only the fields you need.

telemetry.New(telemetry.Config{
    Content: telemetry.ContentConfig{
        // Remove a sensitive field from the recorded input.
        InputTransform: func(v any) any {
            if m, ok := v.(map[string]any); ok {
                delete(m, "password")
            }
            return v
        },
        // Do not record the output at all.
        ExcludeOutput: true,
        Operations: telemetry.ContentOperations{
            Overrides: []telemetry.OperationOverride{
                // Record this operation's result, masking one field.
                {OperationName: "fetch-user", Result: func(v any) any {
                    if m, ok := v.(map[string]any); ok {
                        m["email"] = "***"
                    }
                    return v
                }},
                // Drop this operation from the record.
                {OperationName: "debug-dump", Exclude: true},
            },
        },
    },
})

Content fields are JSON values (json.RawMessage):

  • Record.Input and Record.Output are the handler's typed input and result, marshaled once. A Go string becomes a JSON string. A nil pointer, map, or slice becomes null, the same text the SDK checkpoints for that value.
  • OperationRecord.Result is the checkpointed result. It is recorded only for operations named by an override that sets Result.

A transform receives the content decoded as a JSON value: map[string]any, []any, string, json.Number, bool, or nil. A struct arrives as a map[string]any, so a field can be deleted or overwritten by key. Numbers arrive as json.Number rather than float64. A float64 holds 53 bits of mantissa, so an integer above 2^53 would be rounded by the decode alone. json.Number keeps the exact digits; call Int64() or Float64() on it to do arithmetic. The return value is what the record carries. InputTransform and OutputTransform are not called when the matching Exclude* field is true.

OperationOverride.Result receives the checkpointed text decoded as a JSON value, or the text itself as a string when it is not valid JSON. The plugin does not run the operation's own serializer, so an operation with a custom Serdes delivers its serialized form. To record a result unchanged, return the argument.

The value a transform returns is marshaled again. Marshaling writes no whitespace and sorts the keys of every object. So a recorded result can differ from the checkpointed text in key order, even when the transform returns its argument.

Every exporter and RenderRecord write <, > and & literally, as the SDK does in checkpoints. So a text search for one of these characters matches the record. Calling json.Marshal on a Record yourself escapes them as \u003c, \u003e and \u0026; use RenderRecord, or a json.Encoder with SetEscapeHTML(false).

If a transform panics, or returns a value that cannot be marshaled, the field is omitted. The untransformed value is never recorded in its place.

Overrides are matched by OperationName. When several entries share a name, the last one wins. ExcludeErrors omits the error field from operation entries; the execution-level error on the record is always included.

There is no per-field length cap. An exporter that implements SizeLimiter bounds the whole record instead; see Size Limits.

Transforms may run concurrently across invocations. Their functions and captured state must support concurrent use. They run only for sampled-in invocations, and must not block. For how often each transform runs, see Invocation lifetime and concurrency.

OperationsFormat

Some exporters support an OperationsFormat field that controls how operations are rendered in the serialized JSON body:

Value Behavior
OperationsFormatArray Standard operations JSON array. This is the default.
OperationsFormatByName Operations grouped by name (operationsByName), no array.
OperationsFormatBoth Both the array and the by-name aggregation.

Exporters that support this: File, HTTP, SQS, EventBridge, Firehose.

The LambdaLog and CloudWatch Logs exporters have no OperationsFormat field. They always render OperationsFormatByName. See LambdaLog and CloudWatch Logs.

Exporters

The plugin dispatches records to all configured exporters concurrently. Each exporter's errors are swallowed independently (never failing the execution or affecting other exporters). If no exporters are configured, a single LambdaLogExporter is used by default.

LambdaLog (default)

Writes records as a single JSON line to stdout. Lambda captures this to the function's CloudWatch log group automatically.

exporter := telemetry.NewLambdaLogExporter()

Each line carries operationsByName and no operations array. The format is fixed. CloudWatch Logs Insights selects a field by a dotted path such as operationsByName.increment.count. A query cannot select an element of the operations array by its name.

Size limit: 256 KB (configurable via MaxSizeBytes). The limit applies to the written line: the operationsByName rendering and the newline that ends it.

File

Writes each record as a JSON file to a local directory.

exporter := telemetry.NewFileExporter(telemetry.FileExporterConfig{
    Dir:              "/tmp/telemetry-records",
    OperationsFormat: telemetry.OperationsFormatByName, // optional
})

A custom FileNamer function can override the default filename scheme ({hash8}_{timestamp}.json).

HTTP

POSTs JSON records to an HTTP endpoint. Retries on 5xx responses with exponential backoff.

exporter := telemetry.NewHTTPExporter(telemetry.HTTPExporterConfig{
    URL: "https://example.com/ingest",
    Headers: map[string]string{
        "Authorization": "Bearer my-token",
    },
    Timeout:    5 * time.Second,
    MaxRetries: 2,
})
S3

Writes records as JSON objects to an S3 bucket with configurable partitioning.

exporter := &telemetry.S3Exporter{
    API:          s3Client,
    Bucket:       "my-telemetry-bucket",
    Prefix:       "durable-execution-telemetry/", // default
    Partitioning: telemetry.S3PartitioningDate, // default: year=YYYY/month=MM/day=DD/
}

Partitioning options: S3PartitioningDate (default), S3PartitioningFunctionName, S3PartitioningNone.

Size limit: 5 MB (configurable via MaxSizeBytes).

CloudWatch Logs

Writes records to a specific CloudWatch Logs log group via PutLogEvents. Unlike LambdaLog (which relies on Lambda's stdout capture), this exporter lets you centralize records from multiple functions into a shared, queryable log group.

exporter := &telemetry.CloudWatchLogsExporter{
    API:             cwlClient,
    LogGroupName:    "/aws/durable-execution/telemetry",
    LogStreamPrefix: "durable-execution-telemetry/", // default
}

Log streams are named {prefix}{YYYY}/{MM}/{DD} (one per UTC calendar day, auto-created on first write).

Each log event carries operationsByName and no operations array, for the same reason as LambdaLog. The format is fixed.

Size limit: 256 KB (configurable via MaxSizeBytes). The limit applies to the operationsByName rendering.

SQS

Sends records as SQS messages. FIFO mode is auto-detected from a .fifo suffix on the queue URL.

exporter := &telemetry.SQSExporter{
    API:      sqsClient,
    QueueURL: "https://sqs.us-east-1.amazonaws.com/123456789012/telemetry-queue",
}

Message attributes status and functionName are always set for filtering. In FIFO mode, MessageGroupID defaults to the execution ARN and deduplication is based on {ExecutionArn}:{EmittedAt}.

Size limit: 256 KB (configurable via MaxSizeBytes).

EventBridge

Publishes records as EventBridge events for event-driven reactions.

exporter := &telemetry.EventBridgeExporter{
    API:          ebClient,
    EventBusName: "default", // default
    Source:       "software.amazon.lambda.durable.telemetry", // default
}

DetailType is the record's status (SUCCEEDED/FAILED/RUNNING). Detail is the full rendered record JSON.

Size limit: 256 KB (configurable via MaxSizeBytes).

Firehose

Sends records to a Kinesis Data Firehose delivery stream for delivery to S3, Redshift, Splunk, or any configured HTTP endpoint.

exporter := &telemetry.FirehoseExporter{
    API:                firehoseClient,
    DeliveryStreamName: "telemetry-delivery-stream",
}

Records are appended with a trailing newline (NDJSON) so they remain individually parseable when Firehose batches multiple records into a single S3 object.

Size limit: 1 MB (configurable via MaxSizeBytes).

DynamoDB

Writes records as items to a DynamoDB table.

exporter := &telemetry.DynamoDBExporter{
    API:       ddbClient,
    TableName: "durable-execution-telemetry",
}

Partition key (pk by default) is the execution ARN. Sort key (sk by default) is the EmittedAt timestamp. Set SortKeyField to telemetry.NoSortKey to disable the sort key (each export overwrites the previous record for the same execution).

Size limit: 400 KB (configurable via MaxSizeBytes).

OpenSearch

Indexes records as documents via HTTP POST to the OpenSearch _doc API.

exporter := &telemetry.OpenSearchExporter{
    Endpoint:  "https://search-my-domain.us-east-1.es.amazonaws.com",
    IndexName: "durable-execution-telemetry", // default
    Username:  "admin",
    Password:  "secret",
}

Supports Basic auth (Username/Password) or Bearer token authentication.

Size limit: 10 MB (configurable via MaxSizeBytes).

Redshift

Writes records to Amazon Redshift via the Redshift Data API (HTTP-based, no VPC required). Fire-and-forget: does not poll for statement completion.

exporter := &telemetry.RedshiftExporter{
    API:           redshiftDataClient,
    WorkgroupName: "my-workgroup", // serverless mode
    Database:      "analytics",
    TableName:     "durable_execution_telemetry", // default
}

Supports two connection modes: serverless (WorkgroupName) or provisioned (ClusterIdentifier). SQL is always parameterized.

RDS/Aurora

Writes records to Aurora/RDS via the RDS Data API (HTTP-based, no VPC required).

exporter := &telemetry.RDSExporter{
    API:         rdsDataClient,
    ResourceArn: "arn:aws:rds:us-east-1:123456789012:cluster:my-cluster",
    SecretArn:   "arn:aws:secretsmanager:us-east-1:123456789012:secret:my-creds",
    Database:    "app_db",
    TableName:   "durable_execution_telemetry", // default
}

Size limit: 1 MB (configurable via MaxSizeBytes).

Custom Exporters

Implement the Exporter interface to send records to any destination:

type Exporter interface {
    Export(ctx context.Context, record Record) error
}

Optional interfaces an exporter may additionally implement:

  • Flusher - flush buffered data before the invocation returns.
  • SizeLimiter - opt into size-based truncation with a MaxRecordSizeBytes() int method.
  • RenderingSizeLimiter - SizeLimiter plus a Render(Record) ([]byte, error) method for format-aware size checking.
type MyExporter struct{}

func (e *MyExporter) Export(ctx context.Context, record telemetry.Record) error {
    // Send record to your destination
    return nil
}

// Optional: implement Flusher
func (e *MyExporter) Flush(ctx context.Context) error {
    return nil
}

Record Structure

Each emitted Record contains:

Field Type Description
RecordType string Always "DurableExecutionTelemetry".
SchemaVersion string Always "1.0".
ExecutionArn string The durable execution's full ARN.
ExecutionName string Extracted from the execution-name segment of the ARN.
FunctionName string Function name parsed from the execution ARN. Always emitted.
FunctionQualifier string Function qualifier parsed from the execution ARN ($LATEST, a version, or an alias). JSON key functionQualifier. Always emitted.
Region string Region parsed from the execution ARN. Always emitted.
AccountID string Account ID parsed from the execution ARN. JSON key accountId. Always emitted.
Status Status RUNNING, SUCCEEDED, or FAILED.
StartTimestamp *time.Time When the execution started. JSON key startTime. Falls back to the oldest operation start, then the invocation time, when the payload omits it.
EndTimestamp *time.Time When the execution ended (terminal records only). JSON key endTime.
DurationMs *int64 End minus start in milliseconds.
EmittedAt time.Time When this record was emitted. Strictly increases across the records of one invocation.
Operations []OperationRecord Operation snapshots (filtered by OperationDetail).
Error *ErrorRecord Execution-level error (failed executions only).
Input json.RawMessage Execution input as a JSON value (if ContentConfig allows).
Output json.RawMessage Execution output as a JSON value (if ContentConfig allows).
Custom map[string]any User-defined metadata.
Truncated bool True if the record was truncated for size.
DroppedOperations int Number of operations dropped during truncation.
DroppedInput bool True if input was dropped during truncation.
DroppedOutput bool True if output was dropped during truncation.

Each OperationRecord contains ID, Name, Type, SubType, Status, ParentID, timestamps (JSON keys startTime and endTime), DurationMs, Attempt count, Result (a JSON value), and Error.

Every timestamp in the JSON output (emittedAt, startTime, endTime, at both the record and operation level) is UTC with exactly three fraction digits and a Z suffix, for example 2026-10-24T02:22:35.088Z. The Go fields stay time.Time; Record.MarshalJSON and OperationRecord.MarshalJSON apply the format. json.Unmarshal into Record parses it back. The DynamoDB sort key, the SQS FIFO deduplication ID, and the RDS/Redshift emitted_at column use the same format.

Size Limits

Exporters that implement SizeLimiter trigger automatic record truncation when the record exceeds their limit. The size is measured on the json.Marshal form of the record. An exporter that also implements RenderingSizeLimiter is measured on its own Render output instead, which is the form it sends.

Truncation removes content in this order. Each step runs only when the record is still over the limit after the previous step:

  1. Operation results, oldest operation first. Each operation that loses its result is marked Truncated. This step can clear more results than the limit requires. That happens only when the rendering carries the operations array and a cleared result is shorter than 7 bytes.
  2. Whole operations, oldest first. A binary search finds a count that fits. This step then checks the count just below each point where a name's error or result would appear in operationsByName, and keeps the smallest count that fits. Near the limit it can drop more operations than needed, because DroppedOperations or a name's minDurationMs can gain a digit. DroppedOperations counts the dropped operations.
  3. The execution input. DroppedInput is set.
  4. The execution output. DroppedOutput is set.

"Oldest" is the order of the Operations slice. The record's Truncated field is set when anything was removed.

Exporter Default Limit
LambdaLog 256 KB
CloudWatch Logs 256 KB
SQS 256 KB
EventBridge 256 KB
DynamoDB 400 KB
Firehose 1 MB
RDS/Aurora 1 MB
S3 5 MB
OpenSearch 10 MB

All limits are configurable via the exporter's MaxSizeBytes field.

License

Apache-2.0. See LICENSE.

Documentation

Overview

Package telemetry provides observability record types for the AWS Durable Execution SDK. It defines the core schema for durable execution telemetry records that capture execution snapshots for diagnostics and monitoring.

This package is EXPERIMENTAL. It is built on the experimental plugin API of the durable package, so the telemetry module stays at major version 0. Any release may change this package's API and its record schema.

Index

Examples

Constants

View Source
const DefaultEventBridgeSource = "software.amazon.lambda.durable.telemetry"

DefaultEventBridgeSource is the default EventBridge event Source.

View Source
const DefaultHTTPExporterTimeout = 10 * time.Second

DefaultHTTPExporterTimeout is the default request timeout for HTTPExporter when no custom client or timeout is specified.

View Source
const DefaultMaxRecordSizeBytesDynamoDB = 400 * 1024

DefaultMaxRecordSizeBytesDynamoDB is the default maximum record size in bytes for the DynamoDBExporter (DynamoDB 400 KB item limit).

View Source
const DefaultMaxRecordSizeBytesFirehose = 1 * 1024 * 1024

DefaultMaxRecordSizeBytesFirehose is the default maximum record size in bytes for the FirehoseExporter (1 MB).

View Source
const DefaultMaxRecordSizeBytesLog = 256 * 1024

DefaultMaxRecordSizeBytesLog is the default maximum record size in bytes for the LambdaLogExporter (256 KB minus overhead).

View Source
const DefaultMaxRecordSizeBytesOpenSearch = 10 * 1024 * 1024

DefaultMaxRecordSizeBytesOpenSearch is the default maximum record size in bytes for the OpenSearchExporter (practical 10 MB limit).

View Source
const DefaultMaxRecordSizeBytesRDS = 1 * 1024 * 1024

DefaultMaxRecordSizeBytesRDS is the default maximum record size in bytes for the RDSExporter (Data API 1 MB request limit).

View Source
const DefaultMaxRecordSizeBytesS3 = 5 * 1024 * 1024

DefaultMaxRecordSizeBytesS3 is the default maximum record size in bytes for the S3Exporter (5 MB).

View Source
const RecordType = "DurableExecutionTelemetry"

RecordType is the discriminator value on every emitted record.

View Source
const SchemaVersion = "1.0"

SchemaVersion is the fixed schema version for telemetry records.

Variables

View Source
var NoSortKey = new(string)

NoSortKey is a *string pointing at the empty string, for use as DynamoDBExporter.SortKeyField to explicitly disable the sort key.

Functions

func DefaultFileNamer

func DefaultFileNamer(record *Record) string

DefaultFileNamer returns a filename based on the SHA-256 hash of the execution ARN and the current timestamp in UnixNano: "{hash8}_{timestamp}.json".

func GroupOperationsByName

func GroupOperationsByName(ops []OperationRecord) map[string]OperationSummary

GroupOperationsByName aggregates operations into a by-name view. Operations with an empty Name are excluded. The aggregation rules are:

  • Count, FailedCount, MaxAttempt, and duration metrics are aggregated across all occurrences sharing a name.
  • Type, SubType, and Status reflect the last occurrence encountered.
  • Error and Result are included only for names that occurred exactly once.

func RenderRecord

func RenderRecord(record Record, format OperationsFormat) ([]byte, error)

RenderRecord marshals a Record to JSON with operations formatted according to the specified format. When format is empty, it defaults to OperationsFormatArray.

Types

type CloudWatchLogsAPI

type CloudWatchLogsAPI interface {
	PutLogEvents(ctx context.Context, params *cloudwatchlogs.PutLogEventsInput, optFns ...func(*cloudwatchlogs.Options)) (*cloudwatchlogs.PutLogEventsOutput, error)
}

CloudWatchLogsAPI is the subset of *cloudwatchlogs.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.

type CloudWatchLogsExporter

type CloudWatchLogsExporter struct {
	// API is the underlying CloudWatch Logs client. Required.
	API CloudWatchLogsAPI

	// LogGroupName is the target log group. Required.
	LogGroupName string

	// LogStreamPrefix prefixes the daily log stream name. Defaults to
	// "durable-execution-telemetry/" when empty.
	LogStreamPrefix string

	// MaxSizeBytes overrides DefaultMaxRecordSizeBytesLog. Left at its
	// zero value, MaxRecordSizeBytes() returns DefaultMaxRecordSizeBytesLog
	// instead.
	MaxSizeBytes int
	// contains filtered or unexported fields
}

CloudWatchLogsExporter writes records to a specific CloudWatch Logs log group via PutLogEvents. Unlike LambdaLogExporter (which relies on Lambda's automatic stdout capture), this exporter lets the caller centralize records from multiple functions into one shared, queryable log group.

Log stream naming: "{LogStreamPrefix}{YYYY}/{MM}/{DD}" — one stream per UTC calendar day, auto-created on first write.

Each log event carries the record with its operations rendered as operationsByName (see OperationsFormatByName), never as the operations array. CloudWatch Logs Insights can read a field by a dotted path, such as operationsByName.charge.status, but it cannot select the array element whose name is "charge". So the format is fixed and has no option.

func (*CloudWatchLogsExporter) Export

func (e *CloudWatchLogsExporter) Export(ctx context.Context, record Record) error

Export writes record as a single log event to this exporter's target log group, in the daily log stream for the current UTC date — creating that stream first if this is the first Export call for that date.

func (*CloudWatchLogsExporter) MaxRecordSizeBytes

func (e *CloudWatchLogsExporter) MaxRecordSizeBytes() int

MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesLog when unset.

func (*CloudWatchLogsExporter) Render

func (e *CloudWatchLogsExporter) Render(record Record) ([]byte, error)

Render implements RenderingSizeLimiter. It returns the log event message that Export sends: the record rendered with OperationsFormatByName. Export calls Render, so truncation sizes the bytes that are sent.

type Config

type Config struct {
	// EmitMode controls when a record is sent to Exporters. Defaults to
	// EmitOnComplete when left at its zero value.
	EmitMode EmitMode

	// OperationDetail controls which operations appear in the record.
	// Defaults to OperationDetailTopLevel.
	OperationDetail OperationDetail

	// SamplingRate is the fraction of executions (0.0-1.0) that emit
	// records. Defaults to 1.0 (every execution) when left at zero.
	// The decision is per-execution and deterministic (derived from a
	// hash of ExecutionArn).
	SamplingRate float64

	// Exporters is where records are sent in parallel. Defaults to a
	// single LambdaLogExporter when left nil/empty.
	Exporters []Exporter

	// Content controls what data is included in emitted records.
	Content ContentConfig
}

Config configures the Durable Execution Telemetry plugin.

type ContentConfig

type ContentConfig struct {
	// ExcludeInput omits the execution input from records.
	ExcludeInput bool

	// InputTransform, if non-nil, receives the execution input decoded as
	// a JSON value and returns the value to record. Use it to remove or
	// mask sensitive fields. It is not called when ExcludeInput is true.
	InputTransform func(any) any

	// ExcludeOutput omits the execution output from records.
	ExcludeOutput bool

	// OutputTransform, if non-nil, receives the execution output decoded
	// as a JSON value and returns the value to record. It is not called
	// when ExcludeOutput is true.
	OutputTransform func(any) any

	// Operations controls per-operation content.
	Operations ContentOperations
}

ContentConfig controls what content appears in emitted records. The zero value includes the execution input and output unchanged, omits operation results, and includes operation and execution errors.

Content values (Record.Input, Record.Output, OperationRecord.Result) are JSON values. There is no per-field length cap. An exporter that implements SizeLimiter bounds the whole record instead (see Truncate).

A transform function receives the content decoded as a JSON value: a map[string]any, []any, string, json.Number, bool, or nil. Numbers are json.Number rather than float64 so that integers above 2^53 keep their exact digits through the transform; call Int64 or Float64 on the value to do arithmetic. The transform returns the value to record, which is marshaled again. Marshaling writes no whitespace and sorts the keys of every object, so the recorded text can differ from the original text in key order. The characters <, > and & are written literally, not as \u003c, \u003e and \u0026. If a transform panics, or its return value cannot be marshaled, the field is omitted. Transforms may be called concurrently by different invocations. Each function and its captured state must be safe for concurrent use.

Transforms run only for sampled-in invocations; see Config.SamplingRate. InputTransform and OutputTransform run at most once per invocation. A Result transform runs once for an operation's result in an invocation, and again only if that result text changes. Each new invocation replays checkpointed results, so it calls the Result transform again. Transforms run while the plugin holds its per-invocation lock, which delays that invocation's other hooks. So a transform should return quickly and must not block.

type ContentOperations

type ContentOperations struct {
	// Overrides are matched to operations by OperationName. When several
	// entries share a name, the last one wins.
	Overrides []OperationOverride

	// ExcludeErrors omits the error field from every operation entry. The
	// execution-level error on the record is always included.
	ExcludeErrors bool
}

ContentOperations controls the content of operation entries.

type DynamoDBAPI

type DynamoDBAPI interface {
	PutItem(ctx context.Context, params *dynamodb.PutItemInput, optFns ...func(*dynamodb.Options)) (*dynamodb.PutItemOutput, error)
}

DynamoDBAPI is the subset of *dynamodb.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.

type DynamoDBExporter

type DynamoDBExporter struct {
	// API is the underlying DynamoDB client. Required.
	API DynamoDBAPI

	// TableName is the target table. Required.
	TableName string

	// PartitionKeyField is the partition-key attribute name. Defaults to
	// "pk" when empty.
	PartitionKeyField string

	// SortKeyField is the sort-key attribute name, or nil to accept the
	// default ("sk"). Set to NoSortKey (a non-nil pointer to "") to
	// explicitly disable the sort key.
	SortKeyField *string

	// MaxSizeBytes overrides DefaultMaxRecordSizeBytesDynamoDB. Left at
	// its zero value, MaxRecordSizeBytes() returns
	// DefaultMaxRecordSizeBytesDynamoDB instead.
	MaxSizeBytes int
}

DynamoDBExporter writes records to Amazon DynamoDB via PutItem.

Partition key value is the record's ExecutionArn. Sort key value (when present) is the record's EmittedAt timestamp in the record's own timestamp format (UTC, millisecond precision, Z suffix) — so a full history of every record ever emitted for a given execution accumulates as separate items sharing the same partition key.

Set SortKey to NoSortKey to explicitly disable the sort key — PutItem then overwrites the single item for a given partition key on every call (only the latest state is kept).

func (*DynamoDBExporter) Export

func (e *DynamoDBExporter) Export(ctx context.Context, record Record) error

Export writes record as a DynamoDB item.

func (*DynamoDBExporter) MaxRecordSizeBytes

func (e *DynamoDBExporter) MaxRecordSizeBytes() int

MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesDynamoDB when unset.

type EmitMode

type EmitMode string

EmitMode controls when the telemetry plugin emits records.

There is no failure-only mode: use EmitOnComplete for terminal-only emission and filter downstream if needed.

const (
	// EmitAlways emits a record after every invocation regardless of
	// outcome.
	EmitAlways EmitMode = "always"

	// EmitOnChange emits a running snapshot on every operation start/end
	// in addition to the terminal record.
	EmitOnChange EmitMode = "on-change"

	// EmitOnComplete emits a record only when the execution reaches a
	// terminal state (succeeded or failed).
	EmitOnComplete EmitMode = "on-complete"
)

type ErrorRecord

type ErrorRecord struct {
	Type       string `json:"type"`
	Message    string `json:"message"`
	StackTrace string `json:"stackTrace,omitempty"`
}

ErrorRecord carries an error's type and message.

type EventBridgeAPI

type EventBridgeAPI interface {
	PutEvents(ctx context.Context, params *eventbridge.PutEventsInput, optFns ...func(*eventbridge.Options)) (*eventbridge.PutEventsOutput, error)
}

EventBridgeAPI is the subset of *eventbridge.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.

type EventBridgeExporter

type EventBridgeExporter struct {
	// API is the underlying EventBridge client. Required.
	API EventBridgeAPI

	// EventBusName is the target event bus. Defaults to "default" when
	// empty.
	EventBusName string

	// Source is the event Source field. Defaults to
	// DefaultEventBridgeSource when empty.
	Source string

	// OperationsFormat controls how operations are rendered within
	// Detail. Defaults to OperationsFormatArray when left empty.
	OperationsFormat OperationsFormat

	// MaxSizeBytes overrides DefaultMaxRecordSizeBytesLog. Left at its
	// zero value, MaxRecordSizeBytes() returns DefaultMaxRecordSizeBytesLog
	// instead.
	MaxSizeBytes int
}

EventBridgeExporter publishes records to Amazon EventBridge for event-driven reactions.

Source defaults to DefaultEventBridgeSource; DetailType is the record's Status (SUCCEEDED/FAILED/RUNNING); Detail is the full rendered record JSON per OperationsFormat.

func (*EventBridgeExporter) Export

func (e *EventBridgeExporter) Export(ctx context.Context, record Record) error

Export publishes record as a single EventBridge event.

func (*EventBridgeExporter) MaxRecordSizeBytes

func (e *EventBridgeExporter) MaxRecordSizeBytes() int

MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesLog when unset.

func (*EventBridgeExporter) Render

func (e *EventBridgeExporter) Render(record Record) ([]byte, error)

Render implements RenderingSizeLimiter, returning the exact bytes Export would send as the event's Detail field.

type Exporter

type Exporter interface {
	Export(ctx context.Context, record Record) error
}

Exporter is the interface a Durable Execution Telemetry destination implements. Export is called with the record whenever a new snapshot is ready to send, per the configured EmitMode.

Export errors are swallowed by the plugin (never fail the execution or affect other configured exporters).

type FileExporter

type FileExporter struct {
	// contains filtered or unexported fields
}

FileExporter writes each record as a JSON file to a configurable directory on the local filesystem. It implements Exporter and requires no AWS dependencies — only the standard library.

func NewFileExporter

func NewFileExporter(cfg FileExporterConfig) *FileExporter

NewFileExporter returns a FileExporter configured by cfg. Panics if cfg.Dir is empty.

func (*FileExporter) Close

func (e *FileExporter) Close() error

Close is a no-op — FileExporter holds no persistent connections.

func (*FileExporter) Export

func (e *FileExporter) Export(_ context.Context, record Record) error

Export writes record as a JSON file in the configured directory. The directory is created if it does not exist.

type FileExporterConfig

type FileExporterConfig struct {
	// Dir is the directory where record files are written. Required.
	// Created via os.MkdirAll if it does not exist.
	Dir string

	// FileNamer returns the filename (without directory) for a given
	// record. If nil, DefaultFileNamer is used.
	FileNamer func(record *Record) string

	// OperationsFormat controls how operations are rendered in the JSON
	// output. Defaults to OperationsFormatArray when empty.
	OperationsFormat OperationsFormat
}

FileExporterConfig configures a FileExporter.

type FirehoseAPI

type FirehoseAPI interface {
	PutRecord(ctx context.Context, params *firehose.PutRecordInput, optFns ...func(*firehose.Options)) (*firehose.PutRecordOutput, error)
}

FirehoseAPI is the subset of *firehose.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.

type FirehoseExporter

type FirehoseExporter struct {
	// API is the underlying Firehose client. Required.
	API FirehoseAPI

	// DeliveryStreamName is the target delivery stream. Required.
	DeliveryStreamName string

	// OperationsFormat controls how operations are rendered. Defaults to
	// OperationsFormatArray when left empty.
	OperationsFormat OperationsFormat

	// MaxSizeBytes overrides DefaultMaxRecordSizeBytesFirehose. Left at
	// its zero value, MaxRecordSizeBytes() returns
	// DefaultMaxRecordSizeBytesFirehose instead.
	MaxSizeBytes int
}

FirehoseExporter sends records to Amazon Kinesis Data Firehose for delivery to S3, Redshift, Splunk, or any HTTP endpoint the delivery stream is configured with.

Each record is appended with a trailing newline (NDJSON), so records remain individually parseable once Firehose batches multiple PutRecord calls into a single S3 object.

func (*FirehoseExporter) Export

func (e *FirehoseExporter) Export(ctx context.Context, record Record) error

Export sends record as a single NDJSON-formatted PutRecord call.

func (*FirehoseExporter) MaxRecordSizeBytes

func (e *FirehoseExporter) MaxRecordSizeBytes() int

MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesFirehose when unset.

func (*FirehoseExporter) Render

func (e *FirehoseExporter) Render(record Record) ([]byte, error)

Render implements RenderingSizeLimiter, returning the exact bytes Export would send as Record.Data (including the trailing NDJSON newline).

type Flusher

type Flusher interface {
	Flush(ctx context.Context) error
}

Flusher is an optional interface an Exporter may additionally implement to flush any buffered data before the invocation returns. Checked via a type assertion at dispatch time rather than being part of the required Exporter interface.

type HTTPDoer

type HTTPDoer interface {
	Do(req *http.Request) (*http.Response, error)
}

HTTPDoer is the interface for sending HTTP requests. *http.Client satisfies this interface.

type HTTPExporter

type HTTPExporter struct {
	// contains filtered or unexported fields
}

HTTPExporter POSTs JSON-serialized records to a configurable URL. It implements Exporter and requires no AWS dependencies — only the standard library net/http.

func NewHTTPExporter

func NewHTTPExporter(cfg HTTPExporterConfig) *HTTPExporter

NewHTTPExporter returns an HTTPExporter configured by cfg. Panics if cfg.URL is empty.

func (*HTTPExporter) Close

func (e *HTTPExporter) Close() error

Close is a no-op — HTTPExporter holds no persistent connections that require cleanup.

func (*HTTPExporter) Export

func (e *HTTPExporter) Export(ctx context.Context, record Record) error

Export sends record as JSON to the configured URL via HTTP POST. On 5xx responses it retries up to MaxRetries times with a simple exponential backoff (100ms, 200ms, ...).

type HTTPExporterConfig

type HTTPExporterConfig struct {
	// URL is the endpoint to POST records to. Required.
	URL string

	// Client is the HTTP client used for requests. If nil, a default
	// client with Timeout is used.
	Client HTTPDoer

	// Headers are added to every request (e.g. Authorization).
	Headers map[string]string

	// Timeout bounds a single HTTP request. Defaults to
	// DefaultHTTPExporterTimeout when zero and Client is nil. Ignored
	// when Client is provided (the caller's client owns its timeout).
	Timeout time.Duration

	// MaxRetries is the maximum number of retries on 5xx responses.
	// Defaults to 1 when zero.
	MaxRetries int

	// OperationsFormat controls how operations are rendered in the JSON
	// body. Defaults to OperationsFormatArray when empty.
	OperationsFormat OperationsFormat
}

HTTPExporterConfig configures an HTTPExporter.

type LambdaLogExporter

type LambdaLogExporter struct {
	// Writer overrides the default destination (os.Stdout). Primarily
	// for tests.
	Writer io.Writer

	// MaxSizeBytes overrides DefaultMaxRecordSizeBytesLog. Left at zero,
	// MaxRecordSizeBytes() returns DefaultMaxRecordSizeBytesLog.
	MaxSizeBytes int
}

LambdaLogExporter writes records as a single JSON line to stdout. Since Lambda captures stdout to the function's CloudWatch log group automatically, this requires zero IAM permissions and zero setup.

Each line carries the record with its operations rendered as operationsByName (see OperationsFormatByName), never as the operations array. The line lands in CloudWatch Logs, where Logs Insights can read a field by a dotted path but cannot select an array element by its name. So the format is fixed and has no option, as for CloudWatchLogsExporter.

func NewLambdaLogExporter

func NewLambdaLogExporter() *LambdaLogExporter

NewLambdaLogExporter returns a LambdaLogExporter writing to os.Stdout.

func (*LambdaLogExporter) Export

func (e *LambdaLogExporter) Export(_ context.Context, record Record) error

Export writes record as a single line of JSON to the configured Writer (os.Stdout if unset). It writes exactly Render's output.

func (*LambdaLogExporter) MaxRecordSizeBytes

func (e *LambdaLogExporter) MaxRecordSizeBytes() int

MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesLog when unset.

func (*LambdaLogExporter) Render

func (e *LambdaLogExporter) Render(record Record) ([]byte, error)

Render implements RenderingSizeLimiter. It returns the exact bytes Export writes: the record rendered with OperationsFormatByName, then one newline. Truncation sizes Render's output. So the size limit includes the newline.

type OpenSearchExporter

type OpenSearchExporter struct {
	// Client is the HTTP client used for requests. If nil,
	// http.DefaultClient is used.
	Client HTTPDoer

	// Endpoint is the cluster's base URL, e.g.
	// "https://search-my-domain.us-east-1.es.amazonaws.com". Required.
	Endpoint string

	// IndexName is the target index. Defaults to "durable-execution-telemetry"
	// when empty.
	IndexName string

	// Username and Password are used for HTTP Basic authentication.
	// Mutually exclusive with BearerToken.
	Username string
	Password string

	// BearerToken is used for Bearer token authentication. Mutually
	// exclusive with Username/Password.
	BearerToken string

	// MaxSizeBytes overrides DefaultMaxRecordSizeBytesOpenSearch. Left
	// at its zero value, MaxRecordSizeBytes() returns
	// DefaultMaxRecordSizeBytesOpenSearch instead.
	MaxSizeBytes int
}

OpenSearchExporter indexes records as documents in OpenSearch via HTTP POST to the _doc API. Supports basic auth (username/password) or bearer token authentication.

Document creation uses POST to {endpoint}/{index}/_doc, which lets OpenSearch assign a document ID automatically. Each export creates a new document — downstream queries can aggregate by executionArn.

func (*OpenSearchExporter) Export

func (e *OpenSearchExporter) Export(ctx context.Context, record Record) error

Export indexes record as an OpenSearch document.

func (*OpenSearchExporter) MaxRecordSizeBytes

func (e *OpenSearchExporter) MaxRecordSizeBytes() int

MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesOpenSearch when unset.

type OperationDetail

type OperationDetail string

OperationDetail controls which operations appear in a record's Operations array.

const (
	// OperationDetailTopLevel includes only top-level operations
	// (anything with a non-empty ParentID is dropped).
	OperationDetailTopLevel OperationDetail = "top-level"

	// OperationDetailFullTree includes every operation, including
	// children of contexts.
	OperationDetailFullTree OperationDetail = "full-tree"
)

type OperationOverride

type OperationOverride struct {
	// OperationName selects the operations this override applies to.
	OperationName string

	// Exclude drops the operation from the record.
	Exclude bool

	// Result opts the operation's result into the record. Results are
	// omitted unless an override sets this. The function receives the
	// checkpointed result decoded as a JSON value, or the raw text as a
	// string when that text is not valid JSON. The plugin does not run
	// the operation's own serializer, so an operation with a custom
	// Serdes delivers its serialized form here. To record a result
	// unchanged, return the argument. It is not called when Exclude is true.
	Result func(any) any
}

OperationOverride changes the content of every operation with the given name.

type OperationRecord

type OperationRecord struct {
	ID       string `json:"id"`
	Name     string `json:"name,omitempty"`
	Type     string `json:"type"`
	SubType  string `json:"subType,omitempty"`
	Status   string `json:"status"`
	ParentID string `json:"parentId,omitempty"`

	// StartTimestamp and EndTimestamp serialize as the JSON keys
	// startTime and endTime. See Record.MarshalJSON for the format.
	StartTimestamp *time.Time `json:"startTime,omitempty"`
	EndTimestamp   *time.Time `json:"endTime,omitempty"`
	DurationMs     *int64     `json:"durationMs,omitempty"`

	Attempt int `json:"attempt,omitempty"`

	// Result is the operation's result as a JSON value. A result is
	// recorded only through an OperationOverride.Result transform. The
	// transform receives the checkpointed wire text decoded as JSON, or
	// the text as a string when it is not valid JSON. Its return value is
	// marshaled again. So the recorded text has no whitespace and has
	// sorted object keys, even when the transform returns its argument.
	// See ContentConfig.
	Result json.RawMessage `json:"result,omitempty"`
	Error  *ErrorRecord    `json:"error,omitempty"`

	// Truncated indicates this operation's result was dropped due to
	// size constraints (see Truncate).
	Truncated bool `json:"truncated,omitempty"`
}

OperationRecord describes a single durable operation within a Record.

func (OperationRecord) MarshalJSON

func (o OperationRecord) MarshalJSON() ([]byte, error)

MarshalJSON writes the operation timestamps in timestampLayout. See Record.MarshalJSON for the mechanism.

type OperationSummary

type OperationSummary struct {
	Type    string `json:"type"`
	SubType string `json:"subType,omitempty"`

	Count int `json:"count"`

	// The duration fields are nil when no occurrence carried a DurationMs,
	// so the JSON omits them; a recorded duration of 0 ms serializes as 0.
	// This matches the JavaScript implementation.
	MinDurationMs   *int64 `json:"minDurationMs,omitempty"`
	MaxDurationMs   *int64 `json:"maxDurationMs,omitempty"`
	TotalDurationMs *int64 `json:"totalDurationMs,omitempty"`

	FailedCount int `json:"failedCount"`
	MaxAttempt  int `json:"maxAttempt,omitempty"`

	// Status reflects the most recently seen occurrence's status.
	Status string `json:"status"`

	// Error is populated only when the name occurred exactly once and
	// the operation has an error. For repeated names (loops, retries),
	// it is left nil; FailedCount still flags failures.
	Error *ErrorRecord `json:"error,omitempty"`

	// Result is populated only when the name occurred exactly once and
	// the operation has a result.
	Result json.RawMessage `json:"result,omitempty"`
}

OperationSummary aggregates every occurrence of a given operation name within a Record's Operations slice. The CloudWatch Logs and Lambda log exporters always render it. Other exporters render it when configured with OperationsFormatByName or OperationsFormatBoth. DynamoDBExporter does not render it: it writes the Operations array inside recordJson.

type OperationsFormat

type OperationsFormat string

OperationsFormat controls how an exporter renders a record's operations.

const (
	// OperationsFormatArray renders only the canonical Operations array.
	// This is the default.
	OperationsFormatArray OperationsFormat = "array"

	// OperationsFormatByName renders only the by-name aggregation,
	// omitting the Operations array.
	OperationsFormatByName OperationsFormat = "by-name"

	// OperationsFormatBoth renders both the Operations array and the
	// by-name aggregation.
	OperationsFormatBoth OperationsFormat = "both"
)

type RDSDataAPI

type RDSDataAPI interface {
	ExecuteStatement(ctx context.Context, params *rdsdata.ExecuteStatementInput, optFns ...func(*rdsdata.Options)) (*rdsdata.ExecuteStatementOutput, error)
}

RDSDataAPI is the subset of *rdsdata.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.

type RDSExporter

type RDSExporter struct {
	// API is the underlying RDS Data API client. Required.
	API RDSDataAPI

	// ResourceArn is the target Aurora cluster's ARN. Required.
	ResourceArn string

	// SecretArn is the Secrets Manager secret ARN holding the database
	// credentials. Required.
	SecretArn string

	// Database is the target database name. Required.
	Database string

	// TableName is the target table name. Defaults to
	// "durable_execution_telemetry" when empty.
	TableName string

	// MaxSizeBytes overrides DefaultMaxRecordSizeBytesRDS. Left at its
	// zero value, MaxRecordSizeBytes() returns
	// DefaultMaxRecordSizeBytesRDS instead.
	MaxSizeBytes int
}

RDSExporter writes records to an Aurora/RDS cluster via the RDS Data API's ExecuteStatement (HTTP-based — no VPC required).

Record values are always bound via SqlParameters. The table name is the one value embedded in the statement text (identifiers cannot be bound as parameters) and is validated as a plain SQL identifier before use. Uses INSERT INTO with named parameters for the four columns: execution_arn, emitted_at, status, record_json.

func (*RDSExporter) Export

func (e *RDSExporter) Export(ctx context.Context, record Record) error

Export inserts record into the target RDS/Aurora table via a parameterized INSERT statement.

func (*RDSExporter) MaxRecordSizeBytes

func (e *RDSExporter) MaxRecordSizeBytes() int

MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesRDS when unset.

type Record

type Record struct {
	RecordType    string `json:"recordType"`
	SchemaVersion string `json:"schemaVersion"`

	ExecutionArn  string `json:"executionArn"`
	ExecutionName string `json:"executionName,omitempty"`

	// FunctionName, FunctionQualifier, Region, and AccountID are parsed
	// from the execution ARN. They are always emitted, empty when the ARN
	// does not carry them, so every record has the same identity keys.
	FunctionName      string `json:"functionName"`
	FunctionQualifier string `json:"functionQualifier"`
	Region            string `json:"region"`
	AccountID         string `json:"accountId"`

	Status Status `json:"status"`

	// StartTimestamp and EndTimestamp serialize as the JSON keys
	// startTime and endTime. See MarshalJSON for the format.
	StartTimestamp *time.Time `json:"startTime,omitempty"`
	EndTimestamp   *time.Time `json:"endTime,omitempty"`
	DurationMs     *int64     `json:"durationMs,omitempty"`

	// EmittedAt is the time this record was created, refreshed to the
	// emission time when a snapshot is sent.
	EmittedAt time.Time `json:"emittedAt"`

	Operations []OperationRecord `json:"operations"`

	Error *ErrorRecord `json:"error,omitempty"`

	// Input and Output are the execution input and result as JSON values,
	// marshaled once from the handler's typed values. A Go string is a
	// JSON string. See ContentConfig.
	Input  json.RawMessage `json:"input,omitempty"`
	Output json.RawMessage `json:"output,omitempty"`

	Custom map[string]any `json:"custom,omitempty"`

	// Truncation metadata: set when the record was truncated to fit an
	// exporter's size limit (see Truncate).
	Truncated         bool `json:"truncated,omitempty"`
	DroppedOperations int  `json:"droppedOperations,omitempty"`
	DroppedInput      bool `json:"droppedInput,omitempty"`
	DroppedOutput     bool `json:"droppedOutput,omitempty"`
}

Record is the cumulative snapshot a telemetry plugin builds for a single durable execution. It is handed to each configured exporter on emission.

func NewRecord

func NewRecord(executionArn string) *Record

NewRecord creates a Record with standard metadata pre-populated. EmittedAt is set to time.Now() and ExecutionName is extracted from the ARN.

func Truncate

func Truncate(record Record, maxBytes int, render Renderer) Record

Truncate returns a copy of record, best-effort truncated to fit within maxBytes when rendered via render (or canonical JSON if render is nil). Drop order: operation results (oldest first), whole operations (oldest first), then Input, then Output. Identity/timeline fields are never dropped.

When maxBytes <= 0, returns record unchanged (truncation disabled).

func (Record) MarshalJSON

func (r Record) MarshalJSON() ([]byte, error)

MarshalJSON writes every timestamp in timestampLayout. encoding/json would otherwise call time.Time.MarshalJSON, which uses RFC3339Nano.

The method marshals an alias of Record with shadow string fields that carry the same JSON keys as the time.Time fields. encoding/json keeps the shallower field when two fields share a key, so the shadow strings replace the time values in the output. The Go field types stay time.Time, and json.Unmarshal into Record still works because time.Time parses the three-digit form.

The method writes <, > and & literally. A caller's own json.Marshal call on a Record escapes them as \u003c, \u003e and \u0026, because json.Marshal escapes the bytes a json.Marshaler returns. To keep them literal, use RenderRecord or a json.Encoder with SetEscapeHTML(false).

type RedshiftDataAPI

type RedshiftDataAPI interface {
	ExecuteStatement(ctx context.Context, params *redshiftdata.ExecuteStatementInput, optFns ...func(*redshiftdata.Options)) (*redshiftdata.ExecuteStatementOutput, error)
}

RedshiftDataAPI is the subset of *redshiftdata.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.

type RedshiftExporter

type RedshiftExporter struct {
	// API is the underlying Redshift Data API client. Required.
	API RedshiftDataAPI

	// WorkgroupName selects Serverless mode. Mutually exclusive with
	// ClusterIdentifier — set exactly one.
	WorkgroupName string

	// ClusterIdentifier selects provisioned-cluster mode. Mutually
	// exclusive with WorkgroupName — set exactly one.
	ClusterIdentifier string

	// Database is the target database name. Required.
	Database string

	// TableName is the target table name. Defaults to
	// "durable_execution_telemetry" when empty.
	TableName string
}

RedshiftExporter writes records to Amazon Redshift via the Redshift Data API's ExecuteStatement (HTTP-based — no VPC required).

Supports two connection modes:

  • Serverless: set WorkgroupName.
  • Provisioned cluster: set ClusterIdentifier.

Record values are always bound via SqlParameters. The table name is the one value embedded in the statement text (identifiers cannot be bound as parameters) and is validated as a plain SQL identifier before use.

Fire-and-forget: does NOT poll for statement completion.

func (*RedshiftExporter) Export

func (e *RedshiftExporter) Export(ctx context.Context, record Record) error

Export inserts record into the target Redshift table via a parameterized INSERT statement.

type Renderer

type Renderer func(Record) ([]byte, error)

Renderer produces the exact byte shape a Truncate call should size against. Exporters with custom rendering (OperationsFormat) provide their own Renderer so truncation sizes against what is actually sent.

type RenderingSizeLimiter

type RenderingSizeLimiter interface {
	SizeLimiter
	Render(record Record) ([]byte, error)
}

RenderingSizeLimiter is SizeLimiter's companion interface for an exporter whose Export does not send the plain json.Marshal form of the record, such as an exporter with an OperationsFormat option or one that always renders operationsByName. Render must return the exact byte shape this exporter's Export call would actually send, so truncation sizes against the real rendered form.

type S3API

type S3API interface {
	PutObject(ctx context.Context, params *s3.PutObjectInput, optFns ...func(*s3.Options)) (*s3.PutObjectOutput, error)
}

S3API is the subset of *s3.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.

type S3Exporter

type S3Exporter struct {
	// API is the underlying S3 client. Required.
	API S3API

	// Bucket is the target S3 bucket. Required.
	Bucket string

	// Prefix is prepended to every object key. Defaults to
	// "durable-execution-telemetry/" when empty.
	Prefix string

	// Partitioning controls the partition segment inserted between
	// Prefix and the object's own filename. Defaults to
	// S3PartitioningDate when left at its zero value.
	Partitioning S3Partitioning

	// MaxSizeBytes overrides DefaultMaxRecordSizeBytesS3. Left at its
	// zero value, MaxRecordSizeBytes() returns DefaultMaxRecordSizeBytesS3
	// instead.
	MaxSizeBytes int
}

S3Exporter writes records as JSON objects to Amazon S3. Uses the record's own ExecutionName (falling back to a sanitized ExecutionArn when ExecutionName is empty) as the object key, so a later update to the same execution overwrites the same object rather than creating a new one each time.

func (*S3Exporter) Export

func (e *S3Exporter) Export(ctx context.Context, record Record) error

Export writes record as a JSON object to this exporter's bucket, at a key determined by Prefix/Partitioning/the record's own identity.

func (*S3Exporter) MaxRecordSizeBytes

func (e *S3Exporter) MaxRecordSizeBytes() int

MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesS3 when unset.

type S3Partitioning

type S3Partitioning string

S3Partitioning controls how S3Exporter keys objects.

const (
	// S3PartitioningDate keys objects under Hive-style
	// year=YYYY/month=MM/day=DD/ partitions. The default.
	S3PartitioningDate S3Partitioning = "date"

	// S3PartitioningFunctionName keys objects under the record's own
	// FunctionName instead of a date.
	S3PartitioningFunctionName S3Partitioning = "function-name"

	// S3PartitioningNone applies no partitioning prefix at all beyond
	// Prefix itself.
	S3PartitioningNone S3Partitioning = "none"
)

type SQSAPI

type SQSAPI interface {
	SendMessage(ctx context.Context, params *sqs.SendMessageInput, optFns ...func(*sqs.Options)) (*sqs.SendMessageOutput, error)
}

SQSAPI is the subset of *sqs.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.

type SQSExporter

type SQSExporter struct {
	// API is the underlying SQS client. Required.
	API SQSAPI

	// QueueURL is the target queue. Required. A ".fifo" suffix switches
	// this exporter into FIFO mode automatically.
	QueueURL string

	// MessageGroupID overrides the FIFO message group ID. Only
	// meaningful when QueueURL ends in ".fifo". Defaults to the
	// record's ExecutionArn when empty.
	MessageGroupID string

	// OperationsFormat controls how operations are rendered within the
	// message body. Defaults to OperationsFormatArray when left empty.
	OperationsFormat OperationsFormat

	// MaxSizeBytes overrides DefaultMaxRecordSizeBytesLog. Left at its
	// zero value, MaxRecordSizeBytes() returns DefaultMaxRecordSizeBytesLog
	// instead.
	MaxSizeBytes int
}

SQSExporter sends records to Amazon SQS (standard or FIFO) via SendMessage.

FIFO mode is auto-detected from a ".fifo" suffix on QueueURL. In FIFO mode, MessageGroupID defaults to the record's ExecutionArn and MessageDeduplicationId is set to "{ExecutionArn}:{EmittedAt}", with EmittedAt in the record's own timestamp format (UTC, millisecond precision, Z suffix).

Message attributes "status" and "functionName" are always set, enabling SQS message filtering without body parsing.

func (*SQSExporter) Export

func (e *SQSExporter) Export(ctx context.Context, record Record) error

Export sends record as a single SQS message.

func (*SQSExporter) MaxRecordSizeBytes

func (e *SQSExporter) MaxRecordSizeBytes() int

MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesLog when unset.

func (*SQSExporter) Render

func (e *SQSExporter) Render(record Record) ([]byte, error)

Render implements RenderingSizeLimiter, returning the exact bytes Export would send as MessageBody.

type SizeLimiter

type SizeLimiter interface {
	MaxRecordSizeBytes() int
}

SizeLimiter is an optional interface an Exporter may additionally implement to opt into size-based truncation. When an Exporter implements SizeLimiter, the plugin sends that exporter its own, independently truncated copy of the record.

type Status

type Status string

Status represents the execution-level status of a durable workflow.

const (
	StatusRunning   Status = "RUNNING"
	StatusSucceeded Status = "SUCCEEDED"
	StatusFailed    Status = "FAILED"
)

type TelemetryPlugin

type TelemetryPlugin struct {
	// contains filtered or unexported fields
}

TelemetryPlugin is the Durable Execution Telemetry plugin factory. Construct via New; do not construct the zero value directly.

func New

func New(cfg Config) *TelemetryPlugin

New constructs a Durable Execution Telemetry plugin from cfg. A zero-value Config produces documented defaults: EmitOnComplete, SamplingRate 1.0, a single LambdaLogExporter.

Example

ExampleNew demonstrates creating a telemetry plugin with default configuration: emits on completion, 100% sampling, stdout JSON via LambdaLogExporter.

package main

import (
	"context"
	"fmt"

	"github.com/aws/aws-durable-execution-sdk-go/durable"
	"github.com/aws/aws-durable-execution-sdk-go/telemetry"
)

func main() {
	plugin := telemetry.New(telemetry.Config{})

	// Register with durable.Start via durable.WithPlugins(plugin). The
	// handler calls CreatePlugin once per invocation.
	p, err := plugin.CreatePlugin(context.Background(), durable.InvocationHookInfo{})
	fmt.Println(err == nil && p.OnInvocationStart != nil)
}
Output:
true
Example (WithContentConfig)

ExampleNew_withContentConfig demonstrates shaping record content: a sensitive input field is removed, the output is omitted, one operation opts its result in, and another is dropped from the record.

package main

import (
	"context"
	"fmt"

	"github.com/aws/aws-durable-execution-sdk-go/durable"
	"github.com/aws/aws-durable-execution-sdk-go/telemetry"
)

func main() {
	plugin := telemetry.New(telemetry.Config{
		Content: telemetry.ContentConfig{
			InputTransform: func(v any) any {
				if m, ok := v.(map[string]any); ok {
					delete(m, "password")
				}
				return v
			},
			ExcludeOutput: true,
			Operations: telemetry.ContentOperations{
				Overrides: []telemetry.OperationOverride{
					{OperationName: "fetch-user", Result: func(v any) any { return v }},
					{OperationName: "debug-dump", Exclude: true},
				},
			},
		},
	})

	p, _ := plugin.CreatePlugin(context.Background(), durable.InvocationHookInfo{})
	fmt.Println(p.OnOperationEnd != nil)
}
Output:
true
Example (WithMultipleExporters)

ExampleNew_withMultipleExporters demonstrates combining a file exporter (for local debugging) with an S3 exporter (for durable storage).

package main

import (
	"context"
	"fmt"

	"github.com/aws/aws-durable-execution-sdk-go/durable"
	"github.com/aws/aws-durable-execution-sdk-go/telemetry"
)

func main() {
	fileExp := telemetry.NewFileExporter(telemetry.FileExporterConfig{
		Dir: "/tmp/telemetry-records",
	})

	s3Exp := &telemetry.S3Exporter{
		API:    nil, // replace with s3.NewFromConfig(cfg)
		Bucket: "my-telemetry-bucket",
	}

	plugin := telemetry.New(telemetry.Config{
		Exporters: []telemetry.Exporter{fileExp, s3Exp},
	})

	p, _ := plugin.CreatePlugin(context.Background(), durable.InvocationHookInfo{})
	fmt.Println(p.OnOperationStart != nil)
}
Output:
true
Example (WithS3Exporter)

ExampleNew_withS3Exporter demonstrates configuring an S3 exporter with date-based partitioning.

package main

import (
	"context"
	"fmt"

	"github.com/aws/aws-durable-execution-sdk-go/durable"
	"github.com/aws/aws-durable-execution-sdk-go/telemetry"
)

func main() {
	// In production, use a real *s3.Client from aws-sdk-go-v2/service/s3.
	// nil is used here to show the configuration shape without making
	// AWS calls.
	exporter := &telemetry.S3Exporter{
		API:          nil, // replace with s3.NewFromConfig(cfg)
		Bucket:       "my-telemetry-bucket",
		Prefix:       "durable-execution-telemetry/",
		Partitioning: telemetry.S3PartitioningDate,
	}

	plugin := telemetry.New(telemetry.Config{
		Exporters: []telemetry.Exporter{exporter},
	})

	p, _ := plugin.CreatePlugin(context.Background(), durable.InvocationHookInfo{})
	fmt.Println(p.OnInvocationEnd != nil)
}
Output:
true
Example (WithSampling)

ExampleNew_withSampling demonstrates configuring the plugin to emit records for only 10% of executions. The sampling decision is deterministic per execution ARN.

package main

import (
	"context"
	"fmt"

	"github.com/aws/aws-durable-execution-sdk-go/durable"
	"github.com/aws/aws-durable-execution-sdk-go/telemetry"
)

func main() {
	plugin := telemetry.New(telemetry.Config{
		SamplingRate: 0.1, // 10% of executions
		EmitMode:     telemetry.EmitOnComplete,
	})

	p, _ := plugin.CreatePlugin(context.Background(), durable.InvocationHookInfo{})
	fmt.Println(p.OnInvocationStart != nil)
}
Output:
true

func (*TelemetryPlugin) CreatePlugin

CreatePlugin returns hooks bound to fresh invocation-local state. Configuration and exporters remain shared on the factory. The record, sampling decision, mutex, and on-change dispatch state belong to the returned plugin alone.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL