Documentation
¶
Overview ¶
Package telemetry provides observability record types for the AWS Durable Execution SDK. It defines the core schema for durable execution telemetry records that capture execution snapshots for diagnostics and monitoring.
This package is EXPERIMENTAL. It is built on the experimental plugin API of the durable package, so the telemetry module stays at major version 0. Any release may change this package's API and its record schema.
Index ¶
- Constants
- Variables
- func DefaultFileNamer(record *Record) string
- func GroupOperationsByName(ops []OperationRecord) map[string]OperationSummary
- func RenderRecord(record Record, format OperationsFormat) ([]byte, error)
- type CloudWatchLogsAPI
- type CloudWatchLogsExporter
- type Config
- type ContentConfig
- type ContentOperations
- type DynamoDBAPI
- type DynamoDBExporter
- type EmitMode
- type ErrorRecord
- type EventBridgeAPI
- type EventBridgeExporter
- type Exporter
- type FileExporter
- type FileExporterConfig
- type FirehoseAPI
- type FirehoseExporter
- type Flusher
- type HTTPDoer
- type HTTPExporter
- type HTTPExporterConfig
- type LambdaLogExporter
- type OpenSearchExporter
- type OperationDetail
- type OperationOverride
- type OperationRecord
- type OperationSummary
- type OperationsFormat
- type RDSDataAPI
- type RDSExporter
- type Record
- type RedshiftDataAPI
- type RedshiftExporter
- type Renderer
- type RenderingSizeLimiter
- type S3API
- type S3Exporter
- type S3Partitioning
- type SQSAPI
- type SQSExporter
- type SizeLimiter
- type Status
- type TelemetryPlugin
Examples ¶
Constants ¶
const DefaultEventBridgeSource = "software.amazon.lambda.durable.telemetry"
DefaultEventBridgeSource is the default EventBridge event Source.
const DefaultHTTPExporterTimeout = 10 * time.Second
DefaultHTTPExporterTimeout is the default request timeout for HTTPExporter when no custom client or timeout is specified.
const DefaultMaxRecordSizeBytesDynamoDB = 400 * 1024
DefaultMaxRecordSizeBytesDynamoDB is the default maximum record size in bytes for the DynamoDBExporter (DynamoDB 400 KB item limit).
const DefaultMaxRecordSizeBytesFirehose = 1 * 1024 * 1024
DefaultMaxRecordSizeBytesFirehose is the default maximum record size in bytes for the FirehoseExporter (1 MB).
const DefaultMaxRecordSizeBytesLog = 256 * 1024
DefaultMaxRecordSizeBytesLog is the default maximum record size in bytes for the LambdaLogExporter (256 KB minus overhead).
const DefaultMaxRecordSizeBytesOpenSearch = 10 * 1024 * 1024
DefaultMaxRecordSizeBytesOpenSearch is the default maximum record size in bytes for the OpenSearchExporter (practical 10 MB limit).
const DefaultMaxRecordSizeBytesRDS = 1 * 1024 * 1024
DefaultMaxRecordSizeBytesRDS is the default maximum record size in bytes for the RDSExporter (Data API 1 MB request limit).
const DefaultMaxRecordSizeBytesS3 = 5 * 1024 * 1024
DefaultMaxRecordSizeBytesS3 is the default maximum record size in bytes for the S3Exporter (5 MB).
const RecordType = "DurableExecutionTelemetry"
RecordType is the discriminator value on every emitted record.
const SchemaVersion = "1.0"
SchemaVersion is the fixed schema version for telemetry records.
Variables ¶
var NoSortKey = new(string)
NoSortKey is a *string pointing at the empty string, for use as DynamoDBExporter.SortKeyField to explicitly disable the sort key.
Functions ¶
func DefaultFileNamer ¶
DefaultFileNamer returns a filename based on the SHA-256 hash of the execution ARN and the current timestamp in UnixNano: "{hash8}_{timestamp}.json".
func GroupOperationsByName ¶
func GroupOperationsByName(ops []OperationRecord) map[string]OperationSummary
GroupOperationsByName aggregates operations into a by-name view. Operations with an empty Name are excluded. The aggregation rules are:
- Count, FailedCount, MaxAttempt, and duration metrics are aggregated across all occurrences sharing a name.
- Type, SubType, and Status reflect the last occurrence encountered.
- Error and Result are included only for names that occurred exactly once.
func RenderRecord ¶
func RenderRecord(record Record, format OperationsFormat) ([]byte, error)
RenderRecord marshals a Record to JSON with operations formatted according to the specified format. When format is empty, it defaults to OperationsFormatArray.
Types ¶
type CloudWatchLogsAPI ¶
type CloudWatchLogsAPI interface {
PutLogEvents(ctx context.Context, params *cloudwatchlogs.PutLogEventsInput, optFns ...func(*cloudwatchlogs.Options)) (*cloudwatchlogs.PutLogEventsOutput, error)
}
CloudWatchLogsAPI is the subset of *cloudwatchlogs.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.
type CloudWatchLogsExporter ¶
type CloudWatchLogsExporter struct {
// API is the underlying CloudWatch Logs client. Required.
API CloudWatchLogsAPI
// LogGroupName is the target log group. Required.
LogGroupName string
// LogStreamPrefix prefixes the daily log stream name. Defaults to
// "durable-execution-telemetry/" when empty.
LogStreamPrefix string
// MaxSizeBytes overrides DefaultMaxRecordSizeBytesLog. Left at its
// zero value, MaxRecordSizeBytes() returns DefaultMaxRecordSizeBytesLog
// instead.
MaxSizeBytes int
// contains filtered or unexported fields
}
CloudWatchLogsExporter writes records to a specific CloudWatch Logs log group via PutLogEvents. Unlike LambdaLogExporter (which relies on Lambda's automatic stdout capture), this exporter lets the caller centralize records from multiple functions into one shared, queryable log group.
Log stream naming: "{LogStreamPrefix}{YYYY}/{MM}/{DD}" — one stream per UTC calendar day, auto-created on first write.
Each log event carries the record with its operations rendered as operationsByName (see OperationsFormatByName), never as the operations array. CloudWatch Logs Insights can read a field by a dotted path, such as operationsByName.charge.status, but it cannot select the array element whose name is "charge". So the format is fixed and has no option.
func (*CloudWatchLogsExporter) Export ¶
func (e *CloudWatchLogsExporter) Export(ctx context.Context, record Record) error
Export writes record as a single log event to this exporter's target log group, in the daily log stream for the current UTC date — creating that stream first if this is the first Export call for that date.
func (*CloudWatchLogsExporter) MaxRecordSizeBytes ¶
func (e *CloudWatchLogsExporter) MaxRecordSizeBytes() int
MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesLog when unset.
func (*CloudWatchLogsExporter) Render ¶
func (e *CloudWatchLogsExporter) Render(record Record) ([]byte, error)
Render implements RenderingSizeLimiter. It returns the log event message that Export sends: the record rendered with OperationsFormatByName. Export calls Render, so truncation sizes the bytes that are sent.
type Config ¶
type Config struct {
// EmitMode controls when a record is sent to Exporters. Defaults to
// EmitOnComplete when left at its zero value.
EmitMode EmitMode
// OperationDetail controls which operations appear in the record.
// Defaults to OperationDetailTopLevel.
OperationDetail OperationDetail
// SamplingRate is the fraction of executions (0.0-1.0) that emit
// records. Defaults to 1.0 (every execution) when left at zero.
// The decision is per-execution and deterministic (derived from a
// hash of ExecutionArn).
SamplingRate float64
// Exporters is where records are sent in parallel. Defaults to a
// single LambdaLogExporter when left nil/empty.
Exporters []Exporter
// Content controls what data is included in emitted records.
Content ContentConfig
}
Config configures the Durable Execution Telemetry plugin.
type ContentConfig ¶
type ContentConfig struct {
// ExcludeInput omits the execution input from records.
ExcludeInput bool
// InputTransform, if non-nil, receives the execution input decoded as
// a JSON value and returns the value to record. Use it to remove or
// mask sensitive fields. It is not called when ExcludeInput is true.
InputTransform func(any) any
// ExcludeOutput omits the execution output from records.
ExcludeOutput bool
// OutputTransform, if non-nil, receives the execution output decoded
// as a JSON value and returns the value to record. It is not called
// when ExcludeOutput is true.
OutputTransform func(any) any
// Operations controls per-operation content.
Operations ContentOperations
}
ContentConfig controls what content appears in emitted records. The zero value includes the execution input and output unchanged, omits operation results, and includes operation and execution errors.
Content values (Record.Input, Record.Output, OperationRecord.Result) are JSON values. There is no per-field length cap. An exporter that implements SizeLimiter bounds the whole record instead (see Truncate).
A transform function receives the content decoded as a JSON value: a map[string]any, []any, string, json.Number, bool, or nil. Numbers are json.Number rather than float64 so that integers above 2^53 keep their exact digits through the transform; call Int64 or Float64 on the value to do arithmetic. The transform returns the value to record, which is marshaled again. Marshaling writes no whitespace and sorts the keys of every object, so the recorded text can differ from the original text in key order. The characters <, > and & are written literally, not as \u003c, \u003e and \u0026. If a transform panics, or its return value cannot be marshaled, the field is omitted. Transforms may be called concurrently by different invocations. Each function and its captured state must be safe for concurrent use.
Transforms run only for sampled-in invocations; see Config.SamplingRate. InputTransform and OutputTransform run at most once per invocation. A Result transform runs once for an operation's result in an invocation, and again only if that result text changes. Each new invocation replays checkpointed results, so it calls the Result transform again. Transforms run while the plugin holds its per-invocation lock, which delays that invocation's other hooks. So a transform should return quickly and must not block.
type ContentOperations ¶
type ContentOperations struct {
// Overrides are matched to operations by OperationName. When several
// entries share a name, the last one wins.
Overrides []OperationOverride
// ExcludeErrors omits the error field from every operation entry. The
// execution-level error on the record is always included.
ExcludeErrors bool
}
ContentOperations controls the content of operation entries.
type DynamoDBAPI ¶
type DynamoDBAPI interface {
PutItem(ctx context.Context, params *dynamodb.PutItemInput, optFns ...func(*dynamodb.Options)) (*dynamodb.PutItemOutput, error)
}
DynamoDBAPI is the subset of *dynamodb.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.
type DynamoDBExporter ¶
type DynamoDBExporter struct {
// API is the underlying DynamoDB client. Required.
API DynamoDBAPI
// TableName is the target table. Required.
TableName string
// PartitionKeyField is the partition-key attribute name. Defaults to
// "pk" when empty.
PartitionKeyField string
// SortKeyField is the sort-key attribute name, or nil to accept the
// default ("sk"). Set to NoSortKey (a non-nil pointer to "") to
// explicitly disable the sort key.
SortKeyField *string
// MaxSizeBytes overrides DefaultMaxRecordSizeBytesDynamoDB. Left at
// its zero value, MaxRecordSizeBytes() returns
// DefaultMaxRecordSizeBytesDynamoDB instead.
MaxSizeBytes int
}
DynamoDBExporter writes records to Amazon DynamoDB via PutItem.
Partition key value is the record's ExecutionArn. Sort key value (when present) is the record's EmittedAt timestamp in the record's own timestamp format (UTC, millisecond precision, Z suffix) — so a full history of every record ever emitted for a given execution accumulates as separate items sharing the same partition key.
Set SortKey to NoSortKey to explicitly disable the sort key — PutItem then overwrites the single item for a given partition key on every call (only the latest state is kept).
func (*DynamoDBExporter) Export ¶
func (e *DynamoDBExporter) Export(ctx context.Context, record Record) error
Export writes record as a DynamoDB item.
func (*DynamoDBExporter) MaxRecordSizeBytes ¶
func (e *DynamoDBExporter) MaxRecordSizeBytes() int
MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesDynamoDB when unset.
type EmitMode ¶
type EmitMode string
EmitMode controls when the telemetry plugin emits records.
There is no failure-only mode: use EmitOnComplete for terminal-only emission and filter downstream if needed.
const ( // EmitAlways emits a record after every invocation regardless of // outcome. EmitAlways EmitMode = "always" // EmitOnChange emits a running snapshot on every operation start/end // in addition to the terminal record. EmitOnChange EmitMode = "on-change" // EmitOnComplete emits a record only when the execution reaches a // terminal state (succeeded or failed). EmitOnComplete EmitMode = "on-complete" )
type ErrorRecord ¶
type ErrorRecord struct {
Type string `json:"type"`
Message string `json:"message"`
StackTrace string `json:"stackTrace,omitempty"`
}
ErrorRecord carries an error's type and message.
type EventBridgeAPI ¶
type EventBridgeAPI interface {
PutEvents(ctx context.Context, params *eventbridge.PutEventsInput, optFns ...func(*eventbridge.Options)) (*eventbridge.PutEventsOutput, error)
}
EventBridgeAPI is the subset of *eventbridge.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.
type EventBridgeExporter ¶
type EventBridgeExporter struct {
// API is the underlying EventBridge client. Required.
API EventBridgeAPI
// EventBusName is the target event bus. Defaults to "default" when
// empty.
EventBusName string
// Source is the event Source field. Defaults to
// DefaultEventBridgeSource when empty.
Source string
// OperationsFormat controls how operations are rendered within
// Detail. Defaults to OperationsFormatArray when left empty.
OperationsFormat OperationsFormat
// MaxSizeBytes overrides DefaultMaxRecordSizeBytesLog. Left at its
// zero value, MaxRecordSizeBytes() returns DefaultMaxRecordSizeBytesLog
// instead.
MaxSizeBytes int
}
EventBridgeExporter publishes records to Amazon EventBridge for event-driven reactions.
Source defaults to DefaultEventBridgeSource; DetailType is the record's Status (SUCCEEDED/FAILED/RUNNING); Detail is the full rendered record JSON per OperationsFormat.
func (*EventBridgeExporter) Export ¶
func (e *EventBridgeExporter) Export(ctx context.Context, record Record) error
Export publishes record as a single EventBridge event.
func (*EventBridgeExporter) MaxRecordSizeBytes ¶
func (e *EventBridgeExporter) MaxRecordSizeBytes() int
MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesLog when unset.
type Exporter ¶
Exporter is the interface a Durable Execution Telemetry destination implements. Export is called with the record whenever a new snapshot is ready to send, per the configured EmitMode.
Export errors are swallowed by the plugin (never fail the execution or affect other configured exporters).
type FileExporter ¶
type FileExporter struct {
// contains filtered or unexported fields
}
FileExporter writes each record as a JSON file to a configurable directory on the local filesystem. It implements Exporter and requires no AWS dependencies — only the standard library.
func NewFileExporter ¶
func NewFileExporter(cfg FileExporterConfig) *FileExporter
NewFileExporter returns a FileExporter configured by cfg. Panics if cfg.Dir is empty.
func (*FileExporter) Close ¶
func (e *FileExporter) Close() error
Close is a no-op — FileExporter holds no persistent connections.
type FileExporterConfig ¶
type FileExporterConfig struct {
// Dir is the directory where record files are written. Required.
// Created via os.MkdirAll if it does not exist.
Dir string
// FileNamer returns the filename (without directory) for a given
// record. If nil, DefaultFileNamer is used.
FileNamer func(record *Record) string
// OperationsFormat controls how operations are rendered in the JSON
// output. Defaults to OperationsFormatArray when empty.
OperationsFormat OperationsFormat
}
FileExporterConfig configures a FileExporter.
type FirehoseAPI ¶
type FirehoseAPI interface {
PutRecord(ctx context.Context, params *firehose.PutRecordInput, optFns ...func(*firehose.Options)) (*firehose.PutRecordOutput, error)
}
FirehoseAPI is the subset of *firehose.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.
type FirehoseExporter ¶
type FirehoseExporter struct {
// API is the underlying Firehose client. Required.
API FirehoseAPI
// DeliveryStreamName is the target delivery stream. Required.
DeliveryStreamName string
// OperationsFormat controls how operations are rendered. Defaults to
// OperationsFormatArray when left empty.
OperationsFormat OperationsFormat
// MaxSizeBytes overrides DefaultMaxRecordSizeBytesFirehose. Left at
// its zero value, MaxRecordSizeBytes() returns
// DefaultMaxRecordSizeBytesFirehose instead.
MaxSizeBytes int
}
FirehoseExporter sends records to Amazon Kinesis Data Firehose for delivery to S3, Redshift, Splunk, or any HTTP endpoint the delivery stream is configured with.
Each record is appended with a trailing newline (NDJSON), so records remain individually parseable once Firehose batches multiple PutRecord calls into a single S3 object.
func (*FirehoseExporter) Export ¶
func (e *FirehoseExporter) Export(ctx context.Context, record Record) error
Export sends record as a single NDJSON-formatted PutRecord call.
func (*FirehoseExporter) MaxRecordSizeBytes ¶
func (e *FirehoseExporter) MaxRecordSizeBytes() int
MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesFirehose when unset.
type Flusher ¶
Flusher is an optional interface an Exporter may additionally implement to flush any buffered data before the invocation returns. Checked via a type assertion at dispatch time rather than being part of the required Exporter interface.
type HTTPDoer ¶
HTTPDoer is the interface for sending HTTP requests. *http.Client satisfies this interface.
type HTTPExporter ¶
type HTTPExporter struct {
// contains filtered or unexported fields
}
HTTPExporter POSTs JSON-serialized records to a configurable URL. It implements Exporter and requires no AWS dependencies — only the standard library net/http.
func NewHTTPExporter ¶
func NewHTTPExporter(cfg HTTPExporterConfig) *HTTPExporter
NewHTTPExporter returns an HTTPExporter configured by cfg. Panics if cfg.URL is empty.
func (*HTTPExporter) Close ¶
func (e *HTTPExporter) Close() error
Close is a no-op — HTTPExporter holds no persistent connections that require cleanup.
type HTTPExporterConfig ¶
type HTTPExporterConfig struct {
// URL is the endpoint to POST records to. Required.
URL string
// Client is the HTTP client used for requests. If nil, a default
// client with Timeout is used.
Client HTTPDoer
// Headers are added to every request (e.g. Authorization).
Headers map[string]string
// Timeout bounds a single HTTP request. Defaults to
// DefaultHTTPExporterTimeout when zero and Client is nil. Ignored
// when Client is provided (the caller's client owns its timeout).
Timeout time.Duration
// MaxRetries is the maximum number of retries on 5xx responses.
// Defaults to 1 when zero.
MaxRetries int
// OperationsFormat controls how operations are rendered in the JSON
// body. Defaults to OperationsFormatArray when empty.
OperationsFormat OperationsFormat
}
HTTPExporterConfig configures an HTTPExporter.
type LambdaLogExporter ¶
type LambdaLogExporter struct {
// Writer overrides the default destination (os.Stdout). Primarily
// for tests.
Writer io.Writer
// MaxSizeBytes overrides DefaultMaxRecordSizeBytesLog. Left at zero,
// MaxRecordSizeBytes() returns DefaultMaxRecordSizeBytesLog.
MaxSizeBytes int
}
LambdaLogExporter writes records as a single JSON line to stdout. Since Lambda captures stdout to the function's CloudWatch log group automatically, this requires zero IAM permissions and zero setup.
Each line carries the record with its operations rendered as operationsByName (see OperationsFormatByName), never as the operations array. The line lands in CloudWatch Logs, where Logs Insights can read a field by a dotted path but cannot select an array element by its name. So the format is fixed and has no option, as for CloudWatchLogsExporter.
func NewLambdaLogExporter ¶
func NewLambdaLogExporter() *LambdaLogExporter
NewLambdaLogExporter returns a LambdaLogExporter writing to os.Stdout.
func (*LambdaLogExporter) Export ¶
func (e *LambdaLogExporter) Export(_ context.Context, record Record) error
Export writes record as a single line of JSON to the configured Writer (os.Stdout if unset). It writes exactly Render's output.
func (*LambdaLogExporter) MaxRecordSizeBytes ¶
func (e *LambdaLogExporter) MaxRecordSizeBytes() int
MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesLog when unset.
func (*LambdaLogExporter) Render ¶
func (e *LambdaLogExporter) Render(record Record) ([]byte, error)
Render implements RenderingSizeLimiter. It returns the exact bytes Export writes: the record rendered with OperationsFormatByName, then one newline. Truncation sizes Render's output. So the size limit includes the newline.
type OpenSearchExporter ¶
type OpenSearchExporter struct {
// Client is the HTTP client used for requests. If nil,
// http.DefaultClient is used.
Client HTTPDoer
// Endpoint is the cluster's base URL, e.g.
// "https://search-my-domain.us-east-1.es.amazonaws.com". Required.
Endpoint string
// IndexName is the target index. Defaults to "durable-execution-telemetry"
// when empty.
IndexName string
// Username and Password are used for HTTP Basic authentication.
// Mutually exclusive with BearerToken.
Username string
Password string
// BearerToken is used for Bearer token authentication. Mutually
// exclusive with Username/Password.
BearerToken string
// MaxSizeBytes overrides DefaultMaxRecordSizeBytesOpenSearch. Left
// at its zero value, MaxRecordSizeBytes() returns
// DefaultMaxRecordSizeBytesOpenSearch instead.
MaxSizeBytes int
}
OpenSearchExporter indexes records as documents in OpenSearch via HTTP POST to the _doc API. Supports basic auth (username/password) or bearer token authentication.
Document creation uses POST to {endpoint}/{index}/_doc, which lets OpenSearch assign a document ID automatically. Each export creates a new document — downstream queries can aggregate by executionArn.
func (*OpenSearchExporter) Export ¶
func (e *OpenSearchExporter) Export(ctx context.Context, record Record) error
Export indexes record as an OpenSearch document.
func (*OpenSearchExporter) MaxRecordSizeBytes ¶
func (e *OpenSearchExporter) MaxRecordSizeBytes() int
MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesOpenSearch when unset.
type OperationDetail ¶
type OperationDetail string
OperationDetail controls which operations appear in a record's Operations array.
const ( // OperationDetailTopLevel includes only top-level operations // (anything with a non-empty ParentID is dropped). OperationDetailTopLevel OperationDetail = "top-level" // OperationDetailFullTree includes every operation, including // children of contexts. OperationDetailFullTree OperationDetail = "full-tree" )
type OperationOverride ¶
type OperationOverride struct {
// OperationName selects the operations this override applies to.
OperationName string
// Exclude drops the operation from the record.
Exclude bool
// Result opts the operation's result into the record. Results are
// omitted unless an override sets this. The function receives the
// checkpointed result decoded as a JSON value, or the raw text as a
// string when that text is not valid JSON. The plugin does not run
// the operation's own serializer, so an operation with a custom
// Serdes delivers its serialized form here. To record a result
// unchanged, return the argument. It is not called when Exclude is true.
Result func(any) any
}
OperationOverride changes the content of every operation with the given name.
type OperationRecord ¶
type OperationRecord struct {
ID string `json:"id"`
Name string `json:"name,omitempty"`
Type string `json:"type"`
SubType string `json:"subType,omitempty"`
Status string `json:"status"`
ParentID string `json:"parentId,omitempty"`
// StartTimestamp and EndTimestamp serialize as the JSON keys
// startTime and endTime. See Record.MarshalJSON for the format.
StartTimestamp *time.Time `json:"startTime,omitempty"`
EndTimestamp *time.Time `json:"endTime,omitempty"`
DurationMs *int64 `json:"durationMs,omitempty"`
Attempt int `json:"attempt,omitempty"`
// Result is the operation's result as a JSON value. A result is
// recorded only through an OperationOverride.Result transform. The
// transform receives the checkpointed wire text decoded as JSON, or
// the text as a string when it is not valid JSON. Its return value is
// marshaled again. So the recorded text has no whitespace and has
// sorted object keys, even when the transform returns its argument.
// See ContentConfig.
Result json.RawMessage `json:"result,omitempty"`
Error *ErrorRecord `json:"error,omitempty"`
// Truncated indicates this operation's result was dropped due to
// size constraints (see Truncate).
Truncated bool `json:"truncated,omitempty"`
}
OperationRecord describes a single durable operation within a Record.
func (OperationRecord) MarshalJSON ¶
func (o OperationRecord) MarshalJSON() ([]byte, error)
MarshalJSON writes the operation timestamps in timestampLayout. See Record.MarshalJSON for the mechanism.
type OperationSummary ¶
type OperationSummary struct {
Type string `json:"type"`
SubType string `json:"subType,omitempty"`
Count int `json:"count"`
// The duration fields are nil when no occurrence carried a DurationMs,
// so the JSON omits them; a recorded duration of 0 ms serializes as 0.
// This matches the JavaScript implementation.
MinDurationMs *int64 `json:"minDurationMs,omitempty"`
MaxDurationMs *int64 `json:"maxDurationMs,omitempty"`
TotalDurationMs *int64 `json:"totalDurationMs,omitempty"`
FailedCount int `json:"failedCount"`
MaxAttempt int `json:"maxAttempt,omitempty"`
// Status reflects the most recently seen occurrence's status.
Status string `json:"status"`
// Error is populated only when the name occurred exactly once and
// the operation has an error. For repeated names (loops, retries),
// it is left nil; FailedCount still flags failures.
Error *ErrorRecord `json:"error,omitempty"`
// Result is populated only when the name occurred exactly once and
// the operation has a result.
Result json.RawMessage `json:"result,omitempty"`
}
OperationSummary aggregates every occurrence of a given operation name within a Record's Operations slice. The CloudWatch Logs and Lambda log exporters always render it. Other exporters render it when configured with OperationsFormatByName or OperationsFormatBoth. DynamoDBExporter does not render it: it writes the Operations array inside recordJson.
type OperationsFormat ¶
type OperationsFormat string
OperationsFormat controls how an exporter renders a record's operations.
const ( // OperationsFormatArray renders only the canonical Operations array. // This is the default. OperationsFormatArray OperationsFormat = "array" // OperationsFormatByName renders only the by-name aggregation, // omitting the Operations array. OperationsFormatByName OperationsFormat = "by-name" // OperationsFormatBoth renders both the Operations array and the // by-name aggregation. OperationsFormatBoth OperationsFormat = "both" )
type RDSDataAPI ¶
type RDSDataAPI interface {
ExecuteStatement(ctx context.Context, params *rdsdata.ExecuteStatementInput, optFns ...func(*rdsdata.Options)) (*rdsdata.ExecuteStatementOutput, error)
}
RDSDataAPI is the subset of *rdsdata.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.
type RDSExporter ¶
type RDSExporter struct {
// API is the underlying RDS Data API client. Required.
API RDSDataAPI
// ResourceArn is the target Aurora cluster's ARN. Required.
ResourceArn string
// SecretArn is the Secrets Manager secret ARN holding the database
// credentials. Required.
SecretArn string
// Database is the target database name. Required.
Database string
// TableName is the target table name. Defaults to
// "durable_execution_telemetry" when empty.
TableName string
// MaxSizeBytes overrides DefaultMaxRecordSizeBytesRDS. Left at its
// zero value, MaxRecordSizeBytes() returns
// DefaultMaxRecordSizeBytesRDS instead.
MaxSizeBytes int
}
RDSExporter writes records to an Aurora/RDS cluster via the RDS Data API's ExecuteStatement (HTTP-based — no VPC required).
Record values are always bound via SqlParameters. The table name is the one value embedded in the statement text (identifiers cannot be bound as parameters) and is validated as a plain SQL identifier before use. Uses INSERT INTO with named parameters for the four columns: execution_arn, emitted_at, status, record_json.
func (*RDSExporter) Export ¶
func (e *RDSExporter) Export(ctx context.Context, record Record) error
Export inserts record into the target RDS/Aurora table via a parameterized INSERT statement.
func (*RDSExporter) MaxRecordSizeBytes ¶
func (e *RDSExporter) MaxRecordSizeBytes() int
MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesRDS when unset.
type Record ¶
type Record struct {
RecordType string `json:"recordType"`
SchemaVersion string `json:"schemaVersion"`
ExecutionArn string `json:"executionArn"`
ExecutionName string `json:"executionName,omitempty"`
// FunctionName, FunctionQualifier, Region, and AccountID are parsed
// from the execution ARN. They are always emitted, empty when the ARN
// does not carry them, so every record has the same identity keys.
FunctionName string `json:"functionName"`
FunctionQualifier string `json:"functionQualifier"`
Region string `json:"region"`
AccountID string `json:"accountId"`
Status Status `json:"status"`
// StartTimestamp and EndTimestamp serialize as the JSON keys
// startTime and endTime. See MarshalJSON for the format.
StartTimestamp *time.Time `json:"startTime,omitempty"`
EndTimestamp *time.Time `json:"endTime,omitempty"`
DurationMs *int64 `json:"durationMs,omitempty"`
// EmittedAt is the time this record was created, refreshed to the
// emission time when a snapshot is sent.
EmittedAt time.Time `json:"emittedAt"`
Operations []OperationRecord `json:"operations"`
Error *ErrorRecord `json:"error,omitempty"`
// Input and Output are the execution input and result as JSON values,
// marshaled once from the handler's typed values. A Go string is a
// JSON string. See ContentConfig.
Input json.RawMessage `json:"input,omitempty"`
Output json.RawMessage `json:"output,omitempty"`
Custom map[string]any `json:"custom,omitempty"`
// Truncation metadata: set when the record was truncated to fit an
// exporter's size limit (see Truncate).
Truncated bool `json:"truncated,omitempty"`
DroppedOperations int `json:"droppedOperations,omitempty"`
DroppedInput bool `json:"droppedInput,omitempty"`
DroppedOutput bool `json:"droppedOutput,omitempty"`
}
Record is the cumulative snapshot a telemetry plugin builds for a single durable execution. It is handed to each configured exporter on emission.
func NewRecord ¶
NewRecord creates a Record with standard metadata pre-populated. EmittedAt is set to time.Now() and ExecutionName is extracted from the ARN.
func Truncate ¶
Truncate returns a copy of record, best-effort truncated to fit within maxBytes when rendered via render (or canonical JSON if render is nil). Drop order: operation results (oldest first), whole operations (oldest first), then Input, then Output. Identity/timeline fields are never dropped.
When maxBytes <= 0, returns record unchanged (truncation disabled).
func (Record) MarshalJSON ¶
MarshalJSON writes every timestamp in timestampLayout. encoding/json would otherwise call time.Time.MarshalJSON, which uses RFC3339Nano.
The method marshals an alias of Record with shadow string fields that carry the same JSON keys as the time.Time fields. encoding/json keeps the shallower field when two fields share a key, so the shadow strings replace the time values in the output. The Go field types stay time.Time, and json.Unmarshal into Record still works because time.Time parses the three-digit form.
The method writes <, > and & literally. A caller's own json.Marshal call on a Record escapes them as \u003c, \u003e and \u0026, because json.Marshal escapes the bytes a json.Marshaler returns. To keep them literal, use RenderRecord or a json.Encoder with SetEscapeHTML(false).
type RedshiftDataAPI ¶
type RedshiftDataAPI interface {
ExecuteStatement(ctx context.Context, params *redshiftdata.ExecuteStatementInput, optFns ...func(*redshiftdata.Options)) (*redshiftdata.ExecuteStatementOutput, error)
}
RedshiftDataAPI is the subset of *redshiftdata.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.
type RedshiftExporter ¶
type RedshiftExporter struct {
// API is the underlying Redshift Data API client. Required.
API RedshiftDataAPI
// WorkgroupName selects Serverless mode. Mutually exclusive with
// ClusterIdentifier — set exactly one.
WorkgroupName string
// ClusterIdentifier selects provisioned-cluster mode. Mutually
// exclusive with WorkgroupName — set exactly one.
ClusterIdentifier string
// Database is the target database name. Required.
Database string
// TableName is the target table name. Defaults to
// "durable_execution_telemetry" when empty.
TableName string
}
RedshiftExporter writes records to Amazon Redshift via the Redshift Data API's ExecuteStatement (HTTP-based — no VPC required).
Supports two connection modes:
- Serverless: set WorkgroupName.
- Provisioned cluster: set ClusterIdentifier.
Record values are always bound via SqlParameters. The table name is the one value embedded in the statement text (identifiers cannot be bound as parameters) and is validated as a plain SQL identifier before use.
Fire-and-forget: does NOT poll for statement completion.
type Renderer ¶
Renderer produces the exact byte shape a Truncate call should size against. Exporters with custom rendering (OperationsFormat) provide their own Renderer so truncation sizes against what is actually sent.
type RenderingSizeLimiter ¶
type RenderingSizeLimiter interface {
SizeLimiter
Render(record Record) ([]byte, error)
}
RenderingSizeLimiter is SizeLimiter's companion interface for an exporter whose Export does not send the plain json.Marshal form of the record, such as an exporter with an OperationsFormat option or one that always renders operationsByName. Render must return the exact byte shape this exporter's Export call would actually send, so truncation sizes against the real rendered form.
type S3API ¶
type S3API interface {
PutObject(ctx context.Context, params *s3.PutObjectInput, optFns ...func(*s3.Options)) (*s3.PutObjectOutput, error)
}
S3API is the subset of *s3.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.
type S3Exporter ¶
type S3Exporter struct {
// API is the underlying S3 client. Required.
API S3API
// Bucket is the target S3 bucket. Required.
Bucket string
// Prefix is prepended to every object key. Defaults to
// "durable-execution-telemetry/" when empty.
Prefix string
// Partitioning controls the partition segment inserted between
// Prefix and the object's own filename. Defaults to
// S3PartitioningDate when left at its zero value.
Partitioning S3Partitioning
// MaxSizeBytes overrides DefaultMaxRecordSizeBytesS3. Left at its
// zero value, MaxRecordSizeBytes() returns DefaultMaxRecordSizeBytesS3
// instead.
MaxSizeBytes int
}
S3Exporter writes records as JSON objects to Amazon S3. Uses the record's own ExecutionName (falling back to a sanitized ExecutionArn when ExecutionName is empty) as the object key, so a later update to the same execution overwrites the same object rather than creating a new one each time.
func (*S3Exporter) Export ¶
func (e *S3Exporter) Export(ctx context.Context, record Record) error
Export writes record as a JSON object to this exporter's bucket, at a key determined by Prefix/Partitioning/the record's own identity.
func (*S3Exporter) MaxRecordSizeBytes ¶
func (e *S3Exporter) MaxRecordSizeBytes() int
MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesS3 when unset.
type S3Partitioning ¶
type S3Partitioning string
S3Partitioning controls how S3Exporter keys objects.
const ( // S3PartitioningDate keys objects under Hive-style // year=YYYY/month=MM/day=DD/ partitions. The default. S3PartitioningDate S3Partitioning = "date" // S3PartitioningFunctionName keys objects under the record's own // FunctionName instead of a date. S3PartitioningFunctionName S3Partitioning = "function-name" // S3PartitioningNone applies no partitioning prefix at all beyond // Prefix itself. S3PartitioningNone S3Partitioning = "none" )
type SQSAPI ¶
type SQSAPI interface {
SendMessage(ctx context.Context, params *sqs.SendMessageInput, optFns ...func(*sqs.Options)) (*sqs.SendMessageOutput, error)
}
SQSAPI is the subset of *sqs.Client this package depends on, exposed as an interface so tests can substitute a fake without needing real AWS credentials or network access.
type SQSExporter ¶
type SQSExporter struct {
// API is the underlying SQS client. Required.
API SQSAPI
// QueueURL is the target queue. Required. A ".fifo" suffix switches
// this exporter into FIFO mode automatically.
QueueURL string
// MessageGroupID overrides the FIFO message group ID. Only
// meaningful when QueueURL ends in ".fifo". Defaults to the
// record's ExecutionArn when empty.
MessageGroupID string
// OperationsFormat controls how operations are rendered within the
// message body. Defaults to OperationsFormatArray when left empty.
OperationsFormat OperationsFormat
// MaxSizeBytes overrides DefaultMaxRecordSizeBytesLog. Left at its
// zero value, MaxRecordSizeBytes() returns DefaultMaxRecordSizeBytesLog
// instead.
MaxSizeBytes int
}
SQSExporter sends records to Amazon SQS (standard or FIFO) via SendMessage.
FIFO mode is auto-detected from a ".fifo" suffix on QueueURL. In FIFO mode, MessageGroupID defaults to the record's ExecutionArn and MessageDeduplicationId is set to "{ExecutionArn}:{EmittedAt}", with EmittedAt in the record's own timestamp format (UTC, millisecond precision, Z suffix).
Message attributes "status" and "functionName" are always set, enabling SQS message filtering without body parsing.
func (*SQSExporter) Export ¶
func (e *SQSExporter) Export(ctx context.Context, record Record) error
Export sends record as a single SQS message.
func (*SQSExporter) MaxRecordSizeBytes ¶
func (e *SQSExporter) MaxRecordSizeBytes() int
MaxRecordSizeBytes implements SizeLimiter, returning e.MaxSizeBytes or DefaultMaxRecordSizeBytesLog when unset.
type SizeLimiter ¶
type SizeLimiter interface {
MaxRecordSizeBytes() int
}
SizeLimiter is an optional interface an Exporter may additionally implement to opt into size-based truncation. When an Exporter implements SizeLimiter, the plugin sends that exporter its own, independently truncated copy of the record.
type Status ¶
type Status string
Status represents the execution-level status of a durable workflow.
type TelemetryPlugin ¶
type TelemetryPlugin struct {
// contains filtered or unexported fields
}
TelemetryPlugin is the Durable Execution Telemetry plugin factory. Construct via New; do not construct the zero value directly.
func New ¶
func New(cfg Config) *TelemetryPlugin
New constructs a Durable Execution Telemetry plugin from cfg. A zero-value Config produces documented defaults: EmitOnComplete, SamplingRate 1.0, a single LambdaLogExporter.
Example ¶
ExampleNew demonstrates creating a telemetry plugin with default configuration: emits on completion, 100% sampling, stdout JSON via LambdaLogExporter.
package main
import (
"context"
"fmt"
"github.com/aws/aws-durable-execution-sdk-go/durable"
"github.com/aws/aws-durable-execution-sdk-go/telemetry"
)
func main() {
plugin := telemetry.New(telemetry.Config{})
// Register with durable.Start via durable.WithPlugins(plugin). The
// handler calls CreatePlugin once per invocation.
p, err := plugin.CreatePlugin(context.Background(), durable.InvocationHookInfo{})
fmt.Println(err == nil && p.OnInvocationStart != nil)
}
Output: true
Example (WithContentConfig) ¶
ExampleNew_withContentConfig demonstrates shaping record content: a sensitive input field is removed, the output is omitted, one operation opts its result in, and another is dropped from the record.
package main
import (
"context"
"fmt"
"github.com/aws/aws-durable-execution-sdk-go/durable"
"github.com/aws/aws-durable-execution-sdk-go/telemetry"
)
func main() {
plugin := telemetry.New(telemetry.Config{
Content: telemetry.ContentConfig{
InputTransform: func(v any) any {
if m, ok := v.(map[string]any); ok {
delete(m, "password")
}
return v
},
ExcludeOutput: true,
Operations: telemetry.ContentOperations{
Overrides: []telemetry.OperationOverride{
{OperationName: "fetch-user", Result: func(v any) any { return v }},
{OperationName: "debug-dump", Exclude: true},
},
},
},
})
p, _ := plugin.CreatePlugin(context.Background(), durable.InvocationHookInfo{})
fmt.Println(p.OnOperationEnd != nil)
}
Output: true
Example (WithMultipleExporters) ¶
ExampleNew_withMultipleExporters demonstrates combining a file exporter (for local debugging) with an S3 exporter (for durable storage).
package main
import (
"context"
"fmt"
"github.com/aws/aws-durable-execution-sdk-go/durable"
"github.com/aws/aws-durable-execution-sdk-go/telemetry"
)
func main() {
fileExp := telemetry.NewFileExporter(telemetry.FileExporterConfig{
Dir: "/tmp/telemetry-records",
})
s3Exp := &telemetry.S3Exporter{
API: nil, // replace with s3.NewFromConfig(cfg)
Bucket: "my-telemetry-bucket",
}
plugin := telemetry.New(telemetry.Config{
Exporters: []telemetry.Exporter{fileExp, s3Exp},
})
p, _ := plugin.CreatePlugin(context.Background(), durable.InvocationHookInfo{})
fmt.Println(p.OnOperationStart != nil)
}
Output: true
Example (WithS3Exporter) ¶
ExampleNew_withS3Exporter demonstrates configuring an S3 exporter with date-based partitioning.
package main
import (
"context"
"fmt"
"github.com/aws/aws-durable-execution-sdk-go/durable"
"github.com/aws/aws-durable-execution-sdk-go/telemetry"
)
func main() {
// In production, use a real *s3.Client from aws-sdk-go-v2/service/s3.
// nil is used here to show the configuration shape without making
// AWS calls.
exporter := &telemetry.S3Exporter{
API: nil, // replace with s3.NewFromConfig(cfg)
Bucket: "my-telemetry-bucket",
Prefix: "durable-execution-telemetry/",
Partitioning: telemetry.S3PartitioningDate,
}
plugin := telemetry.New(telemetry.Config{
Exporters: []telemetry.Exporter{exporter},
})
p, _ := plugin.CreatePlugin(context.Background(), durable.InvocationHookInfo{})
fmt.Println(p.OnInvocationEnd != nil)
}
Output: true
Example (WithSampling) ¶
ExampleNew_withSampling demonstrates configuring the plugin to emit records for only 10% of executions. The sampling decision is deterministic per execution ARN.
package main
import (
"context"
"fmt"
"github.com/aws/aws-durable-execution-sdk-go/durable"
"github.com/aws/aws-durable-execution-sdk-go/telemetry"
)
func main() {
plugin := telemetry.New(telemetry.Config{
SamplingRate: 0.1, // 10% of executions
EmitMode: telemetry.EmitOnComplete,
})
p, _ := plugin.CreatePlugin(context.Background(), durable.InvocationHookInfo{})
fmt.Println(p.OnInvocationStart != nil)
}
Output: true
func (*TelemetryPlugin) CreatePlugin ¶
func (p *TelemetryPlugin) CreatePlugin(_ context.Context, info durable.InvocationHookInfo) (*durable.Plugin, error)
CreatePlugin returns hooks bound to fresh invocation-local state. Configuration and exporters remain shared on the factory. The record, sampling decision, mutex, and on-change dispatch state belong to the returned plugin alone.
Source Files
¶
- content.go
- exporter.go
- exporter_cloudwatch_logs.go
- exporter_dynamodb.go
- exporter_eventbridge.go
- exporter_firehose.go
- exporter_opensearch.go
- exporter_rds.go
- exporter_redshift.go
- exporter_s3.go
- exporter_sqs.go
- file_exporter.go
- http_exporter.go
- json.go
- lambda_log_exporter.go
- operations_by_name.go
- operations_format.go
- plugin.go
- record.go
- sql_identifier.go
- truncate.go