Documentation
¶
Overview ¶
Package dynaflux runs event-sourced fact streams and materialized views on DynamoDB Global Tables. You declare streams, views and queries in SQL, and the dynaflux command generates typed Go code and infrastructure from them.
An application opens a Store, as Open's example shows, and passes it to the generated package: here flux, the client of a store with a stream page_views and a query PageStats over a view of it:
client := flux.New(store)
An append with Waitable returns a token that a query can wait for, so that the query's rows include the event:
view := flux.PageView{UserID: "u1", Page: "/home", EventTime: time.Now()}
tok, err := client.AppendPageView(ctx, view, dynaflux.Waitable())
if err != nil {
return err
}
day := flux.PageStatsParams{Page: view.Page, From: view.EventTime, To: view.EventTime}
rows, err := client.PageStats(ctx, day, dynaflux.WaitFor(tok, 5*time.Second))
if err != nil {
return err
}
for _, row := range rows {
fmt.Println(row.Day, row.Views, row.Visitors)
}
A stream read is a range loop, which yields at most one error:
key := flux.PageViewsKey{UserID: "u1"}
for ev, err := range client.ReadPageViews(ctx, key, dynaflux.Since(time.Now().Add(-time.Hour))) {
if err != nil {
return err
}
fmt.Println(ev.Page, ev.EventTime)
}
Errors wrap the sentinels below, for errors.Is, and the types ValueError and BatchError, for errors.AsType:
switch ve, ok := errors.AsType[*dynaflux.ValueError](err); {
case ok:
fmt.Println("Dynaflux can't store the value of", ve.Name) // a column or query parameter
case errors.Is(err, dynaflux.ErrDuplicate):
// A stream with DEDUP BY stored the event before; tok names that event.
case errors.Is(err, dynaflux.ErrWaitTimeout):
// The projector hasn't handled the event yet: query again later.
}
The getting-started guide builds and deploys such an application.
Index ¶
Examples ¶
Constants ¶
const GeneratedCodeVersion1 = 1
GeneratedCodeVersion1 is the contract between the code that dynaflux generate writes and this module. Generated code names the generator's current GeneratedCodeVersionN, whatever features the project uses. When new generated code needs something that an older module lacks, the generator names the next N, so that code fails to compile at that line against the older module, not with "undefined: …". Every v1 release keeps every constant it has defined.
Variables ¶
var ( // ErrDuplicate is an event that another append stored before: the same PRIMARY KEY, // event_time and DEDUP BY values. The append returns that event's Token. ErrDuplicate = engine.ErrDuplicate // "dynaflux: duplicate event" // ErrOutsideWindow is an event_time outside the stream's ACCEPT window. An append with // [AsImport] skips the window. ErrOutsideWindow = engine.ErrOutsideWindow // "dynaflux: event_time outside the ACCEPT window" // ErrInvalidValue is what Dynaflux can't store: // - a value, which a [ValueError] names // - an event over DynamoDB's 400 KB item limit // - an event of an archived stream whose archive record could be over Firehose's // record limit of 1,024,000 bytes; the error names the stream and the size ErrInvalidValue = engine.ErrInvalidValue // "invalid value" // ErrNotFound is a :one query that found no row. ErrNotFound = engine.ErrNotFound // "dynaflux: no row found" // ErrTooManyKeys is a query call whose IN parameters make more than 100 key // combinations to read. ErrTooManyKeys = engine.ErrTooManyKeys // "too many key combinations" // ErrEvaluation is an expression of a query that failed on the rows it read, such as an // integer division by zero, or a sum outside its result type; or one of a SYNC view // that failed on an event appended without [AsImport], which then isn't written. The // error names the view version. ErrEvaluation = engine.ErrEvaluation // "evaluation error" // ErrViewNotReady is a query over a view whose version isn't live in the store's region: // no backfill has made it live yet, or it broke or was retired. The error names the // version, its state and the region, and a broken version's failure. [AllowPartial] // reads the view anyway. ErrViewNotReady = engine.ErrViewNotReady // "view version not ready" // ErrWaitTimeout is a query whose [WaitFor] timeout passed before the projector had // handled every event of the token. ErrWaitTimeout = engine.ErrWaitTimeout // "wait for an append timed out" )
Errors to check with errors.Is. Returned errors wrap them with details.
Functions ¶
This section is empty.
Types ¶
type AppendOption ¶
type AppendOption interface {
// contains filtered or unexported methods
}
AppendOption configures an append. Every generated append method takes them.
func AsImport ¶
func AsImport() AppendOption
AsImport appends past events as an import: the append skips the stream's ACCEPT window, and it updates no SYNC view in its transaction, so a SYNC view gives no read-your-writes for them. The projector applies the events to every view, as it does for ASYNC views: an event that a SYNC view can't evaluate is stored anyway, and breaks that view version, as it would an ASYNC one. Combine it with Waitable to wait for them. Into a stream without DEDUP BY, a retried import stores the events again.
func Waitable ¶
func Waitable() AppendOption
Waitable asks the projector of the store's region to write a receipt for each event of the append once it has handled it, so that WaitFor can wait for them: one more write to the store's meta table per event, which other appends don't pay.
type BatchError ¶
type BatchError struct {
Errs []error
}
BatchError is the error of a batch append. Errs lines up with the events, nil for each one the call stored.
Example ¶
A retried batch on a stream with DEDUP BY reports the events that an earlier try stored as ErrDuplicate. errors.Is on the whole error can't tell those from failures, so check each event. Here the retry stored event 0, an earlier try stored event 1, and event 2 isn't stored.
package main
import (
"errors"
"fmt"
"github.com/lucindo/dynaflux"
)
func main() {
err := error(&dynaflux.BatchError{Errs: []error{
nil,
dynaflux.ErrDuplicate,
&dynaflux.ValueError{Name: "price", Err: fmt.Errorf("%w: more digits than its scale", dynaflux.ErrInvalidValue)},
}})
if be, ok := errors.AsType[*dynaflux.BatchError](err); ok {
for i, e := range be.Errs {
if e != nil && !errors.Is(e, dynaflux.ErrDuplicate) {
fmt.Printf("event %d was not stored: %v\n", i, e)
}
}
}
}
Output: event 2 was not stored: dynaflux: price: invalid value: more digits than its scale
func (*BatchError) Error ¶
func (e *BatchError) Error() string
func (*BatchError) Unwrap ¶
func (e *BatchError) Unwrap() []error
Unwrap returns the errors of the events that failed.
type Catalog ¶
type Catalog struct {
// contains filtered or unexported fields
}
Catalog holds a store's definitions, compiled. The generated package declares one, as Catalog in its catalog.go. The zero Catalog can't be used.
type Config ¶
type Config struct {
// AWS configures the DynamoDB client. Its Region is the region the store appends in,
// which events record as origin_region; it must be one of the regions in dynaflux.yaml.
// Dynaflux uses its credentials, retryer, HTTP client and endpoint as they are.
//
// The projector's client is a model to follow:
// - the standard retryer without its client-side retry quota (ratelimit.None)
// - a 10 s HTTP timeout for each attempt
AWS aws.Config
// Prefix is the deployment's prefix of resource names: the templates' Prefix
// parameter, "" for none.
Prefix string
}
Config configures the Store that Open returns.
type Decimal ¶
type Decimal struct {
// contains filtered or unexported fields
}
Decimal is an exact decimal number of at most 38 digits that keeps its scale, the digits after its point: 12.50 has scale 2. The zero value is 0.
== compares the representation, scale included, so 12.5 and 12.50 differ under == and are Decimal.Equal. Values read from one Decimal(P, S) column all have scale S.
func DecimalFromBig ¶
DecimalFromBig returns unscaled × 10^−scale. It fails when unscaled has more than 38 digits or scale is outside 0 to 38.
func ParseDecimal ¶
ParseDecimal parses an optional '-', digits, and optionally '.' and more digits, such as "-12.50". The scale is the number of digits after the point.
Example ¶
A Decimal keeps its scale: == compares the representation, and Equal the value.
package main
import (
"fmt"
"github.com/lucindo/dynaflux"
)
func main() {
a, err := dynaflux.ParseDecimal("12.5")
if err != nil {
fmt.Println(err)
return
}
b, err := dynaflux.ParseDecimal("12.50")
if err != nil {
fmt.Println(err)
return
}
fmt.Println(a, b)
fmt.Println(a == b, a.Equal(b))
}
Output: 12.5 12.50 false true
func (Decimal) Compare ¶
Compare returns -1, 0 or +1 as d is less than, equal to or greater than e, by value.
func (Decimal) MarshalText ¶
MarshalText implements encoding.TextMarshaler with Decimal.String.
func (Decimal) String ¶
String returns d with exactly its scale's digits after the point, such as "-0.05".
func (*Decimal) UnmarshalText ¶
UnmarshalText implements encoding.TextUnmarshaler with ParseDecimal.
type QueryOption ¶
type QueryOption interface {
// contains filtered or unexported methods
}
QueryOption configures a query.
func AllowPartial ¶
func AllowPartial() QueryOption
AllowPartial runs a query over a view whose version isn't live in the store's region, without reading its state, instead of returning ErrViewNotReady. The rows may miss events: those a backfill hasn't applied yet, or every event after the version broke.
func WaitFor ¶
func WaitFor(tok Token, timeout time.Duration) QueryOption
WaitFor makes a query first wait until the projector of the store's region has handled every event of tok, an append's token in this region, for at most timeout. A query over a view then reads its version's state again, and both kinds read strongly consistently, so the rows include the events. A wait takes about the projector's batching window plus one invocation, 1 to 1.5 seconds by default: give it a timeout of a few seconds. When the timeout passes first, the query returns an error that matches ErrWaitTimeout; when ctx ends first, ctx's error.
The query fails at once, before any read:
- in another region than the append's
- for an event appended without Waitable
- for an event that another region stored, as a duplicate of a replicated DEDUP BY event is
- for a token of more than 1,000 events
- for a token of a stream that the store doesn't have
A token can be waited on for 24 hours after its append. The zero Token waits for nothing. Given more than once, WaitFor makes the query wait for each token in turn, each for its own timeout.
type ReadOption ¶
type ReadOption interface {
// contains filtered or unexported methods
}
ReadOption configures a stream read.
func Since ¶
func Since(t time.Time) ReadOption
Since reads the events whose event_time is t or later.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store connects generated clients to every Dynaflux store deployed with Config.Prefix in the region of Config.AWS. In SQL and in errors, a store is the catalog of one generated package, such as store analytics; one Store serves all of them. Open returns one backed by DynamoDB; in tests, fluxtest.NewStore makes one in memory. A Store is safe for concurrent use. The zero Store can't be used.
func Open ¶
Open returns a store backed by the DynamoDB tables that the generated templates create for cfg.Prefix in cfg.AWS.Region, which must be a region code such as us-east-1: for FIPS endpoints, set AWS_USE_FIPS_ENDPOINT instead of naming a FIPS pseudo-region. It makes no AWS call, so a missing table or grant shows on the first append, read or query.
A query over a view checks that the view's version is live in the region, in the view states it read from the store's meta table at most 30 seconds before, and returns ErrViewNotReady when it isn't. The application needs dynamodb:Query on the meta table for that read.
Example ¶
An application's Store, with the AWS client that Config.AWS recommends. The prefix is the one dynaflux dev prints, or the one the templates were deployed with.
package main
import (
"context"
"fmt"
"os"
"time"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/aws/ratelimit"
"github.com/aws/aws-sdk-go-v2/aws/retry"
awshttp "github.com/aws/aws-sdk-go-v2/aws/transport/http"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/lucindo/dynaflux"
)
func main() {
ctx := context.Background()
cfg, err := config.LoadDefaultConfig(ctx,
config.WithRetryer(func() aws.Retryer {
return retry.NewStandard(func(o *retry.StandardOptions) { o.RateLimiter = ratelimit.None })
}),
config.WithHTTPClient(awshttp.NewBuildableClient().WithTimeout(10*time.Second)))
if err != nil {
fmt.Println(err)
return
}
store, err := dynaflux.Open(dynaflux.Config{AWS: cfg, Prefix: os.Getenv("DYNAFLUX_PREFIX")})
if err != nil {
fmt.Println(err)
return
}
_ = store // pass it to the generated package's New
}
Output:
type Token ¶
type Token struct {
// contains filtered or unexported fields
}
Token names the events that an append stored, in the store's region. The zero Token names none.
func ParseToken ¶
ParseToken returns the Token whose text form is s, so that another process in the same region can wait for its events with WaitFor. It checks only the form: a token is unauthenticated text that grants nothing, since WaitFor reads only receipts with it.
The text form is a wire format. Every release reads the token formats of the releases before it, and a release writes a new format only after a release that reads it has shipped. So during a rolling deploy from the previous release, each process reads the tokens of the others.
func (Token) MarshalText ¶
MarshalText implements encoding.TextMarshaler with Token.String; the zero Token is "".
Example ¶
A Token in a JSON document is its text, and the zero Token is "".
package main
import (
"encoding/json"
"fmt"
"github.com/lucindo/dynaflux"
)
func main() {
tok, err := dynaflux.ParseToken("AQVsb2NhbApwYWdlX3ZpZXdzANxqz6wA2Yd-zm02iqwab4DA39mToNcBAA")
if err != nil {
fmt.Println(err)
return
}
type receipt struct {
Token dynaflux.Token `json:"wait"`
Prior dynaflux.Token `json:"prior"`
}
data, err := json.Marshal(receipt{Token: tok})
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(data))
var back receipt
if err = json.Unmarshal(data, &back); err != nil {
fmt.Println(err)
return
}
fmt.Println(back.Token == tok, back.Prior.IsZero())
}
Output: {"wait":"AQVsb2NhbApwYWdlX3ZpZXdzANxqz6wA2Yd-zm02iqwab4DA39mToNcBAA","prior":""} true true
func (Token) String ¶
String returns the token's text form, which is opaque. ParseToken reads it back.
func (*Token) UnmarshalText ¶
UnmarshalText implements encoding.TextUnmarshaler with ParseToken; "" is the zero Token.
type UUID ¶
type UUID [16]byte
UUID is a 128-bit UUID. Its methods behave as those of Go 1.27's uuid.UUID, and the two types convert into each other for free. UUID stays a [16]byte type of its own for all of v1. To make one, read crypto/rand into it, rand.Read(id[:]), or on Go 1.27 convert uuid.NewV7: dynaflux.UUID(uuid.NewV7()).
func ParseUUID ¶
ParseUUID parses a UUID in one of the forms Go 1.27's uuid.Parse accepts, with hex digits in either case:
f81d4fae-7dec-11d0-a765-00a0c91e6bf6
{f81d4fae-7dec-11d0-a765-00a0c91e6bf6}
urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6
f81d4fae7dec11d0a76500a0c91e6bf6
Example ¶
ParseUUID accepts four forms, and String prints one.
package main
import (
"fmt"
"github.com/lucindo/dynaflux"
)
func main() {
for _, s := range []string{
"f81d4fae-7dec-11d0-a765-00a0c91e6bf6",
"{F81D4FAE-7DEC-11D0-A765-00A0C91E6BF6}",
"urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6",
"f81d4fae7dec11d0a76500a0c91e6bf6",
} {
u, err := dynaflux.ParseUUID(s)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(u)
}
}
Output: f81d4fae-7dec-11d0-a765-00a0c91e6bf6 f81d4fae-7dec-11d0-a765-00a0c91e6bf6 f81d4fae-7dec-11d0-a765-00a0c91e6bf6 f81d4fae-7dec-11d0-a765-00a0c91e6bf6
func (UUID) AppendText ¶
AppendText implements encoding.TextAppender with the form of UUID.String.
func (UUID) MarshalText ¶
MarshalText implements encoding.TextMarshaler with the form of UUID.String.
func (*UUID) UnmarshalText ¶
UnmarshalText implements encoding.TextUnmarshaler; it accepts the forms of ParseUUID.
type ValueError ¶
type ValueError struct {
// Name is the column or the query parameter, as in SQL; for a PRIMARY KEY too long for
// DynamoDB, the key's columns joined by ", ", such as "org_id, user_id".
Name string
Err error
}
ValueError is a value that Dynaflux can't store: invalid UTF-8, a Float64 that isn't finite or is outside DynamoDB's range, a Decimal outside its type, a time outside 1970 to 2106, an array of more than 1,000 elements, a PRIMARY KEY longer than DynamoDB allows, or a missing event_time on a stream with DEDUP BY. Err matches ErrInvalidValue.
func (*ValueError) Error ¶
func (e *ValueError) Error() string
func (*ValueError) Unwrap ¶
func (e *ValueError) Unwrap() error
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
dynaflux
command
Dynaflux checks a Dynaflux project's SQL definitions and dynaflux.yaml, explains the layout and cost of its streams, views and queries, generates its typed Go client package and dynaflux.lock, and reads and operates the deployed store.
|
Dynaflux checks a Dynaflux project's SQL definitions and dynaflux.yaml, explains the layout and cost of its streams, views and queries, generates its typed Go client package and dynaflux.lock, and reads and operates the deployed store. |
|
examples
|
|
|
analytics
Package analytics is application code that uses the example's generated client.
|
Package analytics is application code that uses the example's generated client. |
|
analytics/cmd/dynaflux-runtime
command
Command dynaflux-runtime is the Lambda binary of store analytics.
|
Command dynaflux-runtime is the Lambda binary of store analytics. |
|
analytics/internal/flux
Package flux is the typed client of store analytics, generated by dynaflux from its SQL definitions.
|
Package flux is the typed client of store analytics, generated by dynaflux from its SQL definitions. |
|
Package fluxgen holds the functions that code generated by dynaflux generate calls.
|
Package fluxgen holds the functions that code generated by dynaflux generate calls. |
|
Package fluxruntime is the entry point of the Lambda binary that dynaflux generate writes to runtime.out.
|
Package fluxruntime is the entry point of the Lambda binary that dynaflux generate writes to runtime.out. |
|
Package fluxtest runs a Dynaflux store in memory, for tests of code that uses a generated client.
|
Package fluxtest runs a Dynaflux store in memory, for tests of code that uses a generated client. |
|
internal
|
|
|
access
Package access lets fluxgen, fluxtest and fluxruntime build and open the types of package dynaflux, which has no exported constructor that takes internal types.
|
Package access lets fluxgen, fluxtest and fluxruntime build and open the types of package dynaflux, which has no exported constructor that takes internal types. |
|
agg
Package agg implements the aggregate states of docs/spec/encoding.md §7.2–§7.5: what a view stores for an aggregate column, how inputs and other states merge into it, its finalized value, and the largest attribute it can need (§7.8).
|
Package agg implements the aggregate states of docs/spec/encoding.md §7.2–§7.5: what a view stores for an aggregate column, how inputs and other states merge into it, its finalized value, and the largest attribute it can need (§7.8). |
|
archive
Package archive is the schema of the event archive: in each archiving region, an Iceberg table per archived stream, which Firehose writes, and an Athena view of it with one row per event.
|
Package archive is the schema of the event archive: in each archiving region, an Iceberg table per archived stream, which Firehose writes, and an Athena view of it with one row per event. |
|
archive/athenafail
Package athenafail classifies Athena's failures by what a caller does about them, for the setup function's statements and the backfill's exports (ADR 0016).
|
Package athenafail classifies Athena's failures by what a caller does about them, for the setup function's statements and the backfill's exports (ADR 0016). |
|
archive/firehose
Package firehose writes a projector batch's INSERT records to the stream's Firehose stream, which feeds the region's Iceberg table.
|
Package firehose writes a projector batch's INSERT records to the stream's Firehose stream, which feeds the region's Iceberg table. |
|
backfill
Package backfill fills one view version in one region (docs/spec/lifecycle.md §5, ADR 0011).
|
Package backfill fills one view version in one region (docs/spec/lifecycle.md §5, ADR 0011). |
|
backfill/awsbackfill
Package awsbackfill implements the interfaces of internal/backfill with the AWS SDK, for the backfill function of the runtime binary and for dynaflux backfill: staged runs in the region's bucket, a stream's shards, the replicas of a stream table, the projectors' lag, and the archive with what AWS shows of its streams' history.
|
Package awsbackfill implements the interfaces of internal/backfill with the AWS SDK, for the backfill function of the runtime binary and for dynaflux backfill: staged runs in the region's bucket, a stream's shards, the replicas of a stream table, the projectors' lag, and the archive with what AWS shows of its streams' history. |
|
backfill/awsbackfill/awsbackfilltest
Package awsbackfilltest serves, over HTTP, the AWS calls of package awsbackfill that the emulator lacks, for tests: S3 objects in memory, Athena running package archive's statements over records in memory, CloudWatch with series a test sets or without data, Firehose's description of a delivery stream, and Lambda's event source mappings.
|
Package awsbackfilltest serves, over HTTP, the AWS calls of package awsbackfill that the emulator lacks, for tests: S3 objects in memory, Athena running package archive's statements over records in memory, CloudWatch with series a test sets or without data, Firehose's description of a delivery stream, and Lambda's event source mappings. |
|
catalog
Package catalog is the compiled model of a Dynaflux store: its streams, every live version of its views, and its named queries, with expressions as a typed IR.
|
Package catalog is the compiled model of a Dynaflux store: its streams, every live version of its views, and its named queries, with expressions as a typed IR. |
|
catalog/catalogtest
Package catalogtest builds catalogs from SQL for tests.
|
Package catalogtest builds catalogs from SQL for tests. |
|
cli
Package cli implements the dynaflux command.
|
Package cli implements the dynaflux command. |
|
codegen
Package codegen writes the typed Go client that dynaflux generate puts in go.out, and the main.go of the runtime binary that it puts in runtime.out.
|
Package codegen writes the typed Go client that dynaflux generate puts in go.out, and the main.go of the runtime binary that it puts in runtime.out. |
|
config
Package config loads dynaflux.yaml, the configuration of one Dynaflux store, as docs/spec/config.md specifies.
|
Package config loads dynaflux.yaml, the configuration of one Dynaflux store, as docs/spec/config.md specifies. |
|
deployed
Package deployed reads one region of a deployed store for the operator commands: its registered catalog, and from the services around the meta table the backfill jobs' executions, the event source mappings, the archive's metrics, and the gaps that Lambda recorded in the store's bucket.
|
Package deployed reads one region of a deployed store for the operator commands: its registered catalog, and from the services around the meta table the backfill jobs' executions, the event source mappings, the archive's metrics, and the gaps that Lambda recorded in the store's bucket. |
|
diag
Package diag holds the diagnostics the Dynaflux front end reports for SQL definitions, dynaflux.yaml and dynaflux.lock, in the format of docs/spec/sql.md appendix A.1.
|
Package diag holds the diagnostics the Dynaflux front end reports for SQL definitions, dynaflux.yaml and dynaflux.lock, in the format of docs/spec/sql.md appendix A.1. |
|
emf
Package emf publishes counts in a function's log line as CloudWatch metrics, in the embedded metric format, under the namespace and dimension that the generated alarms read.
|
Package emf publishes counts in a function's log line as CloudWatch metrics, in the embedded metric format, under the namespace and dimension that the generated alarms read. |
|
emulatortest
Package emulatortest connects tests to DynamoDB Local (docs/decisions/0009-emulator-choice.md): clients that can reach only a loopback endpoint, tables of their own for each test, and a recorder of every request a client sends.
|
Package emulatortest connects tests to DynamoDB Local (docs/decisions/0009-emulator-choice.md): clients that can reach only a loopback endpoint, tables of their own for each test, and a recorder of every request a client sends. |
|
engine
Package engine appends to and reads streams, runs named queries and reads view rows over a store.Backend, with the rules of sql.md §4.3, §8 and §10, for every way into a store: generated code, the CLI and the query handler.
|
Package engine appends to and reads streams, runs named queries and reads view rows over a store.Backend, with the rules of sql.md §4.3, §8 and §10, for every way into a store: generated code, the CLI and the query handler. |
|
esmlocal
Package esmlocal reads a DynamoDB stream on an emulator and renders its records as the tumbling-window events Lambda's event source mapping sends (docs/decisions/0003).
|
Package esmlocal reads a DynamoDB stream on an emulator and renders its records as the tumbling-window events Lambda's event source mapping sends (docs/decisions/0003). |
|
explain
Package explain estimates what a stream and its views cost to run: request units, row sizes, projector invocations and monthly dollars, per region.
|
Package explain estimates what a stream and its views cost to run: request units, row sizes, projector invocations and monthly dollars, per region. |
|
expr
Package expr evaluates the catalog's expressions over SQL values.
|
Package expr evaluates the catalog's expressions over SQL values. |
|
fanout
Package fanout runs a function over indexes on goroutines, a bounded number at a time.
|
Package fanout runs a function over indexes on goroutines, a bounded number at a time. |
|
goname
Package goname derives the Go names that dynaflux generate declares (sql.md §3.2).
|
Package goname derives the Go names that dynaflux generate declares (sql.md §3.2). |
|
gotype
Package gotype maps SQL types to the Go types of generated code, for code generation and for package fluxgen alike.
|
Package gotype maps SQL types to the Go types of generated code, for code generation and for package fluxgen alike. |
|
infra
Package infra builds the infrastructure manifest: the description of one store's AWS resources that dynaflux generate derives from dynaflux.yaml and the catalog, and that a renderer such as package cloudformation turns into templates.
|
Package infra builds the infrastructure manifest: the description of one store's AWS resources that dynaflux generate derives from dynaflux.yaml and the catalog, and that a renderer such as package cloudformation turns into templates. |
|
infra/cloudformation
Package cloudformation renders an infrastructure manifest as CloudFormation templates, which docs/spec/infra.md describes.
|
Package cloudformation renders an infrastructure manifest as CloudFormation templates, which docs/spec/infra.md describes. |
|
infra/infratest
Package infratest evaluates a manifest's IAM grants for tests, as IAM would for a request.
|
Package infratest evaluates a manifest's IAM grants for tests, as IAM would for a request. |
|
keys
Package keys implements the storage encoding of docs/spec/encoding.md: item keys and sort-key ranges, hashes and identifiers, attribute values, and high-water marks.
|
Package keys implements the storage encoding of docs/spec/encoding.md: item keys and sort-key ranges, hashes and identifiers, attribute values, and high-water marks. |
|
localstore
Package localstore creates and deletes a store's tables on DynamoDB Local as the manifest of internal/infra declares them: keys, on-demand billing, the stream view and the TTL attribute, so a disagreement between the templates and the runtime shows locally.
|
Package localstore creates and deletes a store's tables on DynamoDB Local as the manifest of internal/infra declares them: keys, on-demand billing, the stream view and the TTL attribute, so a disagreement between the templates and the runtime shows locally. |
|
meta
Package meta reads and writes the items of a store's regional meta table (docs/spec/encoding.md §8): one unit item per delivery unit, with its progress and fences; one view-state item per view version; one job item per backfill job; and the deployed catalog.
|
Package meta reads and writes the items of a store's regional meta table (docs/spec/encoding.md §8): one unit item per delivery unit, with its progress and fences; one view-state item per view version; one job item per backfill job; and the deployed catalog. |
|
projector
Package projector keeps views current: it applies a delivery unit's change-stream records to every view version of their stream, with conditional writes that make a redelivered record count once (docs/spec/encoding.md §7).
|
Package projector keeps views current: it applies a delivery unit's change-stream records to every view version of their stream, with conditional writes that make a redelivered record count once (docs/spec/encoding.md §7). |
|
projector/projectortest
Package projectortest selects a backfill's archive records the simple way, as the oracle of tests of the backfill.
|
Package projectortest selects a backfill's archive records the simple way, as the oracle of tests of the backfill. |
|
queryhttp
Package queryhttp serves a store's exposed named queries, those annotated -- expose: http, to Lambda function URLs, API Gateway HTTP and REST APIs, and direct invoke (ADR 0018).
|
Package queryhttp serves a store's exposed named queries, those annotated -- expose: http, to Lambda function URLs, API Gateway HTTP and REST APIs, and direct invoke (ADR 0018). |
|
setup
Package setup answers the CloudFormation custom resources of regional.yaml, which the runtime binary serves in its setup mode (ADR 0008, docs/spec/infra.md §6.2, §7.2):
|
Package setup answers the CloudFormation custom resources of regional.yaml, which the runtime binary serves in its setup mode (ADR 0008, docs/spec/infra.md §6.2, §7.2): |
|
setup/awssetup
Package awssetup implements the interfaces of internal/setup with the AWS SDK, for the setup function of the runtime binary (ADR 0008).
|
Package awssetup implements the interfaces of internal/setup with the AWS SDK, for the setup function of the runtime binary (ADR 0008). |
|
simulation
Package simulation is the deterministic harness for the projector: regions, change streams, Lambda's event source mapping, faults and an oracle, all in one testing/synctest bubble per seed, so a failing seed replays exactly.
|
Package simulation is the deterministic harness for the projector: regions, change streams, Lambda's event source mapping, faults and an oracle, all in one testing/synctest bubble per seed, so a failing seed replays exactly. |
|
simulation/mutants
command
Command mutants runs the simulation harness's negative controls, several at once.
|
Command mutants runs the simulation harness's negative controls, several at once. |
|
sql
Package sql parses Dynaflux SQL files into a syntax tree that keeps the shape and positions of the source.
|
Package sql parses Dynaflux SQL files into a syntax tree that keeps the shape and positions of the source. |
|
store
Package store is the storage boundary: DynamoDB in production, an in-memory store in tests.
|
Package store is the storage boundary: DynamoDB in production, an in-memory store in tests. |
|
store/dynamo
Package dynamo is the DynamoDB store.Backend.
|
Package dynamo is the DynamoDB store.Backend. |
|
store/mem
Package mem is an in-memory store.Backend with the DynamoDB behavior Dynaflux depends on: sort keys in byte order, conditions that return the stored item, size limits, numbers stored in DynamoDB's form, and the requests DynamoDB rejects.
|
Package mem is an in-memory store.Backend with the DynamoDB behavior Dynaflux depends on: sort keys in byte order, conditions that return the stored item, size limits, numbers stored in DynamoDB's form, and the requests DynamoDB rejects. |
|
store/storetest
Package storetest holds the conformance suite that every store.Backend passes, and two faults: an SDK retry after a lost response, and eventually consistent reads that lag.
|
Package storetest holds the conformance suite that every store.Backend passes, and two faults: an SDK retry after a lost response, and eventually consistent reads that lag. |
|
streamevent
Package streamevent decodes the events of a DynamoDB stream with a tumbling window (ADR 0003), as Lambda delivers them to a projector function and esmlocal does in dynaflux dev, and holds the function's reply.
|
Package streamevent decodes the events of a DynamoDB stream with a tumbling window (ADR 0003), as Lambda delivers them to a projector function and esmlocal does in dynaflux dev, and holds the function's reply. |
|
txtar
Package txtar reads and writes txtar archives, with the semantics of golang.org/x/tools/txtar: a free-form comment, then files that each start with a marker line "-- name --".
|
Package txtar reads and writes txtar archives, with the semantics of golang.org/x/tools/txtar: a free-form comment, then files that each start with a marker line "-- name --". |
|
validate
Package validate is the compiler front end of Dynaflux.
|
Package validate is the compiler front end of Dynaflux. |
|
valuetext
Package valuetext reads and writes values of catalog types outside DynamoDB: as text, for command-line flags and tables, and as JSON, for -json output, import and the HTTP query endpoints.
|
Package valuetext reads and writes values of catalog types outside DynamoDB: as text, for command-line flags and tables, and as JSON, for -json output, import and the HTTP query endpoints. |