dynamodb

Overview ¶

Package dynamodb is the docstore.Store for Amazon DynamoDB (and DynamoDB Local / LocalStack for development).

Table layout ¶

One table with a string partition key "pk" and a string sort key "sk":

• documents: pk = "<site>#<collection>", sk = <id>, plus attributes data (S, the JSON body — preserving JSON types exactly), created_at and updated_at (N, unix nanoseconds), created_by / updated_by (S), version (N) and, when Create or UpdateIf wrote the item, w (S, a random identity of that write call).
• collection counters: pk = "<site>#__collections", sk = <collection>, n (N), maintained with atomic ADD on every create and delete. Collections lists counters with n > 0 and Count sums them. Counters are updated right after the document write (not transactionally, to avoid transaction conflicts on hot collections), so a crash between the two calls can leave a counter off by one; Count is only used for the soft per-site quota.

"#" and "__collections" cannot clash with valid site or collection names.

Writes ¶

Create is a conditional PutItem (attribute_not_exists(pk)). Put is a single UpdateItem that preserves created_* with if_not_exists and increments the version atomically. Update reads the document, merges the patch and writes it back conditioned on the item being unchanged since the read (version, created_at and updated_at, so a delete and re-create in between is noticed too), retrying on conflicts (optimistic concurrency), so concurrent updates never lose keys. UpdateIf and DeleteIf write once, conditioned on the revision read.

The SDK sends a write again when its answer is lost (an HTTP 500, which DynamoDB documents as "may have succeeded", a dropped connection), and the condition of that attempt then fails on what the first one stored. Create and UpdateIf give the item they write a random identity of the call (w) and compare the item their condition failed on with the one they wrote: when it is the same, w included, they report their own write as done, not as a conflict. Data, actor, version and updated_at alone do not identify a write: another replica can write the same values in the same microsecond.

Queries (limitation) ¶

DynamoDB cannot evaluate filters inside a JSON string attribute. Query reads the collection's items with a strongly consistent Query on pk — at most database.dynamodb.max_scan items (default 10000), in id order — and then filters, sorts and paginates in memory with the reference evaluator (docstore.Match / docstore.EvalQuery). A query also stops after reading 64 MiB of document data. Results are therefore exact for collections up to max_scan documents (and 64 MiB); for larger collections only the first documents by id are considered (a warning is logged). Each query costs read capacity proportional to the collection size.

Documents are limited by DynamoDB's 400 KB item size; keep database.max_doc_bytes below ~390000.