dynamodb

Overview ¶

Package dynamodb is the docstore.Store for Amazon DynamoDB (and DynamoDB Local / LocalStack for development).

Table layout ¶

One table with a string partition key "pk" and a string sort key "sk":

• documents: pk = "<site>#<collection>", sk = <id>, plus attributes data (S, the JSON body — preserving JSON types exactly), created_at and updated_at (N, unix nanoseconds), created_by / updated_by (S) and version (N).
• collection counters: pk = "<site>#__collections", sk = <collection>, n (N), maintained with atomic ADD on every create and delete. Collections lists counters with n > 0 and Count sums them. Counters are updated right after the document write (not transactionally, to avoid transaction conflicts on hot collections), so a crash between the two calls can leave a counter off by one; Count is only used for the soft per-site quota.

"#" and "__collections" cannot clash with valid site or collection names.

Writes ¶

Create is a conditional PutItem (attribute_not_exists(pk)). Put is a single UpdateItem that preserves created_* with if_not_exists and increments the version atomically. Update reads the document, merges the patch and writes it back conditioned on the item being unchanged since the read (version, created_at and updated_at, so a delete and re-create in between is noticed too), retrying on conflicts (optimistic concurrency), so concurrent updates never lose keys.

Queries (limitation) ¶

DynamoDB cannot evaluate filters inside a JSON string attribute. Query reads the collection's items with a strongly consistent Query on pk — at most database.dynamodb.max_scan items (default 10000), in id order — and then filters, sorts and paginates in memory with the reference evaluator (docstore.Match / docstore.EvalQuery). A query also stops after reading 64 MiB of document data. Results are therefore exact for collections up to max_scan documents (and 64 MiB); for larger collections only the first documents by id are considered (a warning is logged). Each query costs read capacity proportional to the collection size.

Documents are limited by DynamoDB's 400 KB item size; keep database.max_doc_bytes below ~390000.