odatatable

package
v1.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 14, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package odatatable implements the shared entity CRUD / $filter engine behind Azure Table Storage's and Azure Cosmos DB's Table API OData/JSON wire protocol: both are wire-identical (same entity model, same $filter grammar), differing only in auth and endpoint/path conventions, which stay in each caller's own service package. This engine was originally written (and is still wire-tested end to end) as part of services/azuretable; AZURE.md section 9's M6 milestone extracted it here so services/cosmosdb's Table API could import the same engine instead of depending on services/azuretable directly, or duplicating its logic. services/azuretable itself now re-exports this package's types/functions under its historical names so its own public API and tests are unaffected by the move.

See services/azuretable/PARITY.md for the engine's known gaps (batch changesets and continuation-token pagination remain deferred); this package does not keep its own separate PARITY.md since it has no wire surface of its own -- every gap it has is a gap of whichever service wires it up.

Index

Constants

View Source 
const (
	MetadataLevelNoMetadata = "nometadata"
	MetadataLevelMinimal    = "minimalmetadata"
	MetadataLevelFull       = "fullmetadata"
)

OData metadata level names, negotiated by each caller from its own request's Accept header and passed into EncodeEntity to vary response shape. Exported so services/azuretable and services/cosmosdb's Table API can compare their own negotiated level against a shared vocabulary instead of each hand-rolling the same three string literals.

View Source 
const (
	PartitionKeyProperty = "PartitionKey"
	RowKeyProperty       = "RowKey"
	TimestampProperty    = "Timestamp"
)

System entity property names, shared across path-predicate parsing (keys.go), $filter identifier resolution (eval.go), and entity body decode/encode (codec.go).

View Source 
const EntityTimeLayout = "2006-01-02T15:04:05.9999999Z"

EntityTimeLayout formats an Edm.DateTime property/Timestamp value on the wire: a variable-precision (trailing zeros trimmed) RFC3339 string, matching aztables' own EDMDateTime.MarshalText layout exactly so its time.Parse round-trips cleanly.

View Source 
const IfMatchAny = "*"

IfMatchAny is the wildcard If-Match value ("*") meaning "must currently exist, but match unconditionally on ETag" -- as opposed to an empty ifMatch (no header at all, meaning upsert semantics) or a specific ETag string (optimistic-concurrency match required).

Variables

View Source 
var (
	ErrTableNotFound       = errors.New("odatatable: table not found")
	ErrTableAlreadyExists  = errors.New("odatatable: table already exists")
	ErrEntityNotFound      = errors.New("odatatable: entity not found")
	ErrEntityAlreadyExists = errors.New("odatatable: entity already exists")
	ErrETagMismatch        = errors.New("odatatable: etag mismatch")

	// ErrInvalidEntityKey is returned when a request omits PartitionKey or
	// RowKey entirely. Empty-string keys are accepted (matching real Azure
	// Table Storage); only an absent key is rejected. See codec.go.
	ErrInvalidEntityKey = errors.New("odatatable: PartitionKey and RowKey are required")

	// ErrInvalidEntityProperty is returned when an entity property's JSON
	// value cannot be decoded under its (explicit or inferred) EDM type.
	ErrInvalidEntityProperty = errors.New("odatatable: invalid entity property value")

	// ErrFilterParse and ErrFilterTooDeep are returned by ParseFilter. A
	// parse error always surfaces as 400 InvalidInput, never a panic or 500
	// -- see each caller's queryEntities-shaped handler.
	ErrFilterParse   = errors.New("odatatable: $filter parse error")
	ErrFilterTooDeep = errors.New("odatatable: $filter expression nested too deeply")

	// ErrSnapshotTableNull and ErrSnapshotEntityNull are returned by Restore
	// when a snapshot's "tables" map (or a table's "Entities" map) holds a
	// JSON null entry, which decodes to a nil pointer that would panic on
	// first dereference if stored as-is. See persistence.go.
	ErrSnapshotTableNull  = errors.New("odatatable: restore snapshot: table is null")
	ErrSnapshotEntityNull = errors.New("odatatable: restore snapshot: entity is null")

	// ErrSnapshotTableNameMismatch is returned by Restore when a snapshot's
	// "tables" map key differs from that entry's storedTable.Name. Table
	// operations all key off the map, while ListTables reads Name -- a
	// mismatch would let those two views disagree about a table's identity.
	// See persistence.go.
	ErrSnapshotTableNameMismatch = errors.New("odatatable: restore snapshot: table map key does not match Name")

	// ErrSnapshotEntityKeyMismatch is returned by Restore when a table's
	// "Entities" map key differs from that entry's own derived
	// (PartitionKey, RowKey) key. Entity operations all key off the map, so
	// a mismatch would let a stored entity be reachable under one identity
	// while its own fields claim another. See persistence.go.
	ErrSnapshotEntityKeyMismatch = errors.New(
		"odatatable: restore snapshot: entity map key does not match PartitionKey/RowKey",
	)

	// ErrMalformedEntityCompositeKey is returned by
	// entityCompositeKey.UnmarshalText when the persisted key does not
	// decode to exactly two JSON string elements. See models.go.
	ErrMalformedEntityCompositeKey = errors.New("odatatable: malformed entity composite key")
)

Sentinel errors for the shared OData Table entity/$filter engine. These were originally services/azuretable's own sentinel errors; they moved here verbatim (message text aside) when services/azuretable's entity CRUD/ $filter logic was extracted into this package so services/cosmosdb's Table API (see AZURE.md section 9's M6) could import the same engine instead of duplicating it. services/azuretable re-exports every one of these as a package-level var of the same name for backward compatibility with its existing exported API and tests.

Functions

func DecodeEntityBody 

func DecodeEntityBody(body []byte) (string, string, bool, bool, map[string]EntityProperty, error)

DecodeEntityBody parses an entity JSON request body into its PartitionKey/RowKey (if present) and typed custom properties. "@odata.type"-annotated properties are decoded per their declared EDM type; unannotated ones are inferred the same way azure-sdk-for-go/sdk/data/aztables's own EDMEntity.UnmarshalJSON infers them client-side: try Edm.Int32 first, then fall back to the JSON value's natural type (float64 -> Edm.Double, bool -> Edm.Boolean, string -> Edm.String). "Timestamp" is silently ignored (server-managed; a client- supplied value never overwrites it -- the server always wins). Any "odata.*"/"@odata.type" metadata key is skipped.

func EncodeEntity 

func EncodeEntity(entity EntityInfo, table, level, selectParam, endpoint, accountName string) map[string]any

EncodeEntity builds an entity's OData JSON response body at the given metadata level. select, if non-empty, is a comma-separated $select projection list; PartitionKey/RowKey/Timestamp are always included regardless of select (they're the entity's identity and are cheap/always safe to return), while custom properties are filtered to the requested names. endpoint is the caller's own service base URL (e.g. "http://127.0.0.1:10002") and accountName is the fake account name used to build a fullmetadata "odata.type" value (e.g. "devstoreaccount1" for services/azuretable) -- both vary per caller, which is why they're parameters rather than baked into this package.

func EscapeODataKey 

func EscapeODataKey(s string) string

EscapeODataKey escapes a key value for embedding back into a single-quoted OData literal (the inverse of UnquoteODataString).

func EtagFor 

func EtagFor(t time.Time) string

EtagFor exposes etagFor for external tests.

func EvaluateFilter 

func EvaluateFilter(node Node, entity EntityInfo) bool

EvaluateFilter reports whether entity satisfies the parsed $filter tree node. A comparison against a property missing from entity always evaluates to false (never an error, never a panic) -- real Table Storage semantics: an absent property simply never matches. A type mismatch between the two operands of a comparison (e.g. a string compared against a number) likewise evaluates to false rather than erroring.

func ParseEntityKeyPredicate 

func ParseEntityKeyPredicate(predicate string) (string, string, bool)

ParseEntityKeyPredicate parses an entity key predicate ("PartitionKey='p',RowKey='r'", in either key order) into its two values. Returns ok=false for anything malformed.

func ParseTop 

func ParseTop(raw string) (int, error)

ParseTop parses the $top query parameter, defaulting to 0 (unlimited) when absent and clamping (never erroring on) an oversized value, per maxQueryTop's doc comment. A negative value is rejected.

func SelectSet 

func SelectSet(selectParam string) map[string]bool

SelectSet parses a $select query parameter into a lookup set of requested property names, or nil if selectParam is empty (meaning "no projection, return every property").

func SetETagFunc 

func SetETagFunc(b *InMemoryBackend, fn func(time.Time) string)

SetETagFunc replaces the backend's ETag derivation function with fn for deterministic ETag assertions.

func SetNowFunc 

func SetNowFunc(b *InMemoryBackend, fn func() time.Time)

SetNowFunc replaces the backend's time provider with fn for deterministic testing of Timestamp/ETag logic without real sleeps.

func UnquoteODataString 

func UnquoteODataString(s string) (string, bool)

UnquoteODataString unquotes a single '...'-delimited OData string literal (escaped by doubling: a single quote written twice in a row means one literal quote), such as the table-name literal in DELETE /<account>/Tables('foo'). Returns ("", false) for anything else (missing/ mismatched quotes, an unescaped quote inside).

Types

type EdmType 

type EdmType string

EdmType identifies an entity property's OData EDM (Entity Data Model) type. See https://learn.microsoft.com/rest/api/storageservices/payload-format-for-table-service-operations for the wire-level annotation scheme this mirrors. 

const (
	EdmString   EdmType = "Edm.String"
	EdmInt32    EdmType = "Edm.Int32"
	EdmInt64    EdmType = "Edm.Int64"
	EdmDouble   EdmType = "Edm.Double"
	EdmBoolean  EdmType = "Edm.Boolean"
	EdmDateTime EdmType = "Edm.DateTime"
	EdmGUID     EdmType = "Edm.Guid"
	EdmBinary   EdmType = "Edm.Binary"
)

Supported EDM property types. EdmString is the default for a bare JSON string with no "@odata.type" annotation; EdmInt32 is the default for a bare, whole-number JSON number. See codec.go's decodeProperty for the exact inference rules (mirroring azure-sdk-for-go/sdk/data/aztables's own EDMEntity.UnmarshalJSON, so unmodified SDK round trips match).

type EntityInfo 

type EntityInfo struct {
	Timestamp    time.Time
	Properties   map[string]EntityProperty
	PartitionKey string
	RowKey       string
	ETag         string
}

EntityInfo is a read-only snapshot of an entity, returned by the StorageBackend entity accessors. Properties excludes the system properties (PartitionKey, RowKey, Timestamp), which are surfaced via their own fields.

type EntityProperty 

type EntityProperty struct {
	Value any
	Type  EdmType
}

EntityProperty is a single typed entity property value. Value's concrete Go type is determined by Type: 

EdmString    string
EdmInt32     int32
EdmInt64     int64
EdmDouble    float64
EdmBoolean   bool
EdmDateTime  time.Time
EdmGUID      string (canonical UUID string, not validated)
EdmBinary    []byte

MarshalJSON below deliberately keeps a value receiver even though UnmarshalJSON must be a pointer receiver to mutate p: EntityProperty values live inside map[string]EntityProperty (Properties), and Go map values are not addressable, so encoding/json can only discover and call a value-receiver Marshaler when encoding a map's values directly -- a pointer-receiver-only MarshalJSON would silently be skipped for every property in Properties, falling back to the wrong (default reflection) encoding with no error. This mixed-receiver shape is required, not an oversight.

func (EntityProperty) MarshalJSON 

func (p EntityProperty) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler for persistence snapshots (see persistence.go). This is independent of the OData wire encoding codec.go uses for HTTP responses. Any Type/Value mismatch (e.g. an EdmBinary property whose Value isn't []byte) is a real error, not silently dropped: a construction bug elsewhere should fail loudly here rather than writing a corrupt snapshot.

func (*EntityProperty) UnmarshalJSON 

func (p *EntityProperty) UnmarshalJSON(data []byte) error

UnmarshalJSON implements json.Unmarshaler for persistence snapshots. See MarshalJSON's doc comment for why this can't just rely on encoding/json's default `any` decoding. A malformed or type-mismatched snapshot value is a real error, never silently zeroed -- a snapshot that can't be decoded exactly must not be decoded approximately.

type InMemoryBackend 

type InMemoryBackend struct {
	// contains filtered or unexported fields
}

InMemoryBackend implements StorageBackend using an in-memory map guarded by a single RWMutex. Shaped after services/azurequeue's InMemoryBackend.

func NewInMemoryBackend 

func NewInMemoryBackend(label string, version int) *InMemoryBackend

NewInMemoryBackend creates a new empty InMemoryBackend. label identifies the caller for lock-contention metrics (lockmetrics.New) -- distinct labels per caller (e.g. "azuretable" vs. "cosmosdb") keep Azure Table and Cosmos DB's Table API locks distinguishable in metrics even though they share this one engine. version is the snapshot-version value Snapshot writes and Restore enforces (see persistence.go and the version field's own doc comment); callers own their version numbering independently.

func (*InMemoryBackend) CreateTable 

func (b *InMemoryBackend) CreateTable(name string) error

CreateTable creates a new, empty table. Returns ErrTableAlreadyExists if a table with the same name already exists -- unlike services/azurequeue's CreateQueue, Table Storage has no metadata-identical-retry idempotency exception; a duplicate Create is always a conflict.

func (*InMemoryBackend) DeleteEntity 

func (b *InMemoryBackend) DeleteEntity(table, partitionKey, rowKey, ifMatch string) error

DeleteEntity removes an entity after verifying ifMatch. Returns ErrTableNotFound, ErrEntityNotFound, or ErrETagMismatch as appropriate. Callers always pass a non-empty ifMatch ("*" or a specific ETag): an absent If-Match is rejected at the wire layer before reaching the backend.

func (*InMemoryBackend) DeleteTable 

func (b *InMemoryBackend) DeleteTable(name string) error

DeleteTable removes a table and all of its entities. Returns ErrTableNotFound if the table does not exist.

func (*InMemoryBackend) GetEntity 

func (b *InMemoryBackend) GetEntity(table, partitionKey, rowKey string) (EntityInfo, error)

GetEntity retrieves a single entity. Returns ErrTableNotFound or ErrEntityNotFound as appropriate.

func (*InMemoryBackend) InsertEntity 

func (b *InMemoryBackend) InsertEntity(
	table, partitionKey, rowKey string, props map[string]EntityProperty,
) (EntityInfo, error)

InsertEntity creates a new entity in table. Returns ErrTableNotFound if the table does not exist, or ErrEntityAlreadyExists if an entity with the same PartitionKey/RowKey already exists.

func (*InMemoryBackend) ListTables 

func (b *InMemoryBackend) ListTables() []TableInfo

ListTables returns a snapshot of all tables, sorted by name (the order Azure's List Tables returns them in).

func (*InMemoryBackend) MergeEntity 

func (b *InMemoryBackend) MergeEntity(
	table, partitionKey, rowKey string, props map[string]EntityProperty, ifMatch string,
) (EntityInfo, error)

MergeEntity merges props into an existing entity's properties (properties not present in props are left unaffected), or (ifMatch == "") inserts a new entity if absent (Insert-Or-Merge / upsert). See StorageBackend's doc comment for ifMatch's three states.

func (*InMemoryBackend) QueryEntities 

func (b *InMemoryBackend) QueryEntities(table string, filter Node, top int) ([]EntityInfo, error)

QueryEntities returns entities in table matching filter (nil matches all), ordered by (PartitionKey, RowKey), capped at top results (top <= 0 means unlimited). Returns ErrTableNotFound if the table does not exist.

func (*InMemoryBackend) ReplaceEntity 

func (b *InMemoryBackend) ReplaceEntity(
	table, partitionKey, rowKey string, props map[string]EntityProperty, ifMatch string,
) (EntityInfo, error)

ReplaceEntity fully replaces an existing entity's properties, or (ifMatch == "") inserts a new one if absent (Insert-Or-Replace / upsert). See StorageBackend's doc comment for ifMatch's three states.

func (*InMemoryBackend) Reset 

func (b *InMemoryBackend) Reset()

Reset clears all in-memory state.

func (*InMemoryBackend) Restore 

func (b *InMemoryBackend) Restore(ctx context.Context, data []byte) error

Restore loads backend state from a JSON snapshot. It implements persistence.Persistable.

func (*InMemoryBackend) Snapshot 

func (b *InMemoryBackend) Snapshot(ctx context.Context) []byte

Snapshot serialises the backend state to JSON. It implements persistence.Persistable, so an InMemoryBackend can be registered directly as a snapshot participant, or delegated to by a caller's own Handler-level Snapshot method (see services/azuretable/persistence.go and services/cosmosdb/persistence.go for that delegation pattern).

type Node 

type Node interface {
	// contains filtered or unexported methods
}

Node is a node in a parsed $filter expression tree.

func ParseFilter 

func ParseFilter(s string) (Node, error)

ParseFilter parses a complete $filter expression string into a Node tree. Returns ErrFilterParse (wrapped with detail) on any malformed input, ErrFilterTooDeep if nesting exceeds maxFilterDepth, and never panics.

type Parser 

type Parser struct {
	// contains filtered or unexported fields
}

Parser is a recursive-descent parser for $filter expressions.

func NewParser 

func NewParser(s string) *Parser

NewParser creates a Parser over a $filter expression string.

type StorageBackend 

type StorageBackend interface {
	CreateTable(name string) error
	DeleteTable(name string) error
	ListTables() []TableInfo

	InsertEntity(table, partitionKey, rowKey string, props map[string]EntityProperty) (EntityInfo, error)
	GetEntity(table, partitionKey, rowKey string) (EntityInfo, error)
	// QueryEntities returns entities in table matching filter (nil matches
	// everything), ordered by (PartitionKey, RowKey), capped at top results
	// (top <= 0 means unlimited).
	QueryEntities(table string, filter Node, top int) ([]EntityInfo, error)
	ReplaceEntity(
		table, partitionKey, rowKey string,
		props map[string]EntityProperty,
		ifMatch string,
	) (EntityInfo, error)
	MergeEntity(table, partitionKey, rowKey string, props map[string]EntityProperty, ifMatch string) (EntityInfo, error)
	DeleteEntity(table, partitionKey, rowKey, ifMatch string) error

	// Reset clears all in-memory state. Used by callers' own
	// POST /_gopherstack/reset endpoint for CI pipelines and rapid local
	// development.
	Reset()
}

StorageBackend defines the interface for an OData Table entity backend (Azure Table Storage or Azure Cosmos DB's Table API). A narrow, testable seam between a wire handler and storage, so handler tests can substitute a fake.

The ifMatch parameter on ReplaceEntity/MergeEntity/DeleteEntity threads through the three If-Match states the wire protocol distinguishes:

  • "" (no If-Match header): upsert semantics for Replace/MergeEntity (create if absent, otherwise mutate unconditionally); DeleteEntity's caller never passes "" -- an absent If-Match on Delete is rejected at the handler layer before the backend is ever called.
  • IfMatchAny ("*"): the entity must exist, but any current ETag matches.
  • any other string: the entity must exist AND its current ETag must equal this value, else ErrETagMismatch.

type StoredTable 

type StoredTable = storedTable

StoredTable is an exported alias for storedTable, existing solely so a caller package (services/azuretable, and services/cosmosdb should its Table API ever be wired into persistence) can reference this type's name in its own guard-visible *Snapshot-suffixed struct -- see services/azuretable/persistence.go's azureTableSnapshot and pkgs/persistence/snapshotversion_guard_test.go's doc comment for why that struct needs to exist at all. It is never constructed directly outside this package.

type TableInfo 

type TableInfo struct {
	Name string
}

TableInfo is a read-only snapshot of a table's metadata, returned by StorageBackend.ListTables.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.