syncquery

package
v0.19.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 20, 2026 License: AGPL-3.0 Imports: 8 Imported by: 0

Documentation

Overview

Package syncquery builds the filters, options and cursors used by every `since` sync endpoint, so all services share one implementation.

Index

Constants

View Source
const DefaultUpdatedAtField = "updated_at"

DefaultUpdatedAtField is the BSON field every entity sorts and filters on.

Variables

View Source
var ErrInvalidCursor = errors.New("invalid cursor")

ErrInvalidCursor is returned when a cursor cannot be decoded. Callers should map it to HTTP 400 rather than ignoring it — silently dropping a bad cursor would restart pagination and re-deliver the whole collection.

View Source
var ErrZeroCursorID = errors.New("syncquery: cannot build a next cursor from a zero id")

ErrZeroCursorID is returned by SetNextCursor when the last row's id is the zero id. DecodeCursor always rejects such a cursor, so emitting one would make every later sync request answer 400.

Functions

func DualBeforeFilter

func DualBeforeFilter(base bson.M, field string, cutoff time.Time) bson.M

DualBeforeFilter builds the retention filter for a tombstone purge across a collection whose deleted_at may still hold either encoding under one name.

The purge is a range query and is therefore type-bracketed just like a read: a Date operand matches no strings. habit.deleted_at is a string until its backfill completes, so a single-typed purge would match zero habit rows and report success having deleted nothing.

$ne: nil on the string branch is redundant under type bracketing and kept deliberately: this is an unrecoverable hard delete of live rows, and a cheap guard against being wrong about a database subtlety is the fail-closed posture used everywhere else. Its behaviour is proven against a real database in the productivity plan, not by analysis here.

The Date branch needs no equivalent guard: every Date-encoded tombstone in these collections is a non-pointer primitive.DateTime with omitempty, so it can never hold an explicit BSON null — only a string-encoded field like habit.deleted_at can.

func DualSinceFilter

func DualSinceFilter(base bson.M, field string, since time.Time) bson.M

DualSinceFilter builds a read filter for a collection mid-migration from a string updated_at to a BSON Date, where both encodings live under the SAME field name.

MongoDB range operators are type-bracketed, so one query cannot span both: a Date operand matches no Strings and a String operand matches no Dates. Each branch is therefore queried in its own type and the results unioned. No $exists guard is needed — bracketing alone separates them.

Offset mode only. A half-migrated collection has no chronological total order — BSON sorts every String before every Date — so keyset pagination must stay off until the backfill is verified complete.

func EncodeCursor

func EncodeCursor(updatedAt time.Time, cursorID id.ID) string

EncodeCursor renders an opaque, URL-safe cursor token.

func FindOptions

func FindOptions(field string, page, limit *int64, c *Cursor) *options.FindOptions

FindOptions builds the find options for a `since` read.

The sort is always ascending (field, _id). Ascending is what a sync feed needs: a partially consumed response still yields a safe cursor at the last row received. The _id component makes the order total, which is what lets a keyset cursor resume exactly.

skip is applied only in offset mode. In keyset mode the position is already encoded in the filter, and skipping on top of it would advance twice.

func FormatServerTime

func FormatServerTime(t time.Time) string

FormatServerTime renders a server time for the wire: RFC3339, always UTC.

func SampleServerTime

func SampleServerTime() time.Time

SampleServerTime returns the authoritative sync cursor for a response.

Call it BEFORE the repository read, never after. Sampling after the read leaves a window in which a row committed during the read, carrying an updated_at below the sample, is never delivered. Sampling first can only cause overlap, and overlap is free: every client merge is an idempotent upsert keyed by id.

func SinceFilter

func SinceFilter(base bson.M, field string, since time.Time, c *Cursor) bson.M

SinceFilter builds the read filter for a `since` endpoint.

Without a cursor it is a plain `field > since`. With one it is the keyset condition `field > u OR (field == u AND _id > i)`. The _id tiebreak is mandatory: without it, more rows sharing a single timestamp than fit in one page make the cursor stop advancing — an infinite loop, or a skip.

base and the condition are combined with $and rather than merged, because base may already contain an $or and duplicate top-level keys would clobber one another.

since is ignored when c is non-nil: the cursor already encodes a position at or after it.

Types

type Cursor

type Cursor struct {
	UpdatedAt time.Time
	ID        id.ID
}

Cursor is a keyset position in an ascending (updated_at, id) ordering.

func DecodeCursor

func DecodeCursor(raw string) (*Cursor, error)

DecodeCursor parses a token produced by EncodeCursor. An empty string means "no cursor" and yields (nil, nil).

The id is accepted in EITHER native form (see id.Parse); Cursor.ID.Kind() reports which one it was. A consumer whose record ids are of a different kind than the cursor's must not compare ids across kinds — see Cursor.After.

A cursor that fails to decode — corrupt base64/JSON, an unparseable timestamp, or an id that is neither a valid ObjectID nor a valid UUID — always returns ErrInvalidCursor. It is never silently treated as absent: callers map this to HTTP 400 (see ErrInvalidCursor's own doc comment).

func (*Cursor) After

func (c *Cursor) After(updatedAt time.Time, recordID id.ID) bool

After reports whether a record at (updatedAt, recordID) lies strictly after this cursor's keyset position — i.e. belongs on the next page — and is the reference behaviour every backend's query builder must reproduce.

A nil cursor matches everything (there is no position to be after).

When the cursor's id and the record's id are THE SAME KIND, the full (updated_at, id) tie-break applies: a record matches if its updated_at is strictly later, or equal with a later id (id.Compare, which agrees with the ordering of the canonical strings). That comparison is safe only between like kinds, where the canonical forms are fixed-width and position-aligned (24 hex chars, or 36-char hyphenated UUID text).

When the kinds DIFFER, the id tie-break is dropped and updated_at is compared ALONE, with >= rather than >: every record sharing the cursor's exact timestamp is re-delivered instead of resuming mid-tie.

The mismatch runs in both directions. The expected one is an ObjectID cursor meeting uuid-keyed records after the cutover. The reverse appears the moment a cutover is REVERTED, which the plan requires to be possible: clients are then holding uuid cursors while Mongo is authoritative again, and comparing a 36-char uuid against a 24-char hex string yields an ordering that means nothing and can skip records just as badly.

Re-delivery is the safe direction to fail in, because every client here merges by upsert; a skipped record goes silently missing on that device until something touches it again. A tie-break cannot simply be re-derived across the id boundary: mongo2pg derives a row's uuid from a namespace that depends on the deployment's service name (see the id package's doc), so a derived uuid's ordering does not agree with the ObjectID's, and comparing across that boundary can skip a record.

type Envelope

type Envelope struct {
	// ServerTime is the authoritative cursor for the next sync, sampled before
	// the read. RFC3339, always UTC.
	ServerTime string `json:"serverTime"`
	// NextCursor resumes keyset pagination. Empty means the feed is drained.
	NextCursor string `json:"nextCursor,omitempty"`
}

Envelope holds the sync fields every `since` response carries. Embed it in a service's existing paginated response struct so total_count / page / size / total_pages keep their current shape and released clients see no change.

func NewEnvelope

func NewEnvelope(serverTime time.Time) Envelope

NewEnvelope builds an Envelope from a time sampled by SampleServerTime.

func (*Envelope) SetNextCursor

func (e *Envelope) SetNextCursor(returned int, limit *int64, lastUpdatedAt time.Time, lastID id.ID) error

SetNextCursor sets the keyset cursor from the last row of a full page.

A page shorter than the limit means the feed is drained, so no cursor is emitted — otherwise clients would loop forever on an empty tail. An unpaginated read (nil limit) returned everything and has no next page.

A full page whose last row has a zero id (a record that was never given one — a data/programming error) cannot be resumed from: the returned error is ErrZeroCursorID and NextCursor stays empty. The caller MUST treat it as an internal error (HTTP 500) rather than serve the page, because an empty NextCursor would tell the client the feed is drained and silently strand the rest of it. (Call sites that ignore the result still compile.)

type Query

type Query struct {
	Filter  bson.M
	Options *options.FindOptions
}

Query is the filter and find options for one `since` read, built together.

func NewQuery

func NewQuery(base bson.M, field string, since time.Time, c *Cursor, page, limit *int64) Query

NewQuery builds the filter and the find options for a `since` read from one set of inputs.

Prefer it over calling SinceFilter and FindOptions separately. Those take field and c independently, so passing a cursor to one but not the other silently pairs a keyset filter with an offset skip: the position advances twice and rows in between are never delivered. Building both from the same inputs makes that mismatch impossible to express.

The dual-field builders (DualSinceFilter, DualBeforeFilter) stay separate: collections mid-migration cannot use keyset pagination at all, so they have no cursor to keep in step.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL