timeseries

timeseries

Package timeseries stores and reads time-bucketed aggregations for event streams. It supports counts, sums, averages, sample standard deviation, min/max, HLL distinct counts, TDigest percentiles, geometric mean, circular mean, and z-score dependencies through queries.Field.

A *client.Client can be passed directly to Execute, Read, and Delete. Tests can use smaller mocks that implement the operation methods required by the path being tested.

Modes

Request.Range chooses the storage mode:

• Range > 0: fixed time-window mode using deterministic multi-granularity cells in set timeseries_set plus metadata in set timeseries_meta.
• Range == 0: custom all-time mode using one record in timeseries_set.

Request.TTL is required in both modes. Fixed-mode cell TTLs are derived from Range; custom-mode writes use Request.TTL. The maximum range is MaxRange, which is two years.

Strengths

• Execute, Read, and Delete accept small client interfaces, so production callers can pass *client.Client and tests can mock only the operations they exercise.
• Fixed-mode keys are deterministic: a SHA-1 primary key is derived from Name and ordered GroupBy values, and cell keys add hierarchical cell IDs. Delete can reconstruct those keys without a separate key-tracking record.
• Range > 0 mode bounds storage by writing only the active cells for the selected multi-granularity plan, while reads drill into boundary cells instead of scanning every possible fine-grained cell.
• Range == 0 custom mode gives a simple all-time aggregate in one timeseries_set record when callers do not need a wall-clock window.
• The package supports the aggregation fields implemented by queries.Field, including approximate HLL distinct counts and TDigest percentiles.

Weaknesses / Tradeoffs

• TTL must be positive for every request. In fixed mode, cell TTL is derived from Range and metadata TTL is Range * 2; Request.TTL is not the fixed cell-retention knob. In custom mode, writes use Request.TTL.
• Fixed ranges larger than MaxRange are rejected. Larger valid ranges also use coarser granularity levels, which reduces cell volume but gives less fine-grained storage resolution.
• Total storage scales with the number of distinct group scopes in use. Per request, operation count scales with requested fields and cells touched by the selected range. FieldDistinct and FieldPercentile add server sketch/digest storage per active cell.
• Fixed reads and writes use metadata lookups plus batch operations. Min/max fields and stale-cell cleanup can add extra reads or deletes, and deleting a large fixed range enumerates all possible hierarchical cell keys.
• Missing GroupBy values become empty strings with MapSource; missing or non-numeric Value fields become 0. Missing event_time uses time.Now(), so callers that need deterministic event-time bucketing should provide it explicitly.

Request

Important request fields:

• Name: variable name used in key derivation.
• Namespace: FrogoDB namespace.
• GroupBy: source keys whose string values scope the aggregation.
• Range: time window, or zero for all-time custom mode.
• Fields: aggregation flags from package queries.
• Value: source key for the numeric value.
• DistinctBy: source key used for FieldDistinct.
• PercentileP: TDigest quantile for FieldPercentile, expressed from 0 to 1.
• TTL: required positive expiration duration.
• IncludeCurrent: when true, write before reading; when false, read before writing.

GroupBy order is significant for timeseries keys.

Example

func updateStats(ctx context.Context) error {
    c, err := client.New("127.0.0.1:3000")
    if err != nil {
        return err
    }
    defer c.Close()

    src := queries.NewMapSource(map[string]any{
        "event_id":         "evt-123",
        "standard.user_id": "user-42",
        "amount":           42.5,
        "event_time":       time.Now(),
    })

    req := timeseries.Request{
        Name:           "user_amounts_24h",
        Namespace:      "scoring",
        GroupBy:        []string{"standard.user_id"},
        Range:          24 * time.Hour,
        Fields:         queries.FieldCount | queries.FieldAvg | queries.FieldSTD | queries.FieldMin | queries.FieldMax,
        Value:          "amount",
        TTL:            48 * time.Hour,
        IncludeCurrent: true,
    }

    result, err := timeseries.Execute(ctx, c, req, src)
    if err != nil {
        return err
    }

    _ = result.Count
    _ = result.Average()
    _ = result.STD()
    return nil
}

Use Read to query without writing the current event:

result, err := timeseries.Read(ctx, c, req, src)

Use Delete to remove stored data for the same request and source scope:

err := timeseries.Delete(ctx, c, req, src)

Results

Result exposes raw aggregate fields and helper methods:

• Count, Sum, Min, Max, DistinctCount, and Percentile.
• Average() returns Sum / Count, or zero when count is zero.
• STD() returns sample standard deviation, or zero with fewer than two samples.
• GeometricMean() uses SumLog.
• CircularMean() uses SumSin and SumCos.
• ZScore(currentValue) uses the result average and standard deviation.

For FieldDistinct, set DistinctBy; otherwise no HLL element is written. For FieldPercentile, set PercentileP, for example 0.95.

Event time

Writes and range reads use queries.EventTime(src). Provide event_time as a time.Time, int64 Unix seconds, or int Unix seconds when events should be bucketed by event time. If omitted, the helper uses time.Now().