Documentation
¶
Overview ¶
Package memory provides an in-memory implementation of stats.Collector. All primitives are goroutine-safe and live in process memory. Use WithPersistence to periodically snapshot to a file.
Index ¶
- func ArchiveAdapter(store stats.ArchiveStore) func(key string, entry SnapshotEntry) error
- func DefaultKeyDateExtractor(key string) (string, bool)
- type Collector
- func (c *Collector) CleanupNow() int
- func (c *Collector) Close() error
- func (c *Collector) Counter(key string) stats.Counter
- func (c *Collector) Flush() error
- func (c *Collector) Gauge(key string) stats.Gauge
- func (c *Collector) HLL(key string) stats.HLL
- func (c *Collector) KeyCount() int
- func (c *Collector) Restore(snap *Snapshot)
- func (c *Collector) Set(key string) stats.Set
- func (c *Collector) Snapshot() *Snapshot
- func (c *Collector) Timer(key string) stats.Timer
- type Option
- type PersistFunc
- type Snapshot
- type SnapshotEntry
- type TTLConfig
- type TimerSnapshot
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ArchiveAdapter ¶ added in v0.2.1
func ArchiveAdapter(store stats.ArchiveStore) func(key string, entry SnapshotEntry) error
ArchiveAdapter wraps a stats.ArchiveStore and provides an OnExpire callback function compatible with TTLConfig.OnExpire.
This bridges the in-memory TTL expiration to any ArchiveStore implementation (SQLite, MySQL, PostgreSQL, etc.).
Usage:
store, _ := sqlite.New("data/stats.db") // or mysql.New(dsn)
c := memory.New(
memory.WithTTL(memory.TTLConfig{
RetentionDays: 7,
OnExpire: memory.ArchiveAdapter(store),
}),
)
func DefaultKeyDateExtractor ¶ added in v0.2.0
DefaultKeyDateExtractor extracts a "YYYY-MM-DD" date from a key. It searches for a 10-character substring matching the date pattern. Returns ok=false if no date is found.
Types ¶
type Collector ¶
type Collector struct {
// contains filtered or unexported fields
}
Collector implements stats.Collector with in-memory primitives.
func (*Collector) CleanupNow ¶ added in v0.2.0
CleanupNow triggers an immediate TTL cleanup cycle (blocking). Returns the number of keys removed.
func (*Collector) KeyCount ¶ added in v0.2.0
KeyCount returns the total number of keys across all primitive types. Useful for monitoring memory growth.
type Option ¶
type Option func(*Collector)
Option configures the in-memory collector.
func WithBloomSet ¶ added in v0.1.2
WithBloomSet configures all Set() calls to return Bloom filter sets instead of exact map-based sets. This dramatically reduces memory for large-scale deduplication (e.g. 1M users → 1.4 MB vs 80 MB).
Trade-off: Count() is approximate, Members() returns nil, Intersect() returns 0. Has() has no false negatives but may have false positives (~falsePositiveRate).
func WithPersistence ¶
func WithPersistence(fn PersistFunc) Option
WithPersistence sets a persistence function called on Flush().
func WithReservoirTimer ¶ added in v0.1.2
WithReservoirTimer sets a fixed capacity for all Timers, using reservoir sampling instead of storing all samples. This bounds memory usage.
Default (0): store all samples (unbounded). Recommended: 4096 (32 KB per timer, P95 error < 1%).
func WithTTL ¶ added in v0.2.0
WithTTL enables automatic TTL-based cleanup of old keys.
Keys containing a date prefix (e.g. "pv:2026-08-18:/home") older than RetentionDays are automatically expired. Before removal, OnExpire is called so the caller can persist data to a database.
Example:
c := memory.New(
memory.WithReservoirTimer(4096),
memory.WithTTL(memory.TTLConfig{
RetentionDays: 7,
CheckInterval: time.Hour,
OnExpire: func(key string, entry memory.SnapshotEntry) error {
return dbStore.Save(key, entry)
},
}),
)
type PersistFunc ¶
PersistFunc is called during Flush to serialize state.
type Snapshot ¶
type Snapshot struct {
Counters map[string]int64 `json:"counters"`
Gauges map[string]int64 `json:"gauges"`
Sets map[string][]string `json:"sets"`
HLLs map[string][]byte `json:"hlls"`
Timers map[string][]int64 `json:"timers"`
}
Snapshot is a serializable representation of the in-memory collector's state. It can be used with gob/json for file persistence.
type SnapshotEntry ¶ added in v0.2.0
type SnapshotEntry struct {
Type string // "counter", "gauge", "set", "hll", "timer"
Value any // type-specific value
}
SnapshotEntry represents the value of a single key at expiration time.
type TTLConfig ¶ added in v0.2.0
type TTLConfig struct {
// RetentionDays is the number of days of data to keep in memory.
// Keys with dates older than this are expired.
// Default: 7
RetentionDays int
// CheckInterval is how often the cleanup goroutine runs.
// Default: 1 hour
CheckInterval time.Duration
// OnExpire is called for each expired key before removal.
// It receives the key and a SnapshotEntry containing all primitive values.
// If OnExpire returns an error, the key is NOT removed (will retry next cycle).
// If OnExpire is nil, keys are removed without callback.
OnExpire func(key string, entry SnapshotEntry) error
// KeyDateExtractor extracts a date from a key string.
// If it returns ok=false, the key is never expired.
// Default: extracts "YYYY-MM-DD" from any position in the key.
KeyDateExtractor func(key string) (date string, ok bool)
}
TTLConfig configures time-to-live based automatic cleanup for the in-memory collector. Keys containing a date prefix (e.g. "pv:2026-08-18:...") are eligible for expiration.
When a key expires:
- The onExpire callback is invoked with the key and its current value, allowing the caller to persist the data to a database before removal.
- The key is removed from all in-memory maps.
This ensures bounded memory usage for long-running services: only the most recent `retentionDays` of data stays in memory, while older data is flushed to external storage (SQLite, MySQL, etc.).
Source Files