Documentation
¶
Overview ¶
Package memory provides an in-memory implementation of stats.Collector. All primitives are goroutine-safe and live in process memory. Use WithPersistence to periodically snapshot to a file.
Index ¶
- func DefaultKeyDateExtractor(key string) (string, bool)
- type Collector
- func (c *Collector) CleanupNow() int
- func (c *Collector) Close() error
- func (c *Collector) Counter(key string) stats.Counter
- func (c *Collector) Flush() error
- func (c *Collector) Gauge(key string) stats.Gauge
- func (c *Collector) HLL(key string) stats.HLL
- func (c *Collector) KeyCount() int
- func (c *Collector) Restore(snap *Snapshot)
- func (c *Collector) Set(key string) stats.Set
- func (c *Collector) Snapshot() *Snapshot
- func (c *Collector) Timer(key string) stats.Timer
- type Option
- type PersistFunc
- type Snapshot
- type TTLConfig
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DefaultKeyDateExtractor ¶ added in v0.2.0
DefaultKeyDateExtractor extracts a "YYYY-MM-DD" date from a key. It searches for a 10-character substring matching the date pattern. Returns ok=false if no date is found.
Types ¶
type Collector ¶
type Collector struct {
// contains filtered or unexported fields
}
Collector implements stats.Collector with in-memory primitives.
func (*Collector) CleanupNow ¶ added in v0.2.0
CleanupNow triggers an immediate TTL cleanup cycle (blocking). Returns the number of keys removed.
func (*Collector) KeyCount ¶ added in v0.2.0
KeyCount returns the total number of keys across all primitive types. Useful for monitoring memory growth.
type Option ¶
type Option func(*Collector)
Option configures the in-memory collector.
func WithBloomSet ¶ added in v0.1.2
WithBloomSet configures all Set() calls to return Bloom filter sets instead of exact map-based sets. This dramatically reduces memory for large-scale deduplication (e.g. 1M users → 1.4 MB vs 80 MB).
Trade-off: Count() is approximate, Members() returns nil, Intersect() returns 0. Has() has no false negatives but may have false positives (~falsePositiveRate).
func WithPersistence ¶
func WithPersistence(fn PersistFunc) Option
WithPersistence sets a persistence function called on Flush().
func WithReservoirTimer ¶ added in v0.1.2
WithReservoirTimer sets a fixed capacity for all Timers, using reservoir sampling instead of storing all samples. This bounds memory usage.
Default (0): store all samples (unbounded). Recommended: 4096 (32 KB per timer, P95 error < 1%).
func WithTTL ¶ added in v0.2.0
WithTTL enables automatic TTL-based cleanup of old keys.
Keys containing a date prefix (e.g. "pv:2026-08-18:/home") older than RetentionDays are automatically expired. Before removal, OnExpire is called so the caller can persist data anywhere.
OnExpire is a pure callback — no implementation class required:
c := memory.New(
memory.WithTTL(memory.TTLConfig{
RetentionDays: 7,
OnExpire: func(ek stats.ExpiredKey) error {
// Write to any database, file, Kafka, HTTP API...
_, err := db.Exec("INSERT INTO archive ...", ek.Key, ek.Value)
return err
},
}),
)
type PersistFunc ¶
PersistFunc is called during Flush to serialize state.
type Snapshot ¶
type Snapshot struct {
Counters map[string]int64 `json:"counters"`
Gauges map[string]int64 `json:"gauges"`
Sets map[string][]string `json:"sets"`
HLLs map[string][]byte `json:"hlls"`
Timers map[string][]int64 `json:"timers"`
}
Snapshot is a serializable representation of the in-memory collector's state. It can be used with gob/json for file persistence.
type TTLConfig ¶ added in v0.2.0
type TTLConfig struct {
// RetentionDays is the number of days of data to keep in memory.
// Keys with dates older than this are expired.
// Default: 7
RetentionDays int
// CheckInterval is how often the cleanup goroutine runs.
// Default: 1 hour
CheckInterval time.Duration
// OnExpire is called for each expired key before removal.
// If it returns an error, the key is NOT removed (will retry next cycle).
// If nil, keys are removed silently.
OnExpire stats.ExpireFunc
// KeyDateExtractor extracts a date from a key string.
// If it returns ok=false, the key is never expired.
// Default: extracts "YYYY-MM-DD" from any position in the key.
KeyDateExtractor func(key string) (date string, ok bool)
}
TTLConfig configures time-to-live based automatic cleanup for the in-memory collector. Keys containing a date prefix (e.g. "pv:2026-08-18:...") are eligible for expiration.
When a key expires:
- The OnExpire callback is invoked with the key and its final value.
- The key is removed from all in-memory maps.
This ensures bounded memory usage for long-running services: only the most recent `retentionDays` of data stays in memory, while older data is flushed to external storage via the callback.
OnExpire is a pure callback — implement it however you like:
memory.WithTTL(memory.TTLConfig{
RetentionDays: 7,
OnExpire: func(ek stats.ExpiredKey) error {
// Write to SQLite, MySQL, Postgres, Kafka, file, HTTP API...
return db.Save(ek)
},
})
Source Files