advanced-cache

Advanced Cache (advCache)

In-memory HTTP cache & reverse proxy written in Go. Focused on low latency and sustained load: sharded storage, LRU with TinyLFU admission, background refresh, upstream management, and a lightweight worker orchestrator.

Highlights

• Sharded storage with per-shard LRU and a global shard balancer for proportional eviction.
• Admission: W-TinyLFU + Doorkeeper (Count-Min + gated bitset) to protect the hot set.
• Background refresh (TTL, β-staggering, scan-rate, upstream rate limiting).
• Upstream cluster: per-backend rate limiting, health probing, slow-start, quarantine.
• Memory discipline: buffer pools, zero-copy headers, predictable budget.
• Metrics: Prometheus / VictoriaMetrics exposition.
• fasthttp HTTP layer; simple control endpoints; K8s probes.

Repository map

• cmd/ – main entrypoint, flags, wiring, probes.
• internal/cache/api/ – HTTP handlers: main route, on/off, clear, metrics.
• pkg/config/ – YAML config loader and derived settings.
• pkg/storage/{map,lru,lfu}/ – sharded map, LRU backend (“LRU mode”), LFU/TinyLFU admission.
• pkg/upstream/ – backend & cluster (rate-limit, health, proxy).
• pkg/orchestrator/ – worker governor (evictor, refresher, GC and so on).
• pkg/http/server/ – fasthttp server and middlewares.
• pkg/prometheus/metrics/, pkg/pools/, pkg/k8s/, pkg/common/ – metrics, pooling, probes, utils.

Cache key (canonicalization)

• Whitelist: only query params and request headers listed in rules.*.cache_key.{query,headers} participate; everything else is ignored.
• Deterministic sort: selected pairs are sorted (by name, then value) before hashing to keep keys stable.
• Variants: if Accept-Encoding is whitelisted, its normalized value is part of the key to separate gzip/brotli/plain.

Response headers

• Whitelist-forwarding: only headers in rules.*.cache_value.headers are stored and returned; order is preserved (no re-sorting).
• Server-added: Server: <service-name> is always set; the original upstream name (if present) is kept as X-Origin-Server.

Configuration

Example profiles: advcache.cfg.yaml (deployment). A local profile can be supplied alongside (looked up as advcache.cfg.local.yaml if present).

Top-level keys (under cache:):

• runtime.gomaxprocs, api.{name,port}, storage.size
• admission.{table_len_per_shard,estimated_length,door_bits_per_counter,sample_multiplier}
• eviction.{enabled,replicas,scan_rate,soft_limit,hard_limit}
• refresh.{enabled,ttl,beta,rate,replicas,scan_rate,coefficient}
• upstream.{policy,cluster.backends[]}, data.dump.*, metrics.enabled, k8s.probe.timeout, rules.*

Small excerpt:

cache:
  api: { name: "advCache", port: "8020" }
  storage: { size: 53687091200 }        # 50 GiB
  eviction: { soft_limit: 0.9, hard_limit: 0.99, replicas: 4, scan_rate: 8 }
  refresh: { enabled: true, ttl: "3h", beta: 0.5, rate: 1250 }
  metrics: { enabled: true }

Build & Run

Requirements: Go 1.23+

# Build
go build -o advCache ./cmd/main.go

# Run (auto-detects config path)
./advCache                  # tries advcache.cfg.yaml, then advcache.cfg.local.yaml
./advCache -cfg ./advcache.cfg.yaml

# Docker (example)
docker build -t advcache .
docker run --rm -p 8020:8020 -v "$PWD/public/dump:/app/public/dump" advcache -cfg /app/advcache.cfg.yaml

HTTP endpoints

• GET /{any} – main cache endpoint.
• GET /cache/bypass – current state of bypass.
• GET /cache/bypass/on – enable bypass -> disable cache.
• GET /cache/bypass/off – disable bypass -> enable cache.
• GET /cache/clear – two-step clear (issue token, then ?token= to confirm).
• GET /cache/invalidate – removes or marks cache entries for update by query/path.
• GET /cache/http/compression/on – turn on compression middleware.
• GET /cache/http/compression/off – turn off compression middleware.
• GET /cache/http/compression – state of compression middleware.
• GET /cache/upstream/policy/await – switches upstream policy to await.
• GET /cache/upstream/policy/deny – switches upstream policy to deny.
• GET /cache/upstream/policy – state of upstream policy.
• GET /metrics – Prometheus/VictoriaMetrics metrics.
• GET /cache/gc/force – Force GC walkthrough.

Tuning & Ops

• Start with soft_limit: 0.8–0.9, hard_limit: 0.9–0.99; raise eviction.replicas/scan_rate if pressure persists.
• Use upstream.policy: "deny" for fail-fast load tests; "await" for back-pressure in prod.
• Keep gomaxprocs: 0 (auto) unless you need a fixed CPU cap.
• Whitelist only the query/headers that must affect the key; include Accept-Encoding if storing compressed variants.

License

Apache-2.0 — see LICENSE.

Maintainer

Borislav Glazunov — glazunov2142@gmail.com · Telegram @gl_c137