Documentation
¶
Overview ¶
Package bus implements the daemon side of the dws event subsystem: one long-lived process per ClientID that holds the single cloud connection and fans out events to N local consumers over IPC.
Files (mirroring plan §5 layout):
daemon.go main loop: lock → meta → IPC listen → ready → source.Start hub.go consumer registry, per-consumer sendCh, drop-oldest backpressure metrics.go per-event-type + per-consumer received/dropped counters lockfile.go single bus.lock (flock + PID content + stale recovery) meta.go bus.meta JSON (clientID/edition/started_at) for list/status reverse mapping
Lifecycle invariants (plan §4 invariants 1–7):
- emit non-blocking (drop-oldest, never block SDK callback)
- dedup on event_id (LRU) to absorb cloud-side redelivery
- single bus per ClientID (bus.lock enforces, all FS paths use clientIDHash)
- upstream always full subscription; consumer filter only affects bus→consume
- dead-consumer auto-reap on socket EOF
- startup order: lock → meta → IPC listen → ready pipe → Source.Start
- stdio detach when fork'd by busctl/spawn
Index ¶
- Constants
- Variables
- func ApplyEnvTuning(cfg *Config)
- func ReadHolderPID(path string) int
- func Run(ctx context.Context, cfg Config) error
- func WriteMeta(dir string, m Meta) error
- type Config
- type Consumer
- type Hub
- func (h *Hub) Broadcast(frame any) int
- func (h *Hub) Counters() *PerTypeCounters
- func (h *Hub) Deliver(raw *dwsevent.RawEvent)
- func (h *Hub) Len() int
- func (h *Hub) Register(hello transport.Hello) (*Consumer, error)
- func (h *Hub) Snapshot() []transport.StatusConsumer
- func (h *Hub) Unregister(id int)
- type Lock
- type Meta
- type PerTypeCounters
- type RegisterError
- type SourceAdapter
Constants ¶
const ( EnvIdleTimeout = "DWS_EVENT_BUS_IDLE_TIMEOUT" // Go duration, e.g. "10m" EnvConsumerBuffer = "DWS_EVENT_CONSUMER_BUFFER" // integer EnvDedupLRU = "DWS_EVENT_DEDUP_LRU" // integer EnvDropWarnPct = "DWS_EVENT_DROP_WARN_PCT" // integer 1-100 )
Tunable env vars (plan §15 已决项 — surfaced for operators without requiring a new CLI flag for each knob). Each lookup is read-once at bus startup; runtime changes require a bus restart.
const CurrentBusVersion = "v1"
CurrentBusVersion identifies the bus wire/storage compatibility level. Bumped only on breaking changes (IPC protocol, lockfile shape, meta schema). v1 is the initial value; the field is parsed defensively by readers (older readers tolerate unknown fields via encoding/json).
const DefaultDropWarnPercent = 5
DefaultDropWarnPercent is the threshold above which per-event-type drop rate triggers a slog WARN line in bus.log. 5% is the plan default (§15 已决项); overridable via DWS_EVENT_DROP_WARN_PCT.
We use whole-percentage granularity (int) because the counter math is integer; sub-percent precision would just add noise.
const DefaultSendBuffer = 100
DefaultSendBuffer is the per-consumer channel capacity used when the Hub is constructed via NewHub. Sized to absorb a short event burst (~100ms at ~1k evt/s) without backpressure. Overridable via DWS_EVENT_CONSUMER_BUFFER at daemon start (plan §15 已决但暴露方式).
const LockFileName = "bus.lock"
LockFileName is the on-disk name of the bus single-instance lock. It lives inside the bus working directory (<ConfigDir>/events/<edition>/<source_kind>/<identity_hash>/).
const MetaFileName = "bus.meta"
MetaFileName is the on-disk name of the bus metadata file. It lives alongside bus.lock and bus.sock inside the bus working directory.
Variables ¶
var ErrBusy = lock.ErrBusy
ErrBusy is re-exported from lock for callers that only depend on bus.
var ErrStaleOwnerAlive = errors.New("bus: lock file PID is alive but flock was released; assuming live owner")
ErrStaleOwnerAlive indicates the PID stored in bus.lock points at a live process — there is already a bus running for this ClientID and we must not start another one. (This case is hit when the holder is still alive but its flock was somehow released; in practice flock + PID always agree, so this is mostly defensive.)
Functions ¶
func ApplyEnvTuning ¶
func ApplyEnvTuning(cfg *Config)
ApplyEnvTuning fills in Config defaults from the env vars listed above for any fields the caller left at zero. The cobra layer calls this after constructing Config so explicit flag values still win.
Defaults (when env is absent or invalid):
IdleTimeout → 5m ConsumerBuffer → DefaultSendBuffer DedupCapacity → 0 (let dedup package's DefaultCapacity apply) DropWarnPercent → DefaultDropWarnPercent
Invalid env values (non-parseable, out of range) are silently ignored and the default is used. We deliberately do NOT fail the bus on bad env input — operators shouldn't lose a daemon over a typo'd env var.
func ReadHolderPID ¶
ReadHolderPID returns the PID stored in path, or 0 if the file is missing or unreadable. Does NOT attempt to acquire the lock — useful for `event status` to display the holder without contention.
func Run ¶
Run starts the bus daemon. Lifecycle (plan §4 invariant #6):
- Acquire bus.lock (single-instance enforcement)
- Write bus.meta
- Listen IPC (so consumers can connect before SDK starts pushing)
- Signal readiness via ReadyPipe
- Start the Source (cloud SDK); concurrent with consumer accept loop
- Wait on ctx for shutdown signal
- Graceful: broadcast Bye → close listener → close source → release lock
Run blocks until ctx is cancelled, the Source returns an error, or a fatal startup error occurs.
Types ¶
type Config ¶
type Config struct {
// WorkDir is the bus working directory:
// <ConfigDir>/events/<edition>/<source_kind>/<identity_hash>/
// The caller MUST mkdir this with pkg/config.DirPerm before calling Run.
WorkDir string
// IPCEndpoint is the Unix socket path or Windows pipe name. Caller
// computes this from WorkDir (Unix) or edition/clientIDHash (Windows).
IPCEndpoint string
// ClientID is the human-readable identifier written into bus.meta and
// status output. NOT used in any path.
ClientID string
// SourceKind/IdentityHash/SourceID are diagnostic identity fields used by
// list/status. Empty SourceKind is interpreted as app_stream for backward
// compatibility.
SourceKind dwsevent.SourceKind
IdentityHash string
SourceID string
// Edition is written into bus.meta. Comes from edition.Get().Name with
// "open" fallback applied by the caller.
Edition string
// SDKVersion is recorded in bus.meta for diagnostics.
SDKVersion string
// Source is the cloud adapter. Required.
Source SourceAdapter
// IdleTimeout: bus self-exits after this long with zero consumers.
// Zero disables (bus runs until SIGTERM).
IdleTimeout time.Duration
// ConsumerBuffer overrides per-consumer sendCh capacity. Zero uses
// DefaultSendBuffer.
ConsumerBuffer int
// DedupCapacity overrides event_id LRU size. Zero uses dedup.DefaultCapacity.
DedupCapacity int
// DropWarnPercent is the per-event-type drop-rate threshold (whole
// percentage points) that triggers a slog WARN in bus.log. Zero or
// out-of-range values fall back to DefaultDropWarnPercent. Overridable
// via env DWS_EVENT_DROP_WARN_PCT (read by the cobra layer).
DropWarnPercent int
// ReadyPipe receives a single byte ('R' on success, 'E' on failure)
// once the bus has either come up or failed startup, so the parent
// process forked by busctl/spawn can stop polling and either dial or
// surface the error. nil disables (foreground mode).
ReadyPipe *os.File
// Logger sink. Nil → slog.Default.
Logger *slog.Logger
}
Config bundles everything Run needs to start a daemon. All paths and identifiers come from busctl/Source so the bus itself stays oblivious to ConfigDir / edition rules.
type Consumer ¶
type Consumer struct {
ID int // monotonic, assigned by Hub
PID int // from Hello.ConsumerPID
EventTypes []string // raw wildcard patterns from Hello
Filter string // raw regex from Hello (for status display)
SubscribeID string // optional personal subscription label and local isolation key
SubscribedAt time.Time
SendCh chan any // bus → consume frames (Event/SourceState/Heartbeat/Bye)
// contains filtered or unexported fields
}
Consumer represents one registered IPC connection to the bus. Wire-side reader/writer goroutines are owned by daemon.go; the Hub holds the metadata + sendCh.
type Hub ¶
type Hub struct {
// contains filtered or unexported fields
}
Hub is the bus's fan-out engine. It owns the set of registered consumers and the bus-wide per-event-type counters. The Hub is concurrency-safe across Register/Unregister/Deliver/Snapshot; Deliver is the hot path and is RLock-only.
func NewHub ¶
NewHub returns a Hub with the given per-consumer channel buffer size. Zero or negative uses DefaultSendBuffer.
func (*Hub) Broadcast ¶
Broadcast sends the same frame (e.g. SourceState change, Bye on shutdown) to every consumer using drop-oldest semantics. Returns the number of consumers the frame was successfully enqueued for.
func (*Hub) Counters ¶
func (h *Hub) Counters() *PerTypeCounters
Counters exposes the bus-wide per-event-type counter set for daemon-side rendering (status RPC, drop-rate warning).
func (*Hub) Deliver ¶
Deliver fans the raw event out to every matching consumer. Updates bus-wide and per-consumer counters. Always non-blocking — drops the oldest entry in any full sendCh (plan invariant #1).
Called from the bus daemon's main loop after dedup; safe for concurrent callers (Hub uses an RLock, Consumer drop-oldest is single-producer-safe because the daemon serialises Deliver per event).
func (*Hub) Register ¶
Register adds a consumer derived from a Hello frame. Returns a new Consumer with the populated ID + sendCh ready to use, or a RegisterError if the Hello's Filter regex is invalid.
func (*Hub) Snapshot ¶
func (h *Hub) Snapshot() []transport.StatusConsumer
Snapshot returns a deterministic StatusConsumer slice (sorted by PID) for status RPC encoding. Caller MUST NOT mutate the returned slice.
func (*Hub) Unregister ¶
Unregister removes a consumer by ID and closes its sendCh. Idempotent — calling twice or on an unknown ID is a no-op. closeSend shares the same per-consumer lock as Deliver/Broadcast, so a stale Hub snapshot cannot send to the channel after it has been closed.
type Lock ¶
type Lock struct {
// contains filtered or unexported fields
}
Lock represents a held bus.lock. Close releases the flock and removes the PID file, so a subsequent bus can acquire cleanly. A zero Lock is unusable.
func Acquire ¶
Acquire takes the bus lock at path and writes our PID into the file body.
If the file already has a PID written by a previous run:
- Try the flock first — if another process holds it, return ErrBusy (a live bus is running, abort).
- flock acquired but file contains a PID → check if that PID is alive via process.Alive(). If alive → return ErrStaleOwnerAlive (defensive; release our flock first). If dead → take over (orphan cleanup) and overwrite PID with our own.
On success the returned Lock owns an exclusive flock and a file body containing our PID. Concurrent competing processes will get ErrBusy.
func (*Lock) Close ¶
Close releases the flock and best-effort blanks the PID body so a stale reader (e.g. `event status` racing our shutdown) does not see our long-dead PID and try to signal it. The lock file itself is NOT removed — keeping it on disk avoids a race where a competing bus could acquire inode-on-create faster than our truncate.
type Meta ¶
type Meta struct {
ClientID string `json:"client_id"`
Edition string `json:"edition"`
SourceKind dwsevent.SourceKind `json:"source_kind,omitempty"`
IdentityHash string `json:"identity_hash,omitempty"`
SourceID string `json:"source_id,omitempty"`
StartedAt time.Time `json:"started_at"`
SDKVersion string `json:"sdk_version,omitempty"`
BusVersion string `json:"bus_version"`
BusPID int `json:"bus_pid"`
}
Meta is the JSON document written once at bus startup. Its primary purpose is to let `dws event list/status --all` reverse-map directory names (clientIDHash hex) back to the human-readable ClientID. It also records bus identity for protocol-compatibility diagnostics (a future consume client built against bus_version="v2" can refuse to dial a bus_version="v1" bus, etc.).
The file is overwritten on each bus startup (so a previous bus's stale meta does not persist past a fresh boot) and intentionally NOT deleted on Close — keeping it on disk helps `event status` diagnose an orphan (bus.lock empty + bus.meta present + PID dead = clean orphan).
type PerTypeCounters ¶
type PerTypeCounters struct {
// contains filtered or unexported fields
}
PerTypeCounters is bus-wide and tracks how many events of each event_type the bus delivered (or dropped due to backpressure) since startup. Surfaced via `dws event status`. Concurrent-safe.
Implementation note: a map of *atomic uint64 pairs lets us avoid taking the mu in the hot Add() path; mu is only held when inserting a new event_type key.
func NewPerTypeCounters ¶
func NewPerTypeCounters() *PerTypeCounters
NewPerTypeCounters returns an empty counter set.
func (*PerTypeCounters) AddDropped ¶
func (c *PerTypeCounters) AddDropped(eventType string)
AddDropped increments the dropped counter for eventType.
func (*PerTypeCounters) AddReceived ¶
func (c *PerTypeCounters) AddReceived(eventType string)
AddReceived increments the received counter for eventType (allocating the row on first sight of a new type).
func (*PerTypeCounters) DropRatePercent ¶
func (c *PerTypeCounters) DropRatePercent(eventType string) int
DropRatePercent returns rounded (dropped / (received + dropped)) * 100 for the given event_type, or -1 if the type was never observed. Used by the drop-rate stderr warning when crossing a configurable threshold.
func (*PerTypeCounters) Snapshot ¶
func (c *PerTypeCounters) Snapshot() map[string]transport.Counters
Snapshot returns a deterministic point-in-time view, sorted by event_type. Used by status RPC encoding.
func (*PerTypeCounters) SortedTypes ¶
func (c *PerTypeCounters) SortedTypes() []string
SortedTypes returns the known event_types in deterministic order. Useful for human-readable formatting (status table).
type RegisterError ¶
type RegisterError struct{ Err error }
RegisterError wraps the matcher compile error so the daemon can refuse the Hello and return a clean error to the consume client (instead of silently accepting a bad filter regex).
func (*RegisterError) Error ¶
func (e *RegisterError) Error() string
func (*RegisterError) Unwrap ¶
func (e *RegisterError) Unwrap() error
type SourceAdapter ¶
type SourceAdapter interface {
// Start opens the cloud connection and blocks until ctx is cancelled
// or a fatal error occurs. emit is called for each incoming event.
Start(ctx context.Context, emit dwsevent.EmitFn) error
}
SourceAdapter is the interface daemon.go uses to talk to the cloud Source (in practice internal/event/source.DingtalkSource). Kept abstract so the bus daemon can be tested without spinning up the real Stream SDK; the integration test substitutes a fake.