bus

package
v1.0.54 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 21, 2026 License: Apache-2.0 Imports: 22 Imported by: 0

Documentation

Overview

Package bus implements the daemon side of the dws event subsystem: one long-lived process per ClientID that holds the single cloud connection and fans out events to N local consumers over IPC.

Files (mirroring plan §5 layout):

daemon.go    main loop: lock → meta → IPC listen → ready → source.Start
hub.go       consumer registry, per-consumer sendCh, drop-oldest backpressure
metrics.go   per-event-type + per-consumer received/dropped counters
lockfile.go  single bus.lock (flock + PID content + stale recovery)
meta.go      bus.meta JSON (clientID/edition/started_at) for list/status reverse mapping

Lifecycle invariants (plan §4 invariants 1–7):

  1. emit non-blocking (drop-oldest, never block SDK callback)
  2. dedup on event_id (LRU) to absorb cloud-side redelivery
  3. single bus per ClientID (bus.lock enforces, all FS paths use clientIDHash)
  4. upstream always full subscription; consumer filter only affects bus→consume
  5. dead-consumer auto-reap on socket EOF
  6. startup order: lock → meta → IPC listen → ready pipe → Source.Start
  7. stdio detach when fork'd by busctl/spawn

Index

Constants

View Source
const (
	EnvIdleTimeout    = "DWS_EVENT_BUS_IDLE_TIMEOUT" // Go duration, e.g. "10m"
	EnvConsumerBuffer = "DWS_EVENT_CONSUMER_BUFFER"  // integer
	EnvDedupLRU       = "DWS_EVENT_DEDUP_LRU"        // integer
	EnvDropWarnPct    = "DWS_EVENT_DROP_WARN_PCT"    // integer 1-100
)

Tunable env vars (plan §15 已决项 — surfaced for operators without requiring a new CLI flag for each knob). Each lookup is read-once at bus startup; runtime changes require a bus restart.

View Source
const CurrentBusVersion = "v1"

CurrentBusVersion identifies the bus wire/storage compatibility level. Bumped only on breaking changes (IPC protocol, lockfile shape, meta schema). v1 is the initial value; the field is parsed defensively by readers (older readers tolerate unknown fields via encoding/json).

View Source
const DefaultDropWarnPercent = 5

DefaultDropWarnPercent is the threshold above which per-event-type drop rate triggers a slog WARN line in bus.log. 5% is the plan default (§15 已决项); overridable via DWS_EVENT_DROP_WARN_PCT.

We use whole-percentage granularity (int) because the counter math is integer; sub-percent precision would just add noise.

View Source
const DefaultSendBuffer = 100

DefaultSendBuffer is the per-consumer channel capacity used when the Hub is constructed via NewHub. Sized to absorb a short event burst (~100ms at ~1k evt/s) without backpressure. Overridable via DWS_EVENT_CONSUMER_BUFFER at daemon start (plan §15 已决但暴露方式).

View Source
const LockFileName = "bus.lock"

LockFileName is the on-disk name of the bus single-instance lock. It lives inside the bus working directory (<ConfigDir>/events/<edition>/<source_kind>/<identity_hash>/).

View Source
const MetaFileName = "bus.meta"

MetaFileName is the on-disk name of the bus metadata file. It lives alongside bus.lock and bus.sock inside the bus working directory.

Variables

View Source
var ErrBusy = lock.ErrBusy

ErrBusy is re-exported from lock for callers that only depend on bus.

View Source
var ErrStaleOwnerAlive = errors.New("bus: lock file PID is alive but flock was released; assuming live owner")

ErrStaleOwnerAlive indicates the PID stored in bus.lock points at a live process — there is already a bus running for this ClientID and we must not start another one. (This case is hit when the holder is still alive but its flock was somehow released; in practice flock + PID always agree, so this is mostly defensive.)

Functions

func ApplyEnvTuning

func ApplyEnvTuning(cfg *Config)

ApplyEnvTuning fills in Config defaults from the env vars listed above for any fields the caller left at zero. The cobra layer calls this after constructing Config so explicit flag values still win.

Defaults (when env is absent or invalid):

IdleTimeout      → 5m
ConsumerBuffer   → DefaultSendBuffer
DedupCapacity    → 0 (let dedup package's DefaultCapacity apply)
DropWarnPercent  → DefaultDropWarnPercent

Invalid env values (non-parseable, out of range) are silently ignored and the default is used. We deliberately do NOT fail the bus on bad env input — operators shouldn't lose a daemon over a typo'd env var.

func ReadHolderPID

func ReadHolderPID(path string) int

ReadHolderPID returns the PID stored in path, or 0 if the file is missing or unreadable. Does NOT attempt to acquire the lock — useful for `event status` to display the holder without contention.

func Run

func Run(ctx context.Context, cfg Config) error

Run starts the bus daemon. Lifecycle (plan §4 invariant #6):

  1. Acquire bus.lock (single-instance enforcement)
  2. Write bus.meta
  3. Listen IPC (so consumers can connect before SDK starts pushing)
  4. Signal readiness via ReadyPipe
  5. Start the Source (cloud SDK); concurrent with consumer accept loop
  6. Wait on ctx for shutdown signal
  7. Graceful: broadcast Bye → close listener → close source → release lock

Run blocks until ctx is cancelled, the Source returns an error, or a fatal startup error occurs.

func WriteMeta

func WriteMeta(dir string, m Meta) error

WriteMeta atomically writes m to <dir>/bus.meta. Atomic via tmp-file + rename. Directory permissions are not changed; caller must mkdir the containing directory beforehand with pkg/config.DirPerm.

Types

type Config

type Config struct {
	// WorkDir is the bus working directory:
	//   <ConfigDir>/events/<edition>/<source_kind>/<identity_hash>/
	// The caller MUST mkdir this with pkg/config.DirPerm before calling Run.
	WorkDir string

	// IPCEndpoint is the Unix socket path or Windows pipe name. Caller
	// computes this from WorkDir (Unix) or edition/clientIDHash (Windows).
	IPCEndpoint string

	// ClientID is the human-readable identifier written into bus.meta and
	// status output. NOT used in any path.
	ClientID string

	// SourceKind/IdentityHash/SourceID are diagnostic identity fields used by
	// list/status. Empty SourceKind is interpreted as app_stream for backward
	// compatibility.
	SourceKind   dwsevent.SourceKind
	IdentityHash string
	SourceID     string

	// Edition is written into bus.meta. Comes from edition.Get().Name with
	// "open" fallback applied by the caller.
	Edition string

	// SDKVersion is recorded in bus.meta for diagnostics.
	SDKVersion string

	// Source is the cloud adapter. Required.
	Source SourceAdapter

	// IdleTimeout: bus self-exits after this long with zero consumers.
	// Zero disables (bus runs until SIGTERM).
	IdleTimeout time.Duration

	// ConsumerBuffer overrides per-consumer sendCh capacity. Zero uses
	// DefaultSendBuffer.
	ConsumerBuffer int

	// DedupCapacity overrides event_id LRU size. Zero uses dedup.DefaultCapacity.
	DedupCapacity int

	// DropWarnPercent is the per-event-type drop-rate threshold (whole
	// percentage points) that triggers a slog WARN in bus.log. Zero or
	// out-of-range values fall back to DefaultDropWarnPercent. Overridable
	// via env DWS_EVENT_DROP_WARN_PCT (read by the cobra layer).
	DropWarnPercent int

	// ReadyPipe receives a single byte ('R' on success, 'E' on failure)
	// once the bus has either come up or failed startup, so the parent
	// process forked by busctl/spawn can stop polling and either dial or
	// surface the error. nil disables (foreground mode).
	ReadyPipe *os.File

	// Logger sink. Nil → slog.Default.
	Logger *slog.Logger
}

Config bundles everything Run needs to start a daemon. All paths and identifiers come from busctl/Source so the bus itself stays oblivious to ConfigDir / edition rules.

type Consumer

type Consumer struct {
	ID           int      // monotonic, assigned by Hub
	PID          int      // from Hello.ConsumerPID
	EventTypes   []string // raw wildcard patterns from Hello
	Filter       string   // raw regex from Hello (for status display)
	SubscribeID  string   // optional personal subscription label and local isolation key
	SubscribedAt time.Time
	SendCh       chan any // bus → consume frames (Event/SourceState/Heartbeat/Bye)
	// contains filtered or unexported fields
}

Consumer represents one registered IPC connection to the bus. Wire-side reader/writer goroutines are owned by daemon.go; the Hub holds the metadata + sendCh.

type Hub

type Hub struct {
	// contains filtered or unexported fields
}

Hub is the bus's fan-out engine. It owns the set of registered consumers and the bus-wide per-event-type counters. The Hub is concurrency-safe across Register/Unregister/Deliver/Snapshot; Deliver is the hot path and is RLock-only.

func NewHub

func NewHub(bufferSize int) *Hub

NewHub returns a Hub with the given per-consumer channel buffer size. Zero or negative uses DefaultSendBuffer.

func (*Hub) Broadcast

func (h *Hub) Broadcast(frame any) int

Broadcast sends the same frame (e.g. SourceState change, Bye on shutdown) to every consumer using drop-oldest semantics. Returns the number of consumers the frame was successfully enqueued for.

func (*Hub) Counters

func (h *Hub) Counters() *PerTypeCounters

Counters exposes the bus-wide per-event-type counter set for daemon-side rendering (status RPC, drop-rate warning).

func (*Hub) Deliver

func (h *Hub) Deliver(raw *dwsevent.RawEvent)

Deliver fans the raw event out to every matching consumer. Updates bus-wide and per-consumer counters. Always non-blocking — drops the oldest entry in any full sendCh (plan invariant #1).

Called from the bus daemon's main loop after dedup; safe for concurrent callers (Hub uses an RLock, Consumer drop-oldest is single-producer-safe because the daemon serialises Deliver per event).

func (*Hub) Len

func (h *Hub) Len() int

Len returns the current number of registered consumers.

func (*Hub) Register

func (h *Hub) Register(hello transport.Hello) (*Consumer, error)

Register adds a consumer derived from a Hello frame. Returns a new Consumer with the populated ID + sendCh ready to use, or a RegisterError if the Hello's Filter regex is invalid.

func (*Hub) Snapshot

func (h *Hub) Snapshot() []transport.StatusConsumer

Snapshot returns a deterministic StatusConsumer slice (sorted by PID) for status RPC encoding. Caller MUST NOT mutate the returned slice.

func (*Hub) Unregister

func (h *Hub) Unregister(id int)

Unregister removes a consumer by ID and closes its sendCh. Idempotent — calling twice or on an unknown ID is a no-op. closeSend shares the same per-consumer lock as Deliver/Broadcast, so a stale Hub snapshot cannot send to the channel after it has been closed.

type Lock

type Lock struct {
	// contains filtered or unexported fields
}

Lock represents a held bus.lock. Close releases the flock and removes the PID file, so a subsequent bus can acquire cleanly. A zero Lock is unusable.

func Acquire

func Acquire(path string) (*Lock, error)

Acquire takes the bus lock at path and writes our PID into the file body.

If the file already has a PID written by a previous run:

  1. Try the flock first — if another process holds it, return ErrBusy (a live bus is running, abort).
  2. flock acquired but file contains a PID → check if that PID is alive via process.Alive(). If alive → return ErrStaleOwnerAlive (defensive; release our flock first). If dead → take over (orphan cleanup) and overwrite PID with our own.

On success the returned Lock owns an exclusive flock and a file body containing our PID. Concurrent competing processes will get ErrBusy.

func (*Lock) Close

func (l *Lock) Close() error

Close releases the flock and best-effort blanks the PID body so a stale reader (e.g. `event status` racing our shutdown) does not see our long-dead PID and try to signal it. The lock file itself is NOT removed — keeping it on disk avoids a race where a competing bus could acquire inode-on-create faster than our truncate.

func (*Lock) HoldsPath

func (l *Lock) HoldsPath() string

HoldsPath returns the path the lock is held on.

type Meta

type Meta struct {
	ClientID     string              `json:"client_id"`
	Edition      string              `json:"edition"`
	SourceKind   dwsevent.SourceKind `json:"source_kind,omitempty"`
	IdentityHash string              `json:"identity_hash,omitempty"`
	SourceID     string              `json:"source_id,omitempty"`
	StartedAt    time.Time           `json:"started_at"`
	SDKVersion   string              `json:"sdk_version,omitempty"`
	BusVersion   string              `json:"bus_version"`
	BusPID       int                 `json:"bus_pid"`
}

Meta is the JSON document written once at bus startup. Its primary purpose is to let `dws event list/status --all` reverse-map directory names (clientIDHash hex) back to the human-readable ClientID. It also records bus identity for protocol-compatibility diagnostics (a future consume client built against bus_version="v2" can refuse to dial a bus_version="v1" bus, etc.).

The file is overwritten on each bus startup (so a previous bus's stale meta does not persist past a fresh boot) and intentionally NOT deleted on Close — keeping it on disk helps `event status` diagnose an orphan (bus.lock empty + bus.meta present + PID dead = clean orphan).

func ReadMeta

func ReadMeta(dir string) (*Meta, error)

ReadMeta loads and parses <dir>/bus.meta. Returns (nil, error) when the file is missing or malformed. Used by `event list/status --all` to resolve clientIDHash → original ClientID.

type PerTypeCounters

type PerTypeCounters struct {
	// contains filtered or unexported fields
}

PerTypeCounters is bus-wide and tracks how many events of each event_type the bus delivered (or dropped due to backpressure) since startup. Surfaced via `dws event status`. Concurrent-safe.

Implementation note: a map of *atomic uint64 pairs lets us avoid taking the mu in the hot Add() path; mu is only held when inserting a new event_type key.

func NewPerTypeCounters

func NewPerTypeCounters() *PerTypeCounters

NewPerTypeCounters returns an empty counter set.

func (*PerTypeCounters) AddDropped

func (c *PerTypeCounters) AddDropped(eventType string)

AddDropped increments the dropped counter for eventType.

func (*PerTypeCounters) AddReceived

func (c *PerTypeCounters) AddReceived(eventType string)

AddReceived increments the received counter for eventType (allocating the row on first sight of a new type).

func (*PerTypeCounters) DropRatePercent

func (c *PerTypeCounters) DropRatePercent(eventType string) int

DropRatePercent returns rounded (dropped / (received + dropped)) * 100 for the given event_type, or -1 if the type was never observed. Used by the drop-rate stderr warning when crossing a configurable threshold.

func (*PerTypeCounters) Snapshot

func (c *PerTypeCounters) Snapshot() map[string]transport.Counters

Snapshot returns a deterministic point-in-time view, sorted by event_type. Used by status RPC encoding.

func (*PerTypeCounters) SortedTypes

func (c *PerTypeCounters) SortedTypes() []string

SortedTypes returns the known event_types in deterministic order. Useful for human-readable formatting (status table).

type RegisterError

type RegisterError struct{ Err error }

RegisterError wraps the matcher compile error so the daemon can refuse the Hello and return a clean error to the consume client (instead of silently accepting a bad filter regex).

func (*RegisterError) Error

func (e *RegisterError) Error() string

func (*RegisterError) Unwrap

func (e *RegisterError) Unwrap() error

type SourceAdapter

type SourceAdapter interface {
	// Start opens the cloud connection and blocks until ctx is cancelled
	// or a fatal error occurs. emit is called for each incoming event.
	Start(ctx context.Context, emit dwsevent.EmitFn) error
}

SourceAdapter is the interface daemon.go uses to talk to the cloud Source (in practice internal/event/source.DingtalkSource). Kept abstract so the bus daemon can be tested without spinning up the real Stream SDK; the integration test substitutes a fake.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL