Documentation
¶
Index ¶
- Constants
- func RegisterConfigHandlers(ctx context.Context, configManager *config.Manager, poolManager Manager)
- type ImportAdmission
- type Manager
- type MetricsSnapshot
- type MetricsTracker
- func (mt *MetricsTracker) GetSnapshot() MetricsSnapshot
- func (mt *MetricsTracker) IncArticlesDownloaded()
- func (mt *MetricsTracker) IncArticlesPosted()
- func (mt *MetricsTracker) Reset(ctx context.Context, resetPeak bool, resetTotals bool) error
- func (mt *MetricsTracker) ResetProviderErrors(ctx context.Context) error
- func (mt *MetricsTracker) SetProviderIDs(mapping map[string]string)
- func (mt *MetricsTracker) Start(ctx context.Context)
- func (mt *MetricsTracker) Stop()
- func (mt *MetricsTracker) UpdateDownloadProgress(id string, bytesDownloaded int64)
- type NntpClient
- type ProviderQuotaSnapshot
- type StatsRepository
- type StreamActivitySource
Constants ¶
const MissingRateWarningThreshold = 10.0
MissingRateWarningThreshold is the missing articles per minute rate that triggers a warning.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type ImportAdmission ¶ added in v0.2.1
type ImportAdmission struct {
// contains filtered or unexported fields
}
ImportAdmission is an adaptive counting semaphore that gates how many NZB imports may run concurrently end-to-end. It exposes two caps:
- capIdle: maximum concurrent imports when no stream is active.
- capWhileStreaming: maximum concurrent imports while any stream is active.
A cap value of 0 means "unlimited" for that mode. When both caps are 0 the admission controller is a no-op, which is the default so existing deployments behave exactly as before until the new config knobs are set.
The controller uses a FIFO waiter queue with manual select / channel signalling so we can support ctx cancellation without dropping wake-ups (the classic "lost wakeup" hazard with sync.Cond.Wait + cancellation).
func NewImportAdmission ¶ added in v0.2.1
func NewImportAdmission() *ImportAdmission
NewImportAdmission constructs an admission controller with both caps disabled (0/0 = unlimited). Use SetCaps and SetStreamSource to configure it.
func (*ImportAdmission) Acquire ¶ added in v0.2.1
func (a *ImportAdmission) Acquire(ctx context.Context) (release func(), err error)
Acquire blocks until an admission slot is available or ctx is cancelled. The returned release function MUST be called exactly once when the import is done. When both caps are 0 the call is a fast-path no-op.
func (*ImportAdmission) NotifyStreamChange ¶ added in v0.2.1
func (a *ImportAdmission) NotifyStreamChange()
NotifyStreamChange should be called when the stream count changes so the controller can wake or hold waiters according to the new effective cap.
func (*ImportAdmission) SetCaps ¶ added in v0.2.1
func (a *ImportAdmission) SetCaps(capIdle, capWhileStreaming int)
SetCaps updates both caps. If capWhileStreaming > capIdle and capIdle > 0, the streaming cap is clamped to capIdle. After updating, queued waiters are woken if the effective cap grew.
func (*ImportAdmission) SetStreamSource ¶ added in v0.2.1
func (a *ImportAdmission) SetStreamSource(src StreamActivitySource)
SetStreamSource wires the activity signal. nil sources are tolerated and effectively pin behaviour to capIdle.
type Manager ¶
type Manager interface {
// GetPool returns the current connection pool or error if not available.
// The returned client exposes the narrow NntpClient surface so tests can
// substitute a fake (see internal/testsupport/fakepool). In production it
// is backed by *nntppool.Client.
GetPool() (NntpClient, error)
// SetProviders creates/recreates the pool with new providers
SetProviders(providers []nntppool.Provider) error
// ClearPool shuts down and removes the current pool
ClearPool() error
// HasPool returns true if a pool is currently available
HasPool() bool
// GetMetrics returns the current pool metrics with calculated speeds
GetMetrics() (MetricsSnapshot, error)
// ResetMetrics resets specific cumulative metrics
ResetMetrics(ctx context.Context, resetPeak bool, resetTotals bool) error
// ResetProviderErrors zeroes all per-provider error counts without
// affecting bytes downloaded, peak speed, or history.
ResetProviderErrors(ctx context.Context) error
// IncArticlesDownloaded increments the count of articles successfully downloaded
IncArticlesDownloaded()
// UpdateDownloadProgress updates the bytes downloaded for a specific stream
UpdateDownloadProgress(id string, bytesDownloaded int64)
// IncArticlesPosted increments the count of articles successfully posted
IncArticlesPosted()
// AddProvider adds a single provider to the running pool.
// If no pool exists, a new one is created with this provider.
AddProvider(provider nntppool.Provider) error
// RemoveProvider removes a provider by its stable configuration ID.
// If the last provider is removed, the pool is closed.
RemoveProvider(name string) error
// ResetProviderQuota resets the download quota counter for a provider,
// clearing its consumed-bytes counter and exceeded flag in-place.
ResetProviderQuota(ctx context.Context, poolName string) error
// SetProviderIDs sets a mapping between pool names and configuration IDs.
SetProviderIDs(mapping map[string]string)
// AcquireImportSlot blocks until an admission slot is available for an
// NZB import to start, or ctx is cancelled. The returned release function
// must be called exactly once when the import has finished (success or
// failure). When admission caps are unconfigured (both 0) it is a no-op.
AcquireImportSlot(ctx context.Context) (release func(), err error)
// SetAdmissionCaps configures the two import-concurrency caps:
// capIdle applies when no stream is active; capWhileStreaming applies
// while any stream is active. A cap of 0 means unlimited.
SetAdmissionCaps(capIdle, capWhileStreaming int)
// SetStreamSource wires the activity signal so admission can adapt to
// whether any stream is currently active.
SetStreamSource(src StreamActivitySource)
// NotifyStreamChange must be called by the stream source whenever its
// active stream count changes, so the admission gate can re-evaluate.
NotifyStreamChange()
}
Manager provides centralized NNTP connection pool management.
func NewManager ¶
func NewManager(ctx context.Context, repo StatsRepository) Manager
NewManager creates a new pool manager
type MetricsSnapshot ¶
type MetricsSnapshot struct {
BytesDownloaded int64 `json:"bytes_downloaded"`
BytesUploaded int64 `json:"bytes_uploaded"`
ArticlesDownloaded int64 `json:"articles_downloaded"`
ArticlesPosted int64 `json:"articles_posted"`
TotalErrors int64 `json:"total_errors"`
ProviderErrors map[string]int64 `json:"provider_errors"`
ProviderBytes map[string]int64 `json:"provider_bytes"`
ProviderBytes24h map[string]int64 `json:"provider_bytes_24h"`
ProviderStartedAt map[string]time.Time `json:"provider_started_at"`
ProviderQuotas map[string]ProviderQuotaSnapshot `json:"provider_quotas,omitempty"`
DownloadSpeedBytesPerSec float64 `json:"download_speed_bytes_per_sec"`
MaxDownloadSpeedBytesPerSec float64 `json:"max_download_speed_bytes_per_sec"`
UploadSpeedBytesPerSec float64 `json:"upload_speed_bytes_per_sec"`
Timestamp time.Time `json:"timestamp"`
StartedAt time.Time `json:"started_at"`
ProviderMissingRates map[string]float64 `json:"provider_missing_rates"`
ProviderMissingWarning map[string]bool `json:"provider_missing_warning"`
ProviderSpeeds map[string]float64 `json:"provider_speeds"`
}
MetricsSnapshot represents pool metrics at a point in time with calculated values
type MetricsTracker ¶
type MetricsTracker struct {
// contains filtered or unexported fields
}
MetricsTracker tracks pool metrics over time and calculates rates
func NewMetricsTracker ¶
func NewMetricsTracker(pool *nntppool.Client, repo StatsRepository) *MetricsTracker
NewMetricsTracker creates a new metrics tracker
func (*MetricsTracker) GetSnapshot ¶
func (mt *MetricsTracker) GetSnapshot() MetricsSnapshot
GetSnapshot returns the current metrics with calculated speeds
func (*MetricsTracker) IncArticlesDownloaded ¶
func (mt *MetricsTracker) IncArticlesDownloaded()
IncArticlesDownloaded increments the count of articles successfully downloaded
func (*MetricsTracker) IncArticlesPosted ¶
func (mt *MetricsTracker) IncArticlesPosted()
IncArticlesPosted increments the count of articles successfully posted
func (*MetricsTracker) Reset ¶
Reset resets cumulative metrics both in memory and in the database based on flags
func (*MetricsTracker) ResetProviderErrors ¶ added in v0.2.1
func (mt *MetricsTracker) ResetProviderErrors(ctx context.Context) error
ResetProviderErrors zeroes out all per-provider error counts by offsetting the live pool error counts. Bytes, speed, and history are untouched.
func (*MetricsTracker) SetProviderIDs ¶ added in v0.2.1
func (mt *MetricsTracker) SetProviderIDs(mapping map[string]string)
SetProviderIDs sets the mapping from pool names to config IDs
func (*MetricsTracker) Start ¶
func (mt *MetricsTracker) Start(ctx context.Context)
Start begins collecting metrics samples
func (*MetricsTracker) Stop ¶
func (mt *MetricsTracker) Stop()
Stop stops collecting metrics samples
func (*MetricsTracker) UpdateDownloadProgress ¶
func (mt *MetricsTracker) UpdateDownloadProgress(id string, bytesDownloaded int64)
UpdateDownloadProgress updates the live bytes downloaded counter
type NntpClient ¶ added in v0.2.1
type NntpClient interface {
// Body fetches an article body via the default (non-priority) lane.
// Used by the importer to download NZB segments during scanning.
Body(ctx context.Context, messageID string, onMeta ...func(nntppool.YEncMeta)) (*nntppool.ArticleBody, error)
// BodyAsync fetches an article body asynchronously, streaming the decoded
// payload to w. The returned channel yields exactly one BodyResult.
BodyAsync(ctx context.Context, messageID string, w io.Writer, onMeta ...func(nntppool.YEncMeta)) <-chan nntppool.BodyResult
// BodyPriority fetches an article body via the priority lane. Streaming
// reads use this so live playback isn't queued behind a background import.
BodyPriority(ctx context.Context, messageID string, onMeta ...func(nntppool.YEncMeta)) (*nntppool.ArticleBody, error)
// Stat checks whether an article exists on at least one provider without
// downloading the body. Used by health checks and validation.
Stat(ctx context.Context, messageID string) (*nntppool.StatResult, error)
// Stats returns a snapshot of pool/provider statistics used by the metrics
// tracker and the system handlers.
Stats() nntppool.ClientStats
}
NntpClient is the narrow surface of the underlying nntppool.Client that the rest of AltMount calls through Manager.GetPool. Defining it here lets tests inject a deterministic fake (see internal/testsupport/fakepool) without standing up real NNTP connections, and pins exactly which operations the streaming, import, validation, and metrics paths depend on.
Implementations must be safe for concurrent use. The production implementation is *nntppool.Client; the contract below intentionally mirrors its signatures so the existing client satisfies the interface without an adapter.
Keep this interface small. Anything that needs a behavior not listed here should add the method explicitly so callers stay observable.
type ProviderQuotaSnapshot ¶ added in v0.2.1
type ProviderQuotaSnapshot struct {
QuotaBytes int64 `json:"quota_bytes"`
QuotaUsed int64 `json:"quota_used"`
QuotaResetAt time.Time `json:"quota_reset_at,omitempty"`
QuotaExceeded bool `json:"quota_exceeded"`
}
ProviderQuotaSnapshot holds the quota state for a single provider.
type StatsRepository ¶
type StatsRepository interface {
UpdateSystemStat(ctx context.Context, key string, value int64) error
BatchUpdateSystemStats(ctx context.Context, stats map[string]int64) error
MigrateSystemStats(ctx context.Context, stats map[string]int64, deleteKeys []string) error
GetSystemStats(ctx context.Context) (map[string]int64, error)
AddBytesDownloadedToDailyStat(ctx context.Context, bytes int64) error
AddProviderBytesToHourlyStat(ctx context.Context, providerID string, bytes int64) error
RecordProviderSpeedTest(ctx context.Context, providerID string, speedMbps float64) error
GetProviderHourlyStats(ctx context.Context, hours int) (map[string]int64, error)
ClearProviderHourlyStats(ctx context.Context) error
GetOldestStatDate(ctx context.Context) (time.Time, error)
GetOldestProviderStatDates(ctx context.Context) (map[string]time.Time, error)
}
StatsRepository defines the interface for persisting pool statistics
type StreamActivitySource ¶ added in v0.2.1
type StreamActivitySource interface {
ActiveStreams() int
}
StreamActivitySource reports how many streams are currently active. Implemented by api.StreamTracker; kept here so the dependency flows api -> pool.