nntp

package
v0.4.7-1-testing-2025-... Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 3, 2025 License: GPL-2.0 Imports: 15 Imported by: 0

README ΒΆ

NNTP Server Implementation

🎯 Phase 6: Article Retrieval - 60% COMPLETE!

The NNTP server foundation and core article retrieval commands have been successfully implemented with a clean, modular architecture spread across multiple files.

πŸ“ File Structure

internal/nntp/
β”œβ”€β”€ nntp-auth-manager.go        # Authentication manager logic
β”œβ”€β”€ nntp-backend-pool.go        # Backend connection pool
β”œβ”€β”€ nntp-client-commands.go     # NNTP client command implementations
β”œβ”€β”€ nntp-client.go              # NNTP client core
β”œβ”€β”€ nntp-cmd-article.go         # ARTICLE command handler
β”œβ”€β”€ nntp-cmd-auth.go            # AUTHINFO command handler
β”œβ”€β”€ nntp-cmd-basic.go           # Basic NNTP commands (HELP, QUIT, etc.)
β”œβ”€β”€ nntp-cmd-body.go            # BODY command handler
β”œβ”€β”€ nntp-cmd-group.go           # GROUP command handler
β”œβ”€β”€ nntp-cmd-head.go            # HEAD command handler
β”œβ”€β”€ nntp-cmd-helpers.go         # Shared command helpers/utilities
β”œβ”€β”€ nntp-cmd-list.go            # LIST and LISTGROUP command handlers
β”œβ”€β”€ nntp-cmd-reader.go          # MODE READER and reader mode logic
β”œβ”€β”€ nntp-cmd-stat.go            # STAT command handler
β”œβ”€β”€ nntp-cmd-xhdr.go            # XHDR command handler
β”œβ”€β”€ nntp-cmd-xover.go           # XOVER command handler
β”œβ”€β”€ nntp-server-cliconns.go     # Client connection management
β”œβ”€β”€ nntp-server-statistics.go   # Server statistics and monitoring
└── nntp-server.go              # Main server struct and startup logic

examples/nntp-server/
└── main.go                     # Example server implementation

βœ… Implemented Features

Core Server Infrastructure
  • βœ… NNTPServer struct with configuration integration
  • βœ… TCP listener with TLS/SSL support
  • βœ… Client connection management with proper lifecycle handling
  • βœ… Graceful shutdown with timeout handling
  • βœ… Connection limits and resource management
Essential NNTP Commands
  • βœ… CAPABILITIES - Server capability announcement
  • βœ… MODE READER - Reader mode switching
  • βœ… AUTHINFO USER/PASS - Basic authentication framework
  • βœ… QUIT - Clean connection termination
  • βœ… HELP - Command help system
Group Management Commands
  • βœ… LIST [ACTIVE|NEWSGROUPS] - List newsgroups with database integration
  • βœ… GROUP - Newsgroup selection with database lookup
  • βœ… LISTGROUP - Article listing with database integration
Article Retrieval Commands
  • βœ… STAT - Article existence checking with message-ID/number support
  • βœ… HEAD - Article headers retrieval with full parsing
  • βœ… BODY - Article body retrieval with proper escaping
  • βœ… ARTICLE - Complete article retrieval (headers + body)
  • βœ… XOVER - Article overview with range support
  • βœ… XHDR - Header field retrieval with range support
Database Integration
  • βœ… GetOverviewsRange() - Range-based overview queries for XOVER
  • βœ… GetOverviewByMessageID() - Message-ID based overview lookup
  • βœ… GetHeaderFieldRange() - Header field extraction for XHDR
  • βœ… Article retrieval by number and message-ID
Support Systems
  • βœ… Authentication Manager with database integration
  • βœ… Server Statistics with comprehensive metrics
  • βœ… Configuration Extension for NNTP settings
  • βœ… Error Handling and logging

πŸ”— Database Integration

The NNTP server seamlessly integrates with the existing database layer:

  • Groups: Uses MainDBGetNewsgroups() and MainDBGetNewsgroup()
  • Authentication: Leverages GetUserByUsername() and GetUserPermissions()
  • Articles: Fully integrated with per-group databases
    • GetArticleByNum() - Retrieve articles by number
    • GetArticleByMessageID() - Retrieve articles by message-ID
    • GetOverviewsRange() - Range-based overview queries
    • GetOverviewByMessageID() - Overview lookup by message-ID
    • GetHeaderFieldRange() - Header field extraction

πŸ§ͺ Testing

The server can be tested using the example:

# Build and run the example
cd examples/nntp-server
go build && ./nntp-server

# Test with telnet
telnet localhost 1119

# Try these commands:
CAPABILITIES
MODE READER
LIST
LIST NEWSGROUPS
GROUP comp.programming
LISTGROUP
STAT 1
HEAD 1
BODY 1
ARTICLE 1
XOVER 1-10
XHDR subject 1-5
HELP
QUIT

⚑ Next Steps: Remaining Phase 6 Tasks

Priority 1: Authentication Enhancement
  • Implement proper AUTHINFO USER/PASS sequence
  • Add real password validation with bcrypt
  • Implement group access control and permissions
Priority 2: Advanced Commands
  • HDR and OVER - RFC 3977 standard commands
  • POST - Article posting (optional)
  • DATE - Server date/time
  • NEWGROUPS - New groups since date
  • NEWNEWS - New articles since date
Priority 3: Performance & Security
  • Article caching for frequently accessed content
  • Rate limiting per connection/IP
  • Connection timeout management
  • Comprehensive logging of NNTP operations

πŸŽ‰ Current Status

MAJOR ACHIEVEMENT: All core article retrieval commands are now functional:

  • STAT, HEAD, BODY, ARTICLE - Full article access
  • XOVER, XHDR - Efficient overview and header queries
  • Range support - Multiple article operations
  • Message-ID support - Both number and message-ID addressing
  • Implement STAT command with database lookups
  • Add HEAD command for article headers
  • Add BODY command for article content
  • Add ARTICLE command for complete articles
  • Support both message-ID and article number addressing
Priority 3: Overview Commands
  • Implement XOVER with NOV format generation
  • Add XHDR for header-specific queries
  • Add article range parsing (e.g., "100-200", "150-")

πŸ—οΈ Architecture Highlights

Modular Design
  • Clean separation of concerns across multiple files
  • Easy to extend with new commands
  • Proper error handling and logging
Performance Considerations
  • Connection pooling and resource limits
  • Graceful shutdown with timeout
  • Statistics tracking for monitoring
Integration Ready
  • Uses existing database structures (models.Article, models.Newsgroup)
  • Leverages current authentication system
  • Extends existing configuration seamlessly

πŸ“Š Current Status

  • Phase 6a: βœ… COMPLETE (Foundation)
  • Phase 6b: ⚑ READY TO BEGIN (Core Commands)
  • Estimated Time: 2-3 days for full Phase 6b implementation

The NNTP server foundation provides a solid, production-ready base for implementing the complete RFC 3977 protocol!

Documentation ΒΆ

Index ΒΆ

Constants ΒΆ

View Source
const (
	// NNTPWelcomeCodeMin is the minimum welcome code for NNTP servers.
	NNTPWelcomeCodeMin int = 200
	// NNTPWelcomeCodeMax is the maximum welcome code for NNTP servers.
	NNTPWelcomeCodeMax int = 201
	// NNTPMoreInfoCode indicates more information is required (e.g., password).
	NNTPMoreInfoCode int = 381
	// NNTPAuthSuccess indicates successful authentication.
	NNTPAuthSuccess int = 281

	// ArticleFollows indicates that an article follows (multi-line).
	ArticleFollows int = 220
	// HeadFollows indicates that the head of an article follows (multi-line).
	HeadFollows int = 221
	// BodyFollows indicates that the body of an article follows (multi-line).
	BodyFollows int = 222
	// ArticleExists indicates that the article exists (no body follows).
	ArticleExists int = 223

	// NoSuchArticle indicates that no such article exists.
	NoSuchArticle int = 430
	// DMCA indicates a DMCA takedown.
	DMCA int = 451

	// DefaultConnExpire is the default connection expiration duration.
	DefaultConnExpire = 25 * time.Second

	// MaxReadLines is the maximum lines to read per response (allow for large group lists).
	MaxReadLines = 500000
)
View Source
const (
	// NNTP protocol constants
	DOT  = "."
	CR   = "\r"
	LF   = "\n"
	CRLF = CR + LF
)
View Source
const MaxReadLinesArticle = 256 * 1024

MaxReadLinesArticle Maximum lines for ARTICLE command, including headers and body

MaxReadLinesBody Maximum lines for BODY command, which retrieves the body of an article

View Source
const MaxReadLinesHeaders = 1024

MaxReadLinesHeaders Maximum lines for HEAD command, which only retrieves headers

Variables ΒΆ

View Source
var DefaultBinaryExcludePatterns = []string{
	"@dk.b.*", "@*dvdnordic*",
	"@a.b.*", "@ab.alt.*", "@ab.mom*", "@alt.b.*",
	"@*alt-bin*", "@*alt.bin*", "@*alt.dvd*", "@*alt.hdtv*",
	"@*alt.binaries*", "@*alt.binaries.dvd*", "@*alt.binaries.hdtv*",
	"@*nairies*", "@*naries*", "@*.bain*", "@*.banar*", "@*.banir*",
	"@*.biana*", "@*.bianr*", "@*.biin*", "@*.binar*", "@*.binai*",
	"@*.binaer*", "@*.bineri*", "@*.biniar*", "@*.binira*",
	"@*.binrie*", "@*.biya*", "@*.boneles*", "@*cd.image*",
	"@*dateien*", "@*.files*", "@*.newfiles*", "@*music.bin*",
	"@*nzb*", "@*mp3*", "@*ictures*", "@*iktures*",
	"@*crack*", "@*serial*", "@*warez*",
	"@unidata.*",
}

Default exclusion patterns for binary groups (INN2 style) These @patterns will reject entire articles if any newsgroup matches

View Source
var DefaultNNTPcliconnTimeout = time.Duration(60 * time.Second)
View Source
var DefaultNoSendPatterns = []string{
	"!control", "!control.*",
	"!junk", "!junk.*",
	"!local", "!local.*",
	"!ka.*", "!gmane.*", "!gwene.*",
}

Default exclusion patterns for administrative and unwanted groups (INN2 $DEFAULT) These !patterns exclude specific groups but allow crossposted articles

View Source
var DefaultSexExcludePatterns = []string{
	"@*erotic*", "@*gay*", "@*paedo*", "@*pedo*", "@*porn*", "@*sex*", "@*xxx*",
}
View Source
var ErrArticleNotFound = fmt.Errorf("article not found")
View Source
var ErrArticleRemoved = fmt.Errorf("article removed (DMCA)")
View Source
var ErrNewsgroupNotFound = fmt.Errorf("newsgroup not found")
View Source
var ErrOutOfRange error = fmt.Errorf("end range exceeds group last article number")
View Source
var MaxReadLinesXover int64 = 100 // XOVER command typically retrieves overview lines MaxBatch REFERENCES this in processor!!!

MaxReadLinesXover Maximum lines for XOVER command, which retrieves overview lines

Functions ΒΆ

func GetPatternType ΒΆ

func GetPatternType(pattern string) string

GetPatternType returns the type of pattern (!exclude, @reject, or normal)

func MultiLineHeaderToMergedString ΒΆ

func MultiLineHeaderToMergedString(vals []string) string

func NormalizePattern ΒΆ

func NormalizePattern(pattern string) string

NormalizePattern removes special prefixes and returns the core pattern

func ParseHeaders ΒΆ

func ParseHeaders(article *models.Article, headerLines []string) error

parseHeaders parses header lines into the article headers map

func ParseLegacyArticleLines ΒΆ

func ParseLegacyArticleLines(messageID string, lines []string, bulkmode bool) (*models.Article, error)

ParseArticleLines parses article lines into headers and body

func RemoteAddr ΒΆ

func RemoteAddr(conn net.Conn) string

RemoteAddr extracts the IP address from a connection

func StrIsIPv4 ΒΆ

func StrIsIPv4(address string) bool

StrIsIPv4 checks if a string is a valid IPv4 address

func StrIsIPv6 ΒΆ

func StrIsIPv6(address string) bool

StrIsIPv6 checks if a string contains IPv6 format (simple check)

func ValidatePatterns ΒΆ

func ValidatePatterns(patterns []string) []string

ValidatePatterns validates a list of patterns for syntax errors

Types ΒΆ

type ArticleProcessor ΒΆ

type ArticleProcessor interface {
	ProcessIncomingArticle(article *models.Article) (int, error)
	Lookup(msgIdItem *history.MessageIdItem) (int, error)
}

Forward declaration to avoid circular imports Processor interface for article processing

type ArticleRetrievalResult ΒΆ

type ArticleRetrievalResult struct {
	Article    *models.Article
	Overview   *models.Overview
	ArticleNum int64
	MsgIdItem  *history.MessageIdItem
	GroupDBs   *database.GroupDBs
}

ArticleRetrievalResult contains the result of article lookup

type ArticleRetrievalType ΒΆ

type ArticleRetrievalType int

ArticleRetrievalType defines what content to send

const (
	RetrievalArticle ArticleRetrievalType = iota // Headers + Body
	RetrievalHead                                // Headers only
	RetrievalBody                                // Body only
	RetrievalStat                                // Status only (no content)
)

type AuthManager ΒΆ

type AuthManager struct {
	// contains filtered or unexported fields
}

AuthManager handles NNTP authentication

func NewAuthManager ΒΆ

func NewAuthManager(db *database.Database) *AuthManager

NewAuthManager creates a new authentication manager

func (*AuthManager) AuthenticateUser ΒΆ

func (am *AuthManager) AuthenticateUser(username, password string) (*models.NNTPUser, error)

AuthenticateUser authenticates a user with username and password

func (*AuthManager) CanPost ΒΆ

func (am *AuthManager) CanPost(user *models.NNTPUser) bool

CanPost checks if a user has posting privileges

func (*AuthManager) CheckConnectionLimit ΒΆ

func (am *AuthManager) CheckConnectionLimit(user *models.NNTPUser) bool

CheckConnectionLimit checks if user can create another connection

func (*AuthManager) CheckGroupAccess ΒΆ

func (am *AuthManager) CheckGroupAccess(user *models.NNTPUser, groupName string) bool

CheckGroupAccess checks if a user has access to a specific newsgroup

func (*AuthManager) IsAdmin ΒΆ

func (am *AuthManager) IsAdmin(user *models.NNTPUser) bool

IsAdmin checks if a user has admin privileges

type BackendConfig ΒΆ

type BackendConfig struct {
	Host           string        // hostname or IP address of the NNTP server
	Port           int           // port number for the NNTP server
	SSL            bool          // whether to use SSL/TLS
	Username       string        // username for authentication
	Password       string        // password for authentication
	ConnectTimeout time.Duration // timeout for establishing a connection
	//ReadTimeout    time.Duration    // timeout for reading from the connection
	//WriteTimeout   time.Duration    // timeout for writing to the connection
	MaxConns int              // maximum number of connections to this backend
	Provider *config.Provider // link to provider config
	Mux      sync.Mutex
}

BackendConfig holds configuration for an NNTP client

type BackendConn ΒΆ

type BackendConn struct {
	Backend *BackendConfig

	Pool *Pool // link to parent pool
	// contains filtered or unexported fields
}

BackendConn represents an NNTP connection to a server. It manages the connection state, authentication, and provides methods for interacting with the NNTP server.

func NewConn ΒΆ

func NewConn(backend *BackendConfig) *BackendConn

NewConn creates a new empty NNTP connection with the provided backend configuration.

func (*BackendConn) CloseFromPoolOnly ΒΆ

func (c *BackendConn) CloseFromPoolOnly() error

CloseFromPoolOnly closes a raw NNTP connection

func (*BackendConn) Connect ΒΆ

func (c *BackendConn) Connect() error

Connect establishes connection to the NNTP server

func (*BackendConn) GetArticle ΒΆ

func (c *BackendConn) GetArticle(messageID *string, bulkmode bool) (*models.Article, error)

GetArticle retrieves a complete article from the server

func (*BackendConn) GetBody ΒΆ

func (c *BackendConn) GetBody(messageID string) ([]byte, error)

GetBody retrieves only the body of an article

func (*BackendConn) GetHead ΒΆ

func (c *BackendConn) GetHead(messageID string) (*models.Article, error)

GetHead retrieves only the headers of an article

func (*BackendConn) ListGroup ΒΆ

func (c *BackendConn) ListGroup(groupName string, start, end int64) ([]int64, error)

ListGroup retrieves article numbers for a specific group

func (*BackendConn) ListGroups ΒΆ

func (c *BackendConn) ListGroups() ([]GroupInfo, error)

ListGroups retrieves a list of available newsgroups

func (*BackendConn) ListGroupsLimited ΒΆ

func (c *BackendConn) ListGroupsLimited(maxGroups int) ([]GroupInfo, error)

ListGroupsLimited retrieves a limited number of newsgroups for testing

func (*BackendConn) SelectGroup ΒΆ

func (c *BackendConn) SelectGroup(groupName string) (*GroupInfo, int, error)

SelectGroup selects a newsgroup for operation

func (*BackendConn) StatArticle ΒΆ

func (c *BackendConn) StatArticle(messageID string) (bool, error)

StatArticle checks if an article exists on the server

func (*BackendConn) UpdateLastUsed ΒΆ

func (c *BackendConn) UpdateLastUsed()

UpdateLastUsed updates the last used timestamp

func (*BackendConn) XHdr ΒΆ

func (c *BackendConn) XHdr(groupName, field string, start, end int64) ([]HeaderLine, error)

XHdr retrieves specific header field for a range of articles Automatically limits to max 1000 articles to prevent SQLite overload

func (*BackendConn) XHdrStreamed ΒΆ

func (c *BackendConn) XHdrStreamed(groupName, field string, start, end int64, resultChan chan<- *HeaderLine) error

XHdrStreamed performs XHDR command and streams results line by line through a channel

func (*BackendConn) XOver ΒΆ

func (c *BackendConn) XOver(groupName string, start, end int64, enforceLimit bool) ([]OverviewLine, error)

XOver retrieves overview data for a range of articles This is essential for efficiently building newsgroup databases enforceLimit controls whether to limit to max 1000 articles to prevent SQLite overload

type ClientConnection ΒΆ

type ClientConnection struct {
	// contains filtered or unexported fields
}

ClientConnection represents a client connection to the NNTP server

func NewClientConnection ΒΆ

func NewClientConnection(conn net.Conn, server *NNTPServer, isTLS bool) *ClientConnection

NewClientConnection creates a new client connection

func (*ClientConnection) Close ΒΆ

func (c *ClientConnection) Close()

func (*ClientConnection) Handle ΒΆ

func (c *ClientConnection) Handle() error

Handle processes the client connection

func (*ClientConnection) RemoteAddr ΒΆ

func (c *ClientConnection) RemoteAddr() net.Addr

RemoteAddr returns the remote address of the connection

func (*ClientConnection) UpdateDeadlines ΒΆ

func (c *ClientConnection) UpdateDeadlines()

type GroupInfo ΒΆ

type GroupInfo struct {
	Name      string
	Count     int64
	First     int64
	Last      int64
	PostingOK bool
}

GroupInfo represents newsgroup information

type HeaderLine ΒΆ

type HeaderLine struct {
	ArticleNum int64
	Value      string
}

HeaderLine represents a line from XHDR command

type Local430 ΒΆ

type Local430 struct {
	// contains filtered or unexported fields
}

Local430 is a simple in-memory cache for 430 responses

func (*Local430) Add ΒΆ

func (lc *Local430) Add(msgIdItem *history.MessageIdItem) bool

Add adds a message ID to the cache with the current time

func (*Local430) Check ΒΆ

func (lc *Local430) Check(msgIdItem *history.MessageIdItem) bool

Check checks if a message ID is in the cache

func (*Local430) Cleanup ΒΆ

func (lc *Local430) Cleanup()

Cleanup removes expired entries from the cache

func (*Local430) CronLocal430 ΒΆ

func (lc *Local430) CronLocal430()

NewLocal430 creates a new Local430 cache

type NNTPServer ΒΆ

type NNTPServer struct {
	Config      *config.ServerConfig
	DB          *database.Database
	Listener    net.Listener
	TLSListener net.Listener
	AuthManager *AuthManager
	Stats       *ServerStats
	Processor   ArticleProcessor // For handling incoming articles (POST/IHAVE/TAKETHIS)
	// contains filtered or unexported fields
}

NNTPServer represents the main NNTP server

func NewNNTPServer ΒΆ

func NewNNTPServer(db *database.Database, cfg *config.ServerConfig, mainWG *sync.WaitGroup, processor ArticleProcessor) (*NNTPServer, error)

NewNNTPServer creates a new NNTP server instance

func (*NNTPServer) IsRunning ΒΆ

func (s *NNTPServer) IsRunning() bool

IsRunning returns whether the server is currently running

func (*NNTPServer) Start ΒΆ

func (s *NNTPServer) Start() error

Start starts the NNTP server on the configured ports

func (*NNTPServer) Stop ΒΆ

func (s *NNTPServer) Stop() error

Stop gracefully shuts down the NNTP server

type OverviewLine ΒΆ

type OverviewLine struct {
	ArticleNum int64
	Subject    string
	From       string
	Date       string
	MessageID  string
	References string
	Bytes      int64
	Lines      int64
}

OverviewLine represents a line from XOVER command

type PatternMatchResult ΒΆ

type PatternMatchResult struct {
	Matched     bool   // Whether any pattern matched
	Action      string // "send", "exclude", "reject"
	Pattern     string // The specific pattern that matched
	Explanation string // Human-readable explanation
}

PatternMatchResult represents the result of pattern matching

func MatchArticleForPeer ΒΆ

func MatchArticleForPeer(newsgroups []string, sendPatterns, excludePatterns, rejectPatterns []string) PatternMatchResult

MatchArticleForPeer determines if an article should be sent to a peer Takes into account all newsgroups in the article (crossposting)

func MatchNewsgroupPatterns ΒΆ

func MatchNewsgroupPatterns(newsgroup string, sendPatterns, excludePatterns, rejectPatterns []string) PatternMatchResult

MatchNewsgroupPatterns evaluates newsgroup patterns according to INN2 rules Returns the final decision and which pattern caused it

type Peer ΒΆ

type Peer struct {
	// Basic identification
	Enabled      bool   `json:"enabled" yaml:"enabled"`
	Hostname     string `json:"hostname" yaml:"hostname"`           // FQDN/RDNS for network connections (used as map key)
	PathHostname string `json:"path_hostname" yaml:"path_hostname"` // Hostname used in Path headers (can differ from network hostname)
	DisplayName  string `json:"display_name" yaml:"display_name"`   // Human-readable name
	Description  string `json:"description" yaml:"description"`     // Optional description

	// Network configuration
	Port        int    `json:"port" yaml:"port"`                 // Default: 119
	IPv4Address string `json:"ipv4_address" yaml:"ipv4_address"` // Static IPv4 for outgoing
	IPv6Address string `json:"ipv6_address" yaml:"ipv6_address"` // Static IPv6 for outgoing
	IPv4CIDR    string `json:"ipv4_cidr" yaml:"ipv4_cidr"`       // CIDR for incoming validation
	IPv6CIDR    string `json:"ipv6_cidr" yaml:"ipv6_cidr"`       // CIDR for incoming validation

	// Connection limits and performance
	MaxIncomingConns int   `json:"max_incoming_conns" yaml:"max_incoming_conns"` // Limit incoming from this peer
	MaxOutgoingConns int   `json:"max_outgoing_conns" yaml:"max_outgoing_conns"` // Limit outgoing to this peer
	SpeedLimitKBps   int64 `json:"speed_limit_kbps" yaml:"speed_limit_kbps"`     // Speed limit per connection

	// Authentication
	LocalUsername  string `json:"local_username" yaml:"local_username"`   // Username this peer must use to auth
	LocalPassword  string `json:"local_password" yaml:"local_password"`   // Password this peer must use to auth
	RemoteUsername string `json:"remote_username" yaml:"remote_username"` // Username we use when connecting
	RemotePassword string `json:"remote_password" yaml:"remote_password"` // Password we use when connecting

	// TLS/SSL configuration
	UseSSL      bool `json:"use_ssl" yaml:"use_ssl"`           // Connect using SSL/TLS
	SSLInsecure bool `json:"ssl_insecure" yaml:"ssl_insecure"` // Allow self-signed certs
	RequireSSL  bool `json:"require_ssl" yaml:"require_ssl"`   // Require SSL for incoming

	// Feed configuration (similar to INN2 newsfeeds patterns)
	SendPatterns    []string `json:"send_patterns" yaml:"send_patterns"`       // Newsgroup patterns to send
	AcceptPatterns  []string `json:"accept_patterns" yaml:"accept_patterns"`   // Newsgroup patterns to accept
	ExcludePatterns []string `json:"exclude_patterns" yaml:"exclude_patterns"` // !patterns - exclude specific groups from send
	RejectPatterns  []string `json:"reject_patterns" yaml:"reject_patterns"`   // @patterns - reject entire article if any group matches

	// Feed behavior
	ReadOnlyAccess bool   `json:"read_only_access" yaml:"read_only_access"` // Peer can only read, not post
	Streaming      bool   `json:"streaming" yaml:"streaming"`               // Use streaming mode (CHECK/TAKETHIS)
	MaxArticleSize int64  `json:"max_article_size" yaml:"max_article_size"` // Maximum article size
	FeedMode       string `json:"feed_mode" yaml:"feed_mode"`               // "realtime", "batch", "pull"

	// Statistics and monitoring
	ArticlesSent     int64     `json:"articles_sent" yaml:"-"`
	ArticlesReceived int64     `json:"articles_received" yaml:"-"`
	BytesSent        int64     `json:"bytes_sent" yaml:"-"`
	BytesReceived    int64     `json:"bytes_received" yaml:"-"`
	LastConnected    time.Time `json:"last_connected" yaml:"-"`
	LastError        string    `json:"last_error" yaml:"-"`
	ErrorCount       int64     `json:"error_count" yaml:"-"`

	// Internal state
	ActiveIncoming   int       `json:"-" yaml:"-"` // Current incoming connections
	ActiveOutgoing   int       `json:"-" yaml:"-"` // Current outgoing connections
	LastConfigReload time.Time `json:"-" yaml:"-"` // When config was last reloaded
}

Peer represents a single NNTP peer configuration

type PeeringConfig ΒΆ

type PeeringConfig struct {
	// Server identification
	Hostname string `json:"hostname" yaml:"hostname"`

	// Connection management
	AcceptAllGroups  bool          `json:"accept_all_groups" yaml:"accept_all_groups"`
	AcceptMaxGroups  int           `json:"accept_max_groups" yaml:"accept_max_groups"`
	ConnectionDelay  time.Duration `json:"connection_delay" yaml:"connection_delay"`
	ReloadInterval   time.Duration `json:"reload_interval" yaml:"reload_interval"`
	MaxOutgoingFeeds int           `json:"max_outgoing_feeds" yaml:"max_outgoing_feeds"`
	MaxIncomingFeeds int           `json:"max_incoming_feeds" yaml:"max_incoming_feeds"`

	// Authentication settings
	AuthRequired      bool `json:"auth_required" yaml:"auth_required"`
	MinPasswordLength int  `json:"min_password_length" yaml:"min_password_length"`
	ListRequiresAuth  bool `json:"list_requires_auth" yaml:"list_requires_auth"`

	// DNS and network settings
	DNSQueryLimit      int           `json:"dns_query_limit" yaml:"dns_query_limit"`
	DNSQueryTimeout    time.Duration `json:"dns_query_timeout" yaml:"dns_query_timeout"`
	ForceConnectionACL bool          `json:"force_connection_acl" yaml:"force_connection_acl"`
}

PeeringConfig holds NNTP peering configuration Extends our existing config system rather than replacing it

func DefaultPeeringConfig ΒΆ

func DefaultPeeringConfig() *PeeringConfig

DefaultPeeringConfig returns sensible defaults for peering

type PeeringManager ΒΆ

type PeeringManager struct {
	// contains filtered or unexported fields
}

PeeringManager handles NNTP peer configuration and management This integrates with our existing config and database systems

func NewPeeringManager ΒΆ

func NewPeeringManager(mainConfig *config.MainConfig, dbConfig *database.DBConfig) *PeeringManager

NewPeeringManager creates a new peering manager

func (*PeeringManager) AddPeer ΒΆ

func (pm *PeeringManager) AddPeer(peer *Peer) error

AddPeer adds a new peer to the configuration

func (*PeeringManager) ApplyDefaultBinaryExclusions ΒΆ

func (pm *PeeringManager) ApplyDefaultBinaryExclusions(peer *Peer)

ApplyDefaultBinaryExclusions applies the default binary exclusion patterns to a peer

func (*PeeringManager) ApplyDefaultExclusions ΒΆ

func (pm *PeeringManager) ApplyDefaultExclusions(peer *Peer)

ApplyDefaultExclusions applies both default exclusion patterns and binary rejection patterns to a peer

func (*PeeringManager) CheckConnectionACL ΒΆ

func (pm *PeeringManager) CheckConnectionACL(conn net.Conn) (*Peer, bool)

CheckConnectionACL validates an incoming connection against peer ACLs

func (*PeeringManager) Close ΒΆ

func (pm *PeeringManager) Close() error

Close gracefully shuts down the peering manager

func (*PeeringManager) CreateDefaultPeer ΒΆ

func (pm *PeeringManager) CreateDefaultPeer(hostname, pathHostname string) *Peer

CreateDefaultPeer creates a peer with sensible defaults including default exclusions and binary rejections

func (*PeeringManager) GetAllPeers ΒΆ

func (pm *PeeringManager) GetAllPeers() []Peer

GetAllPeers returns a copy of all peers

func (*PeeringManager) GetPeer ΒΆ

func (pm *PeeringManager) GetPeer(hostname string) (*Peer, bool)

GetPeer retrieves a peer by hostname

func (*PeeringManager) GetStats ΒΆ

func (pm *PeeringManager) GetStats() PeeringStats

GetStats returns current peering statistics

func (*PeeringManager) LoadConfiguration ΒΆ

func (pm *PeeringManager) LoadConfiguration() error

LoadConfiguration loads peering configuration from database or file

func (*PeeringManager) ValidatePeerConfig ΒΆ

func (pm *PeeringManager) ValidatePeerConfig(peer *Peer) error

ValidatePeerConfig validates a peer configuration

type PeeringStats ΒΆ

type PeeringStats struct {
	TotalPeers         int
	EnabledPeers       int
	ActiveConnections  int
	TotalArticlesSent  int64
	TotalArticlesRecvd int64
	TotalBytesSent     int64
	TotalBytesRecvd    int64
	LastReload         time.Time
}

PeeringStats holds statistics for the peering system

type Pool ΒΆ

type Pool struct {
	Backend *BackendConfig
	// contains filtered or unexported fields
}

Pool manages a pool of NNTP client connections

func NewPool ΒΆ

func NewPool(cfg *BackendConfig) *Pool

NewPool creates a new connection pool

func (*Pool) Cleanup ΒΆ

func (p *Pool) Cleanup()

Cleanup periodically cleans up expired connections

func (*Pool) CloseConn ΒΆ

func (p *Pool) CloseConn(client *BackendConn, lock bool) error

Closes a specific connection

func (*Pool) ClosePool ΒΆ

func (p *Pool) ClosePool() error

Close closes all connections in the pool

func (*Pool) Get ΒΆ

func (p *Pool) Get() (*BackendConn, error)

Get retrieves a connection from the pool or creates a new one

func (*Pool) GetArticle ΒΆ

func (p *Pool) GetArticle(messageID *string, bulkmode bool) (*models.Article, error)

func (*Pool) Put ΒΆ

func (p *Pool) Put(client *BackendConn) error

Put returns a connection to the pool

func (*Pool) SelectGroup ΒΆ

func (p *Pool) SelectGroup(group string) (*GroupInfo, error)

func (*Pool) StartCleanupWorker ΒΆ

func (p *Pool) StartCleanupWorker(interval time.Duration)

StartCleanupWorker starts a goroutine that periodically cleans up expired connections

func (*Pool) Stats ΒΆ

func (p *Pool) Stats() PoolStats

Stats returns pool statistics

func (*Pool) XHdr ΒΆ

func (p *Pool) XHdr(group string, header string, start, end int64) ([]HeaderLine, error)

func (*Pool) XHdrStreamed ΒΆ

func (p *Pool) XHdrStreamed(group string, header string, start, end int64, resultChan chan<- *HeaderLine) error

XHdrStreamed performs XHDR command and streams results through a channel The channel will be closed when all results are sent or an error occurs NOTE: This function takes ownership of the connection and will return it to the pool when done

func (*Pool) XOver ΒΆ

func (p *Pool) XOver(group string, start, end int64, enforceLimit bool) ([]OverviewLine, error)

type PoolStats ΒΆ

type PoolStats struct {
	MaxConnections    int
	ActiveConnections int
	IdleConnections   int
	TotalCreated      int64
	TotalClosed       int64
	Closed            bool
}

PoolStats contains pool statistics

type ServerStats ΒΆ

type ServerStats struct {
	// contains filtered or unexported fields
}

ServerStats tracks NNTP server statistics

func NewServerStats ΒΆ

func NewServerStats() *ServerStats

NewServerStats creates a new server statistics tracker

func (*ServerStats) AuthFailure ΒΆ

func (s *ServerStats) AuthFailure()

AuthFailure increments the failed authentication counter

func (*ServerStats) AuthSuccess ΒΆ

func (s *ServerStats) AuthSuccess()

AuthSuccess increments the successful authentication counter

func (*ServerStats) CommandExecuted ΒΆ

func (s *ServerStats) CommandExecuted(command string)

CommandExecuted increments the counter for a specific command

func (*ServerStats) ConnectionEnded ΒΆ

func (s *ServerStats) ConnectionEnded()

ConnectionEnded decrements the active connection counter

func (*ServerStats) ConnectionStarted ΒΆ

func (s *ServerStats) ConnectionStarted()

ConnectionStarted increments the connection counters

func (*ServerStats) GetActiveConnections ΒΆ

func (s *ServerStats) GetActiveConnections() int

GetActiveConnections returns the current number of active connections

func (*ServerStats) GetAllCommandCounts ΒΆ

func (s *ServerStats) GetAllCommandCounts() map[string]int64

GetAllCommandCounts returns a copy of all command counts

func (*ServerStats) GetAuthStats ΒΆ

func (s *ServerStats) GetAuthStats() (successes, failures int64)

GetAuthStats returns authentication statistics

func (*ServerStats) GetCommandCount ΒΆ

func (s *ServerStats) GetCommandCount(command string) int64

GetCommandCount returns the execution count for a specific command

func (*ServerStats) GetTotalConnections ΒΆ

func (s *ServerStats) GetTotalConnections() int64

GetTotalConnections returns the total number of connections since startup

func (*ServerStats) GetUptime ΒΆ

func (s *ServerStats) GetUptime() time.Duration

GetUptime returns how long the server has been running

func (*ServerStats) Reset ΒΆ

func (s *ServerStats) Reset()

Reset resets all statistics (except start time)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL