Documentation
¶
Index ¶
- Constants
- func BotPathPatterns() []string
- func IsBot(userAgent string) bool
- func IsBotPath(path string) bool
- func IsDataCenterIP(ip string) bool
- func IsMaliciousPath(path string) bool
- func IsReferrerSpam(referrer string) bool
- func MaliciousPathPatterns() []string
- type DefaultGeoIPResolver
- type GeoIPResolver
- type NewStoreOptions
- type StoreInterface
- type UserAgentInfo
- type VisitorInterface
- type VisitorQueryInterface
Constants ¶
const ( COLUMN_ID = "id" COLUMN_COUNTRY = "country" COLUMN_CREATED_AT = "created_at" COLUMN_SOFT_DELETED_AT = "soft_deleted_at" COLUMN_IP_ADDRESS = "ip_address" COLUMN_PATH = "path" COLUMN_UPDATED_AT = "updated_at" COLUMN_FINGERPRINT = "fingerprint" COLUMN_USER_AGENT = "user_agent" COLUMN_USER_ACCEPT_LANGUAGE = "user_accept_language" COLUMN_USER_ACCEPT_ENCODING = "user_accept_encoding" COLUMN_USER_BROWSER = "user_browser" COLUMN_USER_BROWSER_VERSION = "user_browser_version" COLUMN_USER_DEVICE = "user_device" COLUMN_USER_DEVICE_TYPE = "user_device_type" COLUMN_USER_OS = "user_os" COLUMN_USER_OS_VERSION = "user_os_version" COLUMN_USER_REFERRER = "user_referrer" )
const ( COLUMN_KEY = "key" COLUMN_VALUE = "value" SETTING_EXCLUDED_IPS = "excluded_ips" )
Settings table column names.
const ( // GeoIPEndpointDefault is the default IP geolocation endpoint (ip2c.org). GeoIPEndpointDefault = "https://ip2c.org/" // GeoIPTimeoutDefault is the default timeout for IP geolocation lookups. GeoIPTimeoutDefault = 5 * time.Second // CountryUnknown is used for IPs that cannot be resolved (e.g. localhost). CountryUnknown = "UN" // GeoIPCacheTTLDefault is the default TTL for the in-memory IP cache. GeoIPCacheTTLDefault = 24 * time.Hour )
const DEFAULT_SETTINGS_TABLE = "statsstore_settings"
Default table name for key-value settings.
const MAX_DATETIME = "9999-12-31 23:59:59"
MAX_DATETIME is a far-future datetime used as the default soft-delete sentinel.
Variables ¶
This section is empty.
Functions ¶
func BotPathPatterns ¶ added in v1.22.0
func BotPathPatterns() []string
BotPathPatterns returns the list of path substrings for bot-only files (robots.txt, ads.txt, sitemap.xml, etc.) that legitimate crawlers request but no human browser does. Consumers can use this list to drive SQL LIKE scans or similar bulk queries.
func IsBot ¶ added in v1.8.0
IsBot checks whether a user-agent string matches known bot/crawler patterns. Broad patterns (bot, crawler, spider, scraper, slurp) are matched with a word-boundary check to avoid false positives (e.g. "bot" inside a non-bot word). Specific patterns (semrush, curl, googlebot, etc.) are matched as plain case-insensitive substrings.
func IsBotPath ¶ added in v1.22.0
IsBotPath reports whether a request path targets a bot-only file (robots.txt, ads.txt, sitemap.xml, etc.) that no human browser requests. Matching is case-insensitive. The pattern is matched as a suffix of the last path segment (filename), so "/app-ads.txt" matches "ads.txt" but "/ads.txt.backup" does not.
func IsDataCenterIP ¶ added in v1.8.0
IsDataCenterIP checks whether an IP address falls within known data center CIDR ranges (AWS, GCP, Azure, DigitalOcean, Oracle Cloud).
func IsMaliciousPath ¶ added in v1.22.0
IsMaliciousPath reports whether a request path targets a universally malicious endpoint (.env, .git/, .svn/, .htpasswd, shell.php) that is never legitimate on any properly-configured site. A single hit is strong evidence of a vulnerability scanner. Stack-dependent patterns are not included — consumers should add their own.
File patterns (e.g. .env, shell.php) are matched as a suffix of the last path segment, so "/.env" matches but "/my.envfile.js" does not. Directory patterns (e.g. .git/, .svn/) are matched against individual path segments, so "/.git/config" matches but "/.github/workflows" does not.
func IsReferrerSpam ¶ added in v1.8.0
IsReferrerSpam checks whether a referrer URL host matches a known spam domain. The referrer can be a full URL or just a domain. Matching is case-insensitive.
func MaliciousPathPatterns ¶ added in v1.22.0
func MaliciousPathPatterns() []string
MaliciousPathPatterns returns the list of path substrings for endpoints that are never legitimate on any properly-configured website (e.g. .env, .git/, shell.php). Stack-dependent patterns like .php or wp-admin are excluded — consumers should add their own based on their tech stack.
Types ¶
type DefaultGeoIPResolver ¶ added in v1.11.0
type DefaultGeoIPResolver struct {
Endpoint string // default: GeoIPEndpointDefault
Timeout time.Duration // default: GeoIPTimeoutDefault
HTTPClient *http.Client // injectable for testing; if nil, a default client is used
CacheTTL time.Duration // default: GeoIPCacheTTLDefault; set to 0 to disable caching
// contains filtered or unexported fields
}
DefaultGeoIPResolver implements GeoIPResolver using the ip2c.org service. It includes an in-memory cache with TTL to avoid duplicate lookups for the same IP within the cache window.
func NewDefaultGeoIPResolver ¶ added in v1.11.0
func NewDefaultGeoIPResolver() *DefaultGeoIPResolver
NewDefaultGeoIPResolver creates a DefaultGeoIPResolver with sensible defaults.
type GeoIPResolver ¶ added in v1.11.0
type GeoIPResolver interface {
// Resolve returns an ISO2 country code (e.g. "US", "GB") for the given IP.
// If the IP cannot be resolved, it returns CountryUnknown and nil error.
// If the lookup itself fails (network error, bad response), it returns
// an empty string and the error so the caller can leave the field empty
// for a retry.
Resolve(ctx context.Context, ip string) (string, error)
}
GeoIPResolver resolves an IP address to an ISO 3166-1 alpha-2 country code. Implementations may use an external API, a local MaxMind database, or any other source. A nil/empty result with no error means the IP could not be resolved and the country should be left empty for a later retry.
type NewStoreOptions ¶
type NewStoreOptions struct {
VisitorTableName string
SettingsTableName string
DB *sql.DB
AutomigrateEnabled bool
DebugEnabled bool
BotFilterEnabled bool
ExcludedPathPrefixes []string
ExcludedIPs []string
GeoIPResolver GeoIPResolver // optional; enables VisitorEnhance for batch country enrichment
EnhanceBatchSize int // number of records per VisitorEnhance call; default 10
}
NewStoreOptions defines the options for creating a new stats store.
type StoreInterface ¶
type StoreInterface interface {
// MigrateDown drops the stats store tables
MigrateDown(ctx context.Context, tx ...*sql.Tx) error
// MigrateUp creates the stats store tables
MigrateUp(ctx context.Context, tx ...*sql.Tx) error
EnableDebug(debug bool)
GetDB() *sql.DB
SetBotFilterEnabled(enabled bool)
IsBotFilterEnabled() bool
SetExcludedPathPrefixes(prefixes []string)
GetExcludedPathPrefixes() []string
SetExcludedIPs(ips []string)
GetExcludedIPs() []string
ExcludedIPList(ctx context.Context) ([]string, error)
ExcludedIPAdd(ctx context.Context, ip string) error
ExcludedIPRemove(ctx context.Context, ip string) error
// SettingGet retrieves a setting value by key. Returns empty string and
// nil error if the key does not exist.
SettingGet(ctx context.Context, key string) (string, error)
// SettingSet stores a setting value by key, using upsert semantics.
SettingSet(ctx context.Context, key, value string) error
// SettingDelete removes a setting by key. No error if the key is absent.
SettingDelete(ctx context.Context, key string) error
// SettingHas reports whether a setting key exists.
SettingHas(ctx context.Context, key string) (bool, error)
// SettingList returns all settings as a map of key to value.
// Returns an empty map if the settings table does not exist or is empty.
SettingList(ctx context.Context) (map[string]string, error)
VisitorCount(ctx context.Context, query VisitorQueryInterface) (int64, error)
VisitorCreate(ctx context.Context, user VisitorInterface) error
VisitorDelete(ctx context.Context, user VisitorInterface) error
VisitorDeleteByID(ctx context.Context, id string) error
VisitorDeleteByIP(ctx context.Context, ip string) (int64, error)
VisitorFindByID(ctx context.Context, userID string) (VisitorInterface, error)
VisitorList(ctx context.Context, query VisitorQueryInterface) ([]VisitorInterface, error)
VisitorRegister(ctx context.Context, r *http.Request) error
VisitorSoftDelete(ctx context.Context, user VisitorInterface) error
VisitorSoftDeleteByID(ctx context.Context, id string) error
VisitorUpdate(ctx context.Context, user VisitorInterface) error
// VisitorEnhance enriches visitor records that have an empty country field.
// It parses the user agent (browser, OS, device type) and looks up the
// country via the configured GeoIPResolver. UA fields are updated even if
// the geo-IP lookup fails; the country stays empty for retry.
// Returns the number of records fully processed (country + UA).
// Call this from a background task/cron on whatever schedule suits your
// traffic (e.g. every 5 minutes).
//
// If no GeoIPResolver was configured, it returns 0 and an error.
VisitorEnhance(ctx context.Context) (int, error)
}
StoreInterface defines the interface for a stats store.
func NewStore ¶
func NewStore(opts NewStoreOptions) (StoreInterface, error)
NewStore creates a new stats store.
type UserAgentInfo ¶ added in v1.16.0
type UserAgentInfo struct {
Browser string
BrowserVersion string
Os string
OsVersion string
Device string
DeviceType string
}
UserAgentInfo holds parsed user-agent data.
func ParseUserAgent ¶ added in v1.16.0
func ParseUserAgent(ua string) UserAgentInfo
ParseUserAgent extracts browser, OS, device, and device-type from a user-agent string using the uasurfer library. Unknown values are left as empty strings.
type VisitorInterface ¶
type VisitorInterface interface {
// Methods
FingerprintCalculate() string
IsSoftDeleted() bool
GetID() string
SetID(id string) VisitorInterface
GetPath() string
SetPath(path string) VisitorInterface
GetCountry() string
SetCountry(country string) VisitorInterface
GetCreatedAt() string
GetCreatedAtCarbon() *carbon.Carbon
SetCreatedAt(createdAt string) VisitorInterface
GetSoftDeletedAt() string
GetSoftDeletedAtCarbon() *carbon.Carbon
SetSoftDeletedAt(deletedAt string) VisitorInterface
GetFingerprint() string
SetFingerprint(fingerprint string) VisitorInterface
GetIpAddress() string
SetIpAddress(ipAddress string) VisitorInterface
GetUpdatedAt() string
GetUpdatedAtCarbon() *carbon.Carbon
SetUpdatedAt(updatedAt string) VisitorInterface
GetUserAcceptLanguage() string
SetUserAcceptLanguage(userAcceptLanguage string) VisitorInterface
GetUserAcceptEncoding() string
SetUserAcceptEncoding(userAcceptEncoding string) VisitorInterface
GetUserAgent() string
SetUserAgent(userAgent string) VisitorInterface
GetUserBrowser() string
SetUserBrowser(userBrowser string) VisitorInterface
GetUserBrowserVersion() string
SetUserBrowserVersion(userBrowserVersion string) VisitorInterface
GetUserDevice() string
SetUserDevice(userDevice string) VisitorInterface
GetUserDeviceType() string
SetUserDeviceType(userDeviceType string) VisitorInterface
GetUserOs() string
SetUserOs(userOs string) VisitorInterface
GetUserOsVersion() string
SetUserOsVersion(userOsVersion string) VisitorInterface
GetUserReferrer() string
SetUserReferrer(userReferrer string) VisitorInterface
}
VisitorInterface defines the interface for a visitor record.
func NewVisitorFromExistingData ¶
func NewVisitorFromExistingData(data map[string]string) VisitorInterface
NewVisitorFromExistingData creates a new visitor from a raw column map.
type VisitorQueryInterface ¶ added in v1.1.0
type VisitorQueryInterface interface {
Validate() error
HasCountry() bool
Country() string
SetCountry(country string) VisitorQueryInterface
HasCreatedAtGte() bool
CreatedAtGte() string
SetCreatedAtGte(createdAtGte string) VisitorQueryInterface
HasCreatedAtLte() bool
CreatedAtLte() string
SetCreatedAtLte(createdAtLte string) VisitorQueryInterface
HasDeviceType() bool
DeviceType() string
SetDeviceType(deviceType string) VisitorQueryInterface
HasDistinct() bool
Distinct() string
SetDistinct(distinct string) VisitorQueryInterface
HasID() bool
ID() string
SetID(id string) VisitorQueryInterface
HasIDIn() bool
IDIn() []string
SetIDIn(idIn []string) VisitorQueryInterface
HasIPIn() bool
IPIn() []string
SetIPIn(ipIn []string) VisitorQueryInterface
HasIPNotIn() bool
IPNotIn() []string
SetIPNotIn(ipNotIn []string) VisitorQueryInterface
HasLimit() bool
Limit() int
SetLimit(limit int) VisitorQueryInterface
HasOffset() bool
Offset() int
SetOffset(offset int) VisitorQueryInterface
HasOrderBy() bool
OrderBy() string
SetOrderBy(orderBy string) VisitorQueryInterface
HasPathContains() bool
PathContains() string
SetPathContains(pathContains string) VisitorQueryInterface
HasPathExact() bool
PathExact() string
SetPathExact(pathExact string) VisitorQueryInterface
HasSortOrder() bool
SortOrder() string
SetSortOrder(sortOrder string) VisitorQueryInterface
HasSoftDeletedIncluded() bool
SoftDeletedIncluded() bool
SetSoftDeletedIncluded(withSoftDeleted bool) VisitorQueryInterface
}
VisitorQueryInterface defines the interface for visitor query operations.
func NewVisitorQuery ¶ added in v1.1.0
func NewVisitorQuery() VisitorQueryInterface
NewVisitorQuery creates a new visitor query.
func VisitorQuery ¶ added in v1.1.0
func VisitorQuery() VisitorQueryInterface
VisitorQuery is a shortcut for NewVisitorQuery.


