peeringdb

package
v1.23.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 10, 2026 License: BSD-3-Clause Imports: 19 Imported by: 0

Documentation

Overview

Package peeringdb provides a client for the PeeringDB API with rate limiting, pagination, retry logic, and response types for all 13 PeeringDB object types.

Index

Constants

View Source 
const (
	TypeOrg        = "org"
	TypeNet        = "net"
	TypeFac        = "fac"
	TypeIX         = "ix"
	TypePoc        = "poc"
	TypeIXLan      = "ixlan"
	TypeIXPfx      = "ixpfx"
	TypeNetIXLan   = "netixlan"
	TypeNetFac     = "netfac"
	TypeIXFac      = "ixfac"
	TypeCarrier    = "carrier"
	TypeCarrierFac = "carrierfac"
	TypeCampus     = "campus"
)

Object type constants for PeeringDB API paths.

View Source 
const FetchByIDsBatchSize = 100

FetchByIDsBatchSize caps the number of IDs concatenated into a single ?id__in= query value so the assembled URL stays well under typical 8 KiB request-line limits. PeeringDB does not document an explicit id__in cardinality limit, but 100 IDs at ~7 characters each (decimal + comma) keeps the query string under 1 KiB with comfortable headroom for path + other params.

Exported so the FK-backfill cap accounting (in internal/sync) can compute how many underlying HTTP requests a given FetchByIDs(ids) call will issue, without re-deriving the chunk size.

Variables

This section is empty.

Functions

func IsWAFBlocked added in v1.18.2 

func IsWAFBlocked(err error) bool

IsWAFBlocked reports whether err wraps the transport's WAF-block sentinel. Exposed for callers (sync worker) that want to distinguish WAF errors from generic 403 auth failures.

Types

type Campus 

type Campus struct {
	ID          int           `json:"id"`
	OrgID       int           `json:"org_id"`
	OrgName     string        `json:"org_name"`
	Name        string        `json:"name"`
	NameLong    *string       `json:"name_long"`
	Aka         *string       `json:"aka"`
	Website     string        `json:"website"`
	SocialMedia []SocialMedia `json:"social_media"`
	Notes       string        `json:"notes"`
	Country     string        `json:"country"`
	City        string        `json:"city"`
	Zipcode     string        `json:"zipcode"`
	State       string        `json:"state"`
	Created     time.Time     `json:"created"`
	Updated     time.Time     `json:"updated"`
	Status      string        `json:"status"`
}

Campus represents a PeeringDB campus.

type Carrier 

type Carrier struct {
	ID          int           `json:"id"`
	OrgID       int           `json:"org_id"`
	OrgName     string        `json:"org_name"`
	Name        string        `json:"name"`
	Aka         string        `json:"aka"`
	NameLong    string        `json:"name_long"`
	Website     string        `json:"website"`
	SocialMedia []SocialMedia `json:"social_media"`
	Notes       string        `json:"notes"`
	FacCount    int           `json:"fac_count"`
	Created     time.Time     `json:"created"`
	Updated     time.Time     `json:"updated"`
	Status      string        `json:"status"`
}

Carrier represents a PeeringDB carrier.

type CarrierFacility 

type CarrierFacility struct {
	ID        int       `json:"id"`
	CarrierID int       `json:"carrier_id"`
	FacID     int       `json:"fac_id"`
	Name      string    `json:"name"`
	Created   time.Time `json:"created"`
	Updated   time.Time `json:"updated"`
	Status    string    `json:"status"`
}

CarrierFacility represents a PeeringDB carrier-facility association (carrierfac).

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client fetches data from the PeeringDB API with rate limiting, pagination, and retry logic.

func NewClient 

func NewClient(baseURL string, logger *slog.Logger, opts ...ClientOption) *Client

NewClient creates a PeeringDB API client with the given base URL and logger. By default, the client enforces a 2 req/sec rate limit and a 30-second HTTP timeout. Use WithAPIKey to enable authenticated access with a higher 60 req/min rate limit; use WithRPS to override the unauthenticated default.

Internal requests bypass otelhttp instrumentation to avoid span bloat during bulk syncs. Metrics and retries are still recorded via events on the parent stream span. The transport wrapper handles per-request limiter wait, bounded 429 retry, and WAF detection on 403 — see internal/peeringdb/transport.go.

func (*Client) FetchAll 

func (c *Client) FetchAll(ctx context.Context, objectType string, opts ...FetchOption) (FetchResult, error)

FetchAll pages through all objects of the given type, collecting each element as a json.RawMessage in a FetchResult. It is a thin wrapper over StreamAll — new callers should prefer StreamAll directly to avoid the allocation of a full []json.RawMessage slice. FetchAll exists for the pdbcompat-check CLI and the conformance test, which consume the full result set as a batch.

The handler clones each raw message because json.Decoder reuses its internal buffer across Decode calls — without the clone, every slice entry would alias the same memory.

func (*Client) FetchByIDs added in v1.18.4 

func (c *Client) FetchByIDs(ctx context.Context, objectType string, ids []int) ([]json.RawMessage, error)

FetchByIDs issues one or more ?since=1&id__in=<csv> requests against the named object type and returns the concatenated raw rows in fetch order (chunk-1 rows then chunk-2 rows then chunk-3 rows). Each chunk of up to FetchByIDsBatchSize IDs becomes ONE HTTP request through the rate-limited transport (consuming ONE limiter token regardless of chunk size). 250 IDs => 3 sequential requests through the limiter.

The FK-backfill dataloader path uses this to collapse N per-row HTTP requests (the old worst-case during truncate-and-resync recovery) into ⌈N/100⌉ batched requests, well under upstream's API_THROTTLE_REPEATED_REQUEST cap.

Empty / nil ids returns (nil, nil) without issuing any HTTP request. On any chunk error, returns (nil, err) — partial results from prior chunks are NOT returned (callers treat this as a transport failure and fall back to drop-on-miss for the affected IDs).

Order within a chunk follows upstream's response order; the function does NOT sort or deduplicate ids — callers are responsible for any upstream-side ordering they need.

func (*Client) FetchRaw added in v1.18.2 

func (c *Client) FetchRaw(ctx context.Context, objectType string, params url.Values) ([]json.RawMessage, error)

FetchRaw issues a single non-paginated request with the given query params and returns each top-level data element as a json.RawMessage clone. Used by the sync FK-backfill path (fetch one row at a time via ?since=1&id__in=N to recover missing parents from upstream before declaring an orphan).

The request goes through doWithRetry (so it observes the rate limiter, the WAF detector, and the bounded 429 ladder). The decoder clones each raw element because json.Decoder reuses its internal buffer between Decode calls — without the clone, every slice entry would alias the same memory.

Empty data → returns an empty slice (not nil) so callers can range over the result without a separate len-check.

func (*Client) FetchRawPage added in v1.14.0 

func (c *Client) FetchRawPage(ctx context.Context, objectType string, page int) ([]byte, error)

FetchRawPage fetches a single page of objects for the given type and returns the raw response bytes verbatim. Unlike FetchAll (which pages internally and parses), this is intended for callers that write byte-for-byte fixtures — see internal/visbaseline.

page is 1-based. Page 1 -> skip=0, page 2 -> skip=250, etc.

FetchRawPage reuses the client's rate limiter, API key, and 429 short-circuit from doWithRetry. On 429, callers get a *RateLimitError back; they MUST sleep for RetryAfter before re-calling. Do not retry inside this method — the short-circuit at client.go:312-327 is intentional to avoid burning quota on failed retries.

func (*Client) SetRateLimit 

func (c *Client) SetRateLimit(limiter *rate.Limiter)

SetRateLimit overrides the default rate limiter. Intended for testing. Also rewires the transport wrapper's limiter pointer so the change takes effect on the very next request — without this, the transport would keep using the limiter captured at NewClient time and tests like TestFetchAllPagination (which set rate.Inf for fast iteration) would still see the default 2 RPS limit applied per request.

func (*Client) SetRetryBaseDelay 

func (c *Client) SetRetryBaseDelay(d time.Duration)

SetRetryBaseDelay overrides the default retry base delay. Intended for testing.

func (*Client) StreamAll added in v1.14.0 

func (c *Client) StreamAll(ctx context.Context, objectType string, handler func(raw json.RawMessage) error, opts ...FetchOption) (FetchMeta, error)

StreamAll fetches objects of the given type and invokes handler for each element of the "data" array as a json.RawMessage. Both {"meta":{...},"data":[...]} and {"data":[...],"meta":{...}} key orderings are supported — the decoder token-walks the outer object and dispatches based on key name.

The handler is invoked synchronously from within the HTTP response body read loop; the HTTP response is held open for the duration of the stream. The raw message passed to the handler is only valid until the handler returns — if the caller needs to retain it, it must copy (bytes.Clone or json.Unmarshal).

If the handler returns an error, StreamAll aborts, closes the response body, and returns the error wrapped as "handler %s element %d: %w".

For callers that need the full []json.RawMessage slice (e.g. the pdbcompat-check CLI), FetchAll wraps StreamAll with a simple append-all handler. Do NOT rewrite FetchAll's callers to decode into the tx — the batch-materialisation strategy in internal/sync/worker.go requires the batch materialised before the tx opens.

type ClientOption added in v1.3.0 

type ClientOption func(*Client)

ClientOption configures optional Client behavior.

func WithAPIKey added in v1.3.0 

func WithAPIKey(key string) ClientOption

WithAPIKey sets the PeeringDB API key for authenticated requests. When set, requests include the Authorization header and the rate limiter increases to 60 req/min — overriding any WithRPS value. (The authenticated quota is fixed by upstream regardless of any operator preference; making this an override is the simplest way to preserve "auth → 1/sec" without re-deriving it from a float.)

func WithRPS added in v1.18.2 

func WithRPS(rps float64) ClientOption

WithRPS sets the unauthenticated sustained requests-per-second cap. Replaces the hardcoded 1/3s with a float knob driven by PDBPLUS_PEERINGDB_RPS. Burst stays at 1 — concurrent bursts against PeeringDB are not desirable. Authenticated clients (WithAPIKey) override this back to 60 req/min in NewClient — the upstream auth quota is fixed at 60/min regardless of operator preference.

Values <= 0 are silently coerced to the default (2 RPS) so a misconfig in main.go cannot accidentally produce a zero-rate limiter that blocks every request forever.

type Facility 

type Facility struct {
	ID                        int           `json:"id"`
	OrgID                     int           `json:"org_id"`
	OrgName                   string        `json:"org_name"`
	CampusID                  *int          `json:"campus_id"`
	Name                      string        `json:"name"`
	Aka                       string        `json:"aka"`
	NameLong                  string        `json:"name_long"`
	Website                   string        `json:"website"`
	SocialMedia               []SocialMedia `json:"social_media"`
	CLLI                      string        `json:"clli"`
	Rencode                   string        `json:"rencode"`
	NPANXX                    string        `json:"npanxx"`
	TechEmail                 string        `json:"tech_email"`
	TechPhone                 string        `json:"tech_phone"`
	SalesEmail                string        `json:"sales_email"`
	SalesPhone                string        `json:"sales_phone"`
	Property                  *string       `json:"property"`
	DiverseServingSubstations *bool         `json:"diverse_serving_substations"`
	AvailableVoltageServices  []string      `json:"available_voltage_services"`
	Notes                     string        `json:"notes"`
	RegionContinent           *string       `json:"region_continent"`
	StatusDashboard           *string       `json:"status_dashboard"`
	NetCount                  int           `json:"net_count"`
	IXCount                   int           `json:"ix_count"`
	CarrierCount              int           `json:"carrier_count"`
	Address1                  string        `json:"address1"`
	Address2                  string        `json:"address2"`
	City                      string        `json:"city"`
	State                     string        `json:"state"`
	Country                   string        `json:"country"`
	Zipcode                   string        `json:"zipcode"`
	Suite                     string        `json:"suite"`
	Floor                     string        `json:"floor"`
	Latitude                  *float64      `json:"latitude"`
	Longitude                 *float64      `json:"longitude"`
	Created                   time.Time     `json:"created"`
	Updated                   time.Time     `json:"updated"`
	Status                    string        `json:"status"`
}

Facility represents a PeeringDB facility (fac).

type FetchMeta added in v1.2.0 

type FetchMeta struct {
	// Generated is the earliest meta.generated epoch across all pages.
	// Zero value means no generated timestamp was found.
	Generated time.Time
}

FetchMeta contains metadata from PeeringDB API responses.

type FetchOption added in v1.2.0 

type FetchOption func(*fetchConfig)

FetchOption configures optional FetchAll behavior.

func WithSince added in v1.2.0 

func WithSince(t time.Time) FetchOption

WithSince sets the ?since= parameter for delta fetches. Only objects modified after the given time are returned.

type FetchResult added in v1.2.0 

type FetchResult struct {
	Data []json.RawMessage
	Meta FetchMeta
}

FetchResult contains the fetched data and response metadata.

type InternetExchange 

type InternetExchange struct {
	ID                     int           `json:"id"`
	OrgID                  int           `json:"org_id"`
	Name                   string        `json:"name"`
	Aka                    string        `json:"aka"`
	NameLong               string        `json:"name_long"`
	City                   string        `json:"city"`
	Country                string        `json:"country"`
	RegionContinent        string        `json:"region_continent"`
	Media                  string        `json:"media"`
	Notes                  string        `json:"notes"`
	ProtoUnicast           bool          `json:"proto_unicast"`
	ProtoMulticast         bool          `json:"proto_multicast"`
	ProtoIPv6              bool          `json:"proto_ipv6"`
	Website                string        `json:"website"`
	SocialMedia            []SocialMedia `json:"social_media"`
	URLStats               string        `json:"url_stats"`
	TechEmail              string        `json:"tech_email"`
	TechPhone              string        `json:"tech_phone"`
	PolicyEmail            string        `json:"policy_email"`
	PolicyPhone            string        `json:"policy_phone"`
	SalesEmail             string        `json:"sales_email"`
	SalesPhone             string        `json:"sales_phone"`
	NetCount               int           `json:"net_count"`
	FacCount               int           `json:"fac_count"`
	IXFNetCount            int           `json:"ixf_net_count"`
	IXFLastImport          *time.Time    `json:"ixf_last_import"`
	IXFImportRequest       *string       `json:"ixf_import_request"`
	IXFImportRequestStatus string        `json:"ixf_import_request_status"`
	ServiceLevel           string        `json:"service_level"`
	Terms                  string        `json:"terms"`
	StatusDashboard        *string       `json:"status_dashboard"`
	Created                time.Time     `json:"created"`
	Updated                time.Time     `json:"updated"`
	Status                 string        `json:"status"`
}

InternetExchange represents a PeeringDB internet exchange (ix).

type IxFacility 

type IxFacility struct {
	ID      int       `json:"id"`
	IXID    int       `json:"ix_id"`
	FacID   int       `json:"fac_id"`
	Name    string    `json:"name"`
	City    string    `json:"city"`
	Country string    `json:"country"`
	Created time.Time `json:"created"`
	Updated time.Time `json:"updated"`
	Status  string    `json:"status"`
}

IxFacility represents a PeeringDB IX-facility association (ixfac).

type IxLan 

type IxLan struct {
	ID                         int       `json:"id"`
	IXID                       int       `json:"ix_id"`
	Name                       string    `json:"name"`
	Descr                      string    `json:"descr"`
	MTU                        int       `json:"mtu"`
	Dot1QSupport               bool      `json:"dot1q_support"`
	RSASN                      *int      `json:"rs_asn"`
	ARPSponge                  *string   `json:"arp_sponge"`
	IXFIXPMemberListURLVisible string    `json:"ixf_ixp_member_list_url_visible"`
	IXFIXPMemberListURL        string    `json:"ixf_ixp_member_list_url,omitempty"`
	IXFIXPImportEnabled        bool      `json:"ixf_ixp_import_enabled"`
	Created                    time.Time `json:"created"`
	Updated                    time.Time `json:"updated"`
	Status                     string    `json:"status"`
}

IxLan represents a PeeringDB IX LAN (ixlan).

type IxPrefix 

type IxPrefix struct {
	ID       int       `json:"id"`
	IXLanID  int       `json:"ixlan_id"`
	Protocol string    `json:"protocol"`
	Prefix   string    `json:"prefix"`
	InDFZ    bool      `json:"in_dfz"`
	Created  time.Time `json:"created"`
	Updated  time.Time `json:"updated"`
	Status   string    `json:"status"`
}

IxPrefix represents a PeeringDB IX prefix (ixpfx).

The "notes" field is dropped to match upstream PeeringDB's live /api/ixpfx response, which omits "notes" from every row.

type Network 

type Network struct {
	ID                      int           `json:"id"`
	OrgID                   int           `json:"org_id"`
	Name                    string        `json:"name"`
	Aka                     string        `json:"aka"`
	NameLong                string        `json:"name_long"`
	Website                 string        `json:"website"`
	SocialMedia             []SocialMedia `json:"social_media"`
	ASN                     int           `json:"asn"`
	LookingGlass            string        `json:"looking_glass"`
	RouteServer             string        `json:"route_server"`
	IRRASSet                string        `json:"irr_as_set"`
	InfoType                string        `json:"info_type"`
	InfoTypes               []string      `json:"info_types"`
	InfoPrefixes4           *int          `json:"info_prefixes4"`
	InfoPrefixes6           *int          `json:"info_prefixes6"`
	InfoTraffic             string        `json:"info_traffic"`
	InfoRatio               string        `json:"info_ratio"`
	InfoScope               string        `json:"info_scope"`
	InfoUnicast             bool          `json:"info_unicast"`
	InfoMulticast           bool          `json:"info_multicast"`
	InfoIPv6                bool          `json:"info_ipv6"`
	InfoNeverViaRouteServer bool          `json:"info_never_via_route_servers"`
	Notes                   string        `json:"notes"`
	PolicyURL               string        `json:"policy_url"`
	PolicyGeneral           string        `json:"policy_general"`
	PolicyLocations         string        `json:"policy_locations"`
	PolicyRatio             bool          `json:"policy_ratio"`
	PolicyContracts         string        `json:"policy_contracts"`
	AllowIXPUpdate          bool          `json:"allow_ixp_update"`
	IxpUpdateExclude        []string      `json:"ixp_update_exclude"`
	StatusDashboard         *string       `json:"status_dashboard"`
	RIRStatus               *string       `json:"rir_status"`
	RIRStatusUpdated        *time.Time    `json:"rir_status_updated"`
	IXCount                 int           `json:"ix_count"`
	FacCount                int           `json:"fac_count"`
	NetIXLanUpdated         *time.Time    `json:"netixlan_updated"`
	NetFacUpdated           *time.Time    `json:"netfac_updated"`
	PocUpdated              *time.Time    `json:"poc_updated"`
	Created                 time.Time     `json:"created"`
	Updated                 time.Time     `json:"updated"`
	Status                  string        `json:"status"`
}

Network represents a PeeringDB network (net).

type NetworkFacility 

type NetworkFacility struct {
	ID       int       `json:"id"`
	NetID    int       `json:"net_id"`
	FacID    int       `json:"fac_id"`
	Name     string    `json:"name"`
	City     string    `json:"city"`
	Country  string    `json:"country"`
	LocalASN int       `json:"local_asn"`
	Created  time.Time `json:"created"`
	Updated  time.Time `json:"updated"`
	Status   string    `json:"status"`
}

NetworkFacility represents a PeeringDB network-facility association (netfac).

type NetworkIxLan 

type NetworkIxLan struct {
	ID          int       `json:"id"`
	NetID       int       `json:"net_id"`
	IXID        int       `json:"ix_id"`
	IXLanID     int       `json:"ixlan_id"`
	Name        string    `json:"name"`
	Notes       string    `json:"notes"`
	Speed       int       `json:"speed"`
	ASN         int       `json:"asn"`
	IPAddr4     *string   `json:"ipaddr4"`
	IPAddr6     *string   `json:"ipaddr6"`
	IsRSPeer    bool      `json:"is_rs_peer"`
	BFDSupport  bool      `json:"bfd_support"`
	Operational bool      `json:"operational"`
	NetSideID   *int      `json:"net_side_id"`
	IXSideID    *int      `json:"ix_side_id"`
	Created     time.Time `json:"created"`
	Updated     time.Time `json:"updated"`
	Status      string    `json:"status"`
}

NetworkIxLan represents a PeeringDB network-IX LAN association (netixlan).

type Organization 

type Organization struct {
	ID          int           `json:"id"`
	Name        string        `json:"name"`
	Aka         string        `json:"aka"`
	NameLong    string        `json:"name_long"`
	Website     string        `json:"website"`
	SocialMedia []SocialMedia `json:"social_media"`
	Notes       string        `json:"notes"`
	Address1    string        `json:"address1"`
	Address2    string        `json:"address2"`
	City        string        `json:"city"`
	State       string        `json:"state"`
	Country     string        `json:"country"`
	Zipcode     string        `json:"zipcode"`
	Suite       string        `json:"suite"`
	Floor       string        `json:"floor"`
	Latitude    *float64      `json:"latitude"`
	Longitude   *float64      `json:"longitude"`
	Created     time.Time     `json:"created"`
	Updated     time.Time     `json:"updated"`
	Status      string        `json:"status"`
}

Organization represents a PeeringDB organization (org).

type Poc 

type Poc struct {
	ID      int       `json:"id"`
	NetID   int       `json:"net_id"`
	Role    string    `json:"role"`
	Visible string    `json:"visible"`
	Name    string    `json:"name"`
	Phone   string    `json:"phone"`
	Email   string    `json:"email"`
	URL     string    `json:"url"`
	Created time.Time `json:"created"`
	Updated time.Time `json:"updated"`
	Status  string    `json:"status"`
}

Poc represents a PeeringDB network point of contact (poc).

type RateLimitError added in v1.14.0 

type RateLimitError struct {
	// URL is the request URL that was rate-limited.
	URL string
	// RetryAfter is the delay parsed from the Retry-After response header.
	// Zero if the header was absent or unparseable.
	RetryAfter time.Duration
	// Status is always http.StatusTooManyRequests for RateLimitError.
	Status int
}

RateLimitError is returned when PeeringDB responds with HTTP 429 Too Many Requests. It carries the Retry-After delay parsed from the response header so upstream callers (sync worker retry loops) can short-circuit their own backoff ladders instead of burning more of our rate-limited quota on retries that are guaranteed to 429 again.

Background: PeeringDB enforces 1 request per distinct query-string per hour for unauthenticated clients. Their 429 response includes a Retry-After header in seconds (e.g. "Retry-After: 2200" = 36m40s). The sync worker's default retry backoff ladder (30s, 2m, 8m) all fall well inside that window, so every retry within a single sync cycle is doomed — and each one consumes another slot against the hourly quota.

func (*RateLimitError) Error added in v1.14.0 

func (e *RateLimitError) Error() string

Error implements the error interface.

type SocialMedia 

type SocialMedia struct {
	Service    string `json:"service"`
	Identifier string `json:"identifier"`
}

SocialMedia represents a social media link on a PeeringDB object.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.