README
¶
PeeringDB Plus
A high-performance, globally distributed, read-only mirror of PeeringDB data. Syncs all PeeringDB objects via full re-fetch (hourly or on-demand), stores them in SQLite on LiteFS for edge-local reads on Fly.io, and exposes the data through five API surfaces.
API Surfaces
| Surface | Path | Description |
|---|---|---|
| Web UI | /ui/ |
templ + htmx + Tailwind CSS (search, detail pages, ASN comparison) |
| GraphQL | /graphql |
gqlgen via entgql, interactive playground |
| REST | /rest/v1/ |
entrest, OpenAPI-compliant |
| PeeringDB Compat | /api/ |
Drop-in replacement for PeeringDB API |
| ConnectRPC/gRPC | /peeringdb.v1.*/ |
Get/List/Stream RPCs for all 13 types |
ConnectRPC / gRPC API
All 13 PeeringDB entity types are available via ConnectRPC with Get, List, and Stream RPCs. The server supports Connect, gRPC, and gRPC-Web protocols on the same endpoints.
Unary RPCs
Each entity type has Get (by ID) and List (paginated with filters) RPCs.
# Get a single network by ID
buf curl --protocol grpc --http2-prior-knowledge \
http://localhost:8080/peeringdb.v1.NetworkService/GetNetwork \
-d '{"id": 42}'
# List networks with filters and pagination
buf curl --protocol grpc --http2-prior-knowledge \
http://localhost:8080/peeringdb.v1.NetworkService/ListNetworks \
-d '{"page_size": 10, "status": "ok"}'
Streaming RPCs
All 13 entity types support server-streaming RPCs for bulk data retrieval. Streaming RPCs deliver records one message at a time, eliminating the need for manual pagination.
Available RPCs:
| Service | RPC | Streams |
|---|---|---|
| CampusService | StreamCampuses | Campus |
| CarrierService | StreamCarriers | Carrier |
| CarrierFacilityService | StreamCarrierFacilities | CarrierFacility |
| FacilityService | StreamFacilities | Facility |
| InternetExchangeService | StreamInternetExchanges | InternetExchange |
| IxFacilityService | StreamIxFacilities | IxFacility |
| IxLanService | StreamIxLans | IxLan |
| IxPrefixService | StreamIxPrefixes | IxPrefix |
| NetworkService | StreamNetworks | Network |
| NetworkFacilityService | StreamNetworkFacilities | NetworkFacility |
| NetworkIxLanService | StreamNetworkIxLans | NetworkIxLan |
| OrganizationService | StreamOrganizations | Organization |
| PocService | StreamPocs | Poc |
Usage with buf curl:
# Stream all networks (proto format)
buf curl --protocol grpc --http2-prior-knowledge \
http://localhost:8080/peeringdb.v1.NetworkService/StreamNetworks \
-d '{}'
# Stream networks filtered by ASN (JSON format)
buf curl --protocol grpc --http2-prior-knowledge \
http://localhost:8080/peeringdb.v1.NetworkService/StreamNetworks \
-d '{"asn": 15169}'
Format negotiation:
ConnectRPC supports multiple formats on the same endpoint:
| Content-Type | Format | Use case |
|---|---|---|
application/proto |
Protocol Buffers | Smallest wire size, fastest parsing |
application/json |
JSON | Human-readable, debugging |
application/grpc |
gRPC (proto) | Standard gRPC clients (grpcurl, etc.) |
application/grpc+json |
gRPC (JSON) | gRPC clients wanting JSON payloads |
Clients set the format via the Content-Type request header. ConnectRPC
handles serialization automatically.
Response headers:
| Header | Value | Description |
|---|---|---|
grpc-total-count |
integer | Approximate total records matching the filter. Available before the first message. May differ from actual streamed count due to concurrent updates. |
Filters:
Each streaming RPC accepts the same optional filter fields as its List
counterpart (minus pagination fields). Multiple filters combine with AND
logic. Omitted fields impose no constraint.
Cancellation:
Clients can cancel a stream at any time by closing the connection. The server terminates the query loop within one batch cycle (~500 rows).
Timeouts:
Streams have a server-side timeout (default 60 seconds, configurable via
PDBPLUS_STREAM_TIMEOUT environment variable). The timeout covers the
entire stream duration, not individual messages.
Development
Prerequisites
- Go 1.26+
All code generation tools (buf, templ, gqlgen) are declared as Go tool
dependencies in go.mod and require no separate installation.
Build
go build ./...
Code Generation
go generate ./...
This runs the full pipeline in dependency order: ent codegen (schemas, GraphQL, REST, protobuf), buf generate (proto Go types), and templ generate (HTML templates).
Test
go test -race ./...
Lint
go vet ./...
golangci-lint run
govulncheck ./...
Configuration
| Environment Variable | Default | Description |
|---|---|---|
PDBPLUS_STREAM_TIMEOUT |
60s |
Maximum duration for streaming RPCs |
PDBPLUS_PEERINGDB_API_KEY |
(none) | Optional PeeringDB API key for higher sync rate limits |
Technology
- Go 1.26 with entgo ORM
- SQLite via modernc.org/sqlite (CGo-free)
- LiteFS for edge replication on Fly.io
- ConnectRPC for gRPC/Connect/gRPC-Web on standard net/http
- OpenTelemetry for tracing, metrics, and logs
- templ + htmx + Tailwind CSS for the web UI
- gqlgen via entgql for GraphQL
- entrest for OpenAPI-compliant REST
- Chainguard base images for minimal container footprint
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
pdb-schema-extract
command
pdb-schema-extract parses PeeringDB Django serializer and model Python source to produce an intermediate JSON schema representation.
|
pdb-schema-extract parses PeeringDB Django serializer and model Python source to produce an intermediate JSON schema representation. |
|
pdb-schema-generate
command
pdb-schema-generate reads an intermediate JSON schema produced by pdb-schema-extract and generates entgo schema Go files.
|
pdb-schema-generate reads an intermediate JSON schema produced by pdb-schema-extract and generates entgo schema Go files. |
|
pdbcompat-check
command
Command pdbcompat-check fetches responses from the PeeringDB API and compares their structure against local golden files to detect drift.
|
Command pdbcompat-check fetches responses from the PeeringDB API and compares their structure against local golden files to detect drift. |
|
peeringdb-plus
command
Package main is the entry point for the peeringdb-plus application.
|
Package main is the entry point for the peeringdb-plus application. |
|
schema
Package schema defines the entgo schema types for PeeringDB objects.
|
Package schema defines the entgo schema types for PeeringDB objects. |
|
gen
|
|
|
Package graph provides the GraphQL resolver layer for the PeeringDB Plus API.
|
Package graph provides the GraphQL resolver layer for the PeeringDB Plus API. |
|
model
Package model defines custom GraphQL model types that are not generated by ent.
|
Package model defines custom GraphQL model types that are not generated by ent. |
|
internal
|
|
|
config
Package config loads application configuration from environment variables.
|
Package config loads application configuration from environment variables. |
|
conformance
Package conformance provides structural comparison of JSON API responses for validating PeeringDB compatibility layer output against the real PeeringDB API.
|
Package conformance provides structural comparison of JSON API responses for validating PeeringDB compatibility layer output against the real PeeringDB API. |
|
database
Package database provides SQLite database setup for the ent ORM client.
|
Package database provides SQLite database setup for the ent ORM client. |
|
graphql
Package graphql provides the HTTP handler factory for the PeeringDB Plus GraphQL API.
|
Package graphql provides the HTTP handler factory for the PeeringDB Plus GraphQL API. |
|
grpcserver
Package grpcserver provides ConnectRPC service handlers for the PeeringDB gRPC API.
|
Package grpcserver provides ConnectRPC service handlers for the PeeringDB gRPC API. |
|
health
Package health provides HTTP handlers for liveness and readiness checks.
|
Package health provides HTTP handlers for liveness and readiness checks. |
|
httperr
Package httperr provides RFC 9457 Problem Details for HTTP API error responses.
|
Package httperr provides RFC 9457 Problem Details for HTTP API error responses. |
|
litefs
Package litefs provides utilities for detecting the role of the current node in a LiteFS cluster (primary vs replica).
|
Package litefs provides utilities for detecting the role of the current node in a LiteFS cluster (primary vs replica). |
|
middleware
Package middleware provides HTTP middleware for the PeeringDB Plus server.
|
Package middleware provides HTTP middleware for the PeeringDB Plus server. |
|
otel
Package otel initializes the OpenTelemetry trace, metric, and log pipelines.
|
Package otel initializes the OpenTelemetry trace, metric, and log pipelines. |
|
pdbcompat
Package pdbcompat provides a PeeringDB-compatible REST API layer that translates Django-style query parameters to ent predicates and serializes ent entities to PeeringDB's exact JSON response format.
|
Package pdbcompat provides a PeeringDB-compatible REST API layer that translates Django-style query parameters to ent predicates and serializes ent entities to PeeringDB's exact JSON response format. |
|
peeringdb
Package peeringdb provides a client for the PeeringDB API with rate limiting, pagination, retry logic, and response types for all 13 PeeringDB object types.
|
Package peeringdb provides a client for the PeeringDB API with rate limiting, pagination, retry logic, and response types for all 13 PeeringDB object types. |
|
sync
Package sync orchestrates data synchronization from PeeringDB into the local SQLite database using the ent ORM.
|
Package sync orchestrates data synchronization from PeeringDB into the local SQLite database using the ent ORM. |
|
testutil
Package testutil provides shared test helpers for ent client setup.
|
Package testutil provides shared test helpers for ent client setup. |
|
testutil/seed
Package seed provides deterministic test data seeding for PeeringDB entity types.
|
Package seed provides deterministic test data seeding for PeeringDB entity types. |
|
web/templates
templ: version: v0.3.1001
|
templ: version: v0.3.1001 |
|
web/termrender
Package termrender provides terminal client detection and ANSI text rendering for serving styled text responses to CLI clients like curl, wget, and HTTPie.
|
Package termrender provides terminal client detection and ANSI text rendering for serving styled text responses to CLI clients like curl, wget, and HTTPie. |
|
Package schema contains the intermediate PeeringDB JSON schema and generate directives for the schema extraction pipeline.
|
Package schema contains the intermediate PeeringDB JSON schema and generate directives for the schema extraction pipeline. |
Click to show internal directories.
Click to hide internal directories.