gigapi-querier

GigAPI Query Engine (go)

GigAPI Go provides a SQL interface to query time-series using GigAPI Catalog Metadata and DuckDB

[!WARNING]
GigAPI is an open beta developed in public. Bugs and changes should be expected. Use at your own risk.

Quick Start

Docker

Run gigapi-querier using Docker making sure the proper data folder or S3 bucket is provided

gigapi-querier:
  image: ghcr.io/gigapi/gigapi-querier:latest
  container_name: gigapi-querier
  hostname: gigapi-querier
  volumes:
    - ./data:/data
  ports:
    - "8080:8080"
  environment:
    - DATA_DIR=/data
    - PORT=8080

Build

# Build from source
go build -o gigapi *.go

# Start the server
PORT=8080 DATA_DIR=./data ./gigapi

Configuration

• PORT: Server port (default: 8080)
• DATA_DIR: Path to data directory (default: ./data)

API Endpoints

Query Data

$ curl -X POST "http://localhost:9999/query?db=mydb" \
  -H "Content-Type: application/json"  \
{"query": "SELECT time, location, temperature FROM weather WHERE time >= '2025-04-01T00:00:00'"}

Series can be used with or without time ranges, ie for counting, calculating averages, etc.

$ curl -X POST "http://localhost:9999/query?db=mydb" \
  -H "Content-Type: application/json"  \
  -d '{"query": "SELECT count(*), avg(temperature) FROM weather"}'

{"results":[{"avg(temperature)":87.025,"count_star()":"40"}]}

The GigAPI Querier can also be used in CLI mode to execute an individual query

$ ./gigapi --query "SELECT count(*), avg(temperature) FROM weather" --db mydb

{"results":[{"avg(temperature)":87.025,"count_star()":"40"}]}

A quick and dirty query user-interface is also provided for testing

Data Structure

/data
  /mydb
    /weather
      /date=2025-04-10
        /hour=14
          *.parquet
          metadata.json

Query Processing Logic

1. Parse SQL query to extract measurement name and time range
2. Find relevant parquet files using metadata
3. Use DuckDB to execute optimized queries against selected files
4. Post-process results to handle BigInt timestamps

Notes for Developers

• File paths in metadata.json may contain absolute paths; the system handles both absolute and relative paths
• Time fields are converted from nanosecond BigInt to ISO strings
• Add ?debug=true to query requests for detailed troubleshooting information

License

Gigapipe is released under the GNU Affero General Public License v3.0 ©️ HEPVEST BV, All Rights Reserved.