baremaps-exporter

baremaps-exporter

baremaps-exporter is a golang drop-in replacement for the baremaps export command from Apache Baremaps (incubator). This exporter can be up to 10-15x faster at exporting large tilesets.

Written in golang, baremaps-exporter exports vector tiles from a PostGIS database. It can export either as .mvt mapbox vector tile files or as .mbtiles archives.

As input, the exporter requires a tiles.json file generated by Apache Baremaps. These tiles.json files include the queries for each tile layer and zoom level, which baremaps-exporter uses to generate the tiles.

The tiles.json file can be extracted from baremaps when the server is running in dev mode. It is typically at http://localhost:9000/tiles.json.

baremaps-exporter requires a database source name (DSN) connection string. This is typically of the format postgresql://localhost:5432/baremaps?&user=baremaps&password=baremaps, or wherever your database is hosted.

The exporter will automatically detect if the output location ends in .mbtiles and switch to mbtiles output format.

Install

Go must be installed, version 1.25 or later.

Then:

go install github.com/flightaware/baremaps-exporter/v2/cmd/baremaps-exporter

Now you're ready to go.

Usage

Run the application with:

baremaps-exporter --help

All of the options:

export baremaps-compatible tilesets from a postgis server
Usage: baremaps-exporter [--output OUTPUT] [--mbtiles] [--dsn DSN] [--init INIT] [--workers WORKERS] [--batch BATCH] [--tileversion TILEVERSION] [--zoom ZOOM] [--file FILE] TILEJSON

Positional arguments:
  TILEJSON               input tilejson file

Options:
  --output OUTPUT, -o OUTPUT
                         output file or directory
  --mbtiles              output mbtiles instead of files (automatically selected if output filename ends in '.mbtiles')
  --dsn DSN, -d DSN      database connection string (dsn) for postgis
  --init INIT            initialization SQL statement that is sent on connection/session start, for any specific optimizations
  --workers WORKERS, -w WORKERS
                         number of workers to spawn [default: 72]
  --batch BATCH, -b BATCH
                         size of the batch to query and write at once [default: 10]
  --tileversion TILEVERSION
                         version of the tileset (string) written to mbtiles metadata
  --zoom ZOOM            comma-delimited set specific zooms to export (eg: 2,4,6,8)
  --file FILE, -f FILE   a list of tiles to also generate, from a file where each line is a z/x/y tile coordinate
  --help, -h             display this help and exit

Typical usage:

baremaps-exporter -o ./tiles/ -d 'postgres://baremaps:baremaps@localhost:5432/baremaps' tiles.json

Performance Tuning

By default, the exporter disables JIT since that often slows down large numbers of queries required to export high volumes of tiles.

Additional SQL configuration commands may be useful for tuning. These commands are executed by the exporter at the start of a connection session and can be specified with --init.

A particularly useful one is: SET enable_bitmapscan = off;. It is common for the planner to think that there's a lot of features and prefer bitmap scans over going to the geospatial index first. This disables the planner from doing that, so it first filters by the features in the geospatial region (tile coordinates), and then filters the features within that geospatial region.

LICENSE

This work is licensed by FlightAware under the BSD 3-Clause License.