---

🧭 Overview

Porter is a DuckDB-backed Arrow server with two transport protocols:

• Flight SQL — gRPC-based Arrow Flight SQL
• WebSocket — HTTP-based Arrow streaming

SQL goes in. Arrow streams out. Everything else is detail.

Both transports share the same execution engine, ensuring identical query semantics.

---

⚡ Key Characteristics

• Streaming-first execution model (Arrow RecordBatch streams)
• Dual transport support: Flight SQL + WebSocket
• Shared execution engine for semantic parity
• Native DuckDB execution via ADBC
• Full prepared statement lifecycle with parameter binding
• TTL-based handle management with background GC

---

🏗️ Architecture

           +-------------------+
           |   Flight Client   |  <-- ADBC / Flight SQL
           +-------------------+
                     |
               gRPC / Flight
                     |
           +-------------------+
           |   Porter Server   |
           |-------------------|
           | Shared Engine     |  <-- BuildStream()
           +-------------------+
                     |
           +-------------------+
           |     DuckDB        |
           |   (via ADBC)     |
           +-------------------+
                     |
           +-------------------+
           | Arrow RecordBatches|
           +-------------------+

The server is intentionally thin: routing, lifecycle, and streaming glue only. DuckDB does the heavy lifting.

---

🚀 Getting Started

You have three ways to run Porter:

• Docker (fastest path)
• go install (clean local toolchain)
• Build from source (full control)

---

🐳 Option 1 — Run with Docker

docker build -t porter .
docker run -p 32010:32010 -p 8080:8080 porter --ws

Run with a persistent database:

docker run -p 32010:32010 -p 8080:8080 -v $(pwd)/data:/data porter --db /data/porter.duckdb --ws

Defaults:

• Flight SQL: 0.0.0.0:32010
• WebSocket: 0.0.0.0:8080 (when --ws enabled)
• Database: in-memory (:memory:)

---

⚙️ Option 2 — Install via go install

1. Install Porter

go install github.com/TFMV/porter/cmd/porter@latest

This installs porter into your $GOBIN.

2. Install ADBC CLI (dbc)

curl -LsSf https://dbc.columnar.tech/install.sh | sh

3. Install DuckDB ADBC driver

dbc install duckdb

Verify installation:

dbc list

You should see duckdb listed.

---

🛠 Option 3 — Build from Source

1. Clone

git clone https://github.com/TFMV/porter.git
cd porter

2. Install DuckDB ADBC driver

./install_duckdb.sh

3. Run

go run ./cmd/porter serve

---

💻 CLI Usage

porter --help

Quick Start

porter              # Start Flight SQL server on :32010
porter serve        # Same as above

With WebSocket

porter --ws                        # Flight SQL + WebSocket
porter serve --ws                   # Same as above
porter serve --ws --ws-port 9090   # Custom WebSocket port

Full Flags

Flag	Description	Default
--db	DuckDB file path	:memory:
--port	Flight SQL port	32010
--ws	Enable WebSocket	false
--ws-port	WebSocket port	8080

Execute a query

porter query "SELECT 1 AS value"

REPL

porter repl

Load Parquet

porter load data.parquet

Inspect schema

porter schema table_name

Environment variables

• PORTER_DB
• PORTER_PORT
• PORTER_WS
• PORTER_WS_PORT

---

🌐 Wire Contract

Flight SQL

Operation	Behavior
SQL Query	Raw SQL → FlightInfo → DoGet stream
Prepared Statements	Handle-based execution with binding
Schema Introspection	Lightweight probe execution
ExecuteUpdate	DDL/DML via DoPutCommandStatementUpdate

WebSocket

Send JSON query request:

{"query": "SELECT * FROM table"}

Receive:

1. Schema message: {"type": "schema", "fields": ["col1", "col2"]}
2. Binary IPC frames containing Arrow RecordBatches

---

🌊 Streaming Core

Both transports use the same execution primitive:

BuildStream(ctx, sql, params) (*arrow.Schema, <-chan StreamChunk, error)

DuckDB → Arrow RecordReader → Channel → StreamChunk

Backpressure is enforced naturally via the channel boundary.

---

🛣️ Roadmap

• Streaming Flight SQL execution
• WebSocket transport
• Shared execution engine
• Prepared statements
• TTL-based lifecycle
• Background GC
• Session context
• Improved schema probing
• Benchmark suite

---

🤝 Contributing

If you've ever looked at a data system and thought:

"Why is this so complicated?"

You're in the right place.

Build it smaller. Make it clearer. Keep it moving.