tigerfs

module
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 27, 2026 License: MIT

README

TigerFS

A filesystem backed by PostgreSQL, and a filesystem interface to PostgreSQL.

Every file is a real PostgreSQL row. Directories are tables. File contents are columns. Multiple agents and humans can read and write the same files concurrently with full ACID guarantees. No sync protocols. No coordination layer. The filesystem is the API.

You can use TigerFS in two ways:

  • File-first: Write markdown with frontmatter or other file types, organize into directories. Writes are atomic, and everything is auto-versioned. Any tool that works with files -- Claude Code, Cursor, grep, vim -- just works. Build lightweight apps via the filesystem: multi-agent task coordination is just mv'ing files between todo/doing/done directories.

  • Data-first: Mount any Postgres database and explore it with ls, cat, grep, and other unix tools. For large databases, chain filters into paths that push down to SQL: .by/customer_id/123/.order/created_at/.last/10/.export/json. No database client or SQL needed, and ships with agent skills.

Both modes are backed by the same transactional database. You get real transactions, true concurrent access, and a SQL escape hatch when you need it.

TigerFS mounts via FUSE on Linux and NFS on macOS, no extra dependencies needed.

Quick Start

# Install (macOS requires no dependencies; Linux needs fuse3)
curl -fsSL https://install.tigerfs.io | sh
Mode You have... You want to...
File-first A new project or workflow Store markdown or other files, and build simple apps via the file system (e.g., task queues, agent workspaces, collaborative docs).
Data-first An existing Postgres database Explore and operate on it with ls, cat, grep instead of SQL.
File-first
# Mount a database and create a markdown app
tigerfs mount postgres://localhost/mydb /mnt/db
echo "markdown,history" > /mnt/db/.build/notes

# Write a file — frontmatter becomes columns, body becomes text
cat > /mnt/db/notes/hello.md << 'EOF'
---
title: Hello World
author: alice
---
# Hello World
EOF

# Search, explore, unmount
grep -l "author: alice" /mnt/db/notes/*.md
ls /mnt/db/notes/
tigerfs unmount /mnt/db
Data-first
# Mount an existing database
tigerfs mount postgres://localhost/mydb /mnt/db

ls /mnt/db/                                    # list tables
ls /mnt/db/users/                              # list rows
cat /mnt/db/users/1.json                       # read a row
cat /mnt/db/users/.by/email/alice@co.com.json  # index lookup

# Chain filters, ordering, pagination — pushed down as one SQL query
cat /mnt/db/orders/.by/customer_id/1/.order/created_at/.last/5/.export/json

File-First: Transactional Workspace

Apps

Apps tell TigerFS how to present a table as a native file format. Write "markdown" to .build/ and the table becomes a directory of .md files with YAML frontmatter:

# Create a markdown app
echo "markdown" > /mnt/db/.build/blog

# Write a post. Frontmatter becomes columns, body becomes text
cat > /mnt/db/blog/hello-world.md << 'EOF'
---
title: Hello World
author: alice
tags: [intro]
---

# Hello World

Welcome to my blog...
EOF

# Search, edit, and manage content with standard tools
grep -l "author: alice" /mnt/db/blog/*.md

Organize files into directories. mkdir creates folders, mv moves files between them:

mkdir /mnt/db/blog/tutorials
mv /mnt/db/blog/hello-world.md /mnt/db/blog/tutorials/

See docs/markdown-app.md for column mapping, frontmatter handling, and use cases.

Version History

Any app can opt into automatic versioning. Every edit and delete is captured as a timestamped snapshot under a read-only .history/ directory.

To enable automatic versioning, write "history" to .build/ when creating the app:

# Create an app with history enabled
echo "markdown,history" > /mnt/db/.build/notes

# Browse past versions of a file
ls /mnt/db/notes/.history/hello.md/
# .id  2026-02-24T150000Z  2026-02-12T013000Z

# Read a past version
cat /mnt/db/notes/.history/hello.md/2026-02-12T013000Z

History tracks files across renames via stable row UUIDs and uses TimescaleDB hypertables for compressed storage.

See docs/history.md for cross-rename tracking, subdirectory scoping, and recovery workflows.

Use Cases

Shared agent workspace. Multiple agents and humans operate on the same knowledge base concurrently. Every edit is automatically versioned, so if one agent overwrites another's work, recover it from .history/.

# Agent A writes research findings
cat > /mnt/db/kb/auth-analysis.md << 'EOF'
---
author: agent-a
---
OAuth 2.0 is the recommended approach because...
EOF

# Agent B reads it immediately, no sync, no pull
cat /mnt/db/kb/auth-analysis.md

Multi-agent task queue. Three directories (todo/, doing/, done/) and mv is your only API. Moves are atomic database operations, so two agents can't claim the same task.

# Set up a task board
echo "markdown,history" > /mnt/db/.build/tasks
mkdir /mnt/db/tasks/todo /mnt/db/tasks/doing /mnt/db/tasks/done

# Agent claims a task by moving it to doing
mv /mnt/db/tasks/todo/fix-auth-bug.md /mnt/db/tasks/doing/fix-auth-bug.md

# Marks it complete
mv /mnt/db/tasks/doing/fix-auth-bug.md /mnt/db/tasks/done/fix-auth-bug.md

# See what everyone is working on
ls /mnt/db/tasks/doing/
grep "author:" /mnt/db/tasks/doing/*.md

Collaborative docs. A human writes a draft, an agent reviews and edits it, another agent summarizes it. All in the same directory, all visible immediately, no pull/push/merge. History shows who changed what and when.

# Human writes a draft
cat > /mnt/db/docs/proposal.md << 'EOF'
---
title: Q2 Proposal
status: draft
---
We should invest in...
EOF

# Agent reads, edits, and updates the status
cat /mnt/db/docs/proposal.md
cat > /mnt/db/docs/proposal.md << 'EOF'
---
title: Q2 Proposal
status: reviewed
reviewer: agent-b
---
We should invest in... (with agent edits)
EOF

# Human sees changes instantly. Browse the full edit trail
ls /mnt/db/docs/.history/proposal.md/
cat /mnt/db/docs/.history/proposal.md/2026-02-25T100000Z  # see previous version

Data-First: Database as Filesystem

Mount any Postgres database and explore it with ls, cat, grep. Every path resolves to optimized SQL pushed down to the database.

Explore an unfamiliar database. Point an agent at a mounted database and it understands the schema immediately using ls and cat. No SQL, no database client, no connection strings to pass around.

Quick data fixes. Update a customer's email, toggle a feature flag, delete a test record. One shell command instead of opening a SQL client, remembering the table schema, and writing a WHERE clause.

Export and analyze. Chain filters, ordering, and pagination into a single path, then pipe the result into jq, awk, or export as CSV for a spreadsheet.

Explore
ls /mnt/db/                                      # List tables
ls /mnt/db/users/                                # List rows (by primary key)
cat /mnt/db/users/123.json                       # Row as JSON
cat /mnt/db/users/123/email.txt                  # Single column
cat /mnt/db/users/.by/email/foo@example.com.json # Index lookup
Modify
echo 'new@example.com' > /mnt/db/users/123/email.txt            # Update column
echo '{"email":"a@b.com","name":"A"}' > /mnt/db/users/ 123.json # Update via JSON (PATCH)
mkdir /mnt/db/users/456                                         # Create row
rm -r /mnt/db/users/456/                                        # Delete row
Pipeline Queries

Chain filters, ordering, and pagination in a single path. The database executes it as one query:

cat /mnt/db/orders/.by/customer_id/123/.order/created_at/.last/10/.export/json

# Select specific columns from a filtered query
cat /mnt/db/orders/.filter/status/shipped/.columns/id,total,created_at/.export/csv

Pipeline segments can be chained in any order. Available segments: .by/ (indexed filter), .filter/ (any column), .order/ (sort), .columns/col1,... (projection), .first/N/, .last/N/, .sample/N/ (pagination), and .export/csv|json|tsv (output format).

Ingest

Bulk-load data from CSV, JSON, or YAML. The write mode is part of the path: .append/ adds rows, .sync/ upserts by primary key, .overwrite/ replaces the table.

cat data.csv > /mnt/db/orders/.import/.append/csv
Schema Management

Create, modify, and delete tables through a staging pattern:

mkdir /mnt/db/.create/orders && echo "CREATE TABLE orders (...)" > /mnt/db/.create/orders/sql
touch /mnt/db/.create/orders/.commit

See docs/native-tables.md for the full reference: row formats, index navigation, pipeline query chaining, schema management workflows, and configuration.

Why TigerFS

If you're building on files (file-first)
  • vs. local files: Instead of a single-writer assumption, TigerFS supports real concurrent access with isolation guarantees.

  • vs. git: Instead of asynchronous collaboration and merges, TigerFS provides immediate visibility with automatic version history.

  • vs. object storage (S3): Instead of blobs, you get structured rows, ACID transactions, and query pushdown.

If you're querying data (data-first)
  • vs. database clients / psql: No SQL to learn. Every agent already speaks files.

  • vs. ORMs and APIs: No schemas to define, no SDK to install. Mount and go.

  • vs. using a database directly: Instead of clients and schemas, you use files. Every tool and every agent already understands the interface.

The result is simple: you delete coordination code from your application.

Cloud Backends

TigerFS works with any PostgreSQL database. Just pass a connection string. It also integrates with Tiger Cloud and Ghost through their CLIs for credential-free mounting. Use a prefix to specify the backend:

# Mount any Postgres database
tigerfs mount postgres://user:pass@host/mydb /mnt/db

# Or mount cloud services by ID
tigerfs mount tiger:abcde12345 /mnt/db
tigerfs mount ghost:fghij67890 /mnt/db

TigerFS calls the backend CLI to retrieve credentials, so there are no passwords in your config. Authenticate once with tiger auth login or ghost login.

Set a default backend to skip the prefix:

# In ~/.config/tigerfs/config.yaml: default_backend: tiger
tigerfs mount abcde12345 /mnt/db    # uses tiger: implicitly
Create and Fork
# Create a new cloud database (auto-mounts)
tigerfs create tiger:my-db
tigerfs create tiger:my-db /mnt/data   # custom mount path
tigerfs create ghost:my-db --no-mount  # create without mounting

# Fork (clone) for safe experimentation
tigerfs fork /mnt/db my-experiment
tigerfs fork tiger:abcde12345 my-experiment

# Inspect a mount
tigerfs info /mnt/db
tigerfs info --json /mnt/db           # JSON output for scripting

Architecture

┌──────────────┐     ┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│  Unix Tools  │────▶│  Filesystem  │────▶│   TigerFS    │────▶│  PostgreSQL  │
│  ls, cat,    │     │   Backend    │     │   Daemon     │     │   Database   │
│  echo, rm    │◀────│  (FUSE/NFS)  │◀────│              │◀────│              │
└──────────────┘     └──────────────┘     └──────────────┘     └──────────────┘

TigerFS replaces "application-level coordination" with database transactions. The filesystem becomes the API.

TigerFS maps filesystem paths to database queries:

  Filesystem                       Database
  ──────────                       ────────
  /mnt/db/                     →   schemas
  /mnt/db/public/              →   tables
  /mnt/db/public/users/        →   rows (by PK)
  /mnt/db/public/users/123/    →   columns as files

FUSE on Linux, NFS on macOS. No external dependencies on either platform.

Design Principles

  • Keep the interface familiar. If you can ls, you can explore a database.
  • Make concurrency safe. Multiple writers without corruption or conflicts.
  • Push logic down. Every path resolves to optimized SQL.
  • Preserve history. Every change is recoverable.
  • Remove coordination code. The database handles it.

Try the Demo

cd scripts/demo
./demo.sh start     # auto-detects platform (--docker or --mac)
./demo.sh shell     # explore: ls, cat users/1.json, etc.
./demo.sh stop

Configuration

Config file: ~/.config/tigerfs/config.yaml. Run tigerfs config show to see all options and their current values. All options support environment variables with TIGERFS_ prefix. See docs/spec.md for the full reference.

Documentation

Guide Description
docs/markdown-app.md Markdown app: column mapping, frontmatter, directories
docs/history.md Version history: snapshots, cross-rename tracking, recovery
docs/native-tables.md Native table access: row formats, indexes, pipeline queries, schema management
docs/quickstart.md Guided scenarios with sample data

Development

git clone https://github.com/timescale/tigerfs.git
cd tigerfs
go build -o bin/tigerfs ./cmd/tigerfs
go test ./...

For development guidelines, architecture details, and the full specification, see CLAUDE.md and docs/spec.md.

Project Status

TigerFS is early, but the core idea is stable: transactional, concurrent files as the foundation for human-agent collaboration.

v0.5.0. Performance and observability — dramatically fewer SQL queries, flexible logging, and column projection.

Highlights:

  • Markdown apps with YAML frontmatter, directory hierarchies, and automatic version history
  • Cloud backends: mount, create, and fork Tiger Cloud and Ghost databases by service ID
  • Pipeline queries with full database pushdown (.by/, .filter/, .order/, .columns/, chained pagination, .export/)
  • DDL staging for tables, indexes, views, and schemas (.create/, .modify/, .delete/)
  • Full CRUD with multiple formats (TSV, CSV, JSON, YAML), index navigation, and PATCH semantics
  • Binary distribution via GoReleaser with install script (curl -fsSL https://install.tigerfs.io | sh)
  • Multi-tier stat caching and query reduction for fast operations over remote databases

Planned:

  • Tables without primary keys (read-only via ctid)
  • TimescaleDB hypertables (time-based navigation)
  • Windows support

Contributing

Contributions are welcome! Please see the development guidelines in CLAUDE.md.

Support

Directories

Path Synopsis
cmd
tigerfs command
internal
tigerfs/backend
Package backend provides a unified interface for cloud database backends (Tiger Cloud, Ghost) used by TigerFS to provision, inspect, and connect to database services.
Package backend provides a unified interface for cloud database backends (Tiger Cloud, Ghost) used by TigerFS to provision, inspect, and connect to database services.
tigerfs/cmd
Package cmd provides CLI commands for TigerFS.
Package cmd provides CLI commands for TigerFS.
tigerfs/fs
Package fs provides the shared core filesystem logic for TigerFS.
Package fs provides the shared core filesystem logic for TigerFS.
tigerfs/fs/synth
Package synth implements synthesized app formats for TigerFS.
Package synth implements synthesized app formats for TigerFS.
tigerfs/fuse
Package fuse provides FUSE filesystem nodes for TigerFS.
Package fuse provides FUSE filesystem nodes for TigerFS.
tigerfs/mount
Package mount provides mount state tracking for TigerFS filesystem instances.
Package mount provides mount state tracking for TigerFS filesystem instances.
tigerfs/util
Package util provides utility functions for TigerFS.
Package util provides utility functions for TigerFS.
test

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL