README
¶
s3lite
Embedded SQLite with S3-backed durability for serverless containers.
s3lite wraps litestream and a CGO-free SQLite driver so your serverless container can use SQLite as if it were a managed database: restore on startup from S3, replicate continuously to S3, and expose a standard *sql.DB.
Use case
A small Go service deployed as a serverless container with ephemeral storage. Data fits comfortably in SQLite. You want managed-database durability without operating a database.
Usage
db, err := s3lite.Open(ctx, s3lite.Config{
LocalPath: "/tmp/db.sqlite3",
RestoreFrom: "s3://my-bucket/db",
BackupTo: "s3://my-bucket/db",
S3: s3lite.S3Config{
Region: os.Getenv("AWS_REGION"),
Endpoint: os.Getenv("AWS_ENDPOINT_URL"), // for MinIO/Scaleway/etc.
AccessKeyID: os.Getenv("AWS_ACCESS_KEY_ID"),
SecretAccessKey: os.Getenv("AWS_SECRET_ACCESS_KEY"),
},
Migrations: []string{
`CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, email TEXT)`,
`CREATE INDEX IF NOT EXISTS users_email ON users(email)`,
},
})
if err != nil {
log.Fatal(err)
}
defer db.Close()
// db embeds *sql.DB — use it directly
rows, err := db.QueryContext(ctx, "SELECT id, email FROM users")
Point RestoreFrom and BackupTo at the same URL — restore what you've been backing up. On first deploy the replica is empty; Open handles that as a no-op and starts with a fresh DB.
Configuration
s3lite itself reads no environment variables. Pass S3 settings via S3Config.
Empty fields fall through to the AWS SDK's default credential chain (env vars,
~/.aws/config, IAM roles), so on EC2/ECS/Lambda you can leave credentials
blank and rely on the instance role.
Limitations
- Single writer. Run exactly one container instance.
- Restore happens on Open — cold starts pay this cost (typically sub-second for small DBs).
- Sub-second write-loss window if the container crashes before litestream syncs.
Documentation
¶
Overview ¶
Package s3lite provides S3-backed embedded SQLite for serverless containers. It wraps litestream for continuous replication and a CGO-free SQLite driver.
Example (Basic) ¶
package main
import (
"context"
"fmt"
"log"
"os"
"path/filepath"
"github.com/atmin/s3lite"
)
func main() {
ctx := context.Background()
dir, err := os.MkdirTemp("", "s3lite-example-*")
if err != nil {
log.Fatal(err)
}
defer os.RemoveAll(dir)
dbPath := filepath.Join(dir, "app.sqlite3")
replicaURL := "file://" + filepath.Join(dir, "replica")
db, err := s3lite.Open(ctx, s3lite.Config{
LocalPath: dbPath,
RestoreFrom: replicaURL,
BackupTo: replicaURL,
Migrations: []string{
`CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, email TEXT)`,
`CREATE INDEX IF NOT EXISTS users_email ON users(email)`,
},
})
if err != nil {
log.Fatal(err)
}
defer db.Close()
if _, err := db.ExecContext(ctx, `INSERT INTO users (email) VALUES (?)`, "alice@example.com"); err != nil {
log.Fatal(err)
}
var email string
if err := db.QueryRowContext(ctx, `SELECT email FROM users LIMIT 1`).Scan(&email); err != nil {
log.Fatal(err)
}
fmt.Println(email)
}
Output: alice@example.com
Example (S3) ¶
Example_s3 shows the S3 replica URL form. The caller sources S3 settings (from env, config file, secret manager, etc.) and passes them explicitly. Empty S3Config fields fall back to the AWS SDK's default credential chain.
package main
import (
"context"
"log"
"os"
"github.com/atmin/s3lite"
)
func main() {
ctx := context.Background()
db, err := s3lite.Open(ctx, s3lite.Config{
LocalPath: "/tmp/app.sqlite3",
RestoreFrom: "s3://my-bucket/db",
BackupTo: "s3://my-bucket/db",
S3: s3lite.S3Config{
Region: os.Getenv("AWS_REGION"),
Endpoint: os.Getenv("AWS_ENDPOINT_URL"),
AccessKeyID: os.Getenv("AWS_ACCESS_KEY_ID"),
SecretAccessKey: os.Getenv("AWS_SECRET_ACCESS_KEY"),
},
Migrations: []string{
`CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, email TEXT)`,
},
})
if err != nil {
log.Fatal(err)
}
defer db.Close()
_ = db // use db as a standard *sql.DB
}
Output:
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// LocalPath is the on-disk path for the SQLite file. Required.
LocalPath string
// RestoreFrom is a replica URL (file:// or s3://) to restore from on startup
// when LocalPath does not yet exist. Ignored if LocalPath already exists.
// Safe to set on first deploy — an empty replica is not an error.
RestoreFrom string
// BackupTo is a replica URL (file:// or s3://) to replicate to continuously.
// Omit to run without replication.
BackupTo string
// Migrations are SQL strings executed in order on every Open. Each must be
// idempotent (e.g. CREATE TABLE IF NOT EXISTS) — there is no version table.
Migrations []string
// S3 configures s3:// replicas. Applies to both RestoreFrom and BackupTo.
// Empty fields fall back to the AWS SDK's default credential and region chain
// (env vars, ~/.aws/config, IAM roles). Set Endpoint for MinIO or other
// S3-compatible providers — path-style addressing is enabled automatically.
S3 S3Config
// Logger receives litestream's log records. When nil, INFO is suppressed
// and WARN+ is written to stderr. Set to slog.Default() to mirror the
// host application's logging.
Logger *slog.Logger
}
Config holds the options for Open.
type DB ¶
DB wraps *sql.DB with optional litestream replication. All *sql.DB methods are available directly via embedding.
func Open ¶
Open opens or creates a SQLite database at cfg.LocalPath.
Lifecycle:
- If RestoreFrom is set and LocalPath does not exist, restore from replica.
- Start litestream replication if BackupTo is set.
- Open the SQLite connection and apply WAL pragmas.
- Run Migrations in order.
Call Close when done to flush replication and release resources.
Source Files