distributed_lock

distributed_lock

Distributed locks in golang - when you have multiple golang processes (on one computer or many across networks) and you need to guarantee - no matter what - that your code is executing all by itself. Comes with a built in watchdog system for lock expiration.

The Basics

Import the package without an alias:

import "github.com/hlfshell/distributed_lock"

Assuming you have a function that you want to run without risking multiple executions:

// Create a lock for the context of "reports:daily".
// It'll expire within a minute if not refreshed by the lock.
// By default, the renewal timer is  expiration / 3, 20 seconds here.
// It'll time out if we don't acquire the lock within 10 seconds.
lock, err := distributed_lock.NewLock("reports:daily",
    distributed_lock.WithExpiration(time.Minute),
    distributed_lock.WithAcquireTimeout(10*time.Second),
)
if err != nil {
    return err
}

// Wait up to ten seconds for ownership. The watchdog starts on success.
if err := lock.Lock(); err != nil {
    return fmt.Errorf("lock daily report: %w", err)
}
defer lock.Release()

// If we've hit this point, we have the lock - do your thing!
return generateReport()

The watchdog refreshes the lock automatically while your function runs. If the process dies, other workers can acquire the lock, assuming the prior lock died.

A lock has three methods:

Options and defaults

Options belong on NewLock, and apply each time that object calls Lock(). Managers can also supply defaults for the locks they create.

An acquisition timeout returns context.DeadlineExceeded; WithNoWait returns ErrLocked when another object owns the name. Backend failures return errors. The expiration is refreshed automatically, so it does not limit total execution time while renewal succeeds. Choose an expiration comfortably above backend latency.

Unexpected expiration

You can react without polling Locked() by setting a callback at construction:

lock, err := distributed_lock.NewLock("reports:daily",
    distributed_lock.WithOnExpiration(func(err error) {
        // Signal your application to stop the protected work.
        // errors.Is(err, distributed_lock.ErrLost) is true.
    }),
)

The callback runs once per unexpected ownership loss, asynchronously outside internal locks. It does not run on normal release or manager closure. It must return and must not panic. Renewal failures are retried within the last confirmed validity window; late responses never restore lost ownership.

Lock types

All types use the same construction options, watchdog, three lock methods, and expiration callback. The snippets below belong inside a function returning error, with a caller-provided ctx. Construct the lock once, call Lock() to begin protected work, and defer Release().

In-memory locks

Use this when only goroutines in one process need to coordinate. A memory manager creates named locks sharing one backend. Managers from separate NewMemory calls are independent:

manager, err := distributed_lock.NewMemory()
if err != nil { return err }
defer manager.Close(context.Background())

lock, err := manager.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

To share one lock namespace between separately constructed managers, give both the same Memory instance:

storage := &distributed_lock.Memory{}
first, err := distributed_lock.New(storage)
if err != nil { return err }
defer first.Close(context.Background())
second, err := distributed_lock.New(storage)
if err != nil { return err }
defer second.Close(context.Background())

lock, err := first.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

contender, err := second.NewLock("reports:daily", distributed_lock.WithNoWait())
if err != nil { return err }
// err is distributed_lock.ErrLocked because first still owns this name.
err = contender.Lock()

Singleton locks

Use package-level construction when you do not want to pass a manager around:

lock, err := distributed_lock.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

Each NewLock returns a separate, initially unlocked object. By default the objects share one in-memory backend within the process. The singleton itself does not coordinate separate processes. Configure it once, before first use, to select a manager with custom defaults or a database backend:

manager, err := distributed_lock.NewMemory(distributed_lock.WithExpiration(2*time.Minute))
if err != nil { return err }
defer manager.Close(context.Background())
if err := distributed_lock.SetDefault(manager); err != nil { return err }

// Anywhere else in this process:
lock, err := distributed_lock.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

To coordinate separate processes, configure a manager from one of the database examples below instead. Default() returns the configured manager. A second SetDefault call, or one after first use, returns an error. Keep the configured manager alive for the application's lifetime.

Database locks

Each backend is a separate Go module inside this repository. Import only the backend you need; the root package has no third-party dependencies. SQL backends use a dedicated lock table, with configurable table and column names. Database connections stay caller-owned. Managers hold no SQL transaction during your work.

PostgreSQL

Use PostgreSQL to coordinate processes across machines through an existing PostgreSQL database. Open uses pgx through database/sql and accepts a URL or DSN. Import github.com/hlfshell/distributed_lock/postgres alongside the root package:

db, err := postgres.Open(ctx, "postgres://user:password@localhost:5432/app?sslmode=disable")
if err != nil { return err }
defer db.Close()

store, err := postgres.New(db, postgres.Config{Table: "job_locks"})
if err != nil { return err }
if err := store.Migrate(ctx); err != nil { return err }
manager, err := distributed_lock.New(store)
if err != nil { return err }
defer manager.Close(context.Background())

lock, err := manager.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

The connection string above targets local development; configure the connection's TLS settings for your deployment. PostgreSQL migrations serialize concurrent table creation. Select a schema through the connection's search path.

MySQL

Use MySQL for the same cross-machine coordination when your application already uses MySQL. Tables use InnoDB. Open accepts a go-sql-driver/mysql DSN. Import github.com/hlfshell/distributed_lock/mysql:

db, err := mysql.Open(ctx, "user:password@tcp(localhost:3306)/app?timeout=5s")
if err != nil { return err }
defer db.Close()

store, err := mysql.New(db, mysql.Config{
    Table: "job_locks",
    KeyColumn: "resource",
    OwnerColumn: "holder",
    ExpiryColumn: "deadline",
})
if err != nil { return err }
if err := store.Migrate(ctx); err != nil { return err }
manager, err := distributed_lock.New(store)
if err != nil { return err }
defer manager.Close(context.Background())

lock, err := manager.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

All clients must use the same table mapping. The column customization shown here also works with PostgreSQL and SQLite.

SQLite

Use SQLite for persistent coordination between processes on the same machine without running a database server. It uses the pure-Go modernc.org/sqlite driver. Import github.com/hlfshell/distributed_lock/sqlite:

db, err := sqlite.Open(ctx,
    "file:/absolute/path/locks.db?_pragma=busy_timeout(5000)&_pragma=journal_mode(WAL)")
if err != nil { return err }
defer db.Close()

store, err := sqlite.New(db, sqlite.Config{})
if err != nil { return err }
if err := store.Migrate(ctx); err != nil { return err }
manager, err := distributed_lock.New(store)
if err != nil { return err }
defer manager.Close(context.Background())

lock, err := manager.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

Use the same absolute file path and table mapping in every process. Configure the busy timeout on every connection. SQLite coordinates through its local file. Use a local filesystem for SQLite, not a network filesystem.

Redis

Use Redis for cross-machine coordination through expiring keys. It requires no migrations. Import the backend and its client package:

import (
    "github.com/hlfshell/distributed_lock/redis"
    goredis "github.com/redis/go-redis/v9"
)

client, err := redis.Open(ctx, &goredis.Options{Addr: "localhost:6379"})
if err != nil { return err }
defer client.Close()

store, err := redis.New(client, redis.Config{Prefix: "myapp:locks:"})
if err != nil { return err }
manager, err := distributed_lock.New(store)
if err != nil { return err }
defer manager.Close(context.Background())

lock, err := manager.NewLock("reports:daily")
if err != nil { return err }
if err := lock.Lock(); err != nil { return err }
defer lock.Release()

The default prefix is distributed_lock:. There are no migrations. Acquisition uses atomic SET NX PX; Lua scripts compare the owner before renewing or deleting. Use a namespace that cannot be evicted or flushed. This is not Redlock and does not guarantee safety across failover that loses acknowledged writes.

redis.Open enables context timeouts and disables command retries. When supplying an existing client to redis.New, configure it to obey operation contexts and use bounded network timeouts. Automatic failover to storage missing a live lock breaks the backend contract even if the client reconnects successfully.

SQL configuration and migrations

Database locks use their own table. The default is distributed_locks, with columns named lock_key, owner_token, and expires_at. Change these with Table, KeyColumn, OwnerColumn, and ExpiryColumn. All processes sharing a lock need to use the same table and column names.

Names can contain letters, numbers, and underscores, up to 63 characters. Start each name with a letter or underscore. For PostgreSQL, set the schema through your connection's search path.

Call Migrate(ctx) during setup to create the table. It's safe to call more than once, including from multiple processes at the same time. Creating a lock manager doesn't run migrations for you. To get the SQL and run it yourself, use MigrationSQL().

Migrations only create the table. They don't change an existing table's columns or provide a way to drop it. Expiration uses the database's clock, stored as microseconds since the Unix epoch. Transactions finish before your function runs.

License

MIT.