usid

package module
v2.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 3, 2026 License: MIT Imports: 14 Imported by: 0

README

usid

Time-ordered 64-bit IDs. Half the size of UUIDv7, fits in a bigint.

UUIDv7:  019234a5-f78b-7c3d-8a1e-3f9b2c8d4e6f  (36 chars, 16 bytes)
usid:    gb61dv03w20                            (<=13 chars, 8 bytes)

How it works

[1 sign][51 bits µs timestamp][6 bits node][6 bits sequence]

Timestamp (51 bits): Microseconds since epoch (~71 years). Time-ordered for index-friendly inserts.

Node ID (6 bits): Identifies which instance generated the ID. Each instance gets its own "lane"—collisions are impossible as long as node IDs are unique.

Sequence (6 bits): Handles multiple IDs within the same microsecond from one instance. You'll never hit this limit in practice.

Installation

go get github.com/beyondoss/usid/v2

Quick start

import "github.com/beyondoss/usid/v2"

func main() {
    usid.SetNodeID(1)  // Assign once at startup

    id := usid.New()
    fmt.Println(id)              // "gb61dv03w20"
    fmt.Println(id.Timestamp())  // 2025-12-16 12:34:56.789
}

API

// Generate
id := usid.New()

// Parse
id, err := usid.Parse("gb61dv03w20")
id := usid.FromStringOrNil("gb61dv03w20")

// Format
str := id.String()                       // uses DefaultFormat (Crockford Base32)
str := id.Format(usid.FormatCrockford)   // "gb61dv03w20"
str := id.Format(usid.FormatBase58)      // "3kTMd92jFk"
str := id.Format(usid.FormatDecimal)     // "10151254716672"
str := id.Format(usid.FormatHash)        // "93b85ee7100"
str := id.Format(usid.FormatBase64)      // "AAAJO4XucQA="

// Extract components
ts := id.Timestamp()  // time.Time
node := id.Node()     // int64
seq := id.Seq()       // int64

// Raw value
n := id.Int64()
bytes := id.Bytes()

The default format is Crockford Base32: lowercase, case-insensitive on decode, and treats I/L as 1 and O as 0 for human-friendliness.

JSON

type User struct {
    ID   usid.ID `json:"id"`
    Name string  `json:"name"`
}
// {"id":"gb61dv03w20","name":"alice"}

type Record struct {
    ID       usid.ID     `json:"id"`
    ParentID usid.NullID `json:"parent_id"`
}
// {"id":"gb61dv03w20","parent_id":null}

Customizing bit allocation

// Before any ID generation or migrations:
usid.NodeBits = 8  // 255 instances
usid.SeqBits = 4   // still plenty of headroom

// Then set node ID
usid.SetNodeID(node)

// And migrate with matching config
postgres.Migrate(ctx, db, postgres.Config{
    Epoch:    usid.Epoch,
    NodeBits: usid.NodeBits,
    SeqBits:  usid.SeqBits,
})

Node ID assignment

Unique node IDs guarantee no collisions—each instance has its own "lane" in the ID space.

Shared node IDs risk collision when two instances generate an ID in the same microsecond. Rough collision rates for two instances sharing a node (assuming uniform distribution):

IDs/sec per instance Collision rate
10 ~1 per 3 hours
100 ~1 per 2 minutes
1,000 ~1 per second

Real traffic is bursty, so these are optimistic. For N instances sharing a node, multiply by N×(N-1)/2 pairs.

If collisions are acceptable (e.g., you retry on unique constraint violation): shared node IDs are fine at low throughput.

If collisions are unacceptable: use unique node IDs.

Size your node bits to your max concurrent instances:

Max instances NodeBits
15 4
63 6 (default)
255 8

Node 0 is reserved for Postgres (see below), so app instances use 1–63.

Assignment strategies
// From database sequence (recommended)
node, _ := postgres.NextNode(ctx, db)
usid.SetNodeID(node)

// From environment
usid.SetNodeID(mustParseInt(os.Getenv("NODE_ID")))

// From Kubernetes pod ordinal
// pod-0 → node 1, pod-1 → node 2, etc.
hostname, _ := os.Hostname()
parts := strings.Split(hostname, "-")
ordinal, _ := strconv.ParseInt(parts[len(parts)-1], 10, 64)
usid.SetNodeID((ordinal % 63) + 1)

Postgres

Store as bigint:

CREATE TABLE users (
    id bigint PRIMARY KEY DEFAULT usid(),
    email text NOT NULL
);

Run migrations to install Postgres functions:

import "github.com/beyondoss/usid/v2/postgres"

postgres.Migrate(ctx, db, postgres.DefaultConfig())

This gives you:

  • usid() — generate IDs in Postgres (uses node 0)
  • usid_to_crockford(id) / crockford_to_usid(str) — Crockford Base32 encoding
  • usid_to_b58(id) / b58_to_usid(str) — Base58 encoding
  • ts_from_usid(id) — extract timestamp
  • usid_next_node() — get next node ID from sequence

Scanning works automatically:

var user User
db.QueryRow("SELECT id, name FROM users WHERE id = $1", id).Scan(&user.ID, &user.Name)
Optional domain type

For type safety in your schema, you can create a usid domain type:

postgres.Migrate(ctx, db, postgres.Config{
    Epoch:        usid.Epoch,
    NodeBits:     usid.NodeBits,
    SeqBits:      usid.SeqBits,
    CreateDomain: true,
})

Then use usid instead of bigint:

CREATE TABLE users (
    id usid PRIMARY KEY DEFAULT usid(),
    email text NOT NULL
);

The domain is an alias for bigint, so all USID functions work with it. ORMs and code generators like sqlc may need configuration to map the custom type.

Why not nanoid?

nanoid generates random IDs with no coordination required. The tradeoffs:

usid nanoid
Storage 8 bytes (bigint) 21+ bytes (string)
Index writes Sequential (fast) Random (fragmented)
Comparisons Integer String
Timestamp Extractable None
Coordination Node ID at startup None

If you need time ordering or care about database performance at scale, use usid. If you just want short random strings and don't want to think about node IDs, nanoid is simpler.

Why not UUIDv7?

UUIDv7 requires no coordination—any instance can generate IDs independently. The tradeoff is size: 16 bytes vs 8 bytes.

If you're storing millions of rows, that's real savings:

  • 47% smaller indexes
  • 27% smaller total table size
  • Faster range scans

If you only have a few thousand rows or coordination is painful, use UUIDv7.

Why not Snowflake?

Snowflake uses dedicated ID generation services that app servers call over RPC. That's the right architecture at Twitter's scale, but overkill for most systems.

usid generates in-process: no network hop, no single point of failure, no batching complexity. The tradeoff is you need to assign node IDs at startup.

Benchmarks

Apple M2:

Operation ns/op allocs
New 38.1 0
Parse 7.9 0
String 21.3 1
Postgres (10M rows, after 10M random updates)
usid UUID v4
Index size 216 MB 418 MB
Leaf fill 98.11% 72.67%
10M updates 28s 56s
Range scan 10K 7.5 ms 82.9 ms
Range scan buffers 106 8,545

UUID v4 indexes fragment over time and require periodic REINDEX to recover ~90% fill. usid stays at ~98% without maintenance.

License

MIT

Documentation

Overview

Package usid provides microsecond-precision, time-ordered unique identifiers.

USIDs are 64-bit IDs with an embedded timestamp, node ID, and sequence number. They sort chronologically, are URL-safe when encoded, and work well as database primary keys.

Basic usage:

usid.SetNodeID(1)  // Call once at startup
id := usid.New()   // Generate IDs
fmt.Println(id)    // Crockford Base32 encoded by default

The bit layout is configurable via Epoch, NodeBits, and SeqBits variables.

Index

Constants

This section is empty.

Variables

View Source
var (
	// Epoch is the custom epoch in microseconds (default: 2025-12-16).
	// IDs store time as microseconds since this epoch.
	Epoch int64 = 1765947799213000

	// NodeBits is the number of bits allocated for the node ID (default: 6, max 64 nodes).
	NodeBits uint8 = 6

	// SeqBits is the number of bits allocated for the sequence number (default: 6, max 64 per µs).
	SeqBits uint8 = 6

	// DefaultFormat is the default string encoding format for IDs.
	DefaultFormat Format = FormatCrockford
)

Configuration variables for USID generation. Modify these before generating any IDs if you need custom bit layouts.

View Source
var DefaultGenerator = NewGenerator(1)

DefaultGenerator is used by New(). Set via SetNodeID().

Functions

func Node deprecated

func Node(id int64) int64

Deprecated: Use ID.Node() instead

func Seq deprecated

func Seq(id int64) int64

Deprecated: Use ID.Seq() instead

func SetNodeID

func SetNodeID(node int64)

SetNodeID initializes the DefaultGenerator with the given node ID. Call this once at startup before using New().

func SetObfuscator

func SetObfuscator(key int64)

SetObfuscator sets the DefaultObfuscator with the given key. Call once at startup to enable obfuscation.

func Timestamp deprecated

func Timestamp(id int64) time.Time

Deprecated: Use ID.Timestamp() instead

Types

type Format

type Format string

Format specifies the string encoding format for IDs.

const (
	FormatCrockford Format = "crockford" // Crockford Base32, case-insensitive (default)
	FormatBase58    Format = "base58"    // URL-safe, compact
	FormatBase64    Format = "base64"    // Standard base64 encoding
	FormatHash      Format = "hash"      // Hexadecimal encoding
	FormatDecimal   Format = "decimal"   // Decimal integer string
)

Supported ID string formats.

type Generator

type Generator struct {
	// contains filtered or unexported fields
}

Generator produces unique IDs for a specific node. Create with NewGenerator and call Generate to produce IDs.

func NewGenerator

func NewGenerator(node int64) *Generator

NewGenerator creates a Generator for the given node ID. The node ID must be in the range [0, 2^NodeBits - 1]. Panics if node is out of range.

func (*Generator) Generate

func (g *Generator) Generate() ID

Generate produces a new unique ID. Safe for concurrent use.

type ID

type ID int64

ID is a 64-bit microsecond-precision time-ordered identifier.

var Nil ID = 0

Nil is the zero ID, representing an absent or invalid ID.

var Omni ID = math.MaxInt64

Omni is the maximum ID value (math.MaxInt64), useful as an upper bound in queries.

func FromBytes

func FromBytes(b []byte) (ID, error)

FromBytes returns an ID from an 8-byte big-endian slice.

func FromBytesOrNil

func FromBytesOrNil(b []byte) ID

FromBytesOrNil returns an ID from an 8-byte slice. Returns Nil on error.

func FromInt64

func FromInt64(n int64) ID

FromInt64 returns an ID from an int64.

func FromString

func FromString(s string) (ID, error)

FromString returns an ID parsed from the input string. Alias for Parse.

func FromStringOrNil

func FromStringOrNil(s string) ID

FromStringOrNil returns an ID parsed from the input string. Returns Nil on error.

func Must

func Must(id ID, err error) ID

Must panics if err is not nil

func New

func New() ID

New generates an ID using the DefaultGenerator. Panics if SetNodeID() hasn't been called.

func Parse

func Parse(s string) (ID, error)

Parse parses a string into an ID using DefaultFormat.

func ParseBase58

func ParseBase58(s string) (ID, error)

ParseBase58 parses a base58-encoded string into an ID.

func ParseBase64

func ParseBase64(s string) (ID, error)

ParseBase64 parses a base64-encoded string into an ID.

func ParseCrockford

func ParseCrockford(s string) (ID, error)

ParseCrockford parses a Crockford Base32-encoded string into an ID.

func ParseDecimal

func ParseDecimal(s string) (ID, error)

ParseDecimal parses a decimal string into an ID.

func ParseHash

func ParseHash(s string) (ID, error)

ParseHash parses a hex-encoded string into an ID.

func (ID) Bytes

func (id ID) Bytes() []byte

Bytes returns the ID as an 8-byte big-endian slice.

func (ID) Format

func (id ID) Format(f ...Format) string

Format returns the ID as a string in the specified format. If no format is provided, uses DefaultFormat.

func (*ID) GobDecode

func (id *ID) GobDecode(data []byte) error

GobDecode implements gob.GobDecoder.

func (ID) GobEncode

func (id ID) GobEncode() ([]byte, error)

GobEncode implements gob.GobEncoder.

func (ID) Hash

func (id ID) Hash() [8]byte

Hash returns the ID as an 8-byte big-endian array (for hex formatting).

func (ID) Int64

func (id ID) Int64() int64

Int64 returns the ID as an int64.

func (ID) IsNil

func (id ID) IsNil() bool

IsNil returns true if the ID is Nil (zero).

func (ID) MarshalBinary

func (id ID) MarshalBinary() ([]byte, error)

MarshalBinary implements encoding.BinaryMarshaler.

func (ID) MarshalJSON

func (id ID) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler

func (ID) MarshalText

func (id ID) MarshalText() ([]byte, error)

MarshalText implements encoding.TextMarshaler

func (ID) Node

func (id ID) Node() int64

Node extracts the node ID component from the ID.

func (*ID) Parse

func (id *ID) Parse(s string) error

Parse parses a string into the ID receiver.

func (*ID) Scan

func (id *ID) Scan(src interface{}) error

Scan implements sql.Scanner for database retrieval

func (ID) Seq

func (id ID) Seq() int64

Seq extracts the sequence number component from the ID.

func (ID) String

func (id ID) String() string

String returns the ID as a string using DefaultFormat.

func (ID) Timestamp

func (id ID) Timestamp() time.Time

Timestamp extracts the creation time from the ID.

func (*ID) UnmarshalBinary

func (id *ID) UnmarshalBinary(data []byte) error

UnmarshalBinary implements encoding.BinaryUnmarshaler.

func (*ID) UnmarshalJSON

func (id *ID) UnmarshalJSON(b []byte) error

UnmarshalJSON implements json.Unmarshaler

func (*ID) UnmarshalText

func (id *ID) UnmarshalText(b []byte) error

UnmarshalText implements encoding.TextUnmarshaler

func (ID) Value

func (id ID) Value() (driver.Value, error)

Value implements driver.Valuer for database storage

type NullID

type NullID struct {
	ID    ID
	Valid bool
}

NullID can be used with the standard sql package to represent an ID value that can be NULL in the database.

func (NullID) MarshalJSON

func (n NullID) MarshalJSON() ([]byte, error)

MarshalJSON marshals the NullID as null or the nested ID as a string.

func (NullID) MarshalText

func (n NullID) MarshalText() ([]byte, error)

MarshalText implements encoding.TextMarshaler.

func (*NullID) Scan

func (n *NullID) Scan(src interface{}) error

Scan implements the sql.Scanner interface.

func (*NullID) UnmarshalJSON

func (n *NullID) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals a NullID.

func (*NullID) UnmarshalText

func (n *NullID) UnmarshalText(b []byte) error

UnmarshalText implements encoding.TextUnmarshaler.

func (NullID) Value

func (n NullID) Value() (driver.Value, error)

Value implements the driver.Valuer interface.

type Obfuscator

type Obfuscator struct {
	// contains filtered or unexported fields
}

Obfuscator XORs IDs with a key to hide timestamps and sequences in external representations.

var DefaultObfuscator *Obfuscator

DefaultObfuscator, when set, obfuscates all external representations (String, Format, JSON, etc.) while keeping internal values raw. Set this once at startup before generating or parsing IDs.

func NewObfuscator

func NewObfuscator(key int64) *Obfuscator

NewObfuscator creates an obfuscator with the given key. Use a random int64 and keep it secret.

func (*Obfuscator) Deobfuscate

func (o *Obfuscator) Deobfuscate(id ID) ID

Deobfuscate reverses obfuscation (XOR is its own inverse).

func (*Obfuscator) Obfuscate

func (o *Obfuscator) Obfuscate(id ID) ID

Obfuscate XORs the ID with the key.

Directories

Path Synopsis
Package base58 provides Base58 encoding and decoding for int64 values.
Package base58 provides Base58 encoding and decoding for int64 values.
Package crockford provides Crockford Base32 encoding and decoding for int64 values.
Package crockford provides Crockford Base32 encoding and decoding for int64 values.
Package postgres provides PostgreSQL integration for USID.
Package postgres provides PostgreSQL integration for USID.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL