snowflake

package module
v0.0.0-...-6ca99fc Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 9, 2025 License: BSD-2-Clause, MIT Imports: 9 Imported by: 0

README

Snowflake ID Generator 雪花識別碼套件

Credits

This project was inspired by and built upon several other repositories (mention in Reference Part below).
I also used OpenAI Codex to help generate and refine parts of the code.

Reference

Overview 概述

  • English: This package provides a lock-free implementation of Twitter's Snowflake ID algorithm for Go. It focuses on predictable latency, monotonic identifiers, and flexible encoding options. The generator allows you to tune node and sequence bit widths and to choose a custom epoch while keeping the classic 64-bit layout.
  • 中文: 這個套件在 Go 中實作 Twitter Snowflake 演算法,採無鎖設計以降低延遲並確保遞增的識別碼。使用者可以調整節點與序列的位元數,指定自訂的時間起點(Epoch),並享有多種編碼格式。

Design Logic 設計邏輯

  • English: Snowflake composes a 64-bit signed integer with the most significant bit fixed at zero. The remaining bits are reserved for timestamp, node identifier, and per-millisecond sequence number. This layout keeps IDs sortable and compact while supporting horizontally scaled emitters.
  • 中文: Snowflake 以 64 位帶符號整數表示,其最高位固定為 0,其餘位元依序代表時間戳、節點編號與同毫秒序號。這樣的配置讓識別碼可排序且體積小,並支援橫向擴充。
Bit Allocation 位元配置
Bits Field Description (EN) 描述(中文)
1 Sign Always 0 to keep IDs positive 永遠為 0,確保識別碼為正數
41 Timestamp (ms) Milliseconds since the configured epoch 自訂 Epoch 起算的毫秒數
10 Node ID Identifies logical node / worker 節點或工作者編號
12 Sequence Per-millisecond counter preventing collisions 同毫秒內的序號,避免碰撞

The node and sequence widths sum to 22 bits by default, but both can be tuned (still 22 bits combined) via options.

Origin snowflake from twitter

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0|                        Timestamp_Mill                       |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   Timestamp_Mill  |  DC_ID  | Mech_ID |       Step_Count      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Snowflake In this Package

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0|                        Timestamp_Mill                       |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   Timestamp_Mill  |      Node_ID      |       Step_Count      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Snowflake vs UUID Comparison 與 UUID 的比較

Property / 性質 Snowflake UUIDv7 UUIDv4
Latency 延遲 Local calculation (~tens of ns) Local calculation (~hundreds of ns) Local calculation (~hundreds of ns)
Sortability 排序性 Naturally ordered by timestamp Mostly ordered (timestamp-first) Unordered
Collision Handling 碰撞處理 Sequence resets per ms; wait for next tick Random 74-bit suffix, extremely low probability Random 122 bits, extremely low probability
High Concurrency 高併發 Up to 4096 IDs/ms/node by default; waits 1 ms when exhausted Similar per-node throughput, but no built-in back-pressure Unlimited but unordered
Payload Size 大小 64 bits (8 bytes) 128 bits (16 bytes) 128 bits (16 bytes)
Encoding 延伸性 Multiple bases (Base58/62/36/32/16/64) Typically hex or Base58 Typically hex or Base58
  • English: Snowflake excels when you need compact, ordered IDs with deterministic latency. UUIDv7 offers similar ordering but doubles the storage footprint. UUIDv4 remains useful for decentralised generation when ordering is irrelevant.
  • 中文: 若需緊湊且可排序的識別碼,Snowflake 提供最穩定的延遲。UUIDv7 雖然也可排序,但大小是 Snowflake 的兩倍。當排序不重要時,UUIDv4 仍適合完全分散式的場景。

Installation 安裝

go get github.com/TinyMurky/snowflake
  • English: Import the module in your project after fetching it.
  • 中文: 下載後即可在專案中引用此模組。

Quick Start 快速上手

package main

import (
	"fmt"
	"log"

	"github.com/TinyMurky/snowflake"
)

func main() {
	gen, err := snowflake.NewGenerator()
	if err != nil {
		log.Fatal(err)
	}

	id, err := gen.NextID(1) // node ID = 1
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(id.Base58())
}
  • English: NewGenerator defaults to Twitter's epoch, 10 node bits, and 12 sequence bits. NextID blocks briefly if a millisecond is saturated.
  • 中文: NewGenerator 預設使用 Twitter Epoch、10 位節點與 12 位序號。若同一毫秒內用量超額,NextID 會稍候至下一毫秒。
Custom Layout 自訂配置
gen, err := snowflake.NewGenerator(
	snowflake.WithNodeBits(8),
	snowflake.WithStepBits(14),
	snowflake.WithEpoch(time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)),
)
  • English: Keep nodeBits + stepBits == 22 to match the constructor guarantees. This combination supports 256 nodes and 16384 IDs per ms.
  • 中文: 請確保 nodeBits + stepBits == 22,如上設定可支援 256 個節點與每毫秒 16,384 筆序號。

Encoding & Decoding 編解碼

  • English: Every SID supports Base58, Base62, Base36, Base32 (Crockford), Base16, and Base64 encoders plus matching Parse... helpers. Base32 is case-insensitive and accepts O/o as zero; Base64 parsing ignores padding.
  • 中文: SID 提供 Base58Base62Base36Crockford Base32Base16Base64 的編碼函式及對應解析器。Base32 解析時不分大小寫且接受 O/o 代表 0;Base64 會忽略結尾的補字元。

Testing 測試

go test ./...
  • English: Unit tests cover the bit constructor, lock-free resolver, and encoding helpers. Integration tests validate chronological ordering and include a dedicated high-concurrency stress suite that saturates the per-ms capacity and verifies recovery.
  • 中文: 單元測試涵蓋位元組合器、無鎖序號解析器與各種編解碼。整合測試則檢查整體排序特性,並包含高併發壓力測試,模擬同毫秒耗盡後的回復行為。

Contributing 貢獻

  • English: Issues and pull requests are welcome. Please accompany new behaviour with tests and update both language sections of the README when documentation changes.
  • 中文: 歡迎提出 Issue 與 PR。新增功能請附上對應測試,並更新 README 的中英文段落。

How to open documentation

  1. install pkgsite
go install golang.org/x/pkgsite/cmd/pkgsite@latest

Documentation

Overview

Package snowflake implements a lock-free Snowflake ID generator alongside encoding and decoding helpers.

Index

Examples

Constants

This section is empty.

Variables

View Source
var ErrNegativeMillisecond = errors.New("millisecond must be >= 0")

ErrNegativeMillisecond is returned if GetNextStep is asked to operate on a negative millisecond offset.

Functions

func AssertEqual

func AssertEqual[T any](t testing.TB, got, want T)

AssertEqual use deep reflect to

func AssertError

func AssertError(t testing.TB, err error)

AssertError test error not be nil

func AssertNoError

func AssertNoError(t testing.TB, err error)

AssertNoError test error to be nil

Types

type AtomicResolver

type AtomicResolver struct {
	// contains filtered or unexported fields
}

AtomicResolver implements StepResolver without locks by packing the current millisecond together with the number of issued sequence values into a single atomic word.

The resolver keeps track of the amount of IDs that have already been emitted in the currently observed millisecond. When the counter reaches the capacity (maxStep + 1), the caller must wait until the clock advances.

func NewAtomicResolver

func NewAtomicResolver(maxStep int64) (*AtomicResolver, error)

NewAtomicResolver builds an AtomicResolver that can serve up to maxStep+1 unique sequence numbers per millisecond.

maxStep must be >= 0. A maxStep of 0 means only a single ID can be emitted within the same millisecond.

func (*AtomicResolver) GetNextStep

func (r *AtomicResolver) GetNextStep(ms int64) (int64, bool, error)

GetNextStep implements StepResolver. Counter start at sequence 0

It returns:

  1. The sequence number for the requested millisecond.
  2. A boolean signalling whether the caller should retry in a newer millisecond (true if the capacity has been exhausted or the clock moved backwards).
  3. An error (non-nil only for invalid input or when the clock regressed).

type ClockBackwardError

type ClockBackwardError struct {
	LastMs int64
}

ClockBackwardError is returned when a caller observes a millisecond earlier than the last one the resolver has already seen. The caller should wait until the clock catches up to LastMs (inclusive) before retrying.

func (*ClockBackwardError) Error

func (e *ClockBackwardError) Error() string

Error implements the error interface.

type Generator

type Generator struct {
	Constructor SIDConstructor
	Resolver    StepResolver
	Epoch       time.Time
}

Generator produces Snowflake IDs with millisecond resolution.

The generator is safe for concurrent use.

Example (ConfigurableEpoch)
package main

import (
	"fmt"
	"log"
	"time"

	"github.com/TinyMurky/snowflake"
)

func main() {
	epoch := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
	gen, err := snowflake.NewGenerator(
		snowflake.WithNodeBits(8),
		snowflake.WithStepBits(14),
		snowflake.WithEpoch(epoch),
	)
	if err != nil {
		log.Fatal(err)
	}

	id, err := gen.NextID(25)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("ID: %d (base36: %s)\n", id, id.Base36())
}

func NewGenerator

func NewGenerator(opts ...Opt) (*Generator, error)

NewGenerator creates a generator using configurable options. By default it allocates 10 bits for the node identifier, 12 bits for the per-millisecond sequence counter and uses the original Twitter epoch (2010-11-04 UTC).

func (*Generator) NextID

func (g *Generator) NextID(nodeID int64) (SID, error)

NextID returns the next Snowflake identifier for the supplied node ID.

The method blocks when the millisecond bucket is exhausted and automatically waits for the next tick. If the system clock moves backwards it waits until the wall clock catches up.

type IDConstructor

type IDConstructor struct {
	// contains filtered or unexported fields
}

IDConstructor construct timestamp, node_id, step_cnt into snowflakeID

func NewDefaultIDConstructor

func NewDefaultIDConstructor() (*IDConstructor, error)

NewDefaultIDConstructor will return a constructor wil format as:

  1. 41 bits timestamp
  2. 10 bits node_id
  3. 12 bits step_cnt

func NewIDConstructor

func NewIDConstructor(nodeBits, stepBits uint8) (*IDConstructor, error)

NewIDConstructor will return a constructor wil format as:

  1. 41 bits timestamp
  2. nodeBits of node_id
  3. stepBits step_cnt

nodeBits and stepBits share 22 bits

func (*IDConstructor) GenID

func (c *IDConstructor) GenID(timestamp, nodeID, stepCnt int64) (SID, error)

GenID construct timestamp, node_id, step_cnt into snowflake ID

func (*IDConstructor) GetMaxStep

func (c *IDConstructor) GetMaxStep() int64

GetMaxStep return the maximum stepCnt allow in a millisecond

type Opt

type Opt interface {
	Apply(*Options)
}

Opt is a functional option that mutates Options at construction time.

Typical usage:

NewGenerator(WithNodeBits(10), WithStepBits(12), WithEpoch(t))

type OptEpoch

type OptEpoch time.Time

OptEpoch is the functional option type for configuring the custom epoch.

func WithEpoch

func WithEpoch(t time.Time) OptEpoch

WithEpoch returns an option that sets the custom epoch (time origin).

Use a constant value (e.g., UTC midnight of a chosen date) so IDs remain comparable across processes and restarts.

func (OptEpoch) Apply

func (o OptEpoch) Apply(opt *Options)

Apply sets the epoch on the provided Options.

The epoch should be a fixed point in time in the past. All generated timestamps are measured as a delta from this epoch to reduce required timestamp bits.

type OptNodeBits

type OptNodeBits uint8

OptNodeBits is the functional option type for configuring nodeBits.

func WithNodeBits

func WithNodeBits(nodeBits uint8) OptNodeBits

WithNodeBits returns an option that sets the number of bits reserved for the node ID.

nodeBits should be chosen so that nodeBits + stepBits (+ timestamp bits in your layout) fits within your target word size (commonly 63 bits for signed 64-bit integers).

func (OptNodeBits) Apply

func (o OptNodeBits) Apply(opt *Options)

Apply sets the nodeBits on the provided Options.

Example: WithNodeBits(10) allows up to 1024 distinct nodes.

type OptStepBits

type OptStepBits uint8

OptStepBits is the functional option type for configuring stepBits.

func WithStepBits

func WithStepBits(stepBits uint8) OptStepBits

WithStepBits returns an option that sets the number of bits for the per-tick sequence.

Pick a value large enough for your expected peak allocation rate per tick.

func (OptStepBits) Apply

func (o OptStepBits) Apply(opt *Options)

Apply sets the stepBits on the provided Options.

Example: WithStepBits(12) allows up to 4096 IDs per tick (e.g., per millisecond).

type Options

type Options struct {
	// contains filtered or unexported fields
}

Options holds internal configuration used when creating a generator.

  • nodeBits: number of bits allocated to the node ID. Max nodes = 2^nodeBits.
  • stepBits: number of bits allocated to the per-tick sequence counter. Max sequence per tick = 2^stepBits - 1.
  • epoch: custom epoch (time origin) used to shrink the timestamp range.

Note: fields are unexported on purpose. Use Opt implementations (With*) to configure these values from outside the package.

type SID

type SID int64

SID represents a Snowflake identifier as a positive 64-bit integer.

Example (Encoding)
package main

import (
	"fmt"
	"log"

	"github.com/TinyMurky/snowflake"
)

func main() {
	constructor, err := snowflake.NewIDConstructor(10, 12)
	if err != nil {
		log.Fatal(err)
	}

	id, err := constructor.GenID(1_234_567, 42, 7)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println("raw:", int64(id))
	fmt.Println("base58:", id.Base58())
	fmt.Println("base62:", id.Base62())
	fmt.Println("base36:", id.Base36())
	fmt.Println("base32:", id.Base32())
	fmt.Println("base16:", id.Base16())
	fmt.Println("base64:", id.Base64())

}
Output:

raw: 5178149478407
base58: 3M2ELaut
base62: 1TABHVOJ
base36: 1U2T4I4NB
base32: 4PPGW5807
base16: 4B5A1C2A007
base64: BLWhwqAH

func ParseBase16

func ParseBase16(code string) (SID, error)

ParseBase16 will parse the SnowflakeID (string) that has been encoded by Base16 (hexadecimal) back to SID.

func ParseBase16Bytes

func ParseBase16Bytes(buf []byte) (SID, error)

ParseBase16Bytes parses a hexadecimal encoded Snowflake ID.

func ParseBase32

func ParseBase32(code string) (SID, error)

ParseBase32 will parse the SnowflakeID (string) that has been encoded by Base32 back to SID.

func ParseBase32Bytes

func ParseBase32Bytes(buf []byte) (SID, error)

ParseBase32Bytes parses a Crockford Base32 encoded Snowflake ID.

func ParseBase36

func ParseBase36(code string) (SID, error)

ParseBase36 will parse the SnowflakeID (string) that has been encoded by Base36 back to SID.

func ParseBase36Bytes

func ParseBase36Bytes(buf []byte) (SID, error)

ParseBase36Bytes parses a Base36 encoded Snowflake ID.

func ParseBase58

func ParseBase58(code string) (SID, error)

ParseBase58 will parse the SnowflakeID (string) that has been encoded by Base58 back to SID

func ParseBase58Bytes

func ParseBase58Bytes(buf []byte) (SID, error)

ParseBase58Bytes will parse the SnowflakeID ([]byte) that has been encoded by Base58 back to SID

func ParseBase62

func ParseBase62(code string) (SID, error)

ParseBase62 will parse the SnowflakeID (string) that has been encoded by Base62 back to SID

func ParseBase62Bytes

func ParseBase62Bytes(buf []byte) (SID, error)

ParseBase62Bytes will parse the SnowflakeID ([]byte) that has been encoded by Base62 back to SID

func ParseBase64

func ParseBase64(code string) (SID, error)

ParseBase64 will parse the SnowflakeID (string) that has been encoded by Base64 back to SID. Padding characters (`=`) are ignored.

func ParseBase64Bytes

func ParseBase64Bytes(buf []byte) (SID, error)

ParseBase64Bytes parses an unpadded Base64 encoded Snowflake ID.

func (SID) Base16

func (sid SID) Base16() string

Base16 returns the hexadecimal representation of the Snowflake ID.

func (SID) Base32

func (sid SID) Base32() string

Base32 encodes the Snowflake ID using the Crockford alphabet.

func (SID) Base36

func (sid SID) Base36() string

Base36 encodes the Snowflake ID using the alphanumeric alphabet (0-9, A-Z).

func (SID) Base58

func (sid SID) Base58() string

Base58 encode snowflakeID into Base58

func (SID) Base62

func (sid SID) Base62() string

Base62 encode snowflakeID into Base62

func (SID) Base64

func (sid SID) Base64() string

Base64 encodes the Snowflake ID using the standard Base64 alphabet without padding.

type SIDConstructor

type SIDConstructor interface {
	GenID(ms, nodeID, stepCnt int64) (SID, error)
	GetMaxStep() int64
}

SIDConstructor formats timestamp, node ID and step counter into a Snowflake ID.

type StepResolver

type StepResolver interface {
	// GetNextStep returns:
	//   1. The sequence/step counter associated with ms.
	//   2. Whether the caller should wait for the next millisecond.
	//   3. An error (non-nil on invalid input or when the clock regresses).
	GetNextStep(ms int64) (int64, bool, error)
}

StepResolver returns the next step counter for a given millisecond.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL