Generate and Analyze Twitter Snowflake ID Tool for Insights
Generate and analyze Twitter Snowflake IDs, unique 64-bit identifiers used in distributed systems. This tool allows you to create new Snowflake IDs and parse existing ones, providing insights into their timestamp, machine ID, and sequence number components.
Snowflake ID Generator
Snowflake ID Generator
Documentation
Snowflake ID Generator
Introduction
A Snowflake ID is a unique identifier used in distributed systems, originally developed by Twitter. This tool allows you to generate and analyze Snowflake IDs, which are 64-bit integers composed of a timestamp, machine ID, and sequence number.
How Snowflake IDs Work
Snowflake IDs are 64-bit integers structured as follows:
- 41 bits: Timestamp (milliseconds since a custom epoch)
- 10 bits: Machine ID (5 bits for data center ID, 5 bits for worker ID)
- 12 bits: Sequence number
This structure allows for the generation of approximately 4,096 unique IDs per millisecond per machine.
Using the Snowflake ID Generator
- (Optional) Set a custom epoch (default is Twitter's epoch: 2010-11-04T01:42:54.657Z)
- Enter a machine ID (0-31) and data center ID (0-31)
- Click "Generate" to create a new Snowflake ID
- The generated ID and its components will be displayed
To parse an existing Snowflake ID, enter it in the "Parse ID" field and click "Parse".
Formula
The Snowflake ID is constructed using bitwise operations:
1ID = (timestamp << 22) | (datacenterId << 17) | (workerId << 12) | sequence
2
Where:
timestamp
is the number of milliseconds since the epochdatacenterId
is a 5-bit integer (0-31)workerId
is a 5-bit integer (0-31)sequence
is a 12-bit integer (0-4095)
Calculation
The Snowflake ID generator performs the following steps:
- Get the current timestamp in milliseconds
- Ensure the timestamp is greater than the last used timestamp (for uniqueness)
- If the timestamp is the same as the last one, increment the sequence number
- If the sequence number overflows (reaches 4096), wait for the next millisecond
- Combine the components using bitwise operations to create the final ID
Use Cases
Snowflake IDs are particularly useful in:
- Distributed Systems: Generate unique IDs across multiple machines without coordination
- High-Volume Data: Create sortable IDs for large datasets
- Microservices: Ensure unique identifiers across different services
- Database Sharding: Use the timestamp or machine ID component for efficient sharding
Alternatives
While Snowflake IDs are powerful, other ID generation systems include:
- UUID (Universally Unique Identifier): Useful when distributed generation is needed without sortability
- Auto-incrementing database IDs: Simple but limited to single database instances
- ULID (Universally Unique Lexicographically Sortable Identifier): Similar to Snowflake, but with a different structure
Edge Cases and Limitations
-
Clock Synchronization: Snowflake IDs rely on system time. If the clock moves backwards due to NTP adjustments or daylight saving time changes, it can cause issues with ID generation.
-
Year 2038 Problem: The 41-bit timestamp will overflow in 2079 (assuming the Twitter epoch). Systems using Snowflake IDs should plan for this eventuality.
-
Machine ID Collisions: In large distributed systems, ensuring unique machine IDs can be challenging and may require additional coordination.
-
Sequence Overflow: In extremely high-throughput scenarios, it's possible to exhaust the 4096 sequences per millisecond, potentially causing delays.
-
Non-monotonicity Across Machines: While IDs are monotonically increasing on a single machine, they may not be strictly monotonic across multiple machines.
History
Snowflake IDs were introduced by Twitter in 2010 to address the need for distributed, time-sortable unique identifiers. They have since been adopted and adapted by many other companies and projects.
Examples
Here are implementations of Snowflake ID generators in various languages:
1class SnowflakeGenerator {
2 constructor(epoch = 1288834974657, datacenterIdBits = 5, workerIdBits = 5, sequenceBits = 12) {
3 this.epoch = BigInt(epoch);
4 this.datacenterIdBits = datacenterIdBits;
5 this.workerIdBits = workerIdBits;
6 this.sequenceBits = sequenceBits;
7 this.maxDatacenterId = -1n ^ (-1n << BigInt(datacenterIdBits));
8 this.maxWorkerId = -1n ^ (-1n << BigInt(workerIdBits));
9 this.sequenceMask = -1n ^ (-1n << BigInt(sequenceBits));
10 this.workerIdShift = BigInt(sequenceBits);
11 this.datacenterIdShift = BigInt(sequenceBits + workerIdBits);
12 this.timestampLeftShift = BigInt(sequenceBits + workerIdBits + datacenterIdBits);
13 this.sequence = 0n;
14 this.lastTimestamp = -1n;
15 }
16
17 nextId(datacenterId, workerId) {
18 let timestamp = this.currentTimestamp();
19
20 if (timestamp < this.lastTimestamp) {
21 throw new Error('Clock moved backwards. Refusing to generate id');
22 }
23
24 if (timestamp === this.lastTimestamp) {
25 this.sequence = (this.sequence + 1n) & this.sequenceMask;
26 if (this.sequence === 0n) {
27 timestamp = this.tilNextMillis(this.lastTimestamp);
28 }
29 } else {
30 this.sequence = 0n;
31 }
32
33 this.lastTimestamp = timestamp;
34
35 return ((timestamp - this.epoch) << this.timestampLeftShift) |
36 (BigInt(datacenterId) << this.datacenterIdShift) |
37 (BigInt(workerId) << this.workerIdShift) |
38 this.sequence;
39 }
40
41 tilNextMillis(lastTimestamp) {
42 let timestamp = this.currentTimestamp();
43 while (timestamp <= lastTimestamp) {
44 timestamp = this.currentTimestamp();
45 }
46 return timestamp;
47 }
48
49 currentTimestamp() {
50 return BigInt(Date.now());
51 }
52}
53
54// Usage
55const generator = new SnowflakeGenerator();
56const id = generator.nextId(1, 1);
57console.log(`Generated Snowflake ID: ${id}`);
58
1import time
2import threading
3
4class SnowflakeGenerator:
5 def __init__(self, datacenter_id, worker_id, sequence=0):
6 self.datacenter_id = datacenter_id
7 self.worker_id = worker_id
8 self.sequence = sequence
9
10 self.last_timestamp = -1
11 self.epoch = 1288834974657
12
13 self.datacenter_id_bits = 5
14 self.worker_id_bits = 5
15 self.sequence_bits = 12
16
17 self.max_datacenter_id = -1 ^ (-1 << self.datacenter_id_bits)
18 self.max_worker_id = -1 ^ (-1 << self.worker_id_bits)
19
20 self.worker_id_shift = self.sequence_bits
21 self.datacenter_id_shift = self.sequence_bits + self.worker_id_bits
22 self.timestamp_left_shift = self.sequence_bits + self.worker_id_bits + self.datacenter_id_bits
23 self.sequence_mask = -1 ^ (-1 << self.sequence_bits)
24
25 self._lock = threading.Lock()
26
27 def _til_next_millis(self, last_timestamp):
28 timestamp = self._get_timestamp()
29 while timestamp <= last_timestamp:
30 timestamp = self._get_timestamp()
31 return timestamp
32
33 def _get_timestamp(self):
34 return int(time.time() * 1000)
35
36 def next_id(self):
37 with self._lock:
38 timestamp = self._get_timestamp()
39
40 if timestamp < self.last_timestamp:
41 raise ValueError("Clock moved backwards. Refusing to generate id")
42
43 if timestamp == self.last_timestamp:
44 self.sequence = (self.sequence + 1) & self.sequence_mask
45 if self.sequence == 0:
46 timestamp = self._til_next_millis(self.last_timestamp)
47 else:
48 self.sequence = 0
49
50 self.last_timestamp = timestamp
51
52 return ((timestamp - self.epoch) << self.timestamp_left_shift) | \
53 (self.datacenter_id << self.datacenter_id_shift) | \
54 (self.worker_id << self.worker_id_shift) | \
55 self.sequence
56
57## Usage
58generator = SnowflakeGenerator(datacenter_id=1, worker_id=1)
59snowflake_id = generator.next_id()
60print(f"Generated Snowflake ID: {snowflake_id}")
61
1import java.util.concurrent.locks.Lock;
2import java.util.concurrent.locks.ReentrantLock;
3
4public class SnowflakeGenerator {
5 private final long epoch;
6 private final long datacenterIdBits;
7 private final long workerIdBits;
8 private final long sequenceBits;
9 private final long maxDatacenterId;
10 private final long maxWorkerId;
11 private final long workerIdShift;
12 private final long datacenterIdShift;
13 private final long timestampLeftShift;
14 private final long sequenceMask;
15
16 private long datacenterId;
17 private long workerId;
18 private long sequence = 0L;
19 private long lastTimestamp = -1L;
20
21 private final Lock lock = new ReentrantLock();
22
23 public SnowflakeGenerator(long datacenterId, long workerId) {
24 this.epoch = 1288834974657L;
25 this.datacenterIdBits = 5L;
26 this.workerIdBits = 5L;
27 this.sequenceBits = 12L;
28
29 this.maxDatacenterId = ~(-1L << datacenterIdBits);
30 this.maxWorkerId = ~(-1L << workerIdBits);
31
32 this.workerIdShift = sequenceBits;
33 this.datacenterIdShift = sequenceBits + workerIdBits;
34 this.timestampLeftShift = sequenceBits + workerIdBits + datacenterIdBits;
35 this.sequenceMask = ~(-1L << sequenceBits);
36
37 if (datacenterId > maxDatacenterId || datacenterId < 0) {
38 throw new IllegalArgumentException("datacenterId can't be greater than maxDatacenterId or less than 0");
39 }
40 if (workerId > maxWorkerId || workerId < 0) {
41 throw new IllegalArgumentException("workerId can't be greater than maxWorkerId or less than 0");
42 }
43 this.datacenterId = datacenterId;
44 this.workerId = workerId;
45 }
46
47 public long nextId() {
48 lock.lock();
49 try {
50 long timestamp = timeGen();
51 if (timestamp < lastTimestamp) {
52 throw new RuntimeException("Clock moved backwards. Refusing to generate id");
53 }
54
55 if (lastTimestamp == timestamp) {
56 sequence = (sequence + 1) & sequenceMask;
57 if (sequence == 0) {
58 timestamp = tilNextMillis(lastTimestamp);
59 }
60 } else {
61 sequence = 0L;
62 }
63
64 lastTimestamp = timestamp;
65
66 return ((timestamp - epoch) << timestampLeftShift) |
67 (datacenterId << datacenterIdShift) |
68 (workerId << workerIdShift) |
69 sequence;
70 } finally {
71 lock.unlock();
72 }
73 }
74
75 private long tilNextMillis(long lastTimestamp) {
76 long timestamp = timeGen();
77 while (timestamp <= lastTimestamp) {
78 timestamp = timeGen();
79 }
80 return timestamp;
81 }
82
83 private long timeGen() {
84 return System.currentTimeMillis();
85 }
86
87 public static void main(String[] args) {
88 SnowflakeGenerator generator = new SnowflakeGenerator(1, 1);
89 long id = generator.nextId();
90 System.out.println("Generated Snowflake ID: " + id);
91 }
92}
93
1require 'time'
2
3class SnowflakeGenerator
4 def initialize(datacenter_id, worker_id, sequence = 0)
5 @datacenter_id = datacenter_id
6 @worker_id = worker_id
7 @sequence = sequence
8 @last_timestamp = -1
9 @epoch = 1288834974657
10
11 @datacenter_id_bits = 5
12 @worker_id_bits = 5
13 @sequence_bits = 12
14
15 @max_datacenter_id = -1 ^ (-1 << @datacenter_id_bits)
16 @max_worker_id = -1 ^ (-1 << @worker_id_bits)
17
18 @worker_id_shift = @sequence_bits
19 @datacenter_id_shift = @sequence_bits + @worker_id_bits
20 @timestamp_left_shift = @sequence_bits + @worker_id_bits + @datacenter_id_bits
21 @sequence_mask = -1 ^ (-1 << @sequence_bits)
22 end
23
24 def next_id
25 timestamp = (Time.now.to_f * 1000).to_i
26
27 raise 'Clock moved backwards' if timestamp < @last_timestamp
28
29 if timestamp == @last_timestamp
30 @sequence = (@sequence + 1) & @sequence_mask
31 timestamp = til_next_millis(@last_timestamp) if @sequence == 0
32 else
33 @sequence = 0
34 end
35
36 @last_timestamp = timestamp
37
38 ((timestamp - @epoch) << @timestamp_left_shift) |
39 (@datacenter_id << @datacenter_id_shift) |
40 (@worker_id << @worker_id_shift) |
41 @sequence
42 end
43
44 private
45
46 def til_next_millis(last_timestamp)
47 timestamp = (Time.now.to_f * 1000).to_i
48 timestamp = (Time.now.to_f * 1000).to_i while timestamp <= last_timestamp
49 timestamp
50 end
51end
52
53## Usage
54generator = SnowflakeGenerator.new(1, 1)
55snowflake_id = generator.next_id
56puts "Generated Snowflake ID: #{snowflake_id}"
57
1<?php
2
3class SnowflakeGenerator {
4 private $epoch;
5 private $datacenterIdBits;
6 private $workerIdBits;
7 private $sequenceBits;
8 private $maxDatacenterId;
9 private $maxWorkerId;
10 private $workerIdShift;
11 private $datacenterIdShift;
12 private $timestampLeftShift;
13 private $sequenceMask;
14
15 private $datacenterId;
16 private $workerId;
17 private $sequence = 0;
18 private $lastTimestamp = -1;
19
20 public function __construct($datacenterId, $workerId) {
21 $this->epoch = 1288834974657;
22 $this->datacenterIdBits = 5;
23 $this->workerIdBits = 5;
24 $this->sequenceBits = 12;
25
26 $this->maxDatacenterId = -1 ^ (-1 << $this->datacenterIdBits);
27 $this->maxWorkerId = -1 ^ (-1 << $this->workerIdBits);
28
29 $this->workerIdShift = $this->sequenceBits;
30 $this->datacenterIdShift = $this->sequenceBits + $this->workerIdBits;
31 $this->timestampLeftShift = $this->sequenceBits + $this->workerIdBits + $this->datacenterIdBits;
32 $this->sequenceMask = -1 ^ (-1 << $this->sequenceBits);
33
34 if ($datacenterId > $this->maxDatacenterId || $datacenterId < 0) {
35 throw new Exception("datacenterId can't be greater than maxDatacenterId or less than 0");
36 }
37 if ($workerId > $this->maxWorkerId || $workerId < 0) {
38 throw new Exception("workerId can't be greater than maxWorkerId or less than 0");
39 }
40 $this->datacenterId = $datacenterId;
41 $this->workerId = $workerId;
42 }
43
44 public function nextId() {
45 $timestamp = $this->timeGen();
46
47 if ($timestamp < $this->lastTimestamp) {
48 throw new Exception("Clock moved backwards. Refusing to generate id");
49 }
50
51 if ($this->lastTimestamp == $timestamp) {
52 $this->sequence = ($this->sequence + 1) & $this->sequenceMask;
53 if ($this->sequence == 0) {
54 $timestamp = $this->tilNextMillis($this->lastTimestamp);
55 }
56 } else {
57 $this->sequence = 0;
58 }
59
60 $this->lastTimestamp = $timestamp;
61
62 return (($timestamp - $this->epoch) << $this->timestampLeftShift) |
63 ($this->datacenterId << $this->datacenterIdShift) |
64 ($this->workerId << $this->workerIdShift) |
65 $this->sequence;
66 }
67
68 private function tilNextMillis($lastTimestamp) {
69 $timestamp = $this->timeGen();
70 while ($timestamp <= $lastTimestamp) {
71 $timestamp = $this->timeGen();
72 }
73 return $timestamp;
74 }
75
76 private function timeGen() {
77 return floor(microtime(true) * 1000);
78 }
79}
80
81// Usage
82$generator = new SnowflakeGenerator(1, 1);
83$id = $generator->nextId();
84echo "Generated Snowflake ID: " . $id . "\n";
85
1using System;
2using System.Threading;
3
4public class SnowflakeGenerator
5{
6 private readonly long _epoch;
7 private readonly int _datacenterIdBits;
8 private readonly int _workerIdBits;
9 private readonly int _sequenceBits;
10 private readonly long _maxDatacenterId;
11 private readonly long _maxWorkerId;
12 private readonly int _workerIdShift;
13 private readonly int _datacenterIdShift;
14 private readonly int _timestampLeftShift;
15 private readonly long _sequenceMask;
16
17 private readonly long _datacenterId;
18 private readonly long _workerId;
19 private long _sequence = 0L;
20 private long _lastTimestamp = -1L;
21
22 private readonly object _lock = new object();
23
24 public SnowflakeGenerator(long datacenterId, long workerId)
25 {
26 _epoch = 1288834974657L;
27 _datacenterIdBits = 5;
28 _workerIdBits = 5;
29 _sequenceBits = 12;
30
31 _maxDatacenterId = -1L ^ (-1L << _datacenterIdBits);
32 _maxWorkerId = -1L ^ (-1L << _workerIdBits);
33
34 _workerIdShift = _sequenceBits;
35 _datacenterIdShift = _sequenceBits + _workerIdBits;
36 _timestampLeftShift = _sequenceBits + _workerIdBits + _datacenterIdBits;
37 _sequenceMask = -1L ^ (-1L << _sequenceBits);
38
39 if (datacenterId > _maxDatacenterId || datacenterId < 0)
40 {
41 throw new ArgumentException($"datacenterId can't be greater than {_maxDatacenterId} or less than 0");
42 }
43 if (workerId > _maxWorkerId || workerId < 0)
44 {
45 throw new ArgumentException($"workerId can't be greater than {_maxWorkerId} or less than 0");
46 }
47 _datacenterId = datacenterId;
48 _workerId = workerId;
49 }
50
51 public long NextId()
52 {
53 lock (_lock)
54 {
55 var timestamp = TimeGen();
56
57 if (timestamp < _lastTimestamp)
58 {
59 throw new Exception("Clock moved backwards. Refusing to generate id");
60 }
61
62 if (_lastTimestamp == timestamp)
63 {
64 _sequence = (_sequence + 1) & _sequenceMask;
65 if (_sequence == 0)
66 {
67 timestamp = TilNextMillis(_lastTimestamp);
68 }
69 }
70 else
71 {
72 _sequence = 0L;
73 }
74
75 _lastTimestamp = timestamp;
76
77 return ((timestamp - _epoch) << _timestampLeftShift) |
78 (_datacenterId << _datacenterIdShift) |
79 (_workerId << _workerIdShift) |
80 _sequence;
81 }
82 }
83
84 private long TilNextMillis(long lastTimestamp)
85 {
86 var timestamp = TimeGen();
87 while (timestamp <= lastTimestamp)
88 {
89 timestamp = TimeGen();
90 }
91 return timestamp;
92 }
93
94 private long TimeGen()
95 {
96 return DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
97 }
98}
99
100// Usage
101class Program
102{
103 static void Main(string[] args)
104 {
105 var generator = new SnowflakeGenerator(1, 1);
106 var id = generator.NextId();
107 Console.WriteLine($"Generated Snowflake ID: {id}");
108 }
109}
110
1package main
2
3import (
4 "fmt"
5 "sync"
6 "time"
7)
8
9type SnowflakeGenerator struct {
10 epoch int64
11 datacenterIdBits uint
12 workerIdBits uint
13 sequenceBits uint
14 maxDatacenterId int64
15 maxWorkerId int64
16 workerIdShift uint
17 datacenterIdShift uint
18 timestampLeftShift uint
19 sequenceMask int64
20
21 datacenterId int64
22 workerId int64
23 sequence int64
24 lastTimestamp int64
25
26 lock sync.Mutex
27}
28
29func NewSnowflakeGenerator(datacenterId, workerId int64) (*SnowflakeGenerator, error) {
30 g := &SnowflakeGenerator{
31 epoch: 1288834974657,
32 datacenterIdBits: 5,
33 workerIdBits: 5,
34 sequenceBits: 12,
35 lastTimestamp: -1,
36 }
37
38 g.maxDatacenterId = -1 ^ (-1 << g.datacenterIdBits)
39 g.maxWorkerId = -1 ^ (-1 << g.workerIdBits)
40
41 g.workerIdShift = g.sequenceBits
42 g.datacenterIdShift = g.sequenceBits + g.workerIdBits
43 g.timestampLeftShift = g.sequenceBits + g.workerIdBits + g.datacenterIdBits
44 g.sequenceMask = -1 ^ (-1 << g.sequenceBits)
45
46 if datacenterId > g.maxDatacenterId || datacenterId < 0 {
47 return nil, fmt.Errorf("datacenterId can't be greater than %d or less than 0", g.maxDatacenterId)
48 }
49 if workerId > g.maxWorkerId || workerId < 0 {
50 return nil, fmt.Errorf("workerId can't be greater than %d or less than 0", g.maxWorkerId)
51 }
52 g.datacenterId = datacenterId
53 g.workerId = workerId
54
55 return g, nil
56}
57
58func (g *SnowflakeGenerator) NextId() (int64, error) {
59 g.lock.Lock()
60 defer g.lock.Unlock()
61
62 timestamp := g.timeGen()
63
64 if timestamp < g.lastTimestamp {
65 return 0, fmt.Errorf("clock moved backwards, refusing to generate id")
66 }
67
68 if g.lastTimestamp == timestamp {
69 g.sequence = (g.sequence + 1) & g.sequenceMask
70 if g.sequence == 0 {
71 timestamp = g.tilNextMillis(g.lastTimestamp)
72 }
73 } else {
74 g.sequence = 0
75 }
76
77 g.lastTimestamp = timestamp
78
79 return ((timestamp - g.epoch) << g.timestampLeftShift) |
80 (g.datacenterId << g.datacenterIdShift) |
81 (g.workerId << g.workerIdShift) |
82 g.sequence, nil
83}
84
85func (g *SnowflakeGenerator) tilNextMillis(lastTimestamp int64) int64 {
86 timestamp := g.timeGen()
87 for timestamp <= lastTimestamp {
88 timestamp = g.timeGen()
89 }
90 return timestamp
91}
92
93func (g *SnowflakeGenerator) timeGen() int64 {
94 return time.Now().UnixNano() / int64(time.Millisecond)
95}
96
97func main() {
98 generator, err := NewSnowflakeGenerator(1, 1)
99 if err != nil {
100 fmt.Printf("Error creating generator: %v\n", err)
101 return
102 }
103
104 id, err := generator.NextId()
105 if err != nil {
106 fmt.Printf("Error generating ID: %v\n", err)
107 return
108 }
109
110 fmt.Printf("Generated Snowflake ID: %d\n", id)
111}
112
Diagram
Here's a visual representation of the Snowflake ID structure:
References
- "Announcing Snowflake." Twitter Engineering Blog, https://blog.twitter.com/engineering/en_us/a/2010/announcing-snowflake
- "Snowflake ID." Wikipedia, https://en.wikipedia.org/wiki/Snowflake_ID
- "Distributed ID Generation in Microservices." Medium, https://medium.com/swlh/distributed-id-generation-in-microservices-b6ce9a8dd93f
Feedback
Click the feedback toast to start giving feedback about this tool
Related Tools
Discover more tools that might be useful for your workflow