Mwanzo wa Kitambulisho cha Snowflake cha Twitter na Uchambuzi
Unda na uchambue Kitambulisho cha Snowflake cha Twitter, vitambulisho vya kipekee vya 64-bit vinavyotumika katika mifumo iliyosambazwa. Chombo hiki kinakuruhusu kuunda vitambulisho vipya vya Snowflake na kuchambua vile vilivyopo, kikitoa maarifa kuhusu sehemu zao za muda, kitambulisho cha mashine, na nambari ya mfuatano.
Mwanzo wa ID ya Snowflake
Mwanzo wa ID ya Snowflake
Hati
1## Snowflake ID Generator
2
3### Introduction
4
5ID ya Snowflake ni kitambulisho cha kipekee kinachotumiwa katika mifumo iliyosambazwa, ambayo awali ilitengenezwa na Twitter. Chombo hiki kinakuwezesha kuzalisha na kuchambua ID za Snowflake, ambazo ni nambari za 64-bit zilizoundwa na muda, kitambulisho cha mashine, na nambari ya mfululizo.
6
7### How Snowflake IDs Work
8
9ID za Snowflake ni nambari za 64-bit zilizo na muundo kama ifuatavyo:
10
11- 41 bits: Muda (millisekunde tangu enzi maalum)
12- 10 bits: Kitambulisho cha Mashine (bit 5 kwa kitambulisho cha kituo cha data, bit 5 kwa kitambulisho cha mfanyakazi)
13- 12 bits: Nambari ya Mfululizo
14
15Muundo huu unaruhusu uzalishaji wa takriban 4,096 ID za kipekee kwa millisekunde kwa mashine.
16
17### Using the Snowflake ID Generator
18
191. (Hiari) Weka enzi maalum (ya kawaida ni enzi ya Twitter: 2010-11-04T01:42:54.657Z)
202. Ingiza kitambulisho cha mashine (0-31) na kitambulisho cha kituo cha data (0-31)
213. Bonyeza "Zalisha" ili kuunda ID mpya ya Snowflake
224. ID iliyozalishwa na vipengele vyake vitatolewa
23
24Ili kuchambua ID iliyopo ya Snowflake, ingiza katika uwanja wa "Parse ID" na bonyeza "Parse".
25
26### Formula
27
28ID ya Snowflake inajengwa kwa kutumia operesheni za bitwise:
29
30plaintext ID = (timestamp << 22) | (datacenterId << 17) | (workerId << 12) | sequence
1
2Ambapo:
3- `timestamp` ni idadi ya millisekunde tangu enzi
4- `datacenterId` ni nambari ya 5-bit (0-31)
5- `workerId` ni nambari ya 5-bit (0-31)
6- `sequence` ni nambari ya 12-bit (0-4095)
7
8### Calculation
9
10Generator ya ID ya Snowflake inatekeleza hatua zifuatazo:
11
121. Pata muda wa sasa kwa millisekunde
132. Hakikisha muda ni mkubwa kuliko muda wa mwisho uliotumika (kwa kipekee)
143. Ikiwa muda ni sawa na wa mwisho, pandisha nambari ya mfululizo
154. Ikiwa nambari ya mfululizo inazidi (inasonga 4096), subiri millisekunde inayofuata
165. Changanya vipengele kwa kutumia operesheni za bitwise ili kuunda ID ya mwisho
17
18### Use Cases
19
20ID za Snowflake ni muhimu hasa katika:
21
221. Mifumo iliyosambazwa: Zalisha ID za kipekee kati ya mashine nyingi bila uratibu
232. Takwimu za Kiwango Kikubwa: Unda ID zinazoweza kuorodheshwa kwa seti kubwa za data
243. Microservices: Hakikisha vitambulisho vya kipekee kati ya huduma tofauti
254. Kugawanya Hifadhidata: Tumia kipengele cha muda au kitambulisho cha mashine kwa kugawanya kwa ufanisi
26
27#### Alternatives
28
29Ingawa ID za Snowflake ni zenye nguvu, mifumo mingine ya uzalishaji wa ID ni pamoja na:
30
311. UUID (Kitambulisho Kifaa cha Kijumla): Inafaa wakati uzalishaji wa kusambazwa unahitajika bila kuweza kuorodheshwa
322. ID zinazoongezeka kiotomatiki katika hifadhidata: Rahisi lakini ina mipaka kwa mifumo ya hifadhidata moja
333. ULID (Kitambulisho Kifaa cha Kijumla na Kinachoweza Kuorodheshwa Kulingana na Kijumla): Kama Snowflake, lakini ina muundo tofauti
34
35### Edge Cases and Limitations
36
371. Ulinganifu wa Saa: ID za Snowflake zinategemea wakati wa mfumo. Ikiwa saa inahamia nyuma kutokana na marekebisho ya NTP au mabadiliko ya wakati wa majira, inaweza kusababisha matatizo katika uzalishaji wa ID.
38
392. Tatizo la Mwaka 2038: Muda wa 41-bit utajaa mwaka 2079 (kulingana na enzi ya Twitter). Mifumo inayotumia ID za Snowflake inapaswa kupanga kwa ajili ya tukio hili.
40
413. Migongano ya Kitambulisho cha Mashine: Katika mifumo mikubwa iliyosambazwa, kuhakikisha vitambulisho vya mashine ni vya kipekee kunaweza kuwa changamoto na inaweza kuhitaji uratibu wa ziada.
42
434. Kufurika kwa Mfululizo: Katika hali za juu za kupitia, inawezekana kupita mipaka ya 4096 ya mfululizo kwa millisekunde, ambayo inaweza kusababisha ucheleweshaji.
44
455. Kutokuwa na Mpangilio Kati ya Mashine: Ingawa ID zinaongezeka kwa mpangilio kwenye mashine moja, zinaweza kuwa si za mpangilio kwa mashine nyingi.
46
47### History
48
49ID za Snowflake zilianzishwa na Twitter mwaka 2010 ili kukidhi hitaji la vitambulisho vya kipekee, vinavyoweza kuorodheshwa kwa wakati katika mazingira yaliyosambazwa. Tangu wakati huo, zimekubaliwa na kubadilishwa na kampuni na miradi mingi mingine.
50
51### Examples
52
53Hapa kuna utekelezaji wa generators za ID za Snowflake katika lugha mbalimbali:
54
55javascript class SnowflakeGenerator { constructor(epoch = 1288834974657, datacenterIdBits = 5, workerIdBits = 5, sequenceBits = 12) { this.epoch = BigInt(epoch); this.datacenterIdBits = datacenterIdBits; this.workerIdBits = workerIdBits; this.sequenceBits = sequenceBits; this.maxDatacenterId = -1n ^ (-1n << BigInt(datacenterIdBits)); this.maxWorkerId = -1n ^ (-1n << BigInt(workerIdBits)); this.sequenceMask = -1n ^ (-1n << BigInt(sequenceBits)); this.workerIdShift = BigInt(sequenceBits); this.datacenterIdShift = BigInt(sequenceBits + workerIdBits); this.timestampLeftShift = BigInt(sequenceBits + workerIdBits + datacenterIdBits); this.sequence = 0n; this.lastTimestamp = -1n; }
nextId(datacenterId, workerId) { let timestamp = this.currentTimestamp();
if (timestamp < this.lastTimestamp) {
throw new Error('Clock moved backwards. Refusing to generate id');
}
if (timestamp === this.lastTimestamp) {
this.sequence = (this.sequence + 1n) & this.sequenceMask;
if (this.sequence === 0n) {
timestamp = this.tilNextMillis(this.lastTimestamp);
}
} else {
this.sequence = 0n;
}
this.lastTimestamp = timestamp;
return ((timestamp - this.epoch) << this.timestampLeftShift) |
(BigInt(datacenterId) << this.datacenterIdShift) |
(BigInt(workerId) << this.workerIdShift) |
this.sequence;
}
tilNextMillis(lastTimestamp) { let timestamp = this.currentTimestamp(); while (timestamp <= lastTimestamp) { timestamp = this.currentTimestamp(); } return timestamp; }
currentTimestamp() { return BigInt(Date.now()); } }
// Usage
const generator = new SnowflakeGenerator();
const id = generator.nextId(1, 1);
console.log(Generated Snowflake ID: ${id});
1
2python import time import threading
class SnowflakeGenerator: def init(self, datacenter_id, worker_id, sequence=0): self.datacenter_id = datacenter_id self.worker_id = worker_id self.sequence = sequence
self.last_timestamp = -1
self.epoch = 1288834974657
self.datacenter_id_bits = 5
self.worker_id_bits = 5
self.sequence_bits = 12
self.max_datacenter_id = -1 ^ (-1 << self.datacenter_id_bits)
self.max_worker_id = -1 ^ (-1 << self.worker_id_bits)
self.worker_id_shift = self.sequence_bits
self.datacenter_id_shift = self.sequence_bits + self.worker_id_bits
self.timestamp_left_shift = self.sequence_bits + self.worker_id_bits + self.datacenter_id_bits
self.sequence_mask = -1 ^ (-1 << self.sequence_bits)
self._lock = threading.Lock()
def _til_next_millis(self, last_timestamp):
timestamp = self._get_timestamp()
while timestamp <= last_timestamp:
timestamp = self._get_timestamp()
return timestamp
def _get_timestamp(self):
return int(time.time() * 1000)
def next_id(self):
with self._lock:
timestamp = self._get_timestamp()
if timestamp < self.last_timestamp:
raise ValueError("Clock moved backwards. Refusing to generate id")
if timestamp == self.last_timestamp:
self.sequence = (self.sequence + 1) & self.sequence_mask
if self.sequence == 0:
timestamp = self._til_next_millis(self.last_timestamp)
else:
self.sequence = 0
self.last_timestamp = timestamp
return ((timestamp - self.epoch) << self.timestamp_left_shift) | \
(self.datacenter_id << self.datacenter_id_shift) | \
(self.worker_id << self.worker_id_shift) | \
self.sequence
Usage
generator = SnowflakeGenerator(datacenter_id=1, worker_id=1) snowflake_id = generator.next_id() print(f"Generated Snowflake ID: {snowflake_id}")
1
2java import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantLock;
public class SnowflakeGenerator { private final long epoch; private final long datacenterIdBits; private final long workerIdBits; private final long sequenceBits; private final long maxDatacenterId; private final long maxWorkerId; private final long workerIdShift; private final long datacenterIdShift; private final long timestampLeftShift; private final long sequenceMask;
private long datacenterId;
private long workerId;
private long sequence = 0L;
private long lastTimestamp = -1L;
private final Lock lock = new ReentrantLock();
public SnowflakeGenerator(long datacenterId, long workerId) {
this.epoch = 1288834974657L;
this.datacenterIdBits = 5L;
this.workerIdBits = 5L;
this.sequenceBits = 12L;
this.maxDatacenterId = ~(-1L << datacenterIdBits);
this.maxWorkerId = ~(-1L << workerIdBits);
this.workerIdShift = sequenceBits;
this.datacenterIdShift = sequenceBits + workerIdBits;
this.timestampLeftShift = sequenceBits + workerIdBits + datacenterIdBits;
this.sequenceMask = ~(-1L << sequenceBits);
if (datacenterId > maxDatacenterId || datacenterId < 0) {
throw new IllegalArgumentException("datacenterId can't be greater than maxDatacenterId or less than 0");
}
if (workerId > maxWorkerId || workerId < 0) {
throw new IllegalArgumentException("workerId can't be greater than maxWorkerId or less than 0");
}
this.datacenterId = datacenterId;
this.workerId = workerId;
}
public long nextId() {
lock.lock();
try {
long timestamp = timeGen();
if (timestamp < lastTimestamp) {
throw new RuntimeException("Clock moved backwards. Refusing to generate id");
}
if (lastTimestamp == timestamp) {
sequence = (sequence + 1) & sequenceMask;
if (sequence == 0) {
timestamp = tilNextMillis(lastTimestamp);
}
} else {
sequence = 0L;
}
lastTimestamp = timestamp;
return ((timestamp - epoch) << timestampLeftShift) |
(datacenterId << datacenterIdShift) |
(workerId << workerIdShift) |
sequence;
} finally {
lock.unlock();
}
}
private long tilNextMillis(long lastTimestamp) {
long timestamp = timeGen();
while (timestamp <= lastTimestamp) {
timestamp = timeGen();
}
return timestamp;
}
private long timeGen() {
return System.currentTimeMillis();
}
public static void main(String[] args) {
SnowflakeGenerator generator = new SnowflakeGenerator(1, 1);
long id = generator.nextId();
System.out.println("Generated Snowflake ID: " + id);
}
}
1
2ruby require 'time'
class SnowflakeGenerator def initialize(datacenter_id, worker_id, sequence = 0) @datacenter_id = datacenter_id @worker_id = worker_id @sequence = sequence @last_timestamp = -1 @epoch = 1288834974657
@datacenter_id_bits = 5
@worker_id_bits = 5
@sequence_bits = 12
@max_datacenter_id = -1 ^ (-1 << @datacenter_id_bits)
@max_worker_id = -1 ^ (-1 << @worker_id_bits)
@worker_id_shift = @sequence_bits
@datacenter_id_shift = @sequence_bits + @worker_id_bits
@timestamp_left_shift = @sequence_bits + @worker_id_bits + @datacenter_id_bits
@sequence_mask = -1 ^ (-1 << @sequence_bits)
end
def next_id timestamp = (Time.now.to_f * 1000).to_i
raise 'Clock moved backwards' if timestamp < @last_timestamp
if timestamp == @last_timestamp
@sequence = (@sequence + 1) & @sequence_mask
timestamp = til_next_millis(@last_timestamp) if @sequence == 0
else
@sequence = 0
end
@last_timestamp = timestamp
((timestamp - @epoch) << @timestamp_left_shift) |
(@datacenter_id << @datacenter_id_shift) |
(@worker_id << @worker_id_shift) |
@sequence
end
private
def til_next_millis(last_timestamp) timestamp = (Time.now.to_f * 1000).to_i timestamp = (Time.now.to_f * 1000).to_i while timestamp <= last_timestamp timestamp end end
Usage
generator = SnowflakeGenerator.new(1, 1) snowflake_id = generator.next_id puts "Generated Snowflake ID: #{snowflake_id}"
1
2php
epoch = 1288834974657; $this->datacenterIdBits = 5; $this->workerIdBits = 5; $this->sequenceBits = 12; $this->maxDatacenterId = -1 ^ (-1 << $this->datacenterIdBits); $this->maxWorkerId = -1 ^ (-1 << $this->workerIdBits); $this->workerIdShift = $this->sequenceBits; $this->datacenterIdShift = $this->sequenceBits + $this->workerIdBits; $this->timestampLeftShift = $this->sequenceBits + $this->workerIdBits + $this->datacenterIdBits; $this->sequenceMask = -1 ^ (-1 << $this->sequenceBits); if ($datacenterId > $this->maxDatacenterId || $datacenterId < 0) { throw new Exception("datacenterId can't be greater than maxDatacenterId or less than 0"); } if ($workerId > $this->maxWorkerId || $workerId < 0) { throw new Exception("workerId can't be greater than maxWorkerId or less than 0"); } $this->datacenterId = $datacenterId; $this->workerId = $workerId; } public function nextId() { $timestamp = $this->timeGen(); if ($timestamp < $this->lastTimestamp) { throw new Exception("Clock moved backwards. Refusing to generate id"); } if ($this->lastTimestamp == $timestamp) { $this->sequence = ($this->sequence + 1) & $this->sequenceMask; if ($this->sequence == 0) { $timestamp = $this->tilNextMillis($this->lastTimestamp); } } else { $this->sequence = 0; } $this->lastTimestamp = $timestamp; return (($timestamp - $this->epoch) << $this->timestampLeftShift) | ($this->datacenterId << $this->datacenterIdShift) | ($this->workerId << $this->workerIdShift) | $this->sequence; } private function tilNextMillis($lastTimestamp) { $timestamp = $this->timeGen(); while ($timestamp <= $lastTimestamp) { $timestamp = $this->timeGen(); } return $timestamp; } private function timeGen() { return floor(microtime(true) * 1000); } } // Usage $generator = new SnowflakeGenerator(1, 1); $id = $generator->nextId(); echo "Generated Snowflake ID: " . $id . "\n";1
2csharp using System; using System.Threading;
public class SnowflakeGenerator { private readonly long _epoch; private readonly int _datacenterIdBits; private readonly int _workerIdBits; private readonly int _sequenceBits; private readonly long _maxDatacenterId; private readonly long _maxWorkerId; private readonly int _workerIdShift; private readonly int _datacenterIdShift; private readonly int _timestampLeftShift; private readonly long _sequenceMask;
private readonly long _datacenterId;
private readonly long _workerId;
private long _sequence = 0L;
private long _lastTimestamp = -1L;
private readonly object _lock = new object();
public SnowflakeGenerator(long datacenterId, long workerId)
{
_epoch = 1288834974657L;
_datacenterIdBits = 5;
_workerIdBits = 5;
_sequenceBits = 12;
_maxDatacenterId = -1L ^ (-1L << _datacenterIdBits);
_maxWorkerId = -1L ^ (-1L << _workerIdBits);
_workerIdShift = _sequenceBits;
_datacenterIdShift = _sequenceBits + _workerIdBits;
_timestampLeftShift = _sequenceBits + _workerIdBits + _datacenterIdBits;
_sequenceMask = -1L ^ (-1L << _sequenceBits);
if (datacenterId > _maxDatacenterId || datacenterId < 0)
{
throw new ArgumentException($"datacenterId can't be greater than {_maxDatacenterId} or less than 0");
}
if (workerId > _maxWorkerId || workerId < 0)
{
throw new ArgumentException($"workerId can't be greater than {_maxWorkerId} or less than 0");
}
_datacenterId = datacenterId;
_workerId = workerId;
}
public long NextId()
{
lock (_lock)
{
var timestamp = TimeGen();
if (timestamp < _lastTimestamp)
{
throw new Exception("Clock moved backwards. Refusing to generate id");
}
if (_lastTimestamp == timestamp)
{
_sequence = (_sequence + 1) & _sequenceMask;
if (_sequence == 0)
{
timestamp = TilNextMillis(_lastTimestamp);
}
}
else
{
_sequence = 0L;
}
_lastTimestamp = timestamp;
return ((timestamp - _epoch) << _timestampLeftShift) |
(_datacenterId << _datacenterIdShift) |
(_workerId << _workerIdShift) |
_sequence;
}
}
private long TilNextMillis(long lastTimestamp)
{
var timestamp = TimeGen();
while (timestamp <= lastTimestamp)
{
timestamp = TimeGen();
}
return timestamp;
}
private long TimeGen()
{
return DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
}
}
// Usage class Program { static void Main(string[] args) { var generator = new SnowflakeGenerator(1, 1); var id = generator.NextId(); Console.WriteLine($"Generated Snowflake ID: {id}"); } }
1
2go package main
import ( "fmt" "sync" "time" )
type SnowflakeGenerator struct { epoch int64 datacenterIdBits uint workerIdBits uint sequenceBits uint maxDatacenterId int64 maxWorkerId int64 workerIdShift uint datacenterIdShift uint timestampLeftShift uint sequenceMask int64
datacenterId int64
workerId int64
sequence int64
lastTimestamp int64
lock sync.Mutex
}
func NewSnowflakeGenerator(datacenterId, workerId int64) (*SnowflakeGenerator, error) { g := &SnowflakeGenerator{ epoch: 1288834974657, datacenterIdBits: 5, workerIdBits: 5, sequenceBits: 12, lastTimestamp: -1, }
g.maxDatacenterId = -1 ^ (-1 << g.datacenterIdBits)
g.maxWorkerId = -1 ^ (-1 << g.workerIdBits)
g.workerIdShift = g.sequenceBits
g.datacenterIdShift = g.sequenceBits + g.workerIdBits
g.timestampLeftShift = g.sequenceBits + g.workerIdBits + g.datacenterIdBits
g.sequenceMask = -1 ^ (-1 << g.sequenceBits)
if datacenterId > g.maxDatacenterId || datacenterId < 0 {
return nil, fmt.Errorf("datacenterId can't be greater than %d or less than 0", g.maxDatacenterId)
}
if workerId > g.maxWorkerId || workerId < 0 {
return nil, fmt.Errorf("workerId can't be greater than %d or less than 0", g.maxWorkerId)
}
g.datacenterId = datacenterId
g.workerId = workerId
return g, nil
}
func (g *SnowflakeGenerator) NextId() (int64, error) { g.lock.Lock() defer g.lock.Unlock()
timestamp := g.timeGen()
if timestamp < g.lastTimestamp {
return 0, fmt.Errorf("clock moved backwards, refusing to generate id")
}
if g.lastTimestamp == timestamp {
g.sequence = (g.sequence + 1) & g.sequenceMask
if g.sequence == 0 {
timestamp = g.tilNextMillis(g.lastTimestamp)
}
} else {
g.sequence = 0
}
g.lastTimestamp = timestamp
return ((timestamp - g.epoch) << g.timestampLeftShift) |
(g.datacenterId << g.datacenterIdShift) |
(g.workerId << g.workerIdShift) |
g.sequence, nil
}
func (g *SnowflakeGenerator) tilNextMillis(lastTimestamp int64) int64 { timestamp := g.timeGen() for timestamp <= lastTimestamp { timestamp = g.timeGen() } return timestamp }
func (g *SnowflakeGenerator) timeGen() int64 { return time.Now().UnixNano() / int64(time.Millisecond) }
func main() { generator, err := NewSnowflakeGenerator(1, 1) if err != nil { fmt.Printf("Error creating generator: %v\n", err) return }
id, err := generator.NextId()
if err != nil {
fmt.Printf("Error generating ID: %v\n", err)
return
}
fmt.Printf("Generated Snowflake ID: %d\n", id)
}
1
2### Diagram
3
4Hapa kuna uwakilishi wa picha wa muundo wa ID ya Snowflake:
5
6<svg width="600" height="100" xmlns="http://www.w3.org/2000/svg">
7 <rect x="0" y="0" width="380" height="50" fill="#4299e1"/>
8 <text x="190" y="30" font-family="Arial" fontSize="14" fill="white" textAnchor="middle">Muda (41 bits)</text>
9
10 <rect x="380" y="0" width="90" height="50" fill="#48bb78"/>
11 <text x="425" y="30" font-family="Arial" fontSize="14" fill="white" textAnchor="middle">Kitambulisho cha Mashine (10 bits)</text>
12
13 <rect x="470" y="0" width="130" height="50" fill="#ed8936"/>
14 <text x="535" y="30" font-family="Arial" fontSize="14" fill="white" textAnchor="middle">Mfululizo (12 bits)</text>
15
16 <text x="300" y="80" font-family="Arial" fontSize="16" fill="black" textAnchor="middle">Muundo wa ID ya Snowflake wa 64-bit</text>
17</svg>
18
19### References
20
211. "Kutangaza Snowflake." Blog ya Uhandisi ya Twitter, https://blog.twitter.com/engineering/en_us/a/2010/announcing-snowflake
222. "ID ya Snowflake." Wikipedia, https://en.wikipedia.org/wiki/Snowflake_ID
233. "Uzazi wa ID iliyosambazwa katika Microservices." Medium, https://medium.com/swlh/distributed-id-generation-in-microservices-b6ce9a8dd93f
24Maoni
Bonyeza toast ya maoni kuanza kutoa maoni kuhusu zana hii
Zana Zinazohusiana
Gundua zana zaidi ambazo zinaweza kuwa na manufaa kwa mtiririko wako wa kazi