-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathatom.xml
More file actions
764 lines (687 loc) · 92.2 KB
/
atom.xml
File metadata and controls
764 lines (687 loc) · 92.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>行胜于言</title>
<link href="http://localhost:4000/atom.xml" rel="self"/>
<link href="http://localhost:4000"/>
<updated>2019-11-17T00:56:36+08:00</updated>
<id>http://localhost:4000</id>
<author>
<name>xiabo2011</name>
<email></email>
</author>
<entry>
<title>Dubbo一致性哈希遇到的坑</title>
<link href="http://localhost:4000/distributed/2018/09/02/dubbo-loadbalance/"/>
<updated>2018-09-02T08:01:01+08:00</updated>
<id>http://localhost:4000/distributed/2018/09/02/dubbo-loadbalance</id>
<content type="html"><p>关于dubbo的负载均衡策略已经有太多blog介绍了，这里说一下我在使用dubbo一致性哈希负载均衡策略时候遇到的一个坑。</p>
<p>在使用dubbo2.8.4版本consistenthash策略的时候，我们根据用户uid做哈希，理论上同一个用户的访问应该打到同一个provider节点上，
但实际上我们发现不同consumer节点过来的同一个uid的请求并不能打到同一个provider节点上。这是因为dubbo2.8.4版本
的一致哈希策略存在问题。如何解决？</p>
<ol>
<li>
<p>dubbo2.8.4实际上是2015年的一个老版本，对应的是当当的开源项目dubbox，我们可以将其升级为dubbo2.6.x（看着是版本号减小啊，
实际上是升级，感兴趣的可以研究一下dubbo版本号~），dubbo2.6.x已经修复了这个问题，但是问题是
实际升级并不会如想象般简单，由于dubbo服务众多，而且版本的不兼容（比如序列化/反序列化的不兼容），升级可能会困难重重。</p>
</li>
<li>
<p>另一种方法是我们实现dubbo的LoadBalance接口，提供一个新的一致哈希实现，我们先看一下dubbo2.8.4的一致性哈希的实现问题在哪。</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="nf">ConsistentHashSelector</span><span class="o">(</span><span class="nc">List</span><span class="o">&lt;</span><span class="nc">Invoker</span><span class="o">&lt;</span><span class="no">T</span><span class="o">&gt;&gt;</span> <span class="n">invokers</span><span class="o">,</span> <span class="nc">String</span> <span class="n">methodName</span><span class="o">,</span> <span class="kt">int</span> <span class="n">identityHashCode</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">virtualInvokers</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">TreeMap</span><span class="o">&lt;</span><span class="nc">Long</span><span class="o">,</span> <span class="nc">Invoker</span><span class="o">&lt;</span><span class="no">T</span><span class="o">&gt;&gt;();</span>
<span class="k">this</span><span class="o">.</span><span class="na">identityHashCode</span> <span class="o">=</span> <span class="nc">System</span><span class="o">.</span><span class="na">identityHashCode</span><span class="o">(</span><span class="n">invokers</span><span class="o">);</span>
<span class="no">URL</span> <span class="n">url</span> <span class="o">=</span> <span class="n">invokers</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="mi">0</span><span class="o">).</span><span class="na">getUrl</span><span class="o">();</span>
<span class="k">this</span><span class="o">.</span><span class="na">replicaNumber</span> <span class="o">=</span> <span class="n">url</span><span class="o">.</span><span class="na">getMethodParameter</span><span class="o">(</span><span class="n">methodName</span><span class="o">,</span> <span class="s">"hash.nodes"</span><span class="o">,</span> <span class="mi">160</span><span class="o">);</span>
<span class="nc">String</span><span class="o">[]</span> <span class="n">index</span> <span class="o">=</span> <span class="nc">Constants</span><span class="o">.</span><span class="na">COMMA_SPLIT_PATTERN</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="n">url</span><span class="o">.</span><span class="na">getMethodParameter</span><span class="o">(</span><span class="n">methodName</span><span class="o">,</span> <span class="s">"hash.arguments"</span><span class="o">,</span> <span class="s">"0"</span><span class="o">));</span>
<span class="n">argumentIndex</span> <span class="o">=</span> <span class="k">new</span> <span class="kt">int</span><span class="o">[</span><span class="n">index</span><span class="o">.</span><span class="na">length</span><span class="o">];</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">i</span> <span class="o">&lt;</span> <span class="n">index</span><span class="o">.</span><span class="na">length</span><span class="o">;</span> <span class="n">i</span> <span class="o">++)</span> <span class="o">{</span>
<span class="n">argumentIndex</span><span class="o">[</span><span class="n">i</span><span class="o">]</span> <span class="o">=</span> <span class="nc">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">index</span><span class="o">[</span><span class="n">i</span><span class="o">]);</span>
<span class="o">}</span>
<span class="k">for</span> <span class="o">(</span><span class="nc">Invoker</span><span class="o">&lt;</span><span class="no">T</span><span class="o">&gt;</span> <span class="n">invoker</span> <span class="o">:</span> <span class="n">invokers</span><span class="o">)</span> <span class="o">{</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">i</span> <span class="o">&lt;</span> <span class="n">replicaNumber</span> <span class="o">/</span> <span class="mi">4</span><span class="o">;</span> <span class="n">i</span><span class="o">++)</span> <span class="o">{</span>
<span class="kt">byte</span><span class="o">[]</span> <span class="n">digest</span> <span class="o">=</span> <span class="n">md5</span><span class="o">(</span><span class="n">invoker</span><span class="o">.</span><span class="na">getUrl</span><span class="o">().</span><span class="na">toFullString</span><span class="o">()</span> <span class="o">+</span> <span class="n">i</span><span class="o">);</span>
<span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">h</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">h</span> <span class="o">&lt;</span> <span class="mi">4</span><span class="o">;</span> <span class="n">h</span><span class="o">++)</span> <span class="o">{</span>
<span class="kt">long</span> <span class="n">m</span> <span class="o">=</span> <span class="n">hash</span><span class="o">(</span><span class="n">digest</span><span class="o">,</span> <span class="n">h</span><span class="o">);</span>
<span class="n">virtualInvokers</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="n">m</span><span class="o">,</span> <span class="n">invoker</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div> </div>
<p>上面代码中通过md5计算digest的时候使用的invoker的url实际包含了和provider相关的参数”&amp;pid=”以及和时间相关的参数”&amp;timestamp=”参数，
所以fix这个问题最直接的方法就是去掉这些参数影响，只要稍作修改即可。</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">byte</span><span class="o">[]</span> <span class="n">digest</span> <span class="o">=</span> <span class="n">md5</span><span class="o">(</span><span class="n">invoker</span><span class="o">.</span><span class="na">getUrl</span><span class="o">().</span><span class="na">getAddress</span><span class="o">()</span> <span class="o">+</span> <span class="n">i</span><span class="o">);</span>
</code></pre></div> </div>
</li>
</ol>
<p>源码中的ConsistentHashLoadBalance全部实现保持不变，但我们将类名修改为ConsistentHashLoadBalance2</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">ConsistentHashLoadBalance2</span> <span class="kd">extends</span> <span class="nc">AbstractLoadBalance</span> <span class="o">{</span>
<span class="o">...</span>
<span class="o">}</span>
</code></pre></div></div>
<p>然后我们在resources/META-INF/dubbo中定义com.alibaba.dubbo.rpc.cluster.LoadBalance文件，内容为</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>consistenthash2=xxx=com.xxx.ConsistentHashLoadBalance2
</code></pre></div></div>
<p>这样只要在consumer侧引入相应的修改，配置loadblance=”consistenthash2”即可。</p>
</content>
</entry>
<entry>
<title>Zookeeper Curator</title>
<link href="http://localhost:4000/distributed/2018/09/02/zookeeper-curator/"/>
<updated>2018-09-02T08:01:01+08:00</updated>
<id>http://localhost:4000/distributed/2018/09/02/zookeeper-curator</id>
<content type="html"><p>Curator是一个Zookeeper的客户端库，是一个分布式协调服务，来帮助你管理zookeeper的连接，减少使用zookeeper的复杂性。
它包含了一套高级API和通用的utilities，提供了一些recipes（如lock，barriers和cache等）和操作（如create, delete, getData等）,
这些都使得编写的应用可以更加鲁棒。</p>
<p>Curator最初是由Netfix实现和贡献的，后来被升级为Apache的顶级项目。</p>
<h2 id="curator-client">Curator Client</h2>
<p>和<a href="/distributed/2018/08/26/zookeeper-api/#more">上一篇</a>提到的使用Zookeeper Client类似，使用Curator Client，我们需要先创建一个CuratorFramework。</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">CuratorFramework</span> <span class="n">zkClient</span> <span class="o">=</span> <span class="nc">CuratorFrameworkFactory</span><span class="o">.</span><span class="na">newClient</span><span class="o">(</span>
<span class="n">connectorString</span><span class="o">,</span> <span class="c1">// servers address e.g.: "127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183"</span>
<span class="n">retryPolicy</span> <span class="c1">// retry policy for retrying operations in the event of disconnections</span>
<span class="o">);</span>
</code></pre></div></div>
<h2 id="fluent-api">Fluent API</h2>
<p>之所以称之为Fluent API是因为，Curator提供的API可以以链式调用的方式来调用。
如果我们使用标准Zookeeper API来同步创建一个znode的调用应该是这样的</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">zk</span><span class="o">.</span><span class="na">create</span><span class="o">(</span>
<span class="s">"/mypath"</span><span class="o">,</span>
<span class="k">new</span> <span class="kt">byte</span><span class="o">[</span><span class="mi">0</span><span class="o">],</span>
<span class="nc">ZooDefs</span><span class="o">.</span><span class="na">Ids</span><span class="o">.</span><span class="na">OPEN_ACL_UNSAFE</span><span class="o">,</span>
<span class="nc">CreateMode</span><span class="o">.</span><span class="na">PERSISTENT</span>
<span class="o">);</span>
</code></pre></div></div>
<p>而如果使用Curator的fluent API完成上面同样的事情应该像下面这样调用</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">zkClient</span><span class="o">.</span><span class="na">create</span><span class="o">()</span>
<span class="o">.</span><span class="na">withMode</span><span class="o">(</span><span class="nc">CreateMode</span><span class="o">.</span><span class="na">PERSISTENT</span><span class="o">)</span>
<span class="o">.</span><span class="na">forPath</span><span class="o">(</span><span class="s">"/mypath"</span><span class="o">,</span> <span class="k">new</span> <span class="kt">byte</span><span class="o">[</span><span class="mi">0</span><span class="o">]);</span>
</code></pre></div></div>
<p>这种模式的调用不仅适用create，也适用其他操作如delete，setData, getData, checkExists和getChildren</p>
<p>如果用异步的方式执行同样的操作，需要添加inBackground()调用:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">zkClient</span><span class="o">.</span><span class="na">create</span><span class="o">()</span>
<span class="o">.</span><span class="na">inBackground</span><span class="o">()</span>
<span class="o">.</span><span class="na">withMode</span><span class="o">(</span><span class="nc">CreateMode</span><span class="o">.</span><span class="na">PERSISTENT</span><span class="o">)</span>
<span class="o">.</span><span class="na">forPath</span><span class="o">(</span><span class="s">"/mypath"</span><span class="o">,</span> <span class="k">new</span> <span class="kt">byte</span><span class="o">[</span><span class="mi">0</span><span class="o">]);</span>
</code></pre></div></div>
<p>同时当zk创建好了znode，我们还需要创建一个或者多个listener来接收callback，来完成异步调用的过程。</p>
<p>同样我们也可以设置一个watcher，watcher触发的通知也是通过listener来完成的，通知将作为WATCHED event传递给listener</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">zkClient</span><span class="o">.</span><span class="na">getData</span><span class="o">().</span><span class="na">inBackground</span><span class="o">().</span><span class="na">watched</span><span class="o">().</span><span class="na">forPath</span><span class="o">(</span><span class="s">"/mypath"</span><span class="o">);</span>
</code></pre></div></div>
<h2 id="listener">Listener</h2>
<p>Curator使用listener来处理callbacks和watch notification。
实现一个处理callback和notification的listener，第一步我们需要按照下面的template</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">CuratorListener</span> <span class="n">masterListener</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">CuratorListener</span><span class="o">()</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">eventReceived</span><span class="o">(</span><span class="nc">CuratorFramework</span> <span class="n">client</span><span class="o">,</span> <span class="nc">CuratorEvent</span> <span class="n">event</span><span class="o">)</span> <span class="o">{</span>
<span class="k">try</span> <span class="o">{</span>
<span class="k">switch</span> <span class="o">(</span><span class="n">event</span><span class="o">.</span><span class="na">getType</span><span class="o">())</span> <span class="o">{</span>
<span class="k">case</span> <span class="nl">CHILDREN:</span>
<span class="o">...</span>
<span class="k">break</span><span class="o">;</span>
<span class="k">case</span> <span class="nl">CREATE:</span>
<span class="o">...</span>
<span class="k">break</span><span class="o">;</span>
<span class="k">case</span> <span class="nl">DELETE:</span>
<span class="o">...</span>
<span class="k">break</span><span class="o">;</span>
<span class="k">case</span> <span class="nl">WATCHED:</span>
<span class="o">...</span>
<span class="k">break</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">Exception</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
<span class="no">LOG</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="s">"Exception while processing event."</span><span class="o">,</span> <span class="n">e</span><span class="o">);</span>
<span class="k">try</span> <span class="o">{</span>
<span class="n">close</span><span class="o">();</span>
<span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">IOException</span> <span class="n">ioe</span><span class="o">)</span> <span class="o">{</span>
<span class="no">LOG</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="s">"IOException while closing."</span><span class="o">,</span> <span class="n">ioe</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">};</span>
</code></pre></div></div>
<p>下一步需要注册这个listener到Curator client</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">zkClient</span><span class="o">.</span><span class="na">getCuratorListenable</span><span class="o">().</span><span class="na">addListener</span><span class="o">(</span><span class="n">masterListener</span><span class="o">);</span>
</code></pre></div></div>
<p>我们还需要定义一个特殊listener来处理background线程出现异常的时候报的错误</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">UnhandledErrorListener</span> <span class="n">errorsListener</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">UnhandledErrorListener</span><span class="o">()</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">unhandledError</span><span class="o">(</span><span class="nc">String</span> <span class="n">message</span><span class="o">,</span> <span class="nc">Throwable</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
<span class="no">LOG</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="s">"Unrecoverable error: "</span> <span class="o">+</span> <span class="n">message</span><span class="o">,</span> <span class="n">e</span><span class="o">);</span>
<span class="k">try</span> <span class="o">{</span>
<span class="n">close</span><span class="o">();</span>
<span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">IOException</span> <span class="n">ioe</span><span class="o">)</span> <span class="o">{</span>
<span class="no">LOG</span><span class="o">.</span><span class="na">warn</span><span class="o">(</span> <span class="s">"Exception when closing."</span><span class="o">,</span> <span class="n">ioe</span> <span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">};</span>
</code></pre></div></div>
<p>同样也需要注册到Curator client</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">client</span><span class="o">.</span><span class="na">getUnhandledErrorListenable</span><span class="o">().</span><span class="na">addListener</span><span class="o">(</span><span class="n">errorsListener</span><span class="o">);</span>
</code></pre></div></div>
</content>
</entry>
<entry>
<title>Zookeeper API使用</title>
<link href="http://localhost:4000/distributed/2018/08/26/zookeeper-api/"/>
<updated>2018-08-26T08:01:01+08:00</updated>
<id>http://localhost:4000/distributed/2018/08/26/zookeeper-api</id>
<content type="html"><h2 id="zookeeper-session">Zookeeper session</h2>
<p>Zookeeper client通过创建一个session与Zookeeper server通信，如果与某一台server的连接断了，session会转移到与另一台server建立的连接。通过org.apache.zookeeper.Zookeeper类构建zookeeper session</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">Zookeeper</span><span class="o">(</span>
<span class="nc">String</span> <span class="n">connectString</span><span class="o">,</span> <span class="c1">// servers address e.g.: "127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183"</span>
<span class="kt">int</span> <span class="n">sessionTimeout</span><span class="o">,</span> <span class="c1">// session timeout in millisecond</span>
<span class="nc">Watcher</span> <span class="n">watcher</span> <span class="c1">// 用来接收session event, watcher是接口，需要传递一个实现接口的实例化object</span>
<span class="o">)</span>
</code></pre></div></div>
<p>为了能够接收zookeeper的通知，需要实现Watcher类，代码如下</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">Master</span> <span class="kd">implements</span> <span class="nc">Watcher</span> <span class="o">{</span>
<span class="kd">private</span> <span class="nc">ZooKeeper</span> <span class="n">zk</span><span class="o">;</span>
<span class="kd">private</span> <span class="nc">String</span> <span class="n">hostPort</span><span class="o">;</span>
<span class="kd">private</span> <span class="kd">volatile</span> <span class="kt">boolean</span> <span class="n">connected</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
<span class="kd">private</span> <span class="kd">volatile</span> <span class="kt">boolean</span> <span class="n">expired</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">startZK</span><span class="o">()</span> <span class="kd">throws</span> <span class="nc">IOException</span> <span class="o">{</span>
<span class="n">zk</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ZooKeeper</span><span class="o">(</span><span class="n">hostPort</span><span class="o">,</span> <span class="mi">10000</span><span class="o">,</span> <span class="k">this</span><span class="o">);</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">stopZK</span><span class="o">()</span> <span class="kd">throws</span> <span class="nc">InterruptedException</span> <span class="o">{</span>
<span class="n">zk</span><span class="o">.</span><span class="na">close</span><span class="o">();</span>
<span class="o">}</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">process</span><span class="o">(</span><span class="nc">WatchedEvent</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
<span class="n">logger</span><span class="o">.</span><span class="na">info</span><span class="o">(</span><span class="s">"Master processing event: {}"</span><span class="o">,</span> <span class="n">e</span><span class="o">.</span><span class="na">toString</span><span class="o">());</span>
<span class="k">if</span> <span class="o">(</span><span class="n">e</span><span class="o">.</span><span class="na">getType</span><span class="o">()</span> <span class="o">==</span> <span class="nc">Event</span><span class="o">.</span><span class="na">EventType</span><span class="o">.</span><span class="na">None</span><span class="o">)</span> <span class="o">{</span>
<span class="k">switch</span> <span class="o">(</span><span class="n">e</span><span class="o">.</span><span class="na">getState</span><span class="o">())</span> <span class="o">{</span>
<span class="k">case</span> <span class="nl">SyncConnected:</span>
<span class="n">connected</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
<span class="k">break</span><span class="o">;</span>
<span class="k">case</span> <span class="nl">Disconnected:</span>
<span class="n">connected</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
<span class="k">break</span><span class="o">;</span>
<span class="k">case</span> <span class="nl">Expired:</span>
<span class="n">expired</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
<span class="n">connected</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
<span class="n">logger</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="s">"Session expiration"</span><span class="o">);</span>
<span class="k">default</span><span class="o">:</span>
<span class="k">break</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">isConnected</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">connected</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">isExpired</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="n">expired</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>WatchEvent包含3个属性，1) EventType, 2) KeeperStatus, 3) path</p>
<ul>
<li>EventType取值包括NodeCreated, NodeDeleted, NodeDataChanged, NodeChildrenChanged, or None；
None表示WatchEvent用来记录zk session的状态变化</li>
<li>KeeperStatus在EventType为None时，表示path节点状态，取值包括Disconnected, SyncConnec ted, AuthFailed, ConnectedReadOnly, SaslAuthenticated, or Expired</li>
<li>path表示在EventType非None时的znode path</li>
</ul>
<h2 id="zookeeper的api">Zookeeper的API</h2>
<p>Zookeeper支持的API包括<strong><em>create</em></strong>, <strong><em>exists</em></strong>, <strong><em>getData</em></strong>, <strong><em>setData</em></strong>, <strong><em>getChildren</em></strong>, <strong><em>delete</em></strong>。
每个API都支持同步调用和异步调用。异步调用的接口增加了callback接口参数，调用方需要实现相应的callback接口，来处理函数调用产生的各种状态码。
下面是 <em>/master</em> znode创建的代码举例。</p>
<p>同步调用<strong><em>create</em></strong></p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="kd">private</span> <span class="kt">boolean</span> <span class="nf">checkMaster</span><span class="o">()</span> <span class="kd">throws</span> <span class="nc">InterruptedException</span> <span class="o">{</span>
<span class="c1">// Logic for checking /master created correctly</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">createMaster</span><span class="o">()</span> <span class="kd">throws</span> <span class="nc">InterruptedException</span> <span class="o">{</span>
<span class="k">while</span> <span class="o">(</span><span class="kc">true</span><span class="o">)</span> <span class="o">{</span>
<span class="k">try</span> <span class="o">{</span>
<span class="n">zk</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="s">"/master"</span><span class="o">,</span> <span class="s">"masterData"</span><span class="o">.</span><span class="na">getBytes</span><span class="o">(),</span> <span class="nc">ZooDefs</span><span class="o">.</span><span class="na">Ids</span><span class="o">.</span><span class="na">OPEN_ACL_UNSAFE</span><span class="o">,</span> <span class="nc">CreateMode</span><span class="o">.</span><span class="na">EPHEMERAL</span><span class="o">);</span>
<span class="n">isLeader</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
<span class="k">break</span><span class="o">;</span>
<span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">KeeperException</span><span class="o">.</span><span class="na">NodeExistsException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
<span class="n">isLeader</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
<span class="k">break</span><span class="o">;</span>
<span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">KeeperException</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
<span class="n">e</span><span class="o">.</span><span class="na">printStackTrace</span><span class="o">();</span>
<span class="o">}</span>
<span class="k">if</span> <span class="o">(</span><span class="n">checkMaster</span><span class="o">())</span> <span class="o">{</span>
<span class="k">break</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>异步调用<strong><em>create</em></strong></p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">createMasterAsync</span><span class="o">()</span> <span class="o">{</span>
<span class="n">zk</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="s">"/master"</span><span class="o">,</span> <span class="s">"masterData"</span><span class="o">.</span><span class="na">getBytes</span><span class="o">(),</span> <span class="nc">ZooDefs</span><span class="o">.</span><span class="na">Ids</span><span class="o">.</span><span class="na">OPEN_ACL_UNSAFE</span><span class="o">,</span> <span class="nc">CreateMode</span><span class="o">.</span><span class="na">EPHEMERAL</span><span class="o">,</span> <span class="n">masterCreateCallback</span><span class="o">,</span> <span class="kc">null</span><span class="o">);</span>
<span class="o">}</span>
<span class="kd">private</span> <span class="nc">AsyncCallback</span><span class="o">.</span><span class="na">StringCallback</span> <span class="n">masterCreateCallback</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">AsyncCallback</span><span class="o">.</span><span class="na">StringCallback</span><span class="o">()</span> <span class="o">{</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">processResult</span><span class="o">(</span><span class="kt">int</span> <span class="n">rc</span><span class="o">,</span> <span class="nc">String</span> <span class="n">path</span><span class="o">,</span> <span class="nc">Object</span> <span class="n">ctx</span><span class="o">,</span> <span class="nc">String</span> <span class="n">name</span><span class="o">)</span> <span class="o">{</span>
<span class="k">switch</span> <span class="o">(</span><span class="nc">KeeperException</span><span class="o">.</span><span class="na">Code</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">rc</span><span class="o">))</span> <span class="o">{</span>
<span class="k">case</span> <span class="nl">OK:</span>
<span class="n">status</span> <span class="o">=</span> <span class="nc">Status</span><span class="o">.</span><span class="na">ELECTED</span><span class="o">;</span>
<span class="n">takeLeadership</span><span class="o">();</span>
<span class="n">logger</span><span class="o">.</span><span class="na">info</span><span class="o">(</span><span class="s">"I'm the leader"</span><span class="o">);</span>
<span class="k">break</span><span class="o">;</span>
<span class="k">case</span> <span class="nl">NODEEXISTS:</span>
<span class="n">status</span> <span class="o">=</span> <span class="nc">Status</span><span class="o">.</span><span class="na">NOT_ELECTED</span><span class="o">;</span>
<span class="n">masterExistsAsync</span><span class="o">();</span>
<span class="k">break</span><span class="o">;</span>
<span class="k">case</span> <span class="nl">CONNECTIONLOSS:</span>
<span class="n">logger</span><span class="o">.</span><span class="na">warn</span><span class="o">(</span><span class="s">"Connection Loss"</span><span class="o">);</span>
<span class="n">checkMasterAsync</span><span class="o">();</span>
<span class="k">break</span><span class="o">;</span>
<span class="k">default</span><span class="o">:</span>
<span class="n">status</span> <span class="o">=</span> <span class="nc">Status</span><span class="o">.</span><span class="na">NOT_ELECTED</span><span class="o">;</span>
<span class="n">logger</span><span class="o">.</span><span class="na">error</span><span class="o">(</span><span class="s">"'create' failed"</span><span class="o">,</span> <span class="nc">KeeperException</span><span class="o">.</span><span class="na">create</span><span class="o">(</span><span class="nc">KeeperException</span><span class="o">.</span><span class="na">Code</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">rc</span><span class="o">),</span> <span class="n">path</span><span class="o">));</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">};</span>
</code></pre></div></div>
<p>一般推荐使用异步API，因为它无需等待同步的create操作完成，线程不会阻塞，不用关心InterruptedException</p>
<h2 id="处理znode状态改变">处理znode状态改变</h2>
<p>对于Zookeeper API的调用，调用方一般遵循固定的pattern</p>
<ol>
<li>调用API异步方法</li>
<li>实现callback object，作为参数传给异步API方法</li>
<li>如果方法需要提供Watcher参数，实现一个Watcher传递给异步API方法</li>
</ol>
<p>代码举例：</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="kd">public</span> <span class="kt">void</span> <span class="nf">masterExistsAsnyc</span><span class="o">()</span> <span class="o">{</span>
<span class="n">zk</span><span class="o">.</span><span class="na">exists</span><span class="o">(</span><span class="s">"/master"</span><span class="o">,</span> <span class="n">masterExistsWatcher</span><span class="o">,</span> <span class="n">masterExistsCallback</span><span class="o">,</span> <span class="kc">null</span><span class="o">);</span>
<span class="o">}</span>
<span class="kd">private</span> <span class="nc">Watcher</span> <span class="n">masterExistsWatcher</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Watcher</span><span class="o">()</span> <span class="o">{</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">process</span><span class="o">(</span><span class="nc">WatchedEvent</span> <span class="n">event</span><span class="o">)</span> <span class="o">{</span>
<span class="k">if</span> <span class="o">(</span><span class="n">event</span><span class="o">.</span><span class="na">getType</span><span class="o">()</span> <span class="o">==</span> <span class="nc">Event</span><span class="o">.</span><span class="na">EventType</span><span class="o">.</span><span class="na">NodeDeleted</span><span class="o">)</span> <span class="o">{</span>
<span class="k">assert</span> <span class="s">"/master"</span><span class="o">.</span><span class="na">equals</span><span class="o">(</span><span class="n">event</span><span class="o">.</span><span class="na">getPath</span><span class="o">());</span>
<span class="n">createMasterAsync</span><span class="o">();</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">};</span>
<span class="kd">private</span> <span class="nc">AsyncCallback</span><span class="o">.</span><span class="na">StatCallback</span> <span class="n">masterExistsCallback</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">AsyncCallback</span><span class="o">.</span><span class="na">StatCallback</span><span class="o">()</span> <span class="o">{</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">processResult</span><span class="o">(</span><span class="kt">int</span> <span class="n">rc</span><span class="o">,</span> <span class="nc">String</span> <span class="n">path</span><span class="o">,</span> <span class="nc">Object</span> <span class="n">ctx</span><span class="o">,</span> <span class="nc">Stat</span> <span class="n">stat</span><span class="o">)</span> <span class="o">{</span>
<span class="k">switch</span> <span class="o">(</span><span class="nc">KeeperException</span><span class="o">.</span><span class="na">Code</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">rc</span><span class="o">))</span> <span class="o">{</span>
<span class="k">case</span> <span class="nl">OK:</span>
<span class="n">looger</span><span class="o">.</span><span class="na">info</span><span class="o">(</span><span class="s">"
break;
case CONNECTIONLOSS:
logger.warn("</span><span class="nc">Connection</span> <span class="nc">Loss</span><span class="s">");
masterExistsA();
break;
case NONODE:
createMasterAsync();
logger.warn("</span><span class="nc">Previous</span> <span class="n">master</span> <span class="n">is</span> <span class="n">gone</span><span class="o">,</span> <span class="n">run</span> <span class="k">for</span> <span class="k">new</span> <span class="n">master</span><span class="err">"</span><span class="o">);</span>
<span class="k">default</span><span class="o">:</span>
<span class="n">checkMasterAsync</span><span class="o">();</span>
<span class="k">break</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">};</span>
</code></pre></div></div>
</content>
</entry>
<entry>
<title>在本地私链上运行智能合约</title>
<link href="http://localhost:4000/blockchain/2018/06/30/first-smart-contract/"/>
<updated>2018-06-30T08:01:01+08:00</updated>
<id>http://localhost:4000/blockchain/2018/06/30/first-smart-contract</id>
<content type="html"><h2 id="准备工作">准备工作</h2>
<ul>
<li>根据上一篇blog搭建私链</li>
<li>下载 <a href="https://github.com/ethereum/remix-ide/tree/gh-pages">remix-ide</a>，我们这里要下载<em>gh-pages</em>分支，该分支为最新的稳定的build。remix-ide是一个用来编译运行智能合约的ide工具</li>
</ul>
<h2 id="启动私链节点">启动私链节点</h2>
<p>相比于上一篇blog提到的启动命令，这里加入rpc相关的参数</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ geth --datadir "$ethereum_home/eth2" --port 30304 --nodiscover --networkid 1234 --rpcport 8545 --rpccorsdomain "*" --unlock 0 console 2&gt;&gt;eth2.log
</code></pre></div></div>
<p>”–unlock 0”表示要求unlock eth.accounts[0]，在提示输入Passphrase后就会解锁eth.accounts[0]</p>
<h2 id="配置remix-ide">配置remix-ide</h2>
<ul>
<li>Run-&gt;Environment: Web3 Provider</li>
<li>Web3 Provider Endpoint: http://localhost:8545</li>
</ul>
<h2 id="新建一个solidity源文件">新建一个solidity源文件</h2>
<p>simple.sol</p>
<pre><code class="language-solidity">pragma solidity ^0.4.0;
contract Adder {
string name;
function setName(string _name) public {
name = _name;
}
function getnName() public constant returns (string) {
return name;
}
function add(int a, int b) public constant returns (int) {
return a + b;
}
}
</code></pre>
<p>点击ide上的<strong>create</strong>按钮，如果上面geth命令没有带上”–unlock 0”就会看到”creation of Adder errored”，需要我们unlock Account。
否则会看到”creation of Adder pending…“字样</p>
<p><img height="220" src="/images/blockchain1.png" />
如上图所示我们看到”1 pending transactions”，这是因为是一个transaction，需要写入区块链才能生效，所以我们需要开启挖矿miner.start()，
执行之后，我们会看到如下图所示结果
<img height="220" src="/images/blockchain2.png" />
从图中我们可以看出，remix-ide中会把需要tranaction写入区块的操作都显示成粉色按钮，这些操作都需要执行挖矿之后才能生效。</p>
<h2 id="从geth-console调用智能合约">从geth console调用智能合约</h2>
<p>我们可以从remix-ide上调用Adder中的方法，但是transaction操作需要执行挖矿才能进行，否则Transaction将一直处于pending状态；
同样我们可以在geth console上调用智能合约。我们需要拷贝下图中所示的ABI内容和上图中的Adder Address，分别赋值给变量abi和address。
<img height="220" src="/images/blockchain3.png" /></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&gt; var abi=[{"constant":true,"inputs":[],"name":"getName","outputs":[{"name":"","type":"string"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":true,"inputs":[{"name":"a","type":"int256"},{"name":"b","type":"int256"}],"name":"add","outputs":[{"name":"","type":"int256"}],"payable":false,"stateMutability":"view","type":"function"},{"constant":false,"inputs":[{"name":"_name","type":"string"}],"name":"setName","outputs":[],"payable":false,"stateMutability":"nonpayable","type":"function"}]
&gt; var address="0x3bf0b1d614df44a610865d62edc27bae12204c55"
&gt; var adder=eth.contract(abi).at(address)
&gt; eth.defaultAccount=eth.accounts[0]
&gt; personal.unlockAccount(eth.defaultAccount)
&gt; adder.setName("Bob")
&gt; eth.pendingTransactions
[{
blockHash: null,
blockNumber: null,
from: "0xd870d9a0cbfa37a53563f12a783fb697027f5668",
gas: 90000,
gasPrice: 18000000000,
hash: "0xef6837fcd5a92720013c964ccc634957635bd71b00281b01c49f8293fdcba93a",
input: "0xc47f002700000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000003426f620000000000000000000000000000000000000000000000000000000000",
nonce: 15,
r: "0xf736b15dd5741a6b95ed6b901eda7c673d27ff8ed02bb536035044523984476a",
s: "0x24c6c4e5d12997f234fd5ef92503928dcccc4330574d243fca6ccf9d30465711",
to: "0x3bf0b1d614df44a610865d62edc27bae12204c55",
transactionIndex: 0,
v: "0x1c",
value: 0
}]
&gt; miner.start()
&gt; eth.pendingTransactions
[]
&gt; miner.stop()
</code></pre></div></div>
</content>
</entry>
<entry>
<title>搭建本地以太坊私链</title>
<link href="http://localhost:4000/blockchain/2018/06/30/local-private-ethereum/"/>
<updated>2018-06-30T08:01:01+08:00</updated>
<id>http://localhost:4000/blockchain/2018/06/30/local-private-ethereum</id>
<content type="html"><h2 id="macos以太坊安装">macOS以太坊安装</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ brew tap ethereum/ethereum
$ brew install ethereum
</code></pre></div></div>
<h2 id="创建ethereum_home">创建ethereum_home</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ cd ~
$ mkdir ethereum
$ echo 'export ethereum_home=$HOME/ethereum' &gt;&gt; ~/.zshrc
$ source ~/.zshrc
</code></pre></div></div>
<h2 id="创建创世纪genesis文件">创建创世纪(Genesis)文件</h2>
<p>创世纪区块是区块链上的第一个区块，Genesis文件是一个JSON文件，描述了初始区块的特性。</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ cd $ethereum_home
$ touch genesis.json
</code></pre></div></div>
<p><strong>genesis.json</strong>内容如下：</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"nonce"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0x0000000000000042"</span><span class="p">,</span><span class="w">
</span><span class="nl">"timestamp"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0x0"</span><span class="p">,</span><span class="w">
</span><span class="nl">"parentHash"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0x0000000000000000000000000000000000000000000000000000000000000000"</span><span class="p">,</span><span class="w">
</span><span class="nl">"extraData"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0x"</span><span class="p">,</span><span class="w">
</span><span class="nl">"gasLimit"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0x8000000"</span><span class="p">,</span><span class="w">
</span><span class="nl">"difficulty"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0x400"</span><span class="p">,</span><span class="w">
</span><span class="nl">"config"</span><span class="p">:{},</span><span class="w">
</span><span class="nl">"mixhash"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0x0000000000000000000000000000000000000000000000000000000000000000"</span><span class="p">,</span><span class="w">
</span><span class="nl">"coinbase"</span><span class="p">:</span><span class="w"> </span><span class="s2">"0x3333333333333333333333333333333333333333"</span><span class="p">,</span><span class="w">
</span><span class="nl">"alloc"</span><span class="p">:</span><span class="w"> </span><span class="p">{}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<h2 id="启动以太坊node">启动以太坊Node</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ geth --datadir "$ethereum_home/eth1" init "$ethereum_home/genesis.json"
$ geth --datadir "$ethereum_home/eth1" --networkid 1234 console 2&gt;&gt;eth1.log
</code></pre></div></div>
<p>通过geth.ipc可以将多个终端attach同一个console</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ geth attach ipc:$ethereum_home/eth1/geth.ipc
</code></pre></div></div>
<h2 id="启动peer-node">启动Peer Node</h2>
<p>采用同样的genesis.json</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ geth --datadir "$ethereum_home/eth2" init "$ethereum_home/genesis.json"
$ geth --datadir "$ethereum_home/eth2" --port 30304 --nodiscover --networkid 1234 console 2&gt;&gt;eth2.log
</code></pre></div></div>
<p>通过admin.nodeInfo获取enode</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&gt; admin.nodeInfo
{
enode: "enode://30312c1a5e16a1698c3b901fec03d5ae81a094dd913f1a480d5d82fad0be26911b2d1abe2b63f0b21948aa0ab4815a70880ca7d8ac19c2abd047d2e75e77552a@[::]:30304?discport=0",
id: "30312c1a5e16a1698c3b901fec03d5ae81a094dd913f1a480d5d82fad0be26911b2d1abe2b63f0b21948aa0ab4815a70880ca7d8ac19c2abd047d2e75e77552a",
ip: "::",
listenAddr: "[::]:30304",
name: "Geth/v1.8.11-stable/darwin-amd64/go1.10.3",
ports: {
discovery: 0,
listener: 30304
},
protocols: {
eth: {
config: {
chainId: null,
eip150Hash: "0x0000000000000000000000000000000000000000000000000000000000000000"
},
difficulty: 1024,
genesis: "0x6231b02ac967dff9b0c799e956094408959de862d720d08776302ffefba0300b",
head: "0x6231b02ac967dff9b0c799e956094408959de862d720d08776302ffefba0300b",
network: 1234
}
}
}
</code></pre></div></div>
<p>切换到console 1, admin.addPeer</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&gt; admin.addPeer("enode://30312c1a5e16a1698c3b901fec03d5ae81a094dd913f1a480d5d82fad0be26911b2d1abe2b63f0b21948aa0ab4815a70880ca7d8ac19c2abd047d2e75e77552a@[::]:30304?discport=0")
&gt; addmin.peers
</code></pre></div></div>
<p>运行<em>admin.peers</em>，我们可以看到peer的信息了</p>
<h2 id="启动挖矿">启动挖矿</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&gt; eth.blockNumber
0
</code></pre></div></div>
<p>这里我们看到区块数目是0，因为还没有开始挖矿。挖矿前需要建立account，否则执行挖矿会报错（Error: etherbase missing）</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&gt; personal.newAccount('123456')
"0x9fc496826bc0745af978c7cad7d90ddee06a6293"
&gt; miner.start(1)
&gt; eth.blockNumber
4
&gt; eth.blockNumber
6
</code></pre></div></div>
<p>我们可以在两个Node的Console上都可以看到blockNumber在不断增加，虽然只是在一个Node上进行挖矿。start(1)这里的1代表一个线程挖矿。
通过下面命令可以获取block信息和最新的block对应的miner</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&gt; eth.getBlock(5)
&gt; eth.getBlock(eth.blockNumber).miner
</code></pre></div></div>
<p>停止挖矿</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">&gt;</span> miner.stop<span class="o">()</span>
</code></pre></div></div>
<h2 id="测试交易">测试交易</h2>
<p>新建一个账户，查看其余额</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&gt; personal.newAccount("123456)
0xd870d9a0cbfa37a53563f12a783fb697027f5668
&gt; eth.getBalance("0xd870d9a0cbfa37a53563f12a783fb697027f5668")
0
</code></pre></div></div>
<p>查看之前挖矿的账户余额，用之前账户转20 Wei</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&gt; eth.getBalance("0x9fc496826bc0745af978c7cad7d90ddee06a6293")
1.01968750000000000002e+21
&gt; eth.sendTransaction({from:"0x9fc496826bc0745af978c7cad7d90ddee06a6293", to:"0xd870d9a0cbfa37a53563f12a783fb697027f5668",value:20})
Error: authentication needed: password or unlock
at web3.js:3143:20
at web3.js:6347:15
at web3.js:5081:36
at &lt;anonymous&gt;:1:1
</code></pre></div></div>
<p>unlockAccount</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&gt; personal.unlockAccount("0x9fc496826bc0745af978c7cad7d90ddee06a6293")
&gt; eth.sendTransaction({from:"0x9fc496826bc0745af978c7cad7d90ddee06a6293", to:"0xd870d9a0cbfa37a53563f12a783fb697027f5668",value:20})
</code></pre></div></div>
<p>这时候我们再次查看新账号余额会发现仍旧是0，这是因为我们之前停止了挖矿，重新开启挖矿，过几秒之后我们发现新账号余额增加了20 wei</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&gt; eth.getBalance("0xd870d9a0cbfa37a53563f12a783fb697027f5668")
0
&gt; miner.start(1)
&gt; eth.getBalance("0xd870d9a0cbfa37a53563f12a783fb697027f5668")
20
</code></pre></div></div>
<h2 id="api">API</h2>
<ul>
<li>admin.*</li>
<li>personal.*</li>
<li>eth.*
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&gt; admin.nodeInfo
&gt; personal.newAccount()
&gt; personal.listAccounts
&gt; eth.blockNumber
</code></pre></div> </div>
</li>
</ul>
</content>
</entry>
<entry>
<title>Zookeeper基础</title>
<link href="http://localhost:4000/distributed/2018/06/03/zookeeper-basics/"/>
<updated>2018-06-03T08:01:01+08:00</updated>
<id>http://localhost:4000/distributed/2018/06/03/zookeeper-basics</id>
<content type="html"><p>一般设计一个分布式协同服务都会提供一组primitives的列表，对于每个primitive暴露一些接口来创建实例，比如分布式锁可以作为一个primitive，
它暴露了一些接口如create, acquire, release等。这种设计会存在一些问题：1）扩展性差，需要提供大量的primitives，还要不断扩展API；
2）灵活性差，对于不同应用适用性会差的较多</p>
<p>Zookeeper的设计并没有直接暴露primitives，而是暴露了一组类似操作文件系统的API来帮助不同应用构建自己的primitives。Zookeeper允许操作
小的数据节点，被称为<strong><em>znode</em></strong>。<strong><em>znode</em></strong>按照树的层级关系组织起来，如下图所示：</p>
<p><img height="220" src="/images/zookeeper1.png" /></p>
<p>Zookeeper的znode中的数据是以byte array形式存储的，byte arrays的具体格式（如Protocol Buffers, Thrift, Avr）是由具体应用自己指定的，
Zookeeper本身不支持对各种格式parse。</p>
<h2 id="zookeeper支持的api包括">Zookeeper支持的API包括：</h2>
<ul>
<li>create /path data</li>
<li>delete /path</li>
<li>get /path</li>
<li>set /path data</li>
<li>ls /path</li>
<li>stat /path</li>
</ul>
<p>Note: 具体API操作说明可以通过help命令查看</p>
<h2 id="znode的模式">ZNode的模式</h2>
<ul>
<li>persistent
<ul>
<li><em>create</em>命令默认创建的模式，只有通过<em>delete</em>操作才能删除掉</li>
</ul>
</li>
<li>ephemeral
<ul>
<li>可以通过<em>create -e</em>命令创建，显示<em>delete</em>或连接session过期都能删除掉</li>
</ul>
</li>
<li>persistent_sequential
<ul>
<li>可以通过<em>create -s</em>命令创建，如连续三次执行<em>create -s /tasks/task- ““</em>会创建<em>task-0000000000</em>，<em>task-0000000001</em>，
<em>task-0000000002</em>三个节点</li>
</ul>
</li>
<li>ephemeral_sequential
<ul>
<li>可以通过<em>create -e -s</em>命令创建</li>
</ul>
</li>
</ul>
<h2 id="watch">Watch</h2>
<p>Zookeeper的client如何知道znode的数据或状态发生变化呢？Zookeeper采用notification的机制，允许client通过对znode设置一个watch来注册接收
notification。<em>Watch</em>是一次性的，一个watch只能接收一次notification。如下图所示：</p>
<p><img height="220" src="/images/zookeeper2.png" /></p>
<p>client每次收到notification之后都需要设置一个新的<em>watch</em>，这期间znode可能会发生新的变化，有人会觉得新的变化会被client错过，但实际上每次
设置<em>watch</em>都会伴随着一次读操作，所以不会错过任何状态变化。</p>
<h2 id="version">Version</h2>
<p>有些API操作需要client带上znode的version作为参数，只有version与server当前的version一致时候，操作才能生效，这样确保在并发操作的情况下
正确性，如图所示：</p>
<p><img height="220" src="/images/zookeeper3.png" /></p>
<h2 id="zookeeper架构">Zookeeper架构</h2>
<p>Zookeeper整体架构如下图所示：</p>
<p><img height="250" src="/images/zookeeper4.png" /></p>
<h3 id="zookeeper-quorum">Zookeeper Quorum</h3>
<p><em>quorum</em>是需要参加投票的最少的server个数，也是确保client数据安全保存了的最小server数目。
Zookeeper集群server的数量最好是奇数。</p>
<h3 id="sessions">Sessions</h3>
<p>Client提交到zookeeper的所有操作都是关联到某一个session的。当session结束时，session内创建的所有ephemeral节点全部被删除。
同一session内的API操作能确保FIFO。client建立的session如果失去了与当前server的连接，client library能确保client重新与其它server建立连接，
但仍沿用同一个session id。</p>
</content>
</entry>
<entry>
<title>Zookeeper服务器集群搭建</title>
<link href="http://localhost:4000/distributed/2018/06/02/zookeeper-getting-started/"/>
<updated>2018-06-02T08:01:01+08:00</updated>
<id>http://localhost:4000/distributed/2018/06/02/zookeeper-getting-started</id>
<content type="html"><h2 id="获取zookeeper">获取zookeeper</h2>
<p>从apache zookeeper官网上获取zookeeper</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>https://www.apache.org/dyn/closer.cgi/zookeeper/
</code></pre></div></div>
<h2 id="zkserversh">zkServer.sh</h2>
<h3 id="配置集群">配置集群</h3>
<p>可以在本机上模拟集群搭建，不同server用不同port即可。可以在 <strong>/conf</strong>目录下创建三个cfg文件：zoo1.cfg, zoo2,cfg, zoo3.cfg, 分别赋予
不同的clientPort（如2181，2182，2183）和dataDir (“./data/zk1”, “./data/zk2”, “./data/zk3”)，其他的配置保持一致，具体配置如下</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=./data/zk1
# the port at which the clients will connect
clientPort=2181
# the first port is for quorum communication, the second port is for leader election
server.1=localhost:2881:3881
server.2=localhost:2882:3882
server.3=localhost:2883:3883
</code></pre></div></div>
<p>在dataDir指定的路径下创建myid文件</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ echo 1 &gt; ./data/zk1/myid
$ echo 2 &gt; ./data/zk2/myid
$ echo 3 &gt; ./data/zk3/myid
</code></pre></div></div>
<h3 id="启动zk集群">启动zk集群</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ bin/zkServer.sh start conf/zoo1.cfg
$ bin/zkServer.sh start conf/zoo2.cfg
$ bin/zkServer.sh start conf/zoo3.cfg
</code></pre></div></div>
<p>zk server的log会记录到zookeeper.out。如果为了方便测试查看server log，可以采用下面命令启动server</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ bin/zkServer.sh start-foreground conf/zoo1.cfg
</code></pre></div></div>
<h3 id="集群测试">集群测试</h3>
<p>可以用zk自带的zkCli.sh对搭建的集群进行一些测试，如创建一个/tmp节点，然后添加test子节点数据。首先需要用<a href="#zk-client"><strong>zkCli.sh</strong></a>
连接到server。</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>...] create /tmp ""
...] create /tmp/test "abc"
...] get /tmp
...] get /tmp/test
...] delete /tmp/test
...] delete /tmp
</code></pre></div></div>
<h2 id="zkclish"><a name="zk-client"></a>zkCli.sh</h2>
<p>通过zkCli.sh连接zookeeper服务器的命令</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ bin/zkCli.sh -server 127.0.0.1:2181
</code></pre></div></div>
<p>这种方式存在的问题是一旦2181端口对应的server挂掉，client会一直尝试连接，并且失败。当2181端口对应的server重启后，client会报连接失败，
session过期，connection会被close掉，
所以-server参数最好带上所有的server地址。</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ bin/zkCli.sh -server 127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
</code></pre></div></div>
<p>当zookeeper集群有server有机器挂掉，存在下面几种情况</p>
<ul>
<li>如果client连接到的server挂了
<ul>
<li>该server是leader，则zookeeper在原来两个follower中重新选举leader后，client会连接到某台server上</li>
<li>该server是follower, client会与另外两台server建立连接</li>
</ul>
</li>
<li>如果不是client连接的那台server挂了
<ul>
<li>如果挂的server是leader，由于要重新选举，client的connection会被refused，client尝试reconnect，选举成功后，client会再连接上，
和之前是同一个session</li>
<li>如果挂的server不是leader，client的连接不受任何影响</li>
</ul>
</li>
<li>如果挂的server数目超过了多数，client将无法连接到server</li>
</ul>
</content>
</entry>
</feed>