On Mon, Nov 25, 2013 at 2:17 AM, Mark Nelson <mark.nelson@xxxxxxxxxxx> wrote:
Great Work! This is very exciting! Did you happen to try RADOS bench at different object sizes and concurrency levels?
Maybe can try it later. :-)
Mark
On 11/24/2013 03:01 AM, Haomai Wang wrote:
Hi all,*Rados Bench*
For Emperor
blueprint(http://wiki.ceph.com/01Planning/02Blueprints/Emperor/Add_LevelDB_support_to_ceph_cluster_backend_store),
I'm sorry to delay the progress. Now, I have done the most of the works
for the blueprint's goal. Because of sage's F
blueprint(http://wiki.ceph.com/index.php?title=01Planning/02Blueprints/Firefly/osd:_new_key%2F%2Fvalue_backend),
I need to adjust some codes to match it. The branch is
here(https://github.com/yuyuyu101/ceph/tree/wip/6173).
I have tested the LevelDB backend on three nodes(eight OSDs) and compare
it to FileStore(ext4). I just use intern benchmark tool "rados bench" to
get the comparison. The default ceph configurations is used and
replication size is 2. The filesystem is ext4 and no others changed. The
results is below:
*Bandwidth(MB/sec)*
*Average Latency*
*Max Latency*
*Min Latency*
*Stddev Latency*
*Stddev Bandwidth(MB/sec)*
*Max Bandwidth(MB/sec)*
*Min Bandwidth(MB/sec)*
*KVStore*
*FileStore*
*KVStore*
*FileStore*
*KVStore*
*FileStore*
*KVStore*
*FileStore*
*KVStore*
*FileStore*
*KVStore*
*FileStore*
*KVStore*
*FileStore*
*KVStore*
*FileStore*
*Write 30**Write 20*
24.590
23.495
4.87257
5.07716
14.752
13.0885
0.580851
0.605118
2.97708
3.30538
9.91938
10.5986
44
76
0
0
*Write 10*
23.515
23.064
3.39745
3.45711
11.6089
11.5996
0.169507
0.138595
2.58285
2.75962
9.14467
8.54156
44
40
0
0
*Write 5*
22.927
21.980
1.73815
1.8198
5.53792
6.46675
0.171028
0.143392
1.05982
1.20303
9.18403
8.74401
44
40
0
0
*Read 30*
19.680
20.017
1.01492
0.997019
3.10783
3.05008
0.143758
0.138161
0.561548
0.571459
5.92575
6.844
36
32
0
0
*Read 20*
65.852
60.688
1.80069
1.96009
9.30039
10.1146
0.115153
0.061657
*Read 10*
59.372
60.738
1.30479
1.28383
6.28435
8.21304
0.016843
0.012073
*Read 5*
65.502
55.814
0.608805
0.7087
3.3917
4.72626
0.016267
0.011998
*RBD Bench-Write*
64.176
54.928
0.307111
0.364077
1.76391
1.90182
0.017174
0.011999
Charts can be view here(http://img42.com/ziwjP+) and
(http://img42.com/LKhoo+)
From above, I'm feeling relieved that the LevelDB backend isn't
useless. Most of metrics are better and if increasing cache size for
LevelDB the results may be more attractive.
Even more, LevelDB backend is used by "KeyValueStore" and much of
optimizations can be done to improve performance such as increase
parallel threads or optimize io path.
Next, I use "rbd bench-write" to test. The result is pity:
*OPS/sec*
*Bytes/sec*
*KVStore*
*FileStore*
*KVStore*
*FileStore*
*Seq 4096 5*
27.42
716.55
111861.51
2492149.21
*Rand 4096 5*_______________________________________________
28.27
504
112331.42
1683151.29
Just because kv backend doesn't support read/write operation with
offset/length argument, each read/write operation will call a additional
read LevelDB api to do. Much of time is consumed by reading entire large
object in rbd situation. There exists some ways to change such as split
large object to multi small objects or save metadata to avoid read
arduous operation.
As sage mentioned in <osd: new key/value
backend>(http://wiki.ceph.com/index.php?title=01Planning/02Blueprints/Firefly/osd:_new_key%2F%2Fvalue_backend),
more kv backends can be added now and I look forward to more people
interested it. I think radosgw situation can fit in kv store in short ti
--
Best Regards,
Wheat
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Best Regards,
Wheat
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
