Hi, Loic, I also suspect the bottleneck is not the CPU. I stored the journal and the data on the same HDD, so the write performance is very poor. Is it the possible limiting factor in my system? Now I am trying to save the journal to SSD insted of the HDD. I will report the results later if I got better results. By the way, is there any commands or methods to know whether the CPU is really accelerated? Or I can only know that from the results of benchmark? I am worry about that the ISA erasure profile is not read by the Ceph... Thanks, Derek 2015-08-07 17:25 GMT+08:00 Loic Dachary <loic@xxxxxxxxxxx>: > Hi, > > On 07/08/2015 03:26, Derek Su wrote: >> Hello, Loic >> the following is my steps and configurations: >> (1) The 11 osd and 3 monitors were ran in the docker container on the >> same host machine. >> (2) Each osd had one 1T HDD. >> >> (3) I set the erasure coding pool profiles: >> ## Jerasure, reed-soloman >> $ ceph osd erasure-code-profile set reed_k4m2_A k=4 m=2 >> directory=/usr/lib64/ceph/erasure-code >> >> ## ISA, reed-soloman >> ceph osd erasure-code-profile set reed_k4m2_isa_A k=4 m=2 >> directory=/usr/lib64/ceph/erasure-code plugin=isa >> technique=reed_sol_van >> >> (4) Then, the erasure pools were created: >> ## Jerasure, reed-soloman >> $ $ceph osd pool create reed_k4m2_A_pool 128 128 erasure reed_k4m2_A >> >> ## ISA, reed-soloman >> $ ceph osd pool create reed_k4m2_isa_A_pool 128 128 erasure reed_k4m2_isa_A >> >> (5) Then, I used the rados benchmark to test the write performance >> ## Jerasure, reed-soloman >> rados bench -p reed_k4m2_A_pool 500 write --no-cleanup >> >> ## ISA, reed-soloman >> rados bench -p reed_k4m2_isa_A_pool write --no-cleanup >> >> ---- >> The results: >> (1) Jerasure/Reed-Soloman >> Write throughput: 136.0 MB/S, Latency: 0.471 >> (2) ISA/Reed-Soloman >> Write throughput: 133.1 MB/S, Latency: 0.481 >> (3) Jerasure/cauchy_good >> Write throughput: 138.3 MB/S, Latency: 0.462 >> (4) ISA/cauchy >> Write throughput: 140.2 MB/S, Latency: 0.452 >> >> -- >> My CPU information: >> Intel(R) Xeon(R) CPU E3-1245 v3 @ 3.40GHz >> >> $ cat /proc/cpuinfo | grep flags >> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge >> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe >> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts >> rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq >> dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid >> sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx >> f16c rdrand lahf_lm abm ida arat epb pln pts dtherm tpr_shadow vnmi >> flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms >> invpcid xsaveopt >> >> Ram: 12 GiB >> >> >> The results of the performance tests seem there are no differences... >> > > I have not conducted such tests myself. I would however expect to see some difference because the ISA plugin goes faster than the Jerasure plugin for writes. Could it be that in the test you run the CPU is not the limiting factor ? > > Cheers > > >> Thanks, :) >> Derek >> >> 2015-08-06 20:31 GMT+08:00 Loic Dachary <loic@xxxxxxxxxxx>: >>> Hi, >>> >>> Could you please publish the benchmark results somewhere ? I should be able to figure out why you don't see a difference. >>> >>> Cheers >>> >>> On 06/08/2015 13:25, Derek Su wrote: >>>> Dear Mr. Dachary and all, >>>> >>>> Recently, I found your blog show the performance tests of erasure >>>> pools (http://dachary.org/?p=3042 , http://dachary.org/?p=3665). >>>> The results indicates the write throughput can be enhanced >>>> significantly using Intel xeon CPU. >>>> >>>> I tried to create an erasure pool with isa plugin, reed_sol_van >>>> technique, and k/m=4/2 on the Intel(R) Xeon(R) CPU E3-1245 v3 @ >>>> 3.40GHz machines. >>>> >>>> However, the results of the rados benchmark showed that there was no >>>> any difference between the jerasure and isa plugins. It seems very >>>> strange. >>>> >>>> Do I need to do other configurations in addition to only setting the >>>> erasure profile? >>>> In addition, how can I know the erasure pool is accelerated by ISA >>>> plugin exactly? Is there any command I can use? >>>> >>>> Thanks, :) >>>> >>>> Derek Su. >>>> >>> >>> -- >>> Loïc Dachary, Artisan Logiciel Libre >>> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > -- > Loïc Dachary, Artisan Logiciel Libre > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
