++CCing gluster-devel to have more eye on this problem. Susant ----- Original Message ----- > From: "Subrata Ghosh" <subrata.ghosh@xxxxxxxxxxxx> > To: "Susant Palai <spalai@xxxxxxxxxx> (spalai@xxxxxxxxxx)" <spalai@xxxxxxxxxx>, "Vijay Bellur <vbellur@xxxxxxxxxx> > (vbellur@xxxxxxxxxx)" <vbellur@xxxxxxxxxx> > Cc: "Subrata Ghosh" <subrata.ghosh@xxxxxxxxxxxx> > Sent: Sunday, 19 July, 2015 7:57:28 PM > Subject: Iusses with Random read/write > > Hi Vijay/Prashant, > > How you are you :). > > We need your immediate help / suggestion to meet our random I/IO > performance metrics. > Currently we have performance issues with random read/write - our basic > requirement 20 MB/sec for random I/O. > > We tried with both "iozone" and "fio", received almost same ( random I/O) > performance which is not meeting our fundamental I/IO requirements. > > Our use case is as below. > > "Application running on different cards Writes/Reads (random) continuous > files to the volume comprising with storage belonging from different cards > in the distributed system, where replica presence across cards and > applications are using non-local storages." > We have verified and identified the bottleneck mostly on Gluster Client side > inside the application, however gluster server to server I/O speed looks > enough good. Performance tuning on gluster server side would not expected to > help. > > We also cross verified checked using NFS client we are getting far better > performance, but we cannot use NFS client /libgfapi because of use case > limitation ( brick failures cases etc..) > > Please throw some lights or thoughts to improve gluster client to achieve > > 20 MB/Secs > > Observations: > > Fio: > > > lease find the test results of Random write & read in 2 APPs scenarios. > > Scenario > > APP_1 > > APP_2 > > File size > > No of AMC's > > Random-Write > > 3.06 MB/s > > 3.02 MB/s > > 100 MB > > 4 > > Random-Read > > 8.1 MB/s > > 8.4 MB/s > > 100 MB > > 4 > > > > Iozone: > > ./iozone -R -l 1 -u 1 -r 4k -s 2G -F /home/cdr/f1 | tee -a > /tmp/iozone_results.txt & > > > APP 1 > > APP2 > > File Size : 2GB > > File size : 2GB > > Record size = 4 Kbytes > > Record size = 4 Kbytes > > Output is in Kbytes/sec > > Output is in Kbytes/sec > > > > > Initial write 41061.78 > > Initial write 41167.36 > > > Rewrite 40395.64 > > Rewrite 40810.41 > > > Read 262685.69 > > Read 269644.62 > > > Re-read 263751.66 > > Re-read 270760.62 > > > Reverse Read 27715.72 > > Reverse Read 28604.22 > > > Stride read 83776.44 > > Stride read 84347.88 > > > Random read 16239.74 (15.8 MB/s ) > > Random read 15815.94 (15.4 MB/s ) > > > Mixed workload 16260.95 > > Mixed workload 15787.55 > > > Random write 3356.57 (3.3 MB/s ) > > Random write 3365.17 ( 3.3 MB/s) > > > Pwrite 40914.55 > > Pwrite 40692.34 > > > Pread 260613.83 > > Pread 269850.59 > > > Fwrite 40412.40 > > Fwrite 40369.78 > > > Fread 261506.61 > > Fread 267142.41 > > > > Some of the info on performance testing is at > http://www.gluster.org/community/documentation/index.php/Performance_Testing > Also pls check iozone limitations listed there. > > "WARNING: random I/O testing in iozone is very restricted by iozone > constraint that it must randomly read then randomly write the entire file! > This is not what we want - instead it should randomly read/write for some > fraction of file size or time duration, allowing us to spread out more on > the disk while not waiting too long for test to finish. This is why fio > (below) is the preferred test tool for random I/O workloads." > > _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
