On 06/10/2010 08:52 PM, Todd Daugherty wrote: > I am not sure what you are saying? The default record size of > ib_write_bw is 64K. But here is 128K. > > Do you think that the QDR Infiniband is not doing 3.3 gigabytes per > seconds? And that is why I am only getting 500 megabytes per seconds > via GlusterFS? You think that it is something wrong with the QDR > setup? What are the actual commands that you want me run on the > machines? I will run them. > > Do you have a single stream GlusterFS that can go OVER 3.3 gigabytes > per second? What setup is it? > Excuse me for not being clear you need to run "iozone" with higher record sizes. Your earlier command suggested you used "16k" for creating a 8GB file. 16k chunk writes over 2.0.9 version has lot of performance penalty, i think that is the reason why you are seeing such a slowness. Please run the below commands "iozone -a -i0 -i1 -s 8192m -r 64k iozone.$$.tmp" "iozone -a -i0 -i1 -s 8192m -r 128k iozone.$$.tmp" "iozone -a -i0 -i1 -s 8192m -r 1m iozone.$$.tmp" Since your infiniband fabric looks quite alright and also your backend disk performing at 1GB/sec. We should be near to 700-850MB/sec over glusterfs for writes as well as reads, if not then its a tuning or configuration issue. Thanks -- Harshavardhana Gluster Inc - http://www.gluster.com +1(408)-770-1887, Ext-113 +1(408)-480-1730
