Please find [root at dsdb1 ~]# cat /proc/sys/vm/drop_caches 3 [root at dsdb1 ~]# dd if=/dev/zero of=/data/big.file bs=128k count=80k oflag=direct 81920+0 records in 81920+0 records out 10737418240 bytes (11 GB) copied, 521.553 seconds, 20.6 MB/s [root at dsdb1 ~]# [root at dsdb1 ~]# dd if=/dev/zero of=/data/big.file bs=128k count=80k iflag=direct dd: opening `/dev/zero': Invalid argument [root at dsdb1 ~]# dd of=/dev/null if=/data/big.file bs=128k iflag=direct 81920+0 records in 81920+0 records out 10737418240 bytes (11 GB) copied, 37.854 seconds, 284 MB/s [root at dsdb1 ~]# On Wed, Apr 20, 2011 at 11:05 AM, Joe Landman <landman at scalableinformatics.com> wrote: > On 04/20/2011 02:01 PM, Mohit Anchlia wrote: >> >> Is 128K block size right size given that file sizes I have is from 70K - >> 2MB? >> >> Please find >> >> [root at dsdb1 ~]# dd if=/dev/zero of=/data/big.file bs=128k count=80k >> 81920+0 records in >> 81920+0 records out >> 10737418240 bytes (11 GB) copied, 14.751 seconds, 728 MB/s > > > >> [root at dsdb1 ~]# echo 3> ?/proc/sys/vm/drop_caches >> [root at dsdb1 ~]# dd of=/dev/null if=/data/big.file bs=128k >> 81920+0 records in >> 81920+0 records out >> 10737418240 bytes (11 GB) copied, 3.10485 seconds, 3.5 GB/s > > Hmm ... this looks like it came from cache. ?4 drive RAID0's aren't even > remotely this fast. > > Add an oflag=direct to the first dd, and an iflag=direct to the second dd so > we can avoid the OS memory cache for the moment (looks like the driver isn't > respecting the drop caches command, or you had no space between the 3 and > the > sign). > > > >> >> On Wed, Apr 20, 2011 at 10:49 AM, Joe Landman >> <landman at scalableinformatics.com> ?wrote: >>> >>> On 04/20/2011 01:42 PM, Mohit Anchlia wrote: >>>> >>>> ?mount >>>> /dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw) >>>> proc on /proc type proc (rw) >>>> sysfs on /sys type sysfs (rw) >>>> devpts on /dev/pts type devpts (rw,gid=5,mode=620) >>>> /dev/sdb1 on /boot type ext3 (rw) >>>> tmpfs on /dev/shm type tmpfs (rw) >>>> none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) >>>> sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) >>>> /dev/sda1 on /data type ext3 (rw) >>>> glusterfs#dsdb1:/stress-volume on /data/mnt-stress type fuse >>>> (rw,allow_other,default_permissions,max_read=131072) >>> >>> ok ... >>> >>> so gluster is running atop /data/mnt-stress, which is on /dev/sda1 and is >>> ext3. >>> >>> Could you do this >>> >>> ? ? ? ?dd if=/dev/zero of=/data/big.file bs=128k count=80k >>> ? ? ? ?echo 3> ?/proc/sys/vm/drop_caches >>> ? ? ? ?dd of=/dev/null if=/data/big.file bs=128k >>> >>> so we can see the write and then read performance using 128k blocks? >>> >>> Also, since you are using the gluster native client, you don't get all >>> the >>> nice NFS caching bits. ?Gluster native client is somewhat slower than the >>> NFS client. >>> >>> So lets start with the write/read speed of the system before we deal with >>> the gluster side of things. >>> >>>> >>>> >>>> On Wed, Apr 20, 2011 at 10:39 AM, Joe Landman >>>> <landman at scalableinformatics.com> ? ?wrote: >>>>> >>>>> On 04/20/2011 01:35 PM, Mohit Anchlia wrote: >>>>>> >>>>>> Should that command be there by default? I couldn't find lsscsi >>>>> >>>>> How about >>>>> >>>>> ? ? ? ?mount >>>>> >>>>> output? >>>>> >>>>> >>>>> -- >>>>> Joseph Landman, Ph.D >>>>> Founder and CEO >>>>> Scalable Informatics Inc. >>>>> email: landman at scalableinformatics.com >>>>> web ?: http://scalableinformatics.com >>>>> ? ? ? http://scalableinformatics.com/sicluster >>>>> phone: +1 734 786 8423 x121 >>>>> fax ?: +1 866 888 3112 >>>>> cell : +1 734 612 4615 >>>>> >>> >>> >>> -- >>> Joseph Landman, Ph.D >>> Founder and CEO >>> Scalable Informatics Inc. >>> email: landman at scalableinformatics.com >>> web ?: http://scalableinformatics.com >>> ? ? ? http://scalableinformatics.com/sicluster >>> phone: +1 734 786 8423 x121 >>> fax ?: +1 866 888 3112 >>> cell : +1 734 612 4615 >>> > > > -- > Joseph Landman, Ph.D > Founder and CEO > Scalable Informatics Inc. > email: landman at scalableinformatics.com > web ?: http://scalableinformatics.com > ? ? ? http://scalableinformatics.com/sicluster > phone: +1 734 786 8423 x121 > fax ?: +1 866 888 3112 > cell : +1 734 612 4615 >
