I have run iozone on the same machine with GlusterFS and without
GlusterFS. (Local storage and Local storage via GlusterFS). I also
included the results from a remote machine. The remote machine and the
local glusterfs are the same. I am getting around 200 megabytes per
second on reads. My local storage will do around 1400 megabytes per
second. Any ideas why?
Todd
Without GlusterFS
iozone -a -i0 -i1 -s 8192m -r 16384 iozone.$$.tmp
Iozone: Performance Test of File I/O
Version $Revision: 3.315 $
Compiled for 64 bit mode.
Build: linux
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.
Run began: Mon Jun 7 12:54:52 2010
Auto Mode
File size set to 8388608 KB
Record Size 16384 KB
Command line used: iozone -a -i0 -i1 -s 8192m -r 16384 iozone.23908.tmp
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random
random bkwd record stride
KB reclen write rewrite read reread read
write read rewrite read fwrite frewrite fread freread
8388608 16384 961438 1053043 1471859 1496125
iozone test complete.
With Glusterfs
iozone -a -i0 -i1 -s 8192m -r 16384 iozone.$$.tmp
Iozone: Performance Test of File I/O
Version $Revision: 3.315 $
Compiled for 64 bit mode.
Build: linux
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.
Run began: Mon Jun 7 12:50:02 2010
Auto Mode
File size set to 8388608 KB
Record Size 16384 KB
Command line used: iozone -a -i0 -i1 -s 8192m -r 16384 iozone.23908.tmp
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random
random bkwd record stride
KB reclen write rewrite read reread read
write read rewrite read fwrite frewrite fread freread
8388608 16384 344149 578252 223183 201110
iozone test complete.
GlusterFS on remote machine.
iozone -a -i0 -i1 -s 8192m -r 16384 iozone.$$.tmp
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.
Run began: Mon Jun 7 12:59:19 2010
Auto Mode
File size set to 8388608 KB
Record Size 16384 KB
Command line used: iozone -a -i0 -i1 -s 8192m -r 16384 iozone.14702.tmp
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random
random bkwd record stride
KB reclen write rewrite read reread read
write read rewrite read fwrite frewrite fread freread
8388608 16384 596662 724340 227773 209839
iozone test complete.
## file auto generated by /usr/bin/glusterfs-volgen (mount.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name neuron --transport ib-verbs
192.168.0.117:/node01/storage
# TRANSPORT-TYPE ib-verbs
volume 192.168.0.117-1
type protocol/client
option transport-type ib-verbs
option remote-host 192.168.0.117
option transport.ib-verbs.port 1
option transport.remote-port 6997
option remote-subvolume brick1
end-volume
volume writebehind
type performance/write-behind
option cache-size 4MB
subvolumes 192.168.0.117-1
end-volume
volume readahead
type performance/read-ahead
option page-count 4
subvolumes writebehind
end-volume
volume iocache
type performance/io-cache
option cache-size 1GB
option cache-timeout 1
subvolumes readahead
end-volume
volume quickread
type performance/quick-read
option cache-timeout 1
option max-file-size 64kB
subvolumes iocache
end-volume
volume statprefetch
type performance/stat-prefetch
subvolumes quickread
end-volume
## file auto generated by /usr/bin/glusterfs-volgen (export.vol)
# Cmd line:
# $ /usr/bin/glusterfs-volgen --name neuron --transport ib-verbs
192.168.0.117:/node01/storage
volume posix1
type storage/posix
option directory /node01/storage
end-volume
volume locks1
type features/locks
subvolumes posix1
end-volume
volume brick1
type performance/io-threads
option thread-count 8
subvolumes locks1
end-volume
volume server-ib-verbs
type protocol/server
option transport-type ib-verbs
option auth.addr.brick1.allow *
option transport.ib-verbs.listen-port 6997
option transport.ib-verbs.port 1
subvolumes brick1
end-volume
