I included a bunch of other info. If there is anything else that will help solve this problem I am very interested in getting to the bottom of this. Side note I built a GlusterFS cluster (2.0.9) from RAM disks (10 gig /dev/ram0 bricks) That performs quite well but still much less than the backend /dev/ram0 bricks. Todd InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev a0) Atto Technology SAS RAID Adapter Subsystem: Atto Technology Device 002d Kernel modules: esasraid 16 SATA 1 TB Disks on two of those Atto Cards so 8 disks each controller. (JMR PE SAN) dd if=/dev/zero of=/node01/zero bs=1M count=16386 oflag=direct 16386+0 records in 16386+0 records out 17181966336 bytes (17 GB) copied, 17.0376 s, 1.0 GB/s 6 Gigabytes of RAM glusterfs -V glusterfs 2.0.9 built on Apr 11 2010 20:40:04 Repository revision: v2.0.9 Copyright (c) 2006-2009 Gluster Inc. <http://www.gluster.com> GlusterFS comes with ABSOLUTELY NO WARRANTY. You may redistribute copies of GlusterFS under the terms of the GNU General Public License. cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz stepping : 5 cpu MHz : 1600.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid bogomips : 5319.77 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz stepping : 5 cpu MHz : 1600.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 1 cpu cores : 4 apicid : 2 initial apicid : 2 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid bogomips : 5319.07 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 2 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz stepping : 5 cpu MHz : 1600.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 2 cpu cores : 4 apicid : 4 initial apicid : 4 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid bogomips : 5319.07 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 3 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz stepping : 5 cpu MHz : 1600.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 3 cpu cores : 4 apicid : 6 initial apicid : 6 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid bogomips : 5319.07 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 4 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz stepping : 5 cpu MHz : 1600.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 4 apicid : 1 initial apicid : 1 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid bogomips : 5319.07 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 5 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz stepping : 5 cpu MHz : 2668.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 1 cpu cores : 4 apicid : 3 initial apicid : 3 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid bogomips : 5319.07 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 6 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz stepping : 5 cpu MHz : 1600.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 2 cpu cores : 4 apicid : 5 initial apicid : 5 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid bogomips : 5319.08 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz stepping : 5 cpu MHz : 1600.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid bogomips : 5319.07 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: cat /proc/meminfo MemTotal: 6060364 kB MemFree: 137120 kB Buffers: 176488 kB Cached: 5225760 kB SwapCached: 0 kB Active: 319020 kB Inactive: 5200428 kB Active(anon): 13216 kB Inactive(anon): 104236 kB Active(file): 305804 kB Inactive(file): 5096192 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 8110072 kB SwapFree: 8110072 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 117264 kB Mapped: 17232 kB Slab: 208168 kB SReclaimable: 177444 kB SUnreclaim: 30724 kB PageTables: 5492 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 11140252 kB Committed_AS: 290736 kB VmallocTotal: 34359738367 kB VmallocUsed: 320548 kB VmallocChunk: 34359397567 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 7744 kB DirectMap2M: 6275072 kB uname -a Linux axon07 2.6.29.4-167.fc11.x86_64 #1 SMP Wed May 27 17:27:08 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux lshw axon07 description: Desktop Computer product: X8SAX vendor: Supermicro version: 1234567890 serial: 1234567890 width: 64 bits capabilities: vsyscall64 vsyscall32 configuration: boot=normal chassis=desktop uuid=00020003-0004-0005-0006-000700080009 *-core description: Motherboard product: X8SAX vendor: Supermicro physical id: 0 version: 1234567890 serial: 1234567890 slot: To Be Filled By O.E.M. *-firmware description: BIOS vendor: American Megatrends Inc. physical id: 0 version: 1.0c (08/05/2009) size: 64KiB capacity: 4032KiB capabilities: isa pci pnp upgrade shadowing escd cdboot bootselect socketedrom edd int13floppy1200 int13floppy720 int13floppy2880 int5printscreen int9keyboard int14serial int17printer int10video acpi usb ls120boot zipboot biosbootspecification *-cpu description: CPU product: Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz vendor: Intel Corp. physical id: 4 bus info: cpu at 0 version: Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz serial: To Be Filled By O.E.M. slot: CPU 1 size: 1600MHz capacity: 1600MHz width: 64 bits clock: 133MHz capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp x86-64 constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid cpufreq *-cache:0 description: L1 cache physical id: 5 slot: L1-Cache size: 256KiB capacity: 256KiB capabilities: internal write-back data *-cache:1 description: L2 cache physical id: 6 slot: L2-Cache size: 1MiB capacity: 1MiB capabilities: internal write-back unified *-cache:2 description: L3 cache physical id: 7 slot: L3-Cache size: 8MiB capacity: 8MiB capabilities: internal write-back unified *-memory:0 description: System Memory physical id: 28 slot: System board or motherboard *-bank:0 description: DIMM 1333 MHz (0.8 ns) vendor: Manufacturer00 physical id: 0 serial: 00000000 slot: DIMM0 size: 2GiB width: 64 bits clock: 1333MHz (0.8ns) *-bank:1 description: DIMM 1333 MHz (0.8 ns) [empty] product: ModulePartNumber01 vendor: Manufacturer01 physical id: 1 serial: SerNum01 slot: DIMM1 width: 64 bits clock: 1333MHz (0.8ns) *-bank:2 description: DIMM 1333 MHz (0.8 ns) vendor: Manufacturer02 physical id: 2 serial: 00000000 slot: DIMM2 size: 2GiB width: 64 bits clock: 1333MHz (0.8ns) *-bank:3 description: DIMM 1333 MHz (0.8 ns) [empty] product: ModulePartNumber03 vendor: Manufacturer03 physical id: 3 serial: SerNum03 slot: DIMM3 width: 64 bits clock: 1333MHz (0.8ns) *-bank:4 description: DIMM 1333 MHz (0.8 ns) vendor: Manufacturer04 physical id: 4 serial: 00000000 slot: DIMM4 size: 2GiB width: 64 bits clock: 1333MHz (0.8ns) *-bank:5 description: DIMM 1333 MHz (0.8 ns) [empty] product: ModulePartNumber05 vendor: Manufacturer05 physical id: 5 serial: SerNum05 slot: DIMM5 width: 64 bits clock: 1333MHz (0.8ns) *-memory:1 UNCLAIMED description: Flash Memory physical id: 36 slot: System board or motherboard capacity: 4MiB *-bank UNCLAIMED description: FLASH Non-volatile 33 MHz (30.3 ns) product: 26DF321 vendor: ATMEL physical id: 0 size: 4MiB width: 8 bits clock: 33MHz (30.3ns) *-memory:2 UNCLAIMED physical id: 1 *-memory:3 UNCLAIMED physical id: 2 *-pci description: Host bridge product: X58 I/O Hub to ESI Port vendor: Intel Corporation physical id: 100 bus info: pci at 0000:00:00.0 version: 13 width: 32 bits clock: 33MHz *-pci:0 description: PCI bridge product: X58 I/O Hub PCI Express Root Port 1 vendor: Intel Corporation physical id: 1 bus info: pci at 0000:00:01.0 version: 13 width: 32 bits clock: 33MHz capabilities: pci msi pciexpress pm normal_decode bus_master cap_list configuration: driver=pcieport-driver resources: irq:24 *-pci description: PCI bridge product: 6702PXH PCI Express-to-PCI Bridge A vendor: Intel Corporation physical id: 0 bus info: pci at 0000:01:00.0 version: 09 width: 32 bits clock: 33MHz capabilities: pci pciexpress msi pm pcix normal_decode bus_master cap_list *-pci:1 description: PCI bridge product: X58 I/O Hub PCI Express Root Port 3 vendor: Intel Corporation physical id: 3 bus info: pci at 0000:00:03.0 version: 13 width: 32 bits clock: 33MHz capabilities: pci msi pciexpress pm normal_decode bus_master cap_list configuration: driver=pcieport-driver resources: irq:25 ioport:a000(size=8192) memory:fb500000-fb9fffff *-pci description: PCI bridge product: PEX 8533 32-lane, 6-port PCI Express Switch vendor: PLX Technology, Inc. physical id: 0 bus info: pci at 0000:03:00.0 version: aa width: 32 bits clock: 33MHz capabilities: pci pm msi pciexpress normal_decode bus_master cap_list configuration: driver=pcieport-driver resources: irq:30 memory:fb5e0000-fb5fffff ioport:a000(size=8192) memory:fb600000-fb9fffff *-pci:0 description: PCI bridge product: PEX 8533 32-lane, 6-port PCI Express Switch vendor: PLX Technology, Inc. physical id: 0 bus info: pci at 0000:04:00.0 version: aa width: 64 bits clock: 33MHz capabilities: pci pm msi pciexpress normal_decode bus_master cap_list configuration: driver=pcieport-driver resources: iomemory:1f10-1f0f irq:31 *-pci:1 description: PCI bridge product: PEX 8533 32-lane, 6-port PCI Express Switch vendor: PLX Technology, Inc. physical id: 1 bus info: pci at 0000:04:01.0 version: aa width: 64 bits clock: 33MHz capabilities: pci pm msi pciexpress normal_decode bus_master cap_list configuration: driver=pcieport-driver resources: iomemory:a1a10-a1a0f irq:32 ioport:a000(size=4096) memory:fb600000-fb7fffff *-storage description: RAID bus controller product: SAS RAID Adapter vendor: Atto Technology physical id: 0 bus info: pci at 0000:06:00.0 logical name: scsi4 version: 00 width: 64 bits clock: 33MHz capabilities: storage pm msi pciexpress bus_master cap_list rom scsi-host configuration: driver=esasraid latency=0 resources: irq:17 memory:fb780000-fb7fffff memory:fb700000-fb77ffff ioport:a800(size=256) memory:fb600000-fb6fffff(prefetchable) *-enclosure UNCLAIMED description: SCSI Enclosure product: PSAN SAS Exp vendor: JMR Elec physical id: 0.0.0 bus info: scsi at 4:0.0.0 version: 2.17 configuration: ansiversion=3 *-disk description: SCSI Disk product: test0100 vendor: ATTO physical id: 0.1.0 bus info: scsi at 4:0.1.0 logical name: /dev/sdb version: 0001 serial: c3da063f774df7d71d996b020ac16ca400000000 size: 7451GiB (8001GB) configuration: ansiversion=5 *-pci:2 description: PCI bridge product: PEX 8533 32-lane, 6-port PCI Express Switch vendor: PLX Technology, Inc. physical id: 9 bus info: pci at 0000:04:09.0 version: aa width: 64 bits clock: 33MHz capabilities: pci pm msi pciexpress normal_decode bus_master cap_list configuration: driver=pcieport-driver resources: iomemory:b1b10-b1b0f irq:33 ioport:b000(size=4096) memory:fb800000-fb9fffff *-storage description: RAID bus controller product: SAS RAID Adapter vendor: Atto Technology physical id: 0 bus info: pci at 0000:07:00.0 logical name: scsi5 version: 00 width: 64 bits clock: 33MHz capabilities: storage pm msi pciexpress bus_master cap_list rom scsi-host configuration: driver=esasraid latency=0 resources: irq:17 memory:fb980000-fb9fffff memory:fb900000-fb97ffff ioport:b800(size=256) memory:fb800000-fb8fffff(prefetchable) *-enclosure UNCLAIMED description: SCSI Enclosure product: PSAN SAS Exp vendor: JMR Elec physical id: 0.0.0 bus info: scsi at 5:0.0.0 version: 2.17 configuration: ansiversion=3 *-disk description: SCSI Disk product: test0200 vendor: ATTO physical id: 0.1.0 bus info: scsi at 5:0.1.0 logical name: /dev/sdc version: 0001 serial: cad720b12a0778fe517f60962c34285400000000 size: 7451GiB (8001GB) configuration: ansiversion=5 *-pci:2 description: PCI bridge product: X58 I/O Hub PCI Express Root Port 7 vendor: Intel Corporation physical id: 7 bus info: pci at 0000:00:07.0 version: 13 width: 32 bits clock: 33MHz capabilities: pci msi pciexpress pm normal_decode bus_master cap_list configuration: driver=pcieport-driver resources: irq:26 ioport:c000(size=4096) memory:fba00000-fbafffff ioport:d0000000(size=268435456) *-display description: VGA compatible controller product: RV710 [Radeon HD 4350] vendor: ATI Technologies Inc physical id: 0 bus info: pci at 0000:08:00.0 version: 00 width: 64 bits clock: 33MHz capabilities: pm pciexpress msi vga_controller bus_master cap_list rom configuration: driver=radeon latency=0 resources: irq:16 memory:d0000000-dfffffff(prefetchable) memory:fbaf0000-fbafffff ioport:c000(size=256) memory:fbac0000-fbadffff(prefetchable) *-multimedia description: Audio device product: R700 Audio Device [Radeon HD 4000 Series] vendor: ATI Technologies Inc physical id: 0.1 bus info: pci at 0000:08:00.1 version: 00 width: 64 bits clock: 33MHz capabilities: pm pciexpress msi bus_master cap_list configuration: driver=HDA Intel latency=0 resources: irq:17 memory:fbaec000-fbaeffff *-generic:0 UNCLAIMED description: PIC product: X58 I/O Hub System Management Registers vendor: Intel Corporation physical id: 14 bus info: pci at 0000:00:14.0 version: 13 width: 32 bits clock: 33MHz capabilities: pciexpress 8259 cap_list configuration: latency=0 *-generic:1 UNCLAIMED description: PIC product: X58 I/O Hub GPIO and Scratch Pad Registers vendor: Intel Corporation physical id: 14.1 bus info: pci at 0000:00:14.1 version: 13 width: 32 bits clock: 33MHz capabilities: pciexpress 8259 cap_list configuration: latency=0 *-generic:2 UNCLAIMED description: PIC product: X58 I/O Hub Control Status and RAS Registers vendor: Intel Corporation physical id: 14.2 bus info: pci at 0000:00:14.2 version: 13 width: 32 bits clock: 33MHz capabilities: pciexpress 8259 cap_list configuration: latency=0 *-generic:3 UNCLAIMED description: PIC product: X58 I/O Hub Throttle Registers vendor: Intel Corporation physical id: 14.3 bus info: pci at 0000:00:14.3 version: 13 width: 32 bits clock: 33MHz capabilities: 8259 configuration: latency=0 *-generic:4 UNCLAIMED description: System peripheral product: X58 Chipset QuickData Technology Device vendor: Intel Corporation physical id: 16 bus info: pci at 0000:00:16.0 version: 13 width: 64 bits clock: 33MHz capabilities: msix pciexpress pm bus_master cap_list configuration: latency=0 resources: memory:fb4fc000-fb4fffff *-generic:5 UNCLAIMED description: System peripheral product: X58 Chipset QuickData Technology Device vendor: Intel Corporation physical id: 16.1 bus info: pci at 0000:00:16.1 version: 13 width: 64 bits clock: 33MHz capabilities: msix pciexpress pm bus_master cap_list configuration: latency=0 resources: memory:fb4f8000-fb4fbfff *-generic:6 UNCLAIMED description: System peripheral product: X58 Chipset QuickData Technology Device vendor: Intel Corporation physical id: 16.2 bus info: pci at 0000:00:16.2 version: 13 width: 64 bits clock: 33MHz capabilities: msix pciexpress pm bus_master cap_list configuration: latency=0 resources: memory:fb4f4000-fb4f7fff *-generic:7 UNCLAIMED description: System peripheral product: X58 Chipset QuickData Technology Device vendor: Intel Corporation physical id: 16.3 bus info: pci at 0000:00:16.3 version: 13 width: 64 bits clock: 33MHz capabilities: msix pciexpress pm bus_master cap_list configuration: latency=0 resources: memory:fb4f0000-fb4f3fff *-generic:8 UNCLAIMED description: System peripheral product: X58 Chipset QuickData Technology Device vendor: Intel Corporation physical id: 16.4 bus info: pci at 0000:00:16.4 version: 13 width: 64 bits clock: 33MHz capabilities: msix pciexpress pm bus_master cap_list configuration: latency=0 resources: memory:fb4ec000-fb4effff *-generic:9 UNCLAIMED description: System peripheral product: X58 Chipset QuickData Technology Device vendor: Intel Corporation physical id: 16.5 bus info: pci at 0000:00:16.5 version: 13 width: 64 bits clock: 33MHz capabilities: msix pciexpress pm bus_master cap_list configuration: latency=0 resources: memory:fb4e8000-fb4ebfff *-generic:10 UNCLAIMED description: System peripheral product: X58 Chipset QuickData Technology Device vendor: Intel Corporation physical id: 16.6 bus info: pci at 0000:00:16.6 version: 13 width: 64 bits clock: 33MHz capabilities: msix pciexpress pm bus_master cap_list configuration: latency=0 resources: memory:fb4e4000-fb4e7fff *-generic:11 UNCLAIMED description: System peripheral product: DMA Engine vendor: Intel Corporation physical id: 16.7 bus info: pci at 0000:00:16.7 version: 13 width: 64 bits clock: 33MHz capabilities: msix pciexpress pm bus_master cap_list configuration: latency=0 resources: memory:fb4e0000-fb4e3fff *-usb:0 description: USB Controller product: 82801JI (ICH10 Family) USB UHCI Controller #4 vendor: Intel Corporation physical id: 1a bus info: pci at 0000:00:1a.0 version: 00 width: 32 bits clock: 33MHz capabilities: uhci bus_master cap_list configuration: driver=uhci_hcd latency=0 resources: irq:20 ioport:9c00(size=32) *-usbhost product: UHCI Host Controller vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd physical id: 1 bus info: usb at 3 logical name: /dev/usb3 version: 2.06 capabilities: usb-1.10 configuration: driver=hub slots=2 speed=12.0MB/s *-usb:1 description: USB Controller product: 82801JI (ICH10 Family) USB UHCI Controller #5 vendor: Intel Corporation physical id: 1a.1 bus info: pci at 0000:00:1a.1 version: 00 width: 32 bits clock: 33MHz capabilities: uhci bus_master cap_list configuration: driver=uhci_hcd latency=0 resources: irq:21 ioport:9880(size=32) *-usbhost product: UHCI Host Controller vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd physical id: 1 bus info: usb at 4 logical name: /dev/usb4 version: 2.06 capabilities: usb-1.10 configuration: driver=hub slots=2 speed=12.0MB/s *-usb:2 description: USB Controller product: 82801JI (ICH10 Family) USB UHCI Controller #6 vendor: Intel Corporation physical id: 1a.2 bus info: pci at 0000:00:1a.2 version: 00 width: 32 bits clock: 33MHz capabilities: uhci bus_master cap_list configuration: driver=uhci_hcd latency=0 resources: irq:23 ioport:9800(size=32) *-usbhost product: UHCI Host Controller vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd physical id: 1 bus info: usb at 5 logical name: /dev/usb5 version: 2.06 capabilities: usb-1.10 configuration: driver=hub slots=2 speed=12.0MB/s *-usb:3 description: USB Controller product: 82801JI (ICH10 Family) USB2 EHCI Controller #2 vendor: Intel Corporation physical id: 1a.7 bus info: pci at 0000:00:1a.7 version: 00 width: 32 bits clock: 33MHz capabilities: pm debug ehci bus_master cap_list configuration: driver=ehci_hcd latency=0 resources: irq:22 memory:fb4de000-fb4de3ff *-usbhost product: EHCI Host Controller vendor: Linux 2.6.29.4-167.fc11.x86_64 ehci_hcd physical id: 1 bus info: usb at 1 logical name: /dev/usb1 version: 2.06 capabilities: usb-2.00 configuration: driver=hub slots=6 speed=480.0MB/s *-multimedia description: Audio device product: 82801JI (ICH10 Family) HD Audio Controller vendor: Intel Corporation physical id: 1b bus info: pci at 0000:00:1b.0 version: 00 width: 64 bits clock: 33MHz capabilities: pm msi pciexpress bus_master cap_list configuration: driver=HDA Intel latency=0 resources: irq:22 memory:fb4d8000-fb4dbfff *-pci:3 description: PCI bridge product: 82801JI (ICH10 Family) PCI Express Port 1 vendor: Intel Corporation physical id: 1c bus info: pci at 0000:00:1c.0 version: 00 width: 32 bits clock: 33MHz capabilities: pci pciexpress msi pm normal_decode bus_master cap_list configuration: driver=pcieport-driver resources: irq:27 memory:fbb00000-fbbfffff ioport:fa800000(size=8388608) *-serial DISABLED description: interface product: MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] vendor: Mellanox Technologies physical id: 0 bus info: pci at 0000:09:00.0 logical name: ib0 version: a0 serial: 80:00:00:48:fe:80 width: 64 bits clock: 33MHz capabilities: pm vpd msix pciexpress bus_master cap_list physical configuration: broadcast=yes driver=ipoib latency=0 multicast=yes resources: irq:16 memory:fbb00000-fbbfffff memory:fa800000-faffffff(prefetchable) *-pci:4 description: PCI bridge product: 82801JI (ICH10 Family) PCI Express Port 5 vendor: Intel Corporation physical id: 1c.4 bus info: pci at 0000:00:1c.4 version: 00 width: 32 bits clock: 33MHz capabilities: pci pciexpress msi pm normal_decode bus_master cap_list configuration: driver=pcieport-driver resources: irq:28 ioport:d000(size=4096) memory:fbc00000-fbcfffff *-network description: Ethernet interface product: 82574L Gigabit Network Connection vendor: Intel Corporation physical id: 0 bus info: pci at 0000:0a:00.0 logical name: eth0 version: 00 serial: 00:30:48:f9:b3:9a size: 1GB/s capacity: 1GB/s width: 32 bits clock: 33MHz capabilities: pm msi pciexpress msix bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation configuration: autonegotiation=on broadcast=yes driver=e1000e driverversion=0.3.3.3-k6 duplex=full firmware=1.8-0 ip=192.168.0.117 latency=0 link=yes multicast=yes port=twisted pair speed=1GB/s resources: irq:16 memory:fbce0000-fbcfffff ioport:dc00(size=32) memory:fbcdc000-fbcdffff *-pci:5 description: PCI bridge product: 82801JI (ICH10 Family) PCI Express Port 6 vendor: Intel Corporation physical id: 1c.5 bus info: pci at 0000:00:1c.5 version: 00 width: 32 bits clock: 33MHz capabilities: pci pciexpress msi pm normal_decode bus_master cap_list configuration: driver=pcieport-driver resources: irq:29 ioport:e000(size=4096) memory:fbd00000-fbdfffff *-network description: Ethernet interface product: 82574L Gigabit Network Connection vendor: Intel Corporation physical id: 0 bus info: pci at 0000:0b:00.0 logical name: eth1 version: 00 serial: 00:30:48:f9:b3:9b capacity: 1GB/s width: 32 bits clock: 33MHz capabilities: pm msi pciexpress msix bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation configuration: autonegotiation=on broadcast=yes driver=e1000e driverversion=0.3.3.3-k6 firmware=1.8-0 latency=0 link=no multicast=yes port=twisted pair resources: irq:17 memory:fbde0000-fbdfffff ioport:ec00(size=32) memory:fbddc000-fbddffff *-usb:4 description: USB Controller product: 82801JI (ICH10 Family) USB UHCI Controller #1 vendor: Intel Corporation physical id: 1d bus info: pci at 0000:00:1d.0 version: 00 width: 32 bits clock: 33MHz capabilities: uhci bus_master cap_list configuration: driver=uhci_hcd latency=0 resources: irq:20 ioport:9480(size=32) *-usbhost product: UHCI Host Controller vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd physical id: 1 bus info: usb at 6 logical name: /dev/usb6 version: 2.06 capabilities: usb-1.10 configuration: driver=hub slots=2 speed=12.0MB/s *-usb:5 description: USB Controller product: 82801JI (ICH10 Family) USB UHCI Controller #2 vendor: Intel Corporation physical id: 1d.1 bus info: pci at 0000:00:1d.1 version: 00 width: 32 bits clock: 33MHz capabilities: uhci bus_master cap_list configuration: driver=uhci_hcd latency=0 resources: irq:21 ioport:9400(size=32) *-usbhost product: UHCI Host Controller vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd physical id: 1 bus info: usb at 7 logical name: /dev/usb7 version: 2.06 capabilities: usb-1.10 configuration: driver=hub slots=2 speed=12.0MB/s *-usb:6 description: USB Controller product: 82801JI (ICH10 Family) USB UHCI Controller #3 vendor: Intel Corporation physical id: 1d.2 bus info: pci at 0000:00:1d.2 version: 00 width: 32 bits clock: 33MHz capabilities: uhci bus_master cap_list configuration: driver=uhci_hcd latency=0 resources: irq:22 ioport:9080(size=32) *-usbhost product: UHCI Host Controller vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd physical id: 1 bus info: usb at 8 logical name: /dev/usb8 version: 2.06 capabilities: usb-1.10 configuration: driver=hub slots=2 speed=12.0MB/s *-usb:7 description: USB Controller product: 82801JI (ICH10 Family) USB2 EHCI Controller #1 vendor: Intel Corporation physical id: 1d.7 bus info: pci at 0000:00:1d.7 version: 00 width: 32 bits clock: 33MHz capabilities: pm debug ehci bus_master cap_list configuration: driver=ehci_hcd latency=0 resources: irq:20 memory:fb4dc000-fb4dc3ff *-usbhost product: EHCI Host Controller vendor: Linux 2.6.29.4-167.fc11.x86_64 ehci_hcd physical id: 1 bus info: usb at 2 logical name: /dev/usb2 version: 2.06 capabilities: usb-2.00 configuration: driver=hub slots=6 speed=480.0MB/s *-pci:6 description: PCI bridge product: 82801 PCI Bridge vendor: Intel Corporation physical id: 1e bus info: pci at 0000:00:1e.0 version: 90 width: 32 bits clock: 33MHz capabilities: pci subtractive_decode bus_master cap_list resources: memory:fbe00000-fbefffff *-firewire description: FireWire (IEEE 1394) product: TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link) vendor: Texas Instruments physical id: 8 bus info: pci at 0000:0c:08.0 version: 00 width: 32 bits clock: 33MHz capabilities: pm ohci bus_master cap_list configuration: driver=firewire_ohci latency=64 maxlatency=4 mingnt=2 resources: irq:17 memory:fbeff800-fbefffff memory:fbef8000-fbefbfff *-isa description: ISA bridge product: 82801JIR (ICH10R) LPC Interface Controller vendor: Intel Corporation physical id: 1f bus info: pci at 0000:00:1f.0 version: 00 width: 32 bits clock: 33MHz capabilities: isa bus_master cap_list configuration: latency=0 *-ide:0 description: IDE interface product: 82801JI (ICH10 Family) 4 port SATA IDE Controller vendor: Intel Corporation physical id: 1f.2 bus info: pci at 0000:00:1f.2 logical name: scsi0 logical name: scsi1 version: 00 width: 32 bits clock: 66MHz capabilities: ide pm bus_master cap_list emulated configuration: driver=ata_piix latency=0 resources: irq:19 ioport:1f0(size=8) ioport:3f6 ioport:170(size=8) ioport:376 ioport:ff90(size=16) ioport:ffa0(size=16) *-disk description: ATA Disk product: SAMSUNG HD753LJ physical id: 0.0.0 bus info: scsi at 0:0.0.0 logical name: /dev/sda version: 1AA0 serial: S13UJ9FZ101808 size: 698GiB (750GB) capabilities: partitioned partitioned:dos configuration: ansiversion=5 signature=0007d825 *-volume:0 description: EXT3 volume vendor: Linux physical id: 1 bus info: scsi at 0:0.0.0,1 logical name: /dev/sda1 logical name: /boot version: 1.0 serial: fc544031-e0c9-4cc9-86c0-583efd5b263d size: 200MiB capacity: 200MiB capabilities: primary bootable journaled extended_attributes ext3 ext2 initialized configuration: created=2010-05-19 21:18:54 filesystem=ext3 modified=2010-06-05 13:26:21 mount.fstype=ext3 mount.options=rw,relatime,errors=continue,data=ordered mounted=2010-06-02 18:15:25 state=mounted *-volume:1 description: Linux LVM Physical Volume partition physical id: 2 bus info: scsi at 0:0.0.0,2 logical name: /dev/sda2 serial: eOHevd-eVLc-aqpu-LN8x-ariz-JPI2-jb0Tkr size: 698GiB capacity: 698GiB capabilities: primary multi lvm2 *-cdrom description: DVD-RAM writer product: DVDRAM GH22NS40 vendor: HL-DT-ST physical id: 0.1.0 bus info: scsi at 1:0.1.0 logical name: /dev/cdrom logical name: /dev/cdrw logical name: /dev/dvd logical name: /dev/dvdrw logical name: /dev/scd0 logical name: /dev/sr0 version: NL02 capabilities: removable audio cd-r cd-rw dvd dvd-r dvd-ram configuration: ansiversion=5 status=nodisc *-serial description: SMBus product: 82801JI (ICH10 Family) SMBus Controller vendor: Intel Corporation physical id: 1f.3 bus info: pci at 0000:00:1f.3 version: 00 width: 64 bits clock: 33MHz configuration: driver=i801_smbus latency=0 resources: irq:18 memory:fb4d6000-fb4d60ff ioport:400(size=32) *-ide:1 description: IDE interface product: 82801JI (ICH10 Family) 2 port SATA IDE Controller vendor: Intel Corporation physical id: 1f.5 bus info: pci at 0000:00:1f.5 version: 00 width: 32 bits clock: 66MHz capabilities: ide pm bus_master cap_list configuration: driver=ata_piix latency=0 resources: irq:19 ioport:8c00(size=8) ioport:8880(size=4) ioport:8800(size=8) ioport:8480(size=4) ioport:8400(size=16) ioport:8080(size=16) On Mon, Jun 7, 2010 at 10:52 PM, Harshavardhana <harsha at gluster.com> wrote: > On 06/07/2010 06:22 AM, Todd Daugherty wrote: >> >> I have run iozone on the same machine with GlusterFS and without >> GlusterFS. (Local storage and Local storage via GlusterFS). I also >> included the results from a remote machine. The remote machine and the >> local glusterfs are the same. I am getting around 200 megabytes per >> second on reads. My local storage will do around 1400 megabytes per >> second. Any ideas why? >> >> Todd >> >> > > To objectively understand the variables here, you need to provide us details > you the hardware including infiniband card, RAM size. ?Backend disk > performance and type of disks (Raid controller). > > Backend disk performance should be with "dd if=/dev/zero > of=/<disk_mountpoint> bs=1M count=16386 oflag=direct" > > Regards > > -- > Harshavardhana > Gluster Inc - http://www.gluster.com > +1(408)-770-1887, Ext-113 > +1(408)-480-1730 > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >
