Performance

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I included a bunch of other info. If there is anything else that will
help solve this problem I am very interested in getting to the bottom
of this. Side note I built a GlusterFS cluster (2.0.9) from RAM disks
(10 gig /dev/ram0 bricks) That performs quite well but still much less
than the backend /dev/ram0 bricks.

Todd

InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s
- IB QDR / 10GigE] (rev a0)
Atto Technology SAS RAID Adapter Subsystem: Atto Technology Device
002d Kernel modules: esasraid
16 SATA 1 TB Disks on two of those Atto Cards so 8 disks each
controller. (JMR PE SAN)

dd if=/dev/zero of=/node01/zero bs=1M count=16386 oflag=direct
16386+0 records in
16386+0 records out
17181966336 bytes (17 GB) copied, 17.0376 s, 1.0 GB/s

6 Gigabytes of RAM


glusterfs -V
glusterfs 2.0.9 built on Apr 11 2010 20:40:04
Repository revision: v2.0.9
Copyright (c) 2006-2009 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU
General Public License.


cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
stepping	: 5
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5319.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
stepping	: 5
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5319.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
stepping	: 5
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5319.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
stepping	: 5
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5319.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
stepping	: 5
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5319.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
stepping	: 5
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 3
initial apicid	: 3
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5319.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
stepping	: 5
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 5
initial apicid	: 5
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5319.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
stepping	: 5
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology
nonstop_tsc pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm
sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5319.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:


cat /proc/meminfo
MemTotal:        6060364 kB
MemFree:          137120 kB
Buffers:          176488 kB
Cached:          5225760 kB
SwapCached:            0 kB
Active:           319020 kB
Inactive:        5200428 kB
Active(anon):      13216 kB
Inactive(anon):   104236 kB
Active(file):     305804 kB
Inactive(file):  5096192 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       8110072 kB
SwapFree:        8110072 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:        117264 kB
Mapped:            17232 kB
Slab:             208168 kB
SReclaimable:     177444 kB
SUnreclaim:        30724 kB
PageTables:         5492 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    11140252 kB
Committed_AS:     290736 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      320548 kB
VmallocChunk:   34359397567 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7744 kB
DirectMap2M:     6275072 kB


uname -a
Linux axon07 2.6.29.4-167.fc11.x86_64 #1 SMP Wed May 27 17:27:08 EDT
2009 x86_64 x86_64 x86_64 GNU/Linux

lshw
axon07
    description: Desktop Computer
    product: X8SAX
    vendor: Supermicro
    version: 1234567890
    serial: 1234567890
    width: 64 bits
    capabilities: vsyscall64 vsyscall32
    configuration: boot=normal chassis=desktop
uuid=00020003-0004-0005-0006-000700080009
  *-core
       description: Motherboard
       product: X8SAX
       vendor: Supermicro
       physical id: 0
       version: 1234567890
       serial: 1234567890
       slot: To Be Filled By O.E.M.
     *-firmware
          description: BIOS
          vendor: American Megatrends Inc.
          physical id: 0
          version: 1.0c (08/05/2009)
          size: 64KiB
          capacity: 4032KiB
          capabilities: isa pci pnp upgrade shadowing escd cdboot
bootselect socketedrom edd int13floppy1200 int13floppy720
int13floppy2880 int5printscreen int9keyboard int14serial int17printer
int10video acpi usb ls120boot zipboot biosbootspecification
     *-cpu
          description: CPU
          product: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
          vendor: Intel Corp.
          physical id: 4
          bus info: cpu at 0
          version: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz
          serial: To Be Filled By O.E.M.
          slot: CPU 1
          size: 1600MHz
          capacity: 1600MHz
          width: 64 bits
          clock: 133MHz
          capabilities: fpu fpu_exception wp vme de pse tsc msr pae
mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse
sse2 ss ht tm pbe syscall nx rdtscp x86-64 constant_tsc arch_perfmon
pebs bts rep_good xtopology nonstop_tsc pni dtes64 monitor ds_cpl vmx
est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 lahf_lm ida tpr_shadow vnmi
flexpriority ept vpid cpufreq
        *-cache:0
             description: L1 cache
             physical id: 5
             slot: L1-Cache
             size: 256KiB
             capacity: 256KiB
             capabilities: internal write-back data
        *-cache:1
             description: L2 cache
             physical id: 6
             slot: L2-Cache
             size: 1MiB
             capacity: 1MiB
             capabilities: internal write-back unified
        *-cache:2
             description: L3 cache
             physical id: 7
             slot: L3-Cache
             size: 8MiB
             capacity: 8MiB
             capabilities: internal write-back unified
     *-memory:0
          description: System Memory
          physical id: 28
          slot: System board or motherboard
        *-bank:0
             description: DIMM 1333 MHz (0.8 ns)
             vendor: Manufacturer00
             physical id: 0
             serial: 00000000
             slot: DIMM0
             size: 2GiB
             width: 64 bits
             clock: 1333MHz (0.8ns)
        *-bank:1
             description: DIMM 1333 MHz (0.8 ns) [empty]
             product: ModulePartNumber01
             vendor: Manufacturer01
             physical id: 1
             serial: SerNum01
             slot: DIMM1
             width: 64 bits
             clock: 1333MHz (0.8ns)
        *-bank:2
             description: DIMM 1333 MHz (0.8 ns)
             vendor: Manufacturer02
             physical id: 2
             serial: 00000000
             slot: DIMM2
             size: 2GiB
             width: 64 bits
             clock: 1333MHz (0.8ns)
        *-bank:3
             description: DIMM 1333 MHz (0.8 ns) [empty]
             product: ModulePartNumber03
             vendor: Manufacturer03
             physical id: 3
             serial: SerNum03
             slot: DIMM3
             width: 64 bits
             clock: 1333MHz (0.8ns)
        *-bank:4
             description: DIMM 1333 MHz (0.8 ns)
             vendor: Manufacturer04
             physical id: 4
             serial: 00000000
             slot: DIMM4
             size: 2GiB
             width: 64 bits
             clock: 1333MHz (0.8ns)
        *-bank:5
             description: DIMM 1333 MHz (0.8 ns) [empty]
             product: ModulePartNumber05
             vendor: Manufacturer05
             physical id: 5
             serial: SerNum05
             slot: DIMM5
             width: 64 bits
             clock: 1333MHz (0.8ns)
     *-memory:1 UNCLAIMED
          description: Flash Memory
          physical id: 36
          slot: System board or motherboard
          capacity: 4MiB
        *-bank UNCLAIMED
             description: FLASH Non-volatile 33 MHz (30.3 ns)
             product: 26DF321
             vendor: ATMEL
             physical id: 0
             size: 4MiB
             width: 8 bits
             clock: 33MHz (30.3ns)
     *-memory:2 UNCLAIMED
          physical id: 1
     *-memory:3 UNCLAIMED
          physical id: 2
     *-pci
          description: Host bridge
          product: X58 I/O Hub to ESI Port
          vendor: Intel Corporation
          physical id: 100
          bus info: pci at 0000:00:00.0
          version: 13
          width: 32 bits
          clock: 33MHz
        *-pci:0
             description: PCI bridge
             product: X58 I/O Hub PCI Express Root Port 1
             vendor: Intel Corporation
             physical id: 1
             bus info: pci at 0000:00:01.0
             version: 13
             width: 32 bits
             clock: 33MHz
             capabilities: pci msi pciexpress pm normal_decode
bus_master cap_list
             configuration: driver=pcieport-driver
             resources: irq:24
           *-pci
                description: PCI bridge
                product: 6702PXH PCI Express-to-PCI Bridge A
                vendor: Intel Corporation
                physical id: 0
                bus info: pci at 0000:01:00.0
                version: 09
                width: 32 bits
                clock: 33MHz
                capabilities: pci pciexpress msi pm pcix normal_decode
bus_master cap_list
        *-pci:1
             description: PCI bridge
             product: X58 I/O Hub PCI Express Root Port 3
             vendor: Intel Corporation
             physical id: 3
             bus info: pci at 0000:00:03.0
             version: 13
             width: 32 bits
             clock: 33MHz
             capabilities: pci msi pciexpress pm normal_decode
bus_master cap_list
             configuration: driver=pcieport-driver
             resources: irq:25 ioport:a000(size=8192) memory:fb500000-fb9fffff
           *-pci
                description: PCI bridge
                product: PEX 8533 32-lane, 6-port PCI Express Switch
                vendor: PLX Technology, Inc.
                physical id: 0
                bus info: pci at 0000:03:00.0
                version: aa
                width: 32 bits
                clock: 33MHz
                capabilities: pci pm msi pciexpress normal_decode
bus_master cap_list
                configuration: driver=pcieport-driver
                resources: irq:30 memory:fb5e0000-fb5fffff
ioport:a000(size=8192) memory:fb600000-fb9fffff
              *-pci:0
                   description: PCI bridge
                   product: PEX 8533 32-lane, 6-port PCI Express Switch
                   vendor: PLX Technology, Inc.
                   physical id: 0
                   bus info: pci at 0000:04:00.0
                   version: aa
                   width: 64 bits
                   clock: 33MHz
                   capabilities: pci pm msi pciexpress normal_decode
bus_master cap_list
                   configuration: driver=pcieport-driver
                   resources: iomemory:1f10-1f0f irq:31
              *-pci:1
                   description: PCI bridge
                   product: PEX 8533 32-lane, 6-port PCI Express Switch
                   vendor: PLX Technology, Inc.
                   physical id: 1
                   bus info: pci at 0000:04:01.0
                   version: aa
                   width: 64 bits
                   clock: 33MHz
                   capabilities: pci pm msi pciexpress normal_decode
bus_master cap_list
                   configuration: driver=pcieport-driver
                   resources: iomemory:a1a10-a1a0f irq:32
ioport:a000(size=4096) memory:fb600000-fb7fffff
                 *-storage
                      description: RAID bus controller
                      product: SAS RAID Adapter
                      vendor: Atto Technology
                      physical id: 0
                      bus info: pci at 0000:06:00.0
                      logical name: scsi4
                      version: 00
                      width: 64 bits
                      clock: 33MHz
                      capabilities: storage pm msi pciexpress
bus_master cap_list rom scsi-host
                      configuration: driver=esasraid latency=0
                      resources: irq:17 memory:fb780000-fb7fffff
memory:fb700000-fb77ffff ioport:a800(size=256)
memory:fb600000-fb6fffff(prefetchable)
                    *-enclosure UNCLAIMED
                         description: SCSI Enclosure
                         product: PSAN SAS Exp
                         vendor: JMR Elec
                         physical id: 0.0.0
                         bus info: scsi at 4:0.0.0
                         version: 2.17
                         configuration: ansiversion=3
                    *-disk
                         description: SCSI Disk
                         product: test0100
                         vendor: ATTO
                         physical id: 0.1.0
                         bus info: scsi at 4:0.1.0
                         logical name: /dev/sdb
                         version: 0001
                         serial: c3da063f774df7d71d996b020ac16ca400000000
                         size: 7451GiB (8001GB)
                         configuration: ansiversion=5
              *-pci:2
                   description: PCI bridge
                   product: PEX 8533 32-lane, 6-port PCI Express Switch
                   vendor: PLX Technology, Inc.
                   physical id: 9
                   bus info: pci at 0000:04:09.0
                   version: aa
                   width: 64 bits
                   clock: 33MHz
                   capabilities: pci pm msi pciexpress normal_decode
bus_master cap_list
                   configuration: driver=pcieport-driver
                   resources: iomemory:b1b10-b1b0f irq:33
ioport:b000(size=4096) memory:fb800000-fb9fffff
                 *-storage
                      description: RAID bus controller
                      product: SAS RAID Adapter
                      vendor: Atto Technology
                      physical id: 0
                      bus info: pci at 0000:07:00.0
                      logical name: scsi5
                      version: 00
                      width: 64 bits
                      clock: 33MHz
                      capabilities: storage pm msi pciexpress
bus_master cap_list rom scsi-host
                      configuration: driver=esasraid latency=0
                      resources: irq:17 memory:fb980000-fb9fffff
memory:fb900000-fb97ffff ioport:b800(size=256)
memory:fb800000-fb8fffff(prefetchable)
                    *-enclosure UNCLAIMED
                         description: SCSI Enclosure
                         product: PSAN SAS Exp
                         vendor: JMR Elec
                         physical id: 0.0.0
                         bus info: scsi at 5:0.0.0
                         version: 2.17
                         configuration: ansiversion=3
                    *-disk
                         description: SCSI Disk
                         product: test0200
                         vendor: ATTO
                         physical id: 0.1.0
                         bus info: scsi at 5:0.1.0
                         logical name: /dev/sdc
                         version: 0001
                         serial: cad720b12a0778fe517f60962c34285400000000
                         size: 7451GiB (8001GB)
                         configuration: ansiversion=5
        *-pci:2
             description: PCI bridge
             product: X58 I/O Hub PCI Express Root Port 7
             vendor: Intel Corporation
             physical id: 7
             bus info: pci at 0000:00:07.0
             version: 13
             width: 32 bits
             clock: 33MHz
             capabilities: pci msi pciexpress pm normal_decode
bus_master cap_list
             configuration: driver=pcieport-driver
             resources: irq:26 ioport:c000(size=4096)
memory:fba00000-fbafffff ioport:d0000000(size=268435456)
           *-display
                description: VGA compatible controller
                product: RV710 [Radeon HD 4350]
                vendor: ATI Technologies Inc
                physical id: 0
                bus info: pci at 0000:08:00.0
                version: 00
                width: 64 bits
                clock: 33MHz
                capabilities: pm pciexpress msi vga_controller
bus_master cap_list rom
                configuration: driver=radeon latency=0
                resources: irq:16
memory:d0000000-dfffffff(prefetchable) memory:fbaf0000-fbafffff
ioport:c000(size=256) memory:fbac0000-fbadffff(prefetchable)
           *-multimedia
                description: Audio device
                product: R700 Audio Device [Radeon HD 4000 Series]
                vendor: ATI Technologies Inc
                physical id: 0.1
                bus info: pci at 0000:08:00.1
                version: 00
                width: 64 bits
                clock: 33MHz
                capabilities: pm pciexpress msi bus_master cap_list
                configuration: driver=HDA Intel latency=0
                resources: irq:17 memory:fbaec000-fbaeffff
        *-generic:0 UNCLAIMED
             description: PIC
             product: X58 I/O Hub System Management Registers
             vendor: Intel Corporation
             physical id: 14
             bus info: pci at 0000:00:14.0
             version: 13
             width: 32 bits
             clock: 33MHz
             capabilities: pciexpress 8259 cap_list
             configuration: latency=0
        *-generic:1 UNCLAIMED
             description: PIC
             product: X58 I/O Hub GPIO and Scratch Pad Registers
             vendor: Intel Corporation
             physical id: 14.1
             bus info: pci at 0000:00:14.1
             version: 13
             width: 32 bits
             clock: 33MHz
             capabilities: pciexpress 8259 cap_list
             configuration: latency=0
        *-generic:2 UNCLAIMED
             description: PIC
             product: X58 I/O Hub Control Status and RAS Registers
             vendor: Intel Corporation
             physical id: 14.2
             bus info: pci at 0000:00:14.2
             version: 13
             width: 32 bits
             clock: 33MHz
             capabilities: pciexpress 8259 cap_list
             configuration: latency=0
        *-generic:3 UNCLAIMED
             description: PIC
             product: X58 I/O Hub Throttle Registers
             vendor: Intel Corporation
             physical id: 14.3
             bus info: pci at 0000:00:14.3
             version: 13
             width: 32 bits
             clock: 33MHz
             capabilities: 8259
             configuration: latency=0
        *-generic:4 UNCLAIMED
             description: System peripheral
             product: X58 Chipset QuickData Technology Device
             vendor: Intel Corporation
             physical id: 16
             bus info: pci at 0000:00:16.0
             version: 13
             width: 64 bits
             clock: 33MHz
             capabilities: msix pciexpress pm bus_master cap_list
             configuration: latency=0
             resources: memory:fb4fc000-fb4fffff
        *-generic:5 UNCLAIMED
             description: System peripheral
             product: X58 Chipset QuickData Technology Device
             vendor: Intel Corporation
             physical id: 16.1
             bus info: pci at 0000:00:16.1
             version: 13
             width: 64 bits
             clock: 33MHz
             capabilities: msix pciexpress pm bus_master cap_list
             configuration: latency=0
             resources: memory:fb4f8000-fb4fbfff
        *-generic:6 UNCLAIMED
             description: System peripheral
             product: X58 Chipset QuickData Technology Device
             vendor: Intel Corporation
             physical id: 16.2
             bus info: pci at 0000:00:16.2
             version: 13
             width: 64 bits
             clock: 33MHz
             capabilities: msix pciexpress pm bus_master cap_list
             configuration: latency=0
             resources: memory:fb4f4000-fb4f7fff
        *-generic:7 UNCLAIMED
             description: System peripheral
             product: X58 Chipset QuickData Technology Device
             vendor: Intel Corporation
             physical id: 16.3
             bus info: pci at 0000:00:16.3
             version: 13
             width: 64 bits
             clock: 33MHz
             capabilities: msix pciexpress pm bus_master cap_list
             configuration: latency=0
             resources: memory:fb4f0000-fb4f3fff
        *-generic:8 UNCLAIMED
             description: System peripheral
             product: X58 Chipset QuickData Technology Device
             vendor: Intel Corporation
             physical id: 16.4
             bus info: pci at 0000:00:16.4
             version: 13
             width: 64 bits
             clock: 33MHz
             capabilities: msix pciexpress pm bus_master cap_list
             configuration: latency=0
             resources: memory:fb4ec000-fb4effff
        *-generic:9 UNCLAIMED
             description: System peripheral
             product: X58 Chipset QuickData Technology Device
             vendor: Intel Corporation
             physical id: 16.5
             bus info: pci at 0000:00:16.5
             version: 13
             width: 64 bits
             clock: 33MHz
             capabilities: msix pciexpress pm bus_master cap_list
             configuration: latency=0
             resources: memory:fb4e8000-fb4ebfff
        *-generic:10 UNCLAIMED
             description: System peripheral
             product: X58 Chipset QuickData Technology Device
             vendor: Intel Corporation
             physical id: 16.6
             bus info: pci at 0000:00:16.6
             version: 13
             width: 64 bits
             clock: 33MHz
             capabilities: msix pciexpress pm bus_master cap_list
             configuration: latency=0
             resources: memory:fb4e4000-fb4e7fff
        *-generic:11 UNCLAIMED
             description: System peripheral
             product: DMA Engine
             vendor: Intel Corporation
             physical id: 16.7
             bus info: pci at 0000:00:16.7
             version: 13
             width: 64 bits
             clock: 33MHz
             capabilities: msix pciexpress pm bus_master cap_list
             configuration: latency=0
             resources: memory:fb4e0000-fb4e3fff
        *-usb:0
             description: USB Controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #4
             vendor: Intel Corporation
             physical id: 1a
             bus info: pci at 0000:00:1a.0
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:20 ioport:9c00(size=32)
           *-usbhost
                product: UHCI Host Controller
                vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
                physical id: 1
                bus info: usb at 3
                logical name: /dev/usb3
                version: 2.06
                capabilities: usb-1.10
                configuration: driver=hub slots=2 speed=12.0MB/s
        *-usb:1
             description: USB Controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #5
             vendor: Intel Corporation
             physical id: 1a.1
             bus info: pci at 0000:00:1a.1
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:21 ioport:9880(size=32)
           *-usbhost
                product: UHCI Host Controller
                vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
                physical id: 1
                bus info: usb at 4
                logical name: /dev/usb4
                version: 2.06
                capabilities: usb-1.10
                configuration: driver=hub slots=2 speed=12.0MB/s
        *-usb:2
             description: USB Controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #6
             vendor: Intel Corporation
             physical id: 1a.2
             bus info: pci at 0000:00:1a.2
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:23 ioport:9800(size=32)
           *-usbhost
                product: UHCI Host Controller
                vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
                physical id: 1
                bus info: usb at 5
                logical name: /dev/usb5
                version: 2.06
                capabilities: usb-1.10
                configuration: driver=hub slots=2 speed=12.0MB/s
        *-usb:3
             description: USB Controller
             product: 82801JI (ICH10 Family) USB2 EHCI Controller #2
             vendor: Intel Corporation
             physical id: 1a.7
             bus info: pci at 0000:00:1a.7
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: pm debug ehci bus_master cap_list
             configuration: driver=ehci_hcd latency=0
             resources: irq:22 memory:fb4de000-fb4de3ff
           *-usbhost
                product: EHCI Host Controller
                vendor: Linux 2.6.29.4-167.fc11.x86_64 ehci_hcd
                physical id: 1
                bus info: usb at 1
                logical name: /dev/usb1
                version: 2.06
                capabilities: usb-2.00
                configuration: driver=hub slots=6 speed=480.0MB/s
        *-multimedia
             description: Audio device
             product: 82801JI (ICH10 Family) HD Audio Controller
             vendor: Intel Corporation
             physical id: 1b
             bus info: pci at 0000:00:1b.0
             version: 00
             width: 64 bits
             clock: 33MHz
             capabilities: pm msi pciexpress bus_master cap_list
             configuration: driver=HDA Intel latency=0
             resources: irq:22 memory:fb4d8000-fb4dbfff
        *-pci:3
             description: PCI bridge
             product: 82801JI (ICH10 Family) PCI Express Port 1
             vendor: Intel Corporation
             physical id: 1c
             bus info: pci at 0000:00:1c.0
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: pci pciexpress msi pm normal_decode
bus_master cap_list
             configuration: driver=pcieport-driver
             resources: irq:27 memory:fbb00000-fbbfffff
ioport:fa800000(size=8388608)
           *-serial DISABLED
                description: interface
                product: MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE]
                vendor: Mellanox Technologies
                physical id: 0
                bus info: pci at 0000:09:00.0
                logical name: ib0
                version: a0
                serial: 80:00:00:48:fe:80
                width: 64 bits
                clock: 33MHz
                capabilities: pm vpd msix pciexpress bus_master
cap_list physical
                configuration: broadcast=yes driver=ipoib latency=0
multicast=yes
                resources: irq:16 memory:fbb00000-fbbfffff
memory:fa800000-faffffff(prefetchable)
        *-pci:4
             description: PCI bridge
             product: 82801JI (ICH10 Family) PCI Express Port 5
             vendor: Intel Corporation
             physical id: 1c.4
             bus info: pci at 0000:00:1c.4
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: pci pciexpress msi pm normal_decode
bus_master cap_list
             configuration: driver=pcieport-driver
             resources: irq:28 ioport:d000(size=4096) memory:fbc00000-fbcfffff
           *-network
                description: Ethernet interface
                product: 82574L Gigabit Network Connection
                vendor: Intel Corporation
                physical id: 0
                bus info: pci at 0000:0a:00.0
                logical name: eth0
                version: 00
                serial: 00:30:48:f9:b3:9a
                size: 1GB/s
                capacity: 1GB/s
                width: 32 bits
                clock: 33MHz
                capabilities: pm msi pciexpress msix bus_master
cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd
autonegotiation
                configuration: autonegotiation=on broadcast=yes
driver=e1000e driverversion=0.3.3.3-k6 duplex=full firmware=1.8-0
ip=192.168.0.117 latency=0 link=yes multicast=yes port=twisted pair
speed=1GB/s
                resources: irq:16 memory:fbce0000-fbcfffff
ioport:dc00(size=32) memory:fbcdc000-fbcdffff
        *-pci:5
             description: PCI bridge
             product: 82801JI (ICH10 Family) PCI Express Port 6
             vendor: Intel Corporation
             physical id: 1c.5
             bus info: pci at 0000:00:1c.5
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: pci pciexpress msi pm normal_decode
bus_master cap_list
             configuration: driver=pcieport-driver
             resources: irq:29 ioport:e000(size=4096) memory:fbd00000-fbdfffff
           *-network
                description: Ethernet interface
                product: 82574L Gigabit Network Connection
                vendor: Intel Corporation
                physical id: 0
                bus info: pci at 0000:0b:00.0
                logical name: eth1
                version: 00
                serial: 00:30:48:f9:b3:9b
                capacity: 1GB/s
                width: 32 bits
                clock: 33MHz
                capabilities: pm msi pciexpress msix bus_master
cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd
autonegotiation
                configuration: autonegotiation=on broadcast=yes
driver=e1000e driverversion=0.3.3.3-k6 firmware=1.8-0 latency=0
link=no multicast=yes port=twisted pair
                resources: irq:17 memory:fbde0000-fbdfffff
ioport:ec00(size=32) memory:fbddc000-fbddffff
        *-usb:4
             description: USB Controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #1
             vendor: Intel Corporation
             physical id: 1d
             bus info: pci at 0000:00:1d.0
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:20 ioport:9480(size=32)
           *-usbhost
                product: UHCI Host Controller
                vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
                physical id: 1
                bus info: usb at 6
                logical name: /dev/usb6
                version: 2.06
                capabilities: usb-1.10
                configuration: driver=hub slots=2 speed=12.0MB/s
        *-usb:5
             description: USB Controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #2
             vendor: Intel Corporation
             physical id: 1d.1
             bus info: pci at 0000:00:1d.1
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:21 ioport:9400(size=32)
           *-usbhost
                product: UHCI Host Controller
                vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
                physical id: 1
                bus info: usb at 7
                logical name: /dev/usb7
                version: 2.06
                capabilities: usb-1.10
                configuration: driver=hub slots=2 speed=12.0MB/s
        *-usb:6
             description: USB Controller
             product: 82801JI (ICH10 Family) USB UHCI Controller #3
             vendor: Intel Corporation
             physical id: 1d.2
             bus info: pci at 0000:00:1d.2
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master cap_list
             configuration: driver=uhci_hcd latency=0
             resources: irq:22 ioport:9080(size=32)
           *-usbhost
                product: UHCI Host Controller
                vendor: Linux 2.6.29.4-167.fc11.x86_64 uhci_hcd
                physical id: 1
                bus info: usb at 8
                logical name: /dev/usb8
                version: 2.06
                capabilities: usb-1.10
                configuration: driver=hub slots=2 speed=12.0MB/s
        *-usb:7
             description: USB Controller
             product: 82801JI (ICH10 Family) USB2 EHCI Controller #1
             vendor: Intel Corporation
             physical id: 1d.7
             bus info: pci at 0000:00:1d.7
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: pm debug ehci bus_master cap_list
             configuration: driver=ehci_hcd latency=0
             resources: irq:20 memory:fb4dc000-fb4dc3ff
           *-usbhost
                product: EHCI Host Controller
                vendor: Linux 2.6.29.4-167.fc11.x86_64 ehci_hcd
                physical id: 1
                bus info: usb at 2
                logical name: /dev/usb2
                version: 2.06
                capabilities: usb-2.00
                configuration: driver=hub slots=6 speed=480.0MB/s
        *-pci:6
             description: PCI bridge
             product: 82801 PCI Bridge
             vendor: Intel Corporation
             physical id: 1e
             bus info: pci at 0000:00:1e.0
             version: 90
             width: 32 bits
             clock: 33MHz
             capabilities: pci subtractive_decode bus_master cap_list
             resources: memory:fbe00000-fbefffff
           *-firewire
                description: FireWire (IEEE 1394)
                product: TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link)
                vendor: Texas Instruments
                physical id: 8
                bus info: pci at 0000:0c:08.0
                version: 00
                width: 32 bits
                clock: 33MHz
                capabilities: pm ohci bus_master cap_list
                configuration: driver=firewire_ohci latency=64
maxlatency=4 mingnt=2
                resources: irq:17 memory:fbeff800-fbefffff
memory:fbef8000-fbefbfff
        *-isa
             description: ISA bridge
             product: 82801JIR (ICH10R) LPC Interface Controller
             vendor: Intel Corporation
             physical id: 1f
             bus info: pci at 0000:00:1f.0
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: isa bus_master cap_list
             configuration: latency=0
        *-ide:0
             description: IDE interface
             product: 82801JI (ICH10 Family) 4 port SATA IDE Controller
             vendor: Intel Corporation
             physical id: 1f.2
             bus info: pci at 0000:00:1f.2
             logical name: scsi0
             logical name: scsi1
             version: 00
             width: 32 bits
             clock: 66MHz
             capabilities: ide pm bus_master cap_list emulated
             configuration: driver=ata_piix latency=0
             resources: irq:19 ioport:1f0(size=8) ioport:3f6
ioport:170(size=8) ioport:376 ioport:ff90(size=16)
ioport:ffa0(size=16)
           *-disk
                description: ATA Disk
                product: SAMSUNG HD753LJ
                physical id: 0.0.0
                bus info: scsi at 0:0.0.0
                logical name: /dev/sda
                version: 1AA0
                serial: S13UJ9FZ101808
                size: 698GiB (750GB)
                capabilities: partitioned partitioned:dos
                configuration: ansiversion=5 signature=0007d825
              *-volume:0
                   description: EXT3 volume
                   vendor: Linux
                   physical id: 1
                   bus info: scsi at 0:0.0.0,1
                   logical name: /dev/sda1
                   logical name: /boot
                   version: 1.0
                   serial: fc544031-e0c9-4cc9-86c0-583efd5b263d
                   size: 200MiB
                   capacity: 200MiB
                   capabilities: primary bootable journaled
extended_attributes ext3 ext2 initialized
                   configuration: created=2010-05-19 21:18:54
filesystem=ext3 modified=2010-06-05 13:26:21 mount.fstype=ext3
mount.options=rw,relatime,errors=continue,data=ordered
mounted=2010-06-02 18:15:25 state=mounted
              *-volume:1
                   description: Linux LVM Physical Volume partition
                   physical id: 2
                   bus info: scsi at 0:0.0.0,2
                   logical name: /dev/sda2
                   serial: eOHevd-eVLc-aqpu-LN8x-ariz-JPI2-jb0Tkr
                   size: 698GiB
                   capacity: 698GiB
                   capabilities: primary multi lvm2
           *-cdrom
                description: DVD-RAM writer
                product: DVDRAM GH22NS40
                vendor: HL-DT-ST
                physical id: 0.1.0
                bus info: scsi at 1:0.1.0
                logical name: /dev/cdrom
                logical name: /dev/cdrw
                logical name: /dev/dvd
                logical name: /dev/dvdrw
                logical name: /dev/scd0
                logical name: /dev/sr0
                version: NL02
                capabilities: removable audio cd-r cd-rw dvd dvd-r dvd-ram
                configuration: ansiversion=5 status=nodisc
        *-serial
             description: SMBus
             product: 82801JI (ICH10 Family) SMBus Controller
             vendor: Intel Corporation
             physical id: 1f.3
             bus info: pci at 0000:00:1f.3
             version: 00
             width: 64 bits
             clock: 33MHz
             configuration: driver=i801_smbus latency=0
             resources: irq:18 memory:fb4d6000-fb4d60ff ioport:400(size=32)
        *-ide:1
             description: IDE interface
             product: 82801JI (ICH10 Family) 2 port SATA IDE Controller
             vendor: Intel Corporation
             physical id: 1f.5
             bus info: pci at 0000:00:1f.5
             version: 00
             width: 32 bits
             clock: 66MHz
             capabilities: ide pm bus_master cap_list
             configuration: driver=ata_piix latency=0
             resources: irq:19 ioport:8c00(size=8) ioport:8880(size=4)
ioport:8800(size=8) ioport:8480(size=4) ioport:8400(size=16)
ioport:8080(size=16)





On Mon, Jun 7, 2010 at 10:52 PM, Harshavardhana <harsha at gluster.com> wrote:
> On 06/07/2010 06:22 AM, Todd Daugherty wrote:
>>
>> I have run iozone on the same machine with GlusterFS and without
>> GlusterFS. (Local storage and Local storage via GlusterFS). I also
>> included the results from a remote machine. The remote machine and the
>> local glusterfs are the same. I am getting around 200 megabytes per
>> second on reads. My local storage will do around 1400 megabytes per
>> second. Any ideas why?
>>
>> Todd
>>
>>
>
> To objectively understand the variables here, you need to provide us details
> you the hardware including infiniband card, RAM size. ?Backend disk
> performance and type of disks (Raid controller).
>
> Backend disk performance should be with "dd if=/dev/zero
> of=/<disk_mountpoint> bs=1M count=16386 oflag=direct"
>
> Regards
>
> --
> Harshavardhana
> Gluster Inc - http://www.gluster.com
> +1(408)-770-1887, Ext-113
> +1(408)-480-1730
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>


[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux