Hello Carlos,
thx for your reply. I generelly think that gluster should run out of the box with an good speed if the mainsystem has good speed. E.g. if i mesure 300MB/s Gluster should min run with 100MB/s, but your right its a complex thematic.
As i wrote above i mesure the performance with the Tool "fio" and run some read and write tests. I read about using of XFS, but EXT4 should do it also, while it is normaly as fast as XFS or even faster. Or is glusterfs optimized for XFS? I never worked with XFS, therefore i use EXT4... Yes i mesured first the hardware and than gluster...
Bevore running Gluster I thought about Hardware and the right Raid. After using a couple of years Raid5 and hat some desaster i decided to use raid6 because of speed and costs. Raid10 is to expensive for me.
Now i take the 2. Disk from the Controller and put it on en external USB on another system. If i now copy the fils over the network the speed goes up to 30-34MB/s. Could it the Controller?
------------------------------------------
Actually there are tons of possible tune ups, but let's start from the beginning.
Gluster documents, Red Hat documents, and several other sources suggest using XFS, and NOT ext4.
Second, How did you measure performance ? What tool ?
From my personal experience, first of all, measure performance of you hardware, before measuring Gluster's.
I found out a major problem with my disk layout. I was naive enough believe that by using a good RAID controller (PERC 6) with fair disks (SAS 7.2 KRPM 3 Gbits) on RAID5 I would be in the safe side. It was a disaster. It never passed 40 MBytes/s.
Changing that to RAID10 made it 10 times faster, at the price of half of the storage space.
Test your hardware.
Start with simple stuff like
dd if=/dev/zero of=filename_on_your_local_disks bs=512k count=10000
check what the result is, and if your hardware is responding as expected.
Only then move to other areas.
Cheers,
Carlos
On Tue, Apr 29, 2014 at 10:47 AM, Dragon <Sunghost@xxxxxx> wrote:
Hello,
i have a 3 Node Gluster with one distributed Volume. All 3 Nodes are identically in their Hardware. All have 6x3TB Seagate ST3000DM001. The disk on each Node are in a Softwareraid6 (Blocksize 4096| Stripe 512) with ext4 filesystem on Debian Wheezy. As the Raid was new i testet it with fio:
[sequential-write]
rw=write
size=5g
direct=0
numjobs=3
group_reporting
name=sequential-write-buffered
bs=4k
runtime=10
rw=write
size=5g
direct=0
numjobs=3
group_reporting
name=sequential-write-buffered
bs=4k
runtime=10
Result: WRITE: io=3014.2MB, aggrb=308456KB/s, minb=308456KB/s, maxb=308456KB/s,
I added a new sata controller Digitus DS-30104-1 4 Port PCIe x4, each Port 6GBits on x16 Slot. Node1 has now 7 Disks in Raid6. I added another Disk 3TB same model too on the Controller and copied files 20-40GB onto the volume. The Volume is mounted on the same node with "mount -t gluster-fs IP-Adress:/vol1" The Problem is that the writespeed is only 20MB/s. This is what i didnt understand. The files are copied on the same machine, not over the network, perhaps the bus-system of the mainboard is to slow? Any kind of performance tuning?
thx
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://supercolony.gluster.org/mailman/listinfo/gluster-users
