This has probably been discussed before, but since I'm new on the list I hope You have patience with me. I have a four brick distributed/replicated setup. The computers are multi-core 16GB memory and 2*2.0TB in raid1 SATA-disks locally. The nodes are connected by 1 GB ethernet. All nodes have glusterfs 3.3beta2 installed and they are running debian 6 64bit. The underlying filesystems are xfs. I have setup a volume like so; gluster volume create virtuals replica 2 transport tcp \ adraste:/data/brick alcippe:/data/brick aethra:/data/brick helen:/data/brick Which resulted in a nice volume; # gluster volume info virtuals Volume Name: virtuals Type: Distributed-Replicate Status: Started Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: adraste:/data/brick Brick2: alcippe:/data/brick Brick3: aethra:/data/brick Brick4: helen:/data/brick All seems OK so far, but write performance seems very slow. When writing to localhost:/virtuals I get single-digit MB/s performance which isn't really what I had expected. I know that the write has to go to at least two (?) nodes at the same time, but still? A single scp of a 1GB file from a node to another gives something like ~100MBps. A copy of a virtual image took 17 minutes; # time cp debtest.raw /gluster/debtest.img real 17m36.727s user 0m1.832s sys 0m14.081s # ls -lah /gluster/debtest.img -rw------- 1 root root 20G Mar 1 12:35 /gluster/debtest.img # du -ah /gluster/debtest.img 4.5G /gluster/debtest.img I noted that the processlist shows that direct-io-mode is disabled. Default should be on, or should it? Any help is really appreciated! -- Harald Hannelius | harald.hannelius/a\arcada.fi | +358 50 594 1020
