Hi! First, let me explain the scenario: Until now, I am one admin of an OpenSuse 11.3 root server which hosts several domains and is mainly used for web and mail. To enhance reliability, availability and performance, we decided to extend this setup to up to three servers on different locations. Now we had to find solutions to sync parts of the filebase (especially web root and mail root). For mysql, solutions exist (either clustering or master-master-replication). So, the task is to maintain 2-3 partitions (or a part of them) in sync between two or three servers, no further clients will access them and the volumes are solely used locally on the two servers. Gluster should simply ensure the sync between the two servers. I found first ocfs2 and later on gluster. Gluster, not requiring own partitions with a special file system (like ocfs2/drbd), seemed to be a good solution. After having worked through the documentation on gluster.org and some further tutorials (there are not real hints on mounting the volumes), I got the things working. Creating and installing the mail root volume (15 GB) worked quite good and after enabling readahead, iocache and writeback volume settings in /usr/local/etc/gluster/glusterfs.vol, I achieved acceptable perfomance and system load parameters. The setup is as follows. The partitions (xfs) are mounted under /data and the volumes reference to these directories. The volumes are then mounted on the two machines to their original location. The volumes are created as replica 2 transport tcp and the necessary ports are open for both servers in both iptables. This morning, I tried to extend this to the web root partition (12.5 GB) and expected similiar results. But it did not run stable. After the first installation, I made an error with the path settings on the second host so I stopped Apache/... and tried to recreate the volume. But Apache conflicted in some way with the ongoing first sync, the httpd-processes freezed and could not be killed. I had to reboot the machine. After coming up, Gluster treated the part which was synced to the backup server as valid and deleted the rest from the mail root partition on the master server. Sometimes, extensive backups are really useful. I recreated the volume and copied the backup data from the second server onto the volume (which synced the data to the main server, too). After this was nearly complete, I restarted Apache on the main server. This combination (restoring the last files and Apache accessing the volume) resulted in an increasing system load (95-97 % wait, load going from 20 up to 300) until I had a hard reset on the machine. After the reboot, everything seemed fine so far. But working with the file system and deleting some duplicate folder trees with mc, single folders were repeatedly reported as non-existent or unaccessible, their content vanished and reappeared. Finally, I dismounted these new gluster volumes and deleted them and remounted the local partitions to their original place and restored again missing files. Additionaly, I observed that in case of a reboot of the backup server, the mounted volumes become inaccessible on the main server although all information is local and the mount.glusterfs command always references the local machine when mounting the volume. Does anybody have a good hint for me to get this working as intended? Best regards.
