Hi, I'm trying to put together a fully redundant, no-SPOF storage system using GlusterFS. Everything works fine as long as I don't introduce locking, which is essential if one wishes to use the storage eg. as a backend for a mail server. On each server I have a features/posix-locks translator on top of a storage/posix brick. This volume is exported via protocol/server, and AFR'ed together with the same (features/posix-locks) bricks from the other servers. The order of the subvolumes for each AFR is the same on all servers. (So that in theory, for the first time, all servers ask the first server to lock a file, then all of them ask the second server, and so on). The resulting AFR volume is exported again, and mounted into a directory (using 'localhost' as remote-host). You can find the exact configuration at aht end of this mail. GlusterFS is a fresh tla checkout. I expect that if a file is locked on one of the servers, the same file should get locked on the other servers, too. So a client trying to lock the same file on another server, it should get blocked or EAGAIN returned. Now I test this setup with locktests, suggested on the GlusterFS QA page. I use the network-based tests to simulate multiple server processes locking and releasing files. Here are the results from the host running the master locking process: > ./locktests -n 30 -f /mnt/glusterfs/locktest -c 2 > Init > process initalization > ............................................................ > -------------------------------------- > > TEST : TRY TO WRITE ON A READ LOCK:============================== > TEST : TRY TO WRITE ON A WRITE LOCK:============================== > TEST : TRY TO READ ON A READ LOCK:============================== > TEST : TRY TO READ ON A WRITE LOCK:============================== > TEST : TRY TO SET A READ LOCK ON A READ LOCK:============================== > TEST : TRY TO SET A WRITE LOCK ON A WRITE LOCK:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx > TEST : TRY TO SET A WRITE LOCK ON A READ LOCK:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx > TEST : TRY TO SET A READ LOCK ON A WRITE LOCK:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx > TEST : TRY TO READ LOCK THE WHOLE FILE BYTE BY BYTE:============================== > TEST : TRY TO WRITE LOCK THE WHOLE FILE BYTE BY BYTE:============================== > > process number : 30 - Remote clients: 2 local client 1 - Total client 3 - Total concurent tests: 90 > process number running test successfully : > 90 process of 90 successfully ran test : WRITE ON A READ LOCK > 90 process of 90 successfully ran test : WRITE ON A WRITE LOCK > 90 process of 90 successfully ran test : READ ON A READ LOCK > 90 process of 90 successfully ran test : READ ON A WRITE LOCK > 90 process of 90 successfully ran test : SET A READ LOCK ON A READ LOCK > 88 process of 90 successfully ran test : SET A WRITE LOCK ON A WRITE LOCK > 88 process of 90 successfully ran test : SET A WRITE LOCK ON A READ LOCK > 30 process of 90 successfully ran test : SET A READ LOCK ON A WRITE LOCK > 30 process of 90 successfully ran test : READ LOCK THE WHOLE FILE BYTE BY BYTE > 30 process of 90 successfully ran test : WRITE LOCK THE WHOLE FILE BYTE BY BYTE On the other two slave servers I see: > 0 > Init > process initalization > ............................................................Maitre CLient - fdServeur=3 > ============================== > ============================== > ============================== > ============================== > ============================== > =xxxxxxxxxxxxxxxxxxxxxxxxxxxxx > =xxxxxxxxxxxxxxxxxxxxxxxxxxxxx > ============================== > > ============================== > > ============================== > Fin du programme en cours... > Bye :) Do you have any idea how to configure GlusterFS so the test runs successfully? Is a setup like this possible with the current posix-locks translator? Thanks, -- cc Here are my configs: ==server (this is from the server named 'drbubo', others are similar)== volume data-posix type storage/posix option directory /srv/glusterfs end-volume volume data-drbubo type features/posix-locks subvolumes data-posix end-volume volume data-ursula type protocol/client option transport-type tcp/client option remote-host ursula option remote-subvolume data-ursula end-volume volume data-sf3 type protocol/client option transport-type tcp/client option remote-host sf3 option remote-subvolume data-sf3 end-volume volume data-afr type cluster/afr subvolumes data-ursula data-drbubo data-sf3 end-volume volume server type protocol/server subvolumes data-drbubo data-afr option transport-type tcp/server option auth.ip.data-drbubo.allow <IPs of all servers> option auth.ip.data-afr.allow 127.0.0.1 end-volume ==client== volume client type protocol/client option transport-type tcp/client option remote-host localhost option remote-subvolume data-afr end-volume
