I posted this a while back:
http://lists.nongnu.org/archive/html/gluster-devel/2007-08/msg00124.html
In summary: Performing a flock operation on a newly created file does
not always lock it from other clients.
Process: Open a new file and obtain a lock (flock in my tests) within
the same process. Before closing the file, open and try to obtain a
lock from a separate process from a different glusterfs client. There
is no blocking.
Notes: If I create the file beforehand (touch it), the flock operation
performs as expected. Looking at the servers, the file is created
immediately from the first client, the second client is just not
blocking on the flock call.
Here's a trasncript of operations on the servers, to illustrate the
problem. Both servers are time synced, drift between them is sub 1 second:
# Client A
# ll /mnt/glusterfs/
total 8
-rwxr-xr-x 1 root root 746 Sep 21 12:02 testlock.pl
# Client B
# ll /mnt/glusterfs/
total 8
-rwxr-xr-x 1 root root 746 Sep 21 12:02 testlock.pl
# Client A
# /mnt/glusterfs/testlock.pl /mnt/glusterfs/testfile 10
testing flock on lockfile /mnt/glusterfs/testfile with 10 second delay
on php5-a.graphita.com
opening file
Fri Sep 21 14:26:19 2007 :: opened file
Fri Sep 21 14:26:19 2007 :: locking file
Fri Sep 21 14:26:19 2007 :: locked file
Fri Sep 21 14:26:29 2007 :: unlocked file
# Client B
# /mnt/glusterfs/testlock.pl /mnt/glusterfs/testfile 10
testing flock on lockfile /mnt/glusterfs/testfile with 10 second delay
on php5-b.graphita.com
opening file
Fri Sep 21 14:26:21 2007 :: opened file
Fri Sep 21 14:26:21 2007 :: locking file
Fri Sep 21 14:26:21 2007 :: locked file
Fri Sep 21 14:26:31 2007 :: unlocked file
Notice that the second client ignored the lock of the first? Not let's
try this again but with the file already existing:
# Client A
# rm -f /mnt/glusterfs/testfile
# ll /mnt/glusterfs/
total 8
-rwxr-xr-x 1 root root 746 Sep 21 12:02 testlock.pl
# touch /mnt/glusterfs/testfile
# ll /mnt/glusterfs/
total 12
-rw-r--r-- 1 root root 0 Sep 21 14:29 testfile
-rwxr-xr-x 1 root root 746 Sep 21 12:02 testlock.pl
# Client B
# ll /mnt/glusterfs/
total 12
-rw-r--r-- 1 root root 0 Sep 21 14:29 testfile
-rwxr-xr-x 1 root root 746 Sep 21 12:02 testlock.pl
# Client A
# /mnt/glusterfs/testlock.pl /mnt/glusterfs/testfile 10
testing flock on lockfile /mnt/glusterfs/testfile with 10 second delay
on php5-a.graphita.com
opening file
Fri Sep 21 14:30:33 2007 :: opened file
Fri Sep 21 14:30:33 2007 :: locking file
Fri Sep 21 14:30:33 2007 :: locked file
Fri Sep 21 14:30:43 2007 :: unlocked file
# Client B
# /mnt/glusterfs/testlock.pl /mnt/glusterfs/testfile 10
testing flock on lockfile /mnt/glusterfs/testfile with 10 second delay
on php5-b.graphita.com
opening file
Fri Sep 21 14:30:36 2007 :: opened file
Fri Sep 21 14:30:36 2007 :: locking file
Fri Sep 21 14:30:43 2007 :: locked file
Fri Sep 21 14:30:53 2007 :: unlocked file
Now it flocks fine. Client B had to wait for Client A to unlock for it
could lock. Order of operations doesn't seem to matter, nor which
client creates the file compared to which locks first when the file is
created prior to lock testing.
Here's the contents of testlock.pl:
#!/usr/bin/perl
$| = 1;
use Fcntl ':flock';
my $file = shift or "./testlockfile";
my $delay = shift or 10;
my $hostname = `hostname`;
chomp($hostname);
print "testing flock on lockfile $file with $delay second delay on
$hostname\n";
print "opening file\n";
open(my $testfile, '>>', $file) or die("$!\n");
print localtime(time())." :: opened file\n";
print localtime(time())." :: locking file\n";
flock($testfile, LOCK_EX) or die("Error locking :: $!\n");
print localtime(time())." :: locked file\n";
print $testfile "$hostname start ".localtime(time())."\n";
sleep $delay;
print $testfile "$hostname end ".localtime(time())."\n";
flock($testfile, LOCK_UN) or die("Error locking :: $!\n");
print localtime(time())." :: unlocked file\n";
exit 0;
