Hi,
I’ll try to make myself clearer than in the earlier thread, since I need some help here. I’m not sure if I am missing something.
I am not able to use atomic operations using glusterfs 3.0.2, on a Arch Linux 2.6.32.7-1 x86_64 server.
I’ve stripped everything down to the most simple configuration
Server: a posix store with locks
Client: a tcp client
I mount the glusterfs on /gfs, and use a “writer” and a “reader” as below:
[writer] root@frustration> while :; do date > /gfs/.file; mv /gfs/.file /gfs/file; done
[reader] root@frustration> while :; do cat /gfs/file > /dev/null; done
Leaving this running, I get a flood of
cat: /gfs/file: No such file or directory
This should not happen on a filesystem that supports atomic operations?!
What this means is that I am unable to replace a file as an atomic operation, creating a race-condition where the file is sometimes not present. Furthermore this actually “desyncs” the server in a more complex setup, requiring a restart of the server part to make “desynced” files available again.
This reproduces using XFS as well as EXT2 as the underlying fs.
[glusterfs.vol]
volume media0
type protocol/client
option transport-type tcp
option remote-host 127.0.0.1
option remote-subvolume media0
end-volume
mounted with “glusterfs -f glusterfs.vol /gfs”
[glusterfsd.vol]
(se below debug log)
-x-
[2010-02-09 18:18:23] D [glusterfsd.c:424:_get_specfp] glusterfs: loading volume file glusterfsd.vol
[2010-02-09 18:18:23] D [xlator.c:739:xlator_set_type] xlator: dlsym(notify) on /usr/lib/glusterfs/3.0.2/xlator/features/locks.so: undefined symbol: notify -- neglecting
================================================================================
Version : glusterfs 3.0.2 built on Feb 8 2010 15:23:34
git: v3.0.2
Starting Time: 2010-02-09 18:18:23
Command line : glusterfs -f glusterfsd.vol --debug
PID : 19486
System name : Linux
Nodename : n100.p1.x0.local
Kernel Release : 2.6.32-ARCH
Hardware Identifier: x86_64
Given volfile:
+------------------------------------------------------------------------------+
1: volume media0-store
2: type storage/posix
3: option directory /mnt
4: end-volume
5:
6: volume media0
7: type features/locks
8: subvolumes media0-store
9: end-volume
10:
11: volume server
12: type protocol/server
13: option transport-type tcp
14: option auth.addr.media0.allow *
15: subvolumes media0
16: end-volume
+------------------------------------------------------------------------------+
[2010-02-09 18:18:23] D [glusterfsd.c:1370:main] glusterfs: running in pid 19486
[2010-02-09 18:18:23] D [transport.c:145:transport_load] transport: attempt to load file /usr/lib/glusterfs/3.0.2/transport/socket.so
[2010-02-09 18:18:23] N [glusterfsd.c:1396:main] glusterfs: Successfully started
[2010-02-09 18:18:43] D [addr.c:190:gf_auth] media0: allowed = "*", received addr = "127.0.0.1"
[2010-02-09 18:18:43] N [server-protocol.c:5812:mop_setvolume] server: accepted client from 127.0.0.1:1023
[2010-02-09 18:18:43] D [addr.c:190:gf_auth] media0: allowed = "*", received addr = "127.0.0.1"
[2010-02-09 18:18:43] N [server-protocol.c:5812:mop_setvolume] server: accepted client from 127.0.0.1:1022
[2010-02-09 18:18:58] D [server-resolve.c:238:resolve_path_deep] media0: RESOLVE RENAME() seeking deep resolution of /file
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f79b
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7b3
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7cb
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f79b
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7b3
[2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7cb
[2010-02-09 18:19:15] D [inode.c:916:inode_path] media0/inode: no dentry for non-root inode 502725
[2010-02-09 18:19:34] D [inode.c:916:inode_path] media0/inode: no dentry for non-root inode 502734
[2010-02-09 18:19:56] D [inode.c:916:inode_path] media0/inode: no dentry for non-root inode 502734
[…] repeated many times. Frequency of the “no dentry” entries depends on how aggressive the reader is.
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxx
http://lists.nongnu.org/mailman/listinfo/gluster-devel
On Tue, Feb 9, 2010 at 11:18 AM, Fredrik Widlund <fredrik.widlund@xxxxxxxxxx> wrote:
