Hi, Using samba on top of gluster, and when one brick goes down I can't see files that are actively being made on functional bricks. Running 2.0.0, I have 2 machines in a NUFA arrangement (log below), gluster mounted on /mnt, sharing /mnt out from one node via samba. On the machine that is doing the samba I do a mkdir /mnt/me, then start copying a big file into /mnt/me. I can do a ls -l /mnt/me and see the file growing in there. I take down my second node (umount /mnt), back on the first node everything is fine, the copy continues since the file is going to a local brick anyway. I can do a ls -l /mnt/me and see the file growing there fine. I repeat the above but I'm copying a file into /mnt/me via samba. When the second node goes down, the copy continues, but I can't see the file in /mnt/me (a ls -l /mnt/me/nameoffile shows it, but a ls -l /mnt/me doesn't). If I start a new copy while the second node is down, then a ls -l /mnt/me shows that file fine. It is only files that are in the process of being copied while the second node goes down, and only under samba, that I see the problem. I thought it might be something to do with samba preallocating the entire file, then filling it up during the copy. I set 'strict allocate = yes' in my smb.conf, which doesn't stop the preallocating but it does make the preallocation real slow, slow enough that i can watch the file growing and then take the second node down, but then wham it is then not visible via a ls -l on its directory. It might not sound like a huge issue, since the copy works. But I am currently testing our gluster and demonstrating to a user that it will work OK in the situation when a brick other than actively used one on the samba server goes down. Please, any help much appreciated. Matt --- Version : glusterfs 2.0.0 built on May 6 2009 09:02:18 TLA Revision : 7b2e459db65edd302aa12476bc73b3b7a17b1410 Starting Time: 2009-05-06 09:05:19 Command line : /usr/local/sbin/glusterfs --log-level=NORMAL --volfile=/usr/local/etc/glusterfs/glusterfsd3.vol /mnt PID : 20866 System name : Linux Nodename : mu-rhdev1 Kernel Release : 2.6.18-128.1.6.el5 Hardware Identifier: x86_64 Given volfile: +------------------------------------------------------------------------------+ 1: volume posix0 2: type storage/posix 3: option directory /export/brick0 4: end-volume 5: 6: volume locks0 7: type features/locks 8: subvolumes posix0 9: end-volume 10: 11: volume brick0 12: type performance/io-threads 13: subvolumes locks0 14: end-volume 15: 16: volume posix1 17: type storage/posix 18: option directory /export/brick1 19: end-volume 20: 21: volume locks1 22: type features/locks 23: subvolumes posix1 24: end-volume 25: 26: volume brick1 27: type performance/io-threads 28: subvolumes locks1 29: end-volume 30: 31: volume server 32: type protocol/server 33: option transport-type tcp 34: option auth.addr.brick0.allow * 35: option auth.addr.brick1.allow * 36: subvolumes brick0 brick1 37: end-volume 38: 39: volume mu-rhdev1-0 40: type protocol/client 41: option transport-type tcp 42: option remote-host mu-rhdev1 43: option remote-subvolume brick0 44: end-volume 45: 46: volume mu-rhdev1-1 47: type protocol/client 48: option transport-type tcp 49: option remote-host mu-rhdev1 50: option remote-subvolume brick1 51: end-volume 52: 53: volume mu-rhdev2-0 54: type protocol/client 55: option transport-type tcp 56: option remote-host mu-rhdev2 57: option remote-subvolume brick0 58: end-volume 59: 60: volume nufa 61: type cluster/nufa 62: option local-volume-name mu-rhdev1-0 63: subvolumes mu-rhdev1-0 mu-rhdev1-1 mu-rhdev2-0 64: end-volume 65: 66: volume writebehind 67: type performance/write-behind 68: option cache-size 1MB 69: subvolumes nufa 70: end-volume 71: 72: # before or after writebehind? 73: volume ra 74: type performance/read-ahead 75: subvolumes writebehind 76: end-volume 77: 78: volume cache 79: type performance/io-cache 80: option cache-size 512MB 81: subvolumes ra 82: end-volume 83: +------------------------------------------------------------------------------+ 2009-05-06 10:33:15 N [glusterfsd.c:1152:main] glusterfs: Successfully started 2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused) 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected 2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused) 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected 2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-1: connection failed (Connection refused) 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected 2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-1: connection failed (Connection refused) 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected 2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-0: connection failed (Connection refused) 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected 2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-0: connection failed (Connection refused) 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected 2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected 2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1014 2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1017 2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1016 2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1015 2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1019 2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-0: attaching to the local volume 'brick0' 2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-0: connection and handshake succeeded 2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1018 2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-0: attaching to the local volume 'brick0' 2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-0: connection and handshake succeeded 2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1017 2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-1: attaching to the local volume 'brick1' 2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-1: connection and handshake succeeded 2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1016 2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-1: attaching to the local volume 'brick1' 2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-1: connection and handshake succeeded 2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded 2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded 2009-05-06 10:33:29 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected 2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1014 disconnected 2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1015 disconnected 2009-05-06 10:33:29 N [server-helpers.c:723:server_connection_destroy] server: destroyed connection of mu-rhdev2-7959-2009/05/06-10:33:16:708168-mu-rhdev1-1 2009-05-06 10:33:29 E [socket.c:102:__socket_rwv] server: readv failed (Connection reset by peer) 2009-05-06 10:33:29 E [socket.c:562:__socket_proto_state_machine] server: read (Connection reset by peer) in state 1 (130.123.131.13:1016) 2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1016 disconnected 2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1017 disconnected 2009-05-06 10:33:29 N [server-helpers.c:723:server_connection_destroy] server: destroyed connection of mu-rhdev2-7959-2009/05/06-10:33:16:708168-mu-rhdev1-0 2009-05-06 10:33:38 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused) 2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected 2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected 2009-05-06 10:33:38 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused) 2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected 2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected 2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1009 2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1016 2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1011 2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1010 2009-05-06 10:33:45 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded 2009-05-06 10:33:45 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded
