just to know whether it is the problem of samba re-export, instead of re-exporting the client mount point through samba, can you re-export it through glusterfs itself and try whether you still find the problem?
regards,
On Wed, May 6, 2009 at 3:23 AM, <m.c.wilkins@xxxxxxxxxxxx> wrote:
Hi,
Using samba on top of gluster, and when one brick goes down I can't
see files that are actively being made on functional bricks.
Running 2.0.0, I have 2 machines in a NUFA arrangement (log below),
gluster mounted on /mnt, sharing /mnt out from one node via samba. On
the machine that is doing the samba I do a mkdir /mnt/me, then start
copying a big file into /mnt/me. I can do a ls -l /mnt/me and see the
file growing in there. I take down my second node (umount /mnt), back
on the first node everything is fine, the copy continues since the
file is going to a local brick anyway. I can do a ls -l /mnt/me and
see the file growing there fine.
I repeat the above but I'm copying a file into /mnt/me via samba.
When the second node goes down, the copy continues, but I can't see
the file in /mnt/me (a ls -l /mnt/me/nameoffile shows it, but a ls -l
/mnt/me doesn't). If I start a new copy while the second node is
down, then a ls -l /mnt/me shows that file fine. It is only files
that are in the process of being copied while the second node goes
down, and only under samba, that I see the problem.
I thought it might be something to do with samba preallocating the
entire file, then filling it up during the copy. I set 'strict
allocate = yes' in my smb.conf, which doesn't stop the preallocating
but it does make the preallocation real slow, slow enough that i can
watch the file growing and then take the second node down, but then
wham it is then not visible via a ls -l on its directory.
It might not sound like a huge issue, since the copy works. But I am
currently testing our gluster and demonstrating to a user that it will
work OK in the situation when a brick other than actively used one on
the samba server goes down.
Please, any help much appreciated.
Matt
---
Version : glusterfs 2.0.0 built on May 6 2009 09:02:18
TLA Revision : 7b2e459db65edd302aa12476bc73b3b7a17b1410
Starting Time: 2009-05-06 09:05:19
Command line : /usr/local/sbin/glusterfs --log-level=NORMAL --volfile=/usr/local/etc/glusterfs/glusterfsd3.vol /mnt
PID : 20866
System name : Linux
Nodename : mu-rhdev1
Kernel Release : 2.6.18-128.1.6.el5
Hardware Identifier: x86_64
Given volfile:
+------------------------------------------------------------------------------+
1: volume posix0
2: type storage/posix
3: option directory /export/brick0
4: end-volume
5:
6: volume locks0
7: type features/locks
8: subvolumes posix0
9: end-volume
10:
11: volume brick0
12: type performance/io-threads
13: subvolumes locks0
14: end-volume
15:
16: volume posix1
17: type storage/posix
18: option directory /export/brick1
19: end-volume
20:
21: volume locks1
22: type features/locks
23: subvolumes posix1
24: end-volume
25:
26: volume brick1
27: type performance/io-threads
28: subvolumes locks1
29: end-volume
30:
31: volume server
32: type protocol/server
33: option transport-type tcp
34: option auth.addr.brick0.allow *
35: option auth.addr.brick1.allow *
36: subvolumes brick0 brick1
37: end-volume
38:
39: volume mu-rhdev1-0
40: type protocol/client
41: option transport-type tcp
42: option remote-host mu-rhdev1
43: option remote-subvolume brick0
44: end-volume
45:
46: volume mu-rhdev1-1
47: type protocol/client
48: option transport-type tcp
49: option remote-host mu-rhdev1
50: option remote-subvolume brick1
51: end-volume
52:
53: volume mu-rhdev2-0
54: type protocol/client
55: option transport-type tcp
56: option remote-host mu-rhdev2
57: option remote-subvolume brick0
58: end-volume
59:
60: volume nufa
61: type cluster/nufa
62: option local-volume-name mu-rhdev1-0
63: subvolumes mu-rhdev1-0 mu-rhdev1-1 mu-rhdev2-0
64: end-volume
65:
66: volume writebehind
67: type performance/write-behind
68: option cache-size 1MB
69: subvolumes nufa
70: end-volume
71:
72: # before or after writebehind?
73: volume ra
74: type performance/read-ahead
75: subvolumes writebehind
76: end-volume
77:
78: volume cache
79: type performance/io-cache
80: option cache-size 512MB
81: subvolumes ra
82: end-volume
83:
+------------------------------------------------------------------------------+
2009-05-06 10:33:15 N [glusterfsd.c:1152:main] glusterfs: Successfully started
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-1: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-1: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-0: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-0: connection failed (Connection refused)
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1014
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1017
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1016
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1015
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1019
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-0: attaching to the local volume 'brick0'
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-0: connection and handshake succeeded
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1018
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-0: attaching to the local volume 'brick0'
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-0: connection and handshake succeeded
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1017
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-1: attaching to the local volume 'brick1'
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-1: connection and handshake succeeded
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.2:1016
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-1: attaching to the local volume 'brick1'
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-1: connection and handshake succeeded
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded
2009-05-06 10:33:29 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1014 disconnected
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1015 disconnected
2009-05-06 10:33:29 N [server-helpers.c:723:server_connection_destroy] server: destroyed connection of mu-rhdev2-7959-2009/05/06-10:33:16:708168-mu-rhdev1-1
2009-05-06 10:33:29 E [socket.c:102:__socket_rwv] server: readv failed (Connection reset by peer)
2009-05-06 10:33:29 E [socket.c:562:__socket_proto_state_machine] server: read (Connection reset by peer) in state 1 (130.123.131.13:1016)
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1016 disconnected
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: 130.123.131.13:1017 disconnected
2009-05-06 10:33:29 N [server-helpers.c:723:server_connection_destroy] server: destroyed connection of mu-rhdev2-7959-2009/05/06-10:33:16:708168-mu-rhdev1-0
2009-05-06 10:33:38 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:38 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1009
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1016
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1011
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from 130.123.131.13:1010
2009-05-06 10:33:45 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded
2009-05-06 10:33:45 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxx
http://lists.nongnu.org/mailman/listinfo/gluster-devel
--
Raghavendra G
