Hi All,
I am trying to mount a gluster volume from inside a VM (being used
as a VDSM host) and seeing the below issue.
From the VM, I am able to ping, telnet to the gluster host (no
networking issues present)
*Client side*
============
[root@vdsm_tsm_int glusterfs]# mount -t glusterfs 9.121.60.166:dpkvol /mnt
Mount failed. Please check the log file for more details.
(Using IP or hostname in the mount cmdline doesn't change anything)
mnt.log
-------
[2013-05-11 06:38:54.199518] I [glusterfsd.c:1878:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version
3.4.0beta1 (/usr/sbin/glusterfs --volfile-id=dpkvol
--volfile-server=9.121.60.166 /mnt)
[2013-05-11 06:38:54.203474] I [socket.c:3480:socket_init] 0-glusterfs:
SSL support is NOT enabled
[2013-05-11 06:38:54.203652] I [socket.c:3495:socket_init] 0-glusterfs:
using system polling thread
[2013-05-11 06:38:54.206592] W [common-utils.c:2330:gf_ports_reserved]
0-glusterfs-socket: is not a valid port identifier
[2013-05-11 06:38:55.253221] I [socket.c:3480:socket_init]
0-dpkvol-client-0: SSL support is NOT enabled
[2013-05-11 06:38:55.253268] I [socket.c:3495:socket_init]
0-dpkvol-client-0: using system polling thread
[2013-05-11 06:38:55.253300] I [client.c:2154:notify] 0-dpkvol-client-0:
parent translators are ready, attempting connect on transport
[2013-05-11 06:38:55.255834] W [common-utils.c:2330:gf_ports_reserved]
0-glusterfs-socket: is not a valid port identifier
Given volfile:
+------------------------------------------------------------------------------+
1: volume dpkvol-client-0
2: type protocol/client
3: option transport-type tcp
4: option remote-subvolume /home/dpkshetty/brick
5: option remote-host llmvm03
6: end-volume
7:
8: volume dpkvol-dht
9: type cluster/distribute
10: subvolumes dpkvol-client-0
11: end-volume
12:
13: volume dpkvol-write-behind
14: type performance/write-behind
15: subvolumes dpkvol-dht
16: end-volume
17:
18: volume dpkvol-read-ahead
19: type performance/read-ahead
20: subvolumes dpkvol-write-behind
21: end-volume
22:
23: volume dpkvol-io-cache
24: type performance/io-cache
25: subvolumes dpkvol-read-ahead
26: end-volume
27:
28: volume dpkvol-quick-read
29: type performance/quick-read
30: subvolumes dpkvol-io-cache
31: end-volume
32:
33: volume dpkvol-open-behind
34: type performance/open-behind
35: subvolumes dpkvol-quick-read
36: end-volume
37:
38: volume dpkvol-md-cache
39: type performance/md-cache
40: subvolumes dpkvol-open-behind
41: end-volume
42:
43: volume dpkvol
44: type debug/io-stats
45: option count-fop-hits off
46: option latency-measurement off
47: subvolumes dpkvol-md-cache
48: end-volume
+------------------------------------------------------------------------------+
[2013-05-11 06:38:57.007678] I [rpc-clnt.c:1670:rpc_clnt_reconfig]
0-dpkvol-client-0: changing port to 49152 (from 0)
[2013-05-11 06:38:57.007973] W [socket.c:514:__socket_rwv]
0-dpkvol-client-0: readv failed (No data available)
[2013-05-11 06:38:57.020391] W [common-utils.c:2330:gf_ports_reserved]
0-glusterfs-socket: is not a valid port identifier
[2013-05-11 06:38:58.375306] I
[client-handshake.c:1658:select_server_supported_programs]
0-dpkvol-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2013-05-11 06:38:59.077357] W
[client-handshake.c:1365:client_setvolume_cbk] 0-dpkvol-client-0: failed
to set the volume (Permission denied)
[2013-05-11 06:38:59.077535] W
[client-handshake.c:1391:client_setvolume_cbk] 0-dpkvol-client-0: failed
to get 'process-uuid' from reply dict
[2013-05-11 06:38:59.077571] E
[client-handshake.c:1397:client_setvolume_cbk] 0-dpkvol-client-0:
SETVOLUME on remote-host failed: Authentication failed
[2013-05-11 06:38:59.077606] I
[client-handshake.c:1482:client_setvolume_cbk] 0-dpkvol-client-0:
sending AUTH_FAILED event
[2013-05-11 06:38:59.077647] E [fuse-bridge.c:4788:notify] 0-fuse:
Server authenication failed. Shutting down.
[2013-05-11 06:38:59.077680] I [fuse-bridge.c:5212:fini] 0-fuse:
Unmounting '/mnt'.
[2013-05-11 06:38:59.082462] W [glusterfsd.c:970:cleanup_and_exit]
(-->/usr/lib64/libc.so.6(clone+0x6d) [0x3cbd0f199d]
(-->/usr/lib64/libpthread.so.0() [0x3cbd407d14]
(-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xc5) [0x406be5]))) 0-:
received signum (15), shutting down
version
---------
[root@vdsm_tsm_int glusterfs]# gluster --version
glusterfs 3.4.0beta1 built on May 10 2013 17:55:27
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU
General Public License.
*Server side* (gluster host)
=============
brick log
----------
[2013-05-11 06:40:19.912512] E [addr.c:152:gf_auth] 0-auth/addr: client
is bound to port 1070 which is not privileged
[2013-05-11 06:40:19.912610] E [authenticate.c:246:gf_authenticate]
0-auth: no authentication module is interested in accepting
remote-client (null)
[2013-05-11 06:40:19.912639] E [server-handshake.c:587:server_setvolume]
0-dpkvol-server: Cannot authenticate client from
vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0 3.4.0beta1
[2013-05-11 06:40:20.611853] I [server.c:771:server_rpc_notify]
0-dpkvol-server: disconnecting connectionfrom
vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0
[2013-05-11 06:40:20.611908] I
[server-helpers.c:735:server_connection_put] 0-dpkvol-server: Shutting
down connection
vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0
[2013-05-11 06:40:20.611937] I
[server-helpers.c:623:server_connection_destroy] 0-dpkvol-server:
destroyed connection of
vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0
glusterd.vol
------------
volume management
type mgmt/glusterd
option working-directory /var/lib/glusterd
option transport-type socket,rdma
option transport.socket.keepalive-time 10
option transport.socket.keepalive-interval 2
option transport.socket.read-fail-log off
option rpc-auth-allow-insecure on
end-volume
brick vol
----------
(snip..)
volume dpkvol-server
type protocol/server
option rpc-auth-allow-insecure on
option auth.addr./home/dpkshetty/brick.allow null
option auth.login.91e851d7-2f55-4946-abc2-dd18eeba4a93.password
6c740e73-ff98-4991-b08c-1f1e9c0f9ee9
option auth.login./home/dpkshetty/brick.allow
91e851d7-2f55-4946-abc2-dd18eeba4a93
option transport-type tcp
subvolumes /home/dpkshetty/brick
end-volume
(removing auth.login lines also doesnt have any effect)
volume info
-----------
gluster volume info
Volume Name: dpkvol
Type: Distribute
Volume ID: 71fb0238-6661-4c06-ba5c-7f36c399330c
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: llmvm03:/home/dpkshetty/brick
Options Reconfigured:
server.allow-insecure: on
version
-------
gluster --version
glusterfs 3.4.0alpha2 built on Apr 10 2013 16:21:16
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU
General Public License.
*Questions/Observations*
========================
1) Inspite of having server.allow-insecure: on (in volume option) and
option rpc-auth-allow-insecure on (in glusterd volfile)... why do i
still see the non-priv port error in brick logs ?
(I am running the mount cmdline as root inside VM, but its possible QEMU
(which hosts the VM) is translating the n/w port to non-priv, but still
auth allow insecure should have taken care of this)
2) For the same gluster host, if i try to connect from my laptop
(instead of VM), mount works and i don't see any errors in the brick log
3) gluster --remote-host=<server ip/host> volume info works from inside
VM (which means rpc-auth-allow-insecure on (in glusterd volfile) is
working for this case), but not working for the mount case
4) The auth issue (from my obs) is kicking in only when non-priv port is
being detected on the server side.. but that should not be the case as
insecure options are set
5) Could the version mismatch between the client server be any reason
here ?
thanx,
deepak
