Seeing non-priv port + auth issue in the gluster brick log

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi All,
I am trying to mount a gluster volume from inside a VM (being used as a VDSM host) and seeing the below issue. From the VM, I am able to ping, telnet to the gluster host (no networking issues present)

*Client side*
============

[root@vdsm_tsm_int glusterfs]# mount -t glusterfs 9.121.60.166:dpkvol /mnt
Mount failed. Please check the log file for more details.

(Using IP or hostname in the mount cmdline doesn't change anything)

mnt.log
-------

[2013-05-11 06:38:54.199518] I [glusterfsd.c:1878:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.4.0beta1 (/usr/sbin/glusterfs --volfile-id=dpkvol --volfile-server=9.121.60.166 /mnt) [2013-05-11 06:38:54.203474] I [socket.c:3480:socket_init] 0-glusterfs: SSL support is NOT enabled [2013-05-11 06:38:54.203652] I [socket.c:3495:socket_init] 0-glusterfs: using system polling thread [2013-05-11 06:38:54.206592] W [common-utils.c:2330:gf_ports_reserved] 0-glusterfs-socket: is not a valid port identifier [2013-05-11 06:38:55.253221] I [socket.c:3480:socket_init] 0-dpkvol-client-0: SSL support is NOT enabled [2013-05-11 06:38:55.253268] I [socket.c:3495:socket_init] 0-dpkvol-client-0: using system polling thread [2013-05-11 06:38:55.253300] I [client.c:2154:notify] 0-dpkvol-client-0: parent translators are ready, attempting connect on transport [2013-05-11 06:38:55.255834] W [common-utils.c:2330:gf_ports_reserved] 0-glusterfs-socket: is not a valid port identifier
Given volfile:
+------------------------------------------------------------------------------+
  1: volume dpkvol-client-0
  2:     type protocol/client
  3:     option transport-type tcp
  4:     option remote-subvolume /home/dpkshetty/brick
  5:     option remote-host llmvm03
  6: end-volume
  7:
  8: volume dpkvol-dht
  9:     type cluster/distribute
 10:     subvolumes dpkvol-client-0
 11: end-volume
 12:
 13: volume dpkvol-write-behind
 14:     type performance/write-behind
 15:     subvolumes dpkvol-dht
 16: end-volume
 17:
 18: volume dpkvol-read-ahead
 19:     type performance/read-ahead
 20:     subvolumes dpkvol-write-behind
 21: end-volume
 22:
 23: volume dpkvol-io-cache
 24:     type performance/io-cache
 25:     subvolumes dpkvol-read-ahead
 26: end-volume
 27:
 28: volume dpkvol-quick-read
 29:     type performance/quick-read
 30:     subvolumes dpkvol-io-cache
 31: end-volume
 32:
 33: volume dpkvol-open-behind
 34:     type performance/open-behind
 35:     subvolumes dpkvol-quick-read
 36: end-volume
 37:
 38: volume dpkvol-md-cache
 39:     type performance/md-cache
 40:     subvolumes dpkvol-open-behind
 41: end-volume
 42:
 43: volume dpkvol
 44:     type debug/io-stats
 45:     option count-fop-hits off
 46:     option latency-measurement off
 47:     subvolumes dpkvol-md-cache
 48: end-volume

+------------------------------------------------------------------------------+
[2013-05-11 06:38:57.007678] I [rpc-clnt.c:1670:rpc_clnt_reconfig] 0-dpkvol-client-0: changing port to 49152 (from 0) [2013-05-11 06:38:57.007973] W [socket.c:514:__socket_rwv] 0-dpkvol-client-0: readv failed (No data available) [2013-05-11 06:38:57.020391] W [common-utils.c:2330:gf_ports_reserved] 0-glusterfs-socket: is not a valid port identifier [2013-05-11 06:38:58.375306] I [client-handshake.c:1658:select_server_supported_programs] 0-dpkvol-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330) [2013-05-11 06:38:59.077357] W [client-handshake.c:1365:client_setvolume_cbk] 0-dpkvol-client-0: failed to set the volume (Permission denied) [2013-05-11 06:38:59.077535] W [client-handshake.c:1391:client_setvolume_cbk] 0-dpkvol-client-0: failed to get 'process-uuid' from reply dict [2013-05-11 06:38:59.077571] E [client-handshake.c:1397:client_setvolume_cbk] 0-dpkvol-client-0: SETVOLUME on remote-host failed: Authentication failed [2013-05-11 06:38:59.077606] I [client-handshake.c:1482:client_setvolume_cbk] 0-dpkvol-client-0: sending AUTH_FAILED event [2013-05-11 06:38:59.077647] E [fuse-bridge.c:4788:notify] 0-fuse: Server authenication failed. Shutting down. [2013-05-11 06:38:59.077680] I [fuse-bridge.c:5212:fini] 0-fuse: Unmounting '/mnt'. [2013-05-11 06:38:59.082462] W [glusterfsd.c:970:cleanup_and_exit] (-->/usr/lib64/libc.so.6(clone+0x6d) [0x3cbd0f199d] (-->/usr/lib64/libpthread.so.0() [0x3cbd407d14] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xc5) [0x406be5]))) 0-: received signum (15), shutting down

version
---------

[root@vdsm_tsm_int glusterfs]# gluster --version
glusterfs 3.4.0beta1 built on May 10 2013 17:55:27
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.



*Server side* (gluster host)
=============

brick log
----------

[2013-05-11 06:40:19.912512] E [addr.c:152:gf_auth] 0-auth/addr: client is bound to port 1070 which is not privileged [2013-05-11 06:40:19.912610] E [authenticate.c:246:gf_authenticate] 0-auth: no authentication module is interested in accepting remote-client (null) [2013-05-11 06:40:19.912639] E [server-handshake.c:587:server_setvolume] 0-dpkvol-server: Cannot authenticate client from vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0 3.4.0beta1 [2013-05-11 06:40:20.611853] I [server.c:771:server_rpc_notify] 0-dpkvol-server: disconnecting connectionfrom vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0 [2013-05-11 06:40:20.611908] I [server-helpers.c:735:server_connection_put] 0-dpkvol-server: Shutting down connection vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0 [2013-05-11 06:40:20.611937] I [server-helpers.c:623:server_connection_destroy] 0-dpkvol-server: destroyed connection of vdsm_tsm_int-7221-2013/05/11-06:38:54:195128-dpkvol-client-0-0

glusterd.vol
------------

volume management
    type mgmt/glusterd
    option working-directory /var/lib/glusterd
    option transport-type socket,rdma
    option transport.socket.keepalive-time 10
    option transport.socket.keepalive-interval 2
    option transport.socket.read-fail-log off
    option rpc-auth-allow-insecure on
end-volume

brick vol
----------

(snip..)
volume dpkvol-server
    type protocol/server
    option rpc-auth-allow-insecure on
    option auth.addr./home/dpkshetty/brick.allow null
option auth.login.91e851d7-2f55-4946-abc2-dd18eeba4a93.password 6c740e73-ff98-4991-b08c-1f1e9c0f9ee9 option auth.login./home/dpkshetty/brick.allow 91e851d7-2f55-4946-abc2-dd18eeba4a93
    option transport-type tcp
    subvolumes /home/dpkshetty/brick
end-volume

(removing auth.login lines also doesnt have any effect)

volume info
-----------

gluster volume info

Volume Name: dpkvol
Type: Distribute
Volume ID: 71fb0238-6661-4c06-ba5c-7f36c399330c
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: llmvm03:/home/dpkshetty/brick
Options Reconfigured:
server.allow-insecure: on

version
-------

gluster --version
glusterfs 3.4.0alpha2 built on Apr 10 2013 16:21:16
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.



*Questions/Observations*
========================

1) Inspite of having server.allow-insecure: on (in volume option) and option rpc-auth-allow-insecure on (in glusterd volfile)... why do i still see the non-priv port error in brick logs ? (I am running the mount cmdline as root inside VM, but its possible QEMU (which hosts the VM) is translating the n/w port to non-priv, but still auth allow insecure should have taken care of this)

2) For the same gluster host, if i try to connect from my laptop (instead of VM), mount works and i don't see any errors in the brick log

3) gluster --remote-host=<server ip/host> volume info works from inside VM (which means rpc-auth-allow-insecure on (in glusterd volfile) is working for this case), but not working for the mount case

4) The auth issue (from my obs) is kicking in only when non-priv port is being detected on the server side.. but that should not be the case as insecure options are set

5) Could the version mismatch between the client server be any reason here ?

thanx,
deepak






[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux