Hi Atin,
Thanks for reply.On Fri, Apr 1, 2016 at 1:04 PM, Atin Mukherjee <amukherj@xxxxxxxxxx> wrote:
On 04/01/2016 12:10 PM, ABHISHEK PALIWAL wrote:
> Hi,
>
>
> I have the setup of two boards A and B with two bricks in replica mode.
>
> There is one test scenario
>
> 1. A acts as an active board and having the glusterfs mount point on it.
> 2. B acts as Passive board.
> 3. We are repetitively rebooting the B board (In this time period peer
> status on A board will be "peer in cluster (Disconnected)" and brick is
> not present in "gluster volume status") and when Board B comes up,
> starts the gluster daemon.
> 4. if Gluster daemon starts successfully it will make "peer in
> cluster(Connected)"
> 5. At the same with the immediate effect "gluster volume status" command
> should show the brick is available in online.
>
>
> But in my case sometime step 5 takes immediate reflection sometime 10-15
> second and sometime doesn't show brick is online even after the 1minute.
>
> Could you please confirm why this type of unpredictable behavior is
> occuring. It should be reflect with immediate effect in "gluster volume
> status" command.
When glusterd restarts bricks processes are brought up asynchronously
and hence you may not see the brick processes reflecting in gluster
volume status output immediately after restart. 5-10 seconds is an
accepted time frame. However if it doesn't come back online post that
then probably brick fails to start in that case.
Please check the respective brick log file and if you can find any error
logs in it.
>
>
> --
> Regards
> Abhishek Paliwal
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel@xxxxxxxxxxx
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
--
Regards
Abhishek Paliwal
[2016-03-31 11:48:33.233242] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version 3.7.6 (args: /usr/sbin/glusterfsd -s 10.32.1.144 --volfile-id c_glusterfs.10.32.1.144.opt-lvmdir-c2-brick -p /system/glusterd/vols/c_glusterfs/run/10.32.1.144-opt-lvmdir-c2-brick.pid -S /var/run/gluster/697c0e4a16ebc734cd06fd9150723005.socket --brick-name /opt/lvmdir/c2/brick -l /var/log/glusterfs/bricks/opt-lvmdir-c2-brick.log --xlator-option *-posix.glusterd-uuid=2d576ff8-0cea-4f75-9e34-a5674fbf7256 --brick-port 49301 --xlator-option c_glusterfs-server.listen-port=49301) [2016-03-31 11:48:33.252639] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2016-03-31 11:48:33.269339] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-server: adding option 'listen-port' for volume 'c_glusterfs-server' with value '49301' [2016-03-31 11:48:33.269390] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-posix: adding option 'glusterd-uuid' for volume 'c_glusterfs-posix' with value '2d576ff8-0cea-4f75-9e34-a5674fbf7256' [2016-03-31 11:48:33.269816] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2016-03-31 11:48:33.269882] I [MSGID: 115034] [server.c:403:_check_for_auth_option] 0-/opt/lvmdir/c2/brick: skip format check for non-addr auth option auth.login./opt/lvmdir/c2/brick.allow [2016-03-31 11:48:33.269917] I [MSGID: 115034] [server.c:403:_check_for_auth_option] 0-/opt/lvmdir/c2/brick: skip format check for non-addr auth option auth.login.229f077d-bff9-4066-bc5e-88caa83bdd14.password [2016-03-31 11:48:33.271801] I [rpcsvc.c:2215:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured rpc.outstanding-rpc-limit with value 64 [2016-03-31 11:48:33.271984] W [MSGID: 101002] [options.c:957:xl_opt_validate] 0-c_glusterfs-server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction [2016-03-31 11:48:33.275928] I [trash.c:2366:init] 0-c_glusterfs-trash: no option specified for 'eliminate', using NULL [2016-03-31 11:48:33.278025] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-glusterfs' is not recognized [2016-03-31 11:48:33.278142] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-unix' is not recognized [2016-03-31 11:48:33.278175] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-null' is not recognized [2016-03-31 11:48:33.278229] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-quota: option 'timeout' is not recognized [2016-03-31 11:48:33.278261] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-marker: option 'quota-version' is not recognized [2016-03-31 11:48:33.278303] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-trash: option 'brick-path' is not recognized [2016-03-31 11:48:33.289956] W [MSGID: 113036] [posix.c:2018:posix_rename] 0-c_glusterfs-posix: found directory at /opt/lvmdir/c2/brick/.trashcan/ while expecting ENOENT [File exists] Final graph: +------------------------------------------------------------------------------+ 1: volume c_glusterfs-posix 2: type storage/posix 3: option glusterd-uuid 2d576ff8-0cea-4f75-9e34-a5674fbf7256 4: option directory /opt/lvmdir/c2/brick 5: option volume-id 3c9dd564-a282-436b-8ee5-4537d8e7118a 6: end-volume 7: 8: volume c_glusterfs-trash 9: type features/trash 10: option trash-dir .trashcan 11: option brick-path /opt/lvmdir/c2/brick 12: option trash-internal-op off 13: subvolumes c_glusterfs-posix 14: end-volume 15: 16: volume c_glusterfs-changelog 17: type features/changelog 18: option changelog-brick /opt/lvmdir/c2/brick 19: option changelog-dir /opt/lvmdir/c2/brick/.glusterfs/changelogs 20: option changelog-barrier-timeout 120 21: subvolumes c_glusterfs-trash 22: end-volume 23: 24: volume c_glusterfs-bitrot-stub 25: type features/bitrot-stub 26: option export /opt/lvmdir/c2/brick 27: subvolumes c_glusterfs-changelog 28: end-volume 29: 30: volume c_glusterfs-access-control 31: type features/access-control 32: subvolumes c_glusterfs-bitrot-stub 33: end-volume 34: 35: volume c_glusterfs-locks 36: type features/locks 37: subvolumes c_glusterfs-access-control 38: end-volume 39: 40: volume c_glusterfs-upcall 41: type features/upcall 42: option cache-invalidation off 43: subvolumes c_glusterfs-locks 44: end-volume 45: 46: volume c_glusterfs-io-threads 47: type performance/io-threads 48: subvolumes c_glusterfs-upcall 49: end-volume 50: 51: volume c_glusterfs-marker 52: type features/marker 53: option volume-uuid 3c9dd564-a282-436b-8ee5-4537d8e7118a 54: option timestamp-file /system/glusterd/vols/c_glusterfs/marker.tstamp 55: option quota-version 0 56: option xtime off 57: option gsync-force-xtime off 58: option quota off 59: option inode-quota off 60: subvolumes c_glusterfs-io-threads 61: end-volume 62: 63: volume c_glusterfs-barrier 64: type features/barrier 65: option barrier disable 66: option barrier-timeout 120 67: subvolumes c_glusterfs-marker 68: end-volume 69: 70: volume c_glusterfs-index 71: type features/index 72: option index-base /opt/lvmdir/c2/brick/.glusterfs/indices 73: subvolumes c_glusterfs-barrier 74: end-volume 75: 76: volume c_glusterfs-quota 77: type features/quota 78: option volume-uuid c_glusterfs 79: option server-quota off 80: option timeout 0 81: option deem-statfs off 82: subvolumes c_glusterfs-index 83: end-volume 84: 85: volume c_glusterfs-worm 86: type features/worm 87: option worm off 88: subvolumes c_glusterfs-quota 89: end-volume 90: 91: volume c_glusterfs-read-only 92: type features/read-only 93: option read-only off 94: subvolumes c_glusterfs-worm 95: end-volume 96: 97: volume /opt/lvmdir/c2/brick 98: type debug/io-stats 99: option latency-measurement off 100: option count-fop-hits off 101: subvolumes c_glusterfs-read-only 102: end-volume 103: 104: volume c_glusterfs-server 105: type protocol/server 106: option transport.socket.listen-port 49301 107: option rpc-auth.auth-glusterfs on 108: option rpc-auth.auth-unix on 109: option rpc-auth.auth-null on 110: option rpc-auth-allow-insecure on 111: option transport-type tcp 112: option auth.login./opt/lvmdir/c2/brick.allow 229f077d-bff9-4066-bc5e-88caa83bdd14 113: option auth.login.229f077d-bff9-4066-bc5e-88caa83bdd14.password 7b3070d5-f16b-4a79-8250-31394dfb04d6 114: option auth.addr./opt/lvmdir/c2/brick.allow * 115: option transport.tcp-user-timeout 4 116: subvolumes /opt/lvmdir/c2/brick 117: end-volume 118: +------------------------------------------------------------------------------+ [2016-03-31 11:48:38.744361] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:48:38.744480] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-2682-2016/03/31-11:40:19:464042-c_glusterfs-client-9-0-1 (version: 3.7.6) [2016-03-31 11:48:39.225525] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:48:39.225582] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-3199-2016/03/31-11:40:26:159100-c_glusterfs-client-9-0-1 (version: 3.7.6) [2016-03-31 11:48:39.685701] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:48:39.685763] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-12435-2016/03/31-11:48:39:642114-c_glusterfs-client-9-0-0 (version: 3.7.6) [2016-03-31 11:48:39.783125] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:48:39.783203] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-3108-2016/03/31-11:48:39:727985-c_glusterfs-client-9-0-0 (version: 3.7.6) [2016-03-31 11:48:40.195804] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:48:40.195846] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-2686-2016/03/31-11:48:33:196623-c_glusterfs-client-9-0-0 (version: 3.7.6) [2016-03-31 11:48:41.744879] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-12435-2016/03/31-11:48:39:642114-c_glusterfs-client-9-0-0 [2016-03-31 11:48:41.745155] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-12435-2016/03/31-11:48:39:642114-c_glusterfs-client-9-0-0 [2016-03-31 11:51:46.617327] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:51:46.617379] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-26641-2016/03/31-11:51:46:575358-c_glusterfs-client-9-0-0 (version: 3.7.6) [2016-03-31 11:51:48.672337] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-26641-2016/03/31-11:51:46:575358-c_glusterfs-client-9-0-0 [2016-03-31 11:51:48.672420] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-26641-2016/03/31-11:51:46:575358-c_glusterfs-client-9-0-0 [2016-03-31 11:51:48.943715] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:51:48.943752] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-5674-2016/03/31-11:51:48:900721-c_glusterfs-client-9-0-0 (version: 3.7.6) [2016-03-31 11:51:51.003155] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-5674-2016/03/31-11:51:48:900721-c_glusterfs-client-9-0-0 [2016-03-31 11:51:51.003234] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-5674-2016/03/31-11:51:48:900721-c_glusterfs-client-9-0-0 [2016-03-31 11:51:55.730594] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:51:55.730647] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-26939-2016/03/31-11:51:55:689205-c_glusterfs-client-9-0-0 (version: 3.7.6) [2016-03-31 11:51:57.785776] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-26939-2016/03/31-11:51:55:689205-c_glusterfs-client-9-0-0 [2016-03-31 11:51:57.785857] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-26939-2016/03/31-11:51:55:689205-c_glusterfs-client-9-0-0 [2016-03-31 11:51:58.054737] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:51:58.054772] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-5820-2016/03/31-11:51:58:13754-c_glusterfs-client-9-0-0 (version: 3.7.6) [2016-03-31 11:52:00.111881] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-5820-2016/03/31-11:51:58:13754-c_glusterfs-client-9-0-0 [2016-03-31 11:52:00.111977] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-5820-2016/03/31-11:51:58:13754-c_glusterfs-client-9-0-0 [2016-03-31 11:56:22.730982] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-3108-2016/03/31-11:48:39:727985-c_glusterfs-client-9-0-0 [2016-03-31 11:56:22.731075] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-3108-2016/03/31-11:48:39:727985-c_glusterfs-client-9-0-0 [2016-03-31 11:56:23.374784] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-3199-2016/03/31-11:40:26:159100-c_glusterfs-client-9-0-1 [2016-03-31 11:56:23.374875] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /logfiles/systemlog/syslog [2016-03-31 11:56:23.374953] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-3199-2016/03/31-11:40:26:159100-c_glusterfs-client-9-0-1 [2016-03-31 11:56:23.614113] W [socket.c:588:__socket_rwv] 0-glusterfs: readv on 10.32.1.144:24007 failed (No data available) [2016-03-31 11:56:31.222181] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-2682-2016/03/31-11:40:19:464042-c_glusterfs-client-9-0-1 [2016-03-31 11:56:31.222273] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-2682-2016/03/31-11:40:19:464042-c_glusterfs-client-9-0-1 [2016-03-31 11:56:35.254276] E [socket.c:2278:socket_connect_finish] 0-glusterfs: connection to 10.32.1.144:24007 failed (Connection refused) [2016-03-31 11:56:36.876239] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-2686-2016/03/31-11:48:33:196623-c_glusterfs-client-9-0-0 [2016-03-31 11:56:36.876354] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-2686-2016/03/31-11:48:33:196623-c_glusterfs-client-9-0-0 [2016-03-31 11:56:37.883341] I [glusterfsd-mgmt.c:1513:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-03-31 11:56:38.013365] I [glusterfsd-mgmt.c:57:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-03-31 11:56:39.094318] I [glusterfsd-mgmt.c:1513:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-03-31 11:56:39.100393] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:56:39.100450] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-2130-2016/03/31-11:56:34:965176-c_glusterfs-client-9-0-0 (version: 3.7.6) [2016-03-31 11:56:39.115790] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:56:39.115827] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-9531-2016/03/31-11:56:39:31927-c_glusterfs-client-9-0-0 (version: 3.7.6) [2016-03-31 11:56:40.134625] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:56:40.134670] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-2947-2016/03/31-11:56:40:89511-c_glusterfs-client-9-0-0 (version: 3.7.6) [2016-03-31 11:56:48.781641] W [glusterfsd.c:1236:cleanup_and_exit] (-->/lib64/libpthread.so.0() [0x801afbb730] -->/usr/sbin/glusterfsd(glusterfs_sigwaiter-0x1bdfc) [0x10009b04] -->/usr/sbin/glusterfsd(cleanup_and_exit-0x1c02c) [0x100098bc] ) 0-: received signum (15), shutting down [2016-03-31 11:56:52.463817] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version 3.7.6 (args: /usr/sbin/glusterfsd -s 10.32.1.144 --volfile-id c_glusterfs.10.32.1.144.opt-lvmdir-c2-brick -p /system/glusterd/vols/c_glusterfs/run/10.32.1.144-opt-lvmdir-c2-brick.pid -S /var/run/gluster/697c0e4a16ebc734cd06fd9150723005.socket --brick-name /opt/lvmdir/c2/brick -l /var/log/glusterfs/bricks/opt-lvmdir-c2-brick.log --xlator-option *-posix.glusterd-uuid=2d576ff8-0cea-4f75-9e34-a5674fbf7256 --brick-port 49302 --xlator-option c_glusterfs-server.listen-port=49302) [2016-03-31 11:56:52.482047] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2016-03-31 11:56:52.493724] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-server: adding option 'listen-port' for volume 'c_glusterfs-server' with value '49302' [2016-03-31 11:56:52.493774] I [graph.c:269:gf_add_cmdline_options] 0-c_glusterfs-posix: adding option 'glusterd-uuid' for volume 'c_glusterfs-posix' with value '2d576ff8-0cea-4f75-9e34-a5674fbf7256' [2016-03-31 11:56:52.494092] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2016-03-31 11:56:52.494159] I [MSGID: 115034] [server.c:403:_check_for_auth_option] 0-/opt/lvmdir/c2/brick: skip format check for non-addr auth option auth.login./opt/lvmdir/c2/brick.allow [2016-03-31 11:56:52.494190] I [MSGID: 115034] [server.c:403:_check_for_auth_option] 0-/opt/lvmdir/c2/brick: skip format check for non-addr auth option auth.login.229f077d-bff9-4066-bc5e-88caa83bdd14.password [2016-03-31 11:56:52.495590] I [rpcsvc.c:2215:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured rpc.outstanding-rpc-limit with value 64 [2016-03-31 11:56:52.495734] W [MSGID: 101002] [options.c:957:xl_opt_validate] 0-c_glusterfs-server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction [2016-03-31 11:56:52.497898] I [trash.c:2366:init] 0-c_glusterfs-trash: no option specified for 'eliminate', using NULL [2016-03-31 11:56:52.498683] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-glusterfs' is not recognized [2016-03-31 11:56:52.498715] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-unix' is not recognized [2016-03-31 11:56:52.498743] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-server: option 'rpc-auth.auth-null' is not recognized [2016-03-31 11:56:52.498787] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-quota: option 'timeout' is not recognized [2016-03-31 11:56:52.498813] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-marker: option 'quota-version' is not recognized [2016-03-31 11:56:52.498843] W [graph.c:357:_log_if_unknown_option] 0-c_glusterfs-trash: option 'brick-path' is not recognized [2016-03-31 11:56:52.508134] W [MSGID: 113036] [posix.c:2018:posix_rename] 0-c_glusterfs-posix: found directory at /opt/lvmdir/c2/brick/.trashcan/ while expecting ENOENT [File exists] Final graph: +------------------------------------------------------------------------------+ 1: volume c_glusterfs-posix 2: type storage/posix 3: option glusterd-uuid 2d576ff8-0cea-4f75-9e34-a5674fbf7256 4: option directory /opt/lvmdir/c2/brick 5: option volume-id 3c9dd564-a282-436b-8ee5-4537d8e7118a 6: end-volume 7: 8: volume c_glusterfs-trash 9: type features/trash 10: option trash-dir .trashcan 11: option brick-path /opt/lvmdir/c2/brick 12: option trash-internal-op off 13: subvolumes c_glusterfs-posix 14: end-volume 15: 16: volume c_glusterfs-changelog 17: type features/changelog 18: option changelog-brick /opt/lvmdir/c2/brick 19: option changelog-dir /opt/lvmdir/c2/brick/.glusterfs/changelogs 20: option changelog-barrier-timeout 120 21: subvolumes c_glusterfs-trash 22: end-volume 23: 24: volume c_glusterfs-bitrot-stub 25: type features/bitrot-stub 26: option export /opt/lvmdir/c2/brick 27: subvolumes c_glusterfs-changelog 28: end-volume 29: 30: volume c_glusterfs-access-control 31: type features/access-control 32: subvolumes c_glusterfs-bitrot-stub 33: end-volume 34: 35: volume c_glusterfs-locks 36: type features/locks 37: subvolumes c_glusterfs-access-control 38: end-volume 39: 40: volume c_glusterfs-upcall 41: type features/upcall 42: option cache-invalidation off 43: subvolumes c_glusterfs-locks 44: end-volume 45: 46: volume c_glusterfs-io-threads 47: type performance/io-threads 48: subvolumes c_glusterfs-upcall 49: end-volume 50: 51: volume c_glusterfs-marker 52: type features/marker 53: option volume-uuid 3c9dd564-a282-436b-8ee5-4537d8e7118a 54: option timestamp-file /system/glusterd/vols/c_glusterfs/marker.tstamp 55: option quota-version 0 56: option xtime off 57: option gsync-force-xtime off 58: option quota off 59: option inode-quota off 60: subvolumes c_glusterfs-io-threads 61: end-volume 62: 63: volume c_glusterfs-barrier 64: type features/barrier 65: option barrier disable 66: option barrier-timeout 120 67: subvolumes c_glusterfs-marker 68: end-volume 69: 70: volume c_glusterfs-index 71: type features/index 72: option index-base /opt/lvmdir/c2/brick/.glusterfs/indices 73: subvolumes c_glusterfs-barrier 74: end-volume 75: 76: volume c_glusterfs-quota 77: type features/quota 78: option volume-uuid c_glusterfs 79: option server-quota off 80: option timeout 0 81: option deem-statfs off 82: subvolumes c_glusterfs-index 83: end-volume 84: 85: volume c_glusterfs-worm 86: type features/worm 87: option worm off 88: subvolumes c_glusterfs-quota 89: end-volume 90: 91: volume c_glusterfs-read-only 92: type features/read-only 93: option read-only off 94: subvolumes c_glusterfs-worm 95: end-volume 96: 97: volume /opt/lvmdir/c2/brick 98: type debug/io-stats 99: option latency-measurement off 100: option count-fop-hits off 101: subvolumes c_glusterfs-read-only 102: end-volume 103: 104: volume c_glusterfs-server 105: type protocol/server 106: option transport.socket.listen-port 49302 107: option rpc-auth.auth-glusterfs on 108: option rpc-auth.auth-unix on 109: option rpc-auth.auth-null on 110: option rpc-auth-allow-insecure on 111: option transport-type tcp 112: option auth.login./opt/lvmdir/c2/brick.allow 229f077d-bff9-4066-bc5e-88caa83bdd14 113: option auth.login.229f077d-bff9-4066-bc5e-88caa83bdd14.password 7b3070d5-f16b-4a79-8250-31394dfb04d6 114: option auth.addr./opt/lvmdir/c2/brick.allow * 115: option transport.tcp-user-timeout 4 116: subvolumes /opt/lvmdir/c2/brick 117: end-volume 118: +------------------------------------------------------------------------------+ [2016-03-31 11:56:52.568582] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:56:52.568642] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-10939-2016/03/31-11:56:52:528771-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 11:56:52.633807] I [glusterfsd-mgmt.c:57:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-03-31 11:56:52.836860] I [glusterfsd-mgmt.c:1513:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-03-31 11:56:52.837736] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:56:52.837771] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-2130-2016/03/31-11:56:34:965176-c_glusterfs-client-11-4-0 (version: 3.7.6) [2016-03-31 11:56:53.008227] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:56:53.008292] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-5107-2016/03/31-11:56:52:961109-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 11:56:53.048523] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:56:53.048560] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-10999-2016/03/31-11:56:53:596-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 11:56:55.067463] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-5107-2016/03/31-11:56:52:961109-c_glusterfs-client-11-0-0 [2016-03-31 11:56:55.067541] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-5107-2016/03/31-11:56:52:961109-c_glusterfs-client-11-0-0 [2016-03-31 11:56:56.456303] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 11:56:56.456349] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-5061-2016/03/31-11:56:52:439076-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:00:56.237487] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:00:56.237559] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-21747-2016/03/31-12:00:56:192500-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:00:58.301163] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-21747-2016/03/31-12:00:56:192500-c_glusterfs-client-11-0-0 [2016-03-31 12:00:58.301232] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-21747-2016/03/31-12:00:56:192500-c_glusterfs-client-11-0-0 [2016-03-31 12:00:58.583312] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:00:58.583348] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-13576-2016/03/31-12:00:58:541364-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:01:00.639587] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-13576-2016/03/31-12:00:58:541364-c_glusterfs-client-11-0-0 [2016-03-31 12:01:00.639681] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-13576-2016/03/31-12:00:58:541364-c_glusterfs-client-11-0-0 [2016-03-31 12:01:11.498837] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:01:11.498890] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-22190-2016/03/31-12:01:11:454991-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:01:13.554634] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-22190-2016/03/31-12:01:11:454991-c_glusterfs-client-11-0-0 [2016-03-31 12:01:13.554714] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-22190-2016/03/31-12:01:11:454991-c_glusterfs-client-11-0-0 [2016-03-31 12:01:13.832416] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:01:13.832452] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-15098-2016/03/31-12:01:13:789847-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:01:15.887380] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-15098-2016/03/31-12:01:13:789847-c_glusterfs-client-11-0-0 [2016-03-31 12:01:15.887469] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-15098-2016/03/31-12:01:13:789847-c_glusterfs-client-11-0-0 [2016-03-31 12:07:01.242950] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:07:01.243002] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-30681-2016/03/31-12:07:01:198585-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:07:03.300777] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-30681-2016/03/31-12:07:01:198585-c_glusterfs-client-11-0-0 [2016-03-31 12:07:03.300858] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-30681-2016/03/31-12:07:01:198585-c_glusterfs-client-11-0-0 [2016-03-31 12:07:03.573613] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:07:03.573661] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-19201-2016/03/31-12:07:03:532682-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:07:05.630077] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-19201-2016/03/31-12:07:03:532682-c_glusterfs-client-11-0-0 [2016-03-31 12:07:05.630180] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-19201-2016/03/31-12:07:03:532682-c_glusterfs-client-11-0-0 [2016-03-31 12:26:15.314793] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:26:15.314846] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-19954-2016/03/31-12:26:15:271839-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:26:17.370904] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-19954-2016/03/31-12:26:15:271839-c_glusterfs-client-11-0-0 [2016-03-31 12:26:17.370985] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-19954-2016/03/31-12:26:15:271839-c_glusterfs-client-11-0-0 [2016-03-31 12:26:17.655469] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:26:17.655504] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-30971-2016/03/31-12:26:17:612358-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:26:19.710295] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-30971-2016/03/31-12:26:17:612358-c_glusterfs-client-11-0-0 [2016-03-31 12:26:19.710376] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-30971-2016/03/31-12:26:17:612358-c_glusterfs-client-11-0-0 [2016-03-31 12:34:26.685083] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-2130-2016/03/31-11:56:34:965176-c_glusterfs-client-11-4-0 [2016-03-31 12:34:26.685205] W [inodelk.c:404:pl_inodelk_log_cleanup] 0-c_glusterfs-server: releasing lock on e0af6add-9fb1-4995-a76c-902df63f8c01 held by {client=0x3fff8c003740, pid=0 lk-owner=304b2478ff3f0000} [2016-03-31 12:34:26.685254] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /logfiles/systemlog/syslog [2016-03-31 12:34:26.685504] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-2130-2016/03/31-11:56:34:965176-c_glusterfs-client-11-4-0 [2016-03-31 12:34:38.581403] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-5061-2016/03/31-11:56:52:439076-c_glusterfs-client-11-0-0 [2016-03-31 12:34:38.581512] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-5061-2016/03/31-11:56:52:439076-c_glusterfs-client-11-0-0 [2016-03-31 12:34:39.674288] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:34:39.674342] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-30516-2016/03/31-12:34:39:583307-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:34:39.842249] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:34:39.842288] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-4312-2016/03/31-12:34:39:795671-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:34:39.929506] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:34:39.929557] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-30643-2016/03/31-12:34:39:869795-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:34:41.907025] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-4312-2016/03/31-12:34:39:795671-c_glusterfs-client-11-0-0 [2016-03-31 12:34:41.907131] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-4312-2016/03/31-12:34:39:795671-c_glusterfs-client-11-0-0 [2016-03-31 12:38:43.320427] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:38:43.320479] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-2887-2016/03/31-12:38:43:277806-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:38:45.376738] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-2887-2016/03/31-12:38:43:277806-c_glusterfs-client-11-0-0 [2016-03-31 12:38:45.376812] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-2887-2016/03/31-12:38:43:277806-c_glusterfs-client-11-0-0 [2016-03-31 12:38:45.655747] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:38:45.655784] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-22338-2016/03/31-12:38:45:613364-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:38:47.711087] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-22338-2016/03/31-12:38:45:613364-c_glusterfs-client-11-0-0 [2016-03-31 12:38:47.711196] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-22338-2016/03/31-12:38:45:613364-c_glusterfs-client-11-0-0 [2016-03-31 12:38:52.396523] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:38:52.396576] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-3119-2016/03/31-12:38:52:353164-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:38:54.452224] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-3119-2016/03/31-12:38:52:353164-c_glusterfs-client-11-0-0 [2016-03-31 12:38:54.452301] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-3119-2016/03/31-12:38:52:353164-c_glusterfs-client-11-0-0 [2016-03-31 12:38:54.721575] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:38:54.721612] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-22543-2016/03/31-12:38:54:678864-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:38:56.778878] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-22543-2016/03/31-12:38:54:678864-c_glusterfs-client-11-0-0 [2016-03-31 12:38:56.778981] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-22543-2016/03/31-12:38:54:678864-c_glusterfs-client-11-0-0 [2016-03-31 12:44:04.771705] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:44:04.771757] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-7906-2016/03/31-12:44:04:728591-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:44:06.828537] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-7906-2016/03/31-12:44:04:728591-c_glusterfs-client-11-0-0 [2016-03-31 12:44:06.828611] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-7906-2016/03/31-12:44:04:728591-c_glusterfs-client-11-0-0 [2016-03-31 12:44:07.097321] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 12:44:07.097358] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-27870-2016/03/31-12:44:07:55602-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 12:44:09.153875] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-27870-2016/03/31-12:44:07:55602-c_glusterfs-client-11-0-0 [2016-03-31 12:44:09.153965] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-27870-2016/03/31-12:44:07:55602-c_glusterfs-client-11-0-0 [2016-03-31 13:03:16.550402] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:03:16.550452] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-20186-2016/03/31-13:03:16:505808-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:03:18.611645] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-20186-2016/03/31-13:03:16:505808-c_glusterfs-client-11-0-0 [2016-03-31 13:03:18.611720] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-20186-2016/03/31-13:03:16:505808-c_glusterfs-client-11-0-0 [2016-03-31 13:03:18.884504] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:03:18.884541] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-16168-2016/03/31-13:03:18:841079-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:03:20.942016] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-16168-2016/03/31-13:03:18:841079-c_glusterfs-client-11-0-0 [2016-03-31 13:03:20.942108] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-16168-2016/03/31-13:03:18:841079-c_glusterfs-client-11-0-0 [2016-03-31 13:11:21.186448] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-10999-2016/03/31-11:56:53:596-c_glusterfs-client-11-0-0 [2016-03-31 13:11:21.186556] I [MSGID: 115013] [server-helpers.c:294:do_fd_cleanup] 0-c_glusterfs-server: fd cleanup on /logfiles/systemlog/syslog [2016-03-31 13:11:21.186658] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-10999-2016/03/31-11:56:53:596-c_glusterfs-client-11-0-0 [2016-03-31 13:11:22.055025] W [socket.c:588:__socket_rwv] 0-glusterfs: readv on 10.32.1.144:24007 failed (No data available) [2016-03-31 13:11:27.408512] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-10939-2016/03/31-11:56:52:528771-c_glusterfs-client-11-0-0 [2016-03-31 13:11:27.408603] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-10939-2016/03/31-11:56:52:528771-c_glusterfs-client-11-0-0 [2016-03-31 13:11:32.798355] I [glusterfsd-mgmt.c:1513:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-03-31 13:11:33.508979] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:11:33.509031] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-26521-2016/03/31-13:11:33:464049-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:11:33.572616] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:11:33.572661] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-25961-2016/03/31-13:11:33:523664-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:11:35.428843] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:11:35.428880] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-24648-2016/03/31-13:11:28:412032-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:11:35.565254] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-26521-2016/03/31-13:11:33:464049-c_glusterfs-client-11-0-0 [2016-03-31 13:11:35.565325] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-26521-2016/03/31-13:11:33:464049-c_glusterfs-client-11-0-0 [2016-03-31 13:15:14.579361] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:15:14.579414] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-10275-2016/03/31-13:15:14:536684-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:15:16.635761] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-10275-2016/03/31-13:15:14:536684-c_glusterfs-client-11-0-0 [2016-03-31 13:15:16.635830] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-10275-2016/03/31-13:15:14:536684-c_glusterfs-client-11-0-0 [2016-03-31 13:15:16.917235] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:15:16.917274] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-28741-2016/03/31-13:15:16:874880-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:15:18.976539] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-28741-2016/03/31-13:15:16:874880-c_glusterfs-client-11-0-0 [2016-03-31 13:15:18.976614] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-28741-2016/03/31-13:15:16:874880-c_glusterfs-client-11-0-0 [2016-03-31 13:15:23.663614] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:15:23.663666] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-10569-2016/03/31-13:15:23:620204-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:15:25.719705] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-10569-2016/03/31-13:15:23:620204-c_glusterfs-client-11-0-0 [2016-03-31 13:15:25.719781] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-10569-2016/03/31-13:15:23:620204-c_glusterfs-client-11-0-0 [2016-03-31 13:15:25.993165] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:15:25.993201] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-28888-2016/03/31-13:15:25:951072-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:15:28.052192] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-28888-2016/03/31-13:15:25:951072-c_glusterfs-client-11-0-0 [2016-03-31 13:15:28.052281] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-28888-2016/03/31-13:15:25:951072-c_glusterfs-client-11-0-0 [2016-03-31 13:20:54.664925] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:20:54.664979] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-17812-2016/03/31-13:20:54:619389-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:20:56.721041] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-17812-2016/03/31-13:20:54:619389-c_glusterfs-client-11-0-0 [2016-03-31 13:20:56.721137] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-17812-2016/03/31-13:20:54:619389-c_glusterfs-client-11-0-0 [2016-03-31 13:20:57.004201] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:20:57.004242] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-32368-2016/03/31-13:20:56:961847-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:20:59.060855] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-32368-2016/03/31-13:20:56:961847-c_glusterfs-client-11-0-0 [2016-03-31 13:20:59.060956] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-32368-2016/03/31-13:20:56:961847-c_glusterfs-client-11-0-0 [2016-03-31 13:39:56.280384] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:39:56.280437] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 000300-6855-2016/03/31-13:39:56:236365-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:39:58.338982] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 000300-6855-2016/03/31-13:39:56:236365-c_glusterfs-client-11-0-0 [2016-03-31 13:39:58.339060] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 000300-6855-2016/03/31-13:39:56:236365-c_glusterfs-client-11-0-0 [2016-03-31 13:39:58.618897] I [login.c:81:gf_auth] 0-auth/login: allowed user names: 229f077d-bff9-4066-bc5e-88caa83bdd14 [2016-03-31 13:39:58.618933] I [MSGID: 115029] [server-handshake.c:612:server_setvolume] 0-c_glusterfs-server: accepted client from 002500-11651-2016/03/31-13:39:58:577098-c_glusterfs-client-11-0-0 (version: 3.7.6) [2016-03-31 13:40:00.675108] I [MSGID: 115036] [server.c:552:server_rpc_notify] 0-c_glusterfs-server: disconnecting connection from 002500-11651-2016/03/31-13:39:58:577098-c_glusterfs-client-11-0-0 [2016-03-31 13:40:00.675220] I [MSGID: 101055] [client_t.c:419:gf_client_unref] 0-c_glusterfs-server: Shutting down connection 002500-11651-2016/03/31-13:39:58:577098-c_glusterfs-client-11-0-0
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel