Re: Upgrade to 4.1.1 geo-replication does not work

[Marcus Pedersén <marcus.pedersen@xxxxxx>]

Hi again,

I made a mistake when replacing the python files, I missed selinux context, I fixed this but it makes no difference.

All nodes in geo-replication is still in status Stopped, and at a start response is successful but still status Stopped.

I enclose glusterd.log and gsyncd.log and hope that this can give something.

Many thanks for your help!

Best regards

Marcus Pedesén

---

Från: Marcus Pedersén
Skickat: den 13 juli 2018 14:50
Till: Kotresh Hiremath Ravishankar
Kopia: gluster-users@xxxxxxxxxxx
Ämne: Re: Upgrade to 4.1.1 geo-replication does not work

 

Hi Kotresh,

Yes, all nodes have the same version 4.1.1 both master and slave.

All glusterd are crashing on the master side.

Will send logs tonight.      

Thanks,

Marcus 

################
Marcus Pedersén
Systemadministrator
Interbull Centre
################
Sent from my phone
################

Den 13 juli 2018 11:28 skrev Kotresh Hiremath Ravishankar <khiremat@xxxxxxxxxx>:

Hi Marcus,

Is the gluster geo-rep version is same on both master and slave?

Thanks,

Kotresh HR

On Fri, Jul 13, 2018 at 1:26 AM, Marcus Pedersén <marcus.pedersen@xxxxxx> wrote:

Hi Kotresh,

i have replaced both files (gsyncdconfig.py and repce.py) in all nodes both master and slave.

I rebooted all servers but geo-replication status is still Stopped.

I tried to start geo-replication with response Successful but status still show Stopped on all nodes.

Nothing has been written to geo-replication logs since I sent the tail of the log.

So I do not know what info to provide?

Please, help me to find a way to solve this.

Thanks!

Regards

Marcus

---

Från: gluster-users-bounces@gluster.org <gluster-users-bounces@gluster.org> för Marcus Pedersén <marcus.pedersen@xxxxxx>
Skickat: den 12 juli 2018 08:51
Till: Kotresh Hiremath Ravishankar
Kopia: gluster-users@xxxxxxxxxxx
Ämne: Re: Upgrade to 4.1.1 geo-replication does not work

 

Thanks Kotresh,

I installed through the official centos channel, centos-release-gluster41.

Isn't this fix included in centos install?

I will have a look, test it tonight and come back to you!

Thanks a lot!

Regards

Marcus

################
Marcus Pedersén
Systemadministrator
Interbull Centre
################
Sent from my phone
################

Den 12 juli 2018 07:41 skrev Kotresh Hiremath Ravishankar <khiremat@xxxxxxxxxx>:

Hi Marcus,

I think the fix [1] is needed in 4.1

Could you please this out and let us know if that works for you?

[1] https://review.gluster.org/#/c/20207/

Thanks,

Kotresh HR

On Thu, Jul 12, 2018 at 1:49 AM, Marcus Pedersén <marcus.pedersen@xxxxxx> wrote:

Hi all,

I have upgraded from 3.12.9 to 4.1.1 and been following upgrade instructions for offline upgrade.

I upgraded geo-replication side first 1 x (2+1) and the master side after that 2 x (2+1).

Both clusters works the way they should on their own.

After upgrade on master side status for all geo-replication nodes is Stopped.

I tried to start the geo-replication from master node and response back was started successfully.

Status again .... Stopped

Tried to start again and get response started successfully, after that all glusterd crashed on all master nodes.

After a restart of all glusterd the master cluster was up again.

Status for geo-replication is still Stopped and every try to start it after this gives the response successful but still status Stopped.

Please help me get the geo-replication up and running again.

Best regards

Marcus Pedersén

Part of geo-replication log from master node:

[2018-07-11 18:42:48.941760] I [changelogagent(/urd-gds/gluster):73:__init__] ChangelogAgent: Agent listining...
[2018-07-11 18:42:48.947567] I [resource(/urd-gds/gluster):1780:connect_remote] SSH: Initializing SSH connection between master and slave...
[2018-07-11 18:42:49.363514] E [syncdutils(/urd-gds/gluster):304:log_raise_exception] <top>: connection to peer is broken
[2018-07-11 18:42:49.364279] E [resource(/urd-gds/gluster):210:errlog] Popen: command returned error    cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret\
.pem -p 22 -oControlMaster=auto -S /tmp/gsyncd-aux-ssh-hjRhBo/7e5534547f3675a710a107722317484f.sock geouser@urd-gds-geo-000 /nonexistent/gsyncd --session-owner 5e94eb7d-219f-4741-a179-d4ae6b50c7ee --local-id .%\
2Furd-gds%2Fgluster --local-node urd-gds-001 -N --listen --timeout 120 gluster://localhost:urd-gds-volume   error=2
[2018-07-11 18:42:49.364586] E [resource(/urd-gds/gluster):214:logerr] Popen: ssh> usage: gsyncd.py [-h]
[2018-07-11 18:42:49.364799] E [resource(/urd-gds/gluster):214:logerr] Popen: ssh>
[2018-07-11 18:42:49.364989] E [resource(/urd-gds/gluster):214:logerr] Popen: ssh>                  {monitor-status,monitor,worker,agent,slave,status,config-check,config-get,config-set,config-reset,voluuidget,d\
elete}
[2018-07-11 18:42:49.365210] E [resource(/urd-gds/gluster):214:logerr] Popen: ssh>                  ...
[2018-07-11 18:42:49.365408] E [resource(/urd-gds/gluster):214:logerr] Popen: ssh> gsyncd.py: error: argument subcmd: invalid choice: '5e94eb7d-219f-4741-a179-d4ae6b50c7ee' (choose from 'monitor-status', 'monit\
or', 'worker', 'agent', 'slave', 'status', 'config-check', 'config-get', 'config-set', 'config-reset', 'voluuidget', 'delete')
[2018-07-11 18:42:49.365919] I [syncdutils(/urd-gds/gluster):271:finalize] <top>: exiting.
[2018-07-11 18:42:49.369316] I [repce(/urd-gds/gluster):92:service_loop] RepceServer: terminating on reaching EOF.
[2018-07-11 18:42:49.369921] I [syncdutils(/urd-gds/gluster):271:finalize] <top>: exiting.
[2018-07-11 18:42:49.369694] I [monitor(monitor):353:monitor] Monitor: worker died before establishing connection       brick=/urd-gds/gluster
[2018-07-11 18:42:59.492762] I [monitor(monitor):280:monitor] Monitor: starting gsyncd worker   brick=/urd-gds/gluster  slave_node=ssh://geouser@urd-gds-geo-000:gluster://localhost:urd-gds-volume
[2018-07-11 18:42:59.558491] I [resource(/urd-gds/gluster):1780:connect_remote] SSH: Initializing SSH connection between master and slave...
[2018-07-11 18:42:59.559056] I [changelogagent(/urd-gds/gluster):73:__init__] ChangelogAgent: Agent listining...
[2018-07-11 18:42:59.945693] E [syncdutils(/urd-gds/gluster):304:log_raise_exception] <top>: connection to peer is broken
[2018-07-11 18:42:59.946439] E [resource(/urd-gds/gluster):210:errlog] Popen: command returned error    cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret\
.pem -p 22 -oControlMaster=auto -S /tmp/gsyncd-aux-ssh-992bk7/7e5534547f3675a710a107722317484f.sock geouser@urd-gds-geo-000 /nonexistent/gsyncd --session-owner 5e94eb7d-219f-4741-a179-d4ae6b50c7ee --local-id .%\
2Furd-gds%2Fgluster --local-node urd-gds-001 -N --listen --timeout 120 gluster://localhost:urd-gds-volume   error=2
[2018-07-11 18:42:59.946748] E [resource(/urd-gds/gluster):214:logerr] Popen: ssh> usage: gsyncd.py [-h]
[2018-07-11 18:42:59.946962] E [resource(/urd-gds/gluster):214:logerr] Popen: ssh>
[2018-07-11 18:42:59.947150] E [resource(/urd-gds/gluster):214:logerr] Popen: ssh>                  {monitor-status,monitor,worker,agent,slave,status,config-check,config-get,config-set,config-reset,voluuidget,d\
elete}
[2018-07-11 18:42:59.947369] E [resource(/urd-gds/gluster):214:logerr] Popen: ssh>                  ...
[2018-07-11 18:42:59.947552] E [resource(/urd-gds/gluster):214:logerr] Popen: ssh> gsyncd.py: error: argument subcmd: invalid choice: '5e94eb7d-219f-4741-a179-d4ae6b50c7ee' (choose from 'monitor-status', 'monit\
or', 'worker', 'agent', 'slave', 'status', 'config-check', 'config-get', 'config-set', 'config-reset', 'voluuidget', 'delete')
[2018-07-11 18:42:59.948046] I [syncdutils(/urd-gds/gluster):271:finalize] <top>: exiting.
[2018-07-11 18:42:59.951392] I [repce(/urd-gds/gluster):92:service_loop] RepceServer: terminating on reaching EOF.
[2018-07-11 18:42:59.951760] I [syncdutils(/urd-gds/gluster):271:finalize] <top>: exiting.
[2018-07-11 18:42:59.951817] I [monitor(monitor):353:monitor] Monitor: worker died before establishing connection       brick=/urd-gds/gluster
[2018-07-11 18:43:10.54580] I [monitor(monitor):280:monitor] Monitor: starting gsyncd worker    brick=/urd-gds/gluster  slave_node=ssh://geouser@urd-gds-geo-000:gluster://localhost:urd-gds-volume
[2018-07-11 18:43:10.88356] I [monitor(monitor):345:monitor] Monitor: Changelog Agent died, Aborting Worker     brick=/urd-gds/gluster
[2018-07-11 18:43:10.88613] I [monitor(monitor):353:monitor] Monitor: worker died before establishing connection        brick=/urd-gds/gluster
[2018-07-11 18:43:20.112435] I [gsyncdstatus(monitor):242:set_worker_status] GeorepStatus: Worker Status Change status=inconsistent
[2018-07-11 18:43:20.112885] E [syncdutils(monitor):331:log_raise_exception] <top>: FAIL:
Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 361, in twrap
    except:
  File "/usr/libexec/glusterfs/python/syncdaemon/monitor.py", line 428, in wmon
    sys.exit()
TypeError: 'int' object is not iterable
[2018-07-11 18:43:20.114610] I [syncdutils(monitor):271:finalize] <top>: exiting.

---
När du skickar e-post till SLU så innebär detta att SLU behandlar dina personuppgifter. För att läsa mer om hur detta går till, klicka här
E-mailing SLU will result in SLU processing your personal data. For more information on how this is done, click here

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users

--

Thanks and Regards,

Kotresh H R

---
När du skickar e-post till SLU så innebär detta att SLU behandlar dina personuppgifter. För att läsa mer om hur detta går till, klicka här
E-mailing SLU will result in SLU processing your personal data. For more information on how this is done, click here

---
När du skickar e-post till SLU så innebär detta att SLU behandlar dina personuppgifter. För att läsa mer om hur detta går till, klicka här
E-mailing SLU will result in SLU processing your personal data. For more information on how this is done, click here

--

Thanks and Regards,

Kotresh H R

---
När du skickar e-post till SLU så innebär detta att SLU behandlar dina personuppgifter. För att läsa mer om hur detta går till, klicka här
E-mailing SLU will result in SLU processing your personal data. For more information on how this is done, click here

[2018-07-08 01:33:15.306389] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 01:33:28.295399] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 01:43:14.672413] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 01:43:27.684665] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 01:53:15.103908] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 01:53:28.078800] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 02:03:14.484807] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 02:03:27.464752] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 02:13:14.842905] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 02:13:27.860267] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 02:23:15.208786] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 02:23:28.227765] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 02:33:14.584506] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 02:33:27.646060] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 02:43:14.952688] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 02:43:28.086693] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 02:53:15.322772] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 02:53:27.478082] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 03:03:14.682659] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 03:03:27.881446] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 03:13:15.102384] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 03:13:28.262772] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 03:23:14.484050] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 03:23:27.679716] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 03:33:14.849671] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 03:33:28.047201] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 03:43:15.218186] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 03:43:27.443468] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 03:53:14.587688] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 03:53:27.861267] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 04:03:14.978895] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 04:03:28.248243] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 04:13:15.350102] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 04:13:27.635131] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 04:23:14.775625] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 04:23:28.025866] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 04:33:15.136677] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 04:33:27.400340] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 04:43:14.508082] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 04:43:27.774278] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 04:53:14.863395] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 04:53:28.171215] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 05:03:15.230798] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 05:03:27.543763] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 05:13:14.610451] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 05:13:27.951113] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 05:23:14.998841] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 05:23:28.328228] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 05:33:14.369141] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 05:33:27.705018] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 05:43:14.724539] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 05:43:28.105830] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 05:53:15.094325] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 05:53:27.485134] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 06:03:14.466070] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 06:03:27.864164] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 06:13:14.837411] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 06:13:28.245066] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 06:23:15.261967] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 06:23:27.668105] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 06:33:14.649769] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 06:33:28.071174] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 06:43:15.066980] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 06:43:27.455600] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 06:53:14.437869] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 06:53:27.865692] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 07:03:14.809756] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 07:03:28.303463] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 07:13:15.170821] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 07:13:27.704835] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 07:23:14.573772] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 07:23:28.118646] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 07:33:14.944612] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 07:33:27.505251] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 07:43:15.300833] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 07:43:27.867963] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 07:53:14.684496] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 07:53:28.337433] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 08:03:15.055150] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 08:03:27.730495] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 08:13:14.541303] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 08:13:28.101121] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 08:23:14.901733] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 08:23:27.493223] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 08:33:15.257833] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 08:33:27.889339] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 08:43:14.632677] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 08:43:28.255959] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 08:53:15.003251] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 08:53:27.651041] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 09:03:14.375514] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 09:03:28.028094] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 09:13:14.741281] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 09:13:27.403985] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 09:23:15.184259] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 09:23:27.807400] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 09:33:14.626500] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 09:33:28.214812] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 09:43:14.983025] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 09:43:27.590132] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 09:53:15.344538] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 09:53:27.960368] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 10:03:14.779146] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 10:03:28.343828] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 10:13:15.147692] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 10:13:27.785159] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 10:23:14.516441] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 10:23:28.155042] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 10:33:14.876695] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 10:33:27.569801] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 10:43:15.252114] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 10:43:27.958804] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 10:53:14.660762] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 10:53:28.361429] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 11:03:15.018901] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 11:03:27.743707] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 11:13:14.396106] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 11:13:28.212800] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 11:23:14.756727] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 11:23:27.580251] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 11:33:15.137383] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 11:33:27.959662] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 11:43:14.505663] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 11:43:28.323734] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 11:53:14.897626] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 11:53:27.709513] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 12:03:15.255416] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 12:03:28.082048] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 12:13:14.670179] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 12:13:27.517837] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 12:23:15.056203] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 12:23:27.932367] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 12:33:14.431936] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 12:33:28.335761] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 12:43:14.819329] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 12:43:27.711382] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 12:53:15.181343] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 12:53:28.090820] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 13:03:14.576347] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 13:03:27.465779] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 13:13:14.935322] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 13:13:27.825920] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 13:23:15.304604] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 13:23:28.234175] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 13:33:14.664691] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 13:33:27.669452] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 13:43:15.048321] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 13:43:28.056603] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 13:53:14.430215] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 13:53:27.429441] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 14:03:14.876341] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 14:03:27.798578] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 14:13:15.329320] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 14:13:28.271091] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 14:23:14.691926] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 14:23:27.781349] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 14:33:15.054899] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 14:33:28.226340] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 14:43:14.466944] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 14:43:27.613205] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 14:53:14.909377] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 14:53:28.054808] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 15:03:15.298149] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 15:03:27.433903] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 15:13:14.682470] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 15:13:27.805306] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 15:23:15.083062] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 15:23:28.179200] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 15:33:14.495904] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 15:33:27.547535] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 15:43:15.016234] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 15:43:28.096294] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 15:53:14.483188] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 15:53:27.603949] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 16:03:14.908868] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 16:03:28.024928] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 16:13:15.437713] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 16:13:27.534452] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 16:23:14.932713] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 16:23:28.065538] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 16:33:15.469078] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 16:33:27.596365] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 16:43:14.980580] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 16:43:27.999914] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 16:53:15.457274] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 16:53:27.436697] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 17:03:14.984647] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 17:03:27.852694] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 17:13:15.434783] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 17:13:28.340388] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 17:23:14.819963] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 17:23:27.807126] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 17:33:15.311447] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 17:33:28.334754] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 17:43:14.779018] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 17:43:27.849146] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 17:53:15.212298] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 17:53:28.392811] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 18:03:14.726827] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 18:03:27.930889] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 18:13:15.134986] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 18:13:28.435830] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 18:23:14.552345] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 18:23:27.945319] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 18:33:15.034781] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 18:33:28.487141] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 18:43:14.418780] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 18:43:27.989132] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 18:53:14.808503] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 18:53:27.381775] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 19:03:15.189646] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 19:03:27.846814] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 19:13:14.718100] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 19:13:28.366970] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 19:23:15.208136] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 19:23:27.819413] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 19:33:14.727731] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 19:33:28.303976] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 19:43:15.236856] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 19:43:27.736140] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 19:53:14.763691] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 19:53:28.245806] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 20:03:15.296639] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 20:03:27.715314] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 20:13:14.854874] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 20:13:28.151857] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 20:23:15.371059] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 20:23:27.673110] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 20:33:14.799442] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 20:33:28.154575] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 20:43:15.348236] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 20:43:27.715918] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 20:53:14.788751] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 20:53:28.258424] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 21:03:15.238767] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 21:03:27.729758] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 21:13:14.715434] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 21:13:28.240484] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 21:23:15.208819] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 21:23:27.664269] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 21:33:14.597930] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 21:33:28.056200] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 21:43:14.972429] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 21:43:27.519668] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 21:53:15.367992] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 21:53:27.908175] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 22:03:14.756478] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 22:03:28.346475] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 22:13:15.208752] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 22:13:27.871761] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 22:23:14.609239] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 22:23:28.382518] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 22:33:15.053434] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 22:33:27.879162] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 22:43:14.493717] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 22:43:28.393489] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 22:53:15.014055] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 22:53:27.786786] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 23:00:01.704631] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-08 23:03:14.552431] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 23:03:28.172490] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 23:03:55.596153] W [MSGID: 101095] [xlator.c:162:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/3.12.6/xlator/nfs/server.so: cannot open shared object file: No such file or directory
[2018-07-08 23:03:58.076302] E [MSGID: 106116] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Brick ops failed on urd-gds-003. changelog notify failed
[2018-07-08 23:03:58.085386] E [MSGID: 106116] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Brick ops failed on urd-gds-002. changelog notify failed
[2018-07-08 23:03:58.124906] E [MSGID: 106116] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Brick ops failed on urd-gds-004. changelog notify failed
[2018-07-08 23:03:58.304007] E [MSGID: 106116] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Brick ops failed on urd-gds-000. changelog notify failed
[2018-07-08 23:03:58.304171] E [MSGID: 106125] [glusterd-mgmt.c:1394:glusterd_mgmt_v3_brick_op] 0-management: Brick ops failed on peers
[2018-07-08 23:03:58.304246] E [MSGID: 106125] [glusterd-mgmt.c:2464:glusterd_mgmt_v3_initiate_snap_phases] 0-management: Brick Ops Failed
[2018-07-08 23:03:58.304864] W [MSGID: 101095] [xlator.c:162:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/3.12.6/xlator/nfs/server.so: cannot open shared object file: No such file or directory
[2018-07-08 23:03:59.351663] E [MSGID: 106116] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Brick ops failed on urd-gds-002. changelog notify failed
[2018-07-08 23:03:59.359689] E [MSGID: 106116] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Brick ops failed on urd-gds-004. changelog notify failed
[2018-07-08 23:03:59.362615] E [MSGID: 106116] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Brick ops failed on urd-gds-003. changelog notify failed
[2018-07-08 23:03:59.630683] E [MSGID: 106116] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Brick ops failed on urd-gds-000. changelog notify failed
[2018-07-08 23:03:59.630839] E [MSGID: 106125] [glusterd-mgmt.c:1394:glusterd_mgmt_v3_brick_op] 0-management: Brick ops failed on peers
[2018-07-08 23:03:59.630902] E [MSGID: 106125] [glusterd-mgmt.c:2526:glusterd_mgmt_v3_initiate_snap_phases] 0-management: Brick Ops Failed
[2018-07-08 23:08:59.781241] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-08 23:13:15.048445] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 23:13:27.580514] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 23:23:14.537497] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 23:23:27.995648] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 23:33:15.036140] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 23:33:27.504361] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 23:43:14.447529] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 23:43:27.989475] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-08 23:53:14.907903] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-08 23:53:27.521952] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 00:03:15.344723] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 00:03:28.053929] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 00:13:14.789108] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 00:13:27.601557] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 00:23:15.261931] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 00:23:28.117406] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 00:33:14.660228] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 00:33:27.576843] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 00:43:15.072966] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 00:43:27.974613] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 00:53:14.444231] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 00:53:28.436943] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 01:03:14.880501] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 01:03:27.839747] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 01:13:15.410020] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 01:13:28.303810] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 01:23:14.921097] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 01:23:27.700056] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 01:33:15.452773] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 01:33:28.113340] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 01:43:14.969780] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 01:43:27.587718] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 01:53:15.474729] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 01:53:28.068082] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 02:03:14.923216] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 02:03:27.610510] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 02:13:15.370109] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 02:13:28.008172] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 02:23:14.809295] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 02:23:27.459473] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 02:33:15.321897] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 02:33:27.920313] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 02:43:14.762422] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 02:43:28.316926] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 02:53:15.175782] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 02:53:27.716830] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 03:03:14.674427] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 03:03:28.161159] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 03:13:15.107180] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 03:13:27.554802] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 03:23:14.525678] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 03:23:27.965791] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 03:33:14.927641] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 03:33:28.402202] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 03:43:15.315836] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 03:43:27.855775] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 03:53:14.713790] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 03:53:28.292107] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 04:03:15.105342] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 04:03:27.704443] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 04:13:14.508807] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 04:13:28.100175] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 04:23:14.913423] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 04:23:27.560297] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 04:33:15.350226] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 04:33:27.966937] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 04:43:14.732418] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 04:43:28.441834] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 04:53:15.165860] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 04:53:27.885438] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 05:03:14.598837] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 05:03:28.290822] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 05:13:14.989386] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 05:13:27.736757] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 05:23:15.427689] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 05:23:28.150740] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 05:33:14.840890] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 05:33:27.628467] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 05:43:15.260529] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 05:43:28.023142] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 05:53:14.649143] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 05:53:27.485603] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 06:03:15.088002] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 06:03:27.956718] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 06:13:14.469775] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 06:13:28.361245] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 06:23:14.900188] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 06:23:27.789268] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 06:33:15.336790] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 06:33:28.189213] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 06:43:14.811667] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 06:43:27.621878] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 06:53:15.197755] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 06:53:28.082306] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 07:03:14.648190] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 07:03:27.535062] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 07:13:15.090235] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 07:13:27.989577] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 07:23:14.481467] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 07:23:27.384694] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 07:33:14.889499] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 07:33:27.831761] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 07:43:15.266096] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 07:43:28.307725] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 07:53:14.684414] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 07:53:27.751424] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 08:03:15.085504] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 08:03:28.182750] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 08:13:14.495103] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 08:13:27.580654] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 08:23:14.942826] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 08:23:28.037511] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 08:33:15.417924] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 08:33:27.444637] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 08:43:14.819563] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 08:43:27.866803] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 08:53:15.230734] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 08:53:28.268235] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 09:03:14.661203] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 09:03:27.756477] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 09:13:15.043864] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 09:13:28.154052] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 09:23:14.430443] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 09:23:27.591277] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 09:33:14.879249] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 09:33:28.003153] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 09:43:15.290158] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 09:43:27.397263] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 09:53:14.747000] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 09:53:27.802804] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 10:03:15.173185] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 10:03:28.252908] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 10:13:14.613185] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 10:13:27.688562] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 10:23:15.044721] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 10:23:28.129739] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 10:33:14.426695] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 10:33:27.545226] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 10:43:14.815692] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 10:43:28.022398] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 10:53:15.258844] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 10:53:27.416279] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 11:03:14.714865] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 11:03:27.888885] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 11:13:15.176166] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 11:13:28.350735] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 11:23:14.579496] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 11:23:27.837438] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 11:33:15.013377] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 11:33:28.322514] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 11:43:14.431167] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 11:43:27.748722] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 11:53:14.910615] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 11:53:28.181419] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 12:03:15.363013] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 12:03:27.619950] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 12:13:14.796747] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 12:13:28.078455] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 12:23:15.200158] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 12:23:27.484736] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 12:33:14.593458] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 12:33:27.921139] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 12:43:15.020565] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 12:43:28.322028] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 12:53:14.414406] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 12:53:27.781736] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 13:03:14.844265] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 13:03:28.181177] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 13:13:15.233263] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 13:13:27.648216] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 13:23:14.699135] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 13:23:28.114675] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 13:33:15.095729] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 13:33:27.517853] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 13:43:14.510261] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 13:43:27.916111] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 13:53:14.912899] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 13:53:28.317152] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 14:03:15.307100] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 14:03:27.753255] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 14:13:14.715660] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 14:13:28.169499] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 14:23:15.122830] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 14:23:27.580795] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 14:33:14.515877] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 14:33:27.970382] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 14:43:14.902729] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 14:43:28.364271] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 14:53:15.292698] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 14:53:27.829182] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 15:03:14.694469] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 15:03:28.294299] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 15:13:15.095993] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 15:13:27.688954] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 15:23:14.496108] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 15:23:28.172440] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 15:33:14.969021] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 15:33:27.581499] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 15:43:15.409162] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 15:43:28.037217] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 15:53:14.845453] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 15:53:27.437768] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 16:03:15.370700] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 16:03:27.950912] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 16:13:14.892112] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 16:13:28.449619] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 16:23:15.395195] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 16:23:27.974727] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 16:33:14.903118] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 16:33:28.495515] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 16:43:15.418517] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 16:43:27.994884] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 16:53:14.899941] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 16:53:27.547888] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 17:03:15.370033] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 17:03:28.075295] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 17:13:14.864189] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 17:13:27.538639] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 17:23:15.373060] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 17:23:28.016817] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 17:33:14.857261] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 17:33:27.535337] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 17:43:15.310934] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 17:43:28.077723] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 17:53:14.805263] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 17:53:27.612177] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 18:03:15.273160] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 18:03:28.088198] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 18:13:14.737298] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 18:13:27.583166] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 18:18:56.700197] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 18:20:35.707135] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-09 18:20:35.708685] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-09 18:22:18.951033] W [MSGID: 106029] [glusterd-geo-rep.c:2558:glusterd_get_statefile_name] 0-management: Config file (/var/lib/glusterd/geo-replication/urd-gds-volume_geouser_urd-gds-geo-001/gsyncd.conf) missing. Looking for template config file (/var/lib/glusterd/geo-replication/gsyncd_template.conf) [No such file or directory]
[2018-07-09 18:22:18.951227] I [MSGID: 106294] [glusterd-geo-rep.c:2567:glusterd_get_statefile_name] 0-management: Using default config template(/var/lib/glusterd/geo-replication/gsyncd_template.conf).
[2018-07-09 18:22:19.014498] E [MSGID: 106293] [glusterd-geo-rep.c:676:glusterd_query_extutil_generic] 0-management: reading data from child failed
[2018-07-09 18:22:19.014545] E [MSGID: 106305] [glusterd-geo-rep.c:2592:glusterd_get_statefile_name] 0-management: Unable to get configuration data for urd-gds-volume(master), geouser::urd-gds-geo-001::urd-gds-volume(slave) from template config
[2018-07-09 18:22:19.014573] E [MSGID: 106315] [glusterd-geo-rep.c:3523:glusterd_op_stage_gsync_set] 0-management: state_file entry missing in config file (/var/lib/glusterd/geo-replication/urd-gds-volume_geouser_urd-gds-geo-001/gsyncd.conf)
[2018-07-09 18:22:19.014582] E [MSGID: 106301] [glusterd-syncop.c:1353:gd_stage_op_phase] 0-management: Staging of operation 'Volume Geo-replication' failed on localhost    
[2018-07-09 18:22:55.268231] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 18:23:15.246891] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 18:23:28.078801] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 18:33:14.667115] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 18:33:27.567146] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 18:43:15.138118] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 18:43:27.967164] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 18:53:14.546139] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 18:53:28.428738] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:03:14.944897] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 19:03:27.825484] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:04:53.288639] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:04:53.289153] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0x2322a) [0x7fcac878c22a] -->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0x2d198) [0x7fcac8796198] -->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0xe4765) [0x7fcac884d765] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:04:53.289190] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:04:57.671749] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:04:57.821669] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:03.625230] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-09 19:05:03.646051] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:03.646085] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:05:03.661298] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:03.721341] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:05:06.590383] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-09 19:05:06.599112] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:06.599203] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:05:13.154666] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0x2322a) [0x7fcac878c22a] -->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0x2d198) [0x7fcac8796198] -->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0xe4765) [0x7fcac884d765] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:05:13.155248] E [rpc-clnt.c:350:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fcad3cf0edb] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7fcad3ab5e6e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7fcad3ab5f8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x90)[0x7fcad3ab7710] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x2a0)[0x7fcad3ab8200] ))))) 0-management: forced unwinding frame type(Peer mgmt) op(--(4)) called at 2018-07-09 19:05:06.590620 (xid=0xa98c)
[2018-07-09 19:05:13.154740] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:05:13.155290] E [MSGID: 106158] [glusterd-rpc-ops.c:684:__glusterd_friend_update_cbk] 0-management: RPC Error
[2018-07-09 19:05:13.155336] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received RJT from uuid: 00000000-0000-0000-0000-000000000000
[2018-07-09 19:05:14.864317] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:23.636745] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-09 19:05:23.714923] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:05:24.731070] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:24.731238] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:05:25.040782] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:25.116798] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-09 19:05:25.122403] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:25.122416] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:05:25.143885] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:28.803822] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0x2322a) [0x7fcac878c22a] -->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0x2d198) [0x7fcac8796198] -->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0xe4765) [0x7fcac884d765] ) 0-management: Lock for vol urd-gds-volume not held
The message "I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd." repeated 2 times between [2018-07-09 19:04:53.288639] and [2018-07-09 19:05:28.803554]
[2018-07-09 19:05:28.803877] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
The message "I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202" repeated 2 times between [2018-07-09 19:04:57.671749] and [2018-07-09 19:05:30.623331]
[2018-07-09 19:05:30.780616] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:39.649224] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-09 19:05:39.709340] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:05:42.058112] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:42.058276] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:05:42.679641] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-09 19:05:42.686614] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:05:42.686724] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
The message "I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0" repeated 2 times between [2018-07-09 19:05:25.143885] and [2018-07-09 19:05:42.721685]
[2018-07-09 19:07:21.025092] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:07:21.025957] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0x2322a) [0x7fcac878c22a] -->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0x2d198) [0x7fcac8796198] -->/usr/lib64/glusterfs/3.12.6/xlator/mgmt/glusterd.so(+0xe4765) [0x7fcac884d765] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:07:21.025999] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:07:22.879066] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:07:23.020690] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:07:25.449999] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-09 19:07:25.464625] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:07:25.464724] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:07:31.766521] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-09 19:07:31.836533] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:07:32.088068] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:07:32.088114] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:07:32.104365] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:13:07.214015] W [glusterfsd.c:1375:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7fcad2b4ee25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x562be8762455] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x562be876227b] ) 0-: received signum (15), shutting down
[2018-07-09 19:13:09.223522] I [MSGID: 100030] [glusterfsd.c:2511:main] 0-glusterd: Started running glusterd version 3.12.9 (args: glusterd --xlator-option *.upgrade=on -N)
[2018-07-09 19:13:09.401998] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-09 19:13:09.402070] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-09 19:13:09.402093] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-09 19:13:09.415776] E [rpc-transport.c:283:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/3.12.9/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-09 19:13:09.415818] W [rpc-transport.c:287:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-09 19:13:09.415843] W [rpcsvc.c:1682:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-09 19:13:09.415863] E [MSGID: 106243] [glusterd.c:1769:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-09 19:13:09.417890] I [MSGID: 106513] [glusterd-store.c:2241:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-09 19:13:09.579711] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-09 19:13:09.969465] I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-09 19:13:09.969465] and [2018-07-09 19:13:09.969919]
[2018-07-09 19:13:09.969995] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:09.970083] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:09.970263] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:09.982740] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:09.982755] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:09.982821] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:09.986448] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:09.986462] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:09.986500] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:09.990012] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:09.990026] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:09.990063] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:10.773425] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1) already mounted.
[2018-07-09 19:13:10.773789] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1) already mounted.
[2018-07-09 19:13:10.774092] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1) already mounted.
[2018-07-09 19:13:10.774372] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1) already mounted.
[2018-07-09 19:13:10.774661] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1) already mounted.
[2018-07-09 19:13:10.774946] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1) already mounted.
[2018-07-09 19:13:10.775258] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1) already mounted.
[2018-07-09 19:13:10.775577] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1) already mounted.
[2018-07-09 19:13:10.775898] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1) already mounted.
[2018-07-09 19:13:10.776212] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1) already mounted.
[2018-07-09 19:13:10.776586] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/e192789722a94684b2ea4fc5ebc8d6b0/brick1) already mounted.
[2018-07-09 19:13:10.776952] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/90b16c83dccf4d53b8738a8637a949f1/brick1) already mounted.
[2018-07-09 19:13:10.777302] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1) already mounted.
[2018-07-09 19:13:10.777665] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1) already mounted.
[2018-07-09 19:13:10.778025] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1) already mounted.
[2018-07-09 19:13:10.778385] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1) already mounted.
[2018-07-09 19:13:10.778758] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1) already mounted.
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option upgrade on
  9:     option event-threads 1
 10:     option ping-timeout 0
 11:     option transport.socket.read-fail-log off
 12:     option transport.socket.keepalive-interval 2
 13:     option transport.socket.keepalive-time 10
 14:     option transport-type rdma
 15:     option working-directory /var/lib/glusterd
 16: end-volume
 17:  
+------------------------------------------------------------------------------+
[2018-07-09 19:13:11.029771] W [glusterfsd.c:1375:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7f4ef9722e25] -->glusterd(glusterfs_sigwaiter+0xe5) [0x56233966d505] -->glusterd(cleanup_and_exit+0x6b) [0x56233966d37b] ) 0-: received signum (15), shutting down
[2018-07-09 19:13:11.029994] I [MSGID: 101190] [event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-09 19:13:11.063365] I [MSGID: 100030] [glusterfsd.c:2511:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.12.9 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-09 19:13:11.076901] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-09 19:13:11.076996] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-09 19:13:11.077030] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-09 19:13:11.088945] E [rpc-transport.c:283:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/3.12.9/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-09 19:13:11.088988] W [rpc-transport.c:287:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-09 19:13:11.089011] W [rpcsvc.c:1682:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-09 19:13:11.089031] E [MSGID: 106243] [glusterd.c:1769:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-09 19:13:12.410174] I [MSGID: 106513] [glusterd-store.c:2241:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-09 19:13:12.513883] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-09 19:13:12.519931] I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-09 19:13:12.519931] and [2018-07-09 19:13:12.520014]
[2018-07-09 19:13:12.520038] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:12.520056] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:12.520102] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:12.524786] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:12.524800] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:12.524865] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:12.528510] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:12.528524] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:12.528612] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:12.532102] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:12.532120] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:12.532185] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:12.662403] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1) already mounted.
[2018-07-09 19:13:12.662719] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1) already mounted.
[2018-07-09 19:13:12.662987] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1) already mounted.
[2018-07-09 19:13:12.663236] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1) already mounted.
[2018-07-09 19:13:12.663482] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1) already mounted.
[2018-07-09 19:13:12.663742] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1) already mounted.
[2018-07-09 19:13:12.664030] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1) already mounted.
[2018-07-09 19:13:12.664304] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1) already mounted.
[2018-07-09 19:13:12.664603] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1) already mounted.
[2018-07-09 19:13:12.664894] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1) already mounted.
[2018-07-09 19:13:12.665201] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/e192789722a94684b2ea4fc5ebc8d6b0/brick1) already mounted.
[2018-07-09 19:13:12.665507] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/90b16c83dccf4d53b8738a8637a949f1/brick1) already mounted.
[2018-07-09 19:13:12.665831] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1) already mounted.
[2018-07-09 19:13:12.666148] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1) already mounted.
[2018-07-09 19:13:12.666457] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1) already mounted.
[2018-07-09 19:13:12.666788] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1) already mounted.
[2018-07-09 19:13:12.667116] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1) already mounted.
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-09 19:13:12.667760] I [MSGID: 101190] [event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-09 19:13:12.669732] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-09 19:13:15.302256] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 19:13:17.270694] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-09 19:13:17.278873] I [glusterd-utils.c:5953:glusterd_brick_start] 0-management: discovered already-running brick /urd-gds/gluster
[2018-07-09 19:13:17.278890] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-09 19:13:17.278919] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:17.279079] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:13:17.279097] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:17.344566] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:13:17.344782] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-09 19:13:17.344856] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-09 19:13:17.344871] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-09 19:13:17.345030] I [MSGID: 106600] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-09 19:13:17.345051] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-09 19:13:17.345606] I [MSGID: 106568] [glusterd-proc-mgmt.c:87:glusterd_proc_stop] 0-management: Stopping glustershd daemon running in pid: 18404
[2018-07-09 19:13:18.346157] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-09 19:13:18.346203] I [MSGID: 106567] [glusterd-svc-mgmt.c:197:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-09 19:13:19.348272] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-09 19:13:19.348407] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-09 19:13:19.348420] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-09 19:13:19.348439] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-09 19:13:19.348514] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-09 19:13:19.348522] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-09 19:13:19.348548] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-09 19:13:19.348616] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-09 19:13:19.348628] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-09 19:13:19.447312] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:13:19.640489] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-09 19:13:19.645904] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:13:19.645938] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:19.708397] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:13:19.880532] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:13:19.889604] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:13:19.889945] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:13:19.893341] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-09 19:13:19.953052] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:13:19.990709] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-09 19:13:20.001500] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:13:20.035645] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-09 19:13:20.046521] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-09 19:13:20.049739] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:13:20.049765] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:20.063325] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:13:20.063627] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:13:20.063649] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:20.078673] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:13:21.143701] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-09 19:13:21.282268] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:13:21.463318] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:13:21.463391] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:21.480397] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-09 19:13:21.533282] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:13:21.710431] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:13:21.710598] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:13:21.710633] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:21.728600] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:13:21.728682] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:13:21.733247] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:13:21.771024] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:13:21.829616] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-09 19:13:21.838883] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:13:21.881463] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-09 19:13:21.893946] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:13:21.893982] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:21.907562] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:13:21.909690] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:13:21.909714] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:21.925449] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:13:28.212454] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:13:33.843879] W [glusterfsd.c:1375:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7f7f8ce83e25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x557cd392e505] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x557cd392e37b] ) 0-: received signum (15), shutting down
[2018-07-09 19:13:33.879099] I [MSGID: 100030] [glusterfsd.c:2511:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.12.9 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-09 19:13:33.882328] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-09 19:13:33.882352] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-09 19:13:33.882357] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-09 19:13:33.884990] E [rpc-transport.c:283:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/3.12.9/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-09 19:13:33.884999] W [rpc-transport.c:287:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-09 19:13:33.885005] W [rpcsvc.c:1682:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-09 19:13:33.885009] E [MSGID: 106243] [glusterd.c:1769:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-09 19:13:35.142311] I [MSGID: 106513] [glusterd-store.c:2241:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-09 19:13:35.249661] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-09 19:13:35.259469] I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-09 19:13:35.259469] and [2018-07-09 19:13:35.259577]
[2018-07-09 19:13:35.259605] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:35.259624] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:35.259671] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:35.264081] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:35.264096] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:35.264136] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:35.267670] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:35.267684] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:35.267748] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:35.271318] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:35.271332] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:35.271397] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:35.398832] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1) already mounted.
[2018-07-09 19:13:35.399130] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1) already mounted.
[2018-07-09 19:13:35.399387] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1) already mounted.
[2018-07-09 19:13:35.399663] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1) already mounted.
[2018-07-09 19:13:35.399925] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1) already mounted.
[2018-07-09 19:13:35.400172] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1) already mounted.
[2018-07-09 19:13:35.400448] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1) already mounted.
[2018-07-09 19:13:35.400735] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1) already mounted.
[2018-07-09 19:13:35.401023] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1) already mounted.
[2018-07-09 19:13:35.401302] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1) already mounted.
[2018-07-09 19:13:35.401628] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/e192789722a94684b2ea4fc5ebc8d6b0/brick1) already mounted.
[2018-07-09 19:13:35.401946] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/90b16c83dccf4d53b8738a8637a949f1/brick1) already mounted.
[2018-07-09 19:13:35.402255] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1) already mounted.
[2018-07-09 19:13:35.402577] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1) already mounted.
[2018-07-09 19:13:35.402900] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1) already mounted.
[2018-07-09 19:13:35.403219] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1) already mounted.
[2018-07-09 19:13:35.403555] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1) already mounted.
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-09 19:13:35.404166] I [MSGID: 101190] [event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-09 19:13:35.406006] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-09 19:13:44.154408] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-09 19:13:44.165143] I [glusterd-utils.c:5953:glusterd_brick_start] 0-management: discovered already-running brick /urd-gds/gluster
[2018-07-09 19:13:44.165175] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-09 19:13:44.165229] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:44.165926] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:13:44.165966] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:44.188780] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-09 19:13:44.197133] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-09 19:13:44.197722] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-09 19:13:44.197822] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-09 19:13:44.198552] I [MSGID: 106600] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-09 19:13:44.198686] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-09 19:13:44.201082] I [MSGID: 106568] [glusterd-proc-mgmt.c:87:glusterd_proc_stop] 0-management: Stopping glustershd daemon running in pid: 6750
[2018-07-09 19:13:45.202088] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-09 19:13:45.202155] I [MSGID: 106567] [glusterd-svc-mgmt.c:197:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-09 19:13:46.204657] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-09 19:13:46.205371] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-09 19:13:46.205428] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-09 19:13:46.205524] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-09 19:13:46.205956] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-09 19:13:46.206001] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-09 19:13:46.206092] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-09 19:13:46.206612] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-09 19:13:46.206693] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-09 19:13:46.314990] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:13:46.501842] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-09 19:13:46.607125] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:13:46.794108] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-09 19:13:46.862182] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-09 19:13:46.928388] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:13:47.155882] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:13:47.334560] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:13:47.334645] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:13:47.334738] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:13:47.334760] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:47.796413] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:13:47.796455] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:13:47.796553] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:13:48.258442] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:13:48.687095] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:13:48.687241] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:48.867986] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:13:48.926193] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-09 19:13:48.938356] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:13:48.977630] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-09 19:13:48.992292] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:13:48.992329] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:49.013819] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:13:49.013843] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:49.027445] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:13:49.035700] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:13:49.037773] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:13:49.037887] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:49.064756] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:13:49.065383] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:13:49.073577] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:13:49.130485] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-09 19:13:49.150675] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:13:49.186408] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-09 19:13:49.195454] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:13:49.195492] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:49.212934] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-09 19:13:49.213130] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:13:49.213149] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:13:49.213165] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:13:49.235640] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:13:51.875114] W [glusterfsd.c:1375:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7fbdec997e25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x5626dbed7505] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x5626dbed737b] ) 0-: received signum (15), shutting down
[2018-07-09 19:13:51.901585] I [MSGID: 100030] [glusterfsd.c:2511:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.12.9 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-09 19:13:51.914305] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-09 19:13:51.914404] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-09 19:13:51.914428] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-09 19:13:51.927016] E [rpc-transport.c:283:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/3.12.9/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-09 19:13:51.927060] W [rpc-transport.c:287:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-09 19:13:51.927086] W [rpcsvc.c:1682:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-09 19:13:51.927107] E [MSGID: 106243] [glusterd.c:1769:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-09 19:13:53.291994] I [MSGID: 106513] [glusterd-store.c:2241:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-09 19:13:53.420993] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-09 19:13:53.430918] I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-09 19:13:53.430918] and [2018-07-09 19:13:53.431022]
[2018-07-09 19:13:53.431049] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:53.431068] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:53.431113] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:53.437309] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:53.437349] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:53.437441] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:53.441197] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:53.441211] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:53.441275] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:53.444795] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:13:53.444809] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:13:53.444873] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:13:55.725870] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1) already mounted.
[2018-07-09 19:13:55.725964] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1) already mounted.
[2018-07-09 19:13:55.726044] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1) already mounted.
[2018-07-09 19:13:55.726121] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1) already mounted.
[2018-07-09 19:13:55.726197] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1) already mounted.
[2018-07-09 19:13:55.726273] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1) already mounted.
[2018-07-09 19:13:55.726362] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1) already mounted.
[2018-07-09 19:13:55.726447] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1) already mounted.
[2018-07-09 19:13:55.726538] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1) already mounted.
[2018-07-09 19:13:55.726632] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1) already mounted.
[2018-07-09 19:13:55.726731] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/e192789722a94684b2ea4fc5ebc8d6b0/brick1) already mounted.
[2018-07-09 19:13:55.726827] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/90b16c83dccf4d53b8738a8637a949f1/brick1) already mounted.
[2018-07-09 19:13:55.726924] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1) already mounted.
[2018-07-09 19:13:55.727020] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1) already mounted.
[2018-07-09 19:13:55.727116] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1) already mounted.
[2018-07-09 19:13:55.727216] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1) already mounted.
[2018-07-09 19:13:55.727315] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1) already mounted.
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-09 19:13:55.727485] I [MSGID: 101190] [event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-09 19:13:55.728077] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-09 19:14:02.164843] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-09 19:14:02.204074] I [glusterd-utils.c:5953:glusterd_brick_start] 0-management: discovered already-running brick /urd-gds/gluster
[2018-07-09 19:14:02.204092] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-09 19:14:02.204124] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:14:02.204437] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-09 19:14:02.204504] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-09 19:14:02.204519] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-09 19:14:02.204648] I [MSGID: 106600] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-09 19:14:02.204674] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-09 19:14:02.205231] I [MSGID: 106568] [glusterd-proc-mgmt.c:87:glusterd_proc_stop] 0-management: Stopping glustershd daemon running in pid: 8292
[2018-07-09 19:14:03.205703] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-09 19:14:03.205989] I [MSGID: 106567] [glusterd-svc-mgmt.c:197:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-09 19:14:04.213869] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-09 19:14:04.214383] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-09 19:14:04.214439] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-09 19:14:04.214576] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-09 19:14:04.214940] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-09 19:14:04.214980] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-09 19:14:04.215066] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-09 19:14:04.215383] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-09 19:14:04.215420] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-09 19:14:04.336020] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:14:04.515520] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-09 19:14:04.515931] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:14:04.515975] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:14:04.532737] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-09 19:14:04.593826] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-09 19:14:04.600668] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-09 19:14:04.607415] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:14:04.698812] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
The message "I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf)." repeated 2 times between [2018-07-09 19:14:04.698812] and [2018-07-09 19:14:05.333644]
[2018-07-09 19:14:05.516726] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:14:05.516808] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:14:05.516838] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:14:05.532520] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:14:05.537403] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:14:05.537432] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:14:05.537531] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:14:05.537559] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:14:05.553550] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:14:05.559555] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:14:05.559606] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:14:05.577360] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:14:05.583383] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:14:05.656686] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-09 19:14:05.670768] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:14:05.677693] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:14:05.738963] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-09 19:14:05.751751] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:14:05.829978] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-09 19:14:05.841667] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:14:05.841688] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:14:05.866275] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:14:05.866327] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:14:05.913649] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-09 19:14:05.922054] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:14:05.922078] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:14:05.937554] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:14:05.939395] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:14:05.939426] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:14:05.956754] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:14:05.956861] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-09 19:14:05.957306] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:14:05.957382] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:14:05.975476] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:15:24.293186] W [socket.c:593:__socket_rwv] 0-management: readv on 192.168.67.183:24007 failed (No data available)
[2018-07-09 19:15:24.293271] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:15:24.294017] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7ff3a58fe22a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7ff3a5908198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7ff3a59bfd05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:15:24.294061] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:15:29.629997] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:15:29.801182] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:15:34.964863] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-09 19:15:35.038560] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:15:36.702904] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:15:36.702957] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:15:36.993036] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:15:38.148673] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-09 19:15:38.161161] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:15:38.161184] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:15:38.519134] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:15:41.298430] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:15:41.298558] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7ff3a58fe22a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7ff3a5908198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7ff3a59bfd05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:15:41.298568] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:15:43.816570] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:15:43.978788] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:15:45.392730] W [glusterfsd.c:1375:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7ff3afccbe25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x5618db55b505] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x5618db55b37b] ) 0-: received signum (15), shutting down
[2018-07-09 19:15:45.413862] I [MSGID: 100030] [glusterfsd.c:2511:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.12.9 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-09 19:15:45.417823] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-09 19:15:45.417850] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-09 19:15:45.417855] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-09 19:15:45.420810] E [rpc-transport.c:283:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/3.12.9/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-09 19:15:45.420822] W [rpc-transport.c:287:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-09 19:15:45.420828] W [rpcsvc.c:1682:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-09 19:15:45.420833] E [MSGID: 106243] [glusterd.c:1769:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-09 19:15:46.773315] I [MSGID: 106513] [glusterd-store.c:2241:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-09 19:15:46.914049] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-09 19:15:46.920401] I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-09 19:15:46.920401] and [2018-07-09 19:15:46.920590]
[2018-07-09 19:15:46.920638] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:15:46.920673] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:15:46.920747] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:15:46.926789] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:15:46.926809] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:15:46.926874] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:15:46.930897] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:15:46.930916] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:15:46.930956] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:15:46.934475] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:15:46.934490] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:15:46.934537] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:15:47.050410] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1) already mounted.
[2018-07-09 19:15:47.050602] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1) already mounted.
[2018-07-09 19:15:47.050767] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1) already mounted.
[2018-07-09 19:15:47.050919] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1) already mounted.
[2018-07-09 19:15:47.051070] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1) already mounted.
[2018-07-09 19:15:47.051220] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1) already mounted.
[2018-07-09 19:15:47.051395] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1) already mounted.
[2018-07-09 19:15:47.051574] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1) already mounted.
[2018-07-09 19:15:47.051752] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1) already mounted.
[2018-07-09 19:15:47.051925] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1) already mounted.
[2018-07-09 19:15:47.052120] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/e192789722a94684b2ea4fc5ebc8d6b0/brick1) already mounted.
[2018-07-09 19:15:47.052313] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/90b16c83dccf4d53b8738a8637a949f1/brick1) already mounted.
[2018-07-09 19:15:47.052505] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1) already mounted.
[2018-07-09 19:15:47.052703] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1) already mounted.
[2018-07-09 19:15:47.052904] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1) already mounted.
[2018-07-09 19:15:47.053104] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1) already mounted.
[2018-07-09 19:15:47.053310] I [MSGID: 106516] [glusterd-store.c:3499:glusterd_mount_brick_paths] 0-management: brick_mount_path (/run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1) already mounted.
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-09 19:15:47.053693] I [MSGID: 101190] [event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-09 19:15:47.284380] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-09 19:15:47.868666] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-09 19:15:47.877788] I [glusterd-utils.c:5953:glusterd_brick_start] 0-management: discovered already-running brick /urd-gds/gluster
[2018-07-09 19:15:47.877870] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-09 19:15:47.878042] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:15:47.953239] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:15:47.953563] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-09 19:15:47.953665] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-09 19:15:47.953686] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-09 19:15:47.953941] I [MSGID: 106600] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-09 19:15:47.953969] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-09 19:15:47.954649] I [MSGID: 106568] [glusterd-proc-mgmt.c:87:glusterd_proc_stop] 0-management: Stopping glustershd daemon running in pid: 8738
[2018-07-09 19:15:48.955223] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-09 19:15:48.955389] I [MSGID: 106567] [glusterd-svc-mgmt.c:197:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-09 19:15:49.959151] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-09 19:15:49.959285] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-09 19:15:49.959299] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-09 19:15:49.959320] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-09 19:15:49.959399] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-09 19:15:49.959408] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-09 19:15:49.959426] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-09 19:15:49.959493] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-09 19:15:49.959505] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-09 19:15:50.119792] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:15:50.318161] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-09 19:15:50.319226] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:15:50.378572] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:15:50.418561] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-09 19:15:50.488170] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:15:50.682818] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:15:50.682868] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:15:50.733835] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-09 19:15:50.744041] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:15:50.744133] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:15:50.763450] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:15:50.763560] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:15:55.478432] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-09 19:15:56.175331] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:15:56.216582] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:15:56.304954] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-09 19:15:56.371428] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:15:56.571631] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:15:56.571658] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:15:56.597701] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:15:56.603227] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:15:56.651790] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:15:56.732031] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-09 19:15:56.797772] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:15:56.992418] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:15:57.039645] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-09 19:15:57.053739] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:15:57.053762] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:15:57.075880] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:15:57.075974] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:15:57.093021] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:15:57.909018] W [socket.c:593:__socket_rwv] 0-management: readv on 192.168.67.183:24007 failed (No data available)
[2018-07-09 19:15:57.909106] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:15:57.909490] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f51c16cb22a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f51c16d5198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f51c178cd05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:15:57.909523] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:15:57.945835] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-09 19:15:57.952645] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:15:58.180632] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:15:58.806717] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-09 19:15:58.811399] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:15:58.825507] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:15:59.694116] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:15:59.835777] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:16:08.472683] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-09 19:16:08.538022] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:16:11.225359] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:16:11.225396] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:16:11.525852] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:16:11.733864] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-09 19:16:11.738076] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:16:11.738101] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:16:11.835047] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:16:37.486834] W [socket.c:593:__socket_rwv] 0-management: readv on 192.168.67.184:24007 failed (No data available)
[2018-07-09 19:16:37.486863] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:16:37.487098] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f51c16cb22a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f51c16d5198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f51c178cd05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:16:37.487105] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:16:41.848642] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:16:42.000450] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:16:48.481061] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-09 19:16:48.550630] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:16:50.848852] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:16:50.848971] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:16:51.209219] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:16:51.386479] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-09 19:16:51.396788] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:16:51.396904] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:16:51.467861] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:16:56.909089] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:16:56.909457] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f51c16cb22a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f51c16d5198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f51c178cd05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:16:56.909491] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:16:58.557288] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:16:58.719501] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:17:03.318416] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-09 19:17:04.443161] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:17:04.443262] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:17:07.542197] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-09 19:17:07.606925] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:17:07.788735] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:17:07.788794] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:17:07.804279] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:17:13.263259] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:17:13.263672] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f51c16cb22a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f51c16d5198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f51c178cd05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:17:13.263730] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:17:15.085565] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:17:15.211673] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:17:23.488512] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-09 19:17:23.551893] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:17:23.751622] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:17:23.751660] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:17:23.767069] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:17:27.323241] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-09 19:17:27.329259] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:17:27.329324] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:17:27.403451] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:17:49.057618] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:17:49.058333] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f51c16cb22a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f51c16d5198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f51c178cd05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:17:49.058368] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:17:50.772839] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:17:50.916179] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:17:59.505736] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-09 19:17:59.570240] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:17:59.754150] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:18:02.535868] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:18:02.535967] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:18:03.154411] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-09 19:18:03.160891] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:18:03.160986] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:18:03.211820] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:19:05.528601] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:19:05.529353] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f51c16cb22a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f51c16d5198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f51c178cd05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:19:07.188859] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:19:07.352956] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:19:16.519670] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-09 19:19:16.590228] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:19:19.482250] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:19:19.482368] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:19:19.524116] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:19:20.063601] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-09 19:19:20.073737] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:19:20.073829] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:19:20.236635] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:19:05.529388] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:23:14.760226] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 19:23:16.500616] W [socket.c:593:__socket_rwv] 0-management: readv on 192.168.67.182:24007 failed (No data available)
[2018-07-09 19:23:16.500704] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:23:16.501364] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f51c16cb22a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f51c16d5198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f51c178cd05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:23:16.501396] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:23:27.637500] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:24:32.034827] E [socket.c:2369:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (No route to host); disconnecting socket
[2018-07-09 19:26:09.144834] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:26:09.292189] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:26:09.867471] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-09 19:26:09.933447] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:26:10.119462] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:26:10.119491] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:26:10.134847] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:26:13.209221] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-09 19:26:13.217475] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:26:13.217584] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:26:13.287368] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:27:00.922030] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 19:27:16.697709] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:27:45.033239] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:27:45.034065] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f51c16cb22a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f51c16d5198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f51c178cd05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:27:45.034136] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:28:05.298776] E [socket.c:2369:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (No route to host); disconnecting socket
[2018-07-09 19:30:41.747674] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:30:41.926847] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:30:42.001178] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-09 19:30:42.069481] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:30:43.659170] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:30:43.659294] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:30:43.679901] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:30:45.453245] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-09 19:30:45.461644] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:30:45.461733] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:30:45.522602] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:30:51.504958] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:30:58.762664] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 19:31:25.180462] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:31:25.181243] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f51c16cb22a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f51c16d5198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f51c178cd05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:31:25.181289] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:31:36.818782] E [socket.c:2369:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (No route to host); disconnecting socket
[2018-07-09 19:33:15.264919] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 19:33:28.169733] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:34:18.781622] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:34:18.937490] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:34:18.991571] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-09 19:34:19.056116] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:34:20.410124] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:34:20.410228] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:34:20.428024] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:34:22.514743] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-09 19:34:22.520730] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:34:22.520824] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:34:22.575512] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:34:31.578769] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 19:34:38.205821] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:35:14.777223] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 19:35:34.534424] W [glusterfsd.c:1375:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7f51cba98e25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x55b7c3f36505] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x55b7c3f3637b] ) 0-: received signum (15), shutting down
[2018-07-09 19:38:49.282140] I [MSGID: 100030] [glusterfsd.c:2511:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.12.9 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-09 19:38:49.294628] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-09 19:38:49.294671] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-09 19:38:49.294678] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-09 19:38:49.299302] E [rpc-transport.c:283:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/3.12.9/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-09 19:38:49.299318] W [rpc-transport.c:287:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-09 19:38:49.299325] W [rpcsvc.c:1682:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-09 19:38:49.299330] E [MSGID: 106243] [glusterd.c:1769:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-09 19:38:50.693680] I [MSGID: 106513] [glusterd-store.c:2241:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-09 19:38:50.816010] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-09 19:38:50.827322] I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3603:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-09 19:38:50.827322] and [2018-07-09 19:38:50.827882]
[2018-07-09 19:38:50.827909] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:38:50.827929] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:38:50.827975] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:38:50.832505] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:38:50.832523] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:38:50.832561] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:38:50.836146] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:38:50.836161] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:38:50.836236] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:38:50.839764] W [MSGID: 106062] [glusterd-handler.c:3400:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-09 19:38:50.839778] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:38:50.839817] W [MSGID: 101002] [options.c:995:xl_opt_validate] 0-management: option 'address-family' is deprecated, preferred is 'transport.address-family', continuing with correction
[2018-07-09 19:38:50.845559] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1/gluster [No such file or directory]
[2018-07-09 19:38:50.852423] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1/gluster [No such file or directory]
[2018-07-09 19:38:50.862224] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1/gluster [No such file or directory]
[2018-07-09 19:38:50.873745] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1/gluster [No such file or directory]
[2018-07-09 19:38:50.888590] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1/gluster [No such file or directory]
[2018-07-09 19:38:50.906039] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1/gluster [No such file or directory]
[2018-07-09 19:38:50.919618] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1/gluster [No such file or directory]
[2018-07-09 19:38:50.935880] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1/gluster [No such file or directory]
[2018-07-09 19:38:50.951918] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1/gluster [No such file or directory]
[2018-07-09 19:38:50.965092] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1/gluster [No such file or directory]
[2018-07-09 19:38:50.982428] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e192789722a94684b2ea4fc5ebc8d6b0/brick1/gluster [No such file or directory]
[2018-07-09 19:38:50.997025] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/90b16c83dccf4d53b8738a8637a949f1/brick1/gluster [No such file or directory]
[2018-07-09 19:38:51.011362] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1/gluster [No such file or directory]
[2018-07-09 19:38:51.027302] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1/gluster [No such file or directory]
[2018-07-09 19:38:51.040047] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1/gluster [No such file or directory]
[2018-07-09 19:38:51.055159] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1/gluster [No such file or directory]
[2018-07-09 19:38:51.070055] W [MSGID: 106425] [glusterd-store.c:2643:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1/gluster [No such file or directory]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-09 19:38:59.180327] I [MSGID: 101190] [event-epoll.c:613:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-09 19:38:59.369194] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:38:59.449818] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-09 19:38:59.459825] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-09 19:38:59.459912] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-09 19:38:59.459928] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-09 19:38:59.459941] I [MSGID: 106600] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-09 19:38:59.459965] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-09 19:38:59.460547] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped
[2018-07-09 19:38:59.460561] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-09 19:38:59.460577] I [MSGID: 106567] [glusterd-svc-mgmt.c:197:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-09 19:39:00.463149] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-09 19:39:00.463354] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-09 19:39:00.463375] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-09 19:39:00.463413] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-09 19:39:00.463561] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-09 19:39:00.463576] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-09 19:39:00.463609] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-09 19:39:00.463744] I [MSGID: 106132] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-09 19:39:00.463759] I [MSGID: 106568] [glusterd-svc-mgmt.c:229:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-09 19:39:00.463817] I [glusterd-utils.c:6047:glusterd_brick_start] 0-management: starting a fresh brick process for brick /urd-gds/gluster
[2018-07-09 19:39:00.465741] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-09 19:39:00.564211] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:39:00.748242] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-09 19:39:00.807639] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:39:00.998603] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:39:00.999371] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:39:00.999439] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:39:01.074474] I [socket.c:2474:socket_event_handler] 0-transport: EPOLLERR - disconnecting now
[2018-07-09 19:39:01.123141] I [MSGID: 106005] [glusterd-handler.c:6071:__glusterd_brick_rpc_notify] 0-management: Brick urd-gds-001:/urd-gds/gluster has disconnected from glusterd.
[2018-07-09 19:39:01.124052] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:39:01.124183] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:39:01.161922] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-09 19:39:01.171942] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:39:01.171969] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:39:01.185571] I [rpc-clnt.c:1044:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-09 19:39:02.189468] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-09 19:39:02.206425] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-09 19:39:02.365267] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-09 19:39:02.372817] I [glusterd-utils.c:5953:glusterd_brick_start] 0-management: discovered already-running brick /urd-gds/gluster
[2018-07-09 19:39:02.372830] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-09 19:39:02.423348] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:39:02.648602] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:39:03.080222] I [MSGID: 106143] [glusterd-pmap.c:295:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-09 19:39:06.331360] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-09 19:39:06.389982] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:39:06.580157] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:39:06.580226] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:39:06.597266] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:39:06.597413] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:39:06.597448] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:39:06.611878] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
The message "I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202" repeated 2 times between [2018-07-09 19:39:06.611878] and [2018-07-09 19:39:06.622088]
[2018-07-09 19:39:06.641862] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:39:06.690162] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-09 19:39:06.701315] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:39:06.737068] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-09 19:39:06.747715] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:39:06.790989] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-09 19:39:06.800326] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:39:06.800367] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:39:06.816730] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-09 19:39:06.816824] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:39:06.816840] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:39:06.831720] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-09 19:39:06.831751] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:39:06.831769] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:39:06.844898] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:40:47.320645] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:40:48.092953] I [MSGID: 106488] [glusterd-handler.c:1548:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-09 19:41:18.900247] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 19:41:18.900276] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 19:41:18.900581] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 19:41:19.019630] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:42:34.716510] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 19:42:34.716513] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 19:42:34.716892] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 19:42:34.834250] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:43:20.186913] W [socket.c:593:__socket_rwv] 0-management: readv on 192.168.67.183:24007 failed (No data available)
[2018-07-09 19:43:20.186943] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 19:43:20.187053] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f35c8dc522a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f35c8dcf198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f35c8e86d05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 19:43:20.187063] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 19:43:22.687701] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 19:43:22.817279] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:43:27.559792] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:43:30.369386] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-09 19:43:30.436142] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:43:30.630387] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:43:30.630438] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:43:30.812647] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:43:30.932015] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-09 19:43:30.939234] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:43:30.939255] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 19:43:31.081693] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 19:43:39.595864] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 19:43:39.600183] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 19:43:39.600320] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 19:43:39.721956] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:45:15.151877] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 19:47:59.291965] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 19:47:59.291980] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 19:47:59.292304] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 19:47:59.406155] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
The message "I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options" repeated 2 times between [2018-07-09 19:47:59.291965] and [2018-07-09 19:48:17.774826]
The message "I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options" repeated 2 times between [2018-07-09 19:47:59.291980] and [2018-07-09 19:48:17.774829]
The message "W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found" repeated 2 times between [2018-07-09 19:47:59.292304] and [2018-07-09 19:48:17.774853]
The message "I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf)." repeated 2 times between [2018-07-09 19:47:59.406155] and [2018-07-09 19:48:17.887716]
[2018-07-09 19:49:28.005385] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:49:40.938616] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 19:49:40.938644] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 19:49:40.938827] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 19:49:41.059571] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:49:35.708719] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:52:02.327279] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:53:26.216058] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:53:28.031481] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:53:31.345739] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 19:53:31.345753] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 19:53:31.345783] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 19:53:31.456137] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:55:14.518148] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 19:55:33.650933] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 19:55:33.650956] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 19:55:33.651384] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 19:55:33.763273] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:55:50.150581] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:58:12.255114] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:58:54.297316] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 19:58:59.251247] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 19:59:03.660749] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 19:59:03.660773] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 19:59:03.660818] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 19:59:03.766479] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 20:03:24.587398] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 20:03:24.587419] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 20:03:24.587518] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 20:03:24.722106] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 20:03:27.454091] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 20:05:14.883925] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 20:07:14.734343] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 20:07:14.734358] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 20:07:14.734674] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 20:07:14.844476] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 20:08:51.752047] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 20:08:51.752067] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 20:08:51.752421] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 20:08:51.863446] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 20:10:57.637525] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-09 20:10:57.637735] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f35c8dc522a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f35c8dcf198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f35c8e86d05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-09 20:10:57.637744] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-09 20:11:39.639278] E [socket.c:2369:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (No route to host); disconnecting socket
[2018-07-09 20:11:48.009832] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 20:11:48.009856] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 20:11:48.010138] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 20:11:48.120263] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 20:11:58.977730] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 20:11:58.977769] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 20:11:58.977726] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 20:11:59.085214] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 20:13:27.837822] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 20:13:51.154587] I [MSGID: 106163] [glusterd-handshake.c:1316:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-09 20:13:51.273585] I [MSGID: 106490] [glusterd-handler.c:2540:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 20:13:51.318990] I [MSGID: 106493] [glusterd-handler.c:3800:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-09 20:13:51.382776] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 20:13:51.594174] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 20:13:51.594212] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 20:13:51.609057] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 20:13:54.800006] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-09 20:13:54.806019] I [MSGID: 106492] [glusterd-handler.c:2718:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 20:13:54.806043] I [MSGID: 106502] [glusterd-handler.c:2763:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-09 20:13:55.055061] I [MSGID: 106493] [glusterd-rpc-ops.c:701:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-09 20:13:55.926471] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 20:13:55.926491] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 20:13:55.926812] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 20:13:56.039792] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 20:15:09.759472] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 20:15:09.759491] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 20:15:09.759804] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 20:15:09.870003] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 20:15:15.250854] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 20:23:03.221767] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 20:23:03.221781] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 20:23:03.222115] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 20:23:03.332016] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 20:23:28.222542] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 20:25:14.616312] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 20:25:27.109047] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 20:25:27.109068] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 20:25:27.109419] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 20:25:27.219015] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 20:33:27.599391] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 20:35:14.986050] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 20:43:27.981362] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 20:45:15.358082] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 20:53:28.368301] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 20:55:14.732660] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 21:03:27.741821] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 21:03:41.453561] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-09 21:03:41.453575] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-09 21:03:41.453618] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-09 21:03:41.563054] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 21:05:15.097300] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 21:13:28.120980] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 21:15:14.472175] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 21:23:27.511349] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 21:25:14.835297] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 21:33:27.898190] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 21:35:15.199806] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 21:43:28.272981] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 21:45:14.568223] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 21:53:27.651385] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 21:55:14.933064] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 22:03:28.041956] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 22:05:15.292334] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 22:13:27.434955] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 22:15:14.662560] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 22:23:27.816517] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 22:25:15.041359] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 22:33:28.201407] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 22:35:14.406662] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 22:43:27.576370] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 22:45:14.771107] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 22:53:27.957368] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 22:55:15.143813] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 23:00:01.450983] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 23:03:28.391988] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 23:04:45.523610] W [MSGID: 101095] [xlator.c:162:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/3.12.9/xlator/nfs/server.so: cannot open shared object file: No such file or directory
[2018-07-09 23:04:49.062462] E [MSGID: 106116] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Brick ops failed on urd-gds-000. changelog notify failed
[2018-07-09 23:04:49.062611] E [MSGID: 106125] [glusterd-mgmt.c:1394:glusterd_mgmt_v3_brick_op] 0-management: Brick ops failed on peers
[2018-07-09 23:04:49.062666] E [MSGID: 106125] [glusterd-mgmt.c:2464:glusterd_mgmt_v3_initiate_snap_phases] 0-management: Brick Ops Failed
[2018-07-09 23:04:50.453760] E [MSGID: 106125] [glusterd-mgmt.c:1394:glusterd_mgmt_v3_brick_op] 0-management: Brick ops failed on peers
[2018-07-09 23:04:50.453776] E [MSGID: 106125] [glusterd-mgmt.c:2526:glusterd_mgmt_v3_initiate_snap_phases] 0-management: Brick Ops Failed
[2018-07-09 23:04:49.064838] W [MSGID: 101095] [xlator.c:162:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/3.12.9/xlator/nfs/server.so: cannot open shared object file: No such file or directory
[2018-07-09 23:04:50.453712] E [MSGID: 106116] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Brick ops failed on urd-gds-000. changelog notify failed
[2018-07-09 23:05:14.565298] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 23:09:50.582842] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-09 23:13:27.782912] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 23:15:14.931791] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 23:23:28.167692] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 23:25:15.297883] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 23:33:27.544445] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 23:35:14.665279] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 23:43:27.922544] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 23:45:15.030057] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-09 23:53:28.317147] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-09 23:55:14.397240] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 00:03:27.700350] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 00:05:14.766088] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 00:13:28.085760] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 00:15:15.148681] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 00:23:27.474982] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 00:25:14.519288] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 00:33:27.854302] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 00:35:14.891501] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 00:43:28.242438] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 00:45:15.261705] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 00:53:27.622091] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 00:55:14.631703] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 01:03:28.025463] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 01:05:15.001616] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 01:13:27.408858] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 01:15:14.400184] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 01:23:27.811299] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 01:25:14.767305] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 01:33:28.188849] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 01:35:15.154388] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 01:43:27.565565] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 01:45:14.561191] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 01:53:27.946068] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 01:55:14.951265] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 02:03:28.324882] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 02:05:15.351133] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 02:13:27.701534] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 02:15:14.724028] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 02:23:28.091926] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 02:25:15.116308] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 02:33:27.472839] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 02:35:14.509467] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 02:43:27.859122] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 02:45:14.903487] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 02:53:28.273314] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 02:55:15.273158] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 03:03:27.668755] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 03:05:14.659158] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 03:13:28.054657] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 03:15:15.049775] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 03:23:27.496441] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 03:25:14.440698] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 03:33:27.893363] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 03:35:14.826536] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 03:43:28.269310] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 03:45:15.193093] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 03:53:27.683550] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 03:55:14.566363] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 04:03:28.084957] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 04:05:14.934707] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 04:13:27.485147] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 04:15:15.350466] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 04:23:27.883924] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 04:25:14.742505] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 04:33:28.276609] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 04:35:15.130936] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 04:43:27.653995] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 04:45:14.502895] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 04:53:28.043018] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 04:55:14.885778] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 05:03:27.420867] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 05:05:15.275828] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 05:13:27.843745] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 05:15:14.667044] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 05:23:28.226831] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 05:25:15.041314] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 05:33:27.606878] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 05:35:14.431959] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 05:43:27.990868] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 05:45:14.806258] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 05:53:27.373419] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 05:55:15.176117] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 06:03:27.749390] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 06:05:14.567279] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 06:13:28.127516] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 06:15:14.954160] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 06:23:27.505149] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 06:25:15.319863] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 06:33:27.880538] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 06:35:14.707960] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 06:43:28.287989] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 06:45:15.096658] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 06:50:50.573468] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-10 06:50:50.573486] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-10 06:50:50.573836] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-10 06:50:50.685349] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 06:53:27.139178] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-10 06:53:27.139206] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-10 06:53:27.139548] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-10 06:53:27.251607] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 06:53:27.673709] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 06:55:14.466706] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 07:03:28.367746] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 07:05:14.854922] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 07:13:27.778527] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 07:15:15.227774] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 07:23:28.193286] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 07:25:14.614559] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 07:33:27.586160] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 07:35:15.005704] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 07:43:27.987436] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 07:45:14.383335] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 07:53:27.390196] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 07:55:14.790370] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 08:03:27.789700] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 08:05:15.201176] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 08:13:28.197375] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 08:15:14.613585] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 08:23:27.610372] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 08:25:15.008779] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 08:33:28.021863] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 08:35:14.423025] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 08:43:27.420351] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 08:45:14.823367] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 08:53:27.796732] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 08:55:15.226250] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 09:03:28.194415] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 09:05:14.599977] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 09:13:27.571137] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 09:15:14.985252] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 09:23:27.975875] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 09:25:15.383252] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 09:33:28.356191] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 09:35:14.758862] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 09:43:27.739342] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 09:45:15.167264] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 09:53:28.162878] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 09:55:14.546250] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 10:03:27.544443] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 10:05:14.952938] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 10:13:27.958726] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 10:15:15.311864] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 10:23:28.355802] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 10:25:14.681215] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 10:33:27.771398] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 10:35:15.072781] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 10:43:28.155403] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 10:45:14.468915] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 10:53:27.532384] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 10:55:14.880631] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 11:03:27.939702] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 11:05:15.294482] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 11:13:28.326844] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 11:15:14.663830] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 11:23:27.727938] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 11:25:15.030693] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 11:33:28.120471] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 11:35:14.411425] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 11:43:27.524955] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 11:45:14.883825] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 11:53:27.904422] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 11:55:15.276411] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 12:03:28.306015] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 12:05:14.662200] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 12:13:27.700704] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 12:15:15.036356] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 12:23:28.078335] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 12:25:14.409337] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 12:33:27.466601] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 12:35:14.786779] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 12:43:27.844370] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 12:45:15.150523] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 12:53:28.226965] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 12:55:14.525589] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 13:03:27.611006] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 13:05:14.896959] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 13:13:27.986947] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 13:15:15.269006] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 13:23:27.361281] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 13:25:14.632638] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 13:33:27.741081] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 13:35:15.001109] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 13:43:28.121900] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 13:45:14.378393] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 13:53:27.590379] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 13:55:14.750354] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 14:03:27.972384] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 14:05:15.118774] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 14:13:28.357945] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 14:15:14.485508] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 14:23:27.732975] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 14:25:14.856380] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 14:33:28.113895] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 14:35:15.230792] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 14:43:27.488504] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 14:45:14.597477] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 14:53:27.867066] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 14:55:14.970074] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 15:03:28.244821] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 15:05:15.337328] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 15:13:27.617853] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 15:15:14.704773] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 15:23:27.992566] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 15:25:15.090159] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 15:33:27.372745] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 15:35:14.460811] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 15:43:27.758791] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 15:45:14.827096] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 15:53:28.136421] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 15:55:15.191412] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 16:03:27.517964] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 16:05:14.564342] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 16:13:27.901291] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 16:15:14.944445] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 16:23:28.283366] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 16:25:15.325670] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 16:33:27.671335] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 16:35:14.691921] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 16:43:28.056826] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 16:45:15.068862] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 16:53:27.440971] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 16:55:14.447306] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 17:03:27.830838] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 17:05:14.825258] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 17:13:28.213740] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 17:15:15.204714] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 17:23:27.606212] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 17:25:14.591673] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 17:33:27.994356] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 17:35:14.963340] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 17:43:27.398055] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 17:45:15.333680] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 17:53:27.779472] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 17:55:14.716158] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 18:03:28.169335] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 18:05:15.094716] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 18:13:27.552731] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 18:15:14.484981] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 18:23:27.964633] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 18:25:14.863390] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 18:33:28.358052] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 18:35:15.248096] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 18:43:27.742090] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 18:45:14.625407] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 18:53:28.130323] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 18:55:14.994881] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 19:03:27.527368] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 19:05:15.367067] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 19:11:52.874792] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-10 19:11:52.874821] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-10 19:11:52.875068] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-10 19:11:52.990326] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 19:12:48.362677] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 19:13:27.916406] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 19:14:45.036946] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 19:15:15.757018] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 19:23:28.316397] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 19:23:29.539214] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 19:25:16.358991] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 19:33:27.868041] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 19:35:15.908843] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 19:43:28.455878] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 19:45:16.392539] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 19:53:28.017450] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 19:55:15.862074] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 20:03:27.563950] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 20:05:16.406815] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 20:12:31.003031] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 20:13:01.105809] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-10 20:13:01.105825] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-10 20:13:01.105856] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-10 20:13:01.219671] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 20:13:28.056399] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 20:14:50.176915] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 20:15:00.992628] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 20:15:05.554844] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-10 20:15:05.554862] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-10 20:15:05.554896] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-10 20:15:05.671745] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 20:15:15.938716] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 20:18:10.790335] I [MSGID: 106062] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-10 20:18:10.790350] I [MSGID: 106062] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-10 20:18:10.790450] W [MSGID: 106062] [glusterd-geo-rep.c:2292:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-10 20:18:10.904215] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 20:23:27.598810] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 20:25:16.443378] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 20:33:28.134797] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 20:35:15.977743] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 20:43:27.690090] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 20:45:16.558449] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 20:53:28.222272] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 20:55:16.085942] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 21:03:27.638423] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 21:05:15.588975] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 21:13:28.105575] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 21:15:16.034962] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 21:23:27.632510] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 21:25:15.605342] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 21:33:28.209865] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 21:35:16.173468] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 21:43:27.724540] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 21:45:15.734038] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 21:53:28.239053] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 21:55:16.221093] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 22:03:27.813980] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 22:05:15.778140] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 22:13:28.334432] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 22:15:16.340275] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 22:23:27.814836] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 22:25:15.900296] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 22:33:28.374693] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 22:35:16.445882] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 22:43:27.947775] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 22:45:15.998448] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 22:53:28.509857] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 22:55:15.488440] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 23:00:01.556173] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 23:03:28.054159] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 23:05:03.104567] W [MSGID: 101095] [xlator.c:162:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/3.12.9/xlator/nfs/server.so: cannot open shared object file: No such file or directory
[2018-07-10 23:05:56.881828] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 23:06:48.429279] I [MSGID: 106091] [glusterd-snapshot.c:6262:glusterd_snapshot_remove_commit] 0-management: Successfully marked snap ggsnap_urd-gds-volume_20180629_010001 for decommission.
[2018-07-10 23:06:52.794078] I [MSGID: 106091] [glusterd-snapshot.c:6262:glusterd_snapshot_remove_commit] 0-management: Successfully marked snap ggsnap_urd-gds-volume_20180630_010001 for decommission.
The message "W [MSGID: 101095] [xlator.c:162:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/3.12.9/xlator/nfs/server.so: cannot open shared object file: No such file or directory" repeated 3 times between [2018-07-10 23:05:03.104567] and [2018-07-10 23:06:46.890770]
[2018-07-10 23:07:15.983426] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 23:12:00.592301] I [MSGID: 106327] [glusterd-geo-rep.c:2552:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-10 23:13:27.545197] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 23:17:16.381713] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 23:23:27.979805] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 23:27:15.923486] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 23:33:28.523514] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 23:37:16.342444] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 23:43:28.028696] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 23:47:15.929994] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-10 23:53:27.538762] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-10 23:57:16.368393] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 00:03:28.121178] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 00:07:15.872722] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 00:13:27.601702] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 00:17:16.294745] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 00:23:28.167927] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 00:27:15.711810] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 00:33:27.722570] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 00:37:16.184851] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 00:43:28.194147] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 00:47:15.665307] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 00:53:27.664575] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 00:57:16.217440] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 01:03:28.281959] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 01:07:15.729889] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 01:13:27.846175] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 01:17:16.286288] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 01:23:28.427313] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 01:27:15.835276] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 01:33:27.989707] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 01:37:16.404733] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 01:43:27.495087] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 01:47:15.998867] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 01:53:27.988279] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 01:57:15.543081] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 02:03:27.555372] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 02:07:16.103075] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 02:13:28.099225] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 02:17:15.662266] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 02:23:27.658204] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 02:27:16.081455] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 02:33:28.221328] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 02:37:15.492054] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 02:43:27.795715] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 02:47:15.942762] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 02:53:28.330000] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 02:57:16.448693] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 03:03:27.840448] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 03:07:16.019524] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 03:13:28.407583] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 03:17:15.554303] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 03:23:27.898586] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 03:27:16.019270] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 03:33:28.453645] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 03:37:15.444898] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 03:43:27.882760] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 03:47:15.932488] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 03:53:28.319815] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 03:57:16.408367] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 04:03:27.752960] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 04:07:15.949438] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 04:13:28.314175] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 04:17:16.382956] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 04:23:27.874505] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 04:27:15.952317] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 04:33:28.433212] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 04:37:16.517705] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 04:43:28.003468] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 04:47:16.076336] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 04:53:27.429848] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 04:57:15.634715] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 05:03:27.943394] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 05:07:16.188911] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 05:13:28.471476] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 05:17:15.746957] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 05:23:28.034444] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 05:27:16.311196] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 05:33:27.591034] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 05:37:15.765290] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 05:43:28.151753] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 05:47:16.210708] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 05:53:27.602687] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 05:57:15.756682] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 06:03:28.129772] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 06:07:16.261359] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 06:13:27.609328] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 06:17:15.820445] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 06:23:28.041276] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 06:27:16.294370] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 06:33:27.517200] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 06:37:15.716070] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 06:43:27.958880] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 06:47:16.127497] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 06:53:28.451574] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 06:57:15.548491] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 07:03:27.960601] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 07:07:15.966911] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 07:13:28.392105] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 07:17:16.463352] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 07:23:27.843481] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 07:27:15.967134] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 07:33:28.327865] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 07:37:16.490726] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 07:43:27.812394] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 07:47:15.904911] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 07:53:28.303120] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 07:57:16.400819] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 08:03:27.723556] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 08:07:15.820771] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 08:13:28.160755] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 08:17:16.244664] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 08:23:27.691626] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 08:27:15.763902] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 08:33:28.126870] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 08:37:16.260360] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 08:43:27.623334] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 08:47:15.752265] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 08:53:28.050488] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 08:57:16.203414] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 09:03:27.535388] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 09:07:15.669371] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 09:13:28.011935] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 09:17:16.151310] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 09:23:27.446548] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 09:27:15.593950] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 09:33:27.939550] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 09:37:16.069713] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 09:43:28.408365] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 09:47:15.529970] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 09:53:27.839548] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 09:57:15.937064] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 10:03:28.266745] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 10:07:16.365071] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 10:13:27.699051] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 10:17:15.793686] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 10:23:28.178956] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 10:27:16.243150] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 10:33:27.710005] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 10:37:15.696536] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 10:43:28.203127] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 10:47:16.171630] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 10:53:27.708766] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 10:57:15.661048] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 11:03:28.132598] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 11:07:16.089855] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 11:13:27.661343] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 11:17:15.644501] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 11:23:28.090955] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 11:27:16.083027] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 11:33:27.555124] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 11:37:15.540640] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 11:43:28.005676] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 11:47:15.958068] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 11:53:27.439569] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 11:57:16.367235] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 12:03:27.975325] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 12:07:15.835541] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 12:13:28.459477] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 12:17:16.341877] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 12:23:27.938516] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 12:27:15.794706] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 12:33:28.390243] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 12:37:16.204499] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 12:43:27.824558] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 12:47:15.663065] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 12:53:28.257288] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 12:57:16.084377] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 13:03:27.718498] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 13:07:15.505905] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 13:13:28.160959] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 13:17:15.929860] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 13:23:27.637560] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 13:27:16.380560] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 13:33:28.141153] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 13:37:15.872032] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 13:43:27.622681] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 13:47:16.276855] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 13:53:28.064607] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 13:57:15.807072] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 14:03:27.565037] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 14:07:16.298140] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 14:13:27.990117] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 14:17:15.712897] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 14:23:27.475344] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 14:27:16.139970] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 14:33:27.925838] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 14:37:15.611092] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 14:43:28.429751] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 14:47:16.075891] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 14:53:27.867095] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 14:57:15.539458] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 15:03:28.298137] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 15:07:15.972267] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 15:13:27.711303] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 15:17:16.436334] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 15:23:28.125015] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 15:27:15.917114] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 15:33:27.642849] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 15:37:16.378015] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 15:43:28.159800] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 15:47:15.875068] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 15:53:27.636564] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 15:57:16.313487] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 16:03:28.223686] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 16:07:15.839805] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 16:13:27.686114] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 16:17:16.381338] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 16:23:28.231310] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 16:27:15.943138] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 16:33:27.721630] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 16:37:16.484323] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 16:43:28.286455] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 16:47:16.020661] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 16:53:27.850871] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 16:57:15.596432] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 17:03:28.401280] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 17:07:16.073348] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 17:13:27.950349] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 17:17:15.563735] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 17:23:28.474716] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 17:27:16.126452] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 17:33:28.053026] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 17:37:15.660261] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 17:43:27.619071] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 17:47:16.204191] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 17:53:28.114512] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 17:57:15.771760] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 18:03:27.649127] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 18:07:16.330406] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 18:13:28.228056] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 18:17:15.823705] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 18:23:27.645834] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 18:27:16.316335] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 18:28:37.446456] I [MSGID: 106327] [glusterd-geo-rep.c:4448:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 18:33:28.218063] I [MSGID: 106499] [glusterd-handler.c:4303:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 18:37:15.861869] I [MSGID: 106487] [glusterd-handler.c:1485:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 18:37:51.191895] W [socket.c:593:__socket_rwv] 0-management: readv on 192.168.67.180:24007 failed (No data available)
[2018-07-11 18:37:51.191999] I [MSGID: 106004] [glusterd-handler.c:6317:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:37:51.192680] W [glusterd-locks.c:843:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2322a) [0x7f35c8dc522a] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0x2d198) [0x7f35c8dcf198] -->/usr/lib64/glusterfs/3.12.9/xlator/mgmt/glusterd.so(+0xe4d05) [0x7f35c8e86d05] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-11 18:37:51.192715] W [MSGID: 106118] [glusterd-handler.c:6342:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-11 18:38:01.378231] E [socket.c:2369:socket_connect_finish] 0-management: connection to 192.168.67.180:24007 failed (Connection refused); disconnecting socket
[2018-07-11 18:39:01.624710] W [glusterfsd.c:1375:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7f35d3192e25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x559fb85e4505] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x559fb85e437b] ) 0-: received signum (15), shutting down
[2018-07-11 18:43:06.166979] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-glusterd: Started running glusterd version 4.1.1 (args: glusterd --xlator-option *.upgrade=on -N)
[2018-07-11 18:43:06.169898] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-11 18:43:06.169916] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-11 18:43:06.169921] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-11 18:43:06.172858] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-11 18:43:06.172869] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-11 18:43:06.172875] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-11 18:43:06.172879] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-11 18:43:06.173595] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-11 18:43:06.326854] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 18:43:06.334934] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-11 18:43:06.334934] and [2018-07-11 18:43:06.335468]
[2018-07-11 18:43:06.335489] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-11 18:43:06.335510] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 18:43:06.340373] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 18:43:06.344775] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 18:43:06.348380] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-11 18:43:06.335489] and [2018-07-11 18:43:06.348375]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option upgrade on
  9:     option event-threads 1
 10:     option ping-timeout 0
 11:     option transport.socket.read-fail-log off
 12:     option transport.socket.keepalive-interval 2
 13:     option transport.socket.keepalive-time 10
 14:     option transport-type rdma
 15:     option working-directory /var/lib/glusterd
 16: end-volume
 17:  
+------------------------------------------------------------------------------+
[2018-07-11 18:43:07.494537] W [glusterfsd.c:1514:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7f27b7ea8e25] -->glusterd(glusterfs_sigwaiter+0xe5) [0x556678df0d65] -->glusterd(cleanup_and_exit+0x6b) [0x556678df0b8b] ) 0-: received signum (15), shutting down
[2018-07-11 18:43:07.494716] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-11 18:43:07.494879] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (Connection refused); disconnecting socket
[2018-07-11 18:43:07.494973] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:43:07.496320] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (Connection refused); disconnecting socket
[2018-07-11 18:43:07.496388] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:43:07.496684] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (Connection refused); disconnecting socket
[2018-07-11 18:43:07.496734] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:43:07.497007] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.180:24007 failed (Connection refused); disconnecting socket
[2018-07-11 18:43:07.497054] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:45:57.139999] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 4.1.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-11 18:45:57.144644] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-11 18:45:57.144679] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-11 18:45:57.144686] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-11 18:45:57.147629] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-11 18:45:57.147640] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-11 18:45:57.147646] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-11 18:45:57.147651] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-11 18:45:57.830791] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-11 18:45:57.949334] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 18:45:57.955235] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-11 18:45:57.955235] and [2018-07-11 18:45:57.955360]
[2018-07-11 18:45:57.955387] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-11 18:45:57.955404] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 18:45:57.960835] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 18:45:57.964387] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 18:45:57.967897] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-11 18:45:57.955387] and [2018-07-11 18:45:57.967892]
[2018-07-11 18:45:58.117378] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-11 18:45:58.117599] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (Connection refused); disconnecting socket
[2018-07-11 18:45:58.117702] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:45:58.118751] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (Connection refused); disconnecting socket
[2018-07-11 18:45:58.118847] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:45:58.119267] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (Connection refused); disconnecting socket
[2018-07-11 18:45:58.119357] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:45:59.829301] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-11 18:45:59.835030] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-11 18:45:59.835388] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-11 18:45:59.835455] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-11 18:45:59.835502] I [MSGID: 106599] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-11 18:45:59.835604] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-11 18:45:59.838627] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped
[2018-07-11 18:45:59.838689] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-11 18:45:59.838815] I [MSGID: 106567] [glusterd-svc-mgmt.c:203:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-11 18:46:00.845002] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-11 18:46:00.845600] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-11 18:46:00.845655] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-11 18:46:00.845747] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-11 18:46:00.846154] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-11 18:46:00.846225] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-11 18:46:00.846310] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-11 18:46:00.846688] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-11 18:46:00.846726] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-11 18:46:00.846913] I [glusterd-utils.c:6080:glusterd_brick_start] 0-management: starting a fresh brick process for brick /urd-gds/gluster
[2018-07-11 18:46:00.852840] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 18:46:00.938112] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 18:46:01.048324] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 18:46:01.137914] I [socket.c:2625:socket_event_handler] 0-transport: EPOLLERR - disconnecting now
[2018-07-11 18:46:01.234842] I [MSGID: 106005] [glusterd-handler.c:6131:__glusterd_brick_rpc_notify] 0-management: Brick urd-gds-001:/urd-gds/gluster has disconnected from glusterd.
[2018-07-11 18:46:01.316802] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-11 18:46:02.319326] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-gfproxyd: setting frame-timeout to 600
[2018-07-11 18:46:02.320736] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-11 18:46:02.321802] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 18:46:02.336675] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-11 18:46:02.428710] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 18:46:02.463640] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 18:46:02.542871] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-11 18:46:02.554873] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 18:46:02.554920] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:46:02.575963] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 18:46:09.360869] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 18:46:09.422045] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 18:46:10.253450] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-11 18:46:10.294050] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 18:46:10.550484] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 18:46:10.777066] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 18:46:10.777185] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:46:12.191156] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-11 18:46:12.196816] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 18:46:12.196851] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:46:12.223265] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 18:46:18.688771] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 18:46:18.883038] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 18:46:19.342622] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-11 18:46:19.382510] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 18:46:19.648912] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 18:46:21.339799] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 18:46:21.339970] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:46:21.765219] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-11 18:46:21.771692] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 18:46:21.771732] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:46:21.790069] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 18:46:28.664688] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 18:46:28.727783] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 18:46:28.798111] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-11 18:46:28.837031] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 18:46:29.083870] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 18:46:30.094625] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 18:46:30.094827] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:46:31.730212] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-11 18:46:31.738292] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 18:46:31.738382] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:46:31.756479] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 18:46:58.269537] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 18:47:16.276301] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 18:49:23.629436] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.180:24007 failed (No data available)
[2018-07-11 18:49:23.629520] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:49:23.630154] W [glusterd-locks.c:845:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f23d50b330a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f23d50bd540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe8543) [0x7f23d5177543] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-11 18:49:23.630246] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-11 18:49:31.443616] W [glusterfsd.c:1514:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7f23df48be25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x5602966e2d65] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x5602966e2b8b] ) 0-: received signum (15), shutting down
[2018-07-11 18:52:14.789369] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 4.1.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-11 18:52:14.800506] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-11 18:52:14.800557] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-11 18:52:14.800570] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-11 18:52:14.805029] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-11 18:52:14.805048] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-11 18:52:14.805082] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-11 18:52:14.805087] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-11 18:52:15.526015] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-11 18:52:15.641354] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 18:52:15.653214] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-11 18:52:15.653214] and [2018-07-11 18:52:15.653636]
[2018-07-11 18:52:15.653666] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-11 18:52:15.653685] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 18:52:15.658195] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 18:52:15.661713] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 18:52:15.665244] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-11 18:52:15.653666] and [2018-07-11 18:52:15.665239]
[2018-07-11 18:52:15.669220] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.678414] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.688631] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.698656] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.708578] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.713015] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.721177] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.732232] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.741602] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.753640] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.764184] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.774258] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.786503] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.795515] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.806836] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1/gluster [No such file or directory]
[2018-07-11 18:52:15.819392] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/3ceb415b1f254aab9c5efd22b2a10ce6/brick1/gluster [No such file or directory]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-11 18:52:15.830432] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-11 18:52:17.248033] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.180:24007 failed (Connection refused); disconnecting socket
[2018-07-11 18:52:17.248152] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:52:18.575866] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 18:52:18.622692] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 18:52:18.663679] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (No route to host); disconnecting socket
[2018-07-11 18:52:18.663763] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:52:18.690373] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-11 18:52:18.700846] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (No route to host); disconnecting socket
[2018-07-11 18:52:18.700863] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:52:18.700921] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (No route to host); disconnecting socket
[2018-07-11 18:52:18.700955] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 18:52:18.701026] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 18:52:18.701040] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:52:21.185695] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-11 18:52:21.192329] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 18:52:21.192423] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:52:21.210598] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-11 18:52:21.210936] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-11 18:52:21.210994] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-11 18:52:21.211040] I [MSGID: 106599] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-11 18:52:21.211141] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-11 18:52:21.214022] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped
[2018-07-11 18:52:21.214081] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-11 18:52:21.214152] I [MSGID: 106567] [glusterd-svc-mgmt.c:203:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-11 18:52:22.217594] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-11 18:52:22.218155] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-11 18:52:22.218221] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-11 18:52:22.218313] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-11 18:52:22.218745] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-11 18:52:22.218793] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-11 18:52:22.218880] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-11 18:52:22.219256] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-11 18:52:22.219292] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-11 18:52:22.219454] I [glusterd-utils.c:6080:glusterd_brick_start] 0-management: starting a fresh brick process for brick /urd-gds/gluster
[2018-07-11 18:52:22.224591] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 18:52:22.339391] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 18:52:22.451123] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 18:52:22.546804] I [socket.c:2625:socket_event_handler] 0-transport: EPOLLERR - disconnecting now
[2018-07-11 18:52:22.609036] I [MSGID: 106005] [glusterd-handler.c:6131:__glusterd_brick_rpc_notify] 0-management: Brick urd-gds-001:/urd-gds/gluster has disconnected from glusterd.
[2018-07-11 18:52:22.609312] E [MSGID: 101012] [common-utils.c:4011:gf_is_service_running] 0-: Unable to read pidfile: /var/run/gluster/vols/urd-gds-volume/urd-gds-001-urd-gds-gluster.pid
[2018-07-11 18:52:22.693004] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-11 18:52:23.695039] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-gfproxyd: setting frame-timeout to 600
[2018-07-11 18:52:23.696280] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-11 18:52:23.699692] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 18:52:23.714649] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-11 18:52:24.632633] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 18:52:24.731635] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 18:52:24.872991] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-11 18:52:24.882465] I [glusterd-utils.c:5985:glusterd_brick_start] 0-management: discovered already-running brick /urd-gds/gluster
[2018-07-11 18:52:24.882485] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-11 18:52:24.912668] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 18:52:25.162867] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 18:52:26.079718] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 18:52:26.079865] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:52:27.464020] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-11 18:52:27.468914] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 18:52:27.469005] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:52:27.488818] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 18:52:30.173034] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 18:52:30.283165] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 18:52:30.743695] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-11 18:52:31.708051] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 18:52:31.708149] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:52:33.135979] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-11 18:52:33.179797] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 18:52:33.292235] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 18:52:33.292336] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:52:33.306574] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 18:52:33.431120] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 18:52:36.198420] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 18:52:36.334253] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 18:52:36.941246] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-11 18:52:36.984546] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 18:52:37.238512] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 18:52:37.837578] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 18:52:37.837762] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:52:39.435294] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-11 18:52:39.443693] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 18:52:39.443785] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 18:52:39.548249] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 18:53:28.059024] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 18:57:15.700783] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:03:27.469157] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:07:16.119161] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:13:27.947691] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:17:15.639018] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:19:14.642907] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:19:36.509212] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:23:28.426519] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:24:01.492372] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:24:01.492385] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:24:01.492718] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:24:01.559375] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:24:28.512521] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:24:28.584946] E [MSGID: 106308] [glusterd-geo-rep.c:2186:glusterd_op_verify_gsync_running] 0-management: geo-replication session b/w urd-gds-volume & geouser@urd-gds-geo-001::urd-gds-volume is not running on this node.
[2018-07-11 19:24:28.617810] E [MSGID: 106301] [glusterd-syncop.c:1352:gd_stage_op_phase] 0-management: Staging of operation 'Volume Geo-replication' failed on localhost : Geo-replication session between urd-gds-volume and geouser@urd-gds-geo-001::urd-gds-volume is not Paused.
[2018-07-11 19:25:06.001856] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:25:06.001872] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:25:06.002201] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:25:06.066561] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:26:32.847217] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:26:33.391242] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 19:26:47.756347] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:26:47.756362] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:26:47.756395] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:26:47.817450] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:27:03.231267] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:27:03.231298] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:27:14.488029] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:27:14.559964] E [MSGID: 106308] [glusterd-geo-rep.c:2186:glusterd_op_verify_gsync_running] 0-management: geo-replication session b/w urd-gds-volume & geouser@urd-gds-geo-001::urd-gds-volume is not running on this node.
[2018-07-11 19:27:14.817726] W [MSGID: 106325] [glusterd-geo-rep.c:5647:glusterd_op_gsync_set] 0-management: geo-replication is not set up for urd-gds-volume(master) and geouser@urd-gds-geo-001::urd-gds-volume(slave)
[2018-07-11 19:27:14.877617] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.180:24007 failed (Connection reset by peer)
[2018-07-11 19:27:14.877646] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:27:14.877738] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f9d83d5f30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f9d83d69540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f9d83e2335c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 19:27:14.877750] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-11 19:27:14.877953] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f9d8f2d52cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7f9d8f09ed6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7f9d8f09ee8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f9d8f0a04dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7f9d8f0a1048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-11 19:27:14.871313 (xid=0x2e)
[2018-07-11 19:27:14.877972] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-000. Please check log file for details.
[2018-07-11 19:27:14.881453] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.182:24007 failed (Connection reset by peer)
[2018-07-11 19:27:14.881487] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:27:14.881588] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f9d83d5f30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f9d83d69540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f9d83e2335c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 19:27:14.881603] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-11 19:27:14.881797] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f9d8f2d52cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7f9d8f09ed6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7f9d8f09ee8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f9d8f0a04dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7f9d8f0a1048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-11 19:27:14.871286 (xid=0x2e)
[2018-07-11 19:27:14.881815] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-002. Please check log file for details.
[2018-07-11 19:27:14.883906] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.183:24007 failed (Connection reset by peer)
[2018-07-11 19:27:14.883940] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:27:14.884047] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f9d83d5f30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f9d83d69540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f9d83e2335c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 19:27:14.884061] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-11 19:27:14.884278] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f9d8f2d52cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7f9d8f09ed6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7f9d8f09ee8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f9d8f0a04dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7f9d8f0a1048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-11 19:27:14.871297 (xid=0x2e)
[2018-07-11 19:27:14.884300] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-003. Please check log file for details.
[2018-07-11 19:27:14.884363] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.184:24007 failed (No data available)
[2018-07-11 19:27:14.884389] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:27:14.884554] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f9d83d5f30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f9d83d69540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f9d83e2335c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 19:27:14.884581] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-11 19:27:14.884818] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f9d8f2d52cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7f9d8f09ed6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7f9d8f09ee8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f9d8f0a04dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7f9d8f0a1048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-11 19:27:14.871305 (xid=0x2e)
[2018-07-11 19:27:14.884842] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-004. Please check log file for details.
[2018-07-11 19:27:14.884875] E [MSGID: 106151] [glusterd-syncop.c:1640:gd_unlock_op_phase] 0-management: Failed to unlock on some peer(s)
[2018-07-11 19:27:14.885064] E [mem-pool.c:326:__gf_free] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1870) [0x7f9d83e1c870] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1000) [0x7f9d83e1c000] -->/lib64/libglusterfs.so.0(__gf_free+0xa4) [0x7f9d8f2fe494] ) 0-: Assertion failed: GF_MEM_HEADER_MAGIC == header->magic
[2018-07-11 19:27:14.885101] E [mem-pool.c:335:__gf_free] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1870) [0x7f9d83e1c870] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1000) [0x7f9d83e1c000] -->/lib64/libglusterfs.so.0(__gf_free+0x104) [0x7f9d8f2fe4f4] ) 0-: Assertion failed: GF_MEM_TRAILER_MAGIC == *(uint32_t *)((char *)free_ptr + header->size)
[2018-07-11 19:27:03.231265] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:27:03.298037] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
pending frames:
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
patchset: git://git.gluster.org/glusterfs.git
signal received: 11
time of crash: 
2018-07-11 19:27:14
configuration details:
argp 1
backtrace 1
dlfcn 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 4.1.1
/lib64/libglusterfs.so.0(+0x25920)[0x7f9d8f2d3920]
/lib64/libglusterfs.so.0(gf_print_trace+0x334)[0x7f9d8f2dd874]
/lib64/libc.so.6(+0x362f0)[0x7f9d8d9382f0]
/lib64/libpthread.so.0(pthread_mutex_lock+0x0)[0x7f9d8e139c80]
/lib64/libglusterfs.so.0(__gf_free+0x12c)[0x7f9d8f2fe51c]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1000)[0x7f9d83e1c000]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1870)[0x7f9d83e1c870]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xae808)[0x7f9d83de9808]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2436e)[0x7f9d83d5f36e]
/lib64/libglusterfs.so.0(+0x622b0)[0x7f9d8f3102b0]
/lib64/libc.so.6(+0x48030)[0x7f9d8d94a030]
---------
[2018-07-11 19:29:14.898773] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 4.1.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-11 19:29:14.902955] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-11 19:29:14.902988] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-11 19:29:14.902996] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-11 19:29:14.906192] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-11 19:29:14.906203] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-11 19:29:14.906209] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-11 19:29:14.906214] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-11 19:29:15.597117] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-11 19:29:15.719719] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 19:29:15.727622] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-11 19:29:15.727622] and [2018-07-11 19:29:15.727721]
[2018-07-11 19:29:15.727749] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-11 19:29:15.727767] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:29:15.732208] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:29:15.735828] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:29:15.739258] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-11 19:29:15.727749] and [2018-07-11 19:29:15.739253]
[2018-07-11 19:29:15.744467] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.750954] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.755637] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.763028] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.768645] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.774300] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.792127] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.798339] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.806199] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.815431] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.826842] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.835197] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.845546] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.855843] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.864353] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1/gluster [No such file or directory]
[2018-07-11 19:29:15.875642] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/3ceb415b1f254aab9c5efd22b2a10ce6/brick1/gluster [No such file or directory]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-11 19:29:15.885619] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-11 19:29:15.886618] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:29:15.886718] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:29:15.887473] I [glusterd-utils.c:5985:glusterd_brick_start] 0-management: discovered already-running brick /urd-gds/gluster
[2018-07-11 19:29:15.887546] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-11 19:29:15.887641] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:29:15.888101] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:29:15.888154] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:29:15.888350] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:29:15.888396] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:29:15.888583] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.180:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:29:15.888634] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:29:15.941348] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:29:16.457328] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-11 19:29:16.804745] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-11 19:29:26.948998] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
The message "I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume" repeated 2 times between [2018-07-11 19:29:26.948998] and [2018-07-11 19:29:47.187645]
[2018-07-11 19:29:54.821292] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:30:04.941697] W [glusterfsd.c:1514:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7f2773d3ee25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x55c1c974dd65] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x55c1c974db8b] ) 0-: received signum (15), shutting down
[2018-07-11 19:30:04.951851] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 4.1.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-11 19:30:04.955569] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-11 19:30:04.955600] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-11 19:30:04.955607] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-11 19:30:04.958610] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-11 19:30:04.958624] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-11 19:30:04.958631] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-11 19:30:04.958636] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-11 19:30:05.628782] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-11 19:30:05.750708] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 19:30:05.756578] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-11 19:30:05.756578] and [2018-07-11 19:30:05.756689]
[2018-07-11 19:30:05.756717] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-11 19:30:05.756736] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:30:05.761470] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:30:05.768419] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:30:05.774018] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-11 19:30:05.756717] and [2018-07-11 19:30:05.774008]
[2018-07-11 19:30:05.779267] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.784287] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.791717] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.797208] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.802899] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.810067] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.814904] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.821380] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.827983] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.832567] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.839957] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.845944] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.853551] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.865037] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.874034] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1/gluster [No such file or directory]
[2018-07-11 19:30:05.884739] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/3ceb415b1f254aab9c5efd22b2a10ce6/brick1/gluster [No such file or directory]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-11 19:30:05.894670] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-11 19:30:05.894775] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:30:05.894876] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:30:05.895314] I [glusterd-utils.c:5985:glusterd_brick_start] 0-management: discovered already-running brick /urd-gds/gluster
[2018-07-11 19:30:05.895364] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-11 19:30:05.895451] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:30:05.895886] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:30:05.895948] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:30:05.896145] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:30:05.896193] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:30:05.896365] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.180:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:30:05.896409] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:30:07.476595] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-11 19:30:09.163143] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:30:14.149224] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:30:15.819380] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-11 19:30:54.771042] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:31:15.012115] W [glusterfsd.c:1514:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7f9770ce2e25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x561e11a72d65] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x561e11a72b8b] ) 0-: received signum (15), shutting down
[2018-07-11 19:31:37.565942] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 4.1.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-11 19:31:37.570178] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-11 19:31:37.570379] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-11 19:31:37.570387] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-11 19:31:37.573127] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-11 19:31:37.573138] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-11 19:31:37.573143] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-11 19:31:37.573148] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-11 19:31:38.243431] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-11 19:31:38.366566] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 19:31:38.371310] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-11 19:31:38.371310] and [2018-07-11 19:31:38.371402]
[2018-07-11 19:31:38.371429] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-11 19:31:38.371447] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:31:38.376601] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:31:38.380253] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:31:38.383747] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-11 19:31:38.371429] and [2018-07-11 19:31:38.383742]
[2018-07-11 19:31:38.387652] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.394642] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.399940] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.405329] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.412546] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.416992] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.423288] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.429584] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.433923] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.440989] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.446451] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.451742] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.458990] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.463820] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.470636] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1/gluster [No such file or directory]
[2018-07-11 19:31:38.477487] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/3ceb415b1f254aab9c5efd22b2a10ce6/brick1/gluster [No such file or directory]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-11 19:31:38.482695] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-11 19:31:38.482784] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:31:38.482828] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:31:38.483071] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:31:38.483096] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:31:38.483462] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:31:38.483502] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:31:38.483639] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.180:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:31:38.483662] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:32:04.406625] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:32:12.779542] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:32:24.996288] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:32:24.996305] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:32:24.996632] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:32:25.058971] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:32:56.603967] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:33:04.007320] W [glusterfsd.c:1514:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7f375afb2e25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x5621b6ff1d65] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x5621b6ff1b8b] ) 0-: received signum (15), shutting down
[2018-07-11 19:35:34.128992] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 4.1.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-11 19:35:34.138635] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-11 19:35:34.138675] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-11 19:35:34.138683] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-11 19:35:34.142590] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-11 19:35:34.142608] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-11 19:35:34.142624] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-11 19:35:34.142631] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-11 19:35:34.855181] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-11 19:35:34.975319] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 19:35:34.980202] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-11 19:35:34.980202] and [2018-07-11 19:35:34.980303]
[2018-07-11 19:35:34.980330] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-11 19:35:34.980353] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:35:34.984840] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:35:34.988258] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:35:34.991699] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-11 19:35:34.980330] and [2018-07-11 19:35:34.991694]
[2018-07-11 19:35:34.995648] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.003066] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.008551] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.013901] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.021282] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.026028] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.032960] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.039937] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.045781] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.054534] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.064328] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.074868] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.087402] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.097001] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.107981] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1/gluster [No such file or directory]
[2018-07-11 19:35:35.118889] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/3ceb415b1f254aab9c5efd22b2a10ce6/brick1/gluster [No such file or directory]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-11 19:35:35.128989] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-11 19:35:35.129084] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:35:35.129183] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:35:35.129973] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:35:35.130040] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:35:35.130258] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:35:35.130306] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:35:35.130503] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.180:24007 failed (Connection refused); disconnecting socket
[2018-07-11 19:35:35.130549] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 19:36:04.409900] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:36:15.465881] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:36:31.679644] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:36:31.679673] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:36:31.680001] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:36:31.741260] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:37:33.493511] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 19:37:33.494074] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 19:37:48.907265] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:38:56.689570] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 19:38:56.766449] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 19:38:57.636270] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-11 19:38:57.652069] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 19:38:57.652156] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 19:39:00.003554] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-11 19:39:00.010789] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-11 19:39:00.011076] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-11 19:39:00.011127] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-11 19:39:00.011764] I [MSGID: 106599] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-11 19:39:00.011871] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-11 19:39:00.014700] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped
[2018-07-11 19:39:00.014758] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-11 19:39:00.014828] I [MSGID: 106567] [glusterd-svc-mgmt.c:203:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-11 19:39:01.017992] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-11 19:39:01.018560] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-11 19:39:01.018654] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-11 19:39:01.018781] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-11 19:39:01.019186] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-11 19:39:01.019225] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-11 19:39:01.019306] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-11 19:39:01.019701] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-11 19:39:01.019747] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-11 19:39:01.019890] I [glusterd-utils.c:6080:glusterd_brick_start] 0-management: starting a fresh brick process for brick /urd-gds/gluster
[2018-07-11 19:39:01.025052] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-11 19:39:01.115795] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:39:01.223179] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 19:39:01.223269] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 19:39:01.236932] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-11 19:39:01.291714] I [socket.c:2625:socket_event_handler] 0-transport: EPOLLERR - disconnecting now
[2018-07-11 19:39:01.352484] I [MSGID: 106005] [glusterd-handler.c:6131:__glusterd_brick_rpc_notify] 0-management: Brick urd-gds-001:/urd-gds/gluster has disconnected from glusterd.
[2018-07-11 19:39:01.352784] E [MSGID: 101012] [common-utils.c:4011:gf_is_service_running] 0-: Unable to read pidfile: /var/run/gluster/vols/urd-gds-volume/urd-gds-001-urd-gds-gluster.pid
[2018-07-11 19:39:01.436163] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-11 19:39:02.438187] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-gfproxyd: setting frame-timeout to 600
[2018-07-11 19:39:02.441870] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 19:39:02.450427] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-11 19:39:03.942224] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:39:04.823327] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-11 19:40:02.256161] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 19:40:02.388061] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 19:40:04.030056] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-11 19:40:04.070761] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:40:04.315217] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 19:40:05.232464] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 19:40:05.232561] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 19:40:05.503975] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-11 19:40:05.510920] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 19:40:05.511011] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 19:40:05.529788] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-11 19:41:15.826332] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:41:21.229230] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 19:41:21.422758] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 19:41:22.419216] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-11 19:41:22.460319] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:41:22.706456] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 19:41:23.705321] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 19:41:23.705476] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 19:41:24.149053] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-11 19:41:24.156381] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 19:41:24.156471] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 19:41:24.191862] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-11 19:42:31.085266] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-11 19:42:31.311522] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 19:42:31.701247] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-11 19:42:31.744022] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:42:31.993799] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 19:42:33.834976] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 19:42:33.835133] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 19:42:34.312531] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-11 19:42:34.319053] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 19:42:34.319144] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-11 19:42:34.388112] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-11 19:42:56.563807] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:42:58.700116] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:43:05.439328] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:43:05.439359] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:43:05.439403] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:43:05.501570] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:43:23.101156] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:43:23.171670] E [MSGID: 106308] [glusterd-geo-rep.c:2186:glusterd_op_verify_gsync_running] 0-management: geo-replication session b/w urd-gds-volume & geouser@urd-gds-geo-001::urd-gds-volume is not running on this node.
[2018-07-11 19:43:23.202493] E [MSGID: 106301] [glusterd-syncop.c:1352:gd_stage_op_phase] 0-management: Staging of operation 'Volume Geo-replication' failed on localhost : Geo-replication session between urd-gds-volume and geouser@urd-gds-geo-001::urd-gds-volume is not Paused.
[2018-07-11 19:43:27.955822] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:43:30.935699] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:43:30.935727] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:43:30.935759] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:43:30.998877] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:43:39.091007] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:43:39.650330] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 19:43:43.275310] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:43:43.275337] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:43:43.275370] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:43:43.335910] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:43:47.805997] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:43:47.806027] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:43:47.868258] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:44:11.697835] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
The message "I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options" repeated 2 times between [2018-07-11 19:43:43.275310] and [2018-07-11 19:44:16.145418]
[2018-07-11 19:44:16.145419] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:44:16.145469] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:44:16.207698] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:47:14.958330] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:47:14.958348] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:47:14.958714] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:47:15.021406] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:47:27.751680] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:47:28.298863] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 19:47:33.691766] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:47:33.691767] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:47:33.691795] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:47:33.752144] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:47:39.870987] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:48:26.087814] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:49:32.593906] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:49:32.593926] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:49:32.594291] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:49:32.657839] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:48:20.359580] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:49:24.376807] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:49:43.105645] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:49:43.589933] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-11 19:49:57.410591] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:49:57.410608] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:49:57.410655] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:49:57.470250] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 19:50:03.494195] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:50:08.880726] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:51:16.352378] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 19:53:28.362306] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 19:57:52.551190] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-11 19:57:52.551205] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-11 19:57:52.551516] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-11 19:57:52.614553] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 20:01:15.811285] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 20:03:27.799663] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 20:08:36.306302] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 20:08:43.413153] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 20:11:16.309846] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 20:13:28.264493] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 20:21:15.761217] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 20:23:27.672001] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 20:27:32.140410] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 20:27:37.609311] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 20:28:00.086869] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 20:31:16.186846] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 20:33:28.076416] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 20:41:15.658709] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 20:43:27.510431] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 20:51:16.129735] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 20:53:27.968922] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 21:01:15.570543] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 21:03:28.461164] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 21:11:15.971351] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 21:13:27.839781] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 21:21:16.423497] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 21:23:28.311834] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 21:31:15.864479] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 21:33:27.713350] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 21:41:16.310173] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 21:43:28.151314] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 21:51:15.798571] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 21:53:27.616605] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 22:01:16.255896] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 22:03:28.041284] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 22:11:15.709384] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 22:13:27.443768] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 22:21:16.130490] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 22:23:27.883266] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 22:31:15.550592] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 22:33:28.342795] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 22:41:15.986003] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 22:43:27.882754] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 22:51:16.392190] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-11 22:53:28.320408] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-11 23:00:01.303656] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-11 23:00:01.375024] E [MSGID: 106308] [glusterd-geo-rep.c:2186:glusterd_op_verify_gsync_running] 0-management: geo-replication session b/w urd-gds-volume & geouser@urd-gds-geo-001::urd-gds-volume is not running on this node.
[2018-07-11 23:00:01.669433] W [MSGID: 106325] [glusterd-geo-rep.c:5647:glusterd_op_gsync_set] 0-management: geo-replication is not set up for urd-gds-volume(master) and geouser@urd-gds-geo-001::urd-gds-volume(slave)
[2018-07-11 23:00:01.720116] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.180:24007 failed (No data available)
[2018-07-11 23:00:01.720143] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 23:00:01.720249] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7fa29ebf930a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7fa29ec03540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7fa29ecbd35c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 23:00:01.720259] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-11 23:00:01.728765] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.183:24007 failed (No data available)
[2018-07-11 23:00:01.728792] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 23:00:01.728892] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7fa29ebf930a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7fa29ec03540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7fa29ecbd35c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 23:00:01.728905] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-11 23:00:01.763719] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.184:24007 failed (Connection reset by peer)
[2018-07-11 23:00:01.763809] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 23:00:01.764040] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7fa29ebf930a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7fa29ec03540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7fa29ecbd35c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 23:00:01.764091] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-11 23:00:01.764699] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fa2aa16f2cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7fa2a9f38d6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7fa2a9f38e8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7fa2a9f3a4dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7fa2a9f3b048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-11 23:00:01.753193 (xid=0x9a)
[2018-07-11 23:00:01.764762] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-004. Please check log file for details.
[2018-07-11 23:00:01.765909] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.182:24007 failed (Connection reset by peer)
[2018-07-11 23:00:01.765981] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-11 23:00:01.766247] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7fa29ebf930a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7fa29ec03540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7fa29ecbd35c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-11 23:00:01.766283] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-11 23:00:01.766854] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7fa2aa16f2cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7fa2a9f38d6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7fa2a9f38e8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7fa2a9f3a4dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7fa2a9f3b048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-11 23:00:01.753150 (xid=0x9a)
[2018-07-11 23:00:01.766909] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-002. Please check log file for details.
[2018-07-11 23:00:01.767019] E [MSGID: 106151] [glusterd-syncop.c:1640:gd_unlock_op_phase] 0-management: Failed to unlock on some peer(s)
[2018-07-11 23:00:01.767425] E [mem-pool.c:326:__gf_free] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1870) [0x7fa29ecb6870] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1000) [0x7fa29ecb6000] -->/lib64/libglusterfs.so.0(__gf_free+0xa4) [0x7fa2aa198494] ) 0-: Assertion failed: GF_MEM_HEADER_MAGIC == header->magic
pending frames:
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
patchset: git://git.gluster.org/glusterfs.git
signal received: 6
time of crash: 
2018-07-11 23:00:01
configuration details:
argp 1
backtrace 1
dlfcn 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 4.1.1
/lib64/libglusterfs.so.0(+0x25920)[0x7fa2aa16d920]
/lib64/libglusterfs.so.0(gf_print_trace+0x334)[0x7fa2aa177874]
/lib64/libc.so.6(+0x362f0)[0x7fa2a87d22f0]
/lib64/libc.so.6(gsignal+0x37)[0x7fa2a87d2277]
/lib64/libc.so.6(abort+0x148)[0x7fa2a87d3968]
/lib64/libc.so.6(+0x78d37)[0x7fa2a8814d37]
/lib64/libc.so.6(+0x7f5e4)[0x7fa2a881b5e4]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1000)[0x7fa29ecb6000]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1870)[0x7fa29ecb6870]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xae808)[0x7fa29ec83808]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2436e)[0x7fa29ebf936e]
/lib64/libglusterfs.so.0(+0x622b0)[0x7fa2aa1aa2b0]
/lib64/libc.so.6(+0x48030)[0x7fa2a87e4030]
---------
[2018-07-12 11:15:58.924726] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 4.1.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-12 11:15:58.944100] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-12 11:15:58.944215] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-12 11:15:58.944241] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-12 11:15:58.956253] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-12 11:15:58.956308] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-12 11:15:58.956336] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-12 11:15:58.956357] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-12 11:15:59.647169] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-12 11:15:59.768239] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-12 11:15:59.781567] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-12 11:15:59.781567] and [2018-07-12 11:15:59.782290]
[2018-07-12 11:15:59.782340] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-12 11:15:59.782378] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 11:15:59.788548] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 11:15:59.792869] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 11:15:59.796895] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-12 11:15:59.782340] and [2018-07-12 11:15:59.796887]
[2018-07-12 11:15:59.801686] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.808229] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.816566] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.825712] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.831929] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.841260] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.850493] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.858447] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.868173] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.876698] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.887590] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.898655] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.908254] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.921322] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.931492] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1/gluster [No such file or directory]
[2018-07-12 11:15:59.942630] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/3ceb415b1f254aab9c5efd22b2a10ce6/brick1/gluster [No such file or directory]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-12 11:15:59.954006] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-12 11:15:59.954134] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (Connection refused); disconnecting socket
[2018-07-12 11:15:59.954205] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 11:15:59.954537] I [glusterd-utils.c:5985:glusterd_brick_start] 0-management: discovered already-running brick /urd-gds/gluster
[2018-07-12 11:15:59.954579] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-12 11:15:59.954667] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 11:15:59.954949] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (Connection refused); disconnecting socket
[2018-07-12 11:15:59.954985] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 11:15:59.955139] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (Connection refused); disconnecting socket
[2018-07-12 11:15:59.955192] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 11:16:00.549049] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-12 11:16:01.054966] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-12 11:16:01.518931] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-12 11:16:01.529248] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-12 11:16:01.529568] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-12 11:16:01.529638] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-12 11:16:01.529690] I [MSGID: 106599] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-12 11:16:01.529792] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-12 11:16:01.533220] I [MSGID: 106568] [glusterd-proc-mgmt.c:87:glusterd_proc_stop] 0-management: Stopping glustershd daemon running in pid: 2093
[2018-07-12 11:16:02.533734] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-12 11:16:02.533926] I [MSGID: 106567] [glusterd-svc-mgmt.c:203:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-12 11:16:03.540724] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-12 11:16:03.541294] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-12 11:16:03.541350] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-12 11:16:03.541441] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-12 11:16:03.541963] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-12 11:16:03.542040] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-12 11:16:03.542131] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-12 11:16:03.542568] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-12 11:16:03.542628] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-12 11:16:03.649275] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 11:16:03.782823] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 11:16:03.783076] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 11:16:03.813567] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 11:16:03.877549] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-12 11:16:03.888307] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-12 11:16:03.888713] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-gfproxyd: setting frame-timeout to 600
[2018-07-12 11:16:03.890411] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 11:16:03.890432] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 11:16:03.908051] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 11:16:03.942707] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 11:18:08.938184] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 11:18:09.035765] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 11:18:09.745600] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-12 11:18:09.797084] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 11:18:10.040569] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 11:18:11.362213] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 11:18:11.362279] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 11:18:11.639208] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-12 11:18:11.646577] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 11:18:11.646745] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 11:18:11.666261] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 11:19:52.201492] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 11:19:52.393020] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 11:19:52.489363] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-12 11:19:52.552639] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 11:19:52.810743] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 11:19:54.890426] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 11:19:54.890528] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 11:19:55.201068] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-12 11:19:55.208552] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 11:19:55.208693] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 11:19:55.226032] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 11:21:40.398600] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 11:21:40.600097] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 11:21:40.689940] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-12 11:21:40.732496] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 11:21:40.985897] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 11:21:43.174653] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 11:21:43.174714] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 11:21:43.701821] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-12 11:21:43.706606] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 11:21:43.706728] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 11:21:43.746369] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 11:23:28.527260] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 11:25:15.772922] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 11:33:28.067242] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 11:35:16.186537] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 11:43:27.505732] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 11:45:15.612165] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 11:53:27.950757] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 11:55:16.104096] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 12:03:28.416212] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 12:05:15.563468] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 12:13:27.834901] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 12:15:15.978227] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 12:23:28.223531] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 12:25:16.384747] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 12:33:27.627218] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 12:35:15.862153] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 12:43:28.173722] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 12:45:16.350745] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 12:53:27.723014] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 12:55:15.815250] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 13:03:28.139739] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 13:05:16.257690] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 13:13:27.565096] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 13:15:15.684992] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 13:23:28.029826] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 13:25:16.150048] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 13:33:27.511445] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 13:35:15.614182] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 13:43:27.913888] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 13:45:16.006501] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 13:53:28.368651] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 13:55:15.456019] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 14:03:27.764357] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 14:05:15.841837] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 14:13:28.163790] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 14:15:16.233658] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 14:23:27.556079] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 14:25:15.698983] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 14:33:28.048535] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 14:35:16.152743] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 14:43:27.454528] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 14:45:15.549390] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 14:53:27.911272] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 14:55:16.090736] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 15:03:28.398307] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 15:05:15.581707] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 15:13:27.832874] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 15:15:16.078018] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 15:23:28.246869] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 15:25:15.470884] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 15:33:27.648375] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 15:35:15.870872] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 15:43:28.051525] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 15:45:16.252011] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 15:53:27.528798] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 15:55:15.743527] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 16:03:28.007015] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 16:05:16.134452] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 16:13:27.565665] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 16:15:15.599562] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 16:23:28.088385] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 16:25:15.967520] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 16:33:27.484333] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 16:35:16.359526] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 16:43:27.890570] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 16:45:15.838976] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 16:53:28.407872] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 16:55:16.321148] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 17:03:27.827565] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 17:05:15.856441] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 17:13:28.324774] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 17:15:16.390071] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 17:23:27.764931] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 17:25:15.918588] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 17:33:28.277010] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 17:35:16.430243] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 17:43:27.681088] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 17:45:15.954950] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 17:53:28.120789] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 17:55:16.473626] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 18:03:27.661290] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 18:05:15.965786] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 18:13:28.148055] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 18:15:16.390062] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 18:23:27.606171] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 18:25:15.799044] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 18:33:28.135797] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 18:35:16.232751] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 18:43:27.669028] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 18:45:15.617357] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 18:53:28.211388] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 18:55:16.005982] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 18:59:14.843892] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.180:24007 failed (No data available)
[2018-07-12 18:59:14.843981] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 18:59:14.844699] W [glusterd-locks.c:845:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f264d66f30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f264d679540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe8543) [0x7f264d733543] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-12 18:59:14.844745] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-12 18:59:26.802100] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.180:24007 failed (Connection refused); disconnecting socket
[2018-07-12 18:59:40.729982] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 18:59:40.906822] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 18:59:40.950387] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-12 18:59:40.994959] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 18:59:41.109393] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 18:59:41.109456] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 18:59:41.275792] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 18:59:43.057539] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 18:59:43.274317] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-12 18:59:43.282680] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 18:59:43.282794] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 18:59:43.361861] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 18:59:48.384145] W [glusterfsd.c:1514:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7f2657a47e25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x563dfaf4ed65] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x563dfaf4eb8b] ) 0-: received signum (15), shutting down
[2018-07-12 19:00:13.115826] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 4.1.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-12 19:00:13.120228] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-12 19:00:13.120259] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-12 19:00:13.120265] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-12 19:00:13.123814] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-12 19:00:13.123829] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-12 19:00:13.123836] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-12 19:00:13.123841] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-12 19:00:13.870975] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-12 19:00:13.998530] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-12 19:00:14.008161] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-12 19:00:14.008161] and [2018-07-12 19:00:14.008261]
[2018-07-12 19:00:14.008289] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-12 19:00:14.008306] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 19:00:14.012421] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 19:00:14.016045] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 19:00:14.019726] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-12 19:00:14.008289] and [2018-07-12 19:00:14.019721]
[2018-07-12 19:00:14.023764] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.028242] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.034798] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.041188] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.045578] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.052855] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.058536] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.064278] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.073820] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.082901] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.093946] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.104710] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.114155] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.126086] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.136634] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1/gluster [No such file or directory]
[2018-07-12 19:00:14.146832] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/3ceb415b1f254aab9c5efd22b2a10ce6/brick1/gluster [No such file or directory]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-12 19:00:14.158208] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-12 19:00:14.310320] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:00:14.316441] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:00:14.320528] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-12 19:00:14.325111] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:00:14.331451] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:00:14.331496] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:00:14.345326] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-12 19:00:14.345471] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-12 19:00:14.345486] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-12 19:00:14.345497] I [MSGID: 106599] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-12 19:00:14.345519] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-12 19:00:14.346183] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped
[2018-07-12 19:00:14.346199] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-12 19:00:14.346227] I [MSGID: 106567] [glusterd-svc-mgmt.c:203:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-12 19:00:15.348307] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-12 19:00:15.348920] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-12 19:00:15.348982] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-12 19:00:15.349074] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-12 19:00:15.349480] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-12 19:00:15.349518] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-12 19:00:15.349599] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-12 19:00:15.350006] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-12 19:00:15.350044] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-12 19:00:15.350245] I [glusterd-utils.c:6080:glusterd_brick_start] 0-management: starting a fresh brick process for brick /urd-gds/gluster
[2018-07-12 19:00:15.355368] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 19:00:15.469425] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:00:15.579899] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:00:15.612407] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-12 19:00:15.631472] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-12 19:00:15.694144] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:00:15.805982] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-12 19:00:15.808291] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-gfproxyd: setting frame-timeout to 600
[2018-07-12 19:00:15.836556] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:00:15.948268] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:00:15.948323] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:00:15.963407] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-12 19:00:15.997445] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:00:16.108386] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:00:16.108432] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:00:16.124394] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-12 19:00:16.129050] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:00:16.133561] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:00:16.190830] I [socket.c:2625:socket_event_handler] 0-transport: EPOLLERR - disconnecting now
[2018-07-12 19:00:16.256453] I [MSGID: 106005] [glusterd-handler.c:6131:__glusterd_brick_rpc_notify] 0-management: Brick urd-gds-001:/urd-gds/gluster has disconnected from glusterd.
[2018-07-12 19:00:16.257898] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:00:16.258023] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:00:16.276186] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:00:16.276489] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:00:16.357618] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-12 19:00:16.368651] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:00:16.414214] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-12 19:00:16.423370] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:00:16.423394] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:00:16.423433] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:00:16.473181] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-12 19:00:16.482469] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:00:16.482492] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:00:16.499703] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:00:16.499794] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:00:16.517658] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:00:16.523871] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:00:16.523892] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:00:16.538202] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:00:16.539524] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:00:16.539647] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:00:16.539664] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:00:16.554308] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:00:16.561587] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:00:16.589762] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49155
[2018-07-12 19:00:16.718014] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49154
[2018-07-12 19:00:36.715996] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.182:24007 failed (No data available)
[2018-07-12 19:00:36.716086] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 19:00:36.716422] W [glusterd-locks.c:845:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f459d9ee30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f459d9f8540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe8543) [0x7f459dab2543] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-12 19:00:36.716457] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-12 19:00:47.132523] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (Connection refused); disconnecting socket
[2018-07-12 19:01:02.418682] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:01:02.621593] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:01:02.671918] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-12 19:01:02.752124] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:01:03.010273] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:01:04.225251] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:01:04.225368] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:01:04.381394] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:01:04.579884] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-12 19:01:04.586699] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:01:04.586787] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:01:04.641443] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:01:19.675797] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.183:24007 failed (No data available)
[2018-07-12 19:01:19.676197] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 19:01:19.676509] W [glusterd-locks.c:845:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f459d9ee30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f459d9f8540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe8543) [0x7f459dab2543] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-12 19:01:19.676544] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-12 19:01:31.190078] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (Connection refused); disconnecting socket
[2018-07-12 19:01:39.683962] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:01:41.150635] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-12 19:01:41.199558] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:01:41.343306] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:01:41.384898] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-12 19:01:41.395597] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:01:41.395637] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:01:41.453310] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:01:41.668871] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:01:41.686283] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:01:41.686379] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:01:41.937908] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:01:47.381932] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.184:24007 failed (No data available)
[2018-07-12 19:01:47.382055] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 19:01:47.382347] W [glusterd-locks.c:845:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f459d9ee30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f459d9f8540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe8543) [0x7f459dab2543] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-12 19:01:47.382381] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-12 19:01:58.224543] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (Connection refused); disconnecting socket
[2018-07-12 19:02:15.787834] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:02:15.979288] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:02:16.307636] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-12 19:02:16.348634] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:02:16.601240] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:02:17.382630] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:02:17.382778] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:02:17.805832] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:02:17.982082] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-12 19:02:17.989361] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:02:17.989451] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:02:18.069567] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:03:27.640237] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 19:05:06.849039] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 19:05:15.477192] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 19:05:15.492987] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 19:06:43.668446] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:06:43.668461] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:06:43.668817] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:06:43.733361] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:07:07.809633] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:07:08.378669] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:07:12.045284] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:07:25.123802] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
The message "W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found" repeated 2 times between [2018-07-12 19:06:43.668817] and [2018-07-12 19:07:31.456852]
[2018-07-12 19:07:31.522276] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
The message "I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options" repeated 2 times between [2018-07-12 19:06:43.668446] and [2018-07-12 19:07:31.456819]
The message "I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options" repeated 2 times between [2018-07-12 19:06:43.668461] and [2018-07-12 19:07:31.456820]
[2018-07-12 19:09:30.990661] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:09:30.990688] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:09:30.990915] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:09:31.060565] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:13:28.052422] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 19:15:15.948886] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 19:16:43.693215] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:16:43.693233] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:16:43.693544] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:16:43.757568] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:16:48.845932] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:16:49.405325] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:17:00.156953] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:17:00.156954] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:17:00.156982] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:17:00.218572] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:23:27.458577] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 19:24:14.523791] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:24:14.523812] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:24:14.523931] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:24:14.587949] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:24:21.772438] W [glusterfsd.c:1514:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7e25) [0x7f45a7dc6e25] -->/usr/sbin/glusterd(glusterfs_sigwaiter+0xe5) [0x56283cdbdd65] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x56283cdbdb8b] ) 0-: received signum (15), shutting down
[2018-07-12 19:27:02.768000] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 4.1.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-12 19:27:02.776205] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-12 19:27:02.776240] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-12 19:27:02.776249] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-12 19:27:02.779310] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-12 19:27:02.779327] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-12 19:27:02.779339] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-12 19:27:02.779344] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-12 19:27:03.503614] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-12 19:27:03.624897] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-12 19:27:03.636476] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-12 19:27:03.636476] and [2018-07-12 19:27:03.636904]
[2018-07-12 19:27:03.636934] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-12 19:27:03.636955] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 19:27:03.641583] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 19:27:03.645110] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 19:27:03.648555] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-12 19:27:03.636934] and [2018-07-12 19:27:03.648549]
[2018-07-12 19:27:03.652392] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.657952] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.663216] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.670494] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.674832] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.681068] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.687220] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.691857] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.699567] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.705482] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.711826] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.720146] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.727335] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.737535] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.749006] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1/gluster [No such file or directory]
[2018-07-12 19:27:03.758564] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/3ceb415b1f254aab9c5efd22b2a10ce6/brick1/gluster [No such file or directory]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-12 19:27:03.770142] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-12 19:27:03.815674] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:27:03.934829] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-12 19:27:03.951156] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:27:03.993017] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-12 19:27:04.011258] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:27:04.011294] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:27:04.037382] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-12 19:27:04.037465] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-12 19:27:04.037479] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-12 19:27:04.037491] I [MSGID: 106599] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-12 19:27:04.037521] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-12 19:27:04.038217] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: glustershd already stopped
[2018-07-12 19:27:04.038229] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-12 19:27:04.038259] I [MSGID: 106567] [glusterd-svc-mgmt.c:203:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-12 19:27:05.040903] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-12 19:27:05.041479] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-12 19:27:05.041548] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-12 19:27:05.041640] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-12 19:27:05.042058] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-12 19:27:05.042097] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-12 19:27:05.042180] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-12 19:27:05.042569] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-12 19:27:05.042614] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-12 19:27:05.042772] I [glusterd-utils.c:6080:glusterd_brick_start] 0-management: starting a fresh brick process for brick /urd-gds/gluster
[2018-07-12 19:27:05.050136] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-12 19:27:05.156020] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:27:05.272026] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:27:05.324434] I [socket.c:2625:socket_event_handler] 0-transport: EPOLLERR - disconnecting now
[2018-07-12 19:27:05.324514] E [MSGID: 101012] [common-utils.c:4011:gf_is_service_running] 0-: Unable to read pidfile: /var/run/gluster/vols/urd-gds-volume/urd-gds-001-urd-gds-gluster.pid
[2018-07-12 19:27:05.324577] E [MSGID: 101012] [common-utils.c:4011:gf_is_service_running] 0-: Unable to read pidfile: /var/run/gluster/glustershd/glustershd.pid
[2018-07-12 19:27:05.375023] I [MSGID: 106005] [glusterd-handler.c:6131:__glusterd_brick_rpc_notify] 0-management: Brick urd-gds-001:/urd-gds/gluster has disconnected from glusterd.
[2018-07-12 19:27:05.375113] E [MSGID: 101012] [common-utils.c:4011:gf_is_service_running] 0-: Unable to read pidfile: /var/run/gluster/vols/urd-gds-volume/urd-gds-001-urd-gds-gluster.pid
[2018-07-12 19:27:05.375304] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:27:05.375337] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:27:05.393871] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-12 19:27:06.396573] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-gfproxyd: setting frame-timeout to 600
[2018-07-12 19:27:06.397384] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:27:06.433333] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-12 19:27:06.441991] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:27:06.647807] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (No route to host); disconnecting socket
[2018-07-12 19:27:06.647910] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 19:27:06.648177] E [MSGID: 101012] [common-utils.c:4011:gf_is_service_running] 0-: Unable to read pidfile: /var/run/gluster/vols/urd-gds-volume/urd-gds-001-urd-gds-gluster.pid
[2018-07-12 19:27:06.652783] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (No route to host); disconnecting socket
[2018-07-12 19:27:06.652867] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 19:27:06.653124] E [MSGID: 101012] [common-utils.c:4011:gf_is_service_running] 0-: Unable to read pidfile: /var/run/gluster/vols/urd-gds-volume/urd-gds-001-urd-gds-gluster.pid
[2018-07-12 19:27:06.655783] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (No route to host); disconnecting socket
[2018-07-12 19:27:06.655867] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 19:27:06.656127] E [MSGID: 101012] [common-utils.c:4011:gf_is_service_running] 0-: Unable to read pidfile: /var/run/gluster/vols/urd-gds-volume/urd-gds-001-urd-gds-gluster.pid
[2018-07-12 19:27:07.407125] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-12 19:27:14.433996] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:27:14.561404] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:27:15.702999] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-12 19:27:15.965074] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:27:15.965168] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:27:16.469638] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 19:27:17.231382] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-12 19:27:17.237974] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:27:17.238016] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:27:17.292225] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:27:17.405863] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-12 19:27:17.546490] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:27:18.829632] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:27:19.002287] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:27:19.063758] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-12 19:27:19.103129] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:27:19.353686] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:27:20.276829] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:27:20.276930] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:27:21.752431] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-12 19:27:21.758782] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:27:21.758872] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:27:21.778840] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-12 19:28:11.771599] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:28:11.921208] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:28:13.053670] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-12 19:28:13.096612] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:28:13.347007] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:28:13.420305] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:28:13.420348] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:28:15.064531] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-12 19:28:15.069935] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:28:15.070024] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:28:15.151339] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-12 19:29:15.865660] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 19:29:29.206486] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.180:24007 failed (No data available)
[2018-07-12 19:29:29.206658] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 19:29:29.207315] W [glusterd-locks.c:845:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f7d4106d30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f7d41077540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe8543) [0x7f7d41131543] ) 0-management: Lock for vol urd-gds-volume not held
[2018-07-12 19:29:29.207351] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-12 19:30:46.205735] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.180:24007 failed (No route to host); disconnecting socket
[2018-07-12 19:32:11.345200] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-12 19:32:11.555155] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-12 19:32:11.605318] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:32:11.713870] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:32:11.750885] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-12 19:32:11.760256] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:32:11.760279] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:32:11.775558] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:32:11.888419] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:32:12.870041] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:32:12.870143] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-12 19:32:12.906300] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-12 19:32:59.982024] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:32:59.982041] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:32:59.982377] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:33:00.049703] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:33:08.657119] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 19:33:26.966086] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 19:34:38.246180] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:34:38.246203] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:34:38.246358] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:34:38.309336] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:34:56.347310] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:34:56.923475] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:33:27.893010] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 19:35:07.667282] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:35:07.667302] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:35:07.667335] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:35:07.728439] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:35:10.969926] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:35:10.969927] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:35:10.969959] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:35:11.035909] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:39:16.270960] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 19:39:20.854880] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:39:20.928363] E [MSGID: 106308] [glusterd-geo-rep.c:2186:glusterd_op_verify_gsync_running] 0-management: geo-replication session b/w urd-gds-volume & geouser@urd-gds-geo-001::urd-gds-volume is not running on this node.
[2018-07-12 19:39:20.960277] E [MSGID: 106301] [glusterd-syncop.c:1352:gd_stage_op_phase] 0-management: Staging of operation 'Volume Geo-replication' failed on localhost : Geo-replication session between urd-gds-volume and geouser@urd-gds-geo-001::urd-gds-volume is not Paused.
[2018-07-12 19:43:28.377370] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 19:44:00.671011] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:44:00.671026] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:44:00.671070] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:44:00.734641] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:43:54.299897] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 19:45:45.317471] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:45:45.863764] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-12 19:45:50.219796] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:45:50.219812] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:45:50.219848] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:45:50.280450] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:45:53.223265] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:45:53.223298] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:45:53.223262] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:45:53.287144] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:49:15.722969] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 19:53:27.863914] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 19:55:56.560116] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 19:56:59.014116] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:56:59.014133] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:56:59.014165] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:56:59.077199] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:57:05.656711] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 19:57:05.656729] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 19:57:05.656946] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 19:57:05.717416] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 19:59:16.193883] I [MSGID: 106487] [glusterd-handler.c:1486:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
[2018-07-12 20:00:27.483451] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 20:00:34.020966] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-12 20:00:34.020982] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-12 20:00:34.021014] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-12 20:00:34.084954] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 20:03:28.286102] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-12 23:00:01.496923] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-12 23:00:01.570139] E [MSGID: 106308] [glusterd-geo-rep.c:2186:glusterd_op_verify_gsync_running] 0-management: geo-replication session b/w urd-gds-volume & geouser@urd-gds-geo-001::urd-gds-volume is not running on this node.
[2018-07-12 23:00:01.842883] W [MSGID: 106325] [glusterd-geo-rep.c:5647:glusterd_op_gsync_set] 0-management: geo-replication is not set up for urd-gds-volume(master) and geouser@urd-gds-geo-001::urd-gds-volume(slave)
[2018-07-12 23:00:01.905072] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.182:24007 failed (No data available)
[2018-07-12 23:00:01.905108] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 23:00:01.905230] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f7d4106d30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f7d41077540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f7d4113135c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-12 23:00:01.905246] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-12 23:00:01.928054] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.183:24007 failed (Connection reset by peer)
[2018-07-12 23:00:01.928126] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 23:00:01.928369] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f7d4106d30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f7d41077540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f7d4113135c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-12 23:00:01.928400] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-12 23:00:01.928979] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7d4c5e32cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7f7d4c3acd6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7f7d4c3ace8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7d4c3ae4dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7f7d4c3af048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-12 23:00:01.925196 (xid=0x50)
[2018-07-12 23:00:01.929067] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-003. Please check log file for details.
[2018-07-12 23:00:01.937585] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.184:24007 failed (Connection reset by peer)
[2018-07-12 23:00:01.937698] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 23:00:01.937938] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f7d4106d30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f7d41077540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f7d4113135c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-12 23:00:01.937974] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-12 23:00:01.938509] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7d4c5e32cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7f7d4c3acd6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7f7d4c3ace8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7d4c3ae4dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7f7d4c3af048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-12 23:00:01.925238 (xid=0x50)
[2018-07-12 23:00:01.938592] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-004. Please check log file for details.
[2018-07-12 23:00:01.938682] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-12 23:00:01.939054] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f7d4106d30a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f7d41077540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f7d4113135c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-12 23:00:01.939103] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-12 23:00:01.939701] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f7d4c5e32cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7f7d4c3acd6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7f7d4c3ace8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f7d4c3ae4dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7f7d4c3af048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-12 23:00:01.925272 (xid=0x57)
[2018-07-12 23:00:01.939759] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-000. Please check log file for details.
[2018-07-12 23:00:01.939847] E [MSGID: 106151] [glusterd-syncop.c:1640:gd_unlock_op_phase] 0-management: Failed to unlock on some peer(s)
[2018-07-12 23:00:01.940322] E [mem-pool.c:326:__gf_free] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1870) [0x7f7d4112a870] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1000) [0x7f7d4112a000] -->/lib64/libglusterfs.so.0(__gf_free+0xa4) [0x7f7d4c60c494] ) 0-: Assertion failed: GF_MEM_HEADER_MAGIC == header->magic
pending frames:
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
patchset: git://git.gluster.org/glusterfs.git
signal received: 6
time of crash: 
2018-07-12 23:00:01
configuration details:
argp 1
backtrace 1
dlfcn 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 4.1.1
/lib64/libglusterfs.so.0(+0x25920)[0x7f7d4c5e1920]
/lib64/libglusterfs.so.0(gf_print_trace+0x334)[0x7f7d4c5eb874]
/lib64/libc.so.6(+0x362f0)[0x7f7d4ac462f0]
/lib64/libc.so.6(gsignal+0x37)[0x7f7d4ac46277]
/lib64/libc.so.6(abort+0x148)[0x7f7d4ac47968]
/lib64/libc.so.6(+0x78d37)[0x7f7d4ac88d37]
/lib64/libc.so.6(+0x81499)[0x7f7d4ac91499]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1000)[0x7f7d4112a000]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1870)[0x7f7d4112a870]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xae808)[0x7f7d410f7808]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2436e)[0x7f7d4106d36e]
/lib64/libglusterfs.so.0(+0x622b0)[0x7f7d4c61e2b0]
/lib64/libc.so.6(+0x48030)[0x7f7d4ac58030]
---------
[2018-07-13 05:39:14.589579] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 4.1.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-13 05:39:14.593719] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-13 05:39:14.593750] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-13 05:39:14.593757] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-13 05:39:14.596577] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-13 05:39:14.596588] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-13 05:39:14.596593] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-13 05:39:14.596598] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-13 05:39:15.270116] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-13 05:39:15.390457] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-13 05:39:15.398814] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-13 05:39:15.398814] and [2018-07-13 05:39:15.398915]
[2018-07-13 05:39:15.398944] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-13 05:39:15.398962] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-13 05:39:15.403157] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-13 05:39:15.406809] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-13 05:39:15.410351] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-13 05:39:15.398944] and [2018-07-13 05:39:15.410346]
[2018-07-13 05:39:15.414231] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.418738] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.425766] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.430984] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.436460] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.443509] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.447943] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.454179] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.460612] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.464899] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.474064] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.483947] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.493863] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.505355] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.514278] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1/gluster [No such file or directory]
[2018-07-13 05:39:15.525155] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/3ceb415b1f254aab9c5efd22b2a10ce6/brick1/gluster [No such file or directory]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-13 05:39:15.534877] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-13 05:39:15.535055] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (Connection refused); disconnecting socket
[2018-07-13 05:39:15.535146] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 05:39:15.535633] I [glusterd-utils.c:5985:glusterd_brick_start] 0-management: discovered already-running brick /urd-gds/gluster
[2018-07-13 05:39:15.535693] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-13 05:39:15.535786] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-13 05:39:15.536156] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (Connection refused); disconnecting socket
[2018-07-13 05:39:15.536207] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 05:39:15.536398] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (Connection refused); disconnecting socket
[2018-07-13 05:39:15.536444] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 05:39:15.536646] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.180:24007 failed (Connection refused); disconnecting socket
[2018-07-13 05:39:15.536700] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 05:39:15.595654] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-13 05:39:16.309232] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-13 05:39:26.687065] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-13 05:40:42.900335] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-13 05:40:42.976633] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-13 05:40:43.278023] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-13 05:40:43.289330] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-13 05:40:43.289351] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 05:40:45.589239] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-13 05:40:45.595426] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-13 05:40:45.595621] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-13 05:40:45.595659] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-13 05:40:45.595682] I [MSGID: 106599] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-13 05:40:45.595732] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-13 05:40:45.597671] I [MSGID: 106568] [glusterd-proc-mgmt.c:87:glusterd_proc_stop] 0-management: Stopping glustershd daemon running in pid: 1750
[2018-07-13 05:40:46.598061] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-13 05:40:46.598281] I [MSGID: 106567] [glusterd-svc-mgmt.c:203:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-13 05:40:47.604455] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-13 05:40:47.605052] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-13 05:40:47.605107] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-13 05:40:47.605197] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-13 05:40:47.605628] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-13 05:40:47.605677] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-13 05:40:47.605759] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-13 05:40:47.606136] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-13 05:40:47.606173] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-13 05:40:47.699389] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 05:40:47.805920] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-13 05:40:47.806043] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 05:40:47.819973] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-13 05:40:47.880183] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-13 05:40:47.880734] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-gfproxyd: setting frame-timeout to 600
[2018-07-13 05:40:47.949060] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 05:41:34.729887] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-13 05:41:34.799503] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-13 05:41:34.858839] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-13 05:41:34.897295] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 05:41:35.144241] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 05:41:37.109563] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-13 05:41:37.109686] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 05:41:37.308250] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-13 05:41:37.313665] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-13 05:41:37.313754] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 05:41:37.347647] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-13 05:42:30.608635] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-13 05:42:30.735348] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-13 05:42:31.868248] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-13 05:42:31.906605] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 05:42:32.149097] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 05:42:33.718717] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-13 05:42:33.718865] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 05:42:34.144101] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-13 05:42:34.148561] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-13 05:42:34.148581] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 05:42:34.252205] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-13 05:43:26.376597] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-13 05:43:26.507205] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-13 05:43:26.553771] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-13 05:43:26.594582] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 05:43:26.839963] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 05:43:28.800587] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-13 05:43:28.800694] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 05:43:29.189553] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-13 05:43:29.195797] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-13 05:43:29.195886] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 05:43:29.292568] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-13 18:40:29.832230] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-13 18:40:46.091013] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 18:40:46.091487] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 18:41:29.285039] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-13 18:41:29.285055] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-13 18:41:29.285090] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-13 18:41:29.351761] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 18:47:23.699419] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-13 18:47:23.699437] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-13 18:47:23.699469] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-13 18:47:23.761468] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 18:47:34.940251] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 18:47:35.508992] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 18:47:40.631831] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-13 18:47:40.631833] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-13 18:47:40.631861] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-13 18:47:40.699006] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 18:53:18.471897] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-13 18:53:18.471912] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-13 18:53:18.472129] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-13 18:53:18.536277] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 18:53:31.387514] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 18:53:31.462863] E [MSGID: 106308] [glusterd-geo-rep.c:2186:glusterd_op_verify_gsync_running] 0-management: geo-replication session b/w urd-gds-volume & geouser@urd-gds-geo-001::urd-gds-volume is not running on this node.
[2018-07-13 18:53:31.754333] W [MSGID: 106325] [glusterd-geo-rep.c:5647:glusterd_op_gsync_set] 0-management: geo-replication is not set up for urd-gds-volume(master) and geouser@urd-gds-geo-001::urd-gds-volume(slave)
[2018-07-13 18:53:31.816453] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.182:24007 failed (No data available)
[2018-07-13 18:53:31.816607] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 18:53:31.816956] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f517ccd230a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f517ccdc540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f517cd9635c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-13 18:53:31.816994] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-13 18:53:31.842121] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.180:24007 failed (Connection reset by peer)
[2018-07-13 18:53:31.842218] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 18:53:31.842453] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f517ccd230a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f517ccdc540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f517cd9635c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-13 18:53:31.842488] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-13 18:53:31.843055] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f51882482cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7f5188011d6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7f5188011e8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f51880134dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7f5188014048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-13 18:53:31.837687 (xid=0x1c)
[2018-07-13 18:53:31.843121] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-000. Please check log file for details.
[2018-07-13 18:53:31.846761] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.184:24007 failed (Connection reset by peer)
[2018-07-13 18:53:31.846840] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 18:53:31.847090] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f517ccd230a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f517ccdc540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f517cd9635c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-13 18:53:31.847150] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-13 18:53:31.847689] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f51882482cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7f5188011d6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7f5188011e8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f51880134dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7f5188014048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-13 18:53:31.837655 (xid=0x1c)
[2018-07-13 18:53:31.847747] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-004. Please check log file for details.
[2018-07-13 18:53:31.850166] W [socket.c:592:__socket_rwv] 0-management: readv on 192.168.67.183:24007 failed (Connection reset by peer)
[2018-07-13 18:53:31.850241] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 18:53:31.850499] W [glusterd-locks.c:856:glusterd_mgmt_v3_unlock] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2430a) [0x7f517ccd230a] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2e540) [0x7f517ccdc540] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe835c) [0x7f517cd9635c] ) 0-management: Lock owner mismatch. Lock for vol urd-gds-volume held by 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-13 18:53:31.850570] W [MSGID: 106117] [glusterd-handler.c:6407:__glusterd_peer_rpc_notify] 0-management: Lock not released for urd-gds-volume
[2018-07-13 18:53:31.851010] E [rpc-clnt.c:348:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f51882482cb] (--> /lib64/libgfrpc.so.0(+0xcd6e)[0x7f5188011d6e] (--> /lib64/libgfrpc.so.0(+0xce8e)[0x7f5188011e8e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x8d)[0x7f51880134dd] (--> /lib64/libgfrpc.so.0(+0xf048)[0x7f5188014048] ))))) 0-management: forced unwinding frame type(glusterd mgmt v3) op(--(6)) called at 2018-07-13 18:53:31.837602 (xid=0x1c)
[2018-07-13 18:53:31.851061] E [MSGID: 106115] [glusterd-mgmt.c:124:gd_mgmt_v3_collate_errors] 0-management: Unlocking failed on urd-gds-003. Please check log file for details.
[2018-07-13 18:53:31.851136] E [MSGID: 106151] [glusterd-syncop.c:1640:gd_unlock_op_phase] 0-management: Failed to unlock on some peer(s)
[2018-07-13 18:53:31.851477] E [mem-pool.c:326:__gf_free] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1870) [0x7f517cd8f870] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1000) [0x7f517cd8f000] -->/lib64/libglusterfs.so.0(__gf_free+0xa4) [0x7f5188271494] ) 0-: Assertion failed: GF_MEM_HEADER_MAGIC == header->magic
[2018-07-13 18:53:31.851594] E [mem-pool.c:335:__gf_free] (-->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1870) [0x7f517cd8f870] -->/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1000) [0x7f517cd8f000] -->/lib64/libglusterfs.so.0(__gf_free+0x104) [0x7f51882714f4] ) 0-: Assertion failed: GF_MEM_TRAILER_MAGIC == *(uint32_t *)((char *)free_ptr + header->size)
pending frames:
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
frame : type(0) op(0)
patchset: git://git.gluster.org/glusterfs.git
signal received: 11
time of crash: 
2018-07-13 18:53:31
configuration details:
argp 1
backtrace 1
dlfcn 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 4.1.1
/lib64/libglusterfs.so.0(+0x25920)[0x7f5188246920]
/lib64/libglusterfs.so.0(gf_print_trace+0x334)[0x7f5188250874]
/lib64/libc.so.6(+0x362f0)[0x7f51868ab2f0]
/lib64/libpthread.so.0(pthread_mutex_lock+0x0)[0x7f51870acc80]
/lib64/libglusterfs.so.0(__gf_free+0x12c)[0x7f518827151c]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1000)[0x7f517cd8f000]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xe1870)[0x7f517cd8f870]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0xae808)[0x7f517cd5c808]
/usr/lib64/glusterfs/4.1.1/xlator/mgmt/glusterd.so(+0x2436e)[0x7f517ccd236e]
/lib64/libglusterfs.so.0(+0x622b0)[0x7f51882832b0]
/lib64/libc.so.6(+0x48030)[0x7f51868bd030]
---------
[2018-07-13 18:54:20.648828] I [MSGID: 100030] [glusterfsd.c:2741:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 4.1.1 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO)
[2018-07-13 18:54:20.652233] I [MSGID: 106478] [glusterd.c:1423:init] 0-management: Maximum allowed open file descriptors set to 65536
[2018-07-13 18:54:20.652263] I [MSGID: 106479] [glusterd.c:1481:init] 0-management: Using /var/lib/glusterd as working directory
[2018-07-13 18:54:20.652270] I [MSGID: 106479] [glusterd.c:1486:init] 0-management: Using /var/run/gluster as pid file working directory
[2018-07-13 18:54:20.655153] E [rpc-transport.c:284:rpc_transport_load] 0-rpc-transport: /usr/lib64/glusterfs/4.1.1/rpc-transport/rdma.so: cannot open shared object file: No such file or directory
[2018-07-13 18:54:20.655164] W [rpc-transport.c:288:rpc_transport_load] 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not valid or not found on this machine
[2018-07-13 18:54:20.655170] W [rpcsvc.c:1781:rpcsvc_create_listener] 0-rpc-service: cannot create listener, initing the transport failed
[2018-07-13 18:54:20.655174] E [MSGID: 106244] [glusterd.c:1764:init] 0-management: creation of 1 listeners failed, continuing with succeeded transport
[2018-07-13 18:54:21.333901] I [MSGID: 106513] [glusterd-store.c:2240:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 31202
[2018-07-13 18:54:21.458747] I [MSGID: 106544] [glusterd.c:158:glusterd_uuid_init] 0-management: retrieved UUID: 912bebfd-1a7f-44dc-b0b7-f001a20d58cd
[2018-07-13 18:54:21.468500] I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0
The message "I [MSGID: 106498] [glusterd-handler.c:3614:glusterd_friend_add_from_peerinfo] 0-management: connect returned 0" repeated 3 times between [2018-07-13 18:54:21.468500] and [2018-07-13 18:54:21.468633]
[2018-07-13 18:54:21.468660] W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout
[2018-07-13 18:54:21.468678] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-13 18:54:21.472845] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-13 18:54:21.476346] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-13 18:54:21.479861] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
The message "W [MSGID: 106061] [glusterd-handler.c:3408:glusterd_transport_inet_options_build] 0-glusterd: Failed to get tcp-user-timeout" repeated 3 times between [2018-07-13 18:54:21.468660] and [2018-07-13 18:54:21.479856]
[2018-07-13 18:54:21.483783] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/96ccd6ce6ebd4e2c9bd64fc2041749fa/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.488276] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/287c97528556476bb8a2612c4f9b97cf/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.494707] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a68ccf841d0c43f0abe91f77f219bbc0/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.500879] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/28f7701242994e778a67da2c3c5daedd/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.505052] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/ff1a9e9f5b724c6fbe3dc8ef04e06120/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.512252] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9cff2c38172f46daab7ef8058fd12c54/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.527370] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/e08c794e61534d6eb390a3435fbbc6e0/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.531864] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/5e50990a04fd44788e35bb6c5efa2665/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.538245] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/8c30ca3d1fa646f89727a471007a3df9/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.548364] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/c92802475d26486ab491e99421fd9736/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.553829] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a0b0d5ec0cf442ccbf9d243b1d31af93/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.559206] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/a18cb6f1beb24d2daeb6591bd5d4a26f/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.566224] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/801d38406c8e44fa87f3174b9c4df742/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.570747] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/49ac3744afea42e3a88c2be306a8a677/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.577017] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/9e711977057b42b58c7a3262413752d2/brick1/gluster [No such file or directory]
[2018-07-13 18:54:21.583291] W [MSGID: 106425] [glusterd-store.c:2642:glusterd_store_retrieve_bricks] 0-management: failed to get statfs() call on brick /run/gluster/snaps/3ceb415b1f254aab9c5efd22b2a10ce6/brick1/gluster [No such file or directory]
Final graph:
+------------------------------------------------------------------------------+
  1: volume management
  2:     type mgmt/glusterd
  3:     option rpc-auth.auth-glusterfs on
  4:     option rpc-auth.auth-unix on
  5:     option rpc-auth.auth-null on
  6:     option rpc-auth-allow-insecure on
  7:     option transport.listen-backlog 10
  8:     option event-threads 1
  9:     option ping-timeout 0
 10:     option transport.socket.read-fail-log off
 11:     option transport.socket.keepalive-interval 2
 12:     option transport.socket.keepalive-time 10
 13:     option transport-type rdma
 14:     option working-directory /var/lib/glusterd
 15: end-volume
 16:  
+------------------------------------------------------------------------------+
[2018-07-13 18:54:21.587843] I [MSGID: 101190] [event-epoll.c:617:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2018-07-13 18:54:21.587894] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.182:24007 failed (Connection refused); disconnecting socket
[2018-07-13 18:54:21.587924] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-002> (<bdbe4622-25f9-4ef1-aad1-639ca52fc7e0>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 18:54:21.588066] I [glusterd-utils.c:5985:glusterd_brick_start] 0-management: discovered already-running brick /urd-gds/gluster
[2018-07-13 18:54:21.588080] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-13 18:54:21.588110] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2018-07-13 18:54:21.588236] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.183:24007 failed (Connection refused); disconnecting socket
[2018-07-13 18:54:21.588252] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-003> (<2a48a3b9-efa0-4fb7-837f-c800f04bf99f>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 18:54:21.588313] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.184:24007 failed (Connection refused); disconnecting socket
[2018-07-13 18:54:21.588327] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-004> (<ad893466-ad09-47f4-8bb4-4cea84085e5b>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 18:54:21.588381] E [socket.c:2517:socket_connect_finish] 0-management: connection to 192.168.67.180:24007 failed (Connection refused); disconnecting socket
[2018-07-13 18:54:21.588394] I [MSGID: 106004] [glusterd-handler.c:6382:__glusterd_peer_rpc_notify] 0-management: Peer <urd-gds-000> (<bfe05382-7e22-4b93-8816-b239b733b610>), in state <Peer in Cluster>, has disconnected from glusterd.
[2018-07-13 18:54:21.660473] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick snapd-urd-gds-volume on port 49153
[2018-07-13 18:54:21.768964] I [MSGID: 106142] [glusterd-pmap.c:297:pmap_registry_bind] 0-pmap: adding brick /urd-gds/gluster on port 49152
[2018-07-13 18:54:30.111616] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-13 18:54:47.283360] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-13 18:54:47.329973] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-13 18:54:48.948320] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-000 (0), ret: 0, op_ret: 0
[2018-07-13 18:54:48.960556] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-13 18:54:48.960578] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 18:54:51.309920] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610, host: urd-gds-000, port: 0
[2018-07-13 18:54:51.317451] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-13 18:54:51.317575] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 18:54:51.335376] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2018-07-13 18:54:51.335720] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs already stopped
[2018-07-13 18:54:51.335788] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: nfs service is stopped
[2018-07-13 18:54:51.335835] I [MSGID: 106599] [glusterd-nfs-svc.c:82:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2018-07-13 18:54:51.335928] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2018-07-13 18:54:51.338834] I [MSGID: 106568] [glusterd-proc-mgmt.c:87:glusterd_proc_stop] 0-management: Stopping glustershd daemon running in pid: 7885
[2018-07-13 18:54:52.339160] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: glustershd service is stopped
[2018-07-13 18:54:52.339329] I [MSGID: 106567] [glusterd-svc-mgmt.c:203:glusterd_svc_start] 0-management: Starting glustershd service
[2018-07-13 18:54:53.344990] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2018-07-13 18:54:53.345601] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: quotad already stopped
[2018-07-13 18:54:53.345666] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: quotad service is stopped
[2018-07-13 18:54:53.345759] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2018-07-13 18:54:53.346165] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: bitd already stopped
[2018-07-13 18:54:53.346205] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: bitd service is stopped
[2018-07-13 18:54:53.346286] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2018-07-13 18:54:53.346703] I [MSGID: 106131] [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: scrub already stopped
[2018-07-13 18:54:53.346752] I [MSGID: 106568] [glusterd-svc-mgmt.c:235:glusterd_svc_stop] 0-management: scrub service is stopped
[2018-07-13 18:54:53.432575] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 18:54:53.543149] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bfe05382-7e22-4b93-8816-b239b733b610
[2018-07-13 18:54:53.618909] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2018-07-13 18:54:53.619341] I [rpc-clnt.c:1059:rpc_clnt_connection_init] 0-gfproxyd: setting frame-timeout to 600
[2018-07-13 18:54:53.686298] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 18:55:27.800302] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-13 18:55:27.899812] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-13 18:55:28.284086] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-004 (0), ret: 0, op_ret: 0
[2018-07-13 18:55:28.295935] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-13 18:55:28.295960] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 18:55:30.726454] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b, host: urd-gds-004, port: 0
[2018-07-13 18:55:30.794875] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 18:55:30.907308] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-13 18:55:30.907418] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 18:55:30.922669] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: ad893466-ad09-47f4-8bb4-4cea84085e5b
[2018-07-13 18:55:31.048144] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 18:55:56.732289] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-13 18:55:56.834760] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-13 18:55:58.413114] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-002 (0), ret: 0, op_ret: 0
[2018-07-13 18:55:58.451294] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 18:55:58.715328] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 18:55:59.146564] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-13 18:55:59.146750] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 18:55:59.548547] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0, host: urd-gds-002, port: 0
[2018-07-13 18:55:59.554264] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-13 18:55:59.554356] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 18:55:59.619955] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: bdbe4622-25f9-4ef1-aad1-639ca52fc7e0
[2018-07-13 18:56:22.036735] I [MSGID: 106163] [glusterd-handshake.c:1356:__glusterd_mgmt_hndsk_versions_ack] 0-management: using the op-version 31202
[2018-07-13 18:56:22.171091] I [MSGID: 106490] [glusterd-handler.c:2548:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-13 18:56:22.481445] I [MSGID: 106493] [glusterd-handler.c:3811:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to urd-gds-003 (0), ret: 0, op_ret: 0
[2018-07-13 18:56:22.530023] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 18:56:22.828754] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 18:56:24.631125] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-13 18:56:24.631437] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 18:56:25.208162] I [MSGID: 106493] [glusterd-rpc-ops.c:486:__glusterd_friend_add_cbk] 0-glusterd: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f, host: urd-gds-003, port: 0
[2018-07-13 18:56:25.213784] I [MSGID: 106492] [glusterd-handler.c:2726:__glusterd_handle_friend_update] 0-glusterd: Received friend update from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-13 18:56:25.213874] I [MSGID: 106502] [glusterd-handler.c:2771:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend
[2018-07-13 18:56:25.339457] I [MSGID: 106493] [glusterd-rpc-ops.c:702:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: 2a48a3b9-efa0-4fb7-837f-c800f04bf99f
[2018-07-13 18:56:38.383468] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-13 19:05:31.380914] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-13 19:05:31.380941] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-13 19:05:31.381155] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-13 19:05:31.445256] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 19:05:40.385115] I [MSGID: 106327] [glusterd-geo-rep.c:2560:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 19:05:40.883914] I [MSGID: 106488] [glusterd-handler.c:1549:__glusterd_handle_cli_get_volume] 0-management: Received get vol req
[2018-07-13 19:05:44.701039] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-13 19:05:44.701041] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-13 19:05:44.701070] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-13 19:05:44.765234] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).
[2018-07-13 19:09:44.895016] I [MSGID: 106499] [glusterd-handler.c:4314:__glusterd_handle_status_volume] 0-management: Received status volume req for volume urd-gds-volume
[2018-07-13 19:09:57.270942] I [MSGID: 106061] [glusterd-geo-rep.c:298:__glusterd_handle_gsync_set] 0-management: master not found, while handling geo-replication options
[2018-07-13 19:09:57.270956] I [MSGID: 106061] [glusterd-geo-rep.c:305:__glusterd_handle_gsync_set] 0-management: slave not found, while handling geo-replication options
[2018-07-13 19:09:57.271000] W [MSGID: 106061] [glusterd-geo-rep.c:2300:glusterd_op_gsync_args_get] 0-management: master not found
[2018-07-13 19:09:57.331172] I [MSGID: 106327] [glusterd-geo-rep.c:4477:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf).

[2018-07-11 18:46:00.972596] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:01.9232] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:01.45558] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:01.307204] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:02.674809] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 18:46:02.675663] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 18:46:02.675908] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 18:46:10.329603] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:10.367683] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:10.406622] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:10.472903] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:10.918355] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 18:46:10.919222] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 18:46:10.919486] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 18:46:19.419262] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:19.461751] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:19.500921] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:19.568293] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:20.67095] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 18:46:20.67421] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 18:46:20.67506] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 18:46:28.870708] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:28.907105] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:28.943600] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:29.7987] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:46:29.425755] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 18:46:29.426619] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 18:46:29.426865] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 18:52:22.374429] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:22.411303] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:22.448626] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:22.682029] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:24.60415] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 18:52:24.61280] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 18:52:24.61553] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 18:52:24.946394] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:24.983776] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:25.21370] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:25.85717] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:25.552233] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 18:52:25.553082] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 18:52:25.553325] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 18:52:33.214688] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:33.252418] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:33.289689] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:33.353214] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:33.821991] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 18:52:33.822839] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 18:52:33.823082] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 18:52:37.19570] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:37.56063] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:37.93385] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:37.161221] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 18:52:37.669277] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 18:52:37.670315] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 18:52:37.670576] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 19:24:01.593698] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:24:01.661168] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:24:28.546472] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:24:28.582575] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:25:06.100366] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:25:06.167346] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:26:32.880868] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:26:32.948668] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:26:33.251960] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:26:33.314723] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:26:33.739454] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 19:26:33.740308] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 19:26:33.740586] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 19:26:47.851152] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:26:47.918591] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:27:03.331748] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:27:03.399030] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:27:14.521934] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:27:14.557883] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:27:14.815604] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:32:25.92350] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:32:25.159305] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:36:31.774912] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:36:31.842792] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:39:01.149439] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:39:01.184963] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:39:01.220755] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:39:01.426500] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:39:02.814904] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 19:39:02.815773] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 19:39:02.816026] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 19:40:04.105716] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:40:04.141788] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:40:04.177258] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:40:04.238610] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:40:04.664236] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 19:40:04.665097] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 19:40:04.665339] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 19:41:22.494477] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:41:22.530086] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:41:22.565391] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:41:22.627418] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:41:23.45649] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 19:41:23.46489] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 19:41:23.46761] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 19:42:31.778270] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:42:31.814096] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:42:31.849760] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:42:31.916712] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:42:32.407974] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 19:42:32.408850] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 19:42:32.409096] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 19:43:05.535235] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:05.604176] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:23.134586] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:23.169622] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:31.32541] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:31.99670] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:39.124305] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:39.191248] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:39.512083] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:39.574281] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:40.3715] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 19:43:40.4632] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 19:43:40.4988] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 19:43:43.369250] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:43.435810] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:47.901721] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:43:47.972584] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:44:16.244288] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:44:16.311671] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:47:15.55670] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:47:15.122472] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:47:27.785249] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:47:27.853568] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:47:28.160084] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:47:28.222697] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:47:28.648097] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 19:47:28.648945] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 19:47:28.649194] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 19:47:33.785439] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:47:33.851994] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:49:32.694803] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:49:32.768234] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:49:43.139094] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:49:43.206923] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:49:43.451411] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:49:43.513808] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:49:43.970031] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-11 19:49:43.971117] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-11 19:49:43.971369] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-11 19:49:57.503648] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:49:57.570086] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:57:52.648350] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 19:57:52.715713] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 20:08:36.340102] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 20:08:36.409012] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 20:08:43.446558] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 20:08:43.514060] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 20:28:00.120551] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 20:28:00.187484] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 23:00:01.337191] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 23:00:01.372724] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-11 23:00:01.667366] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:16:03.686954] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:16:03.737035] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:16:03.776866] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:16:03.847513] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:16:04.276067] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 11:16:04.276525] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 11:16:04.276669] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 11:18:09.830792] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:18:09.866314] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:18:09.902139] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:18:09.964318] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:18:10.624002] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 11:18:10.624917] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 11:18:10.625171] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 11:19:52.587980] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:19:52.625864] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:19:52.663651] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:19:52.730304] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:19:53.101191] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 11:19:53.102066] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 11:19:53.102313] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 11:21:40.768486] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:21:40.804541] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:21:40.841053] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:21:40.905890] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 11:21:41.259705] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 11:21:41.260322] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 11:21:41.260498] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 18:59:41.30365] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 18:59:41.67950] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 18:59:41.106128] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 18:59:41.180444] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 18:59:41.626018] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 18:59:41.626246] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 18:59:41.626302] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:00:15.503459] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:15.540575] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:15.576703] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:15.698096] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:15.729125] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:15.767126] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:15.803778] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:15.870580] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:15.906827] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:15.942703] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:16.31939] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:16.68100] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:16.104348] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:00:17.12828] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:00:17.13695] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:00:17.13956] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:01:02.786992] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:01:02.824158] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:01:02.861016] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:01:02.925090] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:01:03.324991] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:01:03.325731] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:01:03.325862] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:01:41.233608] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:01:41.269445] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:01:41.305212] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:01:41.374399] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:01:41.816917] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:01:41.817774] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:01:41.818026] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:02:16.382872] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:02:16.419821] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:02:16.456195] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:02:16.520958] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:02:16.935372] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:02:16.936239] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:02:16.936494] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:06:43.767630] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:06:43.835450] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:07:07.846982] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:07:07.919399] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:07:08.238946] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:07:08.301964] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:07:08.726736] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:07:08.727579] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:07:08.727849] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:07:12.83807] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:07:12.153878] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:07:31.555654] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:07:31.623157] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:09:31.94974] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:09:31.166317] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:16:43.791282] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:16:43.863884] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:16:48.879342] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:16:48.951347] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:16:49.264714] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:16:49.327752] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:16:49.783997] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:16:49.784878] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:16:49.785151] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:17:00.252207] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:17:00.321073] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:24:14.625897] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:24:14.697271] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:05.191579] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:05.230957] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:05.268224] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:05.367969] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:06.805160] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:27:06.805418] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:27:06.805488] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:27:17.327729] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:17.365370] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:17.403058] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:17.469084] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:17.967286] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:27:17.968156] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:27:17.968573] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:27:19.137671] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:19.175390] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:19.213114] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:19.278481] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:27:19.740205] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:27:19.741061] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:27:19.741365] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:28:13.131153] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:28:13.167129] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:28:13.203408] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:28:13.267200] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:28:13.688924] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:28:13.689774] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:28:13.690021] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:32:11.639394] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:32:11.675209] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:32:11.710990] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:32:11.812243] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:32:12.236826] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:32:12.237684] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:32:12.237931] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:33:00.83970] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:33:00.151719] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:34:38.343343] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:34:38.410737] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:34:56.381022] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:34:56.448412] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:34:56.784137] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:34:56.846595] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:34:57.276308] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:34:57.277175] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:34:57.277422] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:35:07.762607] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:35:07.830307] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:35:11.69685] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:35:11.138045] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:39:20.888933] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:39:20.926026] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:44:00.768810] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:44:00.836147] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:45:45.351203] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:45:45.419083] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:45:45.725073] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:45:45.787451] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:45:46.228259] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-12 19:45:46.229346] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-12 19:45:46.229609] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-12 19:45:50.314266] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:45:50.392043] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:45:53.320902] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:45:53.388271] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:56:59.111072] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:56:59.177916] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:57:05.751108] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 19:57:05.818790] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 20:00:34.118869] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 20:00:34.187098] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 23:00:01.531296] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 23:00:01.567812] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-12 23:00:01.840815] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:40:47.732731] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:40:47.768071] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:40:47.803451] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:40:47.869971] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:40:48.300822] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-13 05:40:48.301686] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-13 05:40:48.301968] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-13 05:41:34.930653] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:41:34.966211] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:41:35.1544] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:41:35.68032] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:41:35.495328] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-13 05:41:35.496197] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-13 05:41:35.496443] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-13 05:42:31.939950] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:42:31.975522] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:42:32.11543] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:42:32.74285] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:42:32.498414] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-13 05:42:32.499285] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-13 05:42:32.499545] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-13 05:43:26.629122] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:43:26.664913] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:43:26.700426] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:43:26.765167] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 05:43:27.183115] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-13 05:43:27.183962] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-13 05:43:27.184216] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-13 18:41:29.391355] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:41:29.461147] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:47:23.795861] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:47:23.863809] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:47:34.974653] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:47:35.44632] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:47:35.364774] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:47:35.428500] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:47:35.866710] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-13 18:47:35.867942] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-13 18:47:35.868225] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-13 18:47:40.735440] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:47:40.811265] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:53:18.570219] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:53:18.637584] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:53:31.423150] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:53:31.460381] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:53:31.752136] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:54:53.466783] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:54:53.502753] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:54:53.540704] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:54:53.608721] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:54:54.28263] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-13 18:54:54.29129] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-13 18:54:54.29376] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-13 18:55:30.830063] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:55:30.867276] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:55:30.904489] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:55:30.970408] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:55:31.459503] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-13 18:55:31.460391] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-13 18:55:31.460679] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-13 18:55:58.484771] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:55:58.520380] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:55:58.556072] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:55:58.618750] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:55:59.143769] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-13 18:55:59.144623] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-13 18:55:59.144879] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-13 18:56:22.564383] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:56:22.600275] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:56:22.636097] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:56:22.698795] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 18:56:23.187536] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-13 18:56:23.188386] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-13 18:56:23.188694] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-13 19:05:31.480751] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 19:05:31.551481] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 19:05:40.419915] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 19:05:40.489955] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 19:05:40.736675] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 19:05:40.804028] I [gsyncd(monitor):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 19:05:41.234556] E [syncdutils(monitor):749:errlog] Popen: command returned error	cmd=ssh -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem -p 22 geouser@urd-gds-geo-001 gluster --xml --remote-host=localhost volume info urd-gds-volume	error=255
[2018-07-13 19:05:41.234812] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to create logfile "/var/log/glusterfs/cli.log" (Permission denied)
[2018-07-13 19:05:41.234894] E [syncdutils(monitor):753:logerr] Popen: ssh> ERROR: failed to open logfile /var/log/glusterfs/cli.log
[2018-07-13 19:05:44.800524] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 19:05:44.870376] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 19:09:57.364545] I [gsyncd(config-get):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf
[2018-07-13 19:09:57.431981] I [gsyncd(status):297:main] <top>: Using session config file	path=/var/lib/glusterd/geo-replication/urd-gds-volume_urd-gds-geo-001_urd-gds-volume/gsyncd.conf

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users