Re: Gluster 3.8.10 rebalance VMs corruption

Mahdi Adnan <mahdi.adnan@xxxxxxxxxxx> · Sun, 19 Mar 2017 09:06:46 +0000

Thanks mate,
Kindly, check the attachment.

-- 

Respectfully

Mahdi A. Mahdi

From: Krutika Dhananjay <kdhananj@xxxxxxxxxx>

Sent: Sunday, March 19, 2017 10:00:22 AM

To: Mahdi Adnan

Cc: gluster-users@xxxxxxxxxxx

Subject: Re:  Gluster 3.8.10 rebalance VMs corruption

In that case could you share the ganesha-gfapi logs?

-Krutika

On Sun, Mar 19, 2017 at 12:13 PM, Mahdi Adnan 
<mahdi.adnan@xxxxxxxxxxx> wrote:

I have two volumes, one is mounted using libgfapi for ovirt mount, the other one is exported via NFS-Ganesha for VMWare which is the one im testing now.

-- 

Respectfully

Mahdi A. Mahdi

From: Krutika Dhananjay <kdhananj@xxxxxxxxxx>

Sent: Sunday, March 19, 2017 8:02:19 AM

To: Mahdi Adnan

Cc: gluster-users@xxxxxxxxxxx

Subject: Re:  Gluster 3.8.10 rebalance VMs corruption

On Sat, Mar 18, 2017 at 10:36 PM, Mahdi Adnan 
<mahdi.adnan@xxxxxxxxxxx> wrote:

Kindly, check the attached new log file, i dont know if it's helpful or not but, i couldn't find the log with the name you just described.

No. Are you using FUSE or libgfapi for accessing the volume? Or is it NFS?

-Krutika

-- 

Respectfully

Mahdi A. Mahdi

From: Krutika Dhananjay <kdhananj@xxxxxxxxxx>

Sent: Saturday, March 18, 2017 6:10:40 PM

To: Mahdi Adnan

Cc: gluster-users@xxxxxxxxxxx

Subject: Re:  Gluster 3.8.10 rebalance VMs corruption

mnt-disk11-vmware2.log seems like a brick log. Could you attach the fuse mount logs? It should be right under /var/log/glusterfs/ directory

named after the mount point name, only hyphenated.

-Krutika

On Sat, Mar 18, 2017 at 7:27 PM, Mahdi Adnan 
<mahdi.adnan@xxxxxxxxxxx> wrote:

Hello Krutika,

Kindly, check the attached logs.

-- 

Respectfully

Mahdi A. Mahdi

From: Krutika Dhananjay <kdhananj@xxxxxxxxxx>

Sent: Saturday, March 18, 2017 3:29:03 PM

To: Mahdi Adnan

Cc: gluster-users@xxxxxxxxxxx

Subject: Re:  Gluster 3.8.10 rebalance VMs corruption

Hi Mahdi,

Could you attach mount, brick and rebalance logs?

-Krutika

On Sat, Mar 18, 2017 at 12:14 AM, Mahdi Adnan 
<mahdi.adnan@xxxxxxxxxxx> wrote:

Hi,

I have upgraded to Gluster 3.8.10 today and ran the add-brick procedure in a volume contains few VMs.

After the completion of rebalance, i have rebooted the VMs, some of ran just fine, and others just crashed.

Windows boot to recovery mode and Linux throw xfs errors and does not boot.

I ran the test again and it happened just as the first one, but i have noticed only VMs doing disk IOs are affected by this bug.

The VMs in power off mode started fine and even md5 of the disk file did not change after the rebalance.

anyone else can confirm this ?

Volume info:

Volume Name: vmware2
Type: Distributed-Replicate
Volume ID: 02328d46-a285-4533-aa3a-fb9bfeb688bf
Status: Started
Snapshot Count: 0
Number of Bricks: 22 x 2 = 44
Transport-type: tcp
Bricks:
Brick1: gluster01:/mnt/disk1/vmware2
Brick2: gluster03:/mnt/disk1/vmware2
Brick3: gluster02:/mnt/disk1/vmware2
Brick4: gluster04:/mnt/disk1/vmware2
Brick5: gluster01:/mnt/disk2/vmware2
Brick6: gluster03:/mnt/disk2/vmware2
Brick7: gluster02:/mnt/disk2/vmware2
Brick8: gluster04:/mnt/disk2/vmware2
Brick9: gluster01:/mnt/disk3/vmware2
Brick10: gluster03:/mnt/disk3/vmware2
Brick11: gluster02:/mnt/disk3/vmware2
Brick12: gluster04:/mnt/disk3/vmware2
Brick13: gluster01:/mnt/disk4/vmware2
Brick14: gluster03:/mnt/disk4/vmware2
Brick15: gluster02:/mnt/disk4/vmware2
Brick16: gluster04:/mnt/disk4/vmware2
Brick17: gluster01:/mnt/disk5/vmware2
Brick18: gluster03:/mnt/disk5/vmware2
Brick19: gluster02:/mnt/disk5/vmware2
Brick20: gluster04:/mnt/disk5/vmware2
Brick21: gluster01:/mnt/disk6/vmware2
Brick22: gluster03:/mnt/disk6/vmware2
Brick23: gluster02:/mnt/disk6/vmware2
Brick24: gluster04:/mnt/disk6/vmware2
Brick25: gluster01:/mnt/disk7/vmware2
Brick26: gluster03:/mnt/disk7/vmware2
Brick27: gluster02:/mnt/disk7/vmware2
Brick28: gluster04:/mnt/disk7/vmware2
Brick29: gluster01:/mnt/disk8/vmware2
Brick30: gluster03:/mnt/disk8/vmware2
Brick31: gluster02:/mnt/disk8/vmware2
Brick32: gluster04:/mnt/disk8/vmware2
Brick33: gluster01:/mnt/disk9/vmware2
Brick34: gluster03:/mnt/disk9/vmware2
Brick35: gluster02:/mnt/disk9/vmware2
Brick36: gluster04:/mnt/disk9/vmware2
Brick37: gluster01:/mnt/disk10/vmware2
Brick38: gluster03:/mnt/disk10/vmware2
Brick39: gluster02:/mnt/disk10/vmware2
Brick40: gluster04:/mnt/disk10/vmware2
Brick41: gluster01:/mnt/disk11/vmware2
Brick42: gluster03:/mnt/disk11/vmware2
Brick43: gluster02:/mnt/disk11/vmware2
Brick44: gluster04:/mnt/disk11/vmware2
Options Reconfigured:
cluster.server-quorum-type: server
nfs.disable: on
performance.readdir-ahead: on
transport.address-family: inet
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.stat-prefetch: off
cluster.eager-lock: enable
network.remote-dio: enable
features.shard: on
cluster.data-self-heal-algorithm: full
features.cache-invalidation: on
ganesha.enable: on
features.shard-block-size: 256MB
client.event-threads: 2
server.event-threads: 2
cluster.favorite-child-policy: size
storage.build-pgfid: off
network.ping-timeout: 5
cluster.enable-shared-storage: enable
nfs-ganesha: enable
cluster.server-quorum-ratio: 51%

Adding bricks:
gluster volume add-brick vmware2 replica 2 gluster01:/mnt/disk11/vmware2 gluster03:/mnt/disk11/vmware2 gluster02:/mnt/disk11/vmware2 gluster04:/mnt/disk11/vmware2

starting fix layout:
gluster volume rebalance vmware2 fix-layout start

Starting rebalance:
gluster volume rebalance vmware2  start

-- 

Respectfully

Mahdi A. Mahdi

_______________________________________________

Gluster-users mailing list

Gluster-users@xxxxxxxxxxx

http://lists.gluster.org/mailman/listinfo/gluster-users

[2017-03-17 12:53:01.389453] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 10-vmware2-client-34: changing port to 49422 (from 0)
[2017-03-17 12:53:01.389474] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 10-vmware2-client-38: changing port to 49423 (from 0)
[2017-03-17 12:53:01.389493] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-21: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.389557] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-17: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.389619] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-25: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.389668] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-13: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.389717] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-29: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.389778] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-33: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.404732] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-15: Server lk version = 1
[2017-03-17 12:53:01.404831] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-19: Connected to vmware2-client-19, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 12:53:01.404851] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-19: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.404899] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 10-vmware2-replicate-9: Subvolume 'vmware2-client-19' came back up; going online.
[2017-03-17 12:53:01.411085] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-23: Connected to vmware2-client-23, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 12:53:01.411113] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-23: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.411163] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 10-vmware2-replicate-11: Subvolume 'vmware2-client-23' came back up; going online.
[2017-03-17 12:53:01.411201] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-27: Connected to vmware2-client-27, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 12:53:01.411212] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-27: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.411247] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 10-vmware2-replicate-13: Subvolume 'vmware2-client-27' came back up; going online.
[2017-03-17 12:53:01.413686] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-31: Connected to vmware2-client-31, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 12:53:01.413709] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-31: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.413750] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 10-vmware2-replicate-15: Subvolume 'vmware2-client-31' came back up; going online.
[2017-03-17 12:53:01.419794] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-35: Connected to vmware2-client-35, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 12:53:01.419816] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-35: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.419856] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 10-vmware2-replicate-17: Subvolume 'vmware2-client-35' came back up; going online.
[2017-03-17 12:53:01.419888] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-39: Connected to vmware2-client-39, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 12:53:01.419897] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-39: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.419924] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 10-vmware2-replicate-19: Subvolume 'vmware2-client-39' came back up; going online.
[2017-03-17 12:53:01.419974] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-1: Connected to vmware2-client-1, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 12:53:01.419983] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-1: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.420099] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-5: Connected to vmware2-client-5, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 12:53:01.420118] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-5: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.420208] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-9: Connected to vmware2-client-9, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 12:53:01.420230] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-9: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.420313] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-21: Connected to vmware2-client-21, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 12:53:01.420324] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-21: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.420406] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-25: Connected to vmware2-client-25, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 12:53:01.420416] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-25: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.420497] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-17: Connected to vmware2-client-17, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 12:53:01.420522] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-17: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.420625] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-13: Connected to vmware2-client-13, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 12:53:01.420638] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-13: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.420729] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-29: Connected to vmware2-client-29, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 12:53:01.420741] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-29: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.420838] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-33: Connected to vmware2-client-33, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 12:53:01.420854] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-33: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.421036] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-19: Server lk version = 1
[2017-03-17 12:53:01.421099] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-23: Server lk version = 1
[2017-03-17 12:53:01.421124] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-27: Server lk version = 1
[2017-03-17 12:53:01.421155] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-31: Server lk version = 1
[2017-03-17 12:53:01.421200] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-35: Server lk version = 1
[2017-03-17 12:53:01.421215] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-39: Server lk version = 1
[2017-03-17 12:53:01.421226] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-1: Server lk version = 1
[2017-03-17 12:53:01.421238] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-5: Server lk version = 1
[2017-03-17 12:53:01.421259] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-9: Server lk version = 1
[2017-03-17 12:53:01.421290] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-21: Server lk version = 1
[2017-03-17 12:53:01.421303] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-25: Server lk version = 1
[2017-03-17 12:53:01.421333] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-17: Server lk version = 1
[2017-03-17 12:53:01.421345] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-13: Server lk version = 1
[2017-03-17 12:53:01.421368] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-29: Server lk version = 1
[2017-03-17 12:53:01.421379] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-33: Server lk version = 1
[2017-03-17 12:53:01.421394] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-37: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.421453] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-6: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.421497] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.421538] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-14: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.421592] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-10: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.421640] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-22: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.421680] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-18: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.421755] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-26: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.421813] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-30: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.421864] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-34: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.421998] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-37: Connected to vmware2-client-37, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 12:53:01.422011] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-37: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.422172] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-6: Connected to vmware2-client-6, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 12:53:01.422184] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-6: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.422256] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-2: Connected to vmware2-client-2, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 12:53:01.422266] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.422349] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-14: Connected to vmware2-client-14, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 12:53:01.422369] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-14: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.422455] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-10: Connected to vmware2-client-10, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 12:53:01.422467] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-10: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.422551] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-18: Connected to vmware2-client-18, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 12:53:01.422561] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-18: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.422645] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-22: Connected to vmware2-client-22, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 12:53:01.422656] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-22: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.422714] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-37: Server lk version = 1
[2017-03-17 12:53:01.422737] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-26: Connected to vmware2-client-26, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 12:53:01.422744] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-26: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.422837] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-30: Connected to vmware2-client-30, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 12:53:01.422853] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-30: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.422933] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-34: Connected to vmware2-client-34, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 12:53:01.422944] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-34: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.423003] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-6: Server lk version = 1
[2017-03-17 12:53:01.423035] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-2: Server lk version = 1
[2017-03-17 12:53:01.423049] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-14: Server lk version = 1
[2017-03-17 12:53:01.423070] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-10: Server lk version = 1
[2017-03-17 12:53:01.423087] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-18: Server lk version = 1
[2017-03-17 12:53:01.423119] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-22: Server lk version = 1
[2017-03-17 12:53:01.423138] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-26: Server lk version = 1
[2017-03-17 12:53:01.423154] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-30: Server lk version = 1
[2017-03-17 12:53:01.423194] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-34: Server lk version = 1
[2017-03-17 12:53:01.423694] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 10-vmware2-client-38: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 12:53:01.424183] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 10-vmware2-client-38: Connected to vmware2-client-38, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 12:53:01.424208] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 10-vmware2-client-38: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 12:53:01.440281] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 10-vmware2-client-38: Server lk version = 1
[2017-03-17 12:53:01.441714] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 12:53:01.444290] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 12:53:01.444340] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 12:53:01.444674] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 12:53:01.445040] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 12:53:01.445086] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 12:53:01.445120] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 12:53:01.445250] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 12:53:01.445303] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 12:53:01.445379] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 12:53:01.472424] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.472508] W [MSGID: 104033] [glfs-resolve.c:654:glfs_migrate_fd_safe] 0-vmware2: fsync() failed (Transport endpoint is not connected) on 32671af8-f2f3-4949-bdd7-fe113f7b2836 graph 676c7573-7465-7230-312d-313938392d32 (7) [Transport endpoint is not connected]
[2017-03-17 12:53:01.476202] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-12: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4_3-flat.vmdk (877544d3-d928-4888-a9f0-ec9ec307affb) [No such file or directory]
[2017-03-17 12:53:01.476306] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-13: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4_3-flat.vmdk (877544d3-d928-4888-a9f0-ec9ec307affb) [No such file or directory]
[2017-03-17 12:53:01.477727] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-12: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4_3-flat.vmdk (877544d3-d928-4888-a9f0-ec9ec307affb) [No such file or directory]
[2017-03-17 12:53:01.477754] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-13: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4_3-flat.vmdk (877544d3-d928-4888-a9f0-ec9ec307affb) [No such file or directory]
[2017-03-17 12:53:01.478965] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.479036] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.480292] I [MSGID: 114024] [client-helpers.c:99:this_fd_set_ctx] 7-vmware2-client-15: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4_3-flat.vmdk (877544d3-d928-4888-a9f0-ec9ec307affb): trying duplicate remote fd set. 
[2017-03-17 12:53:01.480310] I [MSGID: 114024] [client-helpers.c:99:this_fd_set_ctx] 7-vmware2-client-14: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4_3-flat.vmdk (877544d3-d928-4888-a9f0-ec9ec307affb): trying duplicate remote fd set. 
[2017-03-17 12:53:01.481798] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-13: remote operation failed. Path: <gfid:877544d3-d928-4888-a9f0-ec9ec307affb> (877544d3-d928-4888-a9f0-ec9ec307affb) [No such file or directory]
[2017-03-17 12:53:01.481844] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-12: remote operation failed. Path: <gfid:877544d3-d928-4888-a9f0-ec9ec307affb> (877544d3-d928-4888-a9f0-ec9ec307affb) [No such file or directory]
[2017-03-17 12:53:01.485261] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.485321] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.485518] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-35: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/vmware-4.log (ddca6658-45e5-4508-820f-bea48759c62d) [No such file or directory]
[2017-03-17 12:53:01.485569] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-34: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/vmware-4.log (ddca6658-45e5-4508-820f-bea48759c62d) [No such file or directory]
[2017-03-17 12:53:01.488516] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-35: remote operation failed. Path: <gfid:ddca6658-45e5-4508-820f-bea48759c62d> (ddca6658-45e5-4508-820f-bea48759c62d) [No such file or directory]
[2017-03-17 12:53:01.488572] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-34: remote operation failed. Path: <gfid:ddca6658-45e5-4508-820f-bea48759c62d> (ddca6658-45e5-4508-820f-bea48759c62d) [No such file or directory]
[2017-03-17 12:53:01.492369] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.492411] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.492522] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-33: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/vmware-5.log (16348bbe-b135-466a-8a9e-b88ecaf2111d) [No such file or directory]
[2017-03-17 12:53:01.492586] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-32: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/vmware-5.log (16348bbe-b135-466a-8a9e-b88ecaf2111d) [No such file or directory]
[2017-03-17 12:53:01.495377] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-32: remote operation failed. Path: <gfid:16348bbe-b135-466a-8a9e-b88ecaf2111d> (16348bbe-b135-466a-8a9e-b88ecaf2111d) [No such file or directory]
[2017-03-17 12:53:01.495437] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-33: remote operation failed. Path: <gfid:16348bbe-b135-466a-8a9e-b88ecaf2111d> (16348bbe-b135-466a-8a9e-b88ecaf2111d) [No such file or directory]
[2017-03-17 12:53:01.503460] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-17: remote operation failed. Path: <gfid:ff370fe6-19d1-4ced-9818-9870ab6d1e19> (ff370fe6-19d1-4ced-9818-9870ab6d1e19) [No such file or directory]
[2017-03-17 12:53:01.503598] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-16: remote operation failed. Path: <gfid:ff370fe6-19d1-4ced-9818-9870ab6d1e19> (ff370fe6-19d1-4ced-9818-9870ab6d1e19) [No such file or directory]
[2017-03-17 12:53:01.514177] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-32: remote operation failed. Path: <gfid:75e7b3d5-ef69-46b7-90ef-4fead132fefe> (75e7b3d5-ef69-46b7-90ef-4fead132fefe) [No such file or directory]
[2017-03-17 12:53:01.514179] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-33: remote operation failed. Path: <gfid:75e7b3d5-ef69-46b7-90ef-4fead132fefe> (75e7b3d5-ef69-46b7-90ef-4fead132fefe) [No such file or directory]
[2017-03-17 12:53:01.516775] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-19: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4.vmx (84739892-a549-475a-9ffc-72b27f28e5fd) [No such file or directory]
[2017-03-17 12:53:01.516837] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-18: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4.vmx (84739892-a549-475a-9ffc-72b27f28e5fd) [No such file or directory]
[2017-03-17 12:53:01.517204] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.517248] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.519342] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-18: remote operation failed. Path: <gfid:84739892-a549-475a-9ffc-72b27f28e5fd> (84739892-a549-475a-9ffc-72b27f28e5fd) [No such file or directory]
[2017-03-17 12:53:01.519412] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-19: remote operation failed. Path: <gfid:84739892-a549-475a-9ffc-72b27f28e5fd> (84739892-a549-475a-9ffc-72b27f28e5fd) [No such file or directory]
[2017-03-17 12:53:01.522508] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-22: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4.vmdk (adc57041-620c-45bd-a7b3-d55df1e0f699) [No such file or directory]
[2017-03-17 12:53:01.522537] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-23: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4.vmdk (adc57041-620c-45bd-a7b3-d55df1e0f699) [No such file or directory]
[2017-03-17 12:53:01.522790] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.522854] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.525283] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-22: remote operation failed. Path: <gfid:adc57041-620c-45bd-a7b3-d55df1e0f699> (adc57041-620c-45bd-a7b3-d55df1e0f699) [No such file or directory]
[2017-03-17 12:53:01.525333] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-23: remote operation failed. Path: <gfid:adc57041-620c-45bd-a7b3-d55df1e0f699> (adc57041-620c-45bd-a7b3-d55df1e0f699) [No such file or directory]
[2017-03-17 12:53:01.528340] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.528380] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.528493] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-32: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/vmware.log (f7fc20f0-9426-4647-a065-d62d62a41b0a) [No such file or directory]
[2017-03-17 12:53:01.528635] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-33: remote operation failed. Path: <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/vmware.log (f7fc20f0-9426-4647-a065-d62d62a41b0a) [No such file or directory]
[2017-03-17 12:53:01.531073] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-32: remote operation failed. Path: <gfid:f7fc20f0-9426-4647-a065-d62d62a41b0a> (f7fc20f0-9426-4647-a065-d62d62a41b0a) [No such file or directory]
[2017-03-17 12:53:01.531089] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-33: remote operation failed. Path: <gfid:f7fc20f0-9426-4647-a065-d62d62a41b0a> (f7fc20f0-9426-4647-a065-d62d62a41b0a) [No such file or directory]
[2017-03-17 12:53:01.538261] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-29: remote operation failed. Path: <gfid:62c424a1-9895-4cd9-8fc2-9e6fd07fa683> (62c424a1-9895-4cd9-8fc2-9e6fd07fa683) [No such file or directory]
[2017-03-17 12:53:01.538298] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-28: remote operation failed. Path: <gfid:62c424a1-9895-4cd9-8fc2-9e6fd07fa683> (62c424a1-9895-4cd9-8fc2-9e6fd07fa683) [No such file or directory]
[2017-03-17 12:53:01.540708] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:fcd748e8-cd38-45b4-80fc-637ce00f58c8> (fcd748e8-cd38-45b4-80fc-637ce00f58c8) [No such file or directory]
[2017-03-17 12:53:01.540780] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:fcd748e8-cd38-45b4-80fc-637ce00f58c8> (fcd748e8-cd38-45b4-80fc-637ce00f58c8) [No such file or directory]
[2017-03-17 12:53:01.552629] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-17: remote operation failed. Path: /New Virtual Machine/New Virtual Machine.vmdk (5eee0fae-4ebc-4985-9179-48e92e4dc655) [No such file or directory]
[2017-03-17 12:53:01.552673] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-16: remote operation failed. Path: /New Virtual Machine/New Virtual Machine.vmdk (5eee0fae-4ebc-4985-9179-48e92e4dc655) [No such file or directory]
[2017-03-17 12:53:01.552961] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.553006] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.554969] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-17: remote operation failed. Path: <gfid:5eee0fae-4ebc-4985-9179-48e92e4dc655> (5eee0fae-4ebc-4985-9179-48e92e4dc655) [No such file or directory]
[2017-03-17 12:53:01.555025] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-16: remote operation failed. Path: <gfid:5eee0fae-4ebc-4985-9179-48e92e4dc655> (5eee0fae-4ebc-4985-9179-48e92e4dc655) [No such file or directory]
[2017-03-17 12:53:01.557273] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-6: remote operation failed. Path: /Mahdi-Test20122/Mahdi-Test20122.vmdk (5b41b2e7-1c3a-4322-a84a-c33c1e2fe030) [No such file or directory]
[2017-03-17 12:53:01.557330] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-7: remote operation failed. Path: /Mahdi-Test20122/Mahdi-Test20122.vmdk (5b41b2e7-1c3a-4322-a84a-c33c1e2fe030) [No such file or directory]
[2017-03-17 12:53:01.558016] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.558063] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.559896] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-6: remote operation failed. Path: <gfid:5b41b2e7-1c3a-4322-a84a-c33c1e2fe030> (5b41b2e7-1c3a-4322-a84a-c33c1e2fe030) [No such file or directory]
[2017-03-17 12:53:01.559904] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-7: remote operation failed. Path: <gfid:5b41b2e7-1c3a-4322-a84a-c33c1e2fe030> (5b41b2e7-1c3a-4322-a84a-c33c1e2fe030) [No such file or directory]
[2017-03-17 12:53:01.564087] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.564145] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.564201] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-32: remote operation failed. Path: /New Virtual Machine/New Virtual Machine.vmx (b4251d1f-15b0-49bf-a05e-f10438744b55) [No such file or directory]
[2017-03-17 12:53:01.564218] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-33: remote operation failed. Path: /New Virtual Machine/New Virtual Machine.vmx (b4251d1f-15b0-49bf-a05e-f10438744b55) [No such file or directory]
[2017-03-17 12:53:01.567053] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-32: remote operation failed. Path: <gfid:b4251d1f-15b0-49bf-a05e-f10438744b55> (b4251d1f-15b0-49bf-a05e-f10438744b55) [No such file or directory]
[2017-03-17 12:53:01.567090] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-33: remote operation failed. Path: <gfid:b4251d1f-15b0-49bf-a05e-f10438744b55> (b4251d1f-15b0-49bf-a05e-f10438744b55) [No such file or directory]
[2017-03-17 12:53:01.568516] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-30: remote operation failed. Path: <gfid:30986c2a-8346-4b13-a47a-a827dcd74fe6> (30986c2a-8346-4b13-a47a-a827dcd74fe6) [No such file or directory]
[2017-03-17 12:53:01.568519] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-31: remote operation failed. Path: <gfid:30986c2a-8346-4b13-a47a-a827dcd74fe6> (30986c2a-8346-4b13-a47a-a827dcd74fe6) [No such file or directory]
[2017-03-17 12:53:01.570970] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-22: remote operation failed. Path: /Mahdi-Test20122/vmware.log (5df33cef-2e1b-4ffd-b05e-08d6679c0f91) [No such file or directory]
[2017-03-17 12:53:01.571034] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-23: remote operation failed. Path: /Mahdi-Test20122/vmware.log (5df33cef-2e1b-4ffd-b05e-08d6679c0f91) [No such file or directory]
[2017-03-17 12:53:01.571228] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.571261] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.573411] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-22: remote operation failed. Path: <gfid:5df33cef-2e1b-4ffd-b05e-08d6679c0f91> (5df33cef-2e1b-4ffd-b05e-08d6679c0f91) [No such file or directory]
[2017-03-17 12:53:01.573499] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-23: remote operation failed. Path: <gfid:5df33cef-2e1b-4ffd-b05e-08d6679c0f91> (5df33cef-2e1b-4ffd-b05e-08d6679c0f91) [No such file or directory]
[2017-03-17 12:53:01.575301] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.575347] W [MSGID: 104033] [glfs-resolve.c:654:glfs_migrate_fd_safe] 0-vmware2: fsync() failed (Transport endpoint is not connected) on 7fdc37c4-273b-45c7-9c4f-dd9dfd2e0902 graph 676c7573-7465-7230-312d-313938392d32 (7) [Transport endpoint is not connected]
[2017-03-17 12:53:01.578902] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-26: remote operation failed. Path: /Mahdi-Test20122/Mahdi-Test20122.vmx (b979edbc-bfb1-4340-984f-858499524e26) [No such file or directory]
[2017-03-17 12:53:01.578981] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 7-vmware2-client-27: remote operation failed. Path: /Mahdi-Test20122/Mahdi-Test20122.vmx (b979edbc-bfb1-4340-984f-858499524e26) [No such file or directory]
[2017-03-17 12:53:01.579033] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-20: no subvolumes up
[2017-03-17 12:53:01.579076] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 7-vmware2-replicate-21: no subvolumes up
[2017-03-17 12:53:01.581374] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-27: remote operation failed. Path: <gfid:b979edbc-bfb1-4340-984f-858499524e26> (b979edbc-bfb1-4340-984f-858499524e26) [No such file or directory]
[2017-03-17 12:53:01.581444] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-26: remote operation failed. Path: <gfid:b979edbc-bfb1-4340-984f-858499524e26> (b979edbc-bfb1-4340-984f-858499524e26) [No such file or directory]
[2017-03-17 12:53:01.585104] I [MSGID: 104041] [glfs-resolve.c:885:__glfs_active_subvol] 0-vmware2: switched to graph 676c7573-7465-7230-312d-313938392d32 (10)
[2017-03-17 12:53:01.585132] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-0: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585162] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-1: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585196] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-2: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585219] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-3: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585234] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-4: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585266] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-5: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585288] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-6: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585311] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-7: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585331] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-8: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585361] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-9: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585386] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-10: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585415] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-11: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585436] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-12: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585429] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 7-vmware2-client-0: forced unwinding frame type(GlusterFS 3.3) op(RELEASE(41)) called at 2017-03-17 12:53:01.585078 (xid=0xa556b0)
[2017-03-17 12:53:01.585466] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-0: disconnected from vmware2-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585473] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-13: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585508] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-14: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585522] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-2: disconnected from vmware2-client-2. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585544] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-15: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585519] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 7-vmware2-client-1: forced unwinding frame type(GlusterFS 3.3) op(RELEASE(41)) called at 2017-03-17 12:53:01.585092 (xid=0x8963c1)
[2017-03-17 12:53:01.585553] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-3: disconnected from vmware2-client-3. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585588] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-1: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.585592] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-16: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585594] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-1: disconnected from vmware2-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585619] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.585635] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-17: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585657] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-4: disconnected from vmware2-client-4. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585662] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-18: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585689] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-5: disconnected from vmware2-client-5. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585691] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-19: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585714] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-2: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.585734] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-20: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585741] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-6: disconnected from vmware2-client-6. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585762] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-7: disconnected from vmware2-client-7. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585792] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-3: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.585795] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-21: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585815] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-9: disconnected from vmware2-client-9. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585826] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-8: disconnected from vmware2-client-8. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585837] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-10: disconnected from vmware2-client-10. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585837] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-22: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585846] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-4: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.585886] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-23: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585909] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-11: disconnected from vmware2-client-11. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585910] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-24: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585928] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-5: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.585954] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-25: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585958] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-12: disconnected from vmware2-client-12. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585984] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-26: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.585994] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-14: disconnected from vmware2-client-14. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.585994] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-13: disconnected from vmware2-client-13. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586021] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-6: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586025] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-27: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586028] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-15: disconnected from vmware2-client-15. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586046] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-16: disconnected from vmware2-client-16. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586050] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-7: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586057] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-28: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586088] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-18: disconnected from vmware2-client-18. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586086] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-17: disconnected from vmware2-client-17. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586088] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-29: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586108] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-8: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586116] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-19: disconnected from vmware2-client-19. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586130] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-9: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586132] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-30: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586138] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-20: disconnected from vmware2-client-20. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586169] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-21: disconnected from vmware2-client-21. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586171] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-31: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586180] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-10: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586197] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-32: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586202] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-23: disconnected from vmware2-client-23. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586208] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-22: disconnected from vmware2-client-22. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586234] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-11: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586235] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-24: disconnected from vmware2-client-24. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586235] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-33: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586267] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-25: disconnected from vmware2-client-25. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586278] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-34: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586304] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-12: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586292] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-26: disconnected from vmware2-client-26. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586332] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-27: disconnected from vmware2-client-27. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586330] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-35: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586346] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-13: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586348] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-28: disconnected from vmware2-client-28. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586374] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-36: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586375] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-29: disconnected from vmware2-client-29. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586395] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-30: disconnected from vmware2-client-30. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586420] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-37: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586401] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-14: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586449] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-38: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586465] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-31: disconnected from vmware2-client-31. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586472] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-39: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586478] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-15: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586497] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-32: disconnected from vmware2-client-32. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586508] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-33: disconnected from vmware2-client-33. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586500] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-40: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586526] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-16: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586528] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-41: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586548] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-35: disconnected from vmware2-client-35. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586549] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-42: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586548] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-34: disconnected from vmware2-client-34. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586572] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-36: disconnected from vmware2-client-36. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586574] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-17: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586574] I [MSGID: 114021] [client.c:2365:notify] 7-vmware2-client-43: current graph is no longer active, destroying rpc_client 
[2017-03-17 12:53:01.586612] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-38: disconnected from vmware2-client-38. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586613] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-37: disconnected from vmware2-client-37. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586630] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-18: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:53:01.586633] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 7-vmware2-client-39: disconnected from vmware2-client-39. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 12:53:01.586643] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 7-vmware2-replicate-19: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 12:54:54.336357] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid 07b1b1c5-0ebd-45b1-9ba7-383d5d23920b. (Possible split-brain)
[2017-03-17 12:54:54.339136] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:07b1b1c5-0ebd-45b1-9ba7-383d5d23920b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:54:54.339217] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 07b1b1c5-0ebd-45b1-9ba7-383d5d23920b [Stale file handle]
[2017-03-17 12:54:54.408050] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid c2eb9000-ceb4-4d96-aa11-6342fad3a817. (Possible split-brain)
[2017-03-17 12:54:54.411299] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:c2eb9000-ceb4-4d96-aa11-6342fad3a817>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:54:54.411378] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: c2eb9000-ceb4-4d96-aa11-6342fad3a817 [Stale file handle]
[2017-03-17 12:54:54.426057] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid b4339058-be21-405c-8bed-02141e38b5e1. (Possible split-brain)
[2017-03-17 12:54:54.428668] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b4339058-be21-405c-8bed-02141e38b5e1>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:54:54.428726] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b4339058-be21-405c-8bed-02141e38b5e1 [Stale file handle]
[2017-03-17 12:54:54.432294] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid c45182c2-af42-4fe4-a1d1-cff77eb971f4. (Possible split-brain)
[2017-03-17 12:54:54.435041] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:c45182c2-af42-4fe4-a1d1-cff77eb971f4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:54:54.435104] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: c45182c2-af42-4fe4-a1d1-cff77eb971f4 [Stale file handle]
[2017-03-17 12:54:54.561090] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 12:54:54.561552] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 12:54:54.678055] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming <gfid:40a0aeba-5026-4b59-91a0-9706fa800150>/Mahdi-Test20122.vmx~ (hash=vmware2-replicate-13/cache=vmware2-replicate-13) => <gfid:40a0aeba-5026-4b59-91a0-9706fa800150>/Mahdi-Test20122.vmx (hash=vmware2-replicate-7/cache=vmware2-replicate-7)
[2017-03-17 12:54:54.686610] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 12:54:54.687094] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:b979edbc-bfb1-4340-984f-858499524e26> (b979edbc-bfb1-4340-984f-858499524e26) [No such file or directory]
[2017-03-17 12:54:54.687218] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:b979edbc-bfb1-4340-984f-858499524e26> (b979edbc-bfb1-4340-984f-858499524e26) [No such file or directory]
[2017-03-17 12:54:54.687733] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid b979edbc-bfb1-4340-984f-858499524e26. (Possible split-brain)
[2017-03-17 12:54:54.688305] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:b979edbc-bfb1-4340-984f-858499524e26> (b979edbc-bfb1-4340-984f-858499524e26) [No such file or directory]
[2017-03-17 12:54:54.688346] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:b979edbc-bfb1-4340-984f-858499524e26> (b979edbc-bfb1-4340-984f-858499524e26) [No such file or directory]
[2017-03-17 12:54:54.689259] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:b979edbc-bfb1-4340-984f-858499524e26> (b979edbc-bfb1-4340-984f-858499524e26) [No such file or directory]
[2017-03-17 12:54:54.689458] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:b979edbc-bfb1-4340-984f-858499524e26> (b979edbc-bfb1-4340-984f-858499524e26) [No such file or directory]
[2017-03-17 12:54:54.690222] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b979edbc-bfb1-4340-984f-858499524e26>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:54:54.690293] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b979edbc-bfb1-4340-984f-858499524e26 [Stale file handle]
[2017-03-17 12:54:56.574518] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming <gfid:40a0aeba-5026-4b59-91a0-9706fa800150>/Mahdi-Test20122.vmdk~ (hash=vmware2-replicate-4/cache=vmware2-replicate-4) => <gfid:40a0aeba-5026-4b59-91a0-9706fa800150>/Mahdi-Test20122.vmdk (hash=vmware2-replicate-15/cache=vmware2-replicate-15)
[2017-03-17 12:54:56.583157] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-31: remote operation failed [No such file or directory]
[2017-03-17 12:54:56.583664] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-30: remote operation failed. Path: <gfid:5b41b2e7-1c3a-4322-a84a-c33c1e2fe030> (5b41b2e7-1c3a-4322-a84a-c33c1e2fe030) [No such file or directory]
[2017-03-17 12:54:56.583734] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-31: remote operation failed. Path: <gfid:5b41b2e7-1c3a-4322-a84a-c33c1e2fe030> (5b41b2e7-1c3a-4322-a84a-c33c1e2fe030) [No such file or directory]
[2017-03-17 12:54:56.584377] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 5b41b2e7-1c3a-4322-a84a-c33c1e2fe030. (Possible split-brain)
[2017-03-17 12:54:56.584883] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-30: remote operation failed. Path: <gfid:5b41b2e7-1c3a-4322-a84a-c33c1e2fe030> (5b41b2e7-1c3a-4322-a84a-c33c1e2fe030) [No such file or directory]
[2017-03-17 12:54:56.584935] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-31: remote operation failed. Path: <gfid:5b41b2e7-1c3a-4322-a84a-c33c1e2fe030> (5b41b2e7-1c3a-4322-a84a-c33c1e2fe030) [No such file or directory]
[2017-03-17 12:54:56.586597] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-30: remote operation failed. Path: <gfid:5b41b2e7-1c3a-4322-a84a-c33c1e2fe030> (5b41b2e7-1c3a-4322-a84a-c33c1e2fe030) [No such file or directory]
[2017-03-17 12:54:56.586615] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-31: remote operation failed. Path: <gfid:5b41b2e7-1c3a-4322-a84a-c33c1e2fe030> (5b41b2e7-1c3a-4322-a84a-c33c1e2fe030) [No such file or directory]
[2017-03-17 12:54:56.586858] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:5b41b2e7-1c3a-4322-a84a-c33c1e2fe030>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:54:56.586914] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 5b41b2e7-1c3a-4322-a84a-c33c1e2fe030 [Stale file handle]
[2017-03-17 12:54:56.590823] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 8412967b-be8d-49b7-8c42-516dc7d772af. (Possible split-brain)
[2017-03-17 12:54:56.593817] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:8412967b-be8d-49b7-8c42-516dc7d772af>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:54:56.593870] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 8412967b-be8d-49b7-8c42-516dc7d772af [Stale file handle]
[2017-03-17 12:56:50.463206] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmdk~ (hash=vmware2-replicate-2/cache=vmware2-replicate-2) => /gfstest/gfstest.vmdk (hash=vmware2-replicate-8/cache=vmware2-replicate-8)
[2017-03-17 12:56:50.472257] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid b0027665-f8c7-4fff-8fb0-12b5e8e3a278. (Possible split-brain)
[2017-03-17 12:56:50.474844] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b0027665-f8c7-4fff-8fb0-12b5e8e3a278>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:56:50.474906] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b0027665-f8c7-4fff-8fb0-12b5e8e3a278 [Stale file handle]
[2017-03-17 12:56:50.479825] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid b6b3f49f-6650-4454-8b66-c9fc2e784fc6. (Possible split-brain)
[2017-03-17 12:56:50.482986] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b6b3f49f-6650-4454-8b66-c9fc2e784fc6>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:56:50.483079] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b6b3f49f-6650-4454-8b66-c9fc2e784fc6 [Stale file handle]
[2017-03-17 12:56:50.495547] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 12:56:50.533693] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmdk~ (hash=vmware2-replicate-2/cache=vmware2-replicate-2) => /gfstest/gfstest.vmdk (hash=vmware2-replicate-8/cache=vmware2-replicate-2)
[2017-03-17 12:56:50.540634] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 12:56:50.541092] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:1114df72-cdf7-4d0c-bec2-3b0c0615f031> (1114df72-cdf7-4d0c-bec2-3b0c0615f031) [No such file or directory]
[2017-03-17 12:56:50.541148] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:1114df72-cdf7-4d0c-bec2-3b0c0615f031> (1114df72-cdf7-4d0c-bec2-3b0c0615f031) [No such file or directory]
[2017-03-17 12:56:50.541514] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1114df72-cdf7-4d0c-bec2-3b0c0615f031. (Possible split-brain)
[2017-03-17 12:56:50.542012] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:1114df72-cdf7-4d0c-bec2-3b0c0615f031> (1114df72-cdf7-4d0c-bec2-3b0c0615f031) [No such file or directory]
[2017-03-17 12:56:50.542081] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:1114df72-cdf7-4d0c-bec2-3b0c0615f031> (1114df72-cdf7-4d0c-bec2-3b0c0615f031) [No such file or directory]
[2017-03-17 12:56:50.542698] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:1114df72-cdf7-4d0c-bec2-3b0c0615f031> (1114df72-cdf7-4d0c-bec2-3b0c0615f031) [No such file or directory]
[2017-03-17 12:56:50.542776] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:1114df72-cdf7-4d0c-bec2-3b0c0615f031> (1114df72-cdf7-4d0c-bec2-3b0c0615f031) [No such file or directory]
[2017-03-17 12:56:50.543965] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:1114df72-cdf7-4d0c-bec2-3b0c0615f031>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:56:50.544034] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 1114df72-cdf7-4d0c-bec2-3b0c0615f031 [Stale file handle]
[2017-03-17 12:56:50.547785] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 3c4486bc-7ea8-4223-9b4b-9cac5e81349b. (Possible split-brain)
[2017-03-17 12:56:50.550222] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:3c4486bc-7ea8-4223-9b4b-9cac5e81349b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:56:50.550272] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 3c4486bc-7ea8-4223-9b4b-9cac5e81349b [Stale file handle]
[2017-03-17 12:56:50.580094] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmdk~ (hash=vmware2-replicate-2/cache=vmware2-replicate-2) => /gfstest/gfstest.vmdk (hash=vmware2-replicate-8/cache=vmware2-replicate-2)
[2017-03-17 12:56:50.586613] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 12:56:50.587011] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:a08efcd9-720b-4586-ba7e-e345f6da24d4> (a08efcd9-720b-4586-ba7e-e345f6da24d4) [No such file or directory]
[2017-03-17 12:56:50.587203] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:a08efcd9-720b-4586-ba7e-e345f6da24d4> (a08efcd9-720b-4586-ba7e-e345f6da24d4) [No such file or directory]
[2017-03-17 12:56:50.587547] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid a08efcd9-720b-4586-ba7e-e345f6da24d4. (Possible split-brain)
[2017-03-17 12:56:50.588008] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:a08efcd9-720b-4586-ba7e-e345f6da24d4> (a08efcd9-720b-4586-ba7e-e345f6da24d4) [No such file or directory]
[2017-03-17 12:56:50.588017] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:a08efcd9-720b-4586-ba7e-e345f6da24d4> (a08efcd9-720b-4586-ba7e-e345f6da24d4) [No such file or directory]
[2017-03-17 12:56:50.588802] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:a08efcd9-720b-4586-ba7e-e345f6da24d4> (a08efcd9-720b-4586-ba7e-e345f6da24d4) [No such file or directory]
[2017-03-17 12:56:50.588830] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:a08efcd9-720b-4586-ba7e-e345f6da24d4> (a08efcd9-720b-4586-ba7e-e345f6da24d4) [No such file or directory]
[2017-03-17 12:56:50.590462] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:a08efcd9-720b-4586-ba7e-e345f6da24d4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:56:50.590510] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: a08efcd9-720b-4586-ba7e-e345f6da24d4 [Stale file handle]
[2017-03-17 12:56:50.594039] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-12: Unreadable subvolume -1 found with event generation 2 for gfid 8deff067-2e51-4c96-a4f9-1107ddc470f9. (Possible split-brain)
[2017-03-17 12:56:50.596359] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:8deff067-2e51-4c96-a4f9-1107ddc470f9>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:56:50.596425] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 8deff067-2e51-4c96-a4f9-1107ddc470f9 [Stale file handle]
[2017-03-17 12:56:50.615625] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 12:56:50.615865] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 12:56:50.616210] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 12:56:50.616376] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 12:57:28.841062] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 9563095e-ff8f-459e-8903-bd1c89c67171. (Possible split-brain)
[2017-03-17 12:57:28.843773] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:9563095e-ff8f-459e-8903-bd1c89c67171>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:28.843905] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 9563095e-ff8f-459e-8903-bd1c89c67171 [Stale file handle]
[2017-03-17 12:57:28.905538] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid fcd748e8-cd38-45b4-80fc-637ce00f58c8. (Possible split-brain)
[2017-03-17 12:57:28.906951] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:fcd748e8-cd38-45b4-80fc-637ce00f58c8>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:28.907001] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: fcd748e8-cd38-45b4-80fc-637ce00f58c8 [Stale file handle]
[2017-03-17 12:57:28.948582] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid 9fe944fc-c9cf-4392-8bec-d56ee60bfec0. (Possible split-brain)
[2017-03-17 12:57:28.950377] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:9fe944fc-c9cf-4392-8bec-d56ee60bfec0>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:28.950459] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 9fe944fc-c9cf-4392-8bec-d56ee60bfec0 [Stale file handle]
[2017-03-17 12:57:28.957256] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid 62f4bcef-ea7c-4aeb-a4dd-fb8f413f06f8. (Possible split-brain)
[2017-03-17 12:57:28.979794] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:62f4bcef-ea7c-4aeb-a4dd-fb8f413f06f8>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:28.979829] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 62f4bcef-ea7c-4aeb-a4dd-fb8f413f06f8 [Stale file handle]
[2017-03-17 12:57:29.095394] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming <gfid:5f6359a1-5617-418e-9de4-2db3ced45ff5>/New Virtual Machine.vmx~ (hash=vmware2-replicate-17/cache=vmware2-replicate-17) => <gfid:5f6359a1-5617-418e-9de4-2db3ced45ff5>/New Virtual Machine.vmx (hash=vmware2-replicate-7/cache=vmware2-replicate-7)
[2017-03-17 12:57:29.106279] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid b4251d1f-15b0-49bf-a05e-f10438744b55. (Possible split-brain)
[2017-03-17 12:57:29.107945] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b4251d1f-15b0-49bf-a05e-f10438744b55>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:29.107986] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b4251d1f-15b0-49bf-a05e-f10438744b55 [Stale file handle]
[2017-03-17 12:57:29.111741] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 62c424a1-9895-4cd9-8fc2-9e6fd07fa683. (Possible split-brain)
[2017-03-17 12:57:29.113292] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:62c424a1-9895-4cd9-8fc2-9e6fd07fa683>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:29.113321] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 62c424a1-9895-4cd9-8fc2-9e6fd07fa683 [Stale file handle]
[2017-03-17 12:57:29.146232] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-11: Unreadable subvolume -1 found with event generation 2 for gfid 30aa408c-2c9e-45e6-a814-aac76441b8eb. (Possible split-brain)
[2017-03-17 12:57:29.148177] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:30aa408c-2c9e-45e6-a814-aac76441b8eb>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:29.148207] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 30aa408c-2c9e-45e6-a814-aac76441b8eb [Stale file handle]
[2017-03-17 12:57:29.151394] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid f6423582-beb8-46ee-8ce3-d2b7678c2339. (Possible split-brain)
[2017-03-17 12:57:29.181968] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:f6423582-beb8-46ee-8ce3-d2b7678c2339>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:29.182001] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: f6423582-beb8-46ee-8ce3-d2b7678c2339 [Stale file handle]
[2017-03-17 12:57:29.219618] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid d4efd103-df77-4fbd-a0fc-e88246a8e6cc. (Possible split-brain)
[2017-03-17 12:57:29.220866] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:d4efd103-df77-4fbd-a0fc-e88246a8e6cc>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:29.220900] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: d4efd103-df77-4fbd-a0fc-e88246a8e6cc [Stale file handle]
[2017-03-17 12:57:29.253870] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 77b03a81-b7e9-4e14-a39a-2ad6bc3533eb. (Possible split-brain)
[2017-03-17 12:57:29.255324] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:77b03a81-b7e9-4e14-a39a-2ad6bc3533eb>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:29.255367] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 77b03a81-b7e9-4e14-a39a-2ad6bc3533eb [Stale file handle]
[2017-03-17 12:57:29.258666] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 57fae8fc-b224-4206-8d9d-3ad8f5efc9dc. (Possible split-brain)
[2017-03-17 12:57:29.260471] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:57fae8fc-b224-4206-8d9d-3ad8f5efc9dc>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:29.260524] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 57fae8fc-b224-4206-8d9d-3ad8f5efc9dc [Stale file handle]
[2017-03-17 12:57:29.405427] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 12:57:29.405625] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 12:57:44.925248] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid ac77763f-6aa6-4d9b-8b31-76e45b7eeff6. (Possible split-brain)
[2017-03-17 12:57:44.926994] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:ac77763f-6aa6-4d9b-8b31-76e45b7eeff6>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:44.927035] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: ac77763f-6aa6-4d9b-8b31-76e45b7eeff6 [Stale file handle]
[2017-03-17 12:57:45.005308] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid 0a3e5686-821c-4164-ab03-ef26aac80997. (Possible split-brain)
[2017-03-17 12:57:45.007357] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:0a3e5686-821c-4164-ab03-ef26aac80997>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:45.007392] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 0a3e5686-821c-4164-ab03-ef26aac80997 [Stale file handle]
[2017-03-17 12:57:45.041252] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 93ee7170-aafb-4d2e-9ed8-b996ee459e65. (Possible split-brain)
[2017-03-17 12:57:45.042599] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:93ee7170-aafb-4d2e-9ed8-b996ee459e65>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:45.042626] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 93ee7170-aafb-4d2e-9ed8-b996ee459e65 [Stale file handle]
[2017-03-17 12:57:45.046006] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid 3b9d8dd9-9025-485b-8300-0e82f82265f3. (Possible split-brain)
[2017-03-17 12:57:45.047880] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:3b9d8dd9-9025-485b-8300-0e82f82265f3>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:45.047914] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 3b9d8dd9-9025-485b-8300-0e82f82265f3 [Stale file handle]
[2017-03-17 12:57:45.654864] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming <gfid:40a0aeba-5026-4b59-91a0-9706fa800150>/Mahdi-Test20122.vmx~ (hash=vmware2-replicate-13/cache=vmware2-replicate-13) => <gfid:40a0aeba-5026-4b59-91a0-9706fa800150>/Mahdi-Test20122.vmx (hash=vmware2-replicate-7/cache=vmware2-replicate-13)
[2017-03-17 12:57:45.662661] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-27: remote operation failed [No such file or directory]
[2017-03-17 12:57:45.662883] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-27: remote operation failed. Path: <gfid:a27d5056-9a07-4107-9adf-41308e47631b> (a27d5056-9a07-4107-9adf-41308e47631b) [No such file or directory]
[2017-03-17 12:57:45.662908] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-26: remote operation failed. Path: <gfid:a27d5056-9a07-4107-9adf-41308e47631b> (a27d5056-9a07-4107-9adf-41308e47631b) [No such file or directory]
[2017-03-17 12:57:45.663232] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid a27d5056-9a07-4107-9adf-41308e47631b. (Possible split-brain)
[2017-03-17 12:57:45.663467] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-26: remote operation failed. Path: <gfid:a27d5056-9a07-4107-9adf-41308e47631b> (a27d5056-9a07-4107-9adf-41308e47631b) [No such file or directory]
[2017-03-17 12:57:45.663505] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-27: remote operation failed. Path: <gfid:a27d5056-9a07-4107-9adf-41308e47631b> (a27d5056-9a07-4107-9adf-41308e47631b) [No such file or directory]
[2017-03-17 12:57:45.664315] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-27: remote operation failed. Path: <gfid:a27d5056-9a07-4107-9adf-41308e47631b> (a27d5056-9a07-4107-9adf-41308e47631b) [No such file or directory]
[2017-03-17 12:57:45.664333] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-26: remote operation failed. Path: <gfid:a27d5056-9a07-4107-9adf-41308e47631b> (a27d5056-9a07-4107-9adf-41308e47631b) [No such file or directory]
[2017-03-17 12:57:45.664586] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:a27d5056-9a07-4107-9adf-41308e47631b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:45.664610] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: a27d5056-9a07-4107-9adf-41308e47631b [Stale file handle]
[2017-03-17 12:57:45.672325] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid d651deb9-2c77-4982-a1c2-c5c4bcd0b9f5. (Possible split-brain)
[2017-03-17 12:57:45.673818] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:d651deb9-2c77-4982-a1c2-c5c4bcd0b9f5>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:45.673847] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: d651deb9-2c77-4982-a1c2-c5c4bcd0b9f5 [Stale file handle]
[2017-03-17 12:57:45.718175] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid a37a119b-fca4-4fbd-9c08-11861d5d6035. (Possible split-brain)
[2017-03-17 12:57:45.719483] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:a37a119b-fca4-4fbd-9c08-11861d5d6035>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:45.719559] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: a37a119b-fca4-4fbd-9c08-11861d5d6035 [Stale file handle]
[2017-03-17 12:57:45.727465] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 2456f950-798c-4cba-b344-cf7e5623e119. (Possible split-brain)
[2017-03-17 12:57:45.728858] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:2456f950-798c-4cba-b344-cf7e5623e119>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:45.728886] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 2456f950-798c-4cba-b344-cf7e5623e119 [Stale file handle]
[2017-03-17 12:57:45.771809] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 5599d5d8-a1ff-47d7-a9e0-5ad617383c2a. (Possible split-brain)
[2017-03-17 12:57:45.773394] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:5599d5d8-a1ff-47d7-a9e0-5ad617383c2a>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:45.773423] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 5599d5d8-a1ff-47d7-a9e0-5ad617383c2a [Stale file handle]
[2017-03-17 12:57:45.806955] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 882a3d41-59b1-449f-be96-b20197e57257. (Possible split-brain)
[2017-03-17 12:57:45.808432] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:882a3d41-59b1-449f-be96-b20197e57257>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:45.808473] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 882a3d41-59b1-449f-be96-b20197e57257 [Stale file handle]
[2017-03-17 12:57:45.816559] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 247d3b94-f81a-4607-95be-8302e01e7492. (Possible split-brain)
[2017-03-17 12:57:45.817865] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:247d3b94-f81a-4607-95be-8302e01e7492>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:45.817895] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 247d3b94-f81a-4607-95be-8302e01e7492 [Stale file handle]
[2017-03-17 12:57:45.953027] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 12:57:45.953151] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 12:57:45.953488] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 12:57:45.953577] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 12:57:56.207688] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid b07557a4-6d44-4e22-a82c-1919b5195896. (Possible split-brain)
[2017-03-17 12:57:56.209068] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b07557a4-6d44-4e22-a82c-1919b5195896>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.209103] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b07557a4-6d44-4e22-a82c-1919b5195896 [Stale file handle]
[2017-03-17 12:57:56.211328] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid a8e49254-abe5-4c3f-8a44-1d1ee25d1520. (Possible split-brain)
[2017-03-17 12:57:56.215325] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:a8e49254-abe5-4c3f-8a44-1d1ee25d1520>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.215353] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: a8e49254-abe5-4c3f-8a44-1d1ee25d1520 [Stale file handle]
[2017-03-17 12:57:56.255094] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid e25db064-7223-4fa4-b0ad-71158c5f11a2. (Possible split-brain)
[2017-03-17 12:57:56.256539] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 5eee0fae-4ebc-4985-9179-48e92e4dc655. (Possible split-brain)
[2017-03-17 12:57:56.256796] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:e25db064-7223-4fa4-b0ad-71158c5f11a2>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.256831] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: e25db064-7223-4fa4-b0ad-71158c5f11a2 [Stale file handle]
[2017-03-17 12:57:56.258473] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:5eee0fae-4ebc-4985-9179-48e92e4dc655>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.258509] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 5eee0fae-4ebc-4985-9179-48e92e4dc655 [Stale file handle]
[2017-03-17 12:57:56.260098] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid 8f0fc5c8-c2c1-4e95-80a6-cdcfcf6a65b3. (Possible split-brain)
[2017-03-17 12:57:56.261402] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 73d6d71b-91cf-45c1-930c-b8531a4ff9c0. (Possible split-brain)
[2017-03-17 12:57:56.261586] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:8f0fc5c8-c2c1-4e95-80a6-cdcfcf6a65b3>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.261613] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 8f0fc5c8-c2c1-4e95-80a6-cdcfcf6a65b3 [Stale file handle]
[2017-03-17 12:57:56.263267] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:73d6d71b-91cf-45c1-930c-b8531a4ff9c0>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.263290] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 73d6d71b-91cf-45c1-930c-b8531a4ff9c0 [Stale file handle]
[2017-03-17 12:57:56.290798] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 5a1d0ba3-499c-4834-a571-f49a659e3465. (Possible split-brain)
[2017-03-17 12:57:56.292080] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:5a1d0ba3-499c-4834-a571-f49a659e3465>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.292155] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 5a1d0ba3-499c-4834-a571-f49a659e3465 [Stale file handle]
[2017-03-17 12:57:56.296264] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid 04863293-e081-49c9-81e4-30d830594fec. (Possible split-brain)
[2017-03-17 12:57:56.298014] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:04863293-e081-49c9-81e4-30d830594fec>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.298038] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 04863293-e081-49c9-81e4-30d830594fec [Stale file handle]
[2017-03-17 12:57:56.338144] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 12:57:56.338452] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 12:57:56.338849] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 12:57:56.338913] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 12:57:56.338998] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 12:57:56.339083] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 12:57:56.339107] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 12:57:56.339157] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 12:57:56.339159] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 12:57:56.339203] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 12:57:56.487277] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 595c06a2-fd64-4abf-8cb6-6c7edd1e54bd. (Possible split-brain)
[2017-03-17 12:57:56.488947] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:595c06a2-fd64-4abf-8cb6-6c7edd1e54bd>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.488979] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 595c06a2-fd64-4abf-8cb6-6c7edd1e54bd [Stale file handle]
[2017-03-17 12:57:56.494268] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 683f0410-997b-4e31-884b-4e2d61c92a46. (Possible split-brain)
[2017-03-17 12:57:56.496032] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:683f0410-997b-4e31-884b-4e2d61c92a46>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.496062] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 683f0410-997b-4e31-884b-4e2d61c92a46 [Stale file handle]
[2017-03-17 12:57:56.536944] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 30986c2a-8346-4b13-a47a-a827dcd74fe6. (Possible split-brain)
[2017-03-17 12:57:56.540842] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:30986c2a-8346-4b13-a47a-a827dcd74fe6>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.540883] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 30986c2a-8346-4b13-a47a-a827dcd74fe6 [Stale file handle]
[2017-03-17 12:57:56.546430] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 52a06f37-6d70-4d12-9972-08b3117cb3d6. (Possible split-brain)
[2017-03-17 12:57:56.547970] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:52a06f37-6d70-4d12-9972-08b3117cb3d6>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.548173] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 52a06f37-6d70-4d12-9972-08b3117cb3d6 [Stale file handle]
[2017-03-17 12:57:56.552512] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 7fdc37c4-273b-45c7-9c4f-dd9dfd2e0902. (Possible split-brain)
[2017-03-17 12:57:56.553883] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:7fdc37c4-273b-45c7-9c4f-dd9dfd2e0902>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.553909] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 7fdc37c4-273b-45c7-9c4f-dd9dfd2e0902 [Stale file handle]
[2017-03-17 12:57:56.555970] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 3b28d521-0c5e-4b5b-b582-74f36ca6cfdc. (Possible split-brain)
[2017-03-17 12:57:56.558381] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:3b28d521-0c5e-4b5b-b582-74f36ca6cfdc>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.558419] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 3b28d521-0c5e-4b5b-b582-74f36ca6cfdc [Stale file handle]
[2017-03-17 12:57:56.595073] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 5df33cef-2e1b-4ffd-b05e-08d6679c0f91. (Possible split-brain)
[2017-03-17 12:57:56.597630] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:5df33cef-2e1b-4ffd-b05e-08d6679c0f91>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.597660] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 5df33cef-2e1b-4ffd-b05e-08d6679c0f91 [Stale file handle]
[2017-03-17 12:57:56.603737] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 34accd95-2ecf-4397-8848-cb3d63c4d58f. (Possible split-brain)
[2017-03-17 12:57:56.605170] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:34accd95-2ecf-4397-8848-cb3d63c4d58f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.605196] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 34accd95-2ecf-4397-8848-cb3d63c4d58f [Stale file handle]
[2017-03-17 12:57:56.609912] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid 1a397df1-d859-49dd-a4dc-0cee20f63e84. (Possible split-brain)
[2017-03-17 12:57:56.611525] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:1a397df1-d859-49dd-a4dc-0cee20f63e84>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.611549] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 1a397df1-d859-49dd-a4dc-0cee20f63e84 [Stale file handle]
[2017-03-17 12:57:56.618361] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid e085a37d-6d62-43b7-a59a-a21b25643774. (Possible split-brain)
[2017-03-17 12:57:56.619583] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:e085a37d-6d62-43b7-a59a-a21b25643774>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.619620] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: e085a37d-6d62-43b7-a59a-a21b25643774 [Stale file handle]
[2017-03-17 12:57:56.671168] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 81cd65ff-4e72-4431-8fd5-c56b653fb880. (Possible split-brain)
[2017-03-17 12:57:56.672444] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:81cd65ff-4e72-4431-8fd5-c56b653fb880>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.672468] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 81cd65ff-4e72-4431-8fd5-c56b653fb880 [Stale file handle]
[2017-03-17 12:57:56.681817] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid ceeefc49-17f4-49fb-94ac-835556b65413. (Possible split-brain)
[2017-03-17 12:57:56.682086] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a6cbe27b-0765-4fb9-8d24-6074fe1b6df9. (Possible split-brain)
[2017-03-17 12:57:56.684146] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:ceeefc49-17f4-49fb-94ac-835556b65413>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.684180] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: ceeefc49-17f4-49fb-94ac-835556b65413 [Stale file handle]
[2017-03-17 12:57:56.685028] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:a6cbe27b-0765-4fb9-8d24-6074fe1b6df9>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.685065] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: a6cbe27b-0765-4fb9-8d24-6074fe1b6df9 [Stale file handle]
[2017-03-17 12:57:56.691568] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid 238c6f7e-f578-4a5a-a0aa-a3d319adce93. (Possible split-brain)
[2017-03-17 12:57:56.694219] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:238c6f7e-f578-4a5a-a0aa-a3d319adce93>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.694248] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 238c6f7e-f578-4a5a-a0aa-a3d319adce93 [Stale file handle]
[2017-03-17 12:57:56.766045] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 31d3abd8-44a4-4aee-86ae-10e5ed22d669. (Possible split-brain)
[2017-03-17 12:57:56.769583] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:31d3abd8-44a4-4aee-86ae-10e5ed22d669>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.769617] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 31d3abd8-44a4-4aee-86ae-10e5ed22d669 [Stale file handle]
[2017-03-17 12:57:56.775134] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 48e951be-b3a7-44a8-9f8c-4d2d933a0052. (Possible split-brain)
[2017-03-17 12:57:56.778201] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:48e951be-b3a7-44a8-9f8c-4d2d933a0052>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:57:56.778229] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 48e951be-b3a7-44a8-9f8c-4d2d933a0052 [Stale file handle]
[2017-03-17 12:57:57.751064] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 12:57:57.751143] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 12:57:57.751209] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 12:57:57.751641] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 12:57:57.751670] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 12:57:57.751708] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 12:57:57.751802] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 12:57:57.751837] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 12:57:57.751887] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 12:57:57.751922] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 12:58:45.814911] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmdk~ (hash=vmware2-replicate-18/cache=vmware2-replicate-18) => /New Virtual Machine/New Virtual Machine.vmdk (hash=vmware2-replicate-16/cache=vmware2-replicate-16)
[2017-03-17 12:58:45.824358] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid c8475a2e-7a3c-47e1-97d5-fea6fed74b5d. (Possible split-brain)
[2017-03-17 12:58:45.826021] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:c8475a2e-7a3c-47e1-97d5-fea6fed74b5d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:58:45.826089] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: c8475a2e-7a3c-47e1-97d5-fea6fed74b5d [Stale file handle]
[2017-03-17 12:58:45.834980] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid 4d9b17b9-7162-486f-b9c5-7e8982610e43. (Possible split-brain)
[2017-03-17 12:58:45.836352] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:4d9b17b9-7162-486f-b9c5-7e8982610e43>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:58:45.836380] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 4d9b17b9-7162-486f-b9c5-7e8982610e43 [Stale file handle]
[2017-03-17 12:58:45.880700] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 12:58:45.969133] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmdk~ (hash=vmware2-replicate-18/cache=vmware2-replicate-18) => /New Virtual Machine/New Virtual Machine.vmdk (hash=vmware2-replicate-16/cache=vmware2-replicate-18)
[2017-03-17 12:58:45.973292] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-36: remote operation failed [No such file or directory]
[2017-03-17 12:58:45.973471] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b> (1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b) [No such file or directory]
[2017-03-17 12:58:45.973517] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b> (1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b) [No such file or directory]
[2017-03-17 12:58:45.974145] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid 1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b. (Possible split-brain)
[2017-03-17 12:58:45.974376] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b> (1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b) [No such file or directory]
[2017-03-17 12:58:45.974385] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b> (1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b) [No such file or directory]
[2017-03-17 12:58:45.975596] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b> (1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b) [No such file or directory]
[2017-03-17 12:58:45.975809] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b> (1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b) [No such file or directory]
[2017-03-17 12:58:45.975872] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b>: failed to lookup the file on vmware2-dht [No such file or directory]
[2017-03-17 12:58:45.975919] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 1dd73a93-feb9-4686-9b9b-bb19d2b7fd9b [No such file or directory]
[2017-03-17 12:58:45.978069] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid 4fe32f54-87a4-4a36-a55d-b3bfa7a5777a. (Possible split-brain)
[2017-03-17 12:58:45.979578] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:4fe32f54-87a4-4a36-a55d-b3bfa7a5777a>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:58:45.979602] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 4fe32f54-87a4-4a36-a55d-b3bfa7a5777a [Stale file handle]
[2017-03-17 12:58:46.072725] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmdk~ (hash=vmware2-replicate-18/cache=vmware2-replicate-18) => /New Virtual Machine/New Virtual Machine.vmdk (hash=vmware2-replicate-16/cache=vmware2-replicate-18)
[2017-03-17 12:58:46.080211] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-36: remote operation failed [No such file or directory]
[2017-03-17 12:58:46.080436] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:f5b2f145-769a-4702-b309-44b6d953c411> (f5b2f145-769a-4702-b309-44b6d953c411) [No such file or directory]
[2017-03-17 12:58:46.080438] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:f5b2f145-769a-4702-b309-44b6d953c411> (f5b2f145-769a-4702-b309-44b6d953c411) [No such file or directory]
[2017-03-17 12:58:46.080728] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid f5b2f145-769a-4702-b309-44b6d953c411. (Possible split-brain)
[2017-03-17 12:58:46.080925] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:f5b2f145-769a-4702-b309-44b6d953c411> (f5b2f145-769a-4702-b309-44b6d953c411) [No such file or directory]
[2017-03-17 12:58:46.080979] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:f5b2f145-769a-4702-b309-44b6d953c411> (f5b2f145-769a-4702-b309-44b6d953c411) [No such file or directory]
[2017-03-17 12:58:46.081983] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:f5b2f145-769a-4702-b309-44b6d953c411> (f5b2f145-769a-4702-b309-44b6d953c411) [No such file or directory]
[2017-03-17 12:58:46.082088] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:f5b2f145-769a-4702-b309-44b6d953c411> (f5b2f145-769a-4702-b309-44b6d953c411) [No such file or directory]
[2017-03-17 12:58:46.082160] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:f5b2f145-769a-4702-b309-44b6d953c411>: failed to lookup the file on vmware2-dht [No such file or directory]
[2017-03-17 12:58:46.082182] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: f5b2f145-769a-4702-b309-44b6d953c411 [No such file or directory]
[2017-03-17 12:58:46.084223] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 2bc7178a-4d1a-4a8c-9f45-f46221de970b. (Possible split-brain)
[2017-03-17 12:58:46.088757] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:2bc7178a-4d1a-4a8c-9f45-f46221de970b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:58:46.088785] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 2bc7178a-4d1a-4a8c-9f45-f46221de970b [Stale file handle]
[2017-03-17 12:58:46.182058] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 12:58:46.184136] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 12:58:46.184583] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 12:59:58.872597] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmdk~ (hash=vmware2-replicate-2/cache=vmware2-replicate-2) => /gfstest/gfstest.vmdk (hash=vmware2-replicate-8/cache=vmware2-replicate-2)
[2017-03-17 12:59:58.877310] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 12:59:58.877554] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:143c8535-c8dd-48d7-8460-ab865b09edd1> (143c8535-c8dd-48d7-8460-ab865b09edd1) [No such file or directory]
[2017-03-17 12:59:58.877856] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:143c8535-c8dd-48d7-8460-ab865b09edd1> (143c8535-c8dd-48d7-8460-ab865b09edd1) [No such file or directory]
[2017-03-17 12:59:58.878583] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 143c8535-c8dd-48d7-8460-ab865b09edd1. (Possible split-brain)
[2017-03-17 12:59:58.879009] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:143c8535-c8dd-48d7-8460-ab865b09edd1> (143c8535-c8dd-48d7-8460-ab865b09edd1) [No such file or directory]
[2017-03-17 12:59:58.879052] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:143c8535-c8dd-48d7-8460-ab865b09edd1> (143c8535-c8dd-48d7-8460-ab865b09edd1) [No such file or directory]
[2017-03-17 12:59:58.879585] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:143c8535-c8dd-48d7-8460-ab865b09edd1> (143c8535-c8dd-48d7-8460-ab865b09edd1) [No such file or directory]
[2017-03-17 12:59:58.879772] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:143c8535-c8dd-48d7-8460-ab865b09edd1> (143c8535-c8dd-48d7-8460-ab865b09edd1) [No such file or directory]
[2017-03-17 12:59:58.880624] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:143c8535-c8dd-48d7-8460-ab865b09edd1>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:59:58.880662] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 143c8535-c8dd-48d7-8460-ab865b09edd1 [Stale file handle]
[2017-03-17 12:59:58.883899] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid b81a0b11-d161-4e84-8f67-52847cba225e. (Possible split-brain)
[2017-03-17 12:59:58.886184] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b81a0b11-d161-4e84-8f67-52847cba225e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:59:58.886211] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b81a0b11-d161-4e84-8f67-52847cba225e [Stale file handle]
[2017-03-17 12:59:58.889235] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 97a173a2-4a9c-4f58-8cf2-fc3096fa58d9. (Possible split-brain)
[2017-03-17 12:59:58.890995] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:97a173a2-4a9c-4f58-8cf2-fc3096fa58d9>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:59:58.891021] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 97a173a2-4a9c-4f58-8cf2-fc3096fa58d9 [Stale file handle]
[2017-03-17 12:59:58.972342] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-3)
[2017-03-17 12:59:58.978033] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-3: Unreadable subvolume -1 found with event generation 2 for gfid f01f09ee-ac4f-44a3-851a-d39712b29246. (Possible split-brain)
[2017-03-17 12:59:58.980381] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:f01f09ee-ac4f-44a3-851a-d39712b29246>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:59:58.980426] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: f01f09ee-ac4f-44a3-851a-d39712b29246 [Stale file handle]
[2017-03-17 12:59:58.982735] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 9318d286-d3fa-4a54-905f-456e2927206b. (Possible split-brain)
[2017-03-17 12:59:58.984391] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:9318d286-d3fa-4a54-905f-456e2927206b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:59:58.984427] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 9318d286-d3fa-4a54-905f-456e2927206b [Stale file handle]
[2017-03-17 12:59:59.032576] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 12:59:59.033022] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 12:59:59.114120] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid f5825e59-be53-4313-a4b0-75f6ad6df8b6. (Possible split-brain)
[2017-03-17 12:59:59.116659] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:f5825e59-be53-4313-a4b0-75f6ad6df8b6>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:59:59.116705] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: f5825e59-be53-4313-a4b0-75f6ad6df8b6 [Stale file handle]
[2017-03-17 12:59:59.161804] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 12:59:59.165854] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 12:59:59.166105] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:d0439ef6-06bc-4739-a642-0e9aa91d41ac> (d0439ef6-06bc-4739-a642-0e9aa91d41ac) [No such file or directory]
[2017-03-17 12:59:59.166212] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:d0439ef6-06bc-4739-a642-0e9aa91d41ac> (d0439ef6-06bc-4739-a642-0e9aa91d41ac) [No such file or directory]
[2017-03-17 12:59:59.166523] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid d0439ef6-06bc-4739-a642-0e9aa91d41ac. (Possible split-brain)
[2017-03-17 12:59:59.166870] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:d0439ef6-06bc-4739-a642-0e9aa91d41ac> (d0439ef6-06bc-4739-a642-0e9aa91d41ac) [No such file or directory]
[2017-03-17 12:59:59.166968] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:d0439ef6-06bc-4739-a642-0e9aa91d41ac> (d0439ef6-06bc-4739-a642-0e9aa91d41ac) [No such file or directory]
[2017-03-17 12:59:59.167690] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:d0439ef6-06bc-4739-a642-0e9aa91d41ac> (d0439ef6-06bc-4739-a642-0e9aa91d41ac) [No such file or directory]
[2017-03-17 12:59:59.167726] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:d0439ef6-06bc-4739-a642-0e9aa91d41ac> (d0439ef6-06bc-4739-a642-0e9aa91d41ac) [No such file or directory]
[2017-03-17 12:59:59.168532] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:d0439ef6-06bc-4739-a642-0e9aa91d41ac>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:59:59.168565] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: d0439ef6-06bc-4739-a642-0e9aa91d41ac [Stale file handle]
[2017-03-17 12:59:59.172825] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 2b7df5ce-b615-46b5-85a4-a8a81f3ff471. (Possible split-brain)
[2017-03-17 12:59:59.175417] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:2b7df5ce-b615-46b5-85a4-a8a81f3ff471>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:59:59.175453] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 2b7df5ce-b615-46b5-85a4-a8a81f3ff471 [Stale file handle]
[2017-03-17 12:59:59.187959] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid c10d529c-9ac5-487c-8a4e-8c56adb299c8. (Possible split-brain)
[2017-03-17 12:59:59.189202] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:c10d529c-9ac5-487c-8a4e-8c56adb299c8>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:59:59.189256] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: c10d529c-9ac5-487c-8a4e-8c56adb299c8 [Stale file handle]
[2017-03-17 12:59:59.191571] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 10b02ec6-4ec0-462f-87ca-092be1ff1b93. (Possible split-brain)
[2017-03-17 12:59:59.193142] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:10b02ec6-4ec0-462f-87ca-092be1ff1b93>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 12:59:59.193197] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 10b02ec6-4ec0-462f-87ca-092be1ff1b93 [Stale file handle]
[2017-03-17 12:59:59.260755] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 12:59:59.261239] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:01:51.311617] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4.vmx~ (hash=vmware2-replicate-9/cache=vmware2-replicate-9) => <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4.vmx (hash=vmware2-replicate-6/cache=vmware2-replicate-6)
[2017-03-17 13:01:51.327254] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 13:01:51.327518] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-12: remote operation failed. Path: <gfid:84739892-a549-475a-9ffc-72b27f28e5fd> (84739892-a549-475a-9ffc-72b27f28e5fd) [No such file or directory]
[2017-03-17 13:01:51.327675] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-13: remote operation failed. Path: <gfid:84739892-a549-475a-9ffc-72b27f28e5fd> (84739892-a549-475a-9ffc-72b27f28e5fd) [No such file or directory]
[2017-03-17 13:01:51.328439] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 84739892-a549-475a-9ffc-72b27f28e5fd. (Possible split-brain)
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-13: remote operation failed. Path: <gfid:84739892-a549-475a-9ffc-72b27f28e5fd> (84739892-a549-475a-9ffc-72b27f28e5fd) [No such file or directory]" repeated 2 times between [2017-03-17 13:01:51.327675] and [2017-03-17 13:01:51.329331]
[2017-03-17 13:01:51.329907] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:84739892-a549-475a-9ffc-72b27f28e5fd>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:51.329969] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 84739892-a549-475a-9ffc-72b27f28e5fd [Stale file handle]
[2017-03-17 13:01:51.355313] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid d88d9e6c-f2e8-461e-abb0-e2474462d6d4. (Possible split-brain)
[2017-03-17 13:01:51.359700] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:d88d9e6c-f2e8-461e-abb0-e2474462d6d4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:51.359740] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: d88d9e6c-f2e8-461e-abb0-e2474462d6d4 [Stale file handle]
[2017-03-17 13:01:51.378103] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 553d81ae-e762-4534-af66-c0511a624ba1. (Possible split-brain)
[2017-03-17 13:01:51.380178] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:553d81ae-e762-4534-af66-c0511a624ba1>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:51.380211] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 553d81ae-e762-4534-af66-c0511a624ba1 [Stale file handle]
[2017-03-17 13:01:51.398863] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid fcebc8e9-1bf6-4261-90bd-15e4c0a1aacd. (Possible split-brain)
[2017-03-17 13:01:51.402822] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:fcebc8e9-1bf6-4261-90bd-15e4c0a1aacd>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:51.402857] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: fcebc8e9-1bf6-4261-90bd-15e4c0a1aacd [Stale file handle]
[2017-03-17 13:01:51.419749] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid ff370fe6-19d1-4ced-9818-9870ab6d1e19. (Possible split-brain)
[2017-03-17 13:01:51.426897] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:ff370fe6-19d1-4ced-9818-9870ab6d1e19>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:51.426931] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: ff370fe6-19d1-4ced-9818-9870ab6d1e19 [Stale file handle]
[2017-03-17 13:01:51.440455] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-11: Unreadable subvolume -1 found with event generation 2 for gfid 1a6caf86-8b6d-460b-b52f-b08a7587763e. (Possible split-brain)
[2017-03-17 13:01:51.446507] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:1a6caf86-8b6d-460b-b52f-b08a7587763e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:51.446546] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 1a6caf86-8b6d-460b-b52f-b08a7587763e [Stale file handle]
[2017-03-17 13:01:51.460032] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 75e7b3d5-ef69-46b7-90ef-4fead132fefe. (Possible split-brain)
[2017-03-17 13:01:51.466056] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:75e7b3d5-ef69-46b7-90ef-4fead132fefe>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:51.466093] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 75e7b3d5-ef69-46b7-90ef-4fead132fefe [Stale file handle]
[2017-03-17 13:01:51.576564] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 0f36d2e5-1231-42b9-9159-53d79b2f5f96. (Possible split-brain)
[2017-03-17 13:01:51.578739] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:0f36d2e5-1231-42b9-9159-53d79b2f5f96>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:51.578783] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 0f36d2e5-1231-42b9-9159-53d79b2f5f96 [Stale file handle]
[2017-03-17 13:01:51.589012] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid 31edac50-039b-493f-ab86-bdd468f1a254. (Possible split-brain)
[2017-03-17 13:01:51.590928] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:31edac50-039b-493f-ab86-bdd468f1a254>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:51.590957] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 31edac50-039b-493f-ab86-bdd468f1a254 [Stale file handle]
[2017-03-17 13:01:51.593743] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid d7c11a3c-5702-4880-90de-3c49bbfecf41. (Possible split-brain)
[2017-03-17 13:01:51.595733] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:d7c11a3c-5702-4880-90de-3c49bbfecf41>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:51.595776] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: d7c11a3c-5702-4880-90de-3c49bbfecf41 [Stale file handle]
[2017-03-17 13:01:52.382480] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4.vmx~ (hash=vmware2-replicate-9/cache=vmware2-replicate-9) => <gfid:db00a20c-5949-4ebd-95db-b335133b3358>/IT-Mahdi4.vmx (hash=vmware2-replicate-6/cache=vmware2-replicate-9)
[2017-03-17 13:01:52.390673] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-18: remote operation failed [No such file or directory]
[2017-03-17 13:01:52.390929] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-18: remote operation failed. Path: <gfid:b7e959d5-6c40-4d17-8b53-7086bb54af1f> (b7e959d5-6c40-4d17-8b53-7086bb54af1f) [No such file or directory]
[2017-03-17 13:01:52.390977] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-19: remote operation failed. Path: <gfid:b7e959d5-6c40-4d17-8b53-7086bb54af1f> (b7e959d5-6c40-4d17-8b53-7086bb54af1f) [No such file or directory]
[2017-03-17 13:01:52.391302] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid b7e959d5-6c40-4d17-8b53-7086bb54af1f. (Possible split-brain)
[2017-03-17 13:01:52.391561] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-19: remote operation failed. Path: <gfid:b7e959d5-6c40-4d17-8b53-7086bb54af1f> (b7e959d5-6c40-4d17-8b53-7086bb54af1f) [No such file or directory]
[2017-03-17 13:01:52.391583] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-18: remote operation failed. Path: <gfid:b7e959d5-6c40-4d17-8b53-7086bb54af1f> (b7e959d5-6c40-4d17-8b53-7086bb54af1f) [No such file or directory]
[2017-03-17 13:01:52.393259] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-18: remote operation failed. Path: <gfid:b7e959d5-6c40-4d17-8b53-7086bb54af1f> (b7e959d5-6c40-4d17-8b53-7086bb54af1f) [No such file or directory]
[2017-03-17 13:01:52.393422] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-19: remote operation failed. Path: <gfid:b7e959d5-6c40-4d17-8b53-7086bb54af1f> (b7e959d5-6c40-4d17-8b53-7086bb54af1f) [No such file or directory]
[2017-03-17 13:01:52.394738] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b7e959d5-6c40-4d17-8b53-7086bb54af1f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:52.394796] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b7e959d5-6c40-4d17-8b53-7086bb54af1f [Stale file handle]
[2017-03-17 13:01:52.400273] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 4ce0f196-8faf-4646-b567-46297a05ded4. (Possible split-brain)
[2017-03-17 13:01:52.401719] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:4ce0f196-8faf-4646-b567-46297a05ded4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:52.401749] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 4ce0f196-8faf-4646-b567-46297a05ded4 [Stale file handle]
[2017-03-17 13:01:52.412696] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-12: Unreadable subvolume -1 found with event generation 2 for gfid dce2dd29-770c-4a2e-9f4d-3512d505ec21. (Possible split-brain)
[2017-03-17 13:01:52.414363] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:dce2dd29-770c-4a2e-9f4d-3512d505ec21>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:52.414418] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: dce2dd29-770c-4a2e-9f4d-3512d505ec21 [Stale file handle]
[2017-03-17 13:01:52.416806] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid ce08ee3c-12de-42c2-ba11-652250608620. (Possible split-brain)
[2017-03-17 13:01:52.418564] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:ce08ee3c-12de-42c2-ba11-652250608620>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:52.418604] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: ce08ee3c-12de-42c2-ba11-652250608620 [Stale file handle]
[2017-03-17 13:01:52.439961] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 4dbd460d-12ce-486a-81a5-d3c4d6874ef8. (Possible split-brain)
[2017-03-17 13:01:52.442246] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:4dbd460d-12ce-486a-81a5-d3c4d6874ef8>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:52.442272] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 4dbd460d-12ce-486a-81a5-d3c4d6874ef8 [Stale file handle]
[2017-03-17 13:01:52.452323] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid fc922c7c-5d2c-47a3-9b6f-bcfc48cc301f. (Possible split-brain)
[2017-03-17 13:01:52.454274] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:fc922c7c-5d2c-47a3-9b6f-bcfc48cc301f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:52.454314] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: fc922c7c-5d2c-47a3-9b6f-bcfc48cc301f [Stale file handle]
[2017-03-17 13:01:52.456800] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 7767b2e9-0248-4aa3-8b4f-15cdc42c4787. (Possible split-brain)
[2017-03-17 13:01:52.458444] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:7767b2e9-0248-4aa3-8b4f-15cdc42c4787>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:01:52.458497] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 7767b2e9-0248-4aa3-8b4f-15cdc42c4787 [Stale file handle]
[2017-03-17 13:01:52.537941] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:01:52.538130] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 13:01:52.538196] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:01:52.538370] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:01:52.538599] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:01:52.538668] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-16: selecting local read_child vmware2-client-32
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-12: remote operation failed. Path: <gfid:84739892-a549-475a-9ffc-72b27f28e5fd> (84739892-a549-475a-9ffc-72b27f28e5fd) [No such file or directory]" repeated 2 times between [2017-03-17 13:01:51.327518] and [2017-03-17 13:01:51.329288]
[2017-03-17 13:02:03.316469] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:03.321709] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:02:03.322031] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:9c3ad3f0-ee0a-4e4e-8611-36055e5df26f> (9c3ad3f0-ee0a-4e4e-8611-36055e5df26f) [No such file or directory]
[2017-03-17 13:02:03.322165] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:9c3ad3f0-ee0a-4e4e-8611-36055e5df26f> (9c3ad3f0-ee0a-4e4e-8611-36055e5df26f) [No such file or directory]
[2017-03-17 13:02:03.322716] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 9c3ad3f0-ee0a-4e4e-8611-36055e5df26f. (Possible split-brain)
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:9c3ad3f0-ee0a-4e4e-8611-36055e5df26f> (9c3ad3f0-ee0a-4e4e-8611-36055e5df26f) [No such file or directory]" repeated 2 times between [2017-03-17 13:02:03.322165] and [2017-03-17 13:02:03.324225]
[2017-03-17 13:02:03.325515] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:9c3ad3f0-ee0a-4e4e-8611-36055e5df26f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:03.325580] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 9c3ad3f0-ee0a-4e4e-8611-36055e5df26f [Stale file handle]
[2017-03-17 13:02:03.446563] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:03.453240] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 13:02:03.453700] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:59afb95f-3e8a-4e18-964f-7347365d6d9d> (59afb95f-3e8a-4e18-964f-7347365d6d9d) [No such file or directory]
[2017-03-17 13:02:03.453852] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:59afb95f-3e8a-4e18-964f-7347365d6d9d> (59afb95f-3e8a-4e18-964f-7347365d6d9d) [No such file or directory]
[2017-03-17 13:02:03.454548] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 59afb95f-3e8a-4e18-964f-7347365d6d9d. (Possible split-brain)
[2017-03-17 13:02:03.454965] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:59afb95f-3e8a-4e18-964f-7347365d6d9d> (59afb95f-3e8a-4e18-964f-7347365d6d9d) [No such file or directory]
[2017-03-17 13:02:03.455119] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:59afb95f-3e8a-4e18-964f-7347365d6d9d> (59afb95f-3e8a-4e18-964f-7347365d6d9d) [No such file or directory]
[2017-03-17 13:02:03.456188] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:59afb95f-3e8a-4e18-964f-7347365d6d9d> (59afb95f-3e8a-4e18-964f-7347365d6d9d) [No such file or directory]
[2017-03-17 13:02:03.456336] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:59afb95f-3e8a-4e18-964f-7347365d6d9d> (59afb95f-3e8a-4e18-964f-7347365d6d9d) [No such file or directory]
[2017-03-17 13:02:03.457450] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:59afb95f-3e8a-4e18-964f-7347365d6d9d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:03.457507] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 59afb95f-3e8a-4e18-964f-7347365d6d9d [Stale file handle]
[2017-03-17 13:02:03.490965] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:03.497475] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:02:03.497881] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:7deb053b-98db-44fd-9d18-df2f2745d335> (7deb053b-98db-44fd-9d18-df2f2745d335) [No such file or directory]
[2017-03-17 13:02:03.497977] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:7deb053b-98db-44fd-9d18-df2f2745d335> (7deb053b-98db-44fd-9d18-df2f2745d335) [No such file or directory]
[2017-03-17 13:02:03.498562] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 7deb053b-98db-44fd-9d18-df2f2745d335. (Possible split-brain)
[2017-03-17 13:02:03.499032] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:7deb053b-98db-44fd-9d18-df2f2745d335> (7deb053b-98db-44fd-9d18-df2f2745d335) [No such file or directory]
[2017-03-17 13:02:03.499099] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:7deb053b-98db-44fd-9d18-df2f2745d335> (7deb053b-98db-44fd-9d18-df2f2745d335) [No such file or directory]
[2017-03-17 13:02:03.500234] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:7deb053b-98db-44fd-9d18-df2f2745d335> (7deb053b-98db-44fd-9d18-df2f2745d335) [No such file or directory]
[2017-03-17 13:02:03.500286] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:7deb053b-98db-44fd-9d18-df2f2745d335> (7deb053b-98db-44fd-9d18-df2f2745d335) [No such file or directory]
[2017-03-17 13:02:03.501556] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:7deb053b-98db-44fd-9d18-df2f2745d335>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:03.501621] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 7deb053b-98db-44fd-9d18-df2f2745d335 [Stale file handle]
[2017-03-17 13:02:03.528122] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:03.539740] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:02:03.540213] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:7379bf68-befa-4c84-bf36-5eb1235bd494> (7379bf68-befa-4c84-bf36-5eb1235bd494) [No such file or directory]
[2017-03-17 13:02:03.540258] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:7379bf68-befa-4c84-bf36-5eb1235bd494> (7379bf68-befa-4c84-bf36-5eb1235bd494) [No such file or directory]
[2017-03-17 13:02:03.540652] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 7379bf68-befa-4c84-bf36-5eb1235bd494. (Possible split-brain)
[2017-03-17 13:02:03.541565] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:7379bf68-befa-4c84-bf36-5eb1235bd494> (7379bf68-befa-4c84-bf36-5eb1235bd494) [No such file or directory]
[2017-03-17 13:02:03.541625] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:7379bf68-befa-4c84-bf36-5eb1235bd494> (7379bf68-befa-4c84-bf36-5eb1235bd494) [No such file or directory]
[2017-03-17 13:02:03.545047] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:7379bf68-befa-4c84-bf36-5eb1235bd494> (7379bf68-befa-4c84-bf36-5eb1235bd494) [No such file or directory]
[2017-03-17 13:02:03.545319] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:7379bf68-befa-4c84-bf36-5eb1235bd494> (7379bf68-befa-4c84-bf36-5eb1235bd494) [No such file or directory]
[2017-03-17 13:02:03.546662] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:7379bf68-befa-4c84-bf36-5eb1235bd494>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:03.546712] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 7379bf68-befa-4c84-bf36-5eb1235bd494 [Stale file handle]
[2017-03-17 13:02:03.593474] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:03.600998] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:02:03.601339] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:fb6046b2-5826-4579-bb84-aaa652265088> (fb6046b2-5826-4579-bb84-aaa652265088) [No such file or directory]
[2017-03-17 13:02:03.601461] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:fb6046b2-5826-4579-bb84-aaa652265088> (fb6046b2-5826-4579-bb84-aaa652265088) [No such file or directory]
[2017-03-17 13:02:03.601935] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid fb6046b2-5826-4579-bb84-aaa652265088. (Possible split-brain)
[2017-03-17 13:02:03.602545] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:fb6046b2-5826-4579-bb84-aaa652265088> (fb6046b2-5826-4579-bb84-aaa652265088) [No such file or directory]
[2017-03-17 13:02:03.602993] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:fb6046b2-5826-4579-bb84-aaa652265088> (fb6046b2-5826-4579-bb84-aaa652265088) [No such file or directory]
[2017-03-17 13:02:03.609220] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:fb6046b2-5826-4579-bb84-aaa652265088> (fb6046b2-5826-4579-bb84-aaa652265088) [No such file or directory]
[2017-03-17 13:02:03.609490] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:fb6046b2-5826-4579-bb84-aaa652265088> (fb6046b2-5826-4579-bb84-aaa652265088) [No such file or directory]
[2017-03-17 13:02:03.610222] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:fb6046b2-5826-4579-bb84-aaa652265088>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:03.610264] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: fb6046b2-5826-4579-bb84-aaa652265088 [Stale file handle]
[2017-03-17 13:02:03.707270] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:03.713041] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 13:02:03.713376] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:ba3f494e-2dc2-4f55-aecb-435386e88452> (ba3f494e-2dc2-4f55-aecb-435386e88452) [No such file or directory]
[2017-03-17 13:02:03.713513] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:ba3f494e-2dc2-4f55-aecb-435386e88452> (ba3f494e-2dc2-4f55-aecb-435386e88452) [No such file or directory]
[2017-03-17 13:02:03.713981] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid ba3f494e-2dc2-4f55-aecb-435386e88452. (Possible split-brain)
[2017-03-17 13:02:03.714485] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:ba3f494e-2dc2-4f55-aecb-435386e88452> (ba3f494e-2dc2-4f55-aecb-435386e88452) [No such file or directory]
[2017-03-17 13:02:03.714507] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:ba3f494e-2dc2-4f55-aecb-435386e88452> (ba3f494e-2dc2-4f55-aecb-435386e88452) [No such file or directory]
[2017-03-17 13:02:03.715500] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:ba3f494e-2dc2-4f55-aecb-435386e88452> (ba3f494e-2dc2-4f55-aecb-435386e88452) [No such file or directory]
[2017-03-17 13:02:03.715531] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:ba3f494e-2dc2-4f55-aecb-435386e88452> (ba3f494e-2dc2-4f55-aecb-435386e88452) [No such file or directory]
[2017-03-17 13:02:03.717750] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:ba3f494e-2dc2-4f55-aecb-435386e88452>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:03.717796] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: ba3f494e-2dc2-4f55-aecb-435386e88452 [Stale file handle]
[2017-03-17 13:02:03.768709] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:03.773597] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 13:02:03.774013] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:b9a4aa75-bcac-4a3d-be9e-7d5ef28de642> (b9a4aa75-bcac-4a3d-be9e-7d5ef28de642) [No such file or directory]
[2017-03-17 13:02:03.774016] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:b9a4aa75-bcac-4a3d-be9e-7d5ef28de642> (b9a4aa75-bcac-4a3d-be9e-7d5ef28de642) [No such file or directory]
[2017-03-17 13:02:03.774430] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid b9a4aa75-bcac-4a3d-be9e-7d5ef28de642. (Possible split-brain)
[2017-03-17 13:02:03.774823] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:b9a4aa75-bcac-4a3d-be9e-7d5ef28de642> (b9a4aa75-bcac-4a3d-be9e-7d5ef28de642) [No such file or directory]
[2017-03-17 13:02:03.774967] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:b9a4aa75-bcac-4a3d-be9e-7d5ef28de642> (b9a4aa75-bcac-4a3d-be9e-7d5ef28de642) [No such file or directory]
[2017-03-17 13:02:03.775836] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:b9a4aa75-bcac-4a3d-be9e-7d5ef28de642> (b9a4aa75-bcac-4a3d-be9e-7d5ef28de642) [No such file or directory]
[2017-03-17 13:02:03.775884] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:b9a4aa75-bcac-4a3d-be9e-7d5ef28de642> (b9a4aa75-bcac-4a3d-be9e-7d5ef28de642) [No such file or directory]
[2017-03-17 13:02:03.776870] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b9a4aa75-bcac-4a3d-be9e-7d5ef28de642>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:03.776930] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b9a4aa75-bcac-4a3d-be9e-7d5ef28de642 [Stale file handle]
[2017-03-17 13:02:03.863974] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:03.873337] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 13:02:03.873658] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:46ee6295-e0b6-4f20-9570-cac5c91691de> (46ee6295-e0b6-4f20-9570-cac5c91691de) [No such file or directory]
[2017-03-17 13:02:03.873681] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:46ee6295-e0b6-4f20-9570-cac5c91691de> (46ee6295-e0b6-4f20-9570-cac5c91691de) [No such file or directory]
[2017-03-17 13:02:03.874105] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 46ee6295-e0b6-4f20-9570-cac5c91691de. (Possible split-brain)
[2017-03-17 13:02:03.874403] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:46ee6295-e0b6-4f20-9570-cac5c91691de> (46ee6295-e0b6-4f20-9570-cac5c91691de) [No such file or directory]
[2017-03-17 13:02:03.874468] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:46ee6295-e0b6-4f20-9570-cac5c91691de> (46ee6295-e0b6-4f20-9570-cac5c91691de) [No such file or directory]
[2017-03-17 13:02:03.880271] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:46ee6295-e0b6-4f20-9570-cac5c91691de> (46ee6295-e0b6-4f20-9570-cac5c91691de) [No such file or directory]
[2017-03-17 13:02:03.880334] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:46ee6295-e0b6-4f20-9570-cac5c91691de> (46ee6295-e0b6-4f20-9570-cac5c91691de) [No such file or directory]
[2017-03-17 13:02:03.881634] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:46ee6295-e0b6-4f20-9570-cac5c91691de>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:03.881677] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 46ee6295-e0b6-4f20-9570-cac5c91691de [Stale file handle]
[2017-03-17 13:02:03.972572] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:03.978449] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 13:02:03.978894] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:07a892cf-1e3f-484e-9230-b41377a3bcfc> (07a892cf-1e3f-484e-9230-b41377a3bcfc) [No such file or directory]
[2017-03-17 13:02:03.979394] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:07a892cf-1e3f-484e-9230-b41377a3bcfc> (07a892cf-1e3f-484e-9230-b41377a3bcfc) [No such file or directory]
[2017-03-17 13:02:03.979948] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 07a892cf-1e3f-484e-9230-b41377a3bcfc. (Possible split-brain)
[2017-03-17 13:02:03.984313] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:07a892cf-1e3f-484e-9230-b41377a3bcfc> (07a892cf-1e3f-484e-9230-b41377a3bcfc) [No such file or directory]
[2017-03-17 13:02:03.984436] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:07a892cf-1e3f-484e-9230-b41377a3bcfc> (07a892cf-1e3f-484e-9230-b41377a3bcfc) [No such file or directory]
[2017-03-17 13:02:03.985108] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:07a892cf-1e3f-484e-9230-b41377a3bcfc> (07a892cf-1e3f-484e-9230-b41377a3bcfc) [No such file or directory]
[2017-03-17 13:02:03.985209] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:07a892cf-1e3f-484e-9230-b41377a3bcfc> (07a892cf-1e3f-484e-9230-b41377a3bcfc) [No such file or directory]
[2017-03-17 13:02:03.985962] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:07a892cf-1e3f-484e-9230-b41377a3bcfc>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:03.986002] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 07a892cf-1e3f-484e-9230-b41377a3bcfc [Stale file handle]
[2017-03-17 13:02:04.076489] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:04.081906] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:02:04.082194] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:be2429a8-03c3-43be-91b8-f923230dce3f> (be2429a8-03c3-43be-91b8-f923230dce3f) [No such file or directory]
[2017-03-17 13:02:04.082358] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:be2429a8-03c3-43be-91b8-f923230dce3f> (be2429a8-03c3-43be-91b8-f923230dce3f) [No such file or directory]
[2017-03-17 13:02:04.082775] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid be2429a8-03c3-43be-91b8-f923230dce3f. (Possible split-brain)
[2017-03-17 13:02:04.083118] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:be2429a8-03c3-43be-91b8-f923230dce3f> (be2429a8-03c3-43be-91b8-f923230dce3f) [No such file or directory]
[2017-03-17 13:02:04.083227] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:be2429a8-03c3-43be-91b8-f923230dce3f> (be2429a8-03c3-43be-91b8-f923230dce3f) [No such file or directory]
[2017-03-17 13:02:04.084087] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:be2429a8-03c3-43be-91b8-f923230dce3f> (be2429a8-03c3-43be-91b8-f923230dce3f) [No such file or directory]
[2017-03-17 13:02:04.084114] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:be2429a8-03c3-43be-91b8-f923230dce3f> (be2429a8-03c3-43be-91b8-f923230dce3f) [No such file or directory]
[2017-03-17 13:02:04.085137] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:be2429a8-03c3-43be-91b8-f923230dce3f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:04.085169] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: be2429a8-03c3-43be-91b8-f923230dce3f [Stale file handle]
[2017-03-17 13:02:04.108839] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:04.115532] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:02:04.115826] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:ac1646e0-4224-4017-a979-6f7b5bdb5834> (ac1646e0-4224-4017-a979-6f7b5bdb5834) [No such file or directory]
[2017-03-17 13:02:04.115847] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:ac1646e0-4224-4017-a979-6f7b5bdb5834> (ac1646e0-4224-4017-a979-6f7b5bdb5834) [No such file or directory]
[2017-03-17 13:02:04.116164] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid ac1646e0-4224-4017-a979-6f7b5bdb5834. (Possible split-brain)
[2017-03-17 13:02:04.116400] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:ac1646e0-4224-4017-a979-6f7b5bdb5834> (ac1646e0-4224-4017-a979-6f7b5bdb5834) [No such file or directory]
[2017-03-17 13:02:04.116499] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:ac1646e0-4224-4017-a979-6f7b5bdb5834> (ac1646e0-4224-4017-a979-6f7b5bdb5834) [No such file or directory]
[2017-03-17 13:02:04.117277] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:ac1646e0-4224-4017-a979-6f7b5bdb5834> (ac1646e0-4224-4017-a979-6f7b5bdb5834) [No such file or directory]
[2017-03-17 13:02:04.117303] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:ac1646e0-4224-4017-a979-6f7b5bdb5834> (ac1646e0-4224-4017-a979-6f7b5bdb5834) [No such file or directory]
[2017-03-17 13:02:04.118237] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:ac1646e0-4224-4017-a979-6f7b5bdb5834>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:04.118290] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: ac1646e0-4224-4017-a979-6f7b5bdb5834 [Stale file handle]
[2017-03-17 13:02:04.143935] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:04.149457] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:02:04.150132] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:bacf914c-1705-40ea-bf56-af0131773635> (bacf914c-1705-40ea-bf56-af0131773635) [No such file or directory]
[2017-03-17 13:02:04.150175] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:bacf914c-1705-40ea-bf56-af0131773635> (bacf914c-1705-40ea-bf56-af0131773635) [No such file or directory]
[2017-03-17 13:02:04.150480] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid bacf914c-1705-40ea-bf56-af0131773635. (Possible split-brain)
[2017-03-17 13:02:04.150774] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:bacf914c-1705-40ea-bf56-af0131773635> (bacf914c-1705-40ea-bf56-af0131773635) [No such file or directory]
[2017-03-17 13:02:04.150817] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:bacf914c-1705-40ea-bf56-af0131773635> (bacf914c-1705-40ea-bf56-af0131773635) [No such file or directory]
[2017-03-17 13:02:04.151643] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:bacf914c-1705-40ea-bf56-af0131773635> (bacf914c-1705-40ea-bf56-af0131773635) [No such file or directory]
[2017-03-17 13:02:04.151700] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:bacf914c-1705-40ea-bf56-af0131773635> (bacf914c-1705-40ea-bf56-af0131773635) [No such file or directory]
[2017-03-17 13:02:04.152620] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:bacf914c-1705-40ea-bf56-af0131773635>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:04.152680] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: bacf914c-1705-40ea-bf56-af0131773635 [Stale file handle]
[2017-03-17 13:02:04.181541] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:02:04.188586] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 13:02:04.188928] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:be967a29-bf0e-46d2-bece-4a2842e94add> (be967a29-bf0e-46d2-bece-4a2842e94add) [No such file or directory]
[2017-03-17 13:02:04.188939] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:be967a29-bf0e-46d2-bece-4a2842e94add> (be967a29-bf0e-46d2-bece-4a2842e94add) [No such file or directory]
[2017-03-17 13:02:04.189374] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid be967a29-bf0e-46d2-bece-4a2842e94add. (Possible split-brain)
[2017-03-17 13:02:04.189696] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:be967a29-bf0e-46d2-bece-4a2842e94add> (be967a29-bf0e-46d2-bece-4a2842e94add) [No such file or directory]
[2017-03-17 13:02:04.189735] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:be967a29-bf0e-46d2-bece-4a2842e94add> (be967a29-bf0e-46d2-bece-4a2842e94add) [No such file or directory]
[2017-03-17 13:02:04.190784] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:be967a29-bf0e-46d2-bece-4a2842e94add> (be967a29-bf0e-46d2-bece-4a2842e94add) [No such file or directory]
[2017-03-17 13:02:04.190854] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:be967a29-bf0e-46d2-bece-4a2842e94add> (be967a29-bf0e-46d2-bece-4a2842e94add) [No such file or directory]
[2017-03-17 13:02:04.191914] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:be967a29-bf0e-46d2-bece-4a2842e94add>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:04.191961] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: be967a29-bf0e-46d2-bece-4a2842e94add [Stale file handle]
[2017-03-17 13:02:15.608735] I [MSGID: 109045] [dht-common.c:2012:dht_lookup_everywhere_cbk] 10-vmware2-dht: attempting deletion of stale linkfile /.shard/5394ac3a-ece1-4f08-b14a-b0c7d90d6a35.19 on vmware2-replicate-3 (hashed subvol is vmware2-replicate-14)
[2017-03-17 13:02:15.610843] I [MSGID: 109069] [dht-common.c:1324:dht_lookup_unlink_cbk] 10-vmware2-dht: lookup_unlink returned with op_ret -> 0 and op-errno -> 0 for /.shard/5394ac3a-ece1-4f08-b14a-b0c7d90d6a35.19
[2017-03-17 13:02:15.699192] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-11: Unreadable subvolume -1 found with event generation 2 for gfid 5394ac3a-ece1-4f08-b14a-b0c7d90d6a35. (Possible split-brain)
[2017-03-17 13:02:15.701013] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:5394ac3a-ece1-4f08-b14a-b0c7d90d6a35>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:15.701056] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 5394ac3a-ece1-4f08-b14a-b0c7d90d6a35 [Stale file handle]
[2017-03-17 13:02:15.704696] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 81ef6ffd-c177-40f7-9a60-b498aa21ecf5. (Possible split-brain)
[2017-03-17 13:02:15.706003] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:81ef6ffd-c177-40f7-9a60-b498aa21ecf5>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:15.706034] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 81ef6ffd-c177-40f7-9a60-b498aa21ecf5 [Stale file handle]
[2017-03-17 13:02:15.740112] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid adc57041-620c-45bd-a7b3-d55df1e0f699. (Possible split-brain)
[2017-03-17 13:02:15.741921] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:adc57041-620c-45bd-a7b3-d55df1e0f699>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:15.741951] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: adc57041-620c-45bd-a7b3-d55df1e0f699 [Stale file handle]
[2017-03-17 13:02:15.744379] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 1341744c-8da3-4a73-a77b-18a3b9b98830. (Possible split-brain)
[2017-03-17 13:02:15.746387] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:1341744c-8da3-4a73-a77b-18a3b9b98830>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:15.746418] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 1341744c-8da3-4a73-a77b-18a3b9b98830 [Stale file handle]
[2017-03-17 13:02:15.899458] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 994e99c6-80c4-4128-851e-fdd18f848090. (Possible split-brain)
[2017-03-17 13:02:15.901272] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:994e99c6-80c4-4128-851e-fdd18f848090>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:15.901315] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 994e99c6-80c4-4128-851e-fdd18f848090 [Stale file handle]
[2017-03-17 13:02:15.904409] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-11: Unreadable subvolume -1 found with event generation 2 for gfid bddd776f-77b9-47e0-89bd-8c528ec39b27. (Possible split-brain)
[2017-03-17 13:02:15.906459] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:bddd776f-77b9-47e0-89bd-8c528ec39b27>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:15.906499] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: bddd776f-77b9-47e0-89bd-8c528ec39b27 [Stale file handle]
[2017-03-17 13:02:15.935978] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-12: Unreadable subvolume -1 found with event generation 2 for gfid 84c7ccc5-443c-4843-8b97-90ef126644a3. (Possible split-brain)
[2017-03-17 13:02:15.937492] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:84c7ccc5-443c-4843-8b97-90ef126644a3>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:15.937523] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 84c7ccc5-443c-4843-8b97-90ef126644a3 [Stale file handle]
[2017-03-17 13:02:15.939769] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid 15ae4e2c-ce6c-4314-ac1d-3d821a443598. (Possible split-brain)
[2017-03-17 13:02:15.942019] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:15ae4e2c-ce6c-4314-ac1d-3d821a443598>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:15.942070] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 15ae4e2c-ce6c-4314-ac1d-3d821a443598 [Stale file handle]
[2017-03-17 13:02:16.077952] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 8a036b19-77a6-4709-aab3-849bd0afe8f9. (Possible split-brain)
[2017-03-17 13:02:16.083262] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:8a036b19-77a6-4709-aab3-849bd0afe8f9>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.083304] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 8a036b19-77a6-4709-aab3-849bd0afe8f9 [Stale file handle]
[2017-03-17 13:02:16.089451] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid bb57533a-206e-4fd5-8ad1-4e154c8423a1. (Possible split-brain)
[2017-03-17 13:02:16.091932] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:bb57533a-206e-4fd5-8ad1-4e154c8423a1>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.091969] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: bb57533a-206e-4fd5-8ad1-4e154c8423a1 [Stale file handle]
[2017-03-17 13:02:16.125627] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 11280572-607f-4951-b5e6-88da37e7cab9. (Possible split-brain)
[2017-03-17 13:02:16.127848] I [MSGID: 109063] [dht-layout.c:713:dht_layout_normalize] 10-vmware2-dht: Found anomalies in <gfid:11280572-607f-4951-b5e6-88da37e7cab9> (gfid = 11280572-607f-4951-b5e6-88da37e7cab9). Holes=1 overlaps=0
[2017-03-17 13:02:16.127879] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:11280572-607f-4951-b5e6-88da37e7cab9>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.127906] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 11280572-607f-4951-b5e6-88da37e7cab9 [Stale file handle]
[2017-03-17 13:02:16.131917] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 0525c89b-b090-4b0c-b96a-3e550a5c151d. (Possible split-brain)
[2017-03-17 13:02:16.133978] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:0525c89b-b090-4b0c-b96a-3e550a5c151d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.134019] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 0525c89b-b090-4b0c-b96a-3e550a5c151d [Stale file handle]
[2017-03-17 13:02:16.246286] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 877544d3-d928-4888-a9f0-ec9ec307affb. (Possible split-brain)
[2017-03-17 13:02:16.248085] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:877544d3-d928-4888-a9f0-ec9ec307affb>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.248130] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 877544d3-d928-4888-a9f0-ec9ec307affb [Stale file handle]
[2017-03-17 13:02:16.250976] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 6441e418-8ec1-4faf-80ec-0aeab96bfd19. (Possible split-brain)
[2017-03-17 13:02:16.252890] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:6441e418-8ec1-4faf-80ec-0aeab96bfd19>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.252930] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 6441e418-8ec1-4faf-80ec-0aeab96bfd19 [Stale file handle]
[2017-03-17 13:02:16.283369] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 8f063e35-2bcc-44c9-9fde-6e5832ef5578. (Possible split-brain)
[2017-03-17 13:02:16.285323] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:8f063e35-2bcc-44c9-9fde-6e5832ef5578>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.285355] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 8f063e35-2bcc-44c9-9fde-6e5832ef5578 [Stale file handle]
[2017-03-17 13:02:16.288491] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 462146e7-9e57-4e82-8213-03595e51a0e1. (Possible split-brain)
[2017-03-17 13:02:16.290334] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:462146e7-9e57-4e82-8213-03595e51a0e1>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.290365] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 462146e7-9e57-4e82-8213-03595e51a0e1 [Stale file handle]
[2017-03-17 13:02:16.389691] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 89a78fe1-b2b5-476c-9eaa-88c32144bc6a. (Possible split-brain)
[2017-03-17 13:02:16.391185] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:89a78fe1-b2b5-476c-9eaa-88c32144bc6a>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.391216] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 89a78fe1-b2b5-476c-9eaa-88c32144bc6a [Stale file handle]
[2017-03-17 13:02:16.394328] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid cf8280f6-e5da-4655-8776-696e44fe7fbc. (Possible split-brain)
[2017-03-17 13:02:16.396122] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:cf8280f6-e5da-4655-8776-696e44fe7fbc>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.396155] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: cf8280f6-e5da-4655-8776-696e44fe7fbc [Stale file handle]
[2017-03-17 13:02:16.418157] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-11: Unreadable subvolume -1 found with event generation 2 for gfid bdf1eea2-2224-4fee-96c2-220b4ae67b73. (Possible split-brain)
[2017-03-17 13:02:16.419462] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:bdf1eea2-2224-4fee-96c2-220b4ae67b73>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.419493] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: bdf1eea2-2224-4fee-96c2-220b4ae67b73 [Stale file handle]
[2017-03-17 13:02:16.421676] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid 029ac107-4843-4988-aef6-c0d5aa4734f3. (Possible split-brain)
[2017-03-17 13:02:16.425291] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:029ac107-4843-4988-aef6-c0d5aa4734f3>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.425315] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 029ac107-4843-4988-aef6-c0d5aa4734f3 [Stale file handle]
[2017-03-17 13:02:16.527965] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 146bbc82-0b94-4c0b-9b7a-c2827a2dbc23. (Possible split-brain)
[2017-03-17 13:02:16.530210] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:146bbc82-0b94-4c0b-9b7a-c2827a2dbc23>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.530260] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 146bbc82-0b94-4c0b-9b7a-c2827a2dbc23 [Stale file handle]
[2017-03-17 13:02:16.533478] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid bbbaf080-cfcd-4b3f-acd5-874708191024. (Possible split-brain)
[2017-03-17 13:02:16.535421] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:bbbaf080-cfcd-4b3f-acd5-874708191024>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.535456] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: bbbaf080-cfcd-4b3f-acd5-874708191024 [Stale file handle]
[2017-03-17 13:02:16.567382] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-11: Unreadable subvolume -1 found with event generation 2 for gfid e761e623-ff7a-4dc1-b2a2-cb6fc07f4a00. (Possible split-brain)
[2017-03-17 13:02:16.569467] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:e761e623-ff7a-4dc1-b2a2-cb6fc07f4a00>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.569496] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: e761e623-ff7a-4dc1-b2a2-cb6fc07f4a00 [Stale file handle]
[2017-03-17 13:02:16.571861] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid c3d5c8ed-d501-4b62-a7e6-5003340eb88f. (Possible split-brain)
[2017-03-17 13:02:16.573741] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:c3d5c8ed-d501-4b62-a7e6-5003340eb88f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.573773] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: c3d5c8ed-d501-4b62-a7e6-5003340eb88f [Stale file handle]
[2017-03-17 13:02:16.638705] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:02:16.638996] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 13:02:16.639046] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:02:16.639137] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 13:02:16.639194] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:02:16.639282] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:02:16.639351] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:02:16.639479] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:02:16.639510] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:02:16.639542] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:02:16.752344] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid b5292d43-1d47-45fa-bbcf-2c1daec0cf77. (Possible split-brain)
[2017-03-17 13:02:16.753916] I [MSGID: 109063] [dht-layout.c:713:dht_layout_normalize] 10-vmware2-dht: Found anomalies in <gfid:b5292d43-1d47-45fa-bbcf-2c1daec0cf77> (gfid = b5292d43-1d47-45fa-bbcf-2c1daec0cf77). Holes=1 overlaps=0
[2017-03-17 13:02:16.753956] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b5292d43-1d47-45fa-bbcf-2c1daec0cf77>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.753987] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b5292d43-1d47-45fa-bbcf-2c1daec0cf77 [Stale file handle]
[2017-03-17 13:02:16.756850] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid 53c778e2-9008-42de-b298-2730300d032d. (Possible split-brain)
[2017-03-17 13:02:16.758964] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:53c778e2-9008-42de-b298-2730300d032d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.758996] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 53c778e2-9008-42de-b298-2730300d032d [Stale file handle]
[2017-03-17 13:02:16.789687] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-3: Unreadable subvolume -1 found with event generation 2 for gfid ddca6658-45e5-4508-820f-bea48759c62d. (Possible split-brain)
[2017-03-17 13:02:16.791520] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:ddca6658-45e5-4508-820f-bea48759c62d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.791547] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: ddca6658-45e5-4508-820f-bea48759c62d [Stale file handle]
[2017-03-17 13:02:16.794014] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid c5293cb6-081f-46b9-8091-cc35211f226d. (Possible split-brain)
[2017-03-17 13:02:16.795468] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:c5293cb6-081f-46b9-8091-cc35211f226d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.795500] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: c5293cb6-081f-46b9-8091-cc35211f226d [Stale file handle]
[2017-03-17 13:02:16.803547] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid 16348bbe-b135-466a-8a9e-b88ecaf2111d. (Possible split-brain)
[2017-03-17 13:02:16.806760] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:16348bbe-b135-466a-8a9e-b88ecaf2111d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.806794] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 16348bbe-b135-466a-8a9e-b88ecaf2111d [Stale file handle]
[2017-03-17 13:02:16.812148] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 6583aba2-6df1-41ca-a6fd-00beb44b5e8a. (Possible split-brain)
[2017-03-17 13:02:16.813773] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:6583aba2-6df1-41ca-a6fd-00beb44b5e8a>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.813831] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 6583aba2-6df1-41ca-a6fd-00beb44b5e8a [Stale file handle]
[2017-03-17 13:02:16.846858] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid f8ffe247-853e-4e58-b02a-2b545813fbe2. (Possible split-brain)
[2017-03-17 13:02:16.848740] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:f8ffe247-853e-4e58-b02a-2b545813fbe2>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.848771] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: f8ffe247-853e-4e58-b02a-2b545813fbe2 [Stale file handle]
[2017-03-17 13:02:16.852328] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid 943c8f5c-395c-445b-b273-f33bef3cfe91. (Possible split-brain)
[2017-03-17 13:02:16.854555] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:943c8f5c-395c-445b-b273-f33bef3cfe91>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.854579] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 943c8f5c-395c-445b-b273-f33bef3cfe91 [Stale file handle]
[2017-03-17 13:02:16.887752] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 0366e7f8-8ad0-430d-88a9-8ba685083066. (Possible split-brain)
[2017-03-17 13:02:16.889868] I [MSGID: 109063] [dht-layout.c:713:dht_layout_normalize] 10-vmware2-dht: Found anomalies in <gfid:0366e7f8-8ad0-430d-88a9-8ba685083066> (gfid = 0366e7f8-8ad0-430d-88a9-8ba685083066). Holes=1 overlaps=0
[2017-03-17 13:02:16.889994] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:0366e7f8-8ad0-430d-88a9-8ba685083066>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.890030] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 0366e7f8-8ad0-430d-88a9-8ba685083066 [Stale file handle]
[2017-03-17 13:02:16.893139] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid efc0e2f6-32a8-43f0-9a98-8a29b3f11035. (Possible split-brain)
[2017-03-17 13:02:16.894803] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:efc0e2f6-32a8-43f0-9a98-8a29b3f11035>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.894829] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: efc0e2f6-32a8-43f0-9a98-8a29b3f11035 [Stale file handle]
[2017-03-17 13:02:16.930020] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid 32671af8-f2f3-4949-bdd7-fe113f7b2836. (Possible split-brain)
[2017-03-17 13:02:16.932453] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:32671af8-f2f3-4949-bdd7-fe113f7b2836>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.932495] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 32671af8-f2f3-4949-bdd7-fe113f7b2836 [Stale file handle]
[2017-03-17 13:02:16.935670] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 254af70f-6e46-4aa7-bbbf-57987ff0f418. (Possible split-brain)
[2017-03-17 13:02:16.937999] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:254af70f-6e46-4aa7-bbbf-57987ff0f418>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.938024] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 254af70f-6e46-4aa7-bbbf-57987ff0f418 [Stale file handle]
[2017-03-17 13:02:16.971017] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid f7fc20f0-9426-4647-a065-d62d62a41b0a. (Possible split-brain)
[2017-03-17 13:02:16.973437] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:f7fc20f0-9426-4647-a065-d62d62a41b0a>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.973468] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: f7fc20f0-9426-4647-a065-d62d62a41b0a [Stale file handle]
[2017-03-17 13:02:16.976358] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 063cc9dd-af05-4f43-97a7-331b98836dfe. (Possible split-brain)
[2017-03-17 13:02:16.977891] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:063cc9dd-af05-4f43-97a7-331b98836dfe>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:16.977941] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 063cc9dd-af05-4f43-97a7-331b98836dfe [Stale file handle]
[2017-03-17 13:02:17.009813] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid 2a23e9eb-08a2-4f63-bd5d-4673b40fd6eb. (Possible split-brain)
[2017-03-17 13:02:17.011768] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:2a23e9eb-08a2-4f63-bd5d-4673b40fd6eb>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:17.011799] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 2a23e9eb-08a2-4f63-bd5d-4673b40fd6eb [Stale file handle]
[2017-03-17 13:02:17.015235] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 16be971c-85fb-452e-aac1-650567cfcb28. (Possible split-brain)
[2017-03-17 13:02:17.016932] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:16be971c-85fb-452e-aac1-650567cfcb28>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:17.016966] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 16be971c-85fb-452e-aac1-650567cfcb28 [Stale file handle]
[2017-03-17 13:02:17.070249] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 7f283e41-efa7-440b-ad75-86d2f0aaf714. (Possible split-brain)
[2017-03-17 13:02:17.072330] I [MSGID: 109063] [dht-layout.c:713:dht_layout_normalize] 10-vmware2-dht: Found anomalies in <gfid:7f283e41-efa7-440b-ad75-86d2f0aaf714> (gfid = 7f283e41-efa7-440b-ad75-86d2f0aaf714). Holes=1 overlaps=0
[2017-03-17 13:02:17.072361] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:7f283e41-efa7-440b-ad75-86d2f0aaf714>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:17.072390] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 7f283e41-efa7-440b-ad75-86d2f0aaf714 [Stale file handle]
[2017-03-17 13:02:17.075359] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid b7b8934a-9619-468b-be37-fde2e696cabb. (Possible split-brain)
[2017-03-17 13:02:17.077650] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b7b8934a-9619-468b-be37-fde2e696cabb>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:17.077694] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b7b8934a-9619-468b-be37-fde2e696cabb [Stale file handle]
[2017-03-17 13:02:17.119485] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid 29482b10-6807-410f-883b-a858d7465feb. (Possible split-brain)
[2017-03-17 13:02:17.121020] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:29482b10-6807-410f-883b-a858d7465feb>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:17.121054] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 29482b10-6807-410f-883b-a858d7465feb [Stale file handle]
[2017-03-17 13:02:17.124088] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 5070af3b-1945-40b4-8595-bac280f0bbf3. (Possible split-brain)
[2017-03-17 13:02:17.126201] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:5070af3b-1945-40b4-8595-bac280f0bbf3>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:17.126239] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 5070af3b-1945-40b4-8595-bac280f0bbf3 [Stale file handle]
[2017-03-17 13:02:18.168375] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:02:18.168482] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:02:18.168814] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 13:02:18.168949] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 13:02:18.169073] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:02:18.169168] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:02:18.169183] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:02:18.169325] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:02:18.169344] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:02:18.169364] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:02:27.700058] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmdk~ (hash=vmware2-replicate-2/cache=vmware2-replicate-2) => /gfstest/gfstest.vmdk (hash=vmware2-replicate-8/cache=vmware2-replicate-2)
[2017-03-17 13:02:27.704873] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 13:02:27.705135] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:4963a9ff-62e3-4273-a1fc-862b92dae614> (4963a9ff-62e3-4273-a1fc-862b92dae614) [No such file or directory]
[2017-03-17 13:02:27.705655] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:4963a9ff-62e3-4273-a1fc-862b92dae614> (4963a9ff-62e3-4273-a1fc-862b92dae614) [No such file or directory]
[2017-03-17 13:02:27.706334] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 4963a9ff-62e3-4273-a1fc-862b92dae614. (Possible split-brain)
[2017-03-17 13:02:27.706562] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:4963a9ff-62e3-4273-a1fc-862b92dae614> (4963a9ff-62e3-4273-a1fc-862b92dae614) [No such file or directory]
[2017-03-17 13:02:27.706842] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:4963a9ff-62e3-4273-a1fc-862b92dae614> (4963a9ff-62e3-4273-a1fc-862b92dae614) [No such file or directory]
[2017-03-17 13:02:27.707290] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-4: remote operation failed. Path: <gfid:4963a9ff-62e3-4273-a1fc-862b92dae614> (4963a9ff-62e3-4273-a1fc-862b92dae614) [No such file or directory]
[2017-03-17 13:02:27.707467] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-5: remote operation failed. Path: <gfid:4963a9ff-62e3-4273-a1fc-862b92dae614> (4963a9ff-62e3-4273-a1fc-862b92dae614) [No such file or directory]
[2017-03-17 13:02:27.708384] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:4963a9ff-62e3-4273-a1fc-862b92dae614>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:27.708434] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 4963a9ff-62e3-4273-a1fc-862b92dae614 [Stale file handle]
[2017-03-17 13:02:27.711336] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f6e1105b-8698-48e9-abaa-cac8daab9e14. (Possible split-brain)
[2017-03-17 13:02:27.713788] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:f6e1105b-8698-48e9-abaa-cac8daab9e14>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:02:27.713818] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: f6e1105b-8698-48e9-abaa-cac8daab9e14 [Stale file handle]
[2017-03-17 13:02:54.508628] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:02:54.508836] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 13:03:20.674431] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:03:20.679870] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 13:03:20.680072] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:459a0b53-c8a0-4593-bf89-0b91108def1f> (459a0b53-c8a0-4593-bf89-0b91108def1f) [No such file or directory]
[2017-03-17 13:03:20.680171] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:459a0b53-c8a0-4593-bf89-0b91108def1f> (459a0b53-c8a0-4593-bf89-0b91108def1f) [No such file or directory]
[2017-03-17 13:03:20.680863] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 459a0b53-c8a0-4593-bf89-0b91108def1f. (Possible split-brain)
[2017-03-17 13:03:20.681143] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:459a0b53-c8a0-4593-bf89-0b91108def1f> (459a0b53-c8a0-4593-bf89-0b91108def1f) [No such file or directory]
[2017-03-17 13:03:20.681154] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:459a0b53-c8a0-4593-bf89-0b91108def1f> (459a0b53-c8a0-4593-bf89-0b91108def1f) [No such file or directory]
[2017-03-17 13:03:20.682019] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-15: remote operation failed. Path: <gfid:459a0b53-c8a0-4593-bf89-0b91108def1f> (459a0b53-c8a0-4593-bf89-0b91108def1f) [No such file or directory]
[2017-03-17 13:03:20.682216] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:459a0b53-c8a0-4593-bf89-0b91108def1f> (459a0b53-c8a0-4593-bf89-0b91108def1f) [No such file or directory]
[2017-03-17 13:03:20.682801] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:459a0b53-c8a0-4593-bf89-0b91108def1f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:03:20.682864] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 459a0b53-c8a0-4593-bf89-0b91108def1f [Stale file handle]
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-14: remote operation failed. Path: <gfid:9c3ad3f0-ee0a-4e4e-8611-36055e5df26f> (9c3ad3f0-ee0a-4e4e-8611-36055e5df26f) [No such file or directory]" repeated 2 times between [2017-03-17 13:02:03.322031] and [2017-03-17 13:02:03.324209]
[2017-03-17 13:06:28.423629] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmdk~ (hash=vmware2-replicate-18/cache=vmware2-replicate-18) => /New Virtual Machine/New Virtual Machine.vmdk (hash=vmware2-replicate-16/cache=vmware2-replicate-18)
[2017-03-17 13:06:28.427752] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-36: remote operation failed [No such file or directory]
[2017-03-17 13:06:28.428013] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:71a5094c-ad30-45d4-b463-48006ee24b2f> (71a5094c-ad30-45d4-b463-48006ee24b2f) [No such file or directory]
[2017-03-17 13:06:28.428032] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:71a5094c-ad30-45d4-b463-48006ee24b2f> (71a5094c-ad30-45d4-b463-48006ee24b2f) [No such file or directory]
[2017-03-17 13:06:28.428721] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid 71a5094c-ad30-45d4-b463-48006ee24b2f. (Possible split-brain)
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:71a5094c-ad30-45d4-b463-48006ee24b2f> (71a5094c-ad30-45d4-b463-48006ee24b2f) [No such file or directory]" repeated 2 times between [2017-03-17 13:06:28.428032] and [2017-03-17 13:06:28.430900]
[2017-03-17 13:06:28.431025] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:71a5094c-ad30-45d4-b463-48006ee24b2f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:06:28.431112] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 71a5094c-ad30-45d4-b463-48006ee24b2f [Stale file handle]
[2017-03-17 13:06:28.433595] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid a7c528f6-ef05-45f5-96ef-448681cf204f. (Possible split-brain)
[2017-03-17 13:06:28.435518] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:a7c528f6-ef05-45f5-96ef-448681cf204f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:06:28.435563] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: a7c528f6-ef05-45f5-96ef-448681cf204f [Stale file handle]
[2017-03-17 13:06:28.437840] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 28007560-72f7-4618-bd3f-ead302d0bd87. (Possible split-brain)
[2017-03-17 13:06:28.439309] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:28007560-72f7-4618-bd3f-ead302d0bd87>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:06:28.439358] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 28007560-72f7-4618-bd3f-ead302d0bd87 [Stale file handle]
[2017-03-17 13:06:28.574240] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-0/cache=vmware2-replicate-0) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-17)
[2017-03-17 13:06:28.582300] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid b94fdbbf-d1a8-4233-be9e-ab72087692ca. (Possible split-brain)
[2017-03-17 13:06:28.584795] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b94fdbbf-d1a8-4233-be9e-ab72087692ca>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:06:28.584845] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b94fdbbf-d1a8-4233-be9e-ab72087692ca [Stale file handle]
[2017-03-17 13:06:28.588245] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid b6b2b5ec-be37-4aa8-8d73-c67a969efe7e. (Possible split-brain)
[2017-03-17 13:06:28.590353] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:b6b2b5ec-be37-4aa8-8d73-c67a969efe7e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:06:28.590419] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: b6b2b5ec-be37-4aa8-8d73-c67a969efe7e [Stale file handle]
[2017-03-17 13:06:28.624085] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:06:28.624149] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:06:28.624370] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:06:28.686665] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 47e7f9ef-29d2-40ab-a9c9-f5d8f64f4a5d. (Possible split-brain)
[2017-03-17 13:06:28.689274] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:47e7f9ef-29d2-40ab-a9c9-f5d8f64f4a5d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:06:28.689326] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 47e7f9ef-29d2-40ab-a9c9-f5d8f64f4a5d [Stale file handle]
[2017-03-17 13:06:28.735153] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-0/cache=vmware2-replicate-0) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-0)
[2017-03-17 13:06:28.741889] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-0: remote operation failed [No such file or directory]
[2017-03-17 13:06:28.742282] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:a60daec3-87fa-414e-b981-b0a9205c604b> (a60daec3-87fa-414e-b981-b0a9205c604b) [No such file or directory]
[2017-03-17 13:06:28.742396] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:a60daec3-87fa-414e-b981-b0a9205c604b> (a60daec3-87fa-414e-b981-b0a9205c604b) [No such file or directory]
[2017-03-17 13:06:28.742780] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid a60daec3-87fa-414e-b981-b0a9205c604b. (Possible split-brain)
[2017-03-17 13:06:28.743225] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:a60daec3-87fa-414e-b981-b0a9205c604b> (a60daec3-87fa-414e-b981-b0a9205c604b) [No such file or directory]
[2017-03-17 13:06:28.743366] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:a60daec3-87fa-414e-b981-b0a9205c604b> (a60daec3-87fa-414e-b981-b0a9205c604b) [No such file or directory]
[2017-03-17 13:06:28.743829] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:a60daec3-87fa-414e-b981-b0a9205c604b> (a60daec3-87fa-414e-b981-b0a9205c604b) [No such file or directory]
[2017-03-17 13:06:28.744052] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:a60daec3-87fa-414e-b981-b0a9205c604b> (a60daec3-87fa-414e-b981-b0a9205c604b) [No such file or directory]
[2017-03-17 13:06:28.745550] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:a60daec3-87fa-414e-b981-b0a9205c604b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:06:28.745607] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: a60daec3-87fa-414e-b981-b0a9205c604b [Stale file handle]
[2017-03-17 13:06:28.749453] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 3a23b38b-9881-40ef-8bf3-34dcd77a5ee9. (Possible split-brain)
[2017-03-17 13:06:28.751883] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:3a23b38b-9881-40ef-8bf3-34dcd77a5ee9>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:06:28.751938] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 3a23b38b-9881-40ef-8bf3-34dcd77a5ee9 [Stale file handle]
[2017-03-17 13:06:29.271091] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid fdb2a0cd-9837-41f6-9494-fd9be874902f. (Possible split-brain)
[2017-03-17 13:06:29.274033] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:fdb2a0cd-9837-41f6-9494-fd9be874902f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:06:29.274114] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: fdb2a0cd-9837-41f6-9494-fd9be874902f [Stale file handle]
[2017-03-17 13:06:29.278126] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 33fd068f-53e5-45f2-a617-e5ebb6a5bb3f. (Possible split-brain)
[2017-03-17 13:06:29.281077] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:33fd068f-53e5-45f2-a617-e5ebb6a5bb3f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:06:29.281134] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 33fd068f-53e5-45f2-a617-e5ebb6a5bb3f [Stale file handle]
[2017-03-17 13:06:29.324167] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:06:29.325066] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-14: selecting local read_child vmware2-client-28
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:71a5094c-ad30-45d4-b463-48006ee24b2f> (71a5094c-ad30-45d4-b463-48006ee24b2f) [No such file or directory]" repeated 2 times between [2017-03-17 13:06:28.428013] and [2017-03-17 13:06:28.430894]
[2017-03-17 13:08:47.095252] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-0/cache=vmware2-replicate-0) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-0)
[2017-03-17 13:08:47.103101] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-0: remote operation failed [No such file or directory]
[2017-03-17 13:08:47.103678] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:8e5ffc66-bb8b-4467-bab6-7cbfbbe9ed9d> (8e5ffc66-bb8b-4467-bab6-7cbfbbe9ed9d) [No such file or directory]
[2017-03-17 13:08:47.103684] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:8e5ffc66-bb8b-4467-bab6-7cbfbbe9ed9d> (8e5ffc66-bb8b-4467-bab6-7cbfbbe9ed9d) [No such file or directory]
[2017-03-17 13:08:47.105049] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid 8e5ffc66-bb8b-4467-bab6-7cbfbbe9ed9d. (Possible split-brain)
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:8e5ffc66-bb8b-4467-bab6-7cbfbbe9ed9d> (8e5ffc66-bb8b-4467-bab6-7cbfbbe9ed9d) [No such file or directory]" repeated 2 times between [2017-03-17 13:08:47.103684] and [2017-03-17 13:08:47.106405]
[2017-03-17 13:08:47.108240] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:8e5ffc66-bb8b-4467-bab6-7cbfbbe9ed9d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:08:47.108339] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 8e5ffc66-bb8b-4467-bab6-7cbfbbe9ed9d [Stale file handle]
[2017-03-17 13:08:47.157540] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-0: remote operation failed [No such file or directory]
[2017-03-17 13:08:47.158057] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:08649c7d-58cc-44f0-9815-6c0800e4ff7e> (08649c7d-58cc-44f0-9815-6c0800e4ff7e) [No such file or directory]
[2017-03-17 13:08:47.158102] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:08649c7d-58cc-44f0-9815-6c0800e4ff7e> (08649c7d-58cc-44f0-9815-6c0800e4ff7e) [No such file or directory]
[2017-03-17 13:08:47.158669] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid 08649c7d-58cc-44f0-9815-6c0800e4ff7e. (Possible split-brain)
[2017-03-17 13:08:47.159218] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:08649c7d-58cc-44f0-9815-6c0800e4ff7e> (08649c7d-58cc-44f0-9815-6c0800e4ff7e) [No such file or directory]
[2017-03-17 13:08:47.159336] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:08649c7d-58cc-44f0-9815-6c0800e4ff7e> (08649c7d-58cc-44f0-9815-6c0800e4ff7e) [No such file or directory]
[2017-03-17 13:08:47.159905] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:08649c7d-58cc-44f0-9815-6c0800e4ff7e> (08649c7d-58cc-44f0-9815-6c0800e4ff7e) [No such file or directory]
[2017-03-17 13:08:47.160116] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:08649c7d-58cc-44f0-9815-6c0800e4ff7e> (08649c7d-58cc-44f0-9815-6c0800e4ff7e) [No such file or directory]
[2017-03-17 13:08:47.162022] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:08649c7d-58cc-44f0-9815-6c0800e4ff7e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:08:47.162076] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 08649c7d-58cc-44f0-9815-6c0800e4ff7e [Stale file handle]
The message "I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-0/cache=vmware2-replicate-0) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-0)" repeated 2 times between [2017-03-17 13:08:47.095252] and [2017-03-17 13:08:47.191876]
[2017-03-17 13:08:47.198789] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-0: remote operation failed [No such file or directory]
[2017-03-17 13:08:47.199242] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:27a4d340-147f-454a-b581-c9525fbc49e8> (27a4d340-147f-454a-b581-c9525fbc49e8) [No such file or directory]
[2017-03-17 13:08:47.199298] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:27a4d340-147f-454a-b581-c9525fbc49e8> (27a4d340-147f-454a-b581-c9525fbc49e8) [No such file or directory]
[2017-03-17 13:08:47.199703] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid 27a4d340-147f-454a-b581-c9525fbc49e8. (Possible split-brain)
[2017-03-17 13:08:47.200220] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:27a4d340-147f-454a-b581-c9525fbc49e8> (27a4d340-147f-454a-b581-c9525fbc49e8) [No such file or directory]
[2017-03-17 13:08:47.200298] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:27a4d340-147f-454a-b581-c9525fbc49e8> (27a4d340-147f-454a-b581-c9525fbc49e8) [No such file or directory]
[2017-03-17 13:08:47.200929] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:27a4d340-147f-454a-b581-c9525fbc49e8> (27a4d340-147f-454a-b581-c9525fbc49e8) [No such file or directory]
[2017-03-17 13:08:47.201020] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:27a4d340-147f-454a-b581-c9525fbc49e8> (27a4d340-147f-454a-b581-c9525fbc49e8) [No such file or directory]
[2017-03-17 13:08:47.202732] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:27a4d340-147f-454a-b581-c9525fbc49e8>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:08:47.202824] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 27a4d340-147f-454a-b581-c9525fbc49e8 [Stale file handle]
[2017-03-17 13:08:47.247436] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:08:47.281158] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-0/cache=vmware2-replicate-0) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-0)
[2017-03-17 13:08:47.288365] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-0: remote operation failed [No such file or directory]
[2017-03-17 13:08:47.288822] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:ecd9e573-b528-470f-8d7c-79f4c4676597> (ecd9e573-b528-470f-8d7c-79f4c4676597) [No such file or directory]
[2017-03-17 13:08:47.288915] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:ecd9e573-b528-470f-8d7c-79f4c4676597> (ecd9e573-b528-470f-8d7c-79f4c4676597) [No such file or directory]
[2017-03-17 13:08:47.290209] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid ecd9e573-b528-470f-8d7c-79f4c4676597. (Possible split-brain)
[2017-03-17 13:08:47.290601] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:ecd9e573-b528-470f-8d7c-79f4c4676597> (ecd9e573-b528-470f-8d7c-79f4c4676597) [No such file or directory]
[2017-03-17 13:08:47.290754] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:ecd9e573-b528-470f-8d7c-79f4c4676597> (ecd9e573-b528-470f-8d7c-79f4c4676597) [No such file or directory]
[2017-03-17 13:08:47.291241] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:ecd9e573-b528-470f-8d7c-79f4c4676597> (ecd9e573-b528-470f-8d7c-79f4c4676597) [No such file or directory]
[2017-03-17 13:08:47.291407] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:ecd9e573-b528-470f-8d7c-79f4c4676597> (ecd9e573-b528-470f-8d7c-79f4c4676597) [No such file or directory]
[2017-03-17 13:08:47.292902] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:ecd9e573-b528-470f-8d7c-79f4c4676597>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:08:47.292971] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: ecd9e573-b528-470f-8d7c-79f4c4676597 [Stale file handle]
[2017-03-17 13:08:47.323426] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-0/cache=vmware2-replicate-0) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-0)
[2017-03-17 13:08:47.330462] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-0: remote operation failed [No such file or directory]
[2017-03-17 13:08:47.330918] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:ec534911-a824-4f10-8643-31fb2b734cb4> (ec534911-a824-4f10-8643-31fb2b734cb4) [No such file or directory]
[2017-03-17 13:08:47.331057] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:ec534911-a824-4f10-8643-31fb2b734cb4> (ec534911-a824-4f10-8643-31fb2b734cb4) [No such file or directory]
[2017-03-17 13:08:47.331551] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid ec534911-a824-4f10-8643-31fb2b734cb4. (Possible split-brain)
[2017-03-17 13:08:47.332030] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:ec534911-a824-4f10-8643-31fb2b734cb4> (ec534911-a824-4f10-8643-31fb2b734cb4) [No such file or directory]
[2017-03-17 13:08:47.332184] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:ec534911-a824-4f10-8643-31fb2b734cb4> (ec534911-a824-4f10-8643-31fb2b734cb4) [No such file or directory]
[2017-03-17 13:08:47.332785] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:ec534911-a824-4f10-8643-31fb2b734cb4> (ec534911-a824-4f10-8643-31fb2b734cb4) [No such file or directory]
[2017-03-17 13:08:47.332900] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:ec534911-a824-4f10-8643-31fb2b734cb4> (ec534911-a824-4f10-8643-31fb2b734cb4) [No such file or directory]
[2017-03-17 13:08:47.334726] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:ec534911-a824-4f10-8643-31fb2b734cb4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:08:47.334805] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: ec534911-a824-4f10-8643-31fb2b734cb4 [Stale file handle]
[2017-03-17 13:08:47.370462] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-0/cache=vmware2-replicate-0) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-0)
[2017-03-17 13:08:47.377609] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-0: remote operation failed [No such file or directory]
[2017-03-17 13:08:47.378040] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:634dc087-aad5-47ba-ae35-13a4b2d1a840> (634dc087-aad5-47ba-ae35-13a4b2d1a840) [No such file or directory]
[2017-03-17 13:08:47.378211] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:634dc087-aad5-47ba-ae35-13a4b2d1a840> (634dc087-aad5-47ba-ae35-13a4b2d1a840) [No such file or directory]
[2017-03-17 13:08:47.378720] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid 634dc087-aad5-47ba-ae35-13a4b2d1a840. (Possible split-brain)
[2017-03-17 13:08:47.379244] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:634dc087-aad5-47ba-ae35-13a4b2d1a840> (634dc087-aad5-47ba-ae35-13a4b2d1a840) [No such file or directory]
[2017-03-17 13:08:47.379360] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:634dc087-aad5-47ba-ae35-13a4b2d1a840> (634dc087-aad5-47ba-ae35-13a4b2d1a840) [No such file or directory]
[2017-03-17 13:08:47.379940] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:634dc087-aad5-47ba-ae35-13a4b2d1a840> (634dc087-aad5-47ba-ae35-13a4b2d1a840) [No such file or directory]
[2017-03-17 13:08:47.380000] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:634dc087-aad5-47ba-ae35-13a4b2d1a840> (634dc087-aad5-47ba-ae35-13a4b2d1a840) [No such file or directory]
[2017-03-17 13:08:47.381882] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:634dc087-aad5-47ba-ae35-13a4b2d1a840>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:08:47.381948] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 634dc087-aad5-47ba-ae35-13a4b2d1a840 [Stale file handle]
[2017-03-17 13:08:47.413715] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-0/cache=vmware2-replicate-0) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-0)
[2017-03-17 13:08:47.420743] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-0: remote operation failed [No such file or directory]
[2017-03-17 13:08:47.421211] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:c317c48e-2a54-4ed2-b868-1aeb6e46990e> (c317c48e-2a54-4ed2-b868-1aeb6e46990e) [No such file or directory]
[2017-03-17 13:08:47.421230] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:c317c48e-2a54-4ed2-b868-1aeb6e46990e> (c317c48e-2a54-4ed2-b868-1aeb6e46990e) [No such file or directory]
[2017-03-17 13:08:47.421664] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid c317c48e-2a54-4ed2-b868-1aeb6e46990e. (Possible split-brain)
[2017-03-17 13:08:47.422166] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:c317c48e-2a54-4ed2-b868-1aeb6e46990e> (c317c48e-2a54-4ed2-b868-1aeb6e46990e) [No such file or directory]
[2017-03-17 13:08:47.422287] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:c317c48e-2a54-4ed2-b868-1aeb6e46990e> (c317c48e-2a54-4ed2-b868-1aeb6e46990e) [No such file or directory]
[2017-03-17 13:08:47.422789] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:c317c48e-2a54-4ed2-b868-1aeb6e46990e> (c317c48e-2a54-4ed2-b868-1aeb6e46990e) [No such file or directory]
[2017-03-17 13:08:47.423027] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:c317c48e-2a54-4ed2-b868-1aeb6e46990e> (c317c48e-2a54-4ed2-b868-1aeb6e46990e) [No such file or directory]
[2017-03-17 13:08:47.424959] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:c317c48e-2a54-4ed2-b868-1aeb6e46990e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:08:47.425033] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: c317c48e-2a54-4ed2-b868-1aeb6e46990e [Stale file handle]
[2017-03-17 13:08:47.497973] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-0/cache=vmware2-replicate-0) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-0)
[2017-03-17 13:08:47.504940] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-0: remote operation failed [No such file or directory]
[2017-03-17 13:08:47.505472] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:f60650a7-f92c-46e0-8519-41c69003b965> (f60650a7-f92c-46e0-8519-41c69003b965) [No such file or directory]
[2017-03-17 13:08:47.505515] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:f60650a7-f92c-46e0-8519-41c69003b965> (f60650a7-f92c-46e0-8519-41c69003b965) [No such file or directory]
[2017-03-17 13:08:47.505921] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid f60650a7-f92c-46e0-8519-41c69003b965. (Possible split-brain)
[2017-03-17 13:08:47.506397] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:f60650a7-f92c-46e0-8519-41c69003b965> (f60650a7-f92c-46e0-8519-41c69003b965) [No such file or directory]
[2017-03-17 13:08:47.506514] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:f60650a7-f92c-46e0-8519-41c69003b965> (f60650a7-f92c-46e0-8519-41c69003b965) [No such file or directory]
[2017-03-17 13:08:47.507054] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:f60650a7-f92c-46e0-8519-41c69003b965> (f60650a7-f92c-46e0-8519-41c69003b965) [No such file or directory]
[2017-03-17 13:08:47.507188] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-1: remote operation failed. Path: <gfid:f60650a7-f92c-46e0-8519-41c69003b965> (f60650a7-f92c-46e0-8519-41c69003b965) [No such file or directory]
[2017-03-17 13:08:47.508669] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:f60650a7-f92c-46e0-8519-41c69003b965>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:08:47.508740] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: f60650a7-f92c-46e0-8519-41c69003b965 [Stale file handle]
[2017-03-17 13:08:47.554755] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 10-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:08:49.219397] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 10-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmdk~ (hash=vmware2-replicate-18/cache=vmware2-replicate-18) => /New Virtual Machine/New Virtual Machine.vmdk (hash=vmware2-replicate-16/cache=vmware2-replicate-18)
[2017-03-17 13:08:49.226983] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 10-vmware2-client-36: remote operation failed [No such file or directory]
[2017-03-17 13:08:49.227426] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:a4febee8-d204-449a-ad87-31e54f72c778> (a4febee8-d204-449a-ad87-31e54f72c778) [No such file or directory]
[2017-03-17 13:08:49.227548] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:a4febee8-d204-449a-ad87-31e54f72c778> (a4febee8-d204-449a-ad87-31e54f72c778) [No such file or directory]
[2017-03-17 13:08:49.228845] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid a4febee8-d204-449a-ad87-31e54f72c778. (Possible split-brain)
[2017-03-17 13:08:49.229353] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:a4febee8-d204-449a-ad87-31e54f72c778> (a4febee8-d204-449a-ad87-31e54f72c778) [No such file or directory]
[2017-03-17 13:08:49.229365] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:a4febee8-d204-449a-ad87-31e54f72c778> (a4febee8-d204-449a-ad87-31e54f72c778) [No such file or directory]
[2017-03-17 13:08:49.231725] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-36: remote operation failed. Path: <gfid:a4febee8-d204-449a-ad87-31e54f72c778> (a4febee8-d204-449a-ad87-31e54f72c778) [No such file or directory]
[2017-03-17 13:08:49.231831] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-37: remote operation failed. Path: <gfid:a4febee8-d204-449a-ad87-31e54f72c778> (a4febee8-d204-449a-ad87-31e54f72c778) [No such file or directory]
[2017-03-17 13:08:49.232162] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:a4febee8-d204-449a-ad87-31e54f72c778>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:08:49.232219] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: a4febee8-d204-449a-ad87-31e54f72c778 [Stale file handle]
[2017-03-17 13:08:49.237298] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 10-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 50bbbefa-030c-4749-af76-e26cd46a57dc. (Possible split-brain)
[2017-03-17 13:08:49.240009] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 10-vmware2-dht: <gfid:50bbbefa-030c-4749-af76-e26cd46a57dc>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:08:49.240072] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 10-vmware2-shard: stat failed: 50bbbefa-030c-4749-af76-e26cd46a57dc [Stale file handle]
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 10-vmware2-client-0: remote operation failed. Path: <gfid:8e5ffc66-bb8b-4467-bab6-7cbfbbe9ed9d> (8e5ffc66-bb8b-4467-bab6-7cbfbbe9ed9d) [No such file or directory]" repeated 2 times between [2017-03-17 13:08:47.103678] and [2017-03-17 13:08:47.106387]
[2017-03-17 13:11:39.504200] I [MSGID: 104045] [glfs-master.c:91:notify] 0-gfapi: New graph 676c7573-7465-7230-312d-313938392d32 (12) coming up
[2017-03-17 13:11:39.504252] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-0: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.507221] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-1: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.507456] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-0: changing port to 49512 (from 0)
[2017-03-17 13:11:39.509943] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-2: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.513557] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.514086] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-0: Connected to vmware2-client-0, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:11:39.514115] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.514182] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-0: Subvolume 'vmware2-client-0' came back up; going online.
[2017-03-17 13:11:39.514255] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-0: Server lk version = 1
[2017-03-17 13:11:39.515868] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-3: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.518462] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-4: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.521075] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-5: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.521304] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-4: changing port to 49513 (from 0)
[2017-03-17 13:11:39.523673] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-6: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.527505] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-4: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.528101] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-4: Connected to vmware2-client-4, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:11:39.528130] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-4: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.528197] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-2: Subvolume 'vmware2-client-4' came back up; going online.
[2017-03-17 13:11:39.528261] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-4: Server lk version = 1
[2017-03-17 13:11:39.530053] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-7: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.532667] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-8: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.535276] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-9: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.535482] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-8: changing port to 49514 (from 0)
[2017-03-17 13:11:39.537877] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-10: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.542010] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-8: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.542589] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-8: Connected to vmware2-client-8, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:11:39.542609] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-8: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.542668] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-4: Subvolume 'vmware2-client-8' came back up; going online.
[2017-03-17 13:11:39.542745] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-8: Server lk version = 1
[2017-03-17 13:11:39.544275] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-11: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.546779] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-12: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.549286] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-13: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.549513] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-12: changing port to 49515 (from 0)
[2017-03-17 13:11:39.551788] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-14: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.556048] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-1: changing port to 49489 (from 0)
[2017-03-17 13:11:39.556093] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-5: changing port to 49490 (from 0)
[2017-03-17 13:11:39.558222] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-15: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.562378] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-9: changing port to 49491 (from 0)
[2017-03-17 13:11:39.562426] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-12: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.564675] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-16: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.570989] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-17: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.574867] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-13: changing port to 49492 (from 0)
[2017-03-17 13:11:39.574930] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-12: Connected to vmware2-client-12, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:11:39.574948] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-12: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.575017] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-6: Subvolume 'vmware2-client-12' came back up; going online.
[2017-03-17 13:11:39.575168] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.577026] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-18: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.580954] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-12: Server lk version = 1
[2017-03-17 13:11:39.581045] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-5: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.581108] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-3: changing port to 49385 (from 0)
[2017-03-17 13:11:39.581167] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-7: changing port to 49386 (from 0)
[2017-03-17 13:11:39.581191] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-11: changing port to 49387 (from 0)
[2017-03-17 13:11:39.581222] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-1: Connected to vmware2-client-1, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:11:39.581235] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-1: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.581415] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-9: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.583086] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-19: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.588889] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-20: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.592559] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-15: changing port to 49388 (from 0)
[2017-03-17 13:11:39.594743] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-21: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.598207] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-16: changing port to 49516 (from 0)
[2017-03-17 13:11:39.598280] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-1: Server lk version = 1
[2017-03-17 13:11:39.598342] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-13: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.598416] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-5: Connected to vmware2-client-5, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:11:39.598429] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-5: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.598531] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-9: Connected to vmware2-client-9, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:11:39.598552] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-9: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.600393] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-22: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.606218] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-23: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.609699] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-17: changing port to 49493 (from 0)
[2017-03-17 13:11:39.609744] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-5: Server lk version = 1
[2017-03-17 13:11:39.609784] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-9: Server lk version = 1
[2017-03-17 13:11:39.609864] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-3: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.609933] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-13: Connected to vmware2-client-13, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:11:39.609946] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-13: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.610128] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-7: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.610177] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-2: changing port to 49414 (from 0)
[2017-03-17 13:11:39.610201] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-6: changing port to 49415 (from 0)
[2017-03-17 13:11:39.610219] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-10: changing port to 49416 (from 0)
[2017-03-17 13:11:39.610259] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-11: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.610347] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-14: changing port to 49417 (from 0)
[2017-03-17 13:11:39.611883] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-24: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.615136] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-19: changing port to 49389 (from 0)
[2017-03-17 13:11:39.615177] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-18: changing port to 49418 (from 0)
[2017-03-17 13:11:39.615201] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-13: Server lk version = 1
[2017-03-17 13:11:39.615223] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-20: changing port to 49517 (from 0)
[2017-03-17 13:11:39.617295] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-25: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.622720] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-26: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.628081] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-27: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.631148] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-15: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.633304] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-28: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.636455] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-3: Connected to vmware2-client-3, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:11:39.636479] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-3: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.636529] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-1: Subvolume 'vmware2-client-3' came back up; going online.
[2017-03-17 13:11:39.636585] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-16: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.636652] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-7: Connected to vmware2-client-7, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:11:39.636665] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-7: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.636710] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-3: Subvolume 'vmware2-client-7' came back up; going online.
[2017-03-17 13:11:39.636753] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-11: Connected to vmware2-client-11, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:11:39.636777] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-11: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.636814] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-5: Subvolume 'vmware2-client-11' came back up; going online.
[2017-03-17 13:11:39.638641] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-29: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.644085] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-30: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.649361] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-31: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.652493] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-21: changing port to 49494 (from 0)
[2017-03-17 13:11:39.652599] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-15: Connected to vmware2-client-15, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:11:39.652614] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-15: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.652645] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-7: Subvolume 'vmware2-client-15' came back up; going online.
[2017-03-17 13:11:39.652698] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-3: Server lk version = 1
[2017-03-17 13:11:39.652716] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-22: changing port to 49419 (from 0)
[2017-03-17 13:11:39.652734] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-7: Server lk version = 1
[2017-03-17 13:11:39.652773] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-11: Server lk version = 1
[2017-03-17 13:11:39.652839] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-16: Connected to vmware2-client-16, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:11:39.652851] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-16: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.652895] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-8: Subvolume 'vmware2-client-16' came back up; going online.
[2017-03-17 13:11:39.652964] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-17: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.654613] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-32: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.657585] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.657660] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-10: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.657709] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-6: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.659795] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-33: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.662821] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-15: Server lk version = 1
[2017-03-17 13:11:39.662904] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-14: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.662951] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-23: changing port to 49390 (from 0)
[2017-03-17 13:11:39.662974] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-16: Server lk version = 1
[2017-03-17 13:11:39.663032] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-19: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.663094] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-24: changing port to 49518 (from 0)
[2017-03-17 13:11:39.663143] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-20: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.663192] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-18: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.663244] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-17: Connected to vmware2-client-17, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:11:39.663254] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-17: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.663364] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-25: changing port to 49495 (from 0)
[2017-03-17 13:11:39.663387] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-26: changing port to 49420 (from 0)
[2017-03-17 13:11:39.663411] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-10: Connected to vmware2-client-10, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:11:39.663421] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-10: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.663492] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-2: Connected to vmware2-client-2, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:11:39.663502] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.663567] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-6: Connected to vmware2-client-6, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:11:39.663577] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-6: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.665003] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-34: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.667792] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-27: changing port to 49391 (from 0)
[2017-03-17 13:11:39.667843] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-28: changing port to 49519 (from 0)
[2017-03-17 13:11:39.669970] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-35: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.672805] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-29: changing port to 49496 (from 0)
[2017-03-17 13:11:39.672846] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-30: changing port to 49421 (from 0)
[2017-03-17 13:11:39.672866] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-17: Server lk version = 1
[2017-03-17 13:11:39.674941] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-36: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.680087] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-37: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.682895] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-21: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.683000] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-14: Connected to vmware2-client-14, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:11:39.683013] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-14: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.683080] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-10: Server lk version = 1
[2017-03-17 13:11:39.683106] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-19: Connected to vmware2-client-19, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:11:39.683113] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-19: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.683138] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-9: Subvolume 'vmware2-client-19' came back up; going online.
[2017-03-17 13:11:39.683157] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-2: Server lk version = 1
[2017-03-17 13:11:39.683175] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-20: Connected to vmware2-client-20, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:11:39.683183] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-20: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.683223] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-10: Subvolume 'vmware2-client-20' came back up; going online.
[2017-03-17 13:11:39.683275] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-6: Server lk version = 1
[2017-03-17 13:11:39.683297] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-18: Connected to vmware2-client-18, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:11:39.683304] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-18: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.683379] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-22: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.685182] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-38: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.690036] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-39: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.694880] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-40: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.699711] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-41: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.702354] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-31: changing port to 49392 (from 0)
[2017-03-17 13:11:39.702404] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-14: Server lk version = 1
[2017-03-17 13:11:39.702424] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-19: Server lk version = 1
[2017-03-17 13:11:39.702453] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-20: Server lk version = 1
[2017-03-17 13:11:39.702479] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-21: Connected to vmware2-client-21, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:11:39.702488] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-21: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.702574] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-32: changing port to 49520 (from 0)
[2017-03-17 13:11:39.702602] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-18: Server lk version = 1
[2017-03-17 13:11:39.702648] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-22: Connected to vmware2-client-22, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:11:39.702660] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-22: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.702690] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-11: Subvolume 'vmware2-client-22' came back up; going online.
[2017-03-17 13:11:39.702741] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-23: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.702854] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-24: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.704500] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-42: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.707159] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-25: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.707235] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-26: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.707285] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-21: Server lk version = 1
[2017-03-17 13:11:39.709263] I [MSGID: 114020] [client.c:2356:notify] 12-vmware2-client-43: parent translators are ready, attempting connect on transport
[2017-03-17 13:11:39.711893] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-22: Server lk version = 1
[2017-03-17 13:11:39.711922] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-33: changing port to 49497 (from 0)
[2017-03-17 13:11:39.711972] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-27: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.712039] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-28: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.712084] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-34: changing port to 49422 (from 0)
[2017-03-17 13:11:39.712143] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-29: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.712183] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-30: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.712231] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-23: Connected to vmware2-client-23, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:11:39.712258] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-23: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.712339] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-24: Connected to vmware2-client-24, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:11:39.712349] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-24: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.712395] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-12: Subvolume 'vmware2-client-24' came back up; going online.
[2017-03-17 13:11:39.712448] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-36: changing port to 49521 (from 0)
[2017-03-17 13:11:39.712502] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-35: changing port to 49393 (from 0)
[2017-03-17 13:11:39.712545] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-26: Connected to vmware2-client-26, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:11:39.712554] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-26: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.712580] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-13: Subvolume 'vmware2-client-26' came back up; going online.
[2017-03-17 13:11:39.712604] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-25: Connected to vmware2-client-25, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:11:39.712611] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-25: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.714176] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-37: changing port to 49498 (from 0)
[2017-03-17 13:11:39.716520] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-38: changing port to 49423 (from 0)
[2017-03-17 13:11:39.716551] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-40: changing port to 49523 (from 0)
[2017-03-17 13:11:39.716574] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-39: changing port to 49394 (from 0)
[2017-03-17 13:11:39.716589] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-23: Server lk version = 1
[2017-03-17 13:11:39.716613] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-28: Connected to vmware2-client-28, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:11:39.716621] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-28: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.716662] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-14: Subvolume 'vmware2-client-28' came back up; going online.
[2017-03-17 13:11:39.716700] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-24: Server lk version = 1
[2017-03-17 13:11:39.718752] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-27: Connected to vmware2-client-27, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:11:39.718778] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-27: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.721242] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-31: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.721306] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-29: Connected to vmware2-client-29, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:11:39.721323] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-29: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.721409] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-30: Connected to vmware2-client-30, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:11:39.721420] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-30: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.721445] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-15: Subvolume 'vmware2-client-30' came back up; going online.
[2017-03-17 13:11:39.721463] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-26: Server lk version = 1
[2017-03-17 13:11:39.721506] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-25: Server lk version = 1
[2017-03-17 13:11:39.721522] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-32: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.730288] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-28: Server lk version = 1
[2017-03-17 13:11:39.730340] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-27: Server lk version = 1
[2017-03-17 13:11:39.730382] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-41: changing port to 49500 (from 0)
[2017-03-17 13:11:39.730402] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-29: Server lk version = 1
[2017-03-17 13:11:39.730427] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-30: Server lk version = 1
[2017-03-17 13:11:39.730438] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-42: changing port to 49425 (from 0)
[2017-03-17 13:11:39.730484] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-31: Connected to vmware2-client-31, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:11:39.730510] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-31: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.730579] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-32: Connected to vmware2-client-32, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:11:39.730589] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-32: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.730624] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-16: Subvolume 'vmware2-client-32' came back up; going online.
[2017-03-17 13:11:39.730663] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-33: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.732518] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-34: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.734874] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-36: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.734941] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 12-vmware2-client-43: changing port to 49395 (from 0)
[2017-03-17 13:11:39.734964] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-32: Server lk version = 1
[2017-03-17 13:11:39.734978] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-31: Server lk version = 1
[2017-03-17 13:11:39.735009] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-35: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.735058] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-37: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.735094] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-38: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.735131] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-40: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.735193] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-33: Connected to vmware2-client-33, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:11:39.735204] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-33: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.735319] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-34: Connected to vmware2-client-34, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:11:39.735330] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-34: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.735355] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-17: Subvolume 'vmware2-client-34' came back up; going online.
[2017-03-17 13:11:39.737125] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-33: Server lk version = 1
[2017-03-17 13:11:39.737160] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-36: Connected to vmware2-client-36, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:11:39.737169] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-36: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.737214] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-18: Subvolume 'vmware2-client-36' came back up; going online.
[2017-03-17 13:11:39.737237] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-39: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.737274] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-34: Server lk version = 1
[2017-03-17 13:11:39.737296] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-37: Connected to vmware2-client-37, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:11:39.737303] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-37: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.737375] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-35: Connected to vmware2-client-35, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:11:39.737385] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-35: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.737457] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-38: Connected to vmware2-client-38, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:11:39.737467] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-38: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.737490] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-19: Subvolume 'vmware2-client-38' came back up; going online.
[2017-03-17 13:11:39.737509] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-41: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.737596] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-36: Server lk version = 1
[2017-03-17 13:11:39.737619] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-37: Server lk version = 1
[2017-03-17 13:11:39.737630] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-35: Server lk version = 1
[2017-03-17 13:11:39.737668] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-38: Server lk version = 1
[2017-03-17 13:11:39.737723] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-42: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.737778] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-39: Connected to vmware2-client-39, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:11:39.737791] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-39: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.737956] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-39: Server lk version = 1
[2017-03-17 13:11:39.739811] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 12-vmware2-client-43: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:11:39.740654] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-40: Connected to vmware2-client-40, attached to remote volume '/mnt/disk11/vmware2'.
[2017-03-17 13:11:39.740677] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-40: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.740743] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-20: Subvolume 'vmware2-client-40' came back up; going online.
[2017-03-17 13:11:39.740804] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-40: Server lk version = 1
[2017-03-17 13:11:39.741877] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-42: Connected to vmware2-client-42, attached to remote volume '/mnt/disk11/vmware2'.
[2017-03-17 13:11:39.741898] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-42: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.741935] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 12-vmware2-replicate-21: Subvolume 'vmware2-client-42' came back up; going online.
[2017-03-17 13:11:39.742012] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-42: Server lk version = 1
[2017-03-17 13:11:39.743021] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-41: Connected to vmware2-client-41, attached to remote volume '/mnt/disk11/vmware2'.
[2017-03-17 13:11:39.743060] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-41: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.743225] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-41: Server lk version = 1
[2017-03-17 13:11:39.744751] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 12-vmware2-client-43: Connected to vmware2-client-43, attached to remote volume '/mnt/disk11/vmware2'.
[2017-03-17 13:11:39.744790] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 12-vmware2-client-43: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:11:39.759830] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 12-vmware2-client-43: Server lk version = 1
[2017-03-17 13:11:39.762986] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:11:39.763496] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:11:39.763855] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 13:11:39.764126] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:11:39.764186] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 13:11:39.764331] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:11:39.764369] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:11:39.764390] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:11:39.764410] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:11:39.764459] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:11:39.764463] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-20: selecting local read_child vmware2-client-40
[2017-03-17 13:11:39.808885] I [MSGID: 104041] [glfs-resolve.c:885:__glfs_active_subvol] 0-vmware2: switched to graph 676c7573-7465-7230-312d-313938392d32 (12)
[2017-03-17 13:11:39.808927] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-0: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.808965] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-1: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.808986] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-0: disconnected from vmware2-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.808993] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-2: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809015] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-1: disconnected from vmware2-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809029] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-3: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809048] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809043] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-2: disconnected from vmware2-client-2. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809077] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-4: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809085] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-3: disconnected from vmware2-client-3. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809097] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-1: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809108] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-5: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809115] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-4: disconnected from vmware2-client-4. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809134] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-6: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809144] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-5: disconnected from vmware2-client-5. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809166] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-2: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809170] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-7: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809192] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-6: disconnected from vmware2-client-6. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809203] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-8: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809213] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-7: disconnected from vmware2-client-7. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809224] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-3: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809238] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-9: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809244] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-8: disconnected from vmware2-client-8. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809267] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-10: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809293] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-11: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809299] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-10: disconnected from vmware2-client-10. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809289] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-9: disconnected from vmware2-client-9. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809324] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-11: disconnected from vmware2-client-11. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809328] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-4: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809348] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-12: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809353] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-5: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809383] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-13: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809390] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-12: disconnected from vmware2-client-12. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809411] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-14: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809418] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-13: disconnected from vmware2-client-13. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809428] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-6: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809444] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-14: disconnected from vmware2-client-14. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809450] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-15: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809481] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-16: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809487] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-15: disconnected from vmware2-client-15. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809501] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-7: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809517] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-16: disconnected from vmware2-client-16. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809518] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-17: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809549] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-18: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809557] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-17: disconnected from vmware2-client-17. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809568] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-8: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809573] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-19: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809580] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-18: disconnected from vmware2-client-18. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809601] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-20: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809605] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-19: disconnected from vmware2-client-19. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809623] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-9: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809635] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-21: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809641] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-20: disconnected from vmware2-client-20. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809663] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-22: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809668] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-21: disconnected from vmware2-client-21. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809685] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-10: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809688] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-23: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809695] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-22: disconnected from vmware2-client-22. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809729] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-24: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809730] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-23: disconnected from vmware2-client-23. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809756] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-11: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809782] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-24: disconnected from vmware2-client-24. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809779] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-25: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809824] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-26: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809830] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-25: disconnected from vmware2-client-25. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809846] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-12: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809851] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-27: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809862] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-26: disconnected from vmware2-client-26. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809882] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-28: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809888] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-27: disconnected from vmware2-client-27. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809905] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-13: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809917] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-29: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809923] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-28: disconnected from vmware2-client-28. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809944] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-29: disconnected from vmware2-client-29. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.809941] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-30: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809956] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-14: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.809976] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-31: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.809982] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-30: disconnected from vmware2-client-30. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.810008] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-31: disconnected from vmware2-client-31. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.810005] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-32: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.810026] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-15: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.810054] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-33: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.810060] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-32: disconnected from vmware2-client-32. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.810082] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-34: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.810087] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-33: disconnected from vmware2-client-33. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.810103] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-16: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.810106] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-35: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.810115] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-34: disconnected from vmware2-client-34. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.810134] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-35: disconnected from vmware2-client-35. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.810138] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-36: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.810145] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-17: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.810180] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-37: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.810185] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-36: disconnected from vmware2-client-36. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.810206] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-38: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.810212] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-37: disconnected from vmware2-client-37. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.810228] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-18: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.810231] I [MSGID: 114021] [client.c:2365:notify] 10-vmware2-client-39: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:11:39.810238] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-38: disconnected from vmware2-client-38. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.810260] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 10-vmware2-client-39: disconnected from vmware2-client-39. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:11:39.810270] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 10-vmware2-replicate-19: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:11:39.933509] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 10-vmware2-client-18: remote operation failed
[2017-03-17 13:11:39.933577] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 10-vmware2-client-19: remote operation failed
[2017-03-17 13:11:39.933614] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 10-vmware2-client-18: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:11:39.933636] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 10-vmware2-client-19: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:11:39.933703] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 10-vmware2-client-4: remote operation failed
[2017-03-17 13:11:39.933728] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 10-vmware2-client-5: remote operation failed
[2017-03-17 13:11:39.933744] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 10-vmware2-client-4: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:11:39.933762] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 10-vmware2-client-5: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:11:40.933949] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 10-vmware2-client-22: remote operation failed
[2017-03-17 13:11:40.934005] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 10-vmware2-client-23: remote operation failed
[2017-03-17 13:11:40.934027] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 10-vmware2-client-22: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:11:40.934048] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 10-vmware2-client-23: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:16:53.584389] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-27: remote operation failed. Path: <gfid:7e1768e0-ed80-4b0d-8535-daa8e11ec74c> (7e1768e0-ed80-4b0d-8535-daa8e11ec74c) [No such file or directory]
[2017-03-17 13:16:53.584429] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-26: remote operation failed. Path: <gfid:7e1768e0-ed80-4b0d-8535-daa8e11ec74c> (7e1768e0-ed80-4b0d-8535-daa8e11ec74c) [No such file or directory]
[2017-03-17 13:16:55.426414] W [MSGID: 108019] [afr-lk-common.c:1082:is_blocking_locks_count_sufficient] 12-vmware2-replicate-11: Unable to obtain blocking inode lock on even one child for gfid:986f8c81-6b8b-4d3e-a61e-93165ca7ff7a.
[2017-03-17 13:16:55.426441] I [MSGID: 108019] [afr-transaction.c:1478:afr_post_blocking_inodelk_cbk] 12-vmware2-replicate-11: Blocking inodelks failed.
[2017-03-17 13:16:55.433695] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-22: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-22: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 13:16:55.433695] and [2017-03-17 13:16:55.433821]
[2017-03-17 13:16:55.433885] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-23: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.433941] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-22: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.433973] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-22: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.433994] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-23: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.434014] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-22: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.434029] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-23: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.434034] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-22: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.434074] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-22: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.434095] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-23: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-23: remote operation failed [No such file or directory]" repeated 4 times between [2017-03-17 13:16:55.434095] and [2017-03-17 13:16:55.434288]
[2017-03-17 13:16:55.632692] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-39: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.633040] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-39: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.633087] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-38: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.633386] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid 621839a9-a27e-4a1f-b466-87841be490f7. (Possible split-brain)
[2017-03-17 13:16:55.633693] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-38: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.633739] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-39: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.644869] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.645030] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.645308] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.645972] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 591be5a5-8b3d-40d9-8882-9fe9087ea367. (Possible split-brain)
[2017-03-17 13:16:55.646222] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.646390] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.654629] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.654803] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.655031] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.655199] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 27066740-67e4-408b-a26f-d92539d71487. (Possible split-brain)
[2017-03-17 13:16:55.655360] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 13:16:55.655525] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 13:16:59.928387] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 13:16:59.928598] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 13:16:59.928894] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 13:16:59.929444] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 77cb1e6a-6b47-411c-bdbb-bfddfc9a3cf5. (Possible split-brain)
[2017-03-17 13:16:59.929754] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 13:16:59.929908] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 13:16:59.944259] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-26: remote operation failed [No such file or directory]
[2017-03-17 13:16:59.944674] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-26: remote operation failed [No such file or directory]
[2017-03-17 13:16:59.944679] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-27: remote operation failed [No such file or directory]
[2017-03-17 13:16:59.944952] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 32583b3b-4193-4fdd-89ac-e411784e63e7. (Possible split-brain)
[2017-03-17 13:16:59.945319] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-27: remote operation failed [No such file or directory]
[2017-03-17 13:16:59.945332] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-26: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.001851] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.002176] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.002234] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.002580] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid d59bfc9d-d366-4549-9ce3-e621e76595f2. (Possible split-brain)
[2017-03-17 13:17:00.002947] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.002979] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.188544] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.188881] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.189086] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.189466] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 43dcdd29-1fcd-46df-b277-f11e7390a046. (Possible split-brain)
[2017-03-17 13:17:00.189791] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.189879] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.354643] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.355090] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.355252] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-13: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.355793] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid f9b4bd80-19ca-40f5-bc40-cdb1c01c8ff7. (Possible split-brain)
[2017-03-17 13:17:00.356237] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 13:17:00.356312] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-13: remote operation failed [No such file or directory]
[2017-03-17 13:17:07.463626] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:17:07.464033] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:17:07.464164] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 13:17:07.465439] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid d3181ab1-d719-44d9-9ff0-03ab18de7885. (Possible split-brain)
[2017-03-17 13:17:07.465923] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:17:07.466011] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 13:17:07.992640] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:17:07.993060] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:17:07.993179] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 13:17:07.993912] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 524a67c2-2396-454c-b8b6-e832c9ca6b66. (Possible split-brain)
[2017-03-17 13:17:07.994483] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 13:17:07.994483] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:17:08.647675] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:17:08.648138] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:17:08.648246] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 13:17:08.648857] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a87dd2aa-0f07-403d-ba7b-1644920489cc. (Possible split-brain)
[2017-03-17 13:17:08.649405] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:17:08.649457] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 13:17:13.591365] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-23: remote operation failed [No such file or directory]
[2017-03-17 13:17:13.591811] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-23: remote operation failed [No such file or directory]
[2017-03-17 13:17:13.591848] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-22: remote operation failed [No such file or directory]
[2017-03-17 13:17:13.592500] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-11: Unreadable subvolume -1 found with event generation 2 for gfid a7cff204-f542-4a70-9bd4-4208448abc8e. (Possible split-brain)
[2017-03-17 13:17:13.592964] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-22: remote operation failed [No such file or directory]
[2017-03-17 13:17:13.593056] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-23: remote operation failed [No such file or directory]
[2017-03-17 13:17:14.260022] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 12-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 13:17:14.260582] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-16: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:17:14.260745] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-17: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:17:14.261432] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 178a2423-15aa-4819-900d-b3b641c40e70. (Possible split-brain)
[2017-03-17 13:17:14.262098] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-16: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:17:14.262111] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-17: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:17:14.263698] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-16: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:17:14.263838] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-17: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:17:14.265822] E [shard.c:426:shard_modify_size_and_block_count] (-->/usr/lib64/glusterfs/3.8.10/xlator/cluster/distribute.so(+0x60ed7) [0x7f1268254ed7] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xb6f0) [0x7f125bde96f0] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xaf5d) [0x7f125bde8f5d] ) 12-vmware2-shard: Failed to get trusted.glusterfs.shard.file-size for 178a2423-15aa-4819-900d-b3b641c40e70
[2017-03-17 13:17:43.582365] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-17: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:17:43.582396] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-16: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:17:54.147298] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-0; inode layout - 3435973824 - 3650722187 - 1; disk layout - 3709289934 - 3904515719 - 1
[2017-03-17 13:17:54.147360] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.147401] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-1; inode layout - 3650722188 - 3865470551 - 1; disk layout - 2928386790 - 3123612575 - 1
[2017-03-17 13:17:54.147435] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.147522] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-2; inode layout - 3221225460 - 3435973823 - 1; disk layout - 3318838362 - 3514064147 - 1
[2017-03-17 13:17:54.147549] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.147609] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-3; inode layout - 1932735276 - 2147483639 - 1; disk layout - 1952257860 - 2147483645 - 1
[2017-03-17 13:17:54.147632] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.147686] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-4; inode layout - 858993456 - 1073741819 - 1; disk layout - 780903144 - 976128929 - 1
[2017-03-17 13:17:54.147708] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.147725] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:17:54.147814] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-5; inode layout - 1073741820 - 1288490183 - 1; disk layout - 976128930 - 1171354715 - 1
[2017-03-17 13:17:54.147838] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.147934] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 13:17:54.147943] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-6; inode layout - 0 - 214748363 - 1; disk layout - 0 - 195225785 - 1
[2017-03-17 13:17:54.147987] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148007] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-7; inode layout - 214748364 - 429496727 - 1; disk layout - 195225786 - 390451571 - 1
[2017-03-17 13:17:54.148025] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148143] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-8; inode layout - 3006477096 - 3221225459 - 1; disk layout - 1366580502 - 1561806287 - 1
[2017-03-17 13:17:54.148194] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148223] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-9; inode layout - 644245092 - 858993455 - 1; disk layout - 585677358 - 780903143 - 1
[2017-03-17 13:17:54.148246] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148312] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-10; inode layout - 4080218916 - 4294967295 - 1; disk layout - 4099741506 - 4294967295 - 1
[2017-03-17 13:17:54.148330] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148331] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:17:54.148461] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-11; inode layout - 2576980368 - 2791728731 - 1; disk layout - 2537935218 - 2733161003 - 1
[2017-03-17 13:17:54.148466] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 13:17:54.148495] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148547] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-12; inode layout - 1717986912 - 1932735275 - 1; disk layout - 1757032074 - 1952257859 - 1
[2017-03-17 13:17:54.148565] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148651] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-13; inode layout - 429496728 - 644245091 - 1; disk layout - 390451572 - 585677357 - 1
[2017-03-17 13:17:54.148669] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148686] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-14; inode layout - 2147483640 - 2362232003 - 1; disk layout - 2147483646 - 2342709431 - 1
[2017-03-17 13:17:54.148707] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148777] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-15; inode layout - 2362232004 - 2576980367 - 1; disk layout - 2342709432 - 2537935217 - 1
[2017-03-17 13:17:54.148803] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148904] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-17; inode layout - 2791728732 - 3006477095 - 1; disk layout - 2733161004 - 2928386789 - 1
[2017-03-17 13:17:54.148921] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148937] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-16; inode layout - 3865470552 - 4080218915 - 1; disk layout - 3904515720 - 4099741505 - 1
[2017-03-17 13:17:54.148959] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.148990] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-18; inode layout - 1288490184 - 1503238547 - 1; disk layout - 1561806288 - 1757032073 - 1
[2017-03-17 13:17:54.149007] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.149127] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-19; inode layout - 1503238548 - 1717986911 - 1; disk layout - 3123612576 - 3318838361 - 1
[2017-03-17 13:17:54.149157] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.149240] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-20; inode layout - 0 - 0 - 0; disk layout - 3514064148 - 3709289933 - 1
[2017-03-17 13:17:54.149260] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.149276] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-21; inode layout - 0 - 0 - 0; disk layout - 1171354716 - 1366580501 - 1
[2017-03-17 13:17:54.149297] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:17:54.149343] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-20: selecting local read_child vmware2-client-40
The message "W [MSGID: 108019] [afr-lk-common.c:1082:is_blocking_locks_count_sufficient] 12-vmware2-replicate-11: Unable to obtain blocking inode lock on even one child for gfid:986f8c81-6b8b-4d3e-a61e-93165ca7ff7a." repeated 7 times between [2017-03-17 13:16:55.426414] and [2017-03-17 13:16:55.432482]
The message "I [MSGID: 108019] [afr-transaction.c:1478:afr_post_blocking_inodelk_cbk] 12-vmware2-replicate-11: Blocking inodelks failed." repeated 7 times between [2017-03-17 13:16:55.426441] and [2017-03-17 13:16:55.432483]
[2017-03-17 13:18:26.589554] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-9/cache=vmware2-replicate-9) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-0)
[2017-03-17 13:18:26.598525] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 12-vmware2-client-0: remote operation failed [No such file or directory]
[2017-03-17 13:18:26.599057] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-0: remote operation failed. Path: <gfid:2dd2207f-c243-4a79-ad75-963cda659cdf> (2dd2207f-c243-4a79-ad75-963cda659cdf) [No such file or directory]
[2017-03-17 13:18:26.599190] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-1: remote operation failed. Path: <gfid:2dd2207f-c243-4a79-ad75-963cda659cdf> (2dd2207f-c243-4a79-ad75-963cda659cdf) [No such file or directory]
[2017-03-17 13:18:26.599935] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid 2dd2207f-c243-4a79-ad75-963cda659cdf. (Possible split-brain)
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-1: remote operation failed. Path: <gfid:2dd2207f-c243-4a79-ad75-963cda659cdf> (2dd2207f-c243-4a79-ad75-963cda659cdf) [No such file or directory]" repeated 2 times between [2017-03-17 13:18:26.599190] and [2017-03-17 13:18:26.601395]
[2017-03-17 13:18:26.603189] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:2dd2207f-c243-4a79-ad75-963cda659cdf>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.603267] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 2dd2207f-c243-4a79-ad75-963cda659cdf [Stale file handle]
[2017-03-17 13:18:26.641403] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 7588d56b-a0af-4af6-8e48-7fb83e128b47. (Possible split-brain)
[2017-03-17 13:18:26.644488] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:7588d56b-a0af-4af6-8e48-7fb83e128b47>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.644577] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 7588d56b-a0af-4af6-8e48-7fb83e128b47 [Stale file handle]
[2017-03-17 13:18:26.743291] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 275e4a96-2c3a-4025-98dd-1696cffcc70f. (Possible split-brain)
[2017-03-17 13:18:26.746325] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:275e4a96-2c3a-4025-98dd-1696cffcc70f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.746379] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 275e4a96-2c3a-4025-98dd-1696cffcc70f [Stale file handle]
[2017-03-17 13:18:26.762249] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid 35da3631-32a3-4110-b786-e33f383af534. (Possible split-brain)
[2017-03-17 13:18:26.765550] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:35da3631-32a3-4110-b786-e33f383af534>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.765611] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 35da3631-32a3-4110-b786-e33f383af534 [Stale file handle]
[2017-03-17 13:18:26.769191] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid b74689f3-6282-4d3b-ac48-712d04ccf11d. (Possible split-brain)
[2017-03-17 13:18:26.771640] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:b74689f3-6282-4d3b-ac48-712d04ccf11d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.771700] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: b74689f3-6282-4d3b-ac48-712d04ccf11d [Stale file handle]
[2017-03-17 13:18:26.842096] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-9/cache=vmware2-replicate-9) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-9)
[2017-03-17 13:18:26.849153] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 12-vmware2-client-19: remote operation failed [No such file or directory]
[2017-03-17 13:18:26.849663] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-18: remote operation failed. Path: <gfid:be99ab5d-913a-41d6-be04-c6352c3879df> (be99ab5d-913a-41d6-be04-c6352c3879df) [No such file or directory]
[2017-03-17 13:18:26.849703] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-19: remote operation failed. Path: <gfid:be99ab5d-913a-41d6-be04-c6352c3879df> (be99ab5d-913a-41d6-be04-c6352c3879df) [No such file or directory]
[2017-03-17 13:18:26.850175] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid be99ab5d-913a-41d6-be04-c6352c3879df. (Possible split-brain)
[2017-03-17 13:18:26.850668] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-18: remote operation failed. Path: <gfid:be99ab5d-913a-41d6-be04-c6352c3879df> (be99ab5d-913a-41d6-be04-c6352c3879df) [No such file or directory]
[2017-03-17 13:18:26.850723] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-19: remote operation failed. Path: <gfid:be99ab5d-913a-41d6-be04-c6352c3879df> (be99ab5d-913a-41d6-be04-c6352c3879df) [No such file or directory]
[2017-03-17 13:18:26.851937] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-18: remote operation failed. Path: <gfid:be99ab5d-913a-41d6-be04-c6352c3879df> (be99ab5d-913a-41d6-be04-c6352c3879df) [No such file or directory]
[2017-03-17 13:18:26.851993] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-19: remote operation failed. Path: <gfid:be99ab5d-913a-41d6-be04-c6352c3879df> (be99ab5d-913a-41d6-be04-c6352c3879df) [No such file or directory]
[2017-03-17 13:18:26.853092] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:be99ab5d-913a-41d6-be04-c6352c3879df>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.853159] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: be99ab5d-913a-41d6-be04-c6352c3879df [Stale file handle]
[2017-03-17 13:18:26.858703] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid b45a26ad-4b21-45b8-8c5e-26992c116d6d. (Possible split-brain)
[2017-03-17 13:18:26.862029] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:b45a26ad-4b21-45b8-8c5e-26992c116d6d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.862090] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: b45a26ad-4b21-45b8-8c5e-26992c116d6d [Stale file handle]
[2017-03-17 13:18:26.880142] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 78846fa7-8c73-4e6d-b6ac-58675c9cbfa1. (Possible split-brain)
[2017-03-17 13:18:26.883109] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:78846fa7-8c73-4e6d-b6ac-58675c9cbfa1>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.883169] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 78846fa7-8c73-4e6d-b6ac-58675c9cbfa1 [Stale file handle]
[2017-03-17 13:18:26.887176] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid db3e7f0c-9d92-4589-9944-416be76bb593. (Possible split-brain)
[2017-03-17 13:18:26.889702] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:db3e7f0c-9d92-4589-9944-416be76bb593>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.889759] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: db3e7f0c-9d92-4589-9944-416be76bb593 [Stale file handle]
[2017-03-17 13:18:26.896966] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-16: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:18:26.897151] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-17: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:18:26.916992] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid 7e1768e0-ed80-4b0d-8535-daa8e11ec74c. (Possible split-brain)
[2017-03-17 13:18:26.919832] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:7e1768e0-ed80-4b0d-8535-daa8e11ec74c>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.919901] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 7e1768e0-ed80-4b0d-8535-daa8e11ec74c [Stale file handle]
[2017-03-17 13:18:26.934860] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid a4f5f161-7ac0-4460-9e55-3dbd32fb81af. (Possible split-brain)
[2017-03-17 13:18:26.938090] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:a4f5f161-7ac0-4460-9e55-3dbd32fb81af>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.938150] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: a4f5f161-7ac0-4460-9e55-3dbd32fb81af [Stale file handle]
[2017-03-17 13:18:26.942140] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid c257c9e1-130a-44a8-9e0d-2bbe61c7cd1d. (Possible split-brain)
[2017-03-17 13:18:26.944433] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:c257c9e1-130a-44a8-9e0d-2bbe61c7cd1d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:18:26.944493] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: c257c9e1-130a-44a8-9e0d-2bbe61c7cd1d [Stale file handle]
[2017-03-17 13:18:26.993155] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:18:26.993896] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 13:18:26.994474] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:18:26.994557] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:19:28.063796] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 1dfcf718-b423-4e85-8627-c01d064998b4. (Possible split-brain)
[2017-03-17 13:19:28.068008] E [shard.c:426:shard_modify_size_and_block_count] (-->/usr/lib64/glusterfs/3.8.10/xlator/cluster/distribute.so(+0x60ed7) [0x7f1268254ed7] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xb6f0) [0x7f125bde96f0] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xaf5d) [0x7f125bde8f5d] ) 12-vmware2-shard: Failed to get trusted.glusterfs.shard.file-size for 1dfcf718-b423-4e85-8627-c01d064998b4
[2017-03-17 13:19:28.094950] I [MSGID: 114024] [client-helpers.c:99:this_fd_set_ctx] 12-vmware2-client-6: /gfstest/gfstest.vmx (1dfcf718-b423-4e85-8627-c01d064998b4): trying duplicate remote fd set. 
[2017-03-17 13:19:28.095022] I [MSGID: 114024] [client-helpers.c:99:this_fd_set_ctx] 12-vmware2-client-7: /gfstest/gfstest.vmx (1dfcf718-b423-4e85-8627-c01d064998b4): trying duplicate remote fd set. 
[2017-03-17 13:19:28.095707] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-3)
[2017-03-17 13:19:28.104288] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 12-vmware2-client-7: remote operation failed [No such file or directory]
[2017-03-17 13:19:28.104784] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-6: remote operation failed. Path: <gfid:1dfcf718-b423-4e85-8627-c01d064998b4> (1dfcf718-b423-4e85-8627-c01d064998b4) [No such file or directory]
[2017-03-17 13:19:28.104797] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-7: remote operation failed. Path: <gfid:1dfcf718-b423-4e85-8627-c01d064998b4> (1dfcf718-b423-4e85-8627-c01d064998b4) [No such file or directory]
[2017-03-17 13:19:28.105547] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-3: Unreadable subvolume -1 found with event generation 2 for gfid 1dfcf718-b423-4e85-8627-c01d064998b4. (Possible split-brain)
[2017-03-17 13:19:28.106013] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-6: remote operation failed. Path: <gfid:1dfcf718-b423-4e85-8627-c01d064998b4> (1dfcf718-b423-4e85-8627-c01d064998b4) [No such file or directory]
[2017-03-17 13:19:28.106118] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-7: remote operation failed. Path: <gfid:1dfcf718-b423-4e85-8627-c01d064998b4> (1dfcf718-b423-4e85-8627-c01d064998b4) [No such file or directory]
[2017-03-17 13:19:28.107195] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-7: remote operation failed. Path: <gfid:1dfcf718-b423-4e85-8627-c01d064998b4> (1dfcf718-b423-4e85-8627-c01d064998b4) [No such file or directory]
[2017-03-17 13:19:28.107232] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-6: remote operation failed. Path: <gfid:1dfcf718-b423-4e85-8627-c01d064998b4> (1dfcf718-b423-4e85-8627-c01d064998b4) [No such file or directory]
[2017-03-17 13:19:28.108519] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:1dfcf718-b423-4e85-8627-c01d064998b4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.108590] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 1dfcf718-b423-4e85-8627-c01d064998b4 [Stale file handle]
[2017-03-17 13:19:28.124654] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 20c43b4f-0702-4233-ae5a-3e9ca1bdf87d. (Possible split-brain)
[2017-03-17 13:19:28.127782] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:20c43b4f-0702-4233-ae5a-3e9ca1bdf87d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.127848] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 20c43b4f-0702-4233-ae5a-3e9ca1bdf87d [Stale file handle]
[2017-03-17 13:19:28.130630] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 739913df-e0dc-4bad-82ed-af70a9b90640. (Possible split-brain)
[2017-03-17 13:19:28.133975] E [shard.c:426:shard_modify_size_and_block_count] (-->/usr/lib64/glusterfs/3.8.10/xlator/cluster/distribute.so(+0x60ed7) [0x7f1268254ed7] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xb6f0) [0x7f125bde96f0] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xaf5d) [0x7f125bde8f5d] ) 12-vmware2-shard: Failed to get trusted.glusterfs.shard.file-size for 739913df-e0dc-4bad-82ed-af70a9b90640
[2017-03-17 13:19:28.199615] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid 78e168ac-3c4e-4b02-942d-060337149b4c. (Possible split-brain)
[2017-03-17 13:19:28.202469] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:78e168ac-3c4e-4b02-942d-060337149b4c>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.202535] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 78e168ac-3c4e-4b02-942d-060337149b4c [Stale file handle]
[2017-03-17 13:19:28.219178] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-12: Unreadable subvolume -1 found with event generation 2 for gfid 27c0b12c-e3d4-43d3-8984-287e2e3c101a. (Possible split-brain)
[2017-03-17 13:19:28.222477] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:27c0b12c-e3d4-43d3-8984-287e2e3c101a>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.222529] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 27c0b12c-e3d4-43d3-8984-287e2e3c101a [Stale file handle]
[2017-03-17 13:19:28.226276] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid c3f24a99-5c1c-4a8e-9210-efa28adcf099. (Possible split-brain)
[2017-03-17 13:19:28.228831] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:c3f24a99-5c1c-4a8e-9210-efa28adcf099>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.228886] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: c3f24a99-5c1c-4a8e-9210-efa28adcf099 [Stale file handle]
[2017-03-17 13:19:28.345176] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:19:28.353005] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 12-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:19:28.353517] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:39894805-10d9-4090-9938-0fc8672740dc> (39894805-10d9-4090-9938-0fc8672740dc) [No such file or directory]
[2017-03-17 13:19:28.353579] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:39894805-10d9-4090-9938-0fc8672740dc> (39894805-10d9-4090-9938-0fc8672740dc) [No such file or directory]
[2017-03-17 13:19:28.354068] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 39894805-10d9-4090-9938-0fc8672740dc. (Possible split-brain)
[2017-03-17 13:19:28.354631] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:39894805-10d9-4090-9938-0fc8672740dc> (39894805-10d9-4090-9938-0fc8672740dc) [No such file or directory]
[2017-03-17 13:19:28.354631] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:39894805-10d9-4090-9938-0fc8672740dc> (39894805-10d9-4090-9938-0fc8672740dc) [No such file or directory]
[2017-03-17 13:19:28.355756] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:39894805-10d9-4090-9938-0fc8672740dc> (39894805-10d9-4090-9938-0fc8672740dc) [No such file or directory]
[2017-03-17 13:19:28.355809] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:39894805-10d9-4090-9938-0fc8672740dc> (39894805-10d9-4090-9938-0fc8672740dc) [No such file or directory]
[2017-03-17 13:19:28.357059] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:39894805-10d9-4090-9938-0fc8672740dc>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.357125] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 39894805-10d9-4090-9938-0fc8672740dc [Stale file handle]
[2017-03-17 13:19:28.364159] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid fa486c28-d01d-4250-a834-e6139edadabe. (Possible split-brain)
[2017-03-17 13:19:28.366921] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:fa486c28-d01d-4250-a834-e6139edadabe>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.366970] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: fa486c28-d01d-4250-a834-e6139edadabe [Stale file handle]
[2017-03-17 13:19:28.386414] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-21: Unreadable subvolume -1 found with event generation 2 for gfid 679dfc2e-2455-43bb-80d5-87eff0d4c81d. (Possible split-brain)
[2017-03-17 13:19:28.389037] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:679dfc2e-2455-43bb-80d5-87eff0d4c81d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.389085] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 679dfc2e-2455-43bb-80d5-87eff0d4c81d [Stale file handle]
[2017-03-17 13:19:28.393245] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-12: Unreadable subvolume -1 found with event generation 2 for gfid 2028be7a-2f01-472a-8d5d-c980d22454ad. (Possible split-brain)
[2017-03-17 13:19:28.396352] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:2028be7a-2f01-472a-8d5d-c980d22454ad>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.396417] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 2028be7a-2f01-472a-8d5d-c980d22454ad [Stale file handle]
[2017-03-17 13:19:28.422079] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid d997963b-5385-4019-9089-cbe4edbfa1a5. (Possible split-brain)
[2017-03-17 13:19:28.425270] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:d997963b-5385-4019-9089-cbe4edbfa1a5>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.425320] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: d997963b-5385-4019-9089-cbe4edbfa1a5 [Stale file handle]
[2017-03-17 13:19:28.440511] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-3: Unreadable subvolume -1 found with event generation 2 for gfid ec8a2a94-1091-4e42-a0be-b56f68c4c555. (Possible split-brain)
[2017-03-17 13:19:28.443653] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:ec8a2a94-1091-4e42-a0be-b56f68c4c555>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.443713] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: ec8a2a94-1091-4e42-a0be-b56f68c4c555 [Stale file handle]
[2017-03-17 13:19:28.447666] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid 9d8f6bad-5831-43f3-902b-4b85c35e0895. (Possible split-brain)
[2017-03-17 13:19:28.450090] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:9d8f6bad-5831-43f3-902b-4b85c35e0895>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:28.450155] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 9d8f6bad-5831-43f3-902b-4b85c35e0895 [Stale file handle]
[2017-03-17 13:19:28.499097] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:19:28.499356] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:19:28.499552] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:19:36.221983] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming /New Virtual Machine/vmware.log (hash=vmware2-replicate-19/cache=vmware2-replicate-19) => /New Virtual Machine/vmware-1.log (hash=vmware2-replicate-7/cache=<nul>)
[2017-03-17 13:19:36.228718] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-16: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:19:36.228848] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-17: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:19:36.428483] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-9/cache=vmware2-replicate-9) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-9)
[2017-03-17 13:19:36.435908] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 12-vmware2-client-19: remote operation failed [No such file or directory]
[2017-03-17 13:19:36.436442] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-18: remote operation failed. Path: <gfid:df0b9830-6a91-40fa-b935-68dc9b85d8e9> (df0b9830-6a91-40fa-b935-68dc9b85d8e9) [No such file or directory]
[2017-03-17 13:19:36.436459] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-19: remote operation failed. Path: <gfid:df0b9830-6a91-40fa-b935-68dc9b85d8e9> (df0b9830-6a91-40fa-b935-68dc9b85d8e9) [No such file or directory]
[2017-03-17 13:19:36.437353] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid df0b9830-6a91-40fa-b935-68dc9b85d8e9. (Possible split-brain)
[2017-03-17 13:19:36.437930] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-18: remote operation failed. Path: <gfid:df0b9830-6a91-40fa-b935-68dc9b85d8e9> (df0b9830-6a91-40fa-b935-68dc9b85d8e9) [No such file or directory]
[2017-03-17 13:19:36.437974] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-19: remote operation failed. Path: <gfid:df0b9830-6a91-40fa-b935-68dc9b85d8e9> (df0b9830-6a91-40fa-b935-68dc9b85d8e9) [No such file or directory]
[2017-03-17 13:19:36.439133] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-18: remote operation failed. Path: <gfid:df0b9830-6a91-40fa-b935-68dc9b85d8e9> (df0b9830-6a91-40fa-b935-68dc9b85d8e9) [No such file or directory]
[2017-03-17 13:19:36.439188] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-19: remote operation failed. Path: <gfid:df0b9830-6a91-40fa-b935-68dc9b85d8e9> (df0b9830-6a91-40fa-b935-68dc9b85d8e9) [No such file or directory]
[2017-03-17 13:19:36.440081] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:df0b9830-6a91-40fa-b935-68dc9b85d8e9>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:36.440157] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: df0b9830-6a91-40fa-b935-68dc9b85d8e9 [Stale file handle]
[2017-03-17 13:19:36.528710] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-9/cache=vmware2-replicate-9) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-9)
[2017-03-17 13:19:36.536385] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 12-vmware2-client-19: remote operation failed [No such file or directory]
[2017-03-17 13:19:36.536903] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-19: remote operation failed. Path: <gfid:ebf2b82e-b71f-43ff-bc62-a6c0b8b70286> (ebf2b82e-b71f-43ff-bc62-a6c0b8b70286) [No such file or directory]
[2017-03-17 13:19:36.536923] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-18: remote operation failed. Path: <gfid:ebf2b82e-b71f-43ff-bc62-a6c0b8b70286> (ebf2b82e-b71f-43ff-bc62-a6c0b8b70286) [No such file or directory]
[2017-03-17 13:19:36.537557] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid ebf2b82e-b71f-43ff-bc62-a6c0b8b70286. (Possible split-brain)
[2017-03-17 13:19:36.538069] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-18: remote operation failed. Path: <gfid:ebf2b82e-b71f-43ff-bc62-a6c0b8b70286> (ebf2b82e-b71f-43ff-bc62-a6c0b8b70286) [No such file or directory]
[2017-03-17 13:19:36.538110] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-19: remote operation failed. Path: <gfid:ebf2b82e-b71f-43ff-bc62-a6c0b8b70286> (ebf2b82e-b71f-43ff-bc62-a6c0b8b70286) [No such file or directory]
[2017-03-17 13:19:36.539670] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-19: remote operation failed. Path: <gfid:ebf2b82e-b71f-43ff-bc62-a6c0b8b70286> (ebf2b82e-b71f-43ff-bc62-a6c0b8b70286) [No such file or directory]
[2017-03-17 13:19:36.539689] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-18: remote operation failed. Path: <gfid:ebf2b82e-b71f-43ff-bc62-a6c0b8b70286> (ebf2b82e-b71f-43ff-bc62-a6c0b8b70286) [No such file or directory]
[2017-03-17 13:19:36.541020] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:ebf2b82e-b71f-43ff-bc62-a6c0b8b70286>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:36.541087] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: ebf2b82e-b71f-43ff-bc62-a6c0b8b70286 [Stale file handle]
[2017-03-17 13:19:38.253448] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmdk~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine/New Virtual Machine.vmdk (hash=vmware2-replicate-11/cache=vmware2-replicate-18)
[2017-03-17 13:19:38.262441] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 12-vmware2-client-36: remote operation failed [No such file or directory]
[2017-03-17 13:19:38.262964] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-36: remote operation failed. Path: <gfid:a72a2a70-29b1-4dbd-8a6f-9046101c2657> (a72a2a70-29b1-4dbd-8a6f-9046101c2657) [No such file or directory]
[2017-03-17 13:19:38.262967] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-37: remote operation failed. Path: <gfid:a72a2a70-29b1-4dbd-8a6f-9046101c2657> (a72a2a70-29b1-4dbd-8a6f-9046101c2657) [No such file or directory]
[2017-03-17 13:19:38.263506] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid a72a2a70-29b1-4dbd-8a6f-9046101c2657. (Possible split-brain)
[2017-03-17 13:19:38.263981] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-37: remote operation failed. Path: <gfid:a72a2a70-29b1-4dbd-8a6f-9046101c2657> (a72a2a70-29b1-4dbd-8a6f-9046101c2657) [No such file or directory]
[2017-03-17 13:19:38.264021] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-36: remote operation failed. Path: <gfid:a72a2a70-29b1-4dbd-8a6f-9046101c2657> (a72a2a70-29b1-4dbd-8a6f-9046101c2657) [No such file or directory]
[2017-03-17 13:19:38.265823] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-36: remote operation failed. Path: <gfid:a72a2a70-29b1-4dbd-8a6f-9046101c2657> (a72a2a70-29b1-4dbd-8a6f-9046101c2657) [No such file or directory]
[2017-03-17 13:19:38.265911] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-37: remote operation failed. Path: <gfid:a72a2a70-29b1-4dbd-8a6f-9046101c2657> (a72a2a70-29b1-4dbd-8a6f-9046101c2657) [No such file or directory]
[2017-03-17 13:19:38.266361] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:a72a2a70-29b1-4dbd-8a6f-9046101c2657>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:38.266414] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: a72a2a70-29b1-4dbd-8a6f-9046101c2657 [Stale file handle]
[2017-03-17 13:19:38.270927] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid c23f6b57-e4e1-41d7-8696-9d9598ceec08. (Possible split-brain)
[2017-03-17 13:19:38.273647] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:c23f6b57-e4e1-41d7-8696-9d9598ceec08>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:19:38.273716] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: c23f6b57-e4e1-41d7-8696-9d9598ceec08 [Stale file handle]
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-0: remote operation failed. Path: <gfid:2dd2207f-c243-4a79-ad75-963cda659cdf> (2dd2207f-c243-4a79-ad75-963cda659cdf) [No such file or directory]" repeated 2 times between [2017-03-17 13:18:26.599057] and [2017-03-17 13:18:26.601237]
[2017-03-17 13:21:48.483628] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/vmware.log (hash=vmware2-replicate-9/cache=vmware2-replicate-0) => <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/vmware-1.log (hash=vmware2-replicate-5/cache=<nul>)
[2017-03-17 13:21:48.574831] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:21:48.692146] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:21:48.699485] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 12-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:21:48.699984] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:327e57d4-20e3-40b8-83d4-5d687d35ac6e> (327e57d4-20e3-40b8-83d4-5d687d35ac6e) [No such file or directory]
[2017-03-17 13:21:48.700036] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:327e57d4-20e3-40b8-83d4-5d687d35ac6e> (327e57d4-20e3-40b8-83d4-5d687d35ac6e) [No such file or directory]
[2017-03-17 13:21:48.700759] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 327e57d4-20e3-40b8-83d4-5d687d35ac6e. (Possible split-brain)
[2017-03-17 13:21:48.701196] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:327e57d4-20e3-40b8-83d4-5d687d35ac6e> (327e57d4-20e3-40b8-83d4-5d687d35ac6e) [No such file or directory]
[2017-03-17 13:21:48.701294] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:327e57d4-20e3-40b8-83d4-5d687d35ac6e> (327e57d4-20e3-40b8-83d4-5d687d35ac6e) [No such file or directory]
[2017-03-17 13:21:48.702397] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:327e57d4-20e3-40b8-83d4-5d687d35ac6e> (327e57d4-20e3-40b8-83d4-5d687d35ac6e) [No such file or directory]
[2017-03-17 13:21:48.702450] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:327e57d4-20e3-40b8-83d4-5d687d35ac6e> (327e57d4-20e3-40b8-83d4-5d687d35ac6e) [No such file or directory]
[2017-03-17 13:21:48.703827] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:327e57d4-20e3-40b8-83d4-5d687d35ac6e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:21:48.703915] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 327e57d4-20e3-40b8-83d4-5d687d35ac6e [Stale file handle]
[2017-03-17 13:21:48.779170] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 12-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:21:48.779581] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:eb329ed0-5abb-4476-845d-edbebd6d7d70> (eb329ed0-5abb-4476-845d-edbebd6d7d70) [No such file or directory]
[2017-03-17 13:21:48.779565] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:eb329ed0-5abb-4476-845d-edbebd6d7d70> (eb329ed0-5abb-4476-845d-edbebd6d7d70) [No such file or directory]
[2017-03-17 13:21:48.780057] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid eb329ed0-5abb-4476-845d-edbebd6d7d70. (Possible split-brain)
[2017-03-17 13:21:48.780573] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:eb329ed0-5abb-4476-845d-edbebd6d7d70> (eb329ed0-5abb-4476-845d-edbebd6d7d70) [No such file or directory]
[2017-03-17 13:21:48.780619] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:eb329ed0-5abb-4476-845d-edbebd6d7d70> (eb329ed0-5abb-4476-845d-edbebd6d7d70) [No such file or directory]
[2017-03-17 13:21:48.781886] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:eb329ed0-5abb-4476-845d-edbebd6d7d70> (eb329ed0-5abb-4476-845d-edbebd6d7d70) [No such file or directory]
[2017-03-17 13:21:48.781914] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:eb329ed0-5abb-4476-845d-edbebd6d7d70> (eb329ed0-5abb-4476-845d-edbebd6d7d70) [No such file or directory]
[2017-03-17 13:21:48.783469] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:eb329ed0-5abb-4476-845d-edbebd6d7d70>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:21:48.783515] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: eb329ed0-5abb-4476-845d-edbebd6d7d70 [Stale file handle]
[2017-03-17 13:21:48.772286] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:24:29.083665] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 888bb294-6d66-4be2-881c-ed8c57babe5c. (Possible split-brain)
[2017-03-17 13:24:29.087041] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:888bb294-6d66-4be2-881c-ed8c57babe5c>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:24:29.087136] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 888bb294-6d66-4be2-881c-ed8c57babe5c [Stale file handle]
[2017-03-17 13:24:29.130699] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid eb7d544b-f5f5-411c-be4f-016c098bbe15. (Possible split-brain)
[2017-03-17 13:24:29.133718] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:eb7d544b-f5f5-411c-be4f-016c098bbe15>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:24:29.133809] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: eb7d544b-f5f5-411c-be4f-016c098bbe15 [Stale file handle]
[2017-03-17 13:24:29.149481] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 26428c87-f537-4964-a470-24cf137b0acf. (Possible split-brain)
[2017-03-17 13:24:29.152463] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:26428c87-f537-4964-a470-24cf137b0acf>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:24:29.152534] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 26428c87-f537-4964-a470-24cf137b0acf [Stale file handle]
[2017-03-17 13:24:29.156608] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid db3baadf-0d93-4c87-b17a-a8530c7da335. (Possible split-brain)
[2017-03-17 13:24:29.159156] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:db3baadf-0d93-4c87-b17a-a8530c7da335>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:24:29.159225] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: db3baadf-0d93-4c87-b17a-a8530c7da335 [Stale file handle]
[2017-03-17 13:24:29.288139] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:24:29.288717] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:24:29.402935] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 12-vmware2-dht: renaming <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-7)
[2017-03-17 13:24:29.410004] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 12-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:24:29.410486] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:49cd3775-832f-4f97-8f91-4633059dd60d> (49cd3775-832f-4f97-8f91-4633059dd60d) [No such file or directory]
[2017-03-17 13:24:29.410516] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:49cd3775-832f-4f97-8f91-4633059dd60d> (49cd3775-832f-4f97-8f91-4633059dd60d) [No such file or directory]
[2017-03-17 13:24:29.411042] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 49cd3775-832f-4f97-8f91-4633059dd60d. (Possible split-brain)
[2017-03-17 13:24:29.411579] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:49cd3775-832f-4f97-8f91-4633059dd60d> (49cd3775-832f-4f97-8f91-4633059dd60d) [No such file or directory]
[2017-03-17 13:24:29.411581] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:49cd3775-832f-4f97-8f91-4633059dd60d> (49cd3775-832f-4f97-8f91-4633059dd60d) [No such file or directory]
[2017-03-17 13:24:29.412753] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:49cd3775-832f-4f97-8f91-4633059dd60d> (49cd3775-832f-4f97-8f91-4633059dd60d) [No such file or directory]
[2017-03-17 13:24:29.412856] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:49cd3775-832f-4f97-8f91-4633059dd60d> (49cd3775-832f-4f97-8f91-4633059dd60d) [No such file or directory]
[2017-03-17 13:24:29.414389] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 12-vmware2-dht: <gfid:49cd3775-832f-4f97-8f91-4633059dd60d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:24:29.414451] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 12-vmware2-shard: stat failed: 49cd3775-832f-4f97-8f91-4633059dd60d [Stale file handle]
[2017-03-17 13:27:24.622564] I [MSGID: 109086] [dht-shared.c:297:dht_parse_decommissioned_bricks] 12-vmware2-dht: decommissioning subvolume vmware2-replicate-20
[2017-03-17 13:27:24.622601] I [MSGID: 109086] [dht-shared.c:297:dht_parse_decommissioned_bricks] 12-vmware2-dht: decommissioning subvolume vmware2-replicate-21
[2017-03-17 13:27:24.646882] I [MSGID: 104043] [glfs-mgmt.c:565:glfs_mgmt_getspec_cbk] 0-gfapi: No change in volfile, continuing
[2017-03-17 13:27:33.589909] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-24: remote operation failed. Path: /New Virtual Machine/.lck-9219333f56b36b9c (058bcdef-7095-476b-9f7b-8c4c8cbcef82) [No such file or directory]
[2017-03-17 13:27:33.590196] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-32: remote operation failed. Path: <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/.lck-5279768c111cada5 (457c46d3-2e88-419e-afac-09392ee40920) [No such file or directory]
[2017-03-17 13:27:33.590304] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-25: remote operation failed. Path: /New Virtual Machine/.lck-9219333f56b36b9c (058bcdef-7095-476b-9f7b-8c4c8cbcef82) [No such file or directory]
[2017-03-17 13:27:33.591861] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-33: remote operation failed. Path: <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/.lck-5279768c111cada5 (457c46d3-2e88-419e-afac-09392ee40920) [No such file or directory]
[2017-03-17 13:27:33.595109] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-35: remote operation failed. Path: /New Virtual Machine/.lck-650d7f98b8400d84 (efef7c02-2411-4b92-890e-c07b143aebb2) [No such file or directory]
[2017-03-17 13:27:33.595380] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-31: remote operation failed. Path: /New Virtual Machine/.lck-06bba37fd1c3d499 (330f216d-6833-43ee-8074-74ca3a6e37bf) [No such file or directory]
[2017-03-17 13:27:33.595958] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-34: remote operation failed. Path: /New Virtual Machine/.lck-650d7f98b8400d84 (efef7c02-2411-4b92-890e-c07b143aebb2) [No such file or directory]
[2017-03-17 13:27:33.596692] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-30: remote operation failed. Path: /New Virtual Machine/.lck-06bba37fd1c3d499 (330f216d-6833-43ee-8074-74ca3a6e37bf) [No such file or directory]
[2017-03-17 13:27:42.076144] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-42: remote operation failed. Path: /.shard/7ae534a9-a215-43ea-99d4-c3d17fa3bb06.34 (a87dd2aa-0f07-403d-ba7b-1644920489cc) [No such file or directory]
[2017-03-17 13:27:42.076326] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-43: remote operation failed. Path: /.shard/7ae534a9-a215-43ea-99d4-c3d17fa3bb06.34 (a87dd2aa-0f07-403d-ba7b-1644920489cc) [No such file or directory]
[2017-03-17 13:27:42.129626] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-30: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.129960] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-30: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.130092] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-31: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.130462] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 27066740-67e4-408b-a26f-d92539d71487. (Possible split-brain)
[2017-03-17 13:27:42.130674] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-30: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.130800] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-31: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.162370] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-40: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.162457] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-24: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.162652] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-40: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.162739] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-24: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.162738] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-25: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.162770] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-41: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.163051] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-12: Unreadable subvolume -1 found with event generation 2 for gfid 32583b3b-4193-4fdd-89ac-e411784e63e7. (Possible split-brain)
[2017-03-17 13:27:42.163178] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-20: Unreadable subvolume -1 found with event generation 2 for gfid 591be5a5-8b3d-40d9-8882-9fe9087ea367. (Possible split-brain)
[2017-03-17 13:27:42.163273] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-24: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.163335] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-25: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.163425] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-40: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.163484] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-41: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.876001] W [MSGID: 108019] [afr-lk-common.c:1082:is_blocking_locks_count_sufficient] 12-vmware2-replicate-21: Unable to obtain blocking inode lock on even one child for gfid:524a67c2-2396-454c-b8b6-e832c9ca6b66.
[2017-03-17 13:27:42.876026] I [MSGID: 108019] [afr-transaction.c:1478:afr_post_blocking_inodelk_cbk] 12-vmware2-replicate-21: Blocking inodelks failed.
[2017-03-17 13:27:42.876800] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-42: remote operation failed [No such file or directory]
[2017-03-17 13:27:42.876938] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-43: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.329205] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-42: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.329556] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-42: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.329606] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-43: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.329956] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-21: Unreadable subvolume -1 found with event generation 2 for gfid d3181ab1-d719-44d9-9ff0-03ab18de7885. (Possible split-brain)
[2017-03-17 13:27:44.330352] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-43: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.330361] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-42: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.375877] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-43: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.376408] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-43: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.376486] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-42: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.376753] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-21: Unreadable subvolume -1 found with event generation 2 for gfid 77cb1e6a-6b47-411c-bdbb-bfddfc9a3cf5. (Possible split-brain)
[2017-03-17 13:27:44.377179] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-42: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.377213] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-43: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.582085] W [MSGID: 108019] [afr-lk-common.c:1082:is_blocking_locks_count_sufficient] 12-vmware2-replicate-7: Unable to obtain blocking inode lock on even one child for gfid:621839a9-a27e-4a1f-b466-87841be490f7.
[2017-03-17 13:27:44.582129] I [MSGID: 108019] [afr-transaction.c:1478:afr_post_blocking_inodelk_cbk] 12-vmware2-replicate-7: Blocking inodelks failed.
[2017-03-17 13:27:44.583316] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 13:27:44.583383] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 13:27:45.145240] W [MSGID: 108019] [afr-lk-common.c:1082:is_blocking_locks_count_sufficient] 12-vmware2-replicate-5: Unable to obtain blocking inode lock on even one child for gfid:f9b4bd80-19ca-40f5-bc40-cdb1c01c8ff7.
[2017-03-17 13:27:45.145284] I [MSGID: 108019] [afr-transaction.c:1478:afr_post_blocking_inodelk_cbk] 12-vmware2-replicate-5: Blocking inodelks failed.
[2017-03-17 13:27:45.146439] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 13:27:45.146448] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 13:27:45.702913] W [MSGID: 108019] [afr-lk-common.c:1082:is_blocking_locks_count_sufficient] 12-vmware2-replicate-21: Unable to obtain blocking inode lock on even one child for gfid:43dcdd29-1fcd-46df-b277-f11e7390a046.
[2017-03-17 13:27:45.702941] I [MSGID: 108019] [afr-transaction.c:1478:afr_post_blocking_inodelk_cbk] 12-vmware2-replicate-21: Blocking inodelks failed.
[2017-03-17 13:27:45.703781] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-42: remote operation failed [No such file or directory]
[2017-03-17 13:27:45.703841] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-43: remote operation failed [No such file or directory]
[2017-03-17 13:27:55.199729] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-0; inode layout - 3709289934 - 3904515719 - 1; disk layout - 3435973824 - 3650722187 - 1
[2017-03-17 13:27:55.199828] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.199867] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-2; inode layout - 3318838362 - 3514064147 - 1; disk layout - 3221225460 - 3435973823 - 1
[2017-03-17 13:27:55.199895] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.200081] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-1; inode layout - 2928386790 - 3123612575 - 1; disk layout - 3650722188 - 3865470551 - 1
[2017-03-17 13:27:55.200114] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.200153] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-3; inode layout - 1952257860 - 2147483645 - 1; disk layout - 1932735276 - 2147483639 - 1
[2017-03-17 13:27:55.200178] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.200222] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:27:55.200444] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:27:55.200488] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-4; inode layout - 780903144 - 976128929 - 1; disk layout - 858993456 - 1073741819 - 1
[2017-03-17 13:27:55.200534] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.200779] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-7; inode layout - 195225786 - 390451571 - 1; disk layout - 214748364 - 429496727 - 1
[2017-03-17 13:27:55.200821] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.200944] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-9; inode layout - 585677358 - 780903143 - 1; disk layout - 644245092 - 858993455 - 1
[2017-03-17 13:27:55.200971] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.201054] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-5; inode layout - 976128930 - 1171354715 - 1; disk layout - 1073741820 - 1288490183 - 1
[2017-03-17 13:27:55.201086] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.201237] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-10; inode layout - 4099741506 - 4294967295 - 1; disk layout - 4080218916 - 4294967295 - 1
[2017-03-17 13:27:55.201289] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.201316] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-6; inode layout - 0 - 195225785 - 1; disk layout - 0 - 214748363 - 1
[2017-03-17 13:27:55.201339] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.201478] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-8; inode layout - 1366580502 - 1561806287 - 1; disk layout - 3006477096 - 3221225459 - 1
[2017-03-17 13:27:55.201508] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.201624] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 13:27:55.201918] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-15; inode layout - 2342709432 - 2537935217 - 1; disk layout - 2362232004 - 2576980367 - 1
[2017-03-17 13:27:55.201956] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.201985] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-14; inode layout - 2147483646 - 2342709431 - 1; disk layout - 2147483640 - 2362232003 - 1
[2017-03-17 13:27:55.202009] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.202076] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:27:55.202117] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 13:27:55.202340] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-11; inode layout - 2537935218 - 2733161003 - 1; disk layout - 2576980368 - 2791728731 - 1
[2017-03-17 13:27:55.202382] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.202408] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-12; inode layout - 1757032074 - 1952257859 - 1; disk layout - 1717986912 - 1932735275 - 1
[2017-03-17 13:27:55.202430] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.202582] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-13; inode layout - 390451572 - 585677357 - 1; disk layout - 429496728 - 644245091 - 1
[2017-03-17 13:27:55.202610] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.202738] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-19; inode layout - 3123612576 - 3318838361 - 1; disk layout - 1503238548 - 1717986911 - 1
[2017-03-17 13:27:55.202753] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:27:55.202777] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.203010] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-16; inode layout - 3904515720 - 4099741505 - 1; disk layout - 3865470552 - 4080218915 - 1
[2017-03-17 13:27:55.203041] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.203065] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-17; inode layout - 2733161004 - 2928386789 - 1; disk layout - 2791728732 - 3006477095 - 1
[2017-03-17 13:27:55.203086] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.203158] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:27:55.203184] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:27:55.203209] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:27:55.203336] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-18; inode layout - 1561806288 - 1757032073 - 1; disk layout - 1288490184 - 1503238547 - 1
[2017-03-17 13:27:55.203362] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.203385] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-20; inode layout - 3514064148 - 3709289933 - 1; disk layout - 0 - 0 - 0
[2017-03-17 13:27:55.203407] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.203493] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:27:55.203591] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 12-vmware2-dht: subvol: vmware2-replicate-21; inode layout - 1171354716 - 1366580501 - 1; disk layout - 0 - 0 - 0
[2017-03-17 13:27:55.203619] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 12-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 13:27:55.203663] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 12-vmware2-replicate-20: selecting local read_child vmware2-client-40
[2017-03-17 13:27:59.047753] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-43: remote operation failed [No such file or directory]
[2017-03-17 13:27:59.048253] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-42: remote operation failed [No such file or directory]
[2017-03-17 13:27:59.048300] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-43: remote operation failed [No such file or directory]
[2017-03-17 13:27:59.048973] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-21: Unreadable subvolume -1 found with event generation 2 for gfid d59bfc9d-d366-4549-9ce3-e621e76595f2. (Possible split-brain)
[2017-03-17 13:27:59.049502] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-42: remote operation failed [No such file or directory]
[2017-03-17 13:27:59.049567] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-43: remote operation failed [No such file or directory]
[2017-03-17 13:27:59.150468] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 12-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 13:27:59.150812] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 13:27:59.150963] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 13:27:59.151369] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 12-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid a7cff204-f542-4a70-9bd4-4208448abc8e. (Possible split-brain)
[2017-03-17 13:27:59.151671] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 13:27:59.151828] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 12-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 13:28:02.028791] W [socket.c:590:__socket_rwv] 12-vmware2-client-40: readv on 192.168.209.194:49523 failed (No data available)
[2017-03-17 13:28:02.028833] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-40: disconnected from vmware2-client-40. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:03.176049] W [socket.c:590:__socket_rwv] 12-vmware2-client-41: readv on 192.168.209.196:49500 failed (No data available)
[2017-03-17 13:28:03.176081] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-41: disconnected from vmware2-client-41. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:03.176095] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-20: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:03.176235] W [socket.c:590:__socket_rwv] 12-vmware2-client-43: readv on 192.168.209.197:49395 failed (No data available)
[2017-03-17 13:28:03.176258] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-43: disconnected from vmware2-client-43. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:03.176316] W [socket.c:590:__socket_rwv] 12-vmware2-client-42: readv on 192.168.209.195:49425 failed (No data available)
[2017-03-17 13:28:03.176344] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-42: disconnected from vmware2-client-42. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:03.176354] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-21: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:03.254688] I [MSGID: 104045] [glfs-master.c:91:notify] 0-gfapi: New graph 676c7573-7465-7230-312d-313938392d32 (15) coming up
[2017-03-17 13:28:03.254731] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-0: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.257207] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-1: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.257445] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-0: changing port to 49512 (from 0)
[2017-03-17 13:28:03.259548] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-2: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.264599] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:03.265078] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-0: Connected to vmware2-client-0, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:28:03.265105] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:03.265171] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-0: Subvolume 'vmware2-client-0' came back up; going online.
[2017-03-17 13:28:03.265208] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-0: Server lk version = 1
[2017-03-17 13:28:03.266495] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-3: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.268842] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-4: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.271202] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-5: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.271386] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-4: changing port to 49513 (from 0)
[2017-03-17 13:28:03.273536] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-6: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.277633] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-4: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:03.278008] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-4: Connected to vmware2-client-4, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:28:03.278031] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-4: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:03.278086] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-2: Subvolume 'vmware2-client-4' came back up; going online.
[2017-03-17 13:28:03.278116] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-4: Server lk version = 1
[2017-03-17 13:28:03.279587] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-7: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.281923] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-8: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.284265] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-9: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.284444] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-8: changing port to 49514 (from 0)
[2017-03-17 13:28:03.286597] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-10: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.290543] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-8: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:03.291005] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-8: Connected to vmware2-client-8, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:28:03.291026] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-8: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:03.291078] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-4: Subvolume 'vmware2-client-8' came back up; going online.
[2017-03-17 13:28:03.291137] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-8: Server lk version = 1
[2017-03-17 13:28:03.292499] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-11: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.294840] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-12: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.297193] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-13: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.297314] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-12: changing port to 49515 (from 0)
[2017-03-17 13:28:03.299568] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-14: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.301953] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-12: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:03.302292] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-12: Connected to vmware2-client-12, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:28:03.302304] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-12: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:03.302338] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-6: Subvolume 'vmware2-client-12' came back up; going online.
[2017-03-17 13:28:03.302376] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-12: Server lk version = 1
[2017-03-17 13:28:03.304036] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-15: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.306406] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-16: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.308750] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-17: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.308864] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-16: changing port to 49516 (from 0)
[2017-03-17 13:28:03.311095] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-18: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.313420] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-16: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:03.313686] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-16: Connected to vmware2-client-16, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:28:03.313713] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-16: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:03.313746] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-8: Subvolume 'vmware2-client-16' came back up; going online.
[2017-03-17 13:28:03.313769] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-16: Server lk version = 1
[2017-03-17 13:28:03.315798] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-19: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.318148] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-20: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.320497] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-21: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.320655] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-20: changing port to 49517 (from 0)
[2017-03-17 13:28:03.322847] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-22: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.326256] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-20: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:03.326649] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-20: Connected to vmware2-client-20, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:28:03.326664] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-20: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:03.326709] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-10: Subvolume 'vmware2-client-20' came back up; going online.
[2017-03-17 13:28:03.326755] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-20: Server lk version = 1
[2017-03-17 13:28:03.328336] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-23: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.330690] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-24: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.333050] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-25: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.333179] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-24: changing port to 49518 (from 0)
[2017-03-17 13:28:03.335405] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-26: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.337810] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-24: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:03.338236] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-24: Connected to vmware2-client-24, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:28:03.338251] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-24: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:03.338299] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-12: Subvolume 'vmware2-client-24' came back up; going online.
[2017-03-17 13:28:03.338351] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-24: Server lk version = 1
[2017-03-17 13:28:03.339838] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-27: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.342295] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-28: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.344657] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-29: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.344853] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-28: changing port to 49519 (from 0)
[2017-03-17 13:28:03.347014] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-30: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.352141] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-28: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:03.352589] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-28: Connected to vmware2-client-28, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:28:03.352618] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-28: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:03.352681] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-14: Subvolume 'vmware2-client-28' came back up; going online.
[2017-03-17 13:28:03.352772] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-28: Server lk version = 1
[2017-03-17 13:28:03.354011] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-31: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.356407] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-32: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.358778] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-33: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.358918] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-32: changing port to 49520 (from 0)
[2017-03-17 13:28:03.361133] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-34: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.364172] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-32: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:03.364522] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-32: Connected to vmware2-client-32, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:28:03.364535] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-32: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:03.364577] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-16: Subvolume 'vmware2-client-32' came back up; going online.
[2017-03-17 13:28:03.364599] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-32: Server lk version = 1
[2017-03-17 13:28:03.366371] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-35: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.368729] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-36: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.371123] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-37: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.371295] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-36: changing port to 49521 (from 0)
[2017-03-17 13:28:03.373483] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-38: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:03.377587] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-36: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:03.378033] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-36: Connected to vmware2-client-36, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:28:03.378056] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-36: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:03.378114] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-18: Subvolume 'vmware2-client-36' came back up; going online.
[2017-03-17 13:28:03.378198] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-36: Server lk version = 1
[2017-03-17 13:28:03.379525] I [MSGID: 114020] [client.c:2356:notify] 15-vmware2-client-39: parent translators are ready, attempting connect on transport
[2017-03-17 13:28:04.186821] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-20: no subvolumes up
[2017-03-17 13:28:04.186896] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-21: no subvolumes up
[2017-03-17 13:28:04.368595] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-3: changing port to 49385 (from 0)
[2017-03-17 13:28:04.368727] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-7: changing port to 49386 (from 0)
[2017-03-17 13:28:04.373975] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-11: changing port to 49387 (from 0)
[2017-03-17 13:28:04.374120] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-15: changing port to 49388 (from 0)
[2017-03-17 13:28:04.374171] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-19: changing port to 49389 (from 0)
[2017-03-17 13:28:04.387510] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-23: changing port to 49390 (from 0)
[2017-03-17 13:28:04.387569] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-27: changing port to 49391 (from 0)
[2017-03-17 13:28:04.387599] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-31: changing port to 49392 (from 0)
[2017-03-17 13:28:04.387623] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-35: changing port to 49393 (from 0)
[2017-03-17 13:28:04.387646] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-39: changing port to 49394 (from 0)
[2017-03-17 13:28:04.387675] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-3: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.407846] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-7: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.407940] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-11: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.407998] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-1: changing port to 49489 (from 0)
[2017-03-17 13:28:04.408027] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-5: changing port to 49490 (from 0)
[2017-03-17 13:28:04.408048] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-9: changing port to 49491 (from 0)
[2017-03-17 13:28:04.408068] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-13: changing port to 49492 (from 0)
[2017-03-17 13:28:04.408087] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-17: changing port to 49493 (from 0)
[2017-03-17 13:28:04.408106] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-21: changing port to 49494 (from 0)
[2017-03-17 13:28:04.408124] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-25: changing port to 49495 (from 0)
[2017-03-17 13:28:04.408144] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-29: changing port to 49496 (from 0)
[2017-03-17 13:28:04.408163] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-33: changing port to 49497 (from 0)
[2017-03-17 13:28:04.408183] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-37: changing port to 49498 (from 0)
[2017-03-17 13:28:04.408216] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-3: Connected to vmware2-client-3, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:28:04.408230] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-3: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.408272] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-1: Subvolume 'vmware2-client-3' came back up; going online.
[2017-03-17 13:28:04.408329] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-15: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.446597] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-7: Connected to vmware2-client-7, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:28:04.446627] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-7: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.446689] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-3: Subvolume 'vmware2-client-7' came back up; going online.
[2017-03-17 13:28:04.446723] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-19: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.446790] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-3: Server lk version = 1
[2017-03-17 13:28:04.446831] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-11: Connected to vmware2-client-11, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:28:04.446842] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-11: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.446880] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-5: Subvolume 'vmware2-client-11' came back up; going online.
[2017-03-17 13:28:04.446911] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-23: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.446961] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-31: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.447011] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-35: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.447064] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-27: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.447125] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-15: Connected to vmware2-client-15, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:28:04.447138] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-15: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.447175] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-7: Subvolume 'vmware2-client-15' came back up; going online.
[2017-03-17 13:28:04.447688] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-7: Server lk version = 1
[2017-03-17 13:28:04.447717] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-11: Server lk version = 1
[2017-03-17 13:28:04.447743] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-19: Connected to vmware2-client-19, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:28:04.447752] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-19: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.447803] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-9: Subvolume 'vmware2-client-19' came back up; going online.
[2017-03-17 13:28:04.447833] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-15: Server lk version = 1
[2017-03-17 13:28:04.447851] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-2: changing port to 49414 (from 0)
[2017-03-17 13:28:04.447888] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-31: Connected to vmware2-client-31, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:28:04.447898] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-31: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.447938] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-15: Subvolume 'vmware2-client-31' came back up; going online.
[2017-03-17 13:28:04.447968] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-23: Connected to vmware2-client-23, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:28:04.447985] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-23: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.448025] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-11: Subvolume 'vmware2-client-23' came back up; going online.
[2017-03-17 13:28:04.448047] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-6: changing port to 49415 (from 0)
[2017-03-17 13:28:04.448074] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-14: changing port to 49417 (from 0)
[2017-03-17 13:28:04.448093] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-10: changing port to 49416 (from 0)
[2017-03-17 13:28:04.448108] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-22: changing port to 49419 (from 0)
[2017-03-17 13:28:04.448124] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-18: changing port to 49418 (from 0)
[2017-03-17 13:28:04.448178] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-27: Connected to vmware2-client-27, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:28:04.448189] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-27: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.448230] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-13: Subvolume 'vmware2-client-27' came back up; going online.
[2017-03-17 13:28:04.448277] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-26: changing port to 49420 (from 0)
[2017-03-17 13:28:04.448304] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-39: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.448376] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.448434] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-30: changing port to 49421 (from 0)
[2017-03-17 13:28:04.448455] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-34: changing port to 49422 (from 0)
[2017-03-17 13:28:04.448489] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-35: Connected to vmware2-client-35, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:28:04.448502] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-35: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.448542] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-17: Subvolume 'vmware2-client-35' came back up; going online.
[2017-03-17 13:28:04.448569] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 15-vmware2-client-38: changing port to 49423 (from 0)
[2017-03-17 13:28:04.448605] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-13: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.448659] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-5: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.448746] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-17: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.448827] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-9: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.448897] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-21: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.448952] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-29: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.449016] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-25: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.449068] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-33: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.449131] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-19: Server lk version = 1
[2017-03-17 13:28:04.450504] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-31: Server lk version = 1
[2017-03-17 13:28:04.463592] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-23: Server lk version = 1
[2017-03-17 13:28:04.469932] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-27: Server lk version = 1
[2017-03-17 13:28:04.478822] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-35: Server lk version = 1
[2017-03-17 13:28:04.478869] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-39: Connected to vmware2-client-39, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:28:04.478881] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-39: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.478920] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 15-vmware2-replicate-19: Subvolume 'vmware2-client-39' came back up; going online.
[2017-03-17 13:28:04.478953] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-1: Connected to vmware2-client-1, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:28:04.478961] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-1: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.479097] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-13: Connected to vmware2-client-13, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:28:04.479110] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-13: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.479206] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-5: Connected to vmware2-client-5, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:28:04.479218] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-5: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.479312] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-17: Connected to vmware2-client-17, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:28:04.479324] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-17: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.479420] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-9: Connected to vmware2-client-9, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:28:04.479437] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-9: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.479533] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-21: Connected to vmware2-client-21, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:28:04.479544] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-21: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.479624] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-29: Connected to vmware2-client-29, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:28:04.479642] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-29: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.479722] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-25: Connected to vmware2-client-25, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:28:04.479733] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-25: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.479838] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-33: Connected to vmware2-client-33, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:28:04.479855] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-33: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.480115] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-39: Server lk version = 1
[2017-03-17 13:28:04.480135] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-1: Server lk version = 1
[2017-03-17 13:28:04.480156] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-13: Server lk version = 1
[2017-03-17 13:28:04.480187] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-5: Server lk version = 1
[2017-03-17 13:28:04.480215] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-17: Server lk version = 1
[2017-03-17 13:28:04.480233] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-9: Server lk version = 1
[2017-03-17 13:28:04.480245] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-21: Server lk version = 1
[2017-03-17 13:28:04.480268] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-29: Server lk version = 1
[2017-03-17 13:28:04.480279] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-25: Server lk version = 1
[2017-03-17 13:28:04.480297] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-33: Server lk version = 1
[2017-03-17 13:28:04.480318] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-37: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.480363] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-6: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.480414] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.480455] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-14: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.480507] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-10: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.480564] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-22: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.480604] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-34: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.480643] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-26: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.480681] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-30: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.480729] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-18: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.480857] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-37: Connected to vmware2-client-37, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:28:04.480874] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-37: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.481034] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-6: Connected to vmware2-client-6, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:28:04.481047] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-6: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.481148] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-2: Connected to vmware2-client-2, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:28:04.481160] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.481243] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-10: Connected to vmware2-client-10, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:28:04.481254] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-10: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.481315] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-37: Server lk version = 1
[2017-03-17 13:28:04.481337] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-14: Connected to vmware2-client-14, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:28:04.481344] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-14: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.481409] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-22: Connected to vmware2-client-22, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:28:04.481419] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-22: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.481489] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-34: Connected to vmware2-client-34, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:28:04.481499] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-34: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.481565] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-26: Connected to vmware2-client-26, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:28:04.481575] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-26: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.481633] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-6: Server lk version = 1
[2017-03-17 13:28:04.481678] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-30: Connected to vmware2-client-30, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:28:04.481694] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-30: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.481783] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-18: Connected to vmware2-client-18, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:28:04.481814] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-18: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.481893] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-2: Server lk version = 1
[2017-03-17 13:28:04.481933] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-10: Server lk version = 1
[2017-03-17 13:28:04.481950] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-14: Server lk version = 1
[2017-03-17 13:28:04.481969] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-22: Server lk version = 1
[2017-03-17 13:28:04.481991] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-34: Server lk version = 1
[2017-03-17 13:28:04.482002] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-26: Server lk version = 1
[2017-03-17 13:28:04.482031] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-30: Server lk version = 1
[2017-03-17 13:28:04.482097] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-18: Server lk version = 1
[2017-03-17 13:28:04.482508] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 15-vmware2-client-38: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:28:04.483126] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 15-vmware2-client-38: Connected to vmware2-client-38, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:28:04.483163] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 15-vmware2-client-38: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:28:04.502488] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 15-vmware2-client-38: Server lk version = 1
[2017-03-17 13:28:04.504059] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:28:04.506638] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:28:04.506862] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 13:28:04.507056] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:28:04.507369] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 13:28:04.507595] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:28:04.507680] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:28:04.507791] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:28:04.507891] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:28:04.507921] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:28:04.513435] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-0: remote operation failed. Path: <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/vmware-1.log (730513c7-b1c3-4ced-a52a-5ddee3f199ea) [No such file or directory]
[2017-03-17 13:28:04.513493] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-1: remote operation failed. Path: <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/vmware-1.log (730513c7-b1c3-4ced-a52a-5ddee3f199ea) [No such file or directory]
[2017-03-17 13:28:04.514977] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-20: no subvolumes up
[2017-03-17 13:28:04.515044] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-21: no subvolumes up
[2017-03-17 13:28:04.516980] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-1: remote operation failed. Path: <gfid:730513c7-b1c3-4ced-a52a-5ddee3f199ea> (730513c7-b1c3-4ced-a52a-5ddee3f199ea) [No such file or directory]
[2017-03-17 13:28:04.517047] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-0: remote operation failed. Path: <gfid:730513c7-b1c3-4ced-a52a-5ddee3f199ea> (730513c7-b1c3-4ced-a52a-5ddee3f199ea) [No such file or directory]
[2017-03-17 13:28:04.520422] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-16: remote operation failed. Path: /New Virtual Machine/vmware-1.log (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:28:04.520528] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-17: remote operation failed. Path: /New Virtual Machine/vmware-1.log (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:28:04.520995] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-20: no subvolumes up
[2017-03-17 13:28:04.521047] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-21: no subvolumes up
[2017-03-17 13:28:04.521529] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-38: remote operation failed. Path: /New Virtual Machine/vmware-1.log (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:28:04.521544] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-39: remote operation failed. Path: /New Virtual Machine/vmware-1.log (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:28:04.523547] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-16: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:28:04.523555] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-17: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:28:04.524497] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-38: remote operation failed. Path: <gfid:178a2423-15aa-4819-900d-b3b641c40e70> (178a2423-15aa-4819-900d-b3b641c40e70) [No such file or directory]
[2017-03-17 13:28:04.532736] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-34: remote operation failed. Path: <gfid:efef7c02-2411-4b92-890e-c07b143aebb2> (efef7c02-2411-4b92-890e-c07b143aebb2) [No such file or directory]
[2017-03-17 13:28:04.532799] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-35: remote operation failed. Path: <gfid:efef7c02-2411-4b92-890e-c07b143aebb2> (efef7c02-2411-4b92-890e-c07b143aebb2) [No such file or directory]
[2017-03-17 13:28:04.535245] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-25: remote operation failed. Path: <gfid:058bcdef-7095-476b-9f7b-8c4c8cbcef82> (058bcdef-7095-476b-9f7b-8c4c8cbcef82) [No such file or directory]
[2017-03-17 13:28:04.535296] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-24: remote operation failed. Path: <gfid:058bcdef-7095-476b-9f7b-8c4c8cbcef82> (058bcdef-7095-476b-9f7b-8c4c8cbcef82) [No such file or directory]
[2017-03-17 13:28:04.542213] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-30: remote operation failed. Path: <gfid:330f216d-6833-43ee-8074-74ca3a6e37bf> (330f216d-6833-43ee-8074-74ca3a6e37bf) [No such file or directory]
[2017-03-17 13:28:04.542472] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-31: remote operation failed. Path: <gfid:330f216d-6833-43ee-8074-74ca3a6e37bf> (330f216d-6833-43ee-8074-74ca3a6e37bf) [No such file or directory]
[2017-03-17 13:28:04.545094] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-18: remote operation failed. Path: /New Virtual Machine/New Virtual Machine.vmx (f9f2d22b-c214-43ae-9de3-240bd2697112) [No such file or directory]
[2017-03-17 13:28:04.545210] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-19: remote operation failed. Path: /New Virtual Machine/New Virtual Machine.vmx (f9f2d22b-c214-43ae-9de3-240bd2697112) [No such file or directory]
[2017-03-17 13:28:04.545685] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-20: no subvolumes up
[2017-03-17 13:28:04.545749] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-21: no subvolumes up
[2017-03-17 13:28:04.548558] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-18: remote operation failed. Path: <gfid:f9f2d22b-c214-43ae-9de3-240bd2697112> (f9f2d22b-c214-43ae-9de3-240bd2697112) [No such file or directory]
[2017-03-17 13:28:04.548736] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-19: remote operation failed. Path: <gfid:f9f2d22b-c214-43ae-9de3-240bd2697112> (f9f2d22b-c214-43ae-9de3-240bd2697112) [No such file or directory]
[2017-03-17 13:28:04.553394] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-20: no subvolumes up
[2017-03-17 13:28:04.553438] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-21: no subvolumes up
[2017-03-17 13:28:04.553439] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-28: remote operation failed. Path: /New Virtual Machine/New Virtual Machine.vmdk (e27f3c56-b050-46f3-a51a-a3c28a830471) [No such file or directory]
[2017-03-17 13:28:04.553514] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-29: remote operation failed. Path: /New Virtual Machine/New Virtual Machine.vmdk (e27f3c56-b050-46f3-a51a-a3c28a830471) [No such file or directory]
[2017-03-17 13:28:04.556843] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:e27f3c56-b050-46f3-a51a-a3c28a830471> (e27f3c56-b050-46f3-a51a-a3c28a830471) [No such file or directory]
[2017-03-17 13:28:04.556939] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:e27f3c56-b050-46f3-a51a-a3c28a830471> (e27f3c56-b050-46f3-a51a-a3c28a830471) [No such file or directory]
[2017-03-17 13:28:04.562446] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-32: remote operation failed. Path: <gfid:457c46d3-2e88-419e-afac-09392ee40920> (457c46d3-2e88-419e-afac-09392ee40920) [No such file or directory]
[2017-03-17 13:28:04.562493] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-33: remote operation failed. Path: <gfid:457c46d3-2e88-419e-afac-09392ee40920> (457c46d3-2e88-419e-afac-09392ee40920) [No such file or directory]
[2017-03-17 13:28:04.566962] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-19: remote operation failed. Path: <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/vmware.log (e212debf-87a8-4248-89d3-384e3843c125) [No such file or directory]
[2017-03-17 13:28:04.566971] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-18: remote operation failed. Path: <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/vmware.log (e212debf-87a8-4248-89d3-384e3843c125) [No such file or directory]
[2017-03-17 13:28:04.567192] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-20: no subvolumes up
[2017-03-17 13:28:04.567225] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-21: no subvolumes up
[2017-03-17 13:28:04.569485] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-18: remote operation failed. Path: <gfid:e212debf-87a8-4248-89d3-384e3843c125> (e212debf-87a8-4248-89d3-384e3843c125) [No such file or directory]
[2017-03-17 13:28:04.569593] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-19: remote operation failed. Path: <gfid:e212debf-87a8-4248-89d3-384e3843c125> (e212debf-87a8-4248-89d3-384e3843c125) [No such file or directory]
[2017-03-17 13:28:04.577475] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-14: remote operation failed. Path: <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx (f681b092-1e7c-437d-a74b-18a13ca0d096) [No such file or directory]
[2017-03-17 13:28:04.577580] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 12-vmware2-client-15: remote operation failed. Path: <gfid:f330cb82-e6f6-49d7-98b4-a3325cb25cf4>/gfstest.vmx (f681b092-1e7c-437d-a74b-18a13ca0d096) [No such file or directory]
[2017-03-17 13:28:04.578256] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-20: no subvolumes up
[2017-03-17 13:28:04.578317] I [MSGID: 108006] [afr-common.c:4518:afr_local_init] 12-vmware2-replicate-21: no subvolumes up
[2017-03-17 13:28:04.580626] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-15: remote operation failed. Path: <gfid:f681b092-1e7c-437d-a74b-18a13ca0d096> (f681b092-1e7c-437d-a74b-18a13ca0d096) [No such file or directory]
[2017-03-17 13:28:04.580688] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-14: remote operation failed. Path: <gfid:f681b092-1e7c-437d-a74b-18a13ca0d096> (f681b092-1e7c-437d-a74b-18a13ca0d096) [No such file or directory]
[2017-03-17 13:28:04.587908] I [MSGID: 104041] [glfs-resolve.c:885:__glfs_active_subvol] 0-vmware2: switched to graph 676c7573-7465-7230-312d-313938392d32 (15)
[2017-03-17 13:28:04.587943] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-0: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588001] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-1: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588033] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-2: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588026] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-0: disconnected from vmware2-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588079] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-3: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588092] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-2: disconnected from vmware2-client-2. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588078] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-1: disconnected from vmware2-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588113] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-4: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588150] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.588161] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-5: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588184] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-6: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588203] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-7: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588198] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-3: disconnected from vmware2-client-3. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588207] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-4: disconnected from vmware2-client-4. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588259] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-8: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588266] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-1: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.588297] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-9: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588320] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-10: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588327] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-5: disconnected from vmware2-client-5. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588339] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-11: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588370] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-12: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588365] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-2: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.588390] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-6: disconnected from vmware2-client-6. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588405] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-13: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588424] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-7: disconnected from vmware2-client-7. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588434] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-8: disconnected from vmware2-client-8. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588428] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-14: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588459] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-3: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.588479] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-15: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588501] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-16: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588504] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-9: disconnected from vmware2-client-9. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588521] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-4: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.588527] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-17: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588547] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-10: disconnected from vmware2-client-10. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588551] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-18: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588586] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-11: disconnected from vmware2-client-11. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588593] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-19: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588586] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-12: disconnected from vmware2-client-12. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588611] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-5: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.588627] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-20: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588658] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-21: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588659] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-13: disconnected from vmware2-client-13. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588682] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-14: disconnected from vmware2-client-14. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588691] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-6: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.588692] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-22: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588720] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-23: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588727] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-15: disconnected from vmware2-client-15. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588733] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-16: disconnected from vmware2-client-16. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588757] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-24: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588755] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-7: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.588804] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-25: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588821] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-18: disconnected from vmware2-client-18. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588822] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-17: disconnected from vmware2-client-17. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588829] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-26: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588890] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-27: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588886] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-8: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.588889] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-19: disconnected from vmware2-client-19. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588929] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-9: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.588930] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-28: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.588930] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-20: disconnected from vmware2-client-20. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588960] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-21: disconnected from vmware2-client-21. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588977] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-10: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.588977] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-22: disconnected from vmware2-client-22. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.588978] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-29: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589009] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-23: disconnected from vmware2-client-23. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589023] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-11: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.589018] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-30: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589018] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-24: disconnected from vmware2-client-24. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589051] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-31: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589052] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-25: disconnected from vmware2-client-25. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589076] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-32: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589077] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-26: disconnected from vmware2-client-26. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589076] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-12: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.589114] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-33: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589134] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-27: disconnected from vmware2-client-27. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589135] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-34: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589162] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-35: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589169] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-13: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.589188] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-36: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589220] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-29: disconnected from vmware2-client-29. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589222] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-28: disconnected from vmware2-client-28. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589262] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-14: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.589239] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-37: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589291] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-30: disconnected from vmware2-client-30. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589297] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-38: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589322] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-32: disconnected from vmware2-client-32. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589319] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-31: disconnected from vmware2-client-31. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589321] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-39: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589345] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-15: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.589351] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-33: disconnected from vmware2-client-33. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589367] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-16: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.589369] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-40: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589380] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-34: disconnected from vmware2-client-34. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589387] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-41: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589406] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-35: disconnected from vmware2-client-35. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589422] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-17: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.589425] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-42: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589432] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-36: disconnected from vmware2-client-36. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589441] I [MSGID: 114021] [client.c:2365:notify] 12-vmware2-client-43: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:28:04.589457] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-37: disconnected from vmware2-client-37. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589482] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-18: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:04.589487] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-38: disconnected from vmware2-client-38. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589523] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 12-vmware2-client-39: disconnected from vmware2-client-39. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:28:04.589537] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 12-vmware2-replicate-19: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:28:05.141679] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-8: remote operation failed
[2017-03-17 13:28:05.141739] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-9: remote operation failed
[2017-03-17 13:28:05.141774] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-8: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.141797] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-9: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.141878] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-4: remote operation failed
[2017-03-17 13:28:05.141898] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-5: remote operation failed
[2017-03-17 13:28:05.141913] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-4: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.141929] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-5: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.141979] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-18: remote operation failed
[2017-03-17 13:28:05.141997] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-19: remote operation failed
[2017-03-17 13:28:05.142011] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-18: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.142028] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-19: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.142075] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-18: remote operation failed
[2017-03-17 13:28:05.142092] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-19: remote operation failed
[2017-03-17 13:28:05.142106] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-18: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.142121] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-19: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.142168] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-22: remote operation failed
[2017-03-17 13:28:05.142186] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-23: remote operation failed
[2017-03-17 13:28:05.142201] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-22: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.142217] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-23: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.142265] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-20: remote operation failed
[2017-03-17 13:28:05.142296] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-21: remote operation failed
[2017-03-17 13:28:05.142311] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-20: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.142327] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-21: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.142376] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-34: remote operation failed
[2017-03-17 13:28:05.142393] W [MSGID: 114031] [client-rpc-fops.c:1825:client3_3_fxattrop_cbk] 12-vmware2-client-35: remote operation failed
[2017-03-17 13:28:05.142407] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-34: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:05.142423] E [MSGID: 114031] [client-rpc-fops.c:1596:client3_3_finodelk_cbk] 12-vmware2-client-35: remote operation failed [Transport endpoint is not connected]
[2017-03-17 13:28:42.851433] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 74d70147-7e7a-41c8-90c7-bcdf3d29757a. (Possible split-brain)
[2017-03-17 13:28:42.854292] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:74d70147-7e7a-41c8-90c7-bcdf3d29757a>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:42.854355] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 74d70147-7e7a-41c8-90c7-bcdf3d29757a [Stale file handle]
[2017-03-17 13:28:42.920760] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 56b12f5e-c604-4909-af9f-2a625b891782. (Possible split-brain)
[2017-03-17 13:28:42.923863] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:56b12f5e-c604-4909-af9f-2a625b891782>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:42.923939] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 56b12f5e-c604-4909-af9f-2a625b891782 [Stale file handle]
[2017-03-17 13:28:42.940589] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 9cd0144a-1cf9-4764-9064-8210cca3ed1d. (Possible split-brain)
[2017-03-17 13:28:42.943348] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:9cd0144a-1cf9-4764-9064-8210cca3ed1d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:42.943407] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 9cd0144a-1cf9-4764-9064-8210cca3ed1d [Stale file handle]
[2017-03-17 13:28:42.947267] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 43b5a897-4d73-4c8c-8cd7-9c0da3c0c94e. (Possible split-brain)
[2017-03-17 13:28:42.949977] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:43b5a897-4d73-4c8c-8cd7-9c0da3c0c94e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:42.950048] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 43b5a897-4d73-4c8c-8cd7-9c0da3c0c94e [Stale file handle]
[2017-03-17 13:28:43.070468] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /gfstest/gfstest.vmx~ (hash=vmware2-replicate-7/cache=vmware2-replicate-7) => /gfstest/gfstest.vmx (hash=vmware2-replicate-17/cache=vmware2-replicate-17)
[2017-03-17 13:28:43.078835] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 13:28:43.079309] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-34: remote operation failed. Path: <gfid:f681b092-1e7c-437d-a74b-18a13ca0d096> (f681b092-1e7c-437d-a74b-18a13ca0d096) [No such file or directory]
[2017-03-17 13:28:43.079381] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-35: remote operation failed. Path: <gfid:f681b092-1e7c-437d-a74b-18a13ca0d096> (f681b092-1e7c-437d-a74b-18a13ca0d096) [No such file or directory]
[2017-03-17 13:28:43.080013] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid f681b092-1e7c-437d-a74b-18a13ca0d096. (Possible split-brain)
[2017-03-17 13:28:43.080498] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-35: remote operation failed. Path: <gfid:f681b092-1e7c-437d-a74b-18a13ca0d096> (f681b092-1e7c-437d-a74b-18a13ca0d096) [No such file or directory]
[2017-03-17 13:28:43.080512] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-34: remote operation failed. Path: <gfid:f681b092-1e7c-437d-a74b-18a13ca0d096> (f681b092-1e7c-437d-a74b-18a13ca0d096) [No such file or directory]
[2017-03-17 13:28:43.082204] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-34: remote operation failed. Path: <gfid:f681b092-1e7c-437d-a74b-18a13ca0d096> (f681b092-1e7c-437d-a74b-18a13ca0d096) [No such file or directory]
[2017-03-17 13:28:43.082223] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-35: remote operation failed. Path: <gfid:f681b092-1e7c-437d-a74b-18a13ca0d096> (f681b092-1e7c-437d-a74b-18a13ca0d096) [No such file or directory]
[2017-03-17 13:28:43.082394] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:f681b092-1e7c-437d-a74b-18a13ca0d096>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:43.082440] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: f681b092-1e7c-437d-a74b-18a13ca0d096 [Stale file handle]
[2017-03-17 13:28:43.088533] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid 48f49ab6-9729-4b09-bae0-eca69b1610d3. (Possible split-brain)
[2017-03-17 13:28:43.091561] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:48f49ab6-9729-4b09-bae0-eca69b1610d3>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:43.091624] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 48f49ab6-9729-4b09-bae0-eca69b1610d3 [Stale file handle]
[2017-03-17 13:28:43.108452] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 22fd385d-9269-4ef5-9109-328504c2b1ee. (Possible split-brain)
[2017-03-17 13:28:43.110948] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:22fd385d-9269-4ef5-9109-328504c2b1ee>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:43.110996] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 22fd385d-9269-4ef5-9109-328504c2b1ee [Stale file handle]
[2017-03-17 13:28:43.114874] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid 1a7f0fa2-e89b-411c-8445-bc8e0246b2a3. (Possible split-brain)
[2017-03-17 13:28:43.116882] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:1a7f0fa2-e89b-411c-8445-bc8e0246b2a3>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:43.116928] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 1a7f0fa2-e89b-411c-8445-bc8e0246b2a3 [Stale file handle]
[2017-03-17 13:28:43.143717] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 457c46d3-2e88-419e-afac-09392ee40920. (Possible split-brain)
[2017-03-17 13:28:43.146744] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:457c46d3-2e88-419e-afac-09392ee40920>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:43.146814] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 457c46d3-2e88-419e-afac-09392ee40920 [Stale file handle]
[2017-03-17 13:28:43.161013] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9dbe13de-12f5-4825-a5ad-1c118c767952. (Possible split-brain)
[2017-03-17 13:28:43.163797] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:9dbe13de-12f5-4825-a5ad-1c118c767952>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:43.163861] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 9dbe13de-12f5-4825-a5ad-1c118c767952 [Stale file handle]
[2017-03-17 13:28:43.167674] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid f16fc696-7f57-4c8b-9ba0-29f3813c4ef8. (Possible split-brain)
[2017-03-17 13:28:43.170440] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:f16fc696-7f57-4c8b-9ba0-29f3813c4ef8>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:43.170500] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: f16fc696-7f57-4c8b-9ba0-29f3813c4ef8 [Stale file handle]
[2017-03-17 13:28:43.223843] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:28:43.224095] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:28:43.224835] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:28:43.225313] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:28:48.526812] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-11: Unreadable subvolume -1 found with event generation 2 for gfid 5159ca0a-0ebf-4d2f-8a03-efdf8f71e37d. (Possible split-brain)
[2017-03-17 13:28:48.528734] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:5159ca0a-0ebf-4d2f-8a03-efdf8f71e37d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.528806] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 5159ca0a-0ebf-4d2f-8a03-efdf8f71e37d [Stale file handle]
[2017-03-17 13:28:48.532082] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 3f30ac58-ed00-44c3-acdb-e748ddb44374. (Possible split-brain)
[2017-03-17 13:28:48.533601] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:3f30ac58-ed00-44c3-acdb-e748ddb44374>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.533646] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 3f30ac58-ed00-44c3-acdb-e748ddb44374 [Stale file handle]
[2017-03-17 13:28:48.545726] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 739913df-e0dc-4bad-82ed-af70a9b90640. (Possible split-brain)
[2017-03-17 13:28:48.547615] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:739913df-e0dc-4bad-82ed-af70a9b90640>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.547679] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 739913df-e0dc-4bad-82ed-af70a9b90640 [Stale file handle]
[2017-03-17 13:28:48.550491] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 16c4dbd7-9777-446d-8edd-49871b3fb773. (Possible split-brain)
[2017-03-17 13:28:48.552619] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:16c4dbd7-9777-446d-8edd-49871b3fb773>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.552656] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 16c4dbd7-9777-446d-8edd-49871b3fb773 [Stale file handle]
[2017-03-17 13:28:48.600377] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:28:48.600382] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:28:48.600748] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 13:28:48.600974] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:28:48.601005] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 13:28:48.601237] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:28:48.601335] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:28:48.601399] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:28:48.601480] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:28:48.657311] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 87562bfe-2b35-4028-8ffd-49ca43ac28ed. (Possible split-brain)
[2017-03-17 13:28:48.659442] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:87562bfe-2b35-4028-8ffd-49ca43ac28ed>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.659488] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 87562bfe-2b35-4028-8ffd-49ca43ac28ed [Stale file handle]
[2017-03-17 13:28:48.663183] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid f2a139d3-3e0c-448c-947d-8bdd50fd8bf6. (Possible split-brain)
[2017-03-17 13:28:48.667080] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:f2a139d3-3e0c-448c-947d-8bdd50fd8bf6>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.667121] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: f2a139d3-3e0c-448c-947d-8bdd50fd8bf6 [Stale file handle]
[2017-03-17 13:28:48.680913] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 730513c7-b1c3-4ced-a52a-5ddee3f199ea. (Possible split-brain)
[2017-03-17 13:28:48.683575] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:730513c7-b1c3-4ced-a52a-5ddee3f199ea>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.683612] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 730513c7-b1c3-4ced-a52a-5ddee3f199ea [Stale file handle]
[2017-03-17 13:28:48.687140] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 33810ad8-1b8c-4e97-a2cb-76c57dcf72bd. (Possible split-brain)
[2017-03-17 13:28:48.689849] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:33810ad8-1b8c-4e97-a2cb-76c57dcf72bd>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.689903] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 33810ad8-1b8c-4e97-a2cb-76c57dcf72bd [Stale file handle]
[2017-03-17 13:28:48.702561] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid e212debf-87a8-4248-89d3-384e3843c125. (Possible split-brain)
[2017-03-17 13:28:48.704797] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:e212debf-87a8-4248-89d3-384e3843c125>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.704869] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: e212debf-87a8-4248-89d3-384e3843c125 [Stale file handle]
[2017-03-17 13:28:48.708065] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid a1140f2c-600b-4cdb-9807-04d34d4dbe97. (Possible split-brain)
[2017-03-17 13:28:48.709998] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:a1140f2c-600b-4cdb-9807-04d34d4dbe97>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.710032] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: a1140f2c-600b-4cdb-9807-04d34d4dbe97 [Stale file handle]
[2017-03-17 13:28:48.738501] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid 3b898f16-51d4-40f6-8abd-f99545038fa4. (Possible split-brain)
[2017-03-17 13:28:48.740835] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:3b898f16-51d4-40f6-8abd-f99545038fa4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.740909] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 3b898f16-51d4-40f6-8abd-f99545038fa4 [Stale file handle]
[2017-03-17 13:28:48.744260] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 628046d4-2370-4653-b5d8-9b60e28cdccf. (Possible split-brain)
[2017-03-17 13:28:48.746510] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:628046d4-2370-4653-b5d8-9b60e28cdccf>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.746567] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 628046d4-2370-4653-b5d8-9b60e28cdccf [Stale file handle]
[2017-03-17 13:28:48.766607] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 2ae855ca-0f21-477b-9d6c-a25d15460ff8. (Possible split-brain)
[2017-03-17 13:28:48.768963] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:2ae855ca-0f21-477b-9d6c-a25d15460ff8>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.769013] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 2ae855ca-0f21-477b-9d6c-a25d15460ff8 [Stale file handle]
[2017-03-17 13:28:48.772569] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid bd963c95-d455-46af-8a90-f08b9b87b57d. (Possible split-brain)
[2017-03-17 13:28:48.775313] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:bd963c95-d455-46af-8a90-f08b9b87b57d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:28:48.775356] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: bd963c95-d455-46af-8a90-f08b9b87b57d [Stale file handle]
[2017-03-17 13:28:49.816970] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:28:49.817235] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:28:49.817491] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 13:28:49.817776] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:28:49.817813] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 13:28:49.818210] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:28:49.818235] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:28:49.818343] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:28:49.818503] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:28:49.818566] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:29:21.892937] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /Testse/Testse.vmdk~ (hash=vmware2-replicate-9/cache=vmware2-replicate-9) => /Testse/Testse.vmdk (hash=vmware2-replicate-1/cache=vmware2-replicate-1)
[2017-03-17 13:29:21.903180] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid c45cad0b-551a-4ef7-9ae6-deb157accb85. (Possible split-brain)
[2017-03-17 13:29:21.905438] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:c45cad0b-551a-4ef7-9ae6-deb157accb85>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:29:21.905488] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: c45cad0b-551a-4ef7-9ae6-deb157accb85 [Stale file handle]
[2017-03-17 13:29:21.909618] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid fa77a46e-0926-4501-833e-ff731bf8dd3a. (Possible split-brain)
[2017-03-17 13:29:21.912078] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:fa77a46e-0926-4501-833e-ff731bf8dd3a>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:29:21.912124] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: fa77a46e-0926-4501-833e-ff731bf8dd3a [Stale file handle]
[2017-03-17 13:29:21.925432] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:29:21.966811] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /Testse/Testse.vmdk~ (hash=vmware2-replicate-9/cache=vmware2-replicate-9) => /Testse/Testse.vmdk (hash=vmware2-replicate-1/cache=vmware2-replicate-9)
[2017-03-17 13:29:21.973348] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-18: remote operation failed [No such file or directory]
[2017-03-17 13:29:21.973736] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-18: remote operation failed. Path: <gfid:3170e1d4-af78-4faf-a1a7-2a71de1f59c7> (3170e1d4-af78-4faf-a1a7-2a71de1f59c7) [No such file or directory]
[2017-03-17 13:29:21.973876] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-19: remote operation failed. Path: <gfid:3170e1d4-af78-4faf-a1a7-2a71de1f59c7> (3170e1d4-af78-4faf-a1a7-2a71de1f59c7) [No such file or directory]
[2017-03-17 13:29:21.974261] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid 3170e1d4-af78-4faf-a1a7-2a71de1f59c7. (Possible split-brain)
[2017-03-17 13:29:21.974658] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-18: remote operation failed. Path: <gfid:3170e1d4-af78-4faf-a1a7-2a71de1f59c7> (3170e1d4-af78-4faf-a1a7-2a71de1f59c7) [No such file or directory]
[2017-03-17 13:29:21.974786] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-19: remote operation failed. Path: <gfid:3170e1d4-af78-4faf-a1a7-2a71de1f59c7> (3170e1d4-af78-4faf-a1a7-2a71de1f59c7) [No such file or directory]
[2017-03-17 13:29:21.976065] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-18: remote operation failed. Path: <gfid:3170e1d4-af78-4faf-a1a7-2a71de1f59c7> (3170e1d4-af78-4faf-a1a7-2a71de1f59c7) [No such file or directory]
[2017-03-17 13:29:21.976286] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-19: remote operation failed. Path: <gfid:3170e1d4-af78-4faf-a1a7-2a71de1f59c7> (3170e1d4-af78-4faf-a1a7-2a71de1f59c7) [No such file or directory]
[2017-03-17 13:29:21.977275] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:3170e1d4-af78-4faf-a1a7-2a71de1f59c7>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:29:21.977336] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 3170e1d4-af78-4faf-a1a7-2a71de1f59c7 [Stale file handle]
[2017-03-17 13:29:21.981199] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid 86cea39c-e8f7-49af-bdeb-9c1ac0331557. (Possible split-brain)
[2017-03-17 13:29:21.983876] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:86cea39c-e8f7-49af-bdeb-9c1ac0331557>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:29:21.983918] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 86cea39c-e8f7-49af-bdeb-9c1ac0331557 [Stale file handle]
[2017-03-17 13:29:22.014250] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /Testse/Testse.vmdk~ (hash=vmware2-replicate-9/cache=vmware2-replicate-9) => /Testse/Testse.vmdk (hash=vmware2-replicate-1/cache=vmware2-replicate-9)
[2017-03-17 13:29:22.020898] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-19: remote operation failed [No such file or directory]
[2017-03-17 13:29:22.021416] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-19: remote operation failed. Path: <gfid:91291651-c9c4-4601-8239-e296fa678e48> (91291651-c9c4-4601-8239-e296fa678e48) [No such file or directory]
[2017-03-17 13:29:22.021428] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-18: remote operation failed. Path: <gfid:91291651-c9c4-4601-8239-e296fa678e48> (91291651-c9c4-4601-8239-e296fa678e48) [No such file or directory]
[2017-03-17 13:29:22.021887] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid 91291651-c9c4-4601-8239-e296fa678e48. (Possible split-brain)
[2017-03-17 13:29:22.022420] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-18: remote operation failed. Path: <gfid:91291651-c9c4-4601-8239-e296fa678e48> (91291651-c9c4-4601-8239-e296fa678e48) [No such file or directory]
[2017-03-17 13:29:22.022428] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-19: remote operation failed. Path: <gfid:91291651-c9c4-4601-8239-e296fa678e48> (91291651-c9c4-4601-8239-e296fa678e48) [No such file or directory]
[2017-03-17 13:29:22.023586] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-18: remote operation failed. Path: <gfid:91291651-c9c4-4601-8239-e296fa678e48> (91291651-c9c4-4601-8239-e296fa678e48) [No such file or directory]
[2017-03-17 13:29:22.023699] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-19: remote operation failed. Path: <gfid:91291651-c9c4-4601-8239-e296fa678e48> (91291651-c9c4-4601-8239-e296fa678e48) [No such file or directory]
[2017-03-17 13:29:22.024628] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:91291651-c9c4-4601-8239-e296fa678e48>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:29:22.024679] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 91291651-c9c4-4601-8239-e296fa678e48 [Stale file handle]
[2017-03-17 13:29:22.028405] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3a2a47c9-9516-47d3-b242-de63431bf1e1. (Possible split-brain)
[2017-03-17 13:29:22.030972] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:3a2a47c9-9516-47d3-b242-de63431bf1e1>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:29:22.031016] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 3a2a47c9-9516-47d3-b242-de63431bf1e1 [Stale file handle]
[2017-03-17 13:29:22.053089] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:31:50.402975] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /Testse/Testse.vmdk~ (hash=vmware2-replicate-9/cache=vmware2-replicate-9) => /Testse/Testse.vmdk (hash=vmware2-replicate-1/cache=vmware2-replicate-9)
[2017-03-17 13:31:50.406666] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-19: remote operation failed [No such file or directory]
[2017-03-17 13:31:50.406973] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-19: remote operation failed. Path: <gfid:8d9b5b11-c1b9-4657-b2d8-6a1afc273638> (8d9b5b11-c1b9-4657-b2d8-6a1afc273638) [No such file or directory]
[2017-03-17 13:31:50.407051] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-18: remote operation failed. Path: <gfid:8d9b5b11-c1b9-4657-b2d8-6a1afc273638> (8d9b5b11-c1b9-4657-b2d8-6a1afc273638) [No such file or directory]
[2017-03-17 13:31:50.407599] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid 8d9b5b11-c1b9-4657-b2d8-6a1afc273638. (Possible split-brain)
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-18: remote operation failed. Path: <gfid:8d9b5b11-c1b9-4657-b2d8-6a1afc273638> (8d9b5b11-c1b9-4657-b2d8-6a1afc273638) [No such file or directory]" repeated 2 times between [2017-03-17 13:31:50.407051] and [2017-03-17 13:31:50.408950]
[2017-03-17 13:31:50.409670] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:8d9b5b11-c1b9-4657-b2d8-6a1afc273638>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:31:50.409723] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 8d9b5b11-c1b9-4657-b2d8-6a1afc273638 [Stale file handle]
[2017-03-17 13:31:50.412557] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid 19f85cd8-87e2-4c18-abff-3ea17f628eda. (Possible split-brain)
[2017-03-17 13:31:50.414123] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:19f85cd8-87e2-4c18-abff-3ea17f628eda>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:31:50.414152] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 19f85cd8-87e2-4c18-abff-3ea17f628eda [Stale file handle]
[2017-03-17 13:31:50.416252] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 9c231496-cf42-49d2-a287-7bae5eac8da1. (Possible split-brain)
[2017-03-17 13:31:50.418212] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:9c231496-cf42-49d2-a287-7bae5eac8da1>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:31:50.418238] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 9c231496-cf42-49d2-a287-7bae5eac8da1 [Stale file handle]
[2017-03-17 13:31:50.594838] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-15/cache=vmware2-replicate-15) => /Testse/Testse.vmx (hash=vmware2-replicate-18/cache=vmware2-replicate-18)
[2017-03-17 13:31:50.602171] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid 01967034-a2fb-4220-a4c5-45825b9ddddb. (Possible split-brain)
[2017-03-17 13:31:50.604007] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:01967034-a2fb-4220-a4c5-45825b9ddddb>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:31:50.604035] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 01967034-a2fb-4220-a4c5-45825b9ddddb [Stale file handle]
[2017-03-17 13:31:50.606742] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 37df1398-6312-4940-88f7-15f295393c99. (Possible split-brain)
[2017-03-17 13:31:50.609044] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:37df1398-6312-4940-88f7-15f295393c99>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:31:50.609070] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 37df1398-6312-4940-88f7-15f295393c99 [Stale file handle]
[2017-03-17 13:31:50.661349] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:31:50.662095] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:31:50.662259] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:31:50.718899] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 1c1b86fd-b546-458a-9c56-3f7431106fcb. (Possible split-brain)
[2017-03-17 13:31:50.720998] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:1c1b86fd-b546-458a-9c56-3f7431106fcb>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:31:50.721048] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 1c1b86fd-b546-458a-9c56-3f7431106fcb [Stale file handle]
[2017-03-17 13:31:50.760576] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-15/cache=vmware2-replicate-15) => /Testse/Testse.vmx (hash=vmware2-replicate-18/cache=vmware2-replicate-15)
[2017-03-17 13:31:50.766549] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-31: remote operation failed [No such file or directory]
[2017-03-17 13:31:50.766960] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-31: remote operation failed. Path: <gfid:ad91cba2-3d00-47ee-b4f1-3fccfedc6edd> (ad91cba2-3d00-47ee-b4f1-3fccfedc6edd) [No such file or directory]
[2017-03-17 13:31:50.767012] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-30: remote operation failed. Path: <gfid:ad91cba2-3d00-47ee-b4f1-3fccfedc6edd> (ad91cba2-3d00-47ee-b4f1-3fccfedc6edd) [No such file or directory]
[2017-03-17 13:31:50.767372] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid ad91cba2-3d00-47ee-b4f1-3fccfedc6edd. (Possible split-brain)
[2017-03-17 13:31:50.767876] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-30: remote operation failed. Path: <gfid:ad91cba2-3d00-47ee-b4f1-3fccfedc6edd> (ad91cba2-3d00-47ee-b4f1-3fccfedc6edd) [No such file or directory]
[2017-03-17 13:31:50.767896] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-31: remote operation failed. Path: <gfid:ad91cba2-3d00-47ee-b4f1-3fccfedc6edd> (ad91cba2-3d00-47ee-b4f1-3fccfedc6edd) [No such file or directory]
[2017-03-17 13:31:50.769350] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-30: remote operation failed. Path: <gfid:ad91cba2-3d00-47ee-b4f1-3fccfedc6edd> (ad91cba2-3d00-47ee-b4f1-3fccfedc6edd) [No such file or directory]
[2017-03-17 13:31:50.769506] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-31: remote operation failed. Path: <gfid:ad91cba2-3d00-47ee-b4f1-3fccfedc6edd> (ad91cba2-3d00-47ee-b4f1-3fccfedc6edd) [No such file or directory]
[2017-03-17 13:31:50.769760] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:ad91cba2-3d00-47ee-b4f1-3fccfedc6edd>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:31:50.769806] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: ad91cba2-3d00-47ee-b4f1-3fccfedc6edd [Stale file handle]
[2017-03-17 13:31:50.772927] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid 81cf97ff-a677-49e1-a5e0-5d9e83cc6bdd. (Possible split-brain)
[2017-03-17 13:31:50.774817] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:81cf97ff-a677-49e1-a5e0-5d9e83cc6bdd>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:31:50.774850] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 81cf97ff-a677-49e1-a5e0-5d9e83cc6bdd [Stale file handle]
[2017-03-17 13:31:50.787294] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid 95cd2e34-53c4-44bb-b6f3-284c5ab77692. (Possible split-brain)
[2017-03-17 13:31:50.789467] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:95cd2e34-53c4-44bb-b6f3-284c5ab77692>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:31:50.789500] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 95cd2e34-53c4-44bb-b6f3-284c5ab77692 [Stale file handle]
[2017-03-17 13:31:50.792730] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid d1739d96-7d9b-4853-957c-822bd39a671b. (Possible split-brain)
[2017-03-17 13:31:50.794593] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:d1739d96-7d9b-4853-957c-822bd39a671b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:31:50.794627] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: d1739d96-7d9b-4853-957c-822bd39a671b [Stale file handle]
[2017-03-17 13:31:50.835341] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:31:50.835399] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:31:50.835582] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-18: selecting local read_child vmware2-client-36
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-19: remote operation failed. Path: <gfid:8d9b5b11-c1b9-4657-b2d8-6a1afc273638> (8d9b5b11-c1b9-4657-b2d8-6a1afc273638) [No such file or directory]" repeated 2 times between [2017-03-17 13:31:50.406973] and [2017-03-17 13:31:50.408944]
[2017-03-17 13:32:25.672034] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmdk~ (hash=vmware2-replicate-10/cache=vmware2-replicate-10) => /New Virtual Machine 5/New Virtual Machine 5.vmdk (hash=vmware2-replicate-12/cache=vmware2-replicate-12)
[2017-03-17 13:32:25.680606] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-12: Unreadable subvolume -1 found with event generation 2 for gfid 7051ce16-8e46-430b-a4cf-a62c60bbb525. (Possible split-brain)
[2017-03-17 13:32:25.683733] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:7051ce16-8e46-430b-a4cf-a62c60bbb525>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:32:25.683809] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 7051ce16-8e46-430b-a4cf-a62c60bbb525 [Stale file handle]
[2017-03-17 13:32:25.688050] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 94ba3d9c-d750-4ea7-984e-bef30af377bb. (Possible split-brain)
[2017-03-17 13:32:25.690616] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:94ba3d9c-d750-4ea7-984e-bef30af377bb>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:32:25.690667] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 94ba3d9c-d750-4ea7-984e-bef30af377bb [Stale file handle]
[2017-03-17 13:32:25.704324] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:32:25.742906] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmdk~ (hash=vmware2-replicate-10/cache=vmware2-replicate-10) => /New Virtual Machine 5/New Virtual Machine 5.vmdk (hash=vmware2-replicate-12/cache=vmware2-replicate-10)
[2017-03-17 13:32:25.749919] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-20: remote operation failed [No such file or directory]
[2017-03-17 13:32:25.750425] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-21: remote operation failed. Path: <gfid:9b80a177-7589-4992-b19d-6a40511bdd5e> (9b80a177-7589-4992-b19d-6a40511bdd5e) [No such file or directory]
[2017-03-17 13:32:25.750427] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-20: remote operation failed. Path: <gfid:9b80a177-7589-4992-b19d-6a40511bdd5e> (9b80a177-7589-4992-b19d-6a40511bdd5e) [No such file or directory]
[2017-03-17 13:32:25.750980] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 9b80a177-7589-4992-b19d-6a40511bdd5e. (Possible split-brain)
[2017-03-17 13:32:25.751479] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-20: remote operation failed. Path: <gfid:9b80a177-7589-4992-b19d-6a40511bdd5e> (9b80a177-7589-4992-b19d-6a40511bdd5e) [No such file or directory]
[2017-03-17 13:32:25.751500] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-21: remote operation failed. Path: <gfid:9b80a177-7589-4992-b19d-6a40511bdd5e> (9b80a177-7589-4992-b19d-6a40511bdd5e) [No such file or directory]
[2017-03-17 13:32:25.752622] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-21: remote operation failed. Path: <gfid:9b80a177-7589-4992-b19d-6a40511bdd5e> (9b80a177-7589-4992-b19d-6a40511bdd5e) [No such file or directory]
[2017-03-17 13:32:25.752637] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-20: remote operation failed. Path: <gfid:9b80a177-7589-4992-b19d-6a40511bdd5e> (9b80a177-7589-4992-b19d-6a40511bdd5e) [No such file or directory]
[2017-03-17 13:32:25.753219] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:9b80a177-7589-4992-b19d-6a40511bdd5e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:32:25.753264] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 9b80a177-7589-4992-b19d-6a40511bdd5e [Stale file handle]
[2017-03-17 13:32:25.757111] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid cfda7b5b-b75f-452d-b300-5ab2138c17ca. (Possible split-brain)
[2017-03-17 13:32:25.759935] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:cfda7b5b-b75f-452d-b300-5ab2138c17ca>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:32:25.760006] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: cfda7b5b-b75f-452d-b300-5ab2138c17ca [Stale file handle]
[2017-03-17 13:32:25.788497] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmdk~ (hash=vmware2-replicate-10/cache=vmware2-replicate-10) => /New Virtual Machine 5/New Virtual Machine 5.vmdk (hash=vmware2-replicate-12/cache=vmware2-replicate-10)
[2017-03-17 13:32:25.794699] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-20: remote operation failed [No such file or directory]
[2017-03-17 13:32:25.795162] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-20: remote operation failed. Path: <gfid:66c0058e-1fb0-4f73-9d20-b76bc823386e> (66c0058e-1fb0-4f73-9d20-b76bc823386e) [No such file or directory]
[2017-03-17 13:32:25.795243] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-21: remote operation failed. Path: <gfid:66c0058e-1fb0-4f73-9d20-b76bc823386e> (66c0058e-1fb0-4f73-9d20-b76bc823386e) [No such file or directory]
[2017-03-17 13:32:25.795525] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 66c0058e-1fb0-4f73-9d20-b76bc823386e. (Possible split-brain)
[2017-03-17 13:32:25.795958] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-20: remote operation failed. Path: <gfid:66c0058e-1fb0-4f73-9d20-b76bc823386e> (66c0058e-1fb0-4f73-9d20-b76bc823386e) [No such file or directory]
[2017-03-17 13:32:25.796043] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-21: remote operation failed. Path: <gfid:66c0058e-1fb0-4f73-9d20-b76bc823386e> (66c0058e-1fb0-4f73-9d20-b76bc823386e) [No such file or directory]
[2017-03-17 13:32:25.797228] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-20: remote operation failed. Path: <gfid:66c0058e-1fb0-4f73-9d20-b76bc823386e> (66c0058e-1fb0-4f73-9d20-b76bc823386e) [No such file or directory]
[2017-03-17 13:32:25.797321] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-21: remote operation failed. Path: <gfid:66c0058e-1fb0-4f73-9d20-b76bc823386e> (66c0058e-1fb0-4f73-9d20-b76bc823386e) [No such file or directory]
[2017-03-17 13:32:25.797905] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:66c0058e-1fb0-4f73-9d20-b76bc823386e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:32:25.797958] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 66c0058e-1fb0-4f73-9d20-b76bc823386e [Stale file handle]
[2017-03-17 13:32:25.801281] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid ea89a521-7763-4722-be9a-0ea3e0709ba6. (Possible split-brain)
[2017-03-17 13:32:25.803447] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:ea89a521-7763-4722-be9a-0ea3e0709ba6>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:32:25.803513] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: ea89a521-7763-4722-be9a-0ea3e0709ba6 [Stale file handle]
[2017-03-17 13:32:25.823781] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:32:25.824312] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:32:25.824355] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:34:47.540449] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmdk~ (hash=vmware2-replicate-10/cache=vmware2-replicate-10) => /New Virtual Machine 5/New Virtual Machine 5.vmdk (hash=vmware2-replicate-12/cache=vmware2-replicate-10)
[2017-03-17 13:34:47.544370] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-20: remote operation failed [No such file or directory]
[2017-03-17 13:34:47.544593] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-20: remote operation failed. Path: <gfid:40b0c3ad-8355-4e2b-b1d4-dfcbe17e18f5> (40b0c3ad-8355-4e2b-b1d4-dfcbe17e18f5) [No such file or directory]
[2017-03-17 13:34:47.544706] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-21: remote operation failed. Path: <gfid:40b0c3ad-8355-4e2b-b1d4-dfcbe17e18f5> (40b0c3ad-8355-4e2b-b1d4-dfcbe17e18f5) [No such file or directory]
[2017-03-17 13:34:47.545292] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 40b0c3ad-8355-4e2b-b1d4-dfcbe17e18f5. (Possible split-brain)
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-21: remote operation failed. Path: <gfid:40b0c3ad-8355-4e2b-b1d4-dfcbe17e18f5> (40b0c3ad-8355-4e2b-b1d4-dfcbe17e18f5) [No such file or directory]" repeated 2 times between [2017-03-17 13:34:47.544706] and [2017-03-17 13:34:47.546838]
[2017-03-17 13:34:47.547556] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:40b0c3ad-8355-4e2b-b1d4-dfcbe17e18f5>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:34:47.547620] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 40b0c3ad-8355-4e2b-b1d4-dfcbe17e18f5 [Stale file handle]
[2017-03-17 13:34:47.550398] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid 14d99b2b-bbcf-4b29-b815-65c30faedc2c. (Possible split-brain)
[2017-03-17 13:34:47.551794] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:14d99b2b-bbcf-4b29-b815-65c30faedc2c>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:34:47.551849] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 14d99b2b-bbcf-4b29-b815-65c30faedc2c [Stale file handle]
[2017-03-17 13:34:47.554096] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid d8687951-a556-4747-a1d8-6d0e95256d61. (Possible split-brain)
[2017-03-17 13:34:47.556086] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:d8687951-a556-4747-a1d8-6d0e95256d61>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:34:47.556133] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: d8687951-a556-4747-a1d8-6d0e95256d61 [Stale file handle]
[2017-03-17 13:34:47.559751] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid e607e674-ea62-4e10-85f7-4d2c94516857. (Possible split-brain)
[2017-03-17 13:34:47.561112] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:e607e674-ea62-4e10-85f7-4d2c94516857>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:34:47.561166] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: e607e674-ea62-4e10-85f7-4d2c94516857 [Stale file handle]
[2017-03-17 13:34:47.637800] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-10)
[2017-03-17 13:34:47.644090] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 5df2da07-21a3-4ee9-b722-7b3a684cd645. (Possible split-brain)
[2017-03-17 13:34:47.645862] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:5df2da07-21a3-4ee9-b722-7b3a684cd645>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:34:47.645909] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 5df2da07-21a3-4ee9-b722-7b3a684cd645 [Stale file handle]
[2017-03-17 13:34:47.648609] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid d082d5fd-bede-43f6-883f-f68d2c26b500. (Possible split-brain)
[2017-03-17 13:34:47.650223] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:d082d5fd-bede-43f6-883f-f68d2c26b500>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:34:47.650256] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: d082d5fd-bede-43f6-883f-f68d2c26b500 [Stale file handle]
[2017-03-17 13:34:47.700645] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:34:47.700794] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:34:47.701124] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:34:47.701204] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:34:47.701380] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:34:47.763623] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 5989c469-9f44-41f4-9407-0cfdfa5a5e42. (Possible split-brain)
[2017-03-17 13:34:47.765585] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:5989c469-9f44-41f4-9407-0cfdfa5a5e42>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:34:47.765620] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 5989c469-9f44-41f4-9407-0cfdfa5a5e42 [Stale file handle]
[2017-03-17 13:34:47.808310] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:34:47.814131] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:34:47.814514] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:673f8455-7e43-4362-96c9-2b5216e5cfb8> (673f8455-7e43-4362-96c9-2b5216e5cfb8) [No such file or directory]
[2017-03-17 13:34:47.814613] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:673f8455-7e43-4362-96c9-2b5216e5cfb8> (673f8455-7e43-4362-96c9-2b5216e5cfb8) [No such file or directory]
[2017-03-17 13:34:47.814936] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 673f8455-7e43-4362-96c9-2b5216e5cfb8. (Possible split-brain)
[2017-03-17 13:34:47.815418] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:673f8455-7e43-4362-96c9-2b5216e5cfb8> (673f8455-7e43-4362-96c9-2b5216e5cfb8) [No such file or directory]
[2017-03-17 13:34:47.815459] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:673f8455-7e43-4362-96c9-2b5216e5cfb8> (673f8455-7e43-4362-96c9-2b5216e5cfb8) [No such file or directory]
[2017-03-17 13:34:47.816651] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:673f8455-7e43-4362-96c9-2b5216e5cfb8> (673f8455-7e43-4362-96c9-2b5216e5cfb8) [No such file or directory]
[2017-03-17 13:34:47.816821] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:673f8455-7e43-4362-96c9-2b5216e5cfb8> (673f8455-7e43-4362-96c9-2b5216e5cfb8) [No such file or directory]
[2017-03-17 13:34:47.817191] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:673f8455-7e43-4362-96c9-2b5216e5cfb8>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:34:47.817224] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 673f8455-7e43-4362-96c9-2b5216e5cfb8 [Stale file handle]
[2017-03-17 13:34:47.820555] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid b1d7fb0a-3175-4032-9f52-c4e0dff67f18. (Possible split-brain)
[2017-03-17 13:34:47.822629] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:b1d7fb0a-3175-4032-9f52-c4e0dff67f18>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:34:47.822686] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: b1d7fb0a-3175-4032-9f52-c4e0dff67f18 [Stale file handle]
[2017-03-17 13:34:47.834018] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid 9898af72-b84a-4014-bd95-04a0ca4278e4. (Possible split-brain)
[2017-03-17 13:34:47.835909] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:9898af72-b84a-4014-bd95-04a0ca4278e4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:34:47.835962] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 9898af72-b84a-4014-bd95-04a0ca4278e4 [Stale file handle]
[2017-03-17 13:34:47.839399] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-16: Unreadable subvolume -1 found with event generation 2 for gfid 26496148-df28-46ac-844d-1b5d999db9a8. (Possible split-brain)
[2017-03-17 13:34:47.841756] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:26496148-df28-46ac-844d-1b5d999db9a8>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:34:47.841804] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 26496148-df28-46ac-844d-1b5d999db9a8 [Stale file handle]
[2017-03-17 13:34:47.881345] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:34:47.881603] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:34:47.881710] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-16: selecting local read_child vmware2-client-32
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-20: remote operation failed. Path: <gfid:40b0c3ad-8355-4e2b-b1d4-dfcbe17e18f5> (40b0c3ad-8355-4e2b-b1d4-dfcbe17e18f5) [No such file or directory]" repeated 2 times between [2017-03-17 13:34:47.544593] and [2017-03-17 13:34:47.546770]
[2017-03-17 13:54:37.206473] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:37.213623] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.214118] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:8dfa17cc-62c4-465b-a32c-e982d60e5cdb> (8dfa17cc-62c4-465b-a32c-e982d60e5cdb) [No such file or directory]
[2017-03-17 13:54:37.214176] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:8dfa17cc-62c4-465b-a32c-e982d60e5cdb> (8dfa17cc-62c4-465b-a32c-e982d60e5cdb) [No such file or directory]
[2017-03-17 13:54:37.215268] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 8dfa17cc-62c4-465b-a32c-e982d60e5cdb. (Possible split-brain)
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:8dfa17cc-62c4-465b-a32c-e982d60e5cdb> (8dfa17cc-62c4-465b-a32c-e982d60e5cdb) [No such file or directory]" repeated 2 times between [2017-03-17 13:54:37.214176] and [2017-03-17 13:54:37.217894]
[2017-03-17 13:54:37.218474] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:8dfa17cc-62c4-465b-a32c-e982d60e5cdb>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.218566] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 8dfa17cc-62c4-465b-a32c-e982d60e5cdb [Stale file handle]
[2017-03-17 13:54:37.273389] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:54:37.273725] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:54:37.342261] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.342676] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:98753e77-46a3-4fae-b194-6d61d779d6f5> (98753e77-46a3-4fae-b194-6d61d779d6f5) [No such file or directory]
[2017-03-17 13:54:37.342720] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:98753e77-46a3-4fae-b194-6d61d779d6f5> (98753e77-46a3-4fae-b194-6d61d779d6f5) [No such file or directory]
[2017-03-17 13:54:37.343228] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 98753e77-46a3-4fae-b194-6d61d779d6f5. (Possible split-brain)
[2017-03-17 13:54:37.343839] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:98753e77-46a3-4fae-b194-6d61d779d6f5> (98753e77-46a3-4fae-b194-6d61d779d6f5) [No such file or directory]
[2017-03-17 13:54:37.343835] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:98753e77-46a3-4fae-b194-6d61d779d6f5> (98753e77-46a3-4fae-b194-6d61d779d6f5) [No such file or directory]
[2017-03-17 13:54:37.345591] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:98753e77-46a3-4fae-b194-6d61d779d6f5> (98753e77-46a3-4fae-b194-6d61d779d6f5) [No such file or directory]
[2017-03-17 13:54:37.345640] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:98753e77-46a3-4fae-b194-6d61d779d6f5> (98753e77-46a3-4fae-b194-6d61d779d6f5) [No such file or directory]
[2017-03-17 13:54:37.346141] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:98753e77-46a3-4fae-b194-6d61d779d6f5>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.346174] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 98753e77-46a3-4fae-b194-6d61d779d6f5 [Stale file handle]
The message "I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)" repeated 2 times between [2017-03-17 13:54:37.206473] and [2017-03-17 13:54:37.380434]
[2017-03-17 13:54:37.387088] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.387549] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd> (cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd) [No such file or directory]
[2017-03-17 13:54:37.387561] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd> (cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd) [No such file or directory]
[2017-03-17 13:54:37.388726] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd. (Possible split-brain)
[2017-03-17 13:54:37.389199] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd> (cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd) [No such file or directory]
[2017-03-17 13:54:37.389283] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd> (cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd) [No such file or directory]
[2017-03-17 13:54:37.391083] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd> (cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd) [No such file or directory]
[2017-03-17 13:54:37.391206] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd> (cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd) [No such file or directory]
[2017-03-17 13:54:37.391869] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.391920] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: cb8b5cdb-9f48-43ba-8a16-a5775a7e2fdd [Stale file handle]
[2017-03-17 13:54:37.418894] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:37.425538] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.425976] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:550569b5-3387-49bc-bd26-d17b17c297c6> (550569b5-3387-49bc-bd26-d17b17c297c6) [No such file or directory]
[2017-03-17 13:54:37.426038] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:550569b5-3387-49bc-bd26-d17b17c297c6> (550569b5-3387-49bc-bd26-d17b17c297c6) [No such file or directory]
[2017-03-17 13:54:37.426396] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 550569b5-3387-49bc-bd26-d17b17c297c6. (Possible split-brain)
[2017-03-17 13:54:37.426786] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:550569b5-3387-49bc-bd26-d17b17c297c6> (550569b5-3387-49bc-bd26-d17b17c297c6) [No such file or directory]
[2017-03-17 13:54:37.426880] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:550569b5-3387-49bc-bd26-d17b17c297c6> (550569b5-3387-49bc-bd26-d17b17c297c6) [No such file or directory]
[2017-03-17 13:54:37.428337] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:550569b5-3387-49bc-bd26-d17b17c297c6> (550569b5-3387-49bc-bd26-d17b17c297c6) [No such file or directory]
[2017-03-17 13:54:37.428554] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:550569b5-3387-49bc-bd26-d17b17c297c6> (550569b5-3387-49bc-bd26-d17b17c297c6) [No such file or directory]
[2017-03-17 13:54:37.428948] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:550569b5-3387-49bc-bd26-d17b17c297c6>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.429019] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 550569b5-3387-49bc-bd26-d17b17c297c6 [Stale file handle]
[2017-03-17 13:54:37.454205] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:37.461001] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.461532] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:fe672966-5f54-4141-8dad-3ec2e64fe884> (fe672966-5f54-4141-8dad-3ec2e64fe884) [No such file or directory]
[2017-03-17 13:54:37.461575] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:fe672966-5f54-4141-8dad-3ec2e64fe884> (fe672966-5f54-4141-8dad-3ec2e64fe884) [No such file or directory]
[2017-03-17 13:54:37.461997] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid fe672966-5f54-4141-8dad-3ec2e64fe884. (Possible split-brain)
[2017-03-17 13:54:37.462536] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:fe672966-5f54-4141-8dad-3ec2e64fe884> (fe672966-5f54-4141-8dad-3ec2e64fe884) [No such file or directory]
[2017-03-17 13:54:37.462541] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:fe672966-5f54-4141-8dad-3ec2e64fe884> (fe672966-5f54-4141-8dad-3ec2e64fe884) [No such file or directory]
[2017-03-17 13:54:37.464117] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:fe672966-5f54-4141-8dad-3ec2e64fe884> (fe672966-5f54-4141-8dad-3ec2e64fe884) [No such file or directory]
[2017-03-17 13:54:37.464273] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:fe672966-5f54-4141-8dad-3ec2e64fe884> (fe672966-5f54-4141-8dad-3ec2e64fe884) [No such file or directory]
[2017-03-17 13:54:37.464790] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:fe672966-5f54-4141-8dad-3ec2e64fe884>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.464855] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: fe672966-5f54-4141-8dad-3ec2e64fe884 [Stale file handle]
[2017-03-17 13:54:37.515620] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:54:37.516186] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:54:37.549321] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:37.556812] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.557363] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:e9bca48f-a371-470d-b512-25c5f974f4f0> (e9bca48f-a371-470d-b512-25c5f974f4f0) [No such file or directory]
[2017-03-17 13:54:37.557445] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:e9bca48f-a371-470d-b512-25c5f974f4f0> (e9bca48f-a371-470d-b512-25c5f974f4f0) [No such file or directory]
[2017-03-17 13:54:37.557830] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid e9bca48f-a371-470d-b512-25c5f974f4f0. (Possible split-brain)
[2017-03-17 13:54:37.558348] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:e9bca48f-a371-470d-b512-25c5f974f4f0> (e9bca48f-a371-470d-b512-25c5f974f4f0) [No such file or directory]
[2017-03-17 13:54:37.558424] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:e9bca48f-a371-470d-b512-25c5f974f4f0> (e9bca48f-a371-470d-b512-25c5f974f4f0) [No such file or directory]
[2017-03-17 13:54:37.560197] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:e9bca48f-a371-470d-b512-25c5f974f4f0> (e9bca48f-a371-470d-b512-25c5f974f4f0) [No such file or directory]
[2017-03-17 13:54:37.560269] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:e9bca48f-a371-470d-b512-25c5f974f4f0> (e9bca48f-a371-470d-b512-25c5f974f4f0) [No such file or directory]
[2017-03-17 13:54:37.560677] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:e9bca48f-a371-470d-b512-25c5f974f4f0>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.560747] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: e9bca48f-a371-470d-b512-25c5f974f4f0 [Stale file handle]
[2017-03-17 13:54:37.592337] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:37.598792] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.599277] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:bfd65146-bbd1-4fa9-ae24-fff89321848a> (bfd65146-bbd1-4fa9-ae24-fff89321848a) [No such file or directory]
[2017-03-17 13:54:37.599312] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:bfd65146-bbd1-4fa9-ae24-fff89321848a> (bfd65146-bbd1-4fa9-ae24-fff89321848a) [No such file or directory]
[2017-03-17 13:54:37.599654] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid bfd65146-bbd1-4fa9-ae24-fff89321848a. (Possible split-brain)
[2017-03-17 13:54:37.600106] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:bfd65146-bbd1-4fa9-ae24-fff89321848a> (bfd65146-bbd1-4fa9-ae24-fff89321848a) [No such file or directory]
[2017-03-17 13:54:37.600172] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:bfd65146-bbd1-4fa9-ae24-fff89321848a> (bfd65146-bbd1-4fa9-ae24-fff89321848a) [No such file or directory]
[2017-03-17 13:54:37.601686] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:bfd65146-bbd1-4fa9-ae24-fff89321848a> (bfd65146-bbd1-4fa9-ae24-fff89321848a) [No such file or directory]
[2017-03-17 13:54:37.601927] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:bfd65146-bbd1-4fa9-ae24-fff89321848a> (bfd65146-bbd1-4fa9-ae24-fff89321848a) [No such file or directory]
[2017-03-17 13:54:37.602271] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:bfd65146-bbd1-4fa9-ae24-fff89321848a>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.602326] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: bfd65146-bbd1-4fa9-ae24-fff89321848a [Stale file handle]
[2017-03-17 13:54:37.634759] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:37.641287] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.641836] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:fc774283-db0b-4f76-b618-0bbd71a77a48> (fc774283-db0b-4f76-b618-0bbd71a77a48) [No such file or directory]
[2017-03-17 13:54:37.641840] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:fc774283-db0b-4f76-b618-0bbd71a77a48> (fc774283-db0b-4f76-b618-0bbd71a77a48) [No such file or directory]
[2017-03-17 13:54:37.642342] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid fc774283-db0b-4f76-b618-0bbd71a77a48. (Possible split-brain)
[2017-03-17 13:54:37.642843] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:fc774283-db0b-4f76-b618-0bbd71a77a48> (fc774283-db0b-4f76-b618-0bbd71a77a48) [No such file or directory]
[2017-03-17 13:54:37.642902] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:fc774283-db0b-4f76-b618-0bbd71a77a48> (fc774283-db0b-4f76-b618-0bbd71a77a48) [No such file or directory]
[2017-03-17 13:54:37.644386] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:fc774283-db0b-4f76-b618-0bbd71a77a48> (fc774283-db0b-4f76-b618-0bbd71a77a48) [No such file or directory]
[2017-03-17 13:54:37.644470] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:fc774283-db0b-4f76-b618-0bbd71a77a48> (fc774283-db0b-4f76-b618-0bbd71a77a48) [No such file or directory]
[2017-03-17 13:54:37.644913] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:fc774283-db0b-4f76-b618-0bbd71a77a48>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.644980] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: fc774283-db0b-4f76-b618-0bbd71a77a48 [Stale file handle]
[2017-03-17 13:54:37.674823] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:37.681192] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.681611] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:bfc19b99-1386-48b5-807d-c0734855a4bc> (bfc19b99-1386-48b5-807d-c0734855a4bc) [No such file or directory]
[2017-03-17 13:54:37.681663] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:bfc19b99-1386-48b5-807d-c0734855a4bc> (bfc19b99-1386-48b5-807d-c0734855a4bc) [No such file or directory]
[2017-03-17 13:54:37.682127] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid bfc19b99-1386-48b5-807d-c0734855a4bc. (Possible split-brain)
[2017-03-17 13:54:37.682594] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:bfc19b99-1386-48b5-807d-c0734855a4bc> (bfc19b99-1386-48b5-807d-c0734855a4bc) [No such file or directory]
[2017-03-17 13:54:37.682647] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:bfc19b99-1386-48b5-807d-c0734855a4bc> (bfc19b99-1386-48b5-807d-c0734855a4bc) [No such file or directory]
[2017-03-17 13:54:37.684048] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:bfc19b99-1386-48b5-807d-c0734855a4bc> (bfc19b99-1386-48b5-807d-c0734855a4bc) [No such file or directory]
[2017-03-17 13:54:37.684179] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:bfc19b99-1386-48b5-807d-c0734855a4bc> (bfc19b99-1386-48b5-807d-c0734855a4bc) [No such file or directory]
[2017-03-17 13:54:37.684697] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:bfc19b99-1386-48b5-807d-c0734855a4bc>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.684756] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: bfc19b99-1386-48b5-807d-c0734855a4bc [Stale file handle]
[2017-03-17 13:54:37.741652] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:37.748092] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.748560] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:479e2eab-35c1-4b58-bf4f-130bde88bf23> (479e2eab-35c1-4b58-bf4f-130bde88bf23) [No such file or directory]
[2017-03-17 13:54:37.748647] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:479e2eab-35c1-4b58-bf4f-130bde88bf23> (479e2eab-35c1-4b58-bf4f-130bde88bf23) [No such file or directory]
[2017-03-17 13:54:37.749036] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 479e2eab-35c1-4b58-bf4f-130bde88bf23. (Possible split-brain)
[2017-03-17 13:54:37.749486] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:479e2eab-35c1-4b58-bf4f-130bde88bf23> (479e2eab-35c1-4b58-bf4f-130bde88bf23) [No such file or directory]
[2017-03-17 13:54:37.749575] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:479e2eab-35c1-4b58-bf4f-130bde88bf23> (479e2eab-35c1-4b58-bf4f-130bde88bf23) [No such file or directory]
[2017-03-17 13:54:37.751133] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:479e2eab-35c1-4b58-bf4f-130bde88bf23> (479e2eab-35c1-4b58-bf4f-130bde88bf23) [No such file or directory]
[2017-03-17 13:54:37.751145] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:479e2eab-35c1-4b58-bf4f-130bde88bf23> (479e2eab-35c1-4b58-bf4f-130bde88bf23) [No such file or directory]
[2017-03-17 13:54:37.751702] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:479e2eab-35c1-4b58-bf4f-130bde88bf23>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.751758] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 479e2eab-35c1-4b58-bf4f-130bde88bf23 [Stale file handle]
[2017-03-17 13:54:37.779991] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:37.786459] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.786999] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:26f49700-2658-49da-b38d-b5a5c53f01bc> (26f49700-2658-49da-b38d-b5a5c53f01bc) [No such file or directory]
[2017-03-17 13:54:37.787004] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:26f49700-2658-49da-b38d-b5a5c53f01bc> (26f49700-2658-49da-b38d-b5a5c53f01bc) [No such file or directory]
[2017-03-17 13:54:37.787601] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 26f49700-2658-49da-b38d-b5a5c53f01bc. (Possible split-brain)
[2017-03-17 13:54:37.788071] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:26f49700-2658-49da-b38d-b5a5c53f01bc> (26f49700-2658-49da-b38d-b5a5c53f01bc) [No such file or directory]
[2017-03-17 13:54:37.788140] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:26f49700-2658-49da-b38d-b5a5c53f01bc> (26f49700-2658-49da-b38d-b5a5c53f01bc) [No such file or directory]
[2017-03-17 13:54:37.789646] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:26f49700-2658-49da-b38d-b5a5c53f01bc> (26f49700-2658-49da-b38d-b5a5c53f01bc) [No such file or directory]
[2017-03-17 13:54:37.789814] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:26f49700-2658-49da-b38d-b5a5c53f01bc> (26f49700-2658-49da-b38d-b5a5c53f01bc) [No such file or directory]
[2017-03-17 13:54:37.790330] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:26f49700-2658-49da-b38d-b5a5c53f01bc>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.790389] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 26f49700-2658-49da-b38d-b5a5c53f01bc [Stale file handle]
[2017-03-17 13:54:37.815726] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:37.822032] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.822511] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:e3c4bf0f-a297-47c0-87bb-0be4eba853be> (e3c4bf0f-a297-47c0-87bb-0be4eba853be) [No such file or directory]
[2017-03-17 13:54:37.822562] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:e3c4bf0f-a297-47c0-87bb-0be4eba853be> (e3c4bf0f-a297-47c0-87bb-0be4eba853be) [No such file or directory]
[2017-03-17 13:54:37.822971] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid e3c4bf0f-a297-47c0-87bb-0be4eba853be. (Possible split-brain)
[2017-03-17 13:54:37.823409] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:e3c4bf0f-a297-47c0-87bb-0be4eba853be> (e3c4bf0f-a297-47c0-87bb-0be4eba853be) [No such file or directory]
[2017-03-17 13:54:37.823468] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:e3c4bf0f-a297-47c0-87bb-0be4eba853be> (e3c4bf0f-a297-47c0-87bb-0be4eba853be) [No such file or directory]
[2017-03-17 13:54:37.825032] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:e3c4bf0f-a297-47c0-87bb-0be4eba853be> (e3c4bf0f-a297-47c0-87bb-0be4eba853be) [No such file or directory]
[2017-03-17 13:54:37.825070] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:e3c4bf0f-a297-47c0-87bb-0be4eba853be> (e3c4bf0f-a297-47c0-87bb-0be4eba853be) [No such file or directory]
[2017-03-17 13:54:37.825481] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:e3c4bf0f-a297-47c0-87bb-0be4eba853be>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.825537] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: e3c4bf0f-a297-47c0-87bb-0be4eba853be [Stale file handle]
[2017-03-17 13:54:37.853134] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:37.859721] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:37.860253] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:15b42a46-ecf9-4411-a2cd-bb93389fbddc> (15b42a46-ecf9-4411-a2cd-bb93389fbddc) [No such file or directory]
[2017-03-17 13:54:37.860269] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:15b42a46-ecf9-4411-a2cd-bb93389fbddc> (15b42a46-ecf9-4411-a2cd-bb93389fbddc) [No such file or directory]
[2017-03-17 13:54:37.860703] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 15b42a46-ecf9-4411-a2cd-bb93389fbddc. (Possible split-brain)
[2017-03-17 13:54:37.861190] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:15b42a46-ecf9-4411-a2cd-bb93389fbddc> (15b42a46-ecf9-4411-a2cd-bb93389fbddc) [No such file or directory]
[2017-03-17 13:54:37.861206] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:15b42a46-ecf9-4411-a2cd-bb93389fbddc> (15b42a46-ecf9-4411-a2cd-bb93389fbddc) [No such file or directory]
[2017-03-17 13:54:37.862656] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:15b42a46-ecf9-4411-a2cd-bb93389fbddc> (15b42a46-ecf9-4411-a2cd-bb93389fbddc) [No such file or directory]
[2017-03-17 13:54:37.862718] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:15b42a46-ecf9-4411-a2cd-bb93389fbddc> (15b42a46-ecf9-4411-a2cd-bb93389fbddc) [No such file or directory]
[2017-03-17 13:54:37.863223] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:15b42a46-ecf9-4411-a2cd-bb93389fbddc>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:37.863287] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 15b42a46-ecf9-4411-a2cd-bb93389fbddc [Stale file handle]
[2017-03-17 13:54:37.905479] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:54:37.906683] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 15-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:54:46.198410] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmdk~ (hash=vmware2-replicate-10/cache=vmware2-replicate-10) => /New Virtual Machine 5/New Virtual Machine 5.vmdk (hash=vmware2-replicate-12/cache=vmware2-replicate-10)
[2017-03-17 13:54:46.204952] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-20: remote operation failed [No such file or directory]
[2017-03-17 13:54:46.205463] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-20: remote operation failed. Path: <gfid:8c2b754a-4212-40c5-ac4c-621f282163a8> (8c2b754a-4212-40c5-ac4c-621f282163a8) [No such file or directory]
[2017-03-17 13:54:46.205557] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-21: remote operation failed. Path: <gfid:8c2b754a-4212-40c5-ac4c-621f282163a8> (8c2b754a-4212-40c5-ac4c-621f282163a8) [No such file or directory]
[2017-03-17 13:54:46.206159] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 8c2b754a-4212-40c5-ac4c-621f282163a8. (Possible split-brain)
[2017-03-17 13:54:46.206657] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-21: remote operation failed. Path: <gfid:8c2b754a-4212-40c5-ac4c-621f282163a8> (8c2b754a-4212-40c5-ac4c-621f282163a8) [No such file or directory]
[2017-03-17 13:54:46.206717] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-20: remote operation failed. Path: <gfid:8c2b754a-4212-40c5-ac4c-621f282163a8> (8c2b754a-4212-40c5-ac4c-621f282163a8) [No such file or directory]
[2017-03-17 13:54:46.207993] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-21: remote operation failed. Path: <gfid:8c2b754a-4212-40c5-ac4c-621f282163a8> (8c2b754a-4212-40c5-ac4c-621f282163a8) [No such file or directory]
[2017-03-17 13:54:46.208040] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-20: remote operation failed. Path: <gfid:8c2b754a-4212-40c5-ac4c-621f282163a8> (8c2b754a-4212-40c5-ac4c-621f282163a8) [No such file or directory]
[2017-03-17 13:54:46.208958] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:8c2b754a-4212-40c5-ac4c-621f282163a8>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:46.209034] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 8c2b754a-4212-40c5-ac4c-621f282163a8 [Stale file handle]
[2017-03-17 13:54:46.213787] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-9: Unreadable subvolume -1 found with event generation 2 for gfid 20cd18b8-c164-49a0-8a5f-2b8808ba3d4b. (Possible split-brain)
[2017-03-17 13:54:46.216550] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:20cd18b8-c164-49a0-8a5f-2b8808ba3d4b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:46.216606] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 20cd18b8-c164-49a0-8a5f-2b8808ba3d4b [Stale file handle]
[2017-03-17 13:54:55.880465] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 15-vmware2-dht: renaming /New Virtual Machine 5/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-14/cache=vmware2-replicate-14) => /New Virtual Machine 5/New Virtual Machine 5.vmx (hash=vmware2-replicate-10/cache=vmware2-replicate-14)
[2017-03-17 13:54:55.887552] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 15-vmware2-client-28: remote operation failed [No such file or directory]
[2017-03-17 13:54:55.888027] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:1e47c576-6c3a-42c7-a756-04f07027f1c7> (1e47c576-6c3a-42c7-a756-04f07027f1c7) [No such file or directory]
[2017-03-17 13:54:55.888091] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:1e47c576-6c3a-42c7-a756-04f07027f1c7> (1e47c576-6c3a-42c7-a756-04f07027f1c7) [No such file or directory]
[2017-03-17 13:54:55.888700] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 15-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid 1e47c576-6c3a-42c7-a756-04f07027f1c7. (Possible split-brain)
[2017-03-17 13:54:55.889241] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:1e47c576-6c3a-42c7-a756-04f07027f1c7> (1e47c576-6c3a-42c7-a756-04f07027f1c7) [No such file or directory]
[2017-03-17 13:54:55.889235] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:1e47c576-6c3a-42c7-a756-04f07027f1c7> (1e47c576-6c3a-42c7-a756-04f07027f1c7) [No such file or directory]
[2017-03-17 13:54:55.890857] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:1e47c576-6c3a-42c7-a756-04f07027f1c7> (1e47c576-6c3a-42c7-a756-04f07027f1c7) [No such file or directory]
[2017-03-17 13:54:55.891030] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-29: remote operation failed. Path: <gfid:1e47c576-6c3a-42c7-a756-04f07027f1c7> (1e47c576-6c3a-42c7-a756-04f07027f1c7) [No such file or directory]
[2017-03-17 13:54:55.891469] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 15-vmware2-dht: <gfid:1e47c576-6c3a-42c7-a756-04f07027f1c7>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 13:54:55.891520] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 15-vmware2-shard: stat failed: 1e47c576-6c3a-42c7-a756-04f07027f1c7 [Stale file handle]
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 15-vmware2-client-28: remote operation failed. Path: <gfid:8dfa17cc-62c4-465b-a32c-e982d60e5cdb> (8dfa17cc-62c4-465b-a32c-e982d60e5cdb) [No such file or directory]" repeated 2 times between [2017-03-17 13:54:37.214118] and [2017-03-17 13:54:37.217736]
[2017-03-17 13:57:43.680994] I [MSGID: 104045] [glfs-master.c:91:notify] 0-gfapi: New graph 676c7573-7465-7230-312d-313938392d32 (17) coming up
[2017-03-17 13:57:43.681048] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-0: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.683964] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-1: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.684261] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-0: changing port to 49512 (from 0)
[2017-03-17 13:57:43.686581] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-2: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.691403] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.691915] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-0: Connected to vmware2-client-0, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:57:43.691941] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.692011] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-0: Subvolume 'vmware2-client-0' came back up; going online.
[2017-03-17 13:57:43.692076] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-0: Server lk version = 1
[2017-03-17 13:57:43.693494] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-3: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.696097] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-4: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.698699] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-5: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.698923] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-4: changing port to 49513 (from 0)
[2017-03-17 13:57:43.701292] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-6: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.705638] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-4: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.707305] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-4: Connected to vmware2-client-4, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:57:43.707330] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-4: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.707394] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-2: Subvolume 'vmware2-client-4' came back up; going online.
[2017-03-17 13:57:43.707451] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-4: Server lk version = 1
[2017-03-17 13:57:43.709752] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-7: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.714046] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-8: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.718312] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-9: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.718508] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-8: changing port to 49514 (from 0)
[2017-03-17 13:57:43.722599] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-10: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.726653] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-8: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.727193] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-8: Connected to vmware2-client-8, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:57:43.727221] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-8: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.727287] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-4: Subvolume 'vmware2-client-8' came back up; going online.
[2017-03-17 13:57:43.727372] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-8: Server lk version = 1
[2017-03-17 13:57:43.730257] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-11: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.734192] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-12: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.737922] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-13: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.738074] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-12: changing port to 49515 (from 0)
[2017-03-17 13:57:43.741680] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-14: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.746717] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-1: changing port to 49489 (from 0)
[2017-03-17 13:57:43.746782] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-11: changing port to 49387 (from 0)
[2017-03-17 13:57:43.746819] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-3: changing port to 49385 (from 0)
[2017-03-17 13:57:43.746845] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-5: changing port to 49490 (from 0)
[2017-03-17 13:57:43.746875] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-12: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.746958] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-9: changing port to 49491 (from 0)
[2017-03-17 13:57:43.749717] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-15: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.754202] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-7: changing port to 49386 (from 0)
[2017-03-17 13:57:43.757248] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-16: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.764596] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-17: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.771720] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-18: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.778583] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-19: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.782141] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-12: Connected to vmware2-client-12, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:57:43.782177] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-12: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.782249] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-6: Subvolume 'vmware2-client-12' came back up; going online.
[2017-03-17 13:57:43.784672] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-20: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.788338] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-13: changing port to 49492 (from 0)
[2017-03-17 13:57:43.788721] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-12: Server lk version = 1
[2017-03-17 13:57:43.788807] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.790707] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-21: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.794311] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-11: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.794394] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-3: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.794468] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-2: changing port to 49414 (from 0)
[2017-03-17 13:57:43.794504] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-10: changing port to 49416 (from 0)
[2017-03-17 13:57:43.794524] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-6: changing port to 49415 (from 0)
[2017-03-17 13:57:43.794545] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-9: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.794592] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-5: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.794675] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-14: changing port to 49417 (from 0)
[2017-03-17 13:57:43.794735] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-1: Connected to vmware2-client-1, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:57:43.794748] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-1: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.794946] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-16: changing port to 49516 (from 0)
[2017-03-17 13:57:43.795013] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-7: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.795088] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-15: changing port to 49388 (from 0)
[2017-03-17 13:57:43.796490] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-22: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.802232] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-23: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.807776] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-24: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.811059] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-17: changing port to 49493 (from 0)
[2017-03-17 13:57:43.813351] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-25: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.816629] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-18: changing port to 49418 (from 0)
[2017-03-17 13:57:43.816687] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-11: Connected to vmware2-client-11, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:57:43.816711] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-11: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.816752] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-5: Subvolume 'vmware2-client-11' came back up; going online.
[2017-03-17 13:57:43.816805] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-19: changing port to 49389 (from 0)
[2017-03-17 13:57:43.816832] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-1: Server lk version = 1
[2017-03-17 13:57:43.816859] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-3: Connected to vmware2-client-3, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:57:43.816869] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-3: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.816903] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-1: Subvolume 'vmware2-client-3' came back up; going online.
[2017-03-17 13:57:43.818818] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-26: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.822136] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-13: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.822226] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-5: Connected to vmware2-client-5, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:57:43.822242] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-5: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.822345] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-9: Connected to vmware2-client-9, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:57:43.822359] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-9: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.824270] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-27: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.827587] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-7: Connected to vmware2-client-7, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:57:43.827614] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-7: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.827657] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-3: Subvolume 'vmware2-client-7' came back up; going online.
[2017-03-17 13:57:43.829784] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-28: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.835206] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-29: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.840562] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-30: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.843721] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-11: Server lk version = 1
[2017-03-17 13:57:43.843755] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-3: Server lk version = 1
[2017-03-17 13:57:43.843785] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-20: changing port to 49517 (from 0)
[2017-03-17 13:57:43.843903] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-5: Server lk version = 1
[2017-03-17 13:57:43.843936] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-9: Server lk version = 1
[2017-03-17 13:57:43.843979] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-13: Connected to vmware2-client-13, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:57:43.843990] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-13: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.844151] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-7: Server lk version = 1
[2017-03-17 13:57:43.844208] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-10: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.844265] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.844399] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-6: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.844551] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-14: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.845951] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-31: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.850696] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-16: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.850876] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-13: Server lk version = 1
[2017-03-17 13:57:43.850925] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-21: changing port to 49494 (from 0)
[2017-03-17 13:57:43.851009] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-15: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.851078] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-23: changing port to 49390 (from 0)
[2017-03-17 13:57:43.851141] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-17: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.851204] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-22: changing port to 49419 (from 0)
[2017-03-17 13:57:43.851233] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-24: changing port to 49518 (from 0)
[2017-03-17 13:57:43.851313] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-18: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.851428] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-10: Connected to vmware2-client-10, attached to remote volume '/mnt/disk3/vmware2'.
[2017-03-17 13:57:43.851445] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-10: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.851561] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-19: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.851638] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-2: Connected to vmware2-client-2, attached to remote volume '/mnt/disk1/vmware2'.
[2017-03-17 13:57:43.851654] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.851775] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-6: Connected to vmware2-client-6, attached to remote volume '/mnt/disk2/vmware2'.
[2017-03-17 13:57:43.851811] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-6: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.851931] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-14: Connected to vmware2-client-14, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:57:43.851949] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-14: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.851993] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-7: Subvolume 'vmware2-client-14' came back up; going online.
[2017-03-17 13:57:43.852816] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-32: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.857646] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-25: changing port to 49495 (from 0)
[2017-03-17 13:57:43.857697] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-26: changing port to 49420 (from 0)
[2017-03-17 13:57:43.859777] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-33: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.864308] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-16: Connected to vmware2-client-16, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:57:43.864341] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-16: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.864419] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-8: Subvolume 'vmware2-client-16' came back up; going online.
[2017-03-17 13:57:43.866404] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-34: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.870987] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-27: changing port to 49391 (from 0)
[2017-03-17 13:57:43.873092] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-35: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.877627] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-28: changing port to 49519 (from 0)
[2017-03-17 13:57:43.877692] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-29: changing port to 49496 (from 0)
[2017-03-17 13:57:43.877740] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-17: Connected to vmware2-client-17, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:57:43.877759] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-17: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.877911] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-10: Server lk version = 1
[2017-03-17 13:57:43.877948] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-15: Connected to vmware2-client-15, attached to remote volume '/mnt/disk4/vmware2'.
[2017-03-17 13:57:43.877960] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-15: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.878057] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-2: Server lk version = 1
[2017-03-17 13:57:43.878090] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-18: Connected to vmware2-client-18, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:57:43.878101] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-18: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.878138] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-9: Subvolume 'vmware2-client-18' came back up; going online.
[2017-03-17 13:57:43.878165] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-6: Server lk version = 1
[2017-03-17 13:57:43.878213] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-19: Connected to vmware2-client-19, attached to remote volume '/mnt/disk5/vmware2'.
[2017-03-17 13:57:43.878226] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-19: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.878312] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-14: Server lk version = 1
[2017-03-17 13:57:43.878399] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-20: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.879740] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-36: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.886146] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-37: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.890452] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-16: Server lk version = 1
[2017-03-17 13:57:43.892518] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-38: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.898933] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-39: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.905255] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-40: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.909450] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-17: Server lk version = 1
[2017-03-17 13:57:43.909509] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-15: Server lk version = 1
[2017-03-17 13:57:43.909537] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-18: Server lk version = 1
[2017-03-17 13:57:43.909562] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-19: Server lk version = 1
[2017-03-17 13:57:43.909580] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-30: changing port to 49421 (from 0)
[2017-03-17 13:57:43.909660] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-20: Connected to vmware2-client-20, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:57:43.909675] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-20: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.909736] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-10: Subvolume 'vmware2-client-20' came back up; going online.
[2017-03-17 13:57:43.909831] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-21: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.909995] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-23: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.910124] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-22: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.910182] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-24: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.911616] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-41: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.915582] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-31: changing port to 49392 (from 0)
[2017-03-17 13:57:43.915635] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-20: Server lk version = 1
[2017-03-17 13:57:43.915717] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-25: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.915840] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-32: changing port to 49520 (from 0)
[2017-03-17 13:57:43.915885] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-26: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.915977] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-33: changing port to 49497 (from 0)
[2017-03-17 13:57:43.916004] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-27: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.916071] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-21: Connected to vmware2-client-21, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:57:43.916086] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-21: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.916255] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-28: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.916317] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-34: changing port to 49422 (from 0)
[2017-03-17 13:57:43.916355] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-29: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.916461] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-22: Connected to vmware2-client-22, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:57:43.916475] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-22: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.916512] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-11: Subvolume 'vmware2-client-22' came back up; going online.
[2017-03-17 13:57:43.916547] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-23: Connected to vmware2-client-23, attached to remote volume '/mnt/disk6/vmware2'.
[2017-03-17 13:57:43.916557] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-23: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.916651] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-24: Connected to vmware2-client-24, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:57:43.916665] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-24: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.916715] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-12: Subvolume 'vmware2-client-24' came back up; going online.
[2017-03-17 13:57:43.917748] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-42: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.921797] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-35: changing port to 49393 (from 0)
[2017-03-17 13:57:43.923898] I [MSGID: 114020] [client.c:2356:notify] 17-vmware2-client-43: parent translators are ready, attempting connect on transport
[2017-03-17 13:57:43.927853] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-36: changing port to 49521 (from 0)
[2017-03-17 13:57:43.930048] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-37: changing port to 49498 (from 0)
[2017-03-17 13:57:43.930093] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-25: Connected to vmware2-client-25, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:57:43.930104] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-25: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.930196] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-21: Server lk version = 1
[2017-03-17 13:57:43.933864] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-38: changing port to 49423 (from 0)
[2017-03-17 13:57:43.933929] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-26: Connected to vmware2-client-26, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:57:43.933947] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-26: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.933994] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-13: Subvolume 'vmware2-client-26' came back up; going online.
[2017-03-17 13:57:43.934034] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-27: Connected to vmware2-client-27, attached to remote volume '/mnt/disk7/vmware2'.
[2017-03-17 13:57:43.934045] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-27: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.934127] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-39: changing port to 49394 (from 0)
[2017-03-17 13:57:43.934158] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-22: Server lk version = 1
[2017-03-17 13:57:43.934188] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-28: Connected to vmware2-client-28, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:57:43.934198] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-28: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.934259] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-14: Subvolume 'vmware2-client-28' came back up; going online.
[2017-03-17 13:57:43.934288] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-23: Server lk version = 1
[2017-03-17 13:57:43.934306] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-24: Server lk version = 1
[2017-03-17 13:57:43.934379] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-29: Connected to vmware2-client-29, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:57:43.934392] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-29: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.936058] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-30: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.942142] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-25: Server lk version = 1
[2017-03-17 13:57:43.946077] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-26: Server lk version = 1
[2017-03-17 13:57:43.948226] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-27: Server lk version = 1
[2017-03-17 13:57:43.948272] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-28: Server lk version = 1
[2017-03-17 13:57:43.948285] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-40: changing port to 49523 (from 0)
[2017-03-17 13:57:43.948312] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-29: Server lk version = 1
[2017-03-17 13:57:43.948330] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-31: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.948420] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-32: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.948506] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-30: Connected to vmware2-client-30, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:57:43.948517] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-30: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.948543] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-15: Subvolume 'vmware2-client-30' came back up; going online.
[2017-03-17 13:57:43.948647] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-33: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.952162] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-34: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.952250] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-41: changing port to 49500 (from 0)
[2017-03-17 13:57:43.952280] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-42: changing port to 49425 (from 0)
[2017-03-17 13:57:43.952302] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-30: Server lk version = 1
[2017-03-17 13:57:43.952345] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-35: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.952429] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-36: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.952498] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-43: changing port to 49395 (from 0)
[2017-03-17 13:57:43.952538] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-38: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.952595] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-37: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.952660] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-31: Connected to vmware2-client-31, attached to remote volume '/mnt/disk8/vmware2'.
[2017-03-17 13:57:43.952674] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-31: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.952781] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-32: Connected to vmware2-client-32, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:57:43.952801] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-32: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.952860] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-16: Subvolume 'vmware2-client-32' came back up; going online.
[2017-03-17 13:57:43.952899] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-33: Connected to vmware2-client-33, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:57:43.952910] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-33: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.960419] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-34: Connected to vmware2-client-34, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:57:43.960439] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-34: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.960472] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-17: Subvolume 'vmware2-client-34' came back up; going online.
[2017-03-17 13:57:43.960492] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-31: Server lk version = 1
[2017-03-17 13:57:43.960524] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-35: Connected to vmware2-client-35, attached to remote volume '/mnt/disk9/vmware2'.
[2017-03-17 13:57:43.960535] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-35: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.960598] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-32: Server lk version = 1
[2017-03-17 13:57:43.960621] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-36: Connected to vmware2-client-36, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:57:43.960627] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-36: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.960670] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-18: Subvolume 'vmware2-client-36' came back up; going online.
[2017-03-17 13:57:43.960699] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-38: Connected to vmware2-client-38, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:57:43.960706] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-38: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.960736] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-19: Subvolume 'vmware2-client-38' came back up; going online.
[2017-03-17 13:57:43.960753] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-33: Server lk version = 1
[2017-03-17 13:57:43.960779] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-39: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.960838] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-37: Connected to vmware2-client-37, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:57:43.960847] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-37: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.960997] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-34: Server lk version = 1
[2017-03-17 13:57:43.961014] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-35: Server lk version = 1
[2017-03-17 13:57:43.961026] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-36: Server lk version = 1
[2017-03-17 13:57:43.961047] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-38: Server lk version = 1
[2017-03-17 13:57:43.961084] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-37: Server lk version = 1
[2017-03-17 13:57:43.961150] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-40: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.961206] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-42: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.961259] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-41: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.961433] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-39: Connected to vmware2-client-39, attached to remote volume '/mnt/disk10/vmware2'.
[2017-03-17 13:57:43.961465] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-39: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.961645] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-39: Server lk version = 1
[2017-03-17 13:57:43.964446] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-43: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 13:57:43.965474] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-41: Connected to vmware2-client-41, attached to remote volume '/mnt/disk11/vmware2'.
[2017-03-17 13:57:43.965508] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-41: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.965573] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-20: Subvolume 'vmware2-client-41' came back up; going online.
[2017-03-17 13:57:43.965681] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-41: Server lk version = 1
[2017-03-17 13:57:43.965940] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-40: Connected to vmware2-client-40, attached to remote volume '/mnt/disk11/vmware2'.
[2017-03-17 13:57:43.965962] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-40: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.966126] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-40: Server lk version = 1
[2017-03-17 13:57:43.966463] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-42: Connected to vmware2-client-42, attached to remote volume '/mnt/disk11/vmware2'.
[2017-03-17 13:57:43.966493] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-42: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.966553] I [MSGID: 108005] [afr-common.c:4387:afr_notify] 17-vmware2-replicate-21: Subvolume 'vmware2-client-42' came back up; going online.
[2017-03-17 13:57:43.966653] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-42: Server lk version = 1
[2017-03-17 13:57:43.969130] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-43: Connected to vmware2-client-43, attached to remote volume '/mnt/disk11/vmware2'.
[2017-03-17 13:57:43.969162] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-43: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 13:57:43.987138] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-43: Server lk version = 1
[2017-03-17 13:57:43.991400] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 13:57:43.992211] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 13:57:43.992452] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 13:57:43.992645] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 13:57:43.993054] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 13:57:43.993118] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 13:57:43.993148] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 13:57:43.993259] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 13:57:43.993259] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-16: selecting local read_child vmware2-client-32
[2017-03-17 13:57:43.993341] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-20: selecting local read_child vmware2-client-40
[2017-03-17 13:57:43.993365] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 13:57:44.046149] I [MSGID: 104041] [glfs-resolve.c:885:__glfs_active_subvol] 0-vmware2: switched to graph 676c7573-7465-7230-312d-313938392d32 (17)
[2017-03-17 13:57:44.046183] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-0: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046219] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-1: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046248] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-2: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046244] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-0: disconnected from vmware2-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046282] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-3: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046277] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-1: disconnected from vmware2-client-1. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046305] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.046306] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-4: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046342] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-5: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046344] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-2: disconnected from vmware2-client-2. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046367] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-6: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046370] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-3: disconnected from vmware2-client-3. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046396] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-1: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.046398] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-7: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046421] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-4: disconnected from vmware2-client-4. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046426] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-8: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046429] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-5: disconnected from vmware2-client-5. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046445] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-2: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.046450] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-6: disconnected from vmware2-client-6. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046455] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-9: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046489] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-7: disconnected from vmware2-client-7. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046496] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-10: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046504] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-3: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.046524] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-11: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046525] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-8: disconnected from vmware2-client-8. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046534] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-9: disconnected from vmware2-client-9. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046553] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-12: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046560] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-4: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.046575] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-10: disconnected from vmware2-client-10. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046583] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-13: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046600] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-11: disconnected from vmware2-client-11. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046603] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-14: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046621] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-5: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.046626] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-15: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046666] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-12: disconnected from vmware2-client-12. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046669] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-13: disconnected from vmware2-client-13. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046671] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-16: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046695] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-6: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.046710] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-14: disconnected from vmware2-client-14. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046717] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-15: disconnected from vmware2-client-15. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046735] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-7: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.046738] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-17: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046740] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-16: disconnected from vmware2-client-16. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046798] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-18: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046800] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-17: disconnected from vmware2-client-17. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046821] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-8: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.046824] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-19: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046833] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-18: disconnected from vmware2-client-18. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046851] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-19: disconnected from vmware2-client-19. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046852] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-20: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046866] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-9: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.046887] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-21: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046910] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-22: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046912] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-20: disconnected from vmware2-client-20. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046918] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-21: disconnected from vmware2-client-21. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046939] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-23: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046940] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-10: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.046949] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-22: disconnected from vmware2-client-22. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046963] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-24: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046977] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-23: disconnected from vmware2-client-23. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046990] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-25: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.046996] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-24: disconnected from vmware2-client-24. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.046991] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-11: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.047041] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-26: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047057] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-25: disconnected from vmware2-client-25. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047061] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-27: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047075] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-12: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.047088] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-26: disconnected from vmware2-client-26. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047092] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-28: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047105] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-27: disconnected from vmware2-client-27. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047118] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-13: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.047128] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-29: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047136] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-28: disconnected from vmware2-client-28. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047165] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-30: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047169] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-29: disconnected from vmware2-client-29. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047193] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-14: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.047197] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-31: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047204] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-30: disconnected from vmware2-client-30. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047232] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-32: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047237] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-31: disconnected from vmware2-client-31. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047261] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-15: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.047278] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-33: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047287] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-32: disconnected from vmware2-client-32. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047313] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-34: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047319] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-33: disconnected from vmware2-client-33. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047352] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-16: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.047353] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-35: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047358] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-34: disconnected from vmware2-client-34. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047380] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-36: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047387] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-35: disconnected from vmware2-client-35. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047401] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-17: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.047416] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-37: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047421] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-36: disconnected from vmware2-client-36. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047445] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-38: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047446] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-37: disconnected from vmware2-client-37. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047465] I [MSGID: 114021] [client.c:2365:notify] 15-vmware2-client-39: current graph is no longer active, destroying rpc_client 
[2017-03-17 13:57:44.047466] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-18: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 13:57:44.047481] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-38: disconnected from vmware2-client-38. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047498] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 15-vmware2-client-39: disconnected from vmware2-client-39. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 13:57:44.047511] E [MSGID: 108006] [afr-common.c:4409:afr_notify] 15-vmware2-replicate-19: All subvolumes are down. Going offline until atleast one of them comes back up.
[2017-03-17 14:00:01.660349] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-0; inode layout - 3435973824 - 3650722187 - 1; disk layout - 3709289934 - 3904515719 - 1
[2017-03-17 14:00:01.660399] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.660470] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-1; inode layout - 3650722188 - 3865470551 - 1; disk layout - 2928386790 - 3123612575 - 1
[2017-03-17 14:00:01.660512] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-2; inode layout - 3221225460 - 3435973823 - 1; disk layout - 3318838362 - 3514064147 - 1
[2017-03-17 14:00:01.660651] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-3; inode layout - 1932735276 - 2147483639 - 1; disk layout - 1952257860 - 2147483645 - 1
The message "I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001" repeated 3 times between [2017-03-17 14:00:01.660399] and [2017-03-17 14:00:01.660683]
[2017-03-17 14:00:01.660694] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-4; inode layout - 858993456 - 1073741819 - 1; disk layout - 780903144 - 976128929 - 1
[2017-03-17 14:00:01.660753] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.660791] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-5; inode layout - 1073741820 - 1288490183 - 1; disk layout - 976128930 - 1171354715 - 1
[2017-03-17 14:00:01.660824] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.660864] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-6; inode layout - 0 - 214748363 - 1; disk layout - 0 - 195225785 - 1
[2017-03-17 14:00:01.660883] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.660937] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-7; inode layout - 214748364 - 429496727 - 1; disk layout - 195225786 - 390451571 - 1
[2017-03-17 14:00:01.660960] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.660989] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-8; inode layout - 3006477096 - 3221225459 - 1; disk layout - 1366580502 - 1561806287 - 1
[2017-03-17 14:00:01.661020] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.661067] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-9; inode layout - 644245092 - 858993455 - 1; disk layout - 585677358 - 780903143 - 1
[2017-03-17 14:00:01.661083] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.661101] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-10; inode layout - 4080218916 - 4294967295 - 1; disk layout - 4099741506 - 4294967295 - 1
[2017-03-17 14:00:01.661119] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.661180] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-11; inode layout - 2576980368 - 2791728731 - 1; disk layout - 2537935218 - 2733161003 - 1
[2017-03-17 14:00:01.661195] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.661282] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-12; inode layout - 1717986912 - 1932735275 - 1; disk layout - 1757032074 - 1952257859 - 1
[2017-03-17 14:00:01.661315] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.661337] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-13; inode layout - 429496728 - 644245091 - 1; disk layout - 390451572 - 585677357 - 1
[2017-03-17 14:00:01.661369] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.661442] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-14; inode layout - 2147483640 - 2362232003 - 1; disk layout - 2147483646 - 2342709431 - 1
[2017-03-17 14:00:01.661470] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.661542] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-15; inode layout - 2362232004 - 2576980367 - 1; disk layout - 2342709432 - 2537935217 - 1
[2017-03-17 14:00:01.661556] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.661695] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-16; inode layout - 3865470552 - 4080218915 - 1; disk layout - 3904515720 - 4099741505 - 1
[2017-03-17 14:00:01.661727] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.661749] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-17; inode layout - 2791728732 - 3006477095 - 1; disk layout - 2733161004 - 2928386789 - 1
[2017-03-17 14:00:01.661809] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.661892] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-19; inode layout - 1503238548 - 1717986911 - 1; disk layout - 3123612576 - 3318838361 - 1
[2017-03-17 14:00:01.661907] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.661933] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-18; inode layout - 1288490184 - 1503238547 - 1; disk layout - 1561806288 - 1757032073 - 1
[2017-03-17 14:00:01.661970] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.662058] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-21; inode layout - 0 - 0 - 0; disk layout - 1171354716 - 1366580501 - 1
[2017-03-17 14:00:01.662079] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:00:01.662125] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-20: selecting local read_child vmware2-client-40
[2017-03-17 14:00:01.662142] I [MSGID: 109064] [dht-layout.c:837:dht_layout_dir_mismatch] 17-vmware2-dht: subvol: vmware2-replicate-20; inode layout - 0 - 0 - 0; disk layout - 3514064148 - 3709289933 - 1
[2017-03-17 14:00:01.662176] I [MSGID: 109018] [dht-common.c:1096:dht_revalidate_cbk] 17-vmware2-dht: Mismatching layouts for /, gfid = 00000000-0000-0000-0000-000000000001
[2017-03-17 14:01:03.607386] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-4: remote operation failed. Path: <gfid:53c138b2-86ed-42c0-8b16-b45aebfed3e4> (53c138b2-86ed-42c0-8b16-b45aebfed3e4) [No such file or directory]
[2017-03-17 14:01:03.607630] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-0: remote operation failed. Path: <gfid:058bcdef-7095-476b-9f7b-8c4c8cbcef82> (058bcdef-7095-476b-9f7b-8c4c8cbcef82) [No such file or directory]
[2017-03-17 14:01:03.608326] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-16: remote operation failed. Path: <gfid:efef7c02-2411-4b92-890e-c07b143aebb2> (efef7c02-2411-4b92-890e-c07b143aebb2) [No such file or directory]
[2017-03-17 14:01:03.608402] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-5: remote operation failed. Path: <gfid:53c138b2-86ed-42c0-8b16-b45aebfed3e4> (53c138b2-86ed-42c0-8b16-b45aebfed3e4) [No such file or directory]
[2017-03-17 14:01:03.609222] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-1: remote operation failed. Path: <gfid:058bcdef-7095-476b-9f7b-8c4c8cbcef82> (058bcdef-7095-476b-9f7b-8c4c8cbcef82) [No such file or directory]
[2017-03-17 14:01:03.609568] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-17: remote operation failed. Path: <gfid:efef7c02-2411-4b92-890e-c07b143aebb2> (efef7c02-2411-4b92-890e-c07b143aebb2) [No such file or directory]
[2017-03-17 14:01:11.924772] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.925049] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.925151] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.925446] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 0df52c51-fdc0-4514-b2d9-9e365a3fda40. (Possible split-brain)
[2017-03-17 14:01:11.925590] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.925877] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 0df52c51-fdc0-4514-b2d9-9e365a3fda40. (Possible split-brain)
[2017-03-17 14:01:11.926010] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.926098] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.926340] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.927566] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 0df52c51-fdc0-4514-b2d9-9e365a3fda40. (Possible split-brain)
[2017-03-17 14:01:11.927864] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.927967] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.928275] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 0df52c51-fdc0-4514-b2d9-9e365a3fda40. (Possible split-brain)
[2017-03-17 14:01:11.928526] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.928635] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.929080] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 0df52c51-fdc0-4514-b2d9-9e365a3fda40. (Possible split-brain)
[2017-03-17 14:01:11.929299] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.929724] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:11.936625] W [xdr-rpc.c:55:xdr_to_rpc_call] 17-rpc: failed to decode call msg
[2017-03-17 14:01:11.936648] W [rpc-clnt.c:717:rpc_clnt_handle_cbk] 17-vmware2-client-42: RPC call decoding failed
[2017-03-17 14:01:11.936922] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 17-vmware2-client-42: forced unwinding frame type(GlusterFS 3.3) op(READ(12)) called at 2017-03-17 14:01:11.929998 (xid=0xacf)
[2017-03-17 14:01:11.936939] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.940277] I [socket.c:3401:socket_submit_request] 17-vmware2-client-42: not connected (priv->connected = 0)
[2017-03-17 14:01:11.940295] W [rpc-clnt.c:1657:rpc_clnt_submit] 17-vmware2-client-42: failed to submit rpc-request (XID: 0xad5 Program: GlusterFS 3.3, ProgVers: 330, Proc: 25) to rpc-transport (vmware2-client-42)
[2017-03-17 14:01:11.940310] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.940314] W [rpc-clnt.c:1657:rpc_clnt_submit] 17-vmware2-client-42: failed to submit rpc-request (XID: 0xad6 Program: GlusterFS 3.3, ProgVers: 330, Proc: 12) to rpc-transport (vmware2-client-42)
[2017-03-17 14:01:11.940340] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.940340] W [rpc-clnt.c:1657:rpc_clnt_submit] 17-vmware2-client-42: failed to submit rpc-request (XID: 0xad7 Program: GlusterFS 3.3, ProgVers: 330, Proc: 12) to rpc-transport (vmware2-client-42)
[2017-03-17 14:01:11.940368] W [rpc-clnt.c:1657:rpc_clnt_submit] 17-vmware2-client-42: failed to submit rpc-request (XID: 0xad8 Program: GlusterFS 3.3, ProgVers: 330, Proc: 12) to rpc-transport (vmware2-client-42)
[2017-03-17 14:01:11.940393] W [rpc-clnt.c:1657:rpc_clnt_submit] 17-vmware2-client-42: failed to submit rpc-request (XID: 0xadb Program: GlusterFS 3.3, ProgVers: 330, Proc: 25) to rpc-transport (vmware2-client-42)
[2017-03-17 14:01:11.940408] W [rpc-clnt.c:1657:rpc_clnt_submit] 17-vmware2-client-42: failed to submit rpc-request (XID: 0xada Program: GlusterFS 3.3, ProgVers: 330, Proc: 25) to rpc-transport (vmware2-client-42)
The message "W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]" repeated 2 times between [2017-03-17 14:01:11.940340] and [2017-03-17 14:01:11.940401]
[2017-03-17 14:01:11.940421] W [rpc-clnt.c:1657:rpc_clnt_submit] 17-vmware2-client-42: failed to submit rpc-request (XID: 0xad9 Program: GlusterFS 3.3, ProgVers: 330, Proc: 12) to rpc-transport (vmware2-client-42)
[2017-03-17 14:01:11.940410] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.940443] W [rpc-clnt.c:1657:rpc_clnt_submit] 17-vmware2-client-42: failed to submit rpc-request (XID: 0xadc Program: GlusterFS 3.3, ProgVers: 330, Proc: 25) to rpc-transport (vmware2-client-42)
[2017-03-17 14:01:11.940419] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.940463] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.940457] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.940475] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 17-vmware2-client-42: forced unwinding frame type(GlusterFS 3.3) op(READ(12)) called at 2017-03-17 14:01:11.930661 (xid=0xad0)
[2017-03-17 14:01:11.940493] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.940504] W [rpc-clnt.c:1657:rpc_clnt_submit] 17-vmware2-client-42: failed to submit rpc-request (XID: 0xadd Program: GlusterFS 3.3, ProgVers: 330, Proc: 25) to rpc-transport (vmware2-client-42)
[2017-03-17 14:01:11.940517] W [rpc-clnt.c:1657:rpc_clnt_submit] 17-vmware2-client-42: failed to submit rpc-request (XID: 0xade Program: GlusterFS 3.3, ProgVers: 330, Proc: 25) to rpc-transport (vmware2-client-42)
[2017-03-17 14:01:11.940515] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.940677] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 17-vmware2-client-42: forced unwinding frame type(GlusterFS 3.3) op(LOOKUP(27)) called at 2017-03-17 14:01:11.931177 (xid=0xad1)
[2017-03-17 14:01:11.940525] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.940694] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-42: remote operation failed. Path: /.shard/de789e92-1bc2-430f-a1db-c118a3b70a27.3 (0df52c51-fdc0-4514-b2d9-9e365a3fda40) [Transport endpoint is not connected]
[2017-03-17 14:01:11.940849] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 17-vmware2-client-42: forced unwinding frame type(GlusterFS 3.3) op(READ(12)) called at 2017-03-17 14:01:11.932261 (xid=0xad2)
[2017-03-17 14:01:11.940864] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.941001] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 17-vmware2-client-42: forced unwinding frame type(GlusterFS 3.3) op(READ(12)) called at 2017-03-17 14:01:11.933784 (xid=0xad3)
[2017-03-17 14:01:11.942117] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 17-vmware2-client-42: forced unwinding frame type(GlusterFS 3.3) op(READ(12)) called at 2017-03-17 14:01:11.936310 (xid=0xad4)
The message "W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]" repeated 2 times between [2017-03-17 14:01:11.940864] and [2017-03-17 14:01:11.942131]
[2017-03-17 14:01:11.942171] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 17-vmware2-client-42: disconnected from vmware2-client-42. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 14:01:11.942250] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.943208] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-42: changing port to 49425 (from 0)
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-42: remote operation failed [Transport endpoint is not connected]" repeated 2 times between [2017-03-17 14:01:11.942250] and [2017-03-17 14:01:11.942544]
[2017-03-17 14:01:11.945837] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-42: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 14:01:11.946859] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-42: Connected to vmware2-client-42, attached to remote volume '/mnt/disk11/vmware2'.
[2017-03-17 14:01:11.946881] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-42: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 14:01:11.946974] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-42: Server lk version = 1
[2017-03-17 14:01:11.976056] W [MSGID: 114063] [client-callback.c:110:client_cbk_cache_invalidation] 17-vmware2-client-43: XDR decode of cache_invalidation failed. [Operation not permitted]
[2017-03-17 14:01:11.976101] W [xdr-rpc.c:55:xdr_to_rpc_call] 17-rpc: failed to decode call msg
[2017-03-17 14:01:11.976108] W [rpc-clnt.c:717:rpc_clnt_handle_cbk] 17-vmware2-client-43: RPC call decoding failed
[2017-03-17 14:01:11.976264] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 17-vmware2-client-43: forced unwinding frame type(GlusterFS 3.3) op(READ(12)) called at 2017-03-17 14:01:11.953532 (xid=0xae8)
[2017-03-17 14:01:11.976097] W [MSGID: 114063] [client-callback.c:110:client_cbk_cache_invalidation] 17-vmware2-client-43: XDR decode of cache_invalidation failed. [Operation not permitted]
[2017-03-17 14:01:11.976280] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-43: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.976870] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 17-vmware2-client-43: forced unwinding frame type(GlusterFS 3.3) op(LOOKUP(27)) called at 2017-03-17 14:01:11.953681 (xid=0xae9)
[2017-03-17 14:01:11.976889] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-43: remote operation failed. Path: <gfid:0df52c51-fdc0-4514-b2d9-9e365a3fda40> (0df52c51-fdc0-4514-b2d9-9e365a3fda40) [Transport endpoint is not connected]
[2017-03-17 14:01:11.977005] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 17-vmware2-client-43: forced unwinding frame type(GlusterFS 3.3) op(READ(12)) called at 2017-03-17 14:01:11.964963 (xid=0xaea)
[2017-03-17 14:01:11.977020] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-43: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.979547] I [socket.c:3401:socket_submit_request] 17-vmware2-client-43: not connected (priv->connected = 0)
[2017-03-17 14:01:11.979561] W [rpc-clnt.c:1657:rpc_clnt_submit] 17-vmware2-client-43: failed to submit rpc-request (XID: 0xaec Program: GlusterFS 3.3, ProgVers: 330, Proc: 25) to rpc-transport (vmware2-client-43)
[2017-03-17 14:01:11.979569] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-43: remote operation failed [Transport endpoint is not connected]
[2017-03-17 14:01:11.979988] E [rpc-clnt.c:365:saved_frames_unwind] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f1276a74162] (--> /lib64/libgfrpc.so.0(saved_frames_unwind+0x1de)[0x7f127683b84e] (--> /lib64/libgfrpc.so.0(saved_frames_destroy+0xe)[0x7f127683b95e] (--> /lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x84)[0x7f127683d0c4] (--> /lib64/libgfrpc.so.0(rpc_clnt_notify+0x94)[0x7f127683d954] ))))) 17-vmware2-client-43: forced unwinding frame type(GlusterFS 3.3) op(LOOKUP(27)) called at 2017-03-17 14:01:11.965124 (xid=0xaeb)
[2017-03-17 14:01:11.980007] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-43: remote operation failed. Path: <gfid:0df52c51-fdc0-4514-b2d9-9e365a3fda40> (0df52c51-fdc0-4514-b2d9-9e365a3fda40) [Transport endpoint is not connected]
[2017-03-17 14:01:11.980027] I [MSGID: 114018] [client.c:2280:client_rpc_notify] 17-vmware2-client-43: disconnected from vmware2-client-43. Client process will keep trying to connect to glusterd until brick's port is available
[2017-03-17 14:01:11.980259] I [rpc-clnt.c:1965:rpc_clnt_reconfig] 17-vmware2-client-43: changing port to 49395 (from 0)
[2017-03-17 14:01:11.983132] I [MSGID: 114057] [client-handshake.c:1440:select_server_supported_programs] 17-vmware2-client-43: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2017-03-17 14:01:11.987014] I [MSGID: 114046] [client-handshake.c:1216:client_setvolume_cbk] 17-vmware2-client-43: Connected to vmware2-client-43, attached to remote volume '/mnt/disk11/vmware2'.
[2017-03-17 14:01:11.987038] I [MSGID: 114047] [client-handshake.c:1227:client_setvolume_cbk] 17-vmware2-client-43: Server and Client lk-version numbers are not same, reopening the fds
[2017-03-17 14:01:11.987354] I [MSGID: 114035] [client-handshake.c:202:client_set_lk_version_cbk] 17-vmware2-client-43: Server lk version = 1
[2017-03-17 14:01:15.631755] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:15.632176] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:15.632220] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:15.632726] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid b9951b68-d1b1-451c-82f1-b5b41eff4571. (Possible split-brain)
[2017-03-17 14:01:15.633103] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:15.633233] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:15.634101] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid b9951b68-d1b1-451c-82f1-b5b41eff4571. (Possible split-brain)
[2017-03-17 14:01:15.634404] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:15.634461] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:15.634850] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid b9951b68-d1b1-451c-82f1-b5b41eff4571. (Possible split-brain)
[2017-03-17 14:01:15.635175] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:15.635310] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:18.050884] W [MSGID: 108019] [afr-lk-common.c:1082:is_blocking_locks_count_sufficient] 17-vmware2-replicate-4: Unable to obtain blocking inode lock on even one child for gfid:e1d96b33-702e-4825-820c-58664b37327f.
[2017-03-17 14:01:18.050921] I [MSGID: 108019] [afr-transaction.c:1478:afr_post_blocking_inodelk_cbk] 17-vmware2-replicate-4: Blocking inodelks failed.
[2017-03-17 14:01:18.052000] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:18.052231] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.098630] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]" repeated 3 times between [2017-03-17 14:01:19.098630] and [2017-03-17 14:01:19.098845]
[2017-03-17 14:01:19.099043] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.099115] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.099179] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.099185] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.099234] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.099249] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.099348] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.099379] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.099448] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.099458] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.099438] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.099499] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.099548] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.099630] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.099665] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]" repeated 5 times between [2017-03-17 14:01:19.099665] and [2017-03-17 14:01:19.100008]
[2017-03-17 14:01:19.100105] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.100156] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.100478] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.100755] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)" repeated 2 times between [2017-03-17 14:01:19.100755] and [2017-03-17 14:01:19.100959]
[2017-03-17 14:01:19.101033] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.101074] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.101105] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.101194] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.101310] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.101317] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.101358] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.101457] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.101498] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.101657] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:19.101657] and [2017-03-17 14:01:19.101771]
[2017-03-17 14:01:19.102644] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.102718] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.103150] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.103854] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.103880] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.104243] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.105038] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.105078] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.105433] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.105846] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.105908] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.105913] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.105960] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.105960] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.105973] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.105993] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]" repeated 5 times between [2017-03-17 14:01:19.105993] and [2017-03-17 14:01:19.106308]
[2017-03-17 14:01:19.106341] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.106377] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.106427] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.106463] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.106495] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.106525] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.106561] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.106600] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.106621] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]" repeated 5 times between [2017-03-17 14:01:19.106621] and [2017-03-17 14:01:19.106889]
[2017-03-17 14:01:19.106947] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.106919] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)" repeated 2 times between [2017-03-17 14:01:19.106919] and [2017-03-17 14:01:19.106927]
[2017-03-17 14:01:19.106937] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.106999] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.107105] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 3f6f0040-f3fb-4ddf-9bc6-ecd2d857bbc2. (Possible split-brain)
[2017-03-17 14:01:19.107144] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.107190] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.107334] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.107348] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.107402] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.107487] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.107566] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.107631] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.107702] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.107787] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.108134] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.108245] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.628853] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.629281] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.629424] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.630113] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 5e0b5b4e-66a4-4d67-9cef-adf6d9fd75cc. (Possible split-brain)
[2017-03-17 14:01:19.630177] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 5e0b5b4e-66a4-4d67-9cef-adf6d9fd75cc. (Possible split-brain)
[2017-03-17 14:01:19.630521] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.630589] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.630663] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.630706] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.631639] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 5e0b5b4e-66a4-4d67-9cef-adf6d9fd75cc. (Possible split-brain)
[2017-03-17 14:01:19.632315] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.632464] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 5e0b5b4e-66a4-4d67-9cef-adf6d9fd75cc. (Possible split-brain)
[2017-03-17 14:01:19.632830] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.632901] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 5e0b5b4e-66a4-4d67-9cef-adf6d9fd75cc. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 5e0b5b4e-66a4-4d67-9cef-adf6d9fd75cc. (Possible split-brain)" repeated 2 times between [2017-03-17 14:01:19.632901] and [2017-03-17 14:01:19.634031]
[2017-03-17 14:01:19.634355] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.634417] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.634455] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.634480] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.635349] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.636529] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.636579] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 5e0b5b4e-66a4-4d67-9cef-adf6d9fd75cc. (Possible split-brain)
[2017-03-17 14:01:19.636854] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.638756] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.638839] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 5e0b5b4e-66a4-4d67-9cef-adf6d9fd75cc. (Possible split-brain)
[2017-03-17 14:01:19.638878] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.639237] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.640611] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:19.641850] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.123471] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.123958] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.125735] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.126064] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.126699] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.127550] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.128663] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.128981] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.134401] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.134767] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.134776] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.135105] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.135113] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.135464] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.135471] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.135782] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.135788] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.136404] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.136482] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 38acb790-16a2-42fc-9ec5-d67c0341e441. (Possible split-brain)
[2017-03-17 14:01:23.136873] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.137228] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.137248] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 38acb790-16a2-42fc-9ec5-d67c0341e441. (Possible split-brain)
[2017-03-17 14:01:23.137512] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.138329] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.138371] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 38acb790-16a2-42fc-9ec5-d67c0341e441. (Possible split-brain)
[2017-03-17 14:01:23.138588] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.139308] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.139354] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 38acb790-16a2-42fc-9ec5-d67c0341e441. (Possible split-brain)
[2017-03-17 14:01:23.139615] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.140362] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.140441] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 38acb790-16a2-42fc-9ec5-d67c0341e441. (Possible split-brain)
[2017-03-17 14:01:23.140726] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.141544] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.141608] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 38acb790-16a2-42fc-9ec5-d67c0341e441. (Possible split-brain)
[2017-03-17 14:01:23.141900] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.142721] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.142775] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 38acb790-16a2-42fc-9ec5-d67c0341e441. (Possible split-brain)
[2017-03-17 14:01:23.143046] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.143905] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:23.143976] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]" repeated 4 times between [2017-03-17 14:01:23.143905] and [2017-03-17 14:01:23.143963]
[2017-03-17 14:01:23.143976] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 38acb790-16a2-42fc-9ec5-d67c0341e441. (Possible split-brain)
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:23.143976] and [2017-03-17 14:01:23.144324]
[2017-03-17 14:01:23.144335] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.822323] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:24.822323] and [2017-03-17 14:01:24.822697]
[2017-03-17 14:01:24.822814] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.822925] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.823119] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)" repeated 3 times between [2017-03-17 14:01:24.823119] and [2017-03-17 14:01:24.823355]
[2017-03-17 14:01:24.823421] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.823473] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.823484] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.824004] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.824100] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.824107] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.824372] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.825418] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.825432] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.825422] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.825486] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.825640] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.825671] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.825673] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.825761] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.825809] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.825829] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.825885] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.825923] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.825937] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826016] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826051] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826074] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.826094] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826125] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826185] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826214] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826256] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826282] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826356] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826388] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826432] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826442] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.826459] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826673] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.826729] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.827213] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.827368] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.827554] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.827958] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.829322] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.830607] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.830800] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.831338] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.831335] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.831515] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.831942] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid f9558815-7c77-480b-8800-c1d616131394. (Possible split-brain)
[2017-03-17 14:01:24.832695] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.832775] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.833480] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:24.833549] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.570308] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.570744] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.570852] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.571725] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a51f8d06-de4b-4c0e-8a66-50b9827237f8. (Possible split-brain)
[2017-03-17 14:01:26.571824] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a51f8d06-de4b-4c0e-8a66-50b9827237f8. (Possible split-brain)
[2017-03-17 14:01:26.572042] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.572149] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.572221] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.573496] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.573533] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a51f8d06-de4b-4c0e-8a66-50b9827237f8. (Possible split-brain)
[2017-03-17 14:01:26.573567] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a51f8d06-de4b-4c0e-8a66-50b9827237f8. (Possible split-brain)
[2017-03-17 14:01:26.573996] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.573953] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.574108] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.574115] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.574533] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a51f8d06-de4b-4c0e-8a66-50b9827237f8. (Possible split-brain)
[2017-03-17 14:01:26.575505] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.575541] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.576082] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a51f8d06-de4b-4c0e-8a66-50b9827237f8. (Possible split-brain)
[2017-03-17 14:01:26.576547] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.576602] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.577554] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a51f8d06-de4b-4c0e-8a66-50b9827237f8. (Possible split-brain)
[2017-03-17 14:01:26.577873] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a51f8d06-de4b-4c0e-8a66-50b9827237f8. (Possible split-brain)
[2017-03-17 14:01:26.578015] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.578102] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.578441] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.578446] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a51f8d06-de4b-4c0e-8a66-50b9827237f8. (Possible split-brain)
[2017-03-17 14:01:26.578462] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.578716] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:26.578845] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.129617] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.129823] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.130065] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.130493] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.130948] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.131158] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.131177] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.131180] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.132041] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.132209] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.132216] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.132281] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.132670] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.132890] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.133531] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.133537] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.133600] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.133657] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.133687] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.134176] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.134285] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.134561] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.134866] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.134992] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.135116] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.135285] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.135368] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.135447] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.136441] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.136566] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.137097] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.137277] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.137702] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.137961] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.138107] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.138276] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.138288] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.138799] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.138937] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.139113] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.139229] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.139294] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.139322] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.140183] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid c1880af3-4df9-43d7-bb97-ec136916171f. (Possible split-brain)
[2017-03-17 14:01:27.140411] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.140677] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.140850] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:27.140992] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.891461] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.891865] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.892019] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.892079] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.892497] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.892564] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.892557] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.892595] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.892690] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.892742] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.893225] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.893254] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.893305] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.893338] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.893339] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.893384] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.893775] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.894143] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.894159] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:28.894159] and [2017-03-17 14:01:28.894204]
[2017-03-17 14:01:28.894256] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.894230] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.894288] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.894350] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.894377] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.894444] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.894477] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.894578] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:28.894578] and [2017-03-17 14:01:28.894906]
[2017-03-17 14:01:28.894911] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.894906] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.894957] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.894954] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.894987] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.894984] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]" repeated 4 times between [2017-03-17 14:01:28.894987] and [2017-03-17 14:01:28.895361]
[2017-03-17 14:01:28.895374] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.896322] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.896914] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.897018] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.897369] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.898860] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.898924] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.899360] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.900538] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.900585] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.900597] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.901941] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.902314] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.902342] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.902735] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.904150] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.904206] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.904847] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.905858] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.905909] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.906369] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.907600] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.907651] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.908776] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.909314] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.909312] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.909736] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.911103] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.911136] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.911627] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.912724] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.912809] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.913271] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.914369] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.914424] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.914956] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.915965] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.916008] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.916012] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.916614] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.917992] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 9e4cd8c6-dccd-470a-b1a4-3fcd82ee69ca. (Possible split-brain)
[2017-03-17 14:01:28.918121] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]" repeated 6 times between [2017-03-17 14:01:28.918121] and [2017-03-17 14:01:28.918458]
[2017-03-17 14:01:28.918488] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:28.918540] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]" repeated 5 times between [2017-03-17 14:01:28.918540] and [2017-03-17 14:01:28.922289]
[2017-03-17 14:01:30.195298] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:30.195298] and [2017-03-17 14:01:30.195450]
[2017-03-17 14:01:30.195672] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.195790] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.196092] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.196101] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 171a7070-f4a6-4256-8f9d-82b18dffc316. (Possible split-brain)
[2017-03-17 14:01:30.196122] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.196285] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.196502] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.196535] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 171a7070-f4a6-4256-8f9d-82b18dffc316. (Possible split-brain)
[2017-03-17 14:01:30.196898] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.197181] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.197391] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 171a7070-f4a6-4256-8f9d-82b18dffc316. (Possible split-brain)
[2017-03-17 14:01:30.197806] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.198388] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.198418] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.198431] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 171a7070-f4a6-4256-8f9d-82b18dffc316. (Possible split-brain)
[2017-03-17 14:01:30.198780] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.198935] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 171a7070-f4a6-4256-8f9d-82b18dffc316. (Possible split-brain)
[2017-03-17 14:01:30.199461] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.199737] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 171a7070-f4a6-4256-8f9d-82b18dffc316. (Possible split-brain)
[2017-03-17 14:01:30.199798] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:30.199798] and [2017-03-17 14:01:30.200057]
[2017-03-17 14:01:30.200286] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.201467] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 171a7070-f4a6-4256-8f9d-82b18dffc316. (Possible split-brain)
[2017-03-17 14:01:30.201787] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.202166] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.202385] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.839540] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.840023] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.840224] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.841226] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.841655] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.841749] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.841993] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.842440] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.842485] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.843818] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.844203] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.844240] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.844846] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.845310] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.845317] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.845817] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.846250] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.846304] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.847220] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.847797] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.847852] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.848807] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.849186] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.849538] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.849598] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.850238] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.851427] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.851435] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.851645] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.851838] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.851954] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.852002] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.852058] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.853751] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.854148] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.854191] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.854259] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.854533] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.854684] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.856082] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.856389] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.856444] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.856489] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.856963] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.857955] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.858317] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.858704] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.858705] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.858788] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.859006] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.860422] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.861017] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.861033] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.861103] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.861190] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.861422] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.863516] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.863531] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.863785] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.863793] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.863948] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.864141] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.864228] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.865758] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.865996] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.866247] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.866276] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.866299] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.867298] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.867555] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.867697] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.867723] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.867755] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.868172] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.869023] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.869062] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.869390] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.869502] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.870286] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.870675] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.870739] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.872271] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.872558] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.872695] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.872699] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.872968] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.873125] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.874281] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.874578] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.874583] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.874742] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.874843] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.874894] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.876976] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 60306f66-af42-4b39-b640-2f5a01120888. (Possible split-brain)
[2017-03-17 14:01:30.877328] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:30.877332] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:31.143135] W [MSGID: 108019] [afr-lk-common.c:1082:is_blocking_locks_count_sufficient] 17-vmware2-replicate-7: Unable to obtain blocking inode lock on even one child for gfid:5a0359ab-c4c9-4e5e-a966-4795203aec7f.
[2017-03-17 14:01:31.143188] I [MSGID: 108019] [afr-transaction.c:1478:afr_post_blocking_inodelk_cbk] 17-vmware2-replicate-7: Blocking inodelks failed.
[2017-03-17 14:01:31.143497] W [MSGID: 108019] [afr-lk-common.c:1082:is_blocking_locks_count_sufficient] 17-vmware2-replicate-8: Unable to obtain blocking inode lock on even one child for gfid:77cb1e6a-6b47-411c-bdbb-bfddfc9a3cf5.
[2017-03-17 14:01:31.143513] I [MSGID: 108019] [afr-transaction.c:1478:afr_post_blocking_inodelk_cbk] 17-vmware2-replicate-8: Blocking inodelks failed.
[2017-03-17 14:01:31.144212] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-15: remote operation failed [No such file or directory]
[2017-03-17 14:01:31.144323] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-14: remote operation failed [No such file or directory]
[2017-03-17 14:01:31.145182] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:31.145266] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:33.117263] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:33.117667] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:33.117745] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:33.118412] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid d597e739-7f63-44ab-8c61-ef7ccd8cfe62. (Possible split-brain)
[2017-03-17 14:01:33.118782] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:33.118831] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:33.121089] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid d597e739-7f63-44ab-8c61-ef7ccd8cfe62. (Possible split-brain)
[2017-03-17 14:01:33.121477] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:33.121499] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:33.121806] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid d597e739-7f63-44ab-8c61-ef7ccd8cfe62. (Possible split-brain)
[2017-03-17 14:01:33.122189] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:33.122214] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.866477] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.866971] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.867030] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.867533] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid d3f5eaeb-8641-4c3f-94c3-9142323d2164. (Possible split-brain)
[2017-03-17 14:01:34.867530] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.867952] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.867999] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.868076] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid d3f5eaeb-8641-4c3f-94c3-9142323d2164. (Possible split-brain)
[2017-03-17 14:01:34.868082] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.868148] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.869176] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.869474] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid d3f5eaeb-8641-4c3f-94c3-9142323d2164. (Possible split-brain)
[2017-03-17 14:01:34.869661] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.870089] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid d3f5eaeb-8641-4c3f-94c3-9142323d2164. (Possible split-brain)
[2017-03-17 14:01:34.872256] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.872379] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.872419] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.872412] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid d3f5eaeb-8641-4c3f-94c3-9142323d2164. (Possible split-brain)
[2017-03-17 14:01:34.872519] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.873010] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.873003] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid d3f5eaeb-8641-4c3f-94c3-9142323d2164. (Possible split-brain)
[2017-03-17 14:01:34.873458] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.873602] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:34.874012] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.990125] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.990672] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.990755] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.991299] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.991352] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:37.991413] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:37.991730] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.991808] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.992110] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.992288] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:37.993027] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:37.993057] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.993122] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:37.993133] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.993496] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:37.993496] and [2017-03-17 14:01:37.995126]
[2017-03-17 14:01:37.995264] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:37.995324] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.995339] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:37.995341] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:37.995398] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.996755] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.996846] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:37.997631] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.997797] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:37.998452] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.998534] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.998963] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)" repeated 2 times between [2017-03-17 14:01:37.998963] and [2017-03-17 14:01:37.998969]
[2017-03-17 14:01:37.999186] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.999259] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.999336] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.999720] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.999781] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:37.999835] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.999903] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:37.999960] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.000084] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.001181] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:38.001222] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.002829] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.002876] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.003344] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:38.003499] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.004490] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.007323] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.007364] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.008216] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.008612] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:38.009046] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.009199] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 04d8622a-e778-48ba-ae59-bd8b0a3ddf6b. (Possible split-brain)
[2017-03-17 14:01:38.010141] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.010914] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.012473] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:38.012517] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]" repeated 3 times between [2017-03-17 14:01:38.012517] and [2017-03-17 14:01:38.014878]
[2017-03-17 14:01:39.203729] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.204671] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.204829] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.205345] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.205511] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.205517] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.205712] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.205984] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.207043] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.207057] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.207205] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.208008] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.208060] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.208134] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.208452] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.208485] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.208492] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.208533] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.208563] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.209560] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.210178] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.210882] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.210889] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.211098] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.211436] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.213055] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.213089] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.213168] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.213222] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.213264] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.213308] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.213355] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.213992] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.215394] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.215863] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.217341] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.217950] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid e32452cb-7d8d-4fb1-a529-99784caa1b6a. (Possible split-brain)
[2017-03-17 14:01:39.218031] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.218746] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.220017] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.221968] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:39.222163] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.348100] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.348693] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.348756] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.349048] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.349407] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.349473] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.349546] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.349675] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.350003] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.350146] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.350133] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.350137] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.350215] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.350791] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.351285] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.351348] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.351421] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.351532] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.351702] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.351772] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.352178] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.352349] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.352581] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.352643] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.352755] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.352803] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.352821] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)" repeated 2 times between [2017-03-17 14:01:40.352821] and [2017-03-17 14:01:40.354253]
[2017-03-17 14:01:40.354387] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.355024] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.355136] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.355125] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.355551] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.355629] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.358163] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.358258] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.359151] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.359192] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.359578] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.360099] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.360655] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.361084] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.361234] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.361322] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.361363] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.362060] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.363151] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.363914] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 31226640-8c7f-4169-bc7e-8e688d106c13. (Possible split-brain)
[2017-03-17 14:01:40.366271] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.366368] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-11: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.366907] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-10: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:40.366907] and [2017-03-17 14:01:40.368123]
[2017-03-17 14:01:40.977577] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]" repeated 3 times between [2017-03-17 14:01:40.977577] and [2017-03-17 14:01:40.977952]
[2017-03-17 14:01:40.978052] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.978136] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.978368] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:40.978368] and [2017-03-17 14:01:40.978450]
[2017-03-17 14:01:40.980601] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.980653] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)" repeated 3 times between [2017-03-17 14:01:40.980653] and [2017-03-17 14:01:40.981535]
[2017-03-17 14:01:40.982064] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.982297] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)" repeated 3 times between [2017-03-17 14:01:40.982297] and [2017-03-17 14:01:40.982517]
[2017-03-17 14:01:40.983027] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.983106] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.983099] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.983326] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.983624] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.984239] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.985004] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.985027] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.985065] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.985472] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.986503] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.986510] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.986566] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.986621] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.986650] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.987020] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.987368] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]" repeated 8 times between [2017-03-17 14:01:40.987368] and [2017-03-17 14:01:40.987822]
[2017-03-17 14:01:40.987922] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.987905] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.988349] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.988966] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.989005] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.989131] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.989185] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.989236] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.989294] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.989372] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.989524] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]" repeated 3 times between [2017-03-17 14:01:40.989524] and [2017-03-17 14:01:40.989899]
[2017-03-17 14:01:40.990234] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.990228] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.990586] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.990623] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.990682] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.991059] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.991911] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.991925] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.991963] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.992307] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.993228] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.993268] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.993718] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.994493] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.994540] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.994959] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.995716] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.995761] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.995790] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.995845] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.996201] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.996260] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.996270] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.996311] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.996342] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.996739] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.998013] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 50049553-9e12-4eb1-82ce-597d7c89364f. (Possible split-brain)
[2017-03-17 14:01:40.998414] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.998724] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:40.998748] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]" repeated 8 times between [2017-03-17 14:01:40.998748] and [2017-03-17 14:01:41.003938]
[2017-03-17 14:01:43.502100] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]" repeated 4 times between [2017-03-17 14:01:43.502100] and [2017-03-17 14:01:43.502734]
[2017-03-17 14:01:43.502748] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.502898] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.503387] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.503421] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.503428] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.503959] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.504840] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.505415] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.505426] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.505453] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.505480] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.505697] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.506500] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.506951] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.507774] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.507841] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.507858] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.508053] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.508091] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.508299] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.509081] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.509133] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.510379] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.510475] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.510623] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.510614] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.510686] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.510899] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.512345] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.512874] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.512893] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.512955] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.512982] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.513005] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.513382] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.513408] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.513415] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.513941] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.513976] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.514155] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.514202] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.514355] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.515324] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.515907] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.515948] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.516332] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.516505] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.517062] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.517153] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.517851] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.517855] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.518036] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.518806] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.518845] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.518993] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.518989] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]" repeated 3 times between [2017-03-17 14:01:43.518993] and [2017-03-17 14:01:43.519384]
[2017-03-17 14:01:43.519430] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.519412] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.519925] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.519968] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.520019] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.520068] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.520801] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.521849] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.522071] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.522293] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.522323] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:43.522323] and [2017-03-17 14:01:43.522559]
[2017-03-17 14:01:43.522996] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.523252] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.524091] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.524312] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 751a09d8-21a9-4e90-a511-5addd4ae4c1a. (Possible split-brain)
[2017-03-17 14:01:43.524737] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.524978] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.525235] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.525597] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.525678] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:43.526108] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]" repeated 3 times between [2017-03-17 14:01:43.526108] and [2017-03-17 14:01:43.529070]
[2017-03-17 14:01:47.573859] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]" repeated 3 times between [2017-03-17 14:01:47.573859] and [2017-03-17 14:01:47.574483]
[2017-03-17 14:01:47.574535] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.574579] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.574658] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.574697] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.575755] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.575786] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.575816] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.575872] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.576319] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.576313] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.576435] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.576478] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)" repeated 2 times between [2017-03-17 14:01:47.576478] and [2017-03-17 14:01:47.576507]
[2017-03-17 14:01:47.576576] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.576656] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.576751] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.576756] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.576905] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.576916] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.576961] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.577104] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.577154] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.577503] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]" repeated 9 times between [2017-03-17 14:01:47.577503] and [2017-03-17 14:01:47.577623]
[2017-03-17 14:01:47.578350] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.578528] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.578812] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.578958] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.579212] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.579582] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.579634] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.579831] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.579953] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.579984] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.579983] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.580057] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.580075] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.580146] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.580241] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.580250] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)" repeated 2 times between [2017-03-17 14:01:47.580250] and [2017-03-17 14:01:47.580292]
[2017-03-17 14:01:47.580333] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.580314] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.580432] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.580539] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.580597] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.580695] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.580722] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.580756] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.580862] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]" repeated 4 times between [2017-03-17 14:01:47.580862] and [2017-03-17 14:01:47.581537]
[2017-03-17 14:01:47.582169] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.582150] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.582282] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.582306] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.582781] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.582907] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.584232] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.584215] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.584686] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.584694] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.584692] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.584747] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.585065] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.586568] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.586572] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 1e5369e5-c610-4f20-9315-c0bd2eae856b. (Possible split-brain)
[2017-03-17 14:01:47.586624] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:01:47.586624] and [2017-03-17 14:01:47.586708]
[2017-03-17 14:01:47.587039] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:01:47.587075] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]" repeated 8 times between [2017-03-17 14:01:47.587075] and [2017-03-17 14:01:47.590591]
[2017-03-17 14:01:51.092626] W [MSGID: 108019] [afr-lk-common.c:1082:is_blocking_locks_count_sufficient] 17-vmware2-replicate-17: Unable to obtain blocking inode lock on even one child for gfid:524a67c2-2396-454c-b8b6-e832c9ca6b66.
[2017-03-17 14:01:51.092689] I [MSGID: 108019] [afr-transaction.c:1478:afr_post_blocking_inodelk_cbk] 17-vmware2-replicate-17: Blocking inodelks failed.
[2017-03-17 14:01:51.093909] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:01:51.093992] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:01:51.133283] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:51.133702] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:51.133749] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:51.134133] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid 43dcdd29-1fcd-46df-b277-f11e7390a046. (Possible split-brain)
[2017-03-17 14:01:51.134535] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-16: remote operation failed [No such file or directory]
[2017-03-17 14:01:51.134595] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-17: remote operation failed [No such file or directory]
[2017-03-17 14:01:55.965391] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:55.965884] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:55.965951] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:01:55.966588] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-4: Unreadable subvolume -1 found with event generation 2 for gfid 591be5a5-8b3d-40d9-8882-9fe9087ea367. (Possible split-brain)
[2017-03-17 14:01:55.966993] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-8: remote operation failed [No such file or directory]
[2017-03-17 14:01:55.967102] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-9: remote operation failed [No such file or directory]
[2017-03-17 14:02:36.073940] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:02:36.074481] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:02:36.074646] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:02:36.076161] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid a87dd2aa-0f07-403d-ba7b-1644920489cc. (Possible split-brain)
[2017-03-17 14:02:36.232316] W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:02:36.232710] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:02:36.232879] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:02:36.233416] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-2: Unreadable subvolume -1 found with event generation 2 for gfid 27066740-67e4-408b-a26f-d92539d71487. (Possible split-brain)
[2017-03-17 14:02:36.233934] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-4: remote operation failed [No such file or directory]
[2017-03-17 14:02:36.233954] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-5: remote operation failed [No such file or directory]
[2017-03-17 14:02:55.189782] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 14:02:55.190952] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-2: selecting local read_child vmware2-client-4
[2017-03-17 14:02:55.191117] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-4: selecting local read_child vmware2-client-8
[2017-03-17 14:02:55.191666] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 14:02:55.192154] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-20: selecting local read_child vmware2-client-40
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]" repeated 2 times between [2017-03-17 14:02:36.074646] and [2017-03-17 14:02:58.020328]
[2017-03-17 14:02:58.020372] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 69133ad3-2cd1-45ca-ad4a-7d76c81b6ea7. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 69133ad3-2cd1-45ca-ad4a-7d76c81b6ea7. (Possible split-brain)" repeated 3 times between [2017-03-17 14:02:58.020372] and [2017-03-17 14:02:58.020817]
[2017-03-17 14:02:58.020828] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:02:58.021941] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 69133ad3-2cd1-45ca-ad4a-7d76c81b6ea7. (Possible split-brain)
[2017-03-17 14:02:58.022154] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 69133ad3-2cd1-45ca-ad4a-7d76c81b6ea7. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 69133ad3-2cd1-45ca-ad4a-7d76c81b6ea7. (Possible split-brain)" repeated 3 times between [2017-03-17 14:02:58.021941] and [2017-03-17 14:02:58.022134]
[2017-03-17 14:02:58.022149] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
The message "W [MSGID: 114031] [client-rpc-fops.c:2997:client3_3_readv_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]" repeated 5 times between [2017-03-17 14:02:36.073940] and [2017-03-17 14:02:58.022279]
[2017-03-17 14:02:58.022519] W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
The message "W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 69133ad3-2cd1-45ca-ad4a-7d76c81b6ea7. (Possible split-brain)" repeated 18 times between [2017-03-17 14:02:58.022154] and [2017-03-17 14:02:58.047604]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]" repeated 28 times between [2017-03-17 14:02:58.022519] and [2017-03-17 14:02:58.048066]
The message "W [MSGID: 114031] [client-rpc-fops.c:1494:client3_3_fstat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]" repeated 33 times between [2017-03-17 14:02:36.074481] and [2017-03-17 14:02:58.053178]
[2017-03-17 14:04:26.305014] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid a68ce411-e381-46a3-93cd-d2af6a7c3532. (Possible split-brain)
[2017-03-17 14:04:26.308152] E [shard.c:426:shard_modify_size_and_block_count] (-->/usr/lib64/glusterfs/3.8.10/xlator/cluster/distribute.so(+0x60ed7) [0x7f1268254ed7] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xb6f0) [0x7f125bde96f0] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xaf5d) [0x7f125bde8f5d] ) 17-vmware2-shard: Failed to get trusted.glusterfs.shard.file-size for a68ce411-e381-46a3-93cd-d2af6a7c3532
[2017-03-17 14:04:26.338607] I [MSGID: 114024] [client-helpers.c:99:this_fd_set_ctx] 17-vmware2-client-6: /New Virtual Machine 5/New Virtual Machine 5.vmx (a68ce411-e381-46a3-93cd-d2af6a7c3532): trying duplicate remote fd set. 
[2017-03-17 14:04:26.338719] I [MSGID: 114024] [client-helpers.c:99:this_fd_set_ctx] 17-vmware2-client-7: /New Virtual Machine 5/New Virtual Machine 5.vmx (a68ce411-e381-46a3-93cd-d2af6a7c3532): trying duplicate remote fd set. 
[2017-03-17 14:04:26.339423] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-13/cache=vmware2-replicate-13) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-3)
[2017-03-17 14:04:26.348622] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-7: remote operation failed [No such file or directory]
[2017-03-17 14:04:26.349076] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-7: remote operation failed. Path: <gfid:a68ce411-e381-46a3-93cd-d2af6a7c3532> (a68ce411-e381-46a3-93cd-d2af6a7c3532) [No such file or directory]
[2017-03-17 14:04:26.349114] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-6: remote operation failed. Path: <gfid:a68ce411-e381-46a3-93cd-d2af6a7c3532> (a68ce411-e381-46a3-93cd-d2af6a7c3532) [No such file or directory]
[2017-03-17 14:04:26.349840] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-3: Unreadable subvolume -1 found with event generation 2 for gfid a68ce411-e381-46a3-93cd-d2af6a7c3532. (Possible split-brain)
[2017-03-17 14:04:26.350316] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-6: remote operation failed. Path: <gfid:a68ce411-e381-46a3-93cd-d2af6a7c3532> (a68ce411-e381-46a3-93cd-d2af6a7c3532) [No such file or directory]
[2017-03-17 14:04:26.350507] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-7: remote operation failed. Path: <gfid:a68ce411-e381-46a3-93cd-d2af6a7c3532> (a68ce411-e381-46a3-93cd-d2af6a7c3532) [No such file or directory]
[2017-03-17 14:04:26.351482] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-6: remote operation failed. Path: <gfid:a68ce411-e381-46a3-93cd-d2af6a7c3532> (a68ce411-e381-46a3-93cd-d2af6a7c3532) [No such file or directory]
[2017-03-17 14:04:26.351640] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-7: remote operation failed. Path: <gfid:a68ce411-e381-46a3-93cd-d2af6a7c3532> (a68ce411-e381-46a3-93cd-d2af6a7c3532) [No such file or directory]
[2017-03-17 14:04:26.353444] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:a68ce411-e381-46a3-93cd-d2af6a7c3532>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.353528] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: a68ce411-e381-46a3-93cd-d2af6a7c3532 [Stale file handle]
[2017-03-17 14:04:26.357993] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-11: Unreadable subvolume -1 found with event generation 2 for gfid 9cde660f-a4be-4169-bbd0-02577179adb5. (Possible split-brain)
[2017-03-17 14:04:26.361898] E [shard.c:426:shard_modify_size_and_block_count] (-->/usr/lib64/glusterfs/3.8.10/xlator/cluster/distribute.so(+0x60ed7) [0x7f1268254ed7] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xb6f0) [0x7f125bde96f0] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xaf5d) [0x7f125bde8f5d] ) 17-vmware2-shard: Failed to get trusted.glusterfs.shard.file-size for 9cde660f-a4be-4169-bbd0-02577179adb5
[2017-03-17 14:04:26.363738] I [MSGID: 114024] [client-helpers.c:99:this_fd_set_ctx] 17-vmware2-client-36: /New Virtual Machine 5/New Virtual Machine 5.nvram (9cde660f-a4be-4169-bbd0-02577179adb5): trying duplicate remote fd set. 
[2017-03-17 14:04:26.363743] I [MSGID: 114024] [client-helpers.c:99:this_fd_set_ctx] 17-vmware2-client-37: /New Virtual Machine 5/New Virtual Machine 5.nvram (9cde660f-a4be-4169-bbd0-02577179adb5): trying duplicate remote fd set. 
[2017-03-17 14:04:26.375958] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 47a325ab-6d99-43b1-998a-3f77b63a18c3. (Possible split-brain)
[2017-03-17 14:04:26.378453] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:47a325ab-6d99-43b1-998a-3f77b63a18c3>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.378519] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 47a325ab-6d99-43b1-998a-3f77b63a18c3 [Stale file handle]
[2017-03-17 14:04:26.442741] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-12: Unreadable subvolume -1 found with event generation 2 for gfid 53c138b2-86ed-42c0-8b16-b45aebfed3e4. (Possible split-brain)
[2017-03-17 14:04:26.445439] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:53c138b2-86ed-42c0-8b16-b45aebfed3e4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.445498] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 53c138b2-86ed-42c0-8b16-b45aebfed3e4 [Stale file handle]
[2017-03-17 14:04:26.459511] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid f33e33e3-a52d-44f9-b0bc-481340e1e4e5. (Possible split-brain)
[2017-03-17 14:04:26.462735] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:f33e33e3-a52d-44f9-b0bc-481340e1e4e5>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.462806] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: f33e33e3-a52d-44f9-b0bc-481340e1e4e5 [Stale file handle]
[2017-03-17 14:04:26.466414] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-12: Unreadable subvolume -1 found with event generation 2 for gfid 6e9f46c3-c577-4fd6-acad-50dcf793951e. (Possible split-brain)
[2017-03-17 14:04:26.468984] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:6e9f46c3-c577-4fd6-acad-50dcf793951e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.469055] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 6e9f46c3-c577-4fd6-acad-50dcf793951e [Stale file handle]
[2017-03-17 14:04:26.591798] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-13/cache=vmware2-replicate-13) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-13)
[2017-03-17 14:04:26.599158] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-27: remote operation failed [No such file or directory]
[2017-03-17 14:04:26.599637] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:9ebd4eab-6d58-4e34-b8fc-63010e911f3d> (9ebd4eab-6d58-4e34-b8fc-63010e911f3d) [No such file or directory]
[2017-03-17 14:04:26.599782] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:9ebd4eab-6d58-4e34-b8fc-63010e911f3d> (9ebd4eab-6d58-4e34-b8fc-63010e911f3d) [No such file or directory]
[2017-03-17 14:04:26.600191] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 9ebd4eab-6d58-4e34-b8fc-63010e911f3d. (Possible split-brain)
[2017-03-17 14:04:26.600640] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:9ebd4eab-6d58-4e34-b8fc-63010e911f3d> (9ebd4eab-6d58-4e34-b8fc-63010e911f3d) [No such file or directory]
[2017-03-17 14:04:26.600724] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:9ebd4eab-6d58-4e34-b8fc-63010e911f3d> (9ebd4eab-6d58-4e34-b8fc-63010e911f3d) [No such file or directory]
[2017-03-17 14:04:26.602309] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:9ebd4eab-6d58-4e34-b8fc-63010e911f3d> (9ebd4eab-6d58-4e34-b8fc-63010e911f3d) [No such file or directory]
[2017-03-17 14:04:26.602359] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:9ebd4eab-6d58-4e34-b8fc-63010e911f3d> (9ebd4eab-6d58-4e34-b8fc-63010e911f3d) [No such file or directory]
[2017-03-17 14:04:26.602993] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:9ebd4eab-6d58-4e34-b8fc-63010e911f3d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.603048] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 9ebd4eab-6d58-4e34-b8fc-63010e911f3d [Stale file handle]
[2017-03-17 14:04:26.608557] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 20fa24f7-c54a-4d39-88e0-6355abdcd91a. (Possible split-brain)
[2017-03-17 14:04:26.611118] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:20fa24f7-c54a-4d39-88e0-6355abdcd91a>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.611194] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 20fa24f7-c54a-4d39-88e0-6355abdcd91a [Stale file handle]
[2017-03-17 14:04:26.629035] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-11: Unreadable subvolume -1 found with event generation 2 for gfid 34ca2dee-3fc9-49f2-b39a-53c045f97b89. (Possible split-brain)
[2017-03-17 14:04:26.632069] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:34ca2dee-3fc9-49f2-b39a-53c045f97b89>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.632131] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 34ca2dee-3fc9-49f2-b39a-53c045f97b89 [Stale file handle]
[2017-03-17 14:04:26.636176] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 50aee93b-f444-4b3b-915d-c010284c4be4. (Possible split-brain)
[2017-03-17 14:04:26.639238] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:50aee93b-f444-4b3b-915d-c010284c4be4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.639288] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 50aee93b-f444-4b3b-915d-c010284c4be4 [Stale file handle]
[2017-03-17 14:04:26.666868] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 96d4ef59-05c0-4a8c-aa36-1a3e10da6905. (Possible split-brain)
[2017-03-17 14:04:26.669454] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:96d4ef59-05c0-4a8c-aa36-1a3e10da6905>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.669505] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 96d4ef59-05c0-4a8c-aa36-1a3e10da6905 [Stale file handle]
[2017-03-17 14:04:26.684088] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid b1262b27-ea2f-4716-a887-1b7926a8577a. (Possible split-brain)
[2017-03-17 14:04:26.686938] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:b1262b27-ea2f-4716-a887-1b7926a8577a>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.687003] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: b1262b27-ea2f-4716-a887-1b7926a8577a [Stale file handle]
[2017-03-17 14:04:26.690897] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid d58bc153-68f2-4c66-9d4b-077cfc80ff64. (Possible split-brain)
[2017-03-17 14:04:26.693336] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:d58bc153-68f2-4c66-9d4b-077cfc80ff64>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:04:26.693378] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: d58bc153-68f2-4c66-9d4b-077cfc80ff64 [Stale file handle]
[2017-03-17 14:04:26.741391] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 14:04:26.741760] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 14:04:26.742174] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 14:05:13.056728] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid f443d1fd-b010-4419-805d-aa56ebe25d73. (Possible split-brain)
[2017-03-17 14:05:13.060783] E [shard.c:426:shard_modify_size_and_block_count] (-->/usr/lib64/glusterfs/3.8.10/xlator/cluster/distribute.so(+0x60ed7) [0x7f1268254ed7] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xb6f0) [0x7f125bde96f0] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xaf5d) [0x7f125bde8f5d] ) 17-vmware2-shard: Failed to get trusted.glusterfs.shard.file-size for f443d1fd-b010-4419-805d-aa56ebe25d73
[2017-03-17 14:05:13.287906] I [MSGID: 114024] [client-helpers.c:99:this_fd_set_ctx] 17-vmware2-client-40: /Testse/Testse.vmx (f443d1fd-b010-4419-805d-aa56ebe25d73): trying duplicate remote fd set. 
[2017-03-17 14:05:13.287928] I [MSGID: 114024] [client-helpers.c:99:this_fd_set_ctx] 17-vmware2-client-41: /Testse/Testse.vmx (f443d1fd-b010-4419-805d-aa56ebe25d73): trying duplicate remote fd set. 
[2017-03-17 14:05:13.288024] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-20)
[2017-03-17 14:05:13.297110] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-40: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.297427] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-40: remote operation failed. Path: <gfid:f443d1fd-b010-4419-805d-aa56ebe25d73> (f443d1fd-b010-4419-805d-aa56ebe25d73) [No such file or directory]
[2017-03-17 14:05:13.297694] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-41: remote operation failed. Path: <gfid:f443d1fd-b010-4419-805d-aa56ebe25d73> (f443d1fd-b010-4419-805d-aa56ebe25d73) [No such file or directory]
[2017-03-17 14:05:13.298352] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-20: Unreadable subvolume -1 found with event generation 2 for gfid f443d1fd-b010-4419-805d-aa56ebe25d73. (Possible split-brain)
[2017-03-17 14:05:13.298803] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-40: remote operation failed. Path: <gfid:f443d1fd-b010-4419-805d-aa56ebe25d73> (f443d1fd-b010-4419-805d-aa56ebe25d73) [No such file or directory]
[2017-03-17 14:05:13.298953] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-41: remote operation failed. Path: <gfid:f443d1fd-b010-4419-805d-aa56ebe25d73> (f443d1fd-b010-4419-805d-aa56ebe25d73) [No such file or directory]
[2017-03-17 14:05:13.301523] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-40: remote operation failed. Path: <gfid:f443d1fd-b010-4419-805d-aa56ebe25d73> (f443d1fd-b010-4419-805d-aa56ebe25d73) [No such file or directory]
[2017-03-17 14:05:13.301725] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-41: remote operation failed. Path: <gfid:f443d1fd-b010-4419-805d-aa56ebe25d73> (f443d1fd-b010-4419-805d-aa56ebe25d73) [No such file or directory]
[2017-03-17 14:05:13.301875] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:f443d1fd-b010-4419-805d-aa56ebe25d73>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.301922] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: f443d1fd-b010-4419-805d-aa56ebe25d73 [Stale file handle]
[2017-03-17 14:05:13.412542] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.419172] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.419559] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:f3ee9962-8301-44cc-bedc-47e809af9a43> (f3ee9962-8301-44cc-bedc-47e809af9a43) [No such file or directory]
[2017-03-17 14:05:13.419665] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:f3ee9962-8301-44cc-bedc-47e809af9a43> (f3ee9962-8301-44cc-bedc-47e809af9a43) [No such file or directory]
[2017-03-17 14:05:13.420167] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid f3ee9962-8301-44cc-bedc-47e809af9a43. (Possible split-brain)
[2017-03-17 14:05:13.420658] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:f3ee9962-8301-44cc-bedc-47e809af9a43> (f3ee9962-8301-44cc-bedc-47e809af9a43) [No such file or directory]
[2017-03-17 14:05:13.420742] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:f3ee9962-8301-44cc-bedc-47e809af9a43> (f3ee9962-8301-44cc-bedc-47e809af9a43) [No such file or directory]
[2017-03-17 14:05:13.421947] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:f3ee9962-8301-44cc-bedc-47e809af9a43> (f3ee9962-8301-44cc-bedc-47e809af9a43) [No such file or directory]
[2017-03-17 14:05:13.421952] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:f3ee9962-8301-44cc-bedc-47e809af9a43> (f3ee9962-8301-44cc-bedc-47e809af9a43) [No such file or directory]
[2017-03-17 14:05:13.423668] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:f3ee9962-8301-44cc-bedc-47e809af9a43>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.423732] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: f3ee9962-8301-44cc-bedc-47e809af9a43 [Stale file handle]
[2017-03-17 14:05:13.457378] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.464303] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.464781] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:497aee74-cd18-4c65-8529-b20721d51bbe> (497aee74-cd18-4c65-8529-b20721d51bbe) [No such file or directory]
[2017-03-17 14:05:13.464848] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:497aee74-cd18-4c65-8529-b20721d51bbe> (497aee74-cd18-4c65-8529-b20721d51bbe) [No such file or directory]
[2017-03-17 14:05:13.465166] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 497aee74-cd18-4c65-8529-b20721d51bbe. (Possible split-brain)
[2017-03-17 14:05:13.465550] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:497aee74-cd18-4c65-8529-b20721d51bbe> (497aee74-cd18-4c65-8529-b20721d51bbe) [No such file or directory]
[2017-03-17 14:05:13.465668] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:497aee74-cd18-4c65-8529-b20721d51bbe> (497aee74-cd18-4c65-8529-b20721d51bbe) [No such file or directory]
[2017-03-17 14:05:13.466498] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:497aee74-cd18-4c65-8529-b20721d51bbe> (497aee74-cd18-4c65-8529-b20721d51bbe) [No such file or directory]
[2017-03-17 14:05:13.466498] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:497aee74-cd18-4c65-8529-b20721d51bbe> (497aee74-cd18-4c65-8529-b20721d51bbe) [No such file or directory]
[2017-03-17 14:05:13.467671] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:497aee74-cd18-4c65-8529-b20721d51bbe>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.467725] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 497aee74-cd18-4c65-8529-b20721d51bbe [Stale file handle]
[2017-03-17 14:05:13.492249] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.498503] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.498968] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:593a299f-1e8c-47cc-ba72-e0b8bc51e262> (593a299f-1e8c-47cc-ba72-e0b8bc51e262) [No such file or directory]
[2017-03-17 14:05:13.498980] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:593a299f-1e8c-47cc-ba72-e0b8bc51e262> (593a299f-1e8c-47cc-ba72-e0b8bc51e262) [No such file or directory]
[2017-03-17 14:05:13.499352] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 593a299f-1e8c-47cc-ba72-e0b8bc51e262. (Possible split-brain)
[2017-03-17 14:05:13.499889] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:593a299f-1e8c-47cc-ba72-e0b8bc51e262> (593a299f-1e8c-47cc-ba72-e0b8bc51e262) [No such file or directory]
[2017-03-17 14:05:13.499907] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:593a299f-1e8c-47cc-ba72-e0b8bc51e262> (593a299f-1e8c-47cc-ba72-e0b8bc51e262) [No such file or directory]
[2017-03-17 14:05:13.500803] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:593a299f-1e8c-47cc-ba72-e0b8bc51e262> (593a299f-1e8c-47cc-ba72-e0b8bc51e262) [No such file or directory]
[2017-03-17 14:05:13.500901] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:593a299f-1e8c-47cc-ba72-e0b8bc51e262> (593a299f-1e8c-47cc-ba72-e0b8bc51e262) [No such file or directory]
[2017-03-17 14:05:13.502176] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:593a299f-1e8c-47cc-ba72-e0b8bc51e262>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.502228] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 593a299f-1e8c-47cc-ba72-e0b8bc51e262 [Stale file handle]
[2017-03-17 14:05:13.525795] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.532731] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.533288] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:4621af56-55bd-4ea5-80d4-5a0949390bd4> (4621af56-55bd-4ea5-80d4-5a0949390bd4) [No such file or directory]
[2017-03-17 14:05:13.533361] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:4621af56-55bd-4ea5-80d4-5a0949390bd4> (4621af56-55bd-4ea5-80d4-5a0949390bd4) [No such file or directory]
[2017-03-17 14:05:13.533832] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 4621af56-55bd-4ea5-80d4-5a0949390bd4. (Possible split-brain)
[2017-03-17 14:05:13.534273] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:4621af56-55bd-4ea5-80d4-5a0949390bd4> (4621af56-55bd-4ea5-80d4-5a0949390bd4) [No such file or directory]
[2017-03-17 14:05:13.534332] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:4621af56-55bd-4ea5-80d4-5a0949390bd4> (4621af56-55bd-4ea5-80d4-5a0949390bd4) [No such file or directory]
[2017-03-17 14:05:13.535229] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:4621af56-55bd-4ea5-80d4-5a0949390bd4> (4621af56-55bd-4ea5-80d4-5a0949390bd4) [No such file or directory]
[2017-03-17 14:05:13.535423] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:4621af56-55bd-4ea5-80d4-5a0949390bd4> (4621af56-55bd-4ea5-80d4-5a0949390bd4) [No such file or directory]
[2017-03-17 14:05:13.536971] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:4621af56-55bd-4ea5-80d4-5a0949390bd4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.537027] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 4621af56-55bd-4ea5-80d4-5a0949390bd4 [Stale file handle]
[2017-03-17 14:05:13.580527] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 14:05:13.581662] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-20: selecting local read_child vmware2-client-40
[2017-03-17 14:05:13.617338] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.624631] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.625070] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:bc91e99a-43e1-4b29-85fe-8138299e0a82> (bc91e99a-43e1-4b29-85fe-8138299e0a82) [No such file or directory]
[2017-03-17 14:05:13.625192] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:bc91e99a-43e1-4b29-85fe-8138299e0a82> (bc91e99a-43e1-4b29-85fe-8138299e0a82) [No such file or directory]
[2017-03-17 14:05:13.625578] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid bc91e99a-43e1-4b29-85fe-8138299e0a82. (Possible split-brain)
[2017-03-17 14:05:13.626021] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:bc91e99a-43e1-4b29-85fe-8138299e0a82> (bc91e99a-43e1-4b29-85fe-8138299e0a82) [No such file or directory]
[2017-03-17 14:05:13.626064] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:bc91e99a-43e1-4b29-85fe-8138299e0a82> (bc91e99a-43e1-4b29-85fe-8138299e0a82) [No such file or directory]
[2017-03-17 14:05:13.626778] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:bc91e99a-43e1-4b29-85fe-8138299e0a82> (bc91e99a-43e1-4b29-85fe-8138299e0a82) [No such file or directory]
[2017-03-17 14:05:13.626817] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:bc91e99a-43e1-4b29-85fe-8138299e0a82> (bc91e99a-43e1-4b29-85fe-8138299e0a82) [No such file or directory]
[2017-03-17 14:05:13.628062] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:bc91e99a-43e1-4b29-85fe-8138299e0a82>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.628112] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: bc91e99a-43e1-4b29-85fe-8138299e0a82 [Stale file handle]
[2017-03-17 14:05:13.654404] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.660636] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.660987] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:3779f7a0-4b33-4ce8-a164-8f47fd316597> (3779f7a0-4b33-4ce8-a164-8f47fd316597) [No such file or directory]
[2017-03-17 14:05:13.661091] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:3779f7a0-4b33-4ce8-a164-8f47fd316597> (3779f7a0-4b33-4ce8-a164-8f47fd316597) [No such file or directory]
[2017-03-17 14:05:13.661577] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 3779f7a0-4b33-4ce8-a164-8f47fd316597. (Possible split-brain)
[2017-03-17 14:05:13.661988] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:3779f7a0-4b33-4ce8-a164-8f47fd316597> (3779f7a0-4b33-4ce8-a164-8f47fd316597) [No such file or directory]
[2017-03-17 14:05:13.662062] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:3779f7a0-4b33-4ce8-a164-8f47fd316597> (3779f7a0-4b33-4ce8-a164-8f47fd316597) [No such file or directory]
[2017-03-17 14:05:13.663275] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:3779f7a0-4b33-4ce8-a164-8f47fd316597> (3779f7a0-4b33-4ce8-a164-8f47fd316597) [No such file or directory]
[2017-03-17 14:05:13.663451] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:3779f7a0-4b33-4ce8-a164-8f47fd316597> (3779f7a0-4b33-4ce8-a164-8f47fd316597) [No such file or directory]
[2017-03-17 14:05:13.665338] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:3779f7a0-4b33-4ce8-a164-8f47fd316597>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.665385] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 3779f7a0-4b33-4ce8-a164-8f47fd316597 [Stale file handle]
[2017-03-17 14:05:13.697542] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.704299] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.704810] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:cd3238e7-4ff4-4336-824f-1c9ac8914083> (cd3238e7-4ff4-4336-824f-1c9ac8914083) [No such file or directory]
[2017-03-17 14:05:13.704843] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:cd3238e7-4ff4-4336-824f-1c9ac8914083> (cd3238e7-4ff4-4336-824f-1c9ac8914083) [No such file or directory]
[2017-03-17 14:05:13.705288] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid cd3238e7-4ff4-4336-824f-1c9ac8914083. (Possible split-brain)
[2017-03-17 14:05:13.705670] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:cd3238e7-4ff4-4336-824f-1c9ac8914083> (cd3238e7-4ff4-4336-824f-1c9ac8914083) [No such file or directory]
[2017-03-17 14:05:13.705859] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:cd3238e7-4ff4-4336-824f-1c9ac8914083> (cd3238e7-4ff4-4336-824f-1c9ac8914083) [No such file or directory]
[2017-03-17 14:05:13.706720] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:cd3238e7-4ff4-4336-824f-1c9ac8914083> (cd3238e7-4ff4-4336-824f-1c9ac8914083) [No such file or directory]
[2017-03-17 14:05:13.706789] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:cd3238e7-4ff4-4336-824f-1c9ac8914083> (cd3238e7-4ff4-4336-824f-1c9ac8914083) [No such file or directory]
[2017-03-17 14:05:13.707995] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:cd3238e7-4ff4-4336-824f-1c9ac8914083>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.708041] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: cd3238e7-4ff4-4336-824f-1c9ac8914083 [Stale file handle]
[2017-03-17 14:05:13.735271] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.741255] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.741646] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6> (9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6) [No such file or directory]
[2017-03-17 14:05:13.741712] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6> (9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6) [No such file or directory]
[2017-03-17 14:05:13.742038] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6. (Possible split-brain)
[2017-03-17 14:05:13.742455] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6> (9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6) [No such file or directory]
[2017-03-17 14:05:13.742543] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6> (9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6) [No such file or directory]
[2017-03-17 14:05:13.743318] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6> (9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6) [No such file or directory]
[2017-03-17 14:05:13.743328] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6> (9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6) [No such file or directory]
[2017-03-17 14:05:13.744368] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.744399] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 9e80d6a6-a8fb-422b-b2e8-4d61c16ad7f6 [Stale file handle]
[2017-03-17 14:05:13.798845] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.805059] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.805481] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:cbf41972-cf28-4faf-a16f-7970a723f67b> (cbf41972-cf28-4faf-a16f-7970a723f67b) [No such file or directory]
[2017-03-17 14:05:13.805555] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:cbf41972-cf28-4faf-a16f-7970a723f67b> (cbf41972-cf28-4faf-a16f-7970a723f67b) [No such file or directory]
[2017-03-17 14:05:13.805901] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid cbf41972-cf28-4faf-a16f-7970a723f67b. (Possible split-brain)
[2017-03-17 14:05:13.806413] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:cbf41972-cf28-4faf-a16f-7970a723f67b> (cbf41972-cf28-4faf-a16f-7970a723f67b) [No such file or directory]
[2017-03-17 14:05:13.806454] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:cbf41972-cf28-4faf-a16f-7970a723f67b> (cbf41972-cf28-4faf-a16f-7970a723f67b) [No such file or directory]
[2017-03-17 14:05:13.807530] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:cbf41972-cf28-4faf-a16f-7970a723f67b> (cbf41972-cf28-4faf-a16f-7970a723f67b) [No such file or directory]
[2017-03-17 14:05:13.807597] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:cbf41972-cf28-4faf-a16f-7970a723f67b> (cbf41972-cf28-4faf-a16f-7970a723f67b) [No such file or directory]
[2017-03-17 14:05:13.809167] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:cbf41972-cf28-4faf-a16f-7970a723f67b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.809221] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: cbf41972-cf28-4faf-a16f-7970a723f67b [Stale file handle]
[2017-03-17 14:05:13.835796] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.842429] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.842928] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:a2f5d4c5-db23-4252-930f-b0d437f85a1e> (a2f5d4c5-db23-4252-930f-b0d437f85a1e) [No such file or directory]
[2017-03-17 14:05:13.842956] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:a2f5d4c5-db23-4252-930f-b0d437f85a1e> (a2f5d4c5-db23-4252-930f-b0d437f85a1e) [No such file or directory]
[2017-03-17 14:05:13.843263] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid a2f5d4c5-db23-4252-930f-b0d437f85a1e. (Possible split-brain)
[2017-03-17 14:05:13.843736] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:a2f5d4c5-db23-4252-930f-b0d437f85a1e> (a2f5d4c5-db23-4252-930f-b0d437f85a1e) [No such file or directory]
[2017-03-17 14:05:13.843741] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:a2f5d4c5-db23-4252-930f-b0d437f85a1e> (a2f5d4c5-db23-4252-930f-b0d437f85a1e) [No such file or directory]
[2017-03-17 14:05:13.844856] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:a2f5d4c5-db23-4252-930f-b0d437f85a1e> (a2f5d4c5-db23-4252-930f-b0d437f85a1e) [No such file or directory]
[2017-03-17 14:05:13.844970] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:a2f5d4c5-db23-4252-930f-b0d437f85a1e> (a2f5d4c5-db23-4252-930f-b0d437f85a1e) [No such file or directory]
[2017-03-17 14:05:13.846125] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:a2f5d4c5-db23-4252-930f-b0d437f85a1e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.846195] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: a2f5d4c5-db23-4252-930f-b0d437f85a1e [Stale file handle]
[2017-03-17 14:05:13.869629] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.875711] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.876143] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:e91402e4-caba-4e0a-a4d4-6c8ee173590b> (e91402e4-caba-4e0a-a4d4-6c8ee173590b) [No such file or directory]
[2017-03-17 14:05:13.876184] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:e91402e4-caba-4e0a-a4d4-6c8ee173590b> (e91402e4-caba-4e0a-a4d4-6c8ee173590b) [No such file or directory]
[2017-03-17 14:05:13.876543] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid e91402e4-caba-4e0a-a4d4-6c8ee173590b. (Possible split-brain)
[2017-03-17 14:05:13.877029] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:e91402e4-caba-4e0a-a4d4-6c8ee173590b> (e91402e4-caba-4e0a-a4d4-6c8ee173590b) [No such file or directory]
[2017-03-17 14:05:13.877040] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:e91402e4-caba-4e0a-a4d4-6c8ee173590b> (e91402e4-caba-4e0a-a4d4-6c8ee173590b) [No such file or directory]
[2017-03-17 14:05:13.877867] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:e91402e4-caba-4e0a-a4d4-6c8ee173590b> (e91402e4-caba-4e0a-a4d4-6c8ee173590b) [No such file or directory]
[2017-03-17 14:05:13.877910] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:e91402e4-caba-4e0a-a4d4-6c8ee173590b> (e91402e4-caba-4e0a-a4d4-6c8ee173590b) [No such file or directory]
[2017-03-17 14:05:13.879200] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:e91402e4-caba-4e0a-a4d4-6c8ee173590b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.879240] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: e91402e4-caba-4e0a-a4d4-6c8ee173590b [Stale file handle]
[2017-03-17 14:05:13.905897] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:13.911909] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:13.912296] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8> (ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8) [No such file or directory]
[2017-03-17 14:05:13.912434] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8> (ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8) [No such file or directory]
[2017-03-17 14:05:13.912718] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8. (Possible split-brain)
[2017-03-17 14:05:13.913062] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8> (ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8) [No such file or directory]
[2017-03-17 14:05:13.913242] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8> (ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8) [No such file or directory]
[2017-03-17 14:05:13.914034] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8> (ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8) [No such file or directory]
[2017-03-17 14:05:13.914103] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8> (ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8) [No such file or directory]
[2017-03-17 14:05:13.915057] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:13.915108] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: ceb6e1b2-3b09-4d05-bb6e-25bbad6486e8 [Stale file handle]
[2017-03-17 14:05:13.958873] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 14:05:13.960353] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-20: selecting local read_child vmware2-client-40
[2017-03-17 14:05:23.301417] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmdk~ (hash=vmware2-replicate-16/cache=vmware2-replicate-16) => /Testse/Testse.vmdk (hash=vmware2-replicate-1/cache=vmware2-replicate-1)
[2017-03-17 14:05:23.310984] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-3: remote operation failed [No such file or directory]
[2017-03-17 14:05:23.311545] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-2: remote operation failed. Path: <gfid:282d8f8f-eafb-4233-a939-5b78bfc83ef2> (282d8f8f-eafb-4233-a939-5b78bfc83ef2) [No such file or directory]
[2017-03-17 14:05:23.311592] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-3: remote operation failed. Path: <gfid:282d8f8f-eafb-4233-a939-5b78bfc83ef2> (282d8f8f-eafb-4233-a939-5b78bfc83ef2) [No such file or directory]
[2017-03-17 14:05:23.312213] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid 282d8f8f-eafb-4233-a939-5b78bfc83ef2. (Possible split-brain)
[2017-03-17 14:05:23.312781] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-2: remote operation failed. Path: <gfid:282d8f8f-eafb-4233-a939-5b78bfc83ef2> (282d8f8f-eafb-4233-a939-5b78bfc83ef2) [No such file or directory]
[2017-03-17 14:05:23.312829] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-3: remote operation failed. Path: <gfid:282d8f8f-eafb-4233-a939-5b78bfc83ef2> (282d8f8f-eafb-4233-a939-5b78bfc83ef2) [No such file or directory]
[2017-03-17 14:05:23.313615] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-2: remote operation failed. Path: <gfid:282d8f8f-eafb-4233-a939-5b78bfc83ef2> (282d8f8f-eafb-4233-a939-5b78bfc83ef2) [No such file or directory]
[2017-03-17 14:05:23.313665] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-3: remote operation failed. Path: <gfid:282d8f8f-eafb-4233-a939-5b78bfc83ef2> (282d8f8f-eafb-4233-a939-5b78bfc83ef2) [No such file or directory]
[2017-03-17 14:05:23.315652] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:282d8f8f-eafb-4233-a939-5b78bfc83ef2>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:23.315708] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 282d8f8f-eafb-4233-a939-5b78bfc83ef2 [Stale file handle]
[2017-03-17 14:05:23.320526] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid a7570a5c-9def-4cee-a035-f87755bd39c4. (Possible split-brain)
[2017-03-17 14:05:23.323123] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:a7570a5c-9def-4cee-a035-f87755bd39c4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:23.323181] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: a7570a5c-9def-4cee-a035-f87755bd39c4 [Stale file handle]
[2017-03-17 14:05:38.420550] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:38.426994] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:38.427419] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:a161b905-6ea2-4add-affa-c1513bc1488f> (a161b905-6ea2-4add-affa-c1513bc1488f) [No such file or directory]
[2017-03-17 14:05:38.427485] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:a161b905-6ea2-4add-affa-c1513bc1488f> (a161b905-6ea2-4add-affa-c1513bc1488f) [No such file or directory]
[2017-03-17 14:05:38.428482] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid a161b905-6ea2-4add-affa-c1513bc1488f. (Possible split-brain)
[2017-03-17 14:05:38.428970] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:a161b905-6ea2-4add-affa-c1513bc1488f> (a161b905-6ea2-4add-affa-c1513bc1488f) [No such file or directory]
[2017-03-17 14:05:38.429040] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:a161b905-6ea2-4add-affa-c1513bc1488f> (a161b905-6ea2-4add-affa-c1513bc1488f) [No such file or directory]
[2017-03-17 14:05:38.430054] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:a161b905-6ea2-4add-affa-c1513bc1488f> (a161b905-6ea2-4add-affa-c1513bc1488f) [No such file or directory]
[2017-03-17 14:05:38.430141] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:a161b905-6ea2-4add-affa-c1513bc1488f> (a161b905-6ea2-4add-affa-c1513bc1488f) [No such file or directory]
[2017-03-17 14:05:38.431296] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:a161b905-6ea2-4add-affa-c1513bc1488f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:38.431333] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: a161b905-6ea2-4add-affa-c1513bc1488f [Stale file handle]
[2017-03-17 14:05:50.532854] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:50.540071] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:50.540551] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:363164ff-9a8f-491a-a68e-1535b10bb2ee> (363164ff-9a8f-491a-a68e-1535b10bb2ee) [No such file or directory]
[2017-03-17 14:05:50.540615] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:363164ff-9a8f-491a-a68e-1535b10bb2ee> (363164ff-9a8f-491a-a68e-1535b10bb2ee) [No such file or directory]
[2017-03-17 14:05:50.541313] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 363164ff-9a8f-491a-a68e-1535b10bb2ee. (Possible split-brain)
[2017-03-17 14:05:50.541873] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:363164ff-9a8f-491a-a68e-1535b10bb2ee> (363164ff-9a8f-491a-a68e-1535b10bb2ee) [No such file or directory]
[2017-03-17 14:05:50.541926] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:363164ff-9a8f-491a-a68e-1535b10bb2ee> (363164ff-9a8f-491a-a68e-1535b10bb2ee) [No such file or directory]
[2017-03-17 14:05:50.542780] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:363164ff-9a8f-491a-a68e-1535b10bb2ee> (363164ff-9a8f-491a-a68e-1535b10bb2ee) [No such file or directory]
[2017-03-17 14:05:50.542901] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:363164ff-9a8f-491a-a68e-1535b10bb2ee> (363164ff-9a8f-491a-a68e-1535b10bb2ee) [No such file or directory]
[2017-03-17 14:05:50.543878] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:363164ff-9a8f-491a-a68e-1535b10bb2ee>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.543929] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 363164ff-9a8f-491a-a68e-1535b10bb2ee [Stale file handle]
[2017-03-17 14:05:50.557624] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid d4f47f76-c116-4fef-bd3d-d48127abaaa9. (Possible split-brain)
[2017-03-17 14:05:50.560486] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:d4f47f76-c116-4fef-bd3d-d48127abaaa9>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.560556] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: d4f47f76-c116-4fef-bd3d-d48127abaaa9 [Stale file handle]
[2017-03-17 14:05:50.606064] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid babb98fe-d400-4f46-bea6-d9c211c5ce9d. (Possible split-brain)
[2017-03-17 14:05:50.608614] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:babb98fe-d400-4f46-bea6-d9c211c5ce9d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.608677] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: babb98fe-d400-4f46-bea6-d9c211c5ce9d [Stale file handle]
[2017-03-17 14:05:50.622688] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid ebc164db-946b-46d6-9643-30e388f55425. (Possible split-brain)
[2017-03-17 14:05:50.625559] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:ebc164db-946b-46d6-9643-30e388f55425>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.625620] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: ebc164db-946b-46d6-9643-30e388f55425 [Stale file handle]
[2017-03-17 14:05:50.630299] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-14: Unreadable subvolume -1 found with event generation 2 for gfid c76ea6bd-c965-40c9-b420-e30273ecaae9. (Possible split-brain)
[2017-03-17 14:05:50.632839] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:c76ea6bd-c965-40c9-b420-e30273ecaae9>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.632893] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: c76ea6bd-c965-40c9-b420-e30273ecaae9 [Stale file handle]
[2017-03-17 14:05:50.680255] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /Testse/Testse.vmx~ (hash=vmware2-replicate-6/cache=vmware2-replicate-6) => /Testse/Testse.vmx (hash=vmware2-replicate-20/cache=vmware2-replicate-6)
[2017-03-17 14:05:50.687338] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-12: remote operation failed [No such file or directory]
[2017-03-17 14:05:50.687902] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:6d004ddc-e705-4ac3-beca-f490959dcb22> (6d004ddc-e705-4ac3-beca-f490959dcb22) [No such file or directory]
[2017-03-17 14:05:50.687927] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:6d004ddc-e705-4ac3-beca-f490959dcb22> (6d004ddc-e705-4ac3-beca-f490959dcb22) [No such file or directory]
[2017-03-17 14:05:50.688378] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-6: Unreadable subvolume -1 found with event generation 2 for gfid 6d004ddc-e705-4ac3-beca-f490959dcb22. (Possible split-brain)
[2017-03-17 14:05:50.688973] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:6d004ddc-e705-4ac3-beca-f490959dcb22> (6d004ddc-e705-4ac3-beca-f490959dcb22) [No such file or directory]
[2017-03-17 14:05:50.688975] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:6d004ddc-e705-4ac3-beca-f490959dcb22> (6d004ddc-e705-4ac3-beca-f490959dcb22) [No such file or directory]
[2017-03-17 14:05:50.689912] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-13: remote operation failed. Path: <gfid:6d004ddc-e705-4ac3-beca-f490959dcb22> (6d004ddc-e705-4ac3-beca-f490959dcb22) [No such file or directory]
[2017-03-17 14:05:50.689970] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-12: remote operation failed. Path: <gfid:6d004ddc-e705-4ac3-beca-f490959dcb22> (6d004ddc-e705-4ac3-beca-f490959dcb22) [No such file or directory]
[2017-03-17 14:05:50.691110] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:6d004ddc-e705-4ac3-beca-f490959dcb22>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.691152] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 6d004ddc-e705-4ac3-beca-f490959dcb22 [Stale file handle]
[2017-03-17 14:05:50.696099] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-3: Unreadable subvolume -1 found with event generation 2 for gfid dbd706be-87a7-4f28-9ed1-607ea1670d1c. (Possible split-brain)
[2017-03-17 14:05:50.698865] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:dbd706be-87a7-4f28-9ed1-607ea1670d1c>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.698931] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: dbd706be-87a7-4f28-9ed1-607ea1670d1c [Stale file handle]
[2017-03-17 14:05:50.717259] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid e0572401-7a8e-4040-a22a-568730af9eca. (Possible split-brain)
[2017-03-17 14:05:50.719831] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:e0572401-7a8e-4040-a22a-568730af9eca>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.719884] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: e0572401-7a8e-4040-a22a-568730af9eca [Stale file handle]
[2017-03-17 14:05:50.723588] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-3: Unreadable subvolume -1 found with event generation 2 for gfid 3f1ecbbf-d57e-4f6e-b96d-07601b70e211. (Possible split-brain)
[2017-03-17 14:05:50.726171] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:3f1ecbbf-d57e-4f6e-b96d-07601b70e211>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.726210] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 3f1ecbbf-d57e-4f6e-b96d-07601b70e211 [Stale file handle]
[2017-03-17 14:05:50.747314] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 7e3e70d6-399c-46fc-9337-4d0a1f611c56. (Possible split-brain)
[2017-03-17 14:05:50.749792] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:7e3e70d6-399c-46fc-9337-4d0a1f611c56>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.749840] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 7e3e70d6-399c-46fc-9337-4d0a1f611c56 [Stale file handle]
[2017-03-17 14:05:50.762990] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-8: Unreadable subvolume -1 found with event generation 2 for gfid b77bd01f-28d4-4199-8d9a-d7185f3911b5. (Possible split-brain)
[2017-03-17 14:05:50.765838] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:b77bd01f-28d4-4199-8d9a-d7185f3911b5>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.765903] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: b77bd01f-28d4-4199-8d9a-d7185f3911b5 [Stale file handle]
[2017-03-17 14:05:50.769745] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-15: Unreadable subvolume -1 found with event generation 2 for gfid 7340cf01-f2f2-4758-964f-66294f4df08f. (Possible split-brain)
[2017-03-17 14:05:50.772454] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:7340cf01-f2f2-4758-964f-66294f4df08f>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:05:50.772501] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 7340cf01-f2f2-4758-964f-66294f4df08f [Stale file handle]
[2017-03-17 14:05:50.842321] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-6: selecting local read_child vmware2-client-12
[2017-03-17 14:05:50.842503] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-8: selecting local read_child vmware2-client-16
[2017-03-17 14:05:50.843063] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-14: selecting local read_child vmware2-client-28
[2017-03-17 14:05:50.843599] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-20: selecting local read_child vmware2-client-40
[2017-03-17 14:06:14.676147] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/vmware.log (hash=vmware2-replicate-17/cache=vmware2-replicate-10) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/vmware-1.log (hash=vmware2-replicate-21/cache=<nul>)
[2017-03-17 14:06:14.872579] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-13/cache=vmware2-replicate-13) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-13)
[2017-03-17 14:06:14.880585] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-26: remote operation failed [No such file or directory]
[2017-03-17 14:06:14.881066] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:768fa325-0289-48db-bb91-1e6197067b08> (768fa325-0289-48db-bb91-1e6197067b08) [No such file or directory]
[2017-03-17 14:06:14.881192] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:768fa325-0289-48db-bb91-1e6197067b08> (768fa325-0289-48db-bb91-1e6197067b08) [No such file or directory]
[2017-03-17 14:06:14.881893] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 768fa325-0289-48db-bb91-1e6197067b08. (Possible split-brain)
[2017-03-17 14:06:14.882416] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:768fa325-0289-48db-bb91-1e6197067b08> (768fa325-0289-48db-bb91-1e6197067b08) [No such file or directory]
[2017-03-17 14:06:14.882498] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:768fa325-0289-48db-bb91-1e6197067b08> (768fa325-0289-48db-bb91-1e6197067b08) [No such file or directory]
[2017-03-17 14:06:14.884578] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:768fa325-0289-48db-bb91-1e6197067b08> (768fa325-0289-48db-bb91-1e6197067b08) [No such file or directory]
[2017-03-17 14:06:14.884664] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:768fa325-0289-48db-bb91-1e6197067b08> (768fa325-0289-48db-bb91-1e6197067b08) [No such file or directory]
[2017-03-17 14:06:14.885483] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:768fa325-0289-48db-bb91-1e6197067b08>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:06:14.885550] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 768fa325-0289-48db-bb91-1e6197067b08 [Stale file handle]
[2017-03-17 14:06:14.954589] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-13/cache=vmware2-replicate-13) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-13)
[2017-03-17 14:06:14.962375] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-27: remote operation failed [No such file or directory]
[2017-03-17 14:06:14.962892] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:8e2a921e-03fb-434d-8cb2-3cd40f3bba81> (8e2a921e-03fb-434d-8cb2-3cd40f3bba81) [No such file or directory]
[2017-03-17 14:06:14.962997] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:8e2a921e-03fb-434d-8cb2-3cd40f3bba81> (8e2a921e-03fb-434d-8cb2-3cd40f3bba81) [No such file or directory]
[2017-03-17 14:06:14.963604] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 8e2a921e-03fb-434d-8cb2-3cd40f3bba81. (Possible split-brain)
[2017-03-17 14:06:14.964042] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:8e2a921e-03fb-434d-8cb2-3cd40f3bba81> (8e2a921e-03fb-434d-8cb2-3cd40f3bba81) [No such file or directory]
[2017-03-17 14:06:14.964187] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:8e2a921e-03fb-434d-8cb2-3cd40f3bba81> (8e2a921e-03fb-434d-8cb2-3cd40f3bba81) [No such file or directory]
[2017-03-17 14:06:14.965845] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:8e2a921e-03fb-434d-8cb2-3cd40f3bba81> (8e2a921e-03fb-434d-8cb2-3cd40f3bba81) [No such file or directory]
[2017-03-17 14:06:14.965860] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:8e2a921e-03fb-434d-8cb2-3cd40f3bba81> (8e2a921e-03fb-434d-8cb2-3cd40f3bba81) [No such file or directory]
[2017-03-17 14:06:14.966669] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:8e2a921e-03fb-434d-8cb2-3cd40f3bba81>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:06:14.966730] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 8e2a921e-03fb-434d-8cb2-3cd40f3bba81 [Stale file handle]
[2017-03-17 14:06:22.138656] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmdk~ (hash=vmware2-replicate-17/cache=vmware2-replicate-17) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmdk (hash=vmware2-replicate-19/cache=vmware2-replicate-10)
[2017-03-17 14:06:22.148440] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-20: remote operation failed [No such file or directory]
[2017-03-17 14:06:22.149026] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-21: remote operation failed. Path: <gfid:4dc735da-5318-43b6-a015-90cd8a309b39> (4dc735da-5318-43b6-a015-90cd8a309b39) [No such file or directory]
[2017-03-17 14:06:22.149027] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-20: remote operation failed. Path: <gfid:4dc735da-5318-43b6-a015-90cd8a309b39> (4dc735da-5318-43b6-a015-90cd8a309b39) [No such file or directory]
[2017-03-17 14:06:22.149629] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 4dc735da-5318-43b6-a015-90cd8a309b39. (Possible split-brain)
[2017-03-17 14:06:22.150151] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-20: remote operation failed. Path: <gfid:4dc735da-5318-43b6-a015-90cd8a309b39> (4dc735da-5318-43b6-a015-90cd8a309b39) [No such file or directory]
[2017-03-17 14:06:22.150188] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-21: remote operation failed. Path: <gfid:4dc735da-5318-43b6-a015-90cd8a309b39> (4dc735da-5318-43b6-a015-90cd8a309b39) [No such file or directory]
[2017-03-17 14:06:22.151521] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-20: remote operation failed. Path: <gfid:4dc735da-5318-43b6-a015-90cd8a309b39> (4dc735da-5318-43b6-a015-90cd8a309b39) [No such file or directory]
[2017-03-17 14:06:22.151569] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-21: remote operation failed. Path: <gfid:4dc735da-5318-43b6-a015-90cd8a309b39> (4dc735da-5318-43b6-a015-90cd8a309b39) [No such file or directory]
[2017-03-17 14:06:22.152647] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:4dc735da-5318-43b6-a015-90cd8a309b39>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:06:22.152723] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 4dc735da-5318-43b6-a015-90cd8a309b39 [Stale file handle]
[2017-03-17 14:06:22.157248] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid 6187b5f0-998a-4fd0-ad48-b8cd260130cc. (Possible split-brain)
[2017-03-17 14:06:22.159818] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:6187b5f0-998a-4fd0-ad48-b8cd260130cc>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:06:22.159882] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 6187b5f0-998a-4fd0-ad48-b8cd260130cc [Stale file handle]
[2017-03-17 14:07:55.046540] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 14:07:55.047262] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 14:12:57.395007] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid c85d714c-2d03-4577-93d6-a16c6da5d457. (Possible split-brain)
[2017-03-17 14:12:57.397984] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:c85d714c-2d03-4577-93d6-a16c6da5d457>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:12:57.398047] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: c85d714c-2d03-4577-93d6-a16c6da5d457 [Stale file handle]
[2017-03-17 14:12:57.455393] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid 216a377a-7858-4f21-b9bd-e8b1effc818e. (Possible split-brain)
[2017-03-17 14:12:57.458750] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:216a377a-7858-4f21-b9bd-e8b1effc818e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:12:57.458851] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 216a377a-7858-4f21-b9bd-e8b1effc818e [Stale file handle]
[2017-03-17 14:12:57.474336] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-0: Unreadable subvolume -1 found with event generation 2 for gfid d79c9966-4f99-4666-a174-de8b10447637. (Possible split-brain)
[2017-03-17 14:12:57.477524] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:d79c9966-4f99-4666-a174-de8b10447637>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:12:57.477600] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: d79c9966-4f99-4666-a174-de8b10447637 [Stale file handle]
[2017-03-17 14:12:57.481616] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid 0b7c1957-e63a-488f-9006-0b7f22ddf9d4. (Possible split-brain)
[2017-03-17 14:12:57.484363] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:0b7c1957-e63a-488f-9006-0b7f22ddf9d4>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:12:57.484410] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 0b7c1957-e63a-488f-9006-0b7f22ddf9d4 [Stale file handle]
[2017-03-17 14:12:57.603565] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-13/cache=vmware2-replicate-13) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-13)
[2017-03-17 14:12:57.611478] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-27: remote operation failed [No such file or directory]
[2017-03-17 14:12:57.612003] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:347061a2-3baf-479c-a713-c130c2ded288> (347061a2-3baf-479c-a713-c130c2ded288) [No such file or directory]
[2017-03-17 14:12:57.612011] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:347061a2-3baf-479c-a713-c130c2ded288> (347061a2-3baf-479c-a713-c130c2ded288) [No such file or directory]
[2017-03-17 14:12:57.613430] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 347061a2-3baf-479c-a713-c130c2ded288. (Possible split-brain)
[2017-03-17 14:12:57.613949] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:347061a2-3baf-479c-a713-c130c2ded288> (347061a2-3baf-479c-a713-c130c2ded288) [No such file or directory]
[2017-03-17 14:12:57.614014] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:347061a2-3baf-479c-a713-c130c2ded288> (347061a2-3baf-479c-a713-c130c2ded288) [No such file or directory]
[2017-03-17 14:12:57.615720] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:347061a2-3baf-479c-a713-c130c2ded288> (347061a2-3baf-479c-a713-c130c2ded288) [No such file or directory]
[2017-03-17 14:12:57.615868] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:347061a2-3baf-479c-a713-c130c2ded288> (347061a2-3baf-479c-a713-c130c2ded288) [No such file or directory]
[2017-03-17 14:12:57.616587] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:347061a2-3baf-479c-a713-c130c2ded288>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:12:57.616638] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 347061a2-3baf-479c-a713-c130c2ded288 [Stale file handle]
[2017-03-17 14:12:57.621933] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 276d409e-6aa4-45b0-a59d-96aaa63fadc0. (Possible split-brain)
[2017-03-17 14:12:57.624798] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:276d409e-6aa4-45b0-a59d-96aaa63fadc0>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:12:57.624860] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 276d409e-6aa4-45b0-a59d-96aaa63fadc0 [Stale file handle]
[2017-03-17 14:12:57.640189] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-11: Unreadable subvolume -1 found with event generation 2 for gfid d51d56b0-584e-403d-8c30-ab1af02b9f1e. (Possible split-brain)
[2017-03-17 14:12:57.642894] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:d51d56b0-584e-403d-8c30-ab1af02b9f1e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:12:57.642972] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: d51d56b0-584e-403d-8c30-ab1af02b9f1e [Stale file handle]
[2017-03-17 14:12:57.646878] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-7: Unreadable subvolume -1 found with event generation 2 for gfid 8e533360-1eab-48b3-91a3-8ee978d776d3. (Possible split-brain)
[2017-03-17 14:12:57.649448] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:8e533360-1eab-48b3-91a3-8ee978d776d3>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:12:57.649506] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 8e533360-1eab-48b3-91a3-8ee978d776d3 [Stale file handle]
[2017-03-17 14:12:57.674849] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid ec401caa-a909-48e6-aff3-1ab50771b6fd. (Possible split-brain)
[2017-03-17 14:12:57.677565] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:ec401caa-a909-48e6-aff3-1ab50771b6fd>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:12:57.677621] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: ec401caa-a909-48e6-aff3-1ab50771b6fd [Stale file handle]
[2017-03-17 14:12:57.690936] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid f1c98205-b818-4a6a-ab3a-2e193104576b. (Possible split-brain)
[2017-03-17 14:12:57.693679] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:f1c98205-b818-4a6a-ab3a-2e193104576b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:12:57.693757] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: f1c98205-b818-4a6a-ab3a-2e193104576b [Stale file handle]
[2017-03-17 14:12:57.697655] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid 123686b3-7603-4401-addb-ab59247ef79d. (Possible split-brain)
[2017-03-17 14:12:57.700275] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:123686b3-7603-4401-addb-ab59247ef79d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:12:57.700327] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 123686b3-7603-4401-addb-ab59247ef79d [Stale file handle]
[2017-03-17 14:12:57.751740] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-0: selecting local read_child vmware2-client-0
[2017-03-17 14:13:04.892439] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-10: Unreadable subvolume -1 found with event generation 2 for gfid 330f216d-6833-43ee-8074-74ca3a6e37bf. (Possible split-brain)
[2017-03-17 14:13:04.895655] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:330f216d-6833-43ee-8074-74ca3a6e37bf>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:04.895714] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 330f216d-6833-43ee-8074-74ca3a6e37bf [Stale file handle]
[2017-03-17 14:13:04.901999] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-12: Unreadable subvolume -1 found with event generation 2 for gfid e27f3c56-b050-46f3-a51a-a3c28a830471. (Possible split-brain)
[2017-03-17 14:13:04.905561] E [shard.c:426:shard_modify_size_and_block_count] (-->/usr/lib64/glusterfs/3.8.10/xlator/cluster/distribute.so(+0x60ed7) [0x7f1268254ed7] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xb6f0) [0x7f125bde96f0] -->/usr/lib64/glusterfs/3.8.10/xlator/features/shard.so(+0xaf5d) [0x7f125bde8f5d] ) 17-vmware2-shard: Failed to get trusted.glusterfs.shard.file-size for e27f3c56-b050-46f3-a51a-a3c28a830471
[2017-03-17 14:13:05.014579] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-19: Unreadable subvolume -1 found with event generation 2 for gfid 3de93166-a95b-4da6-8415-2bfe021bea7d. (Possible split-brain)
[2017-03-17 14:13:05.017533] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:3de93166-a95b-4da6-8415-2bfe021bea7d>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:05.017590] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 3de93166-a95b-4da6-8415-2bfe021bea7d [Stale file handle]
[2017-03-17 14:13:05.031785] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-18: Unreadable subvolume -1 found with event generation 2 for gfid 83bf80db-5048-4a14-9e0a-7d1237a2a9e2. (Possible split-brain)
[2017-03-17 14:13:05.034397] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:83bf80db-5048-4a14-9e0a-7d1237a2a9e2>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:05.034456] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 83bf80db-5048-4a14-9e0a-7d1237a2a9e2 [Stale file handle]
[2017-03-17 14:13:05.037947] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid aee4adb8-4058-4c1d-bb09-c6539ff97aba. (Possible split-brain)
[2017-03-17 14:13:05.040288] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:aee4adb8-4058-4c1d-bb09-c6539ff97aba>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:05.040330] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: aee4adb8-4058-4c1d-bb09-c6539ff97aba [Stale file handle]
[2017-03-17 14:13:05.087638] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming /New Virtual Machine/New Virtual Machine.vmx~ (hash=vmware2-replicate-9/cache=vmware2-replicate-9) => /New Virtual Machine/New Virtual Machine.vmx (hash=vmware2-replicate-13/cache=vmware2-replicate-13)
[2017-03-17 14:13:05.096127] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid f9f2d22b-c214-43ae-9de3-240bd2697112. (Possible split-brain)
[2017-03-17 14:13:05.098298] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:f9f2d22b-c214-43ae-9de3-240bd2697112>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:05.098342] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: f9f2d22b-c214-43ae-9de3-240bd2697112 [Stale file handle]
[2017-03-17 14:13:05.103963] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-20: Unreadable subvolume -1 found with event generation 2 for gfid 058bcdef-7095-476b-9f7b-8c4c8cbcef82. (Possible split-brain)
[2017-03-17 14:13:05.106457] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:058bcdef-7095-476b-9f7b-8c4c8cbcef82>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:05.106519] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 058bcdef-7095-476b-9f7b-8c4c8cbcef82 [Stale file handle]
[2017-03-17 14:13:05.123562] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-5: Unreadable subvolume -1 found with event generation 2 for gfid 445453c4-409c-4603-9c6b-b3563f331992. (Possible split-brain)
[2017-03-17 14:13:05.125878] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:445453c4-409c-4603-9c6b-b3563f331992>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:05.125920] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 445453c4-409c-4603-9c6b-b3563f331992 [Stale file handle]
[2017-03-17 14:13:05.129827] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-20: Unreadable subvolume -1 found with event generation 2 for gfid 7d48a628-e673-4492-bcba-2d1b8a19ba44. (Possible split-brain)
[2017-03-17 14:13:05.132157] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:7d48a628-e673-4492-bcba-2d1b8a19ba44>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:05.132201] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 7d48a628-e673-4492-bcba-2d1b8a19ba44 [Stale file handle]
[2017-03-17 14:13:05.155043] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid efef7c02-2411-4b92-890e-c07b143aebb2. (Possible split-brain)
[2017-03-17 14:13:05.157391] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:efef7c02-2411-4b92-890e-c07b143aebb2>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:05.157444] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: efef7c02-2411-4b92-890e-c07b143aebb2 [Stale file handle]
[2017-03-17 14:13:05.171241] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-21: Unreadable subvolume -1 found with event generation 6 for gfid d7c19051-4a23-4241-840d-40b8987f0d65. (Possible split-brain)
[2017-03-17 14:13:05.174440] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:d7c19051-4a23-4241-840d-40b8987f0d65>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:05.174490] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: d7c19051-4a23-4241-840d-40b8987f0d65 [Stale file handle]
[2017-03-17 14:13:05.178175] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 84690ec6-6b2b-443e-b7d0-8e38f97d7f99. (Possible split-brain)
[2017-03-17 14:13:05.180669] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:84690ec6-6b2b-443e-b7d0-8e38f97d7f99>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:05.180735] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 84690ec6-6b2b-443e-b7d0-8e38f97d7f99 [Stale file handle]
[2017-03-17 14:13:05.243781] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-10: selecting local read_child vmware2-client-20
[2017-03-17 14:13:05.244375] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-20: selecting local read_child vmware2-client-40
[2017-03-17 14:13:05.244475] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-18: selecting local read_child vmware2-client-36
[2017-03-17 14:13:36.482915] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/vmware.log (hash=vmware2-replicate-17/cache=vmware2-replicate-17) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/vmware-2.log (hash=vmware2-replicate-12/cache=<nul>)
[2017-03-17 14:13:36.567449] I [MSGID: 108031] [afr-common.c:2157:afr_local_discovery_cbk] 17-vmware2-replicate-12: selecting local read_child vmware2-client-24
[2017-03-17 14:13:36.690481] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-13/cache=vmware2-replicate-13) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-13)
[2017-03-17 14:13:36.698007] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-27: remote operation failed [No such file or directory]
[2017-03-17 14:13:36.698481] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:08a54ff3-9025-4d72-bb4f-eaa805be1d7e> (08a54ff3-9025-4d72-bb4f-eaa805be1d7e) [No such file or directory]
[2017-03-17 14:13:36.698560] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:08a54ff3-9025-4d72-bb4f-eaa805be1d7e> (08a54ff3-9025-4d72-bb4f-eaa805be1d7e) [No such file or directory]
[2017-03-17 14:13:36.699192] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 08a54ff3-9025-4d72-bb4f-eaa805be1d7e. (Possible split-brain)
[2017-03-17 14:13:36.699705] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:08a54ff3-9025-4d72-bb4f-eaa805be1d7e> (08a54ff3-9025-4d72-bb4f-eaa805be1d7e) [No such file or directory]
[2017-03-17 14:13:36.699755] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:08a54ff3-9025-4d72-bb4f-eaa805be1d7e> (08a54ff3-9025-4d72-bb4f-eaa805be1d7e) [No such file or directory]
[2017-03-17 14:13:36.701366] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:08a54ff3-9025-4d72-bb4f-eaa805be1d7e> (08a54ff3-9025-4d72-bb4f-eaa805be1d7e) [No such file or directory]
[2017-03-17 14:13:36.701499] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:08a54ff3-9025-4d72-bb4f-eaa805be1d7e> (08a54ff3-9025-4d72-bb4f-eaa805be1d7e) [No such file or directory]
[2017-03-17 14:13:36.702111] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:08a54ff3-9025-4d72-bb4f-eaa805be1d7e>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:36.702165] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 08a54ff3-9025-4d72-bb4f-eaa805be1d7e [Stale file handle]
[2017-03-17 14:13:36.769958] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-13/cache=vmware2-replicate-13) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-13)
[2017-03-17 14:13:36.777749] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-26: remote operation failed [No such file or directory]
[2017-03-17 14:13:36.778171] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:3de3f6e7-aacc-4335-b499-aec73206b816> (3de3f6e7-aacc-4335-b499-aec73206b816) [No such file or directory]
[2017-03-17 14:13:36.778314] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:3de3f6e7-aacc-4335-b499-aec73206b816> (3de3f6e7-aacc-4335-b499-aec73206b816) [No such file or directory]
[2017-03-17 14:13:36.778711] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 3de3f6e7-aacc-4335-b499-aec73206b816. (Possible split-brain)
[2017-03-17 14:13:36.779250] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:3de3f6e7-aacc-4335-b499-aec73206b816> (3de3f6e7-aacc-4335-b499-aec73206b816) [No such file or directory]
[2017-03-17 14:13:36.779365] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:3de3f6e7-aacc-4335-b499-aec73206b816> (3de3f6e7-aacc-4335-b499-aec73206b816) [No such file or directory]
[2017-03-17 14:13:36.781020] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:3de3f6e7-aacc-4335-b499-aec73206b816> (3de3f6e7-aacc-4335-b499-aec73206b816) [No such file or directory]
[2017-03-17 14:13:36.781085] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:3de3f6e7-aacc-4335-b499-aec73206b816> (3de3f6e7-aacc-4335-b499-aec73206b816) [No such file or directory]
[2017-03-17 14:13:36.781652] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:3de3f6e7-aacc-4335-b499-aec73206b816>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:36.781688] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 3de3f6e7-aacc-4335-b499-aec73206b816 [Stale file handle]
[2017-03-17 14:13:45.090726] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmdk~ (hash=vmware2-replicate-17/cache=vmware2-replicate-17) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmdk (hash=vmware2-replicate-19/cache=vmware2-replicate-17)
[2017-03-17 14:13:45.099070] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-34: remote operation failed [No such file or directory]
[2017-03-17 14:13:45.099540] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-34: remote operation failed. Path: <gfid:dea8822e-c75e-4b02-acbf-651bf115a0be> (dea8822e-c75e-4b02-acbf-651bf115a0be) [No such file or directory]
[2017-03-17 14:13:45.099652] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-35: remote operation failed. Path: <gfid:dea8822e-c75e-4b02-acbf-651bf115a0be> (dea8822e-c75e-4b02-acbf-651bf115a0be) [No such file or directory]
[2017-03-17 14:13:45.100337] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid dea8822e-c75e-4b02-acbf-651bf115a0be. (Possible split-brain)
[2017-03-17 14:13:45.100838] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-34: remote operation failed. Path: <gfid:dea8822e-c75e-4b02-acbf-651bf115a0be> (dea8822e-c75e-4b02-acbf-651bf115a0be) [No such file or directory]
[2017-03-17 14:13:45.100969] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-35: remote operation failed. Path: <gfid:dea8822e-c75e-4b02-acbf-651bf115a0be> (dea8822e-c75e-4b02-acbf-651bf115a0be) [No such file or directory]
[2017-03-17 14:13:45.103054] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-35: remote operation failed. Path: <gfid:dea8822e-c75e-4b02-acbf-651bf115a0be> (dea8822e-c75e-4b02-acbf-651bf115a0be) [No such file or directory]
[2017-03-17 14:13:45.103089] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-34: remote operation failed. Path: <gfid:dea8822e-c75e-4b02-acbf-651bf115a0be> (dea8822e-c75e-4b02-acbf-651bf115a0be) [No such file or directory]
[2017-03-17 14:13:45.103524] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:dea8822e-c75e-4b02-acbf-651bf115a0be>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:45.103581] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: dea8822e-c75e-4b02-acbf-651bf115a0be [Stale file handle]
[2017-03-17 14:13:45.108143] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-1: Unreadable subvolume -1 found with event generation 2 for gfid ffa47c2b-0bb6-4a46-b77a-c6c932f5eb2b. (Possible split-brain)
[2017-03-17 14:13:45.110694] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:ffa47c2b-0bb6-4a46-b77a-c6c932f5eb2b>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:13:45.110747] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: ffa47c2b-0bb6-4a46-b77a-c6c932f5eb2b [Stale file handle]
[2017-03-17 14:23:36.832918] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmdk~ (hash=vmware2-replicate-17/cache=vmware2-replicate-17) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmdk (hash=vmware2-replicate-19/cache=vmware2-replicate-17)
[2017-03-17 14:23:36.843300] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-35: remote operation failed [No such file or directory]
[2017-03-17 14:23:36.843832] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-34: remote operation failed. Path: <gfid:54645d52-46e1-49a9-b595-0d693088defa> (54645d52-46e1-49a9-b595-0d693088defa) [No such file or directory]
[2017-03-17 14:23:36.843876] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-35: remote operation failed. Path: <gfid:54645d52-46e1-49a9-b595-0d693088defa> (54645d52-46e1-49a9-b595-0d693088defa) [No such file or directory]
[2017-03-17 14:23:36.844557] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-17: Unreadable subvolume -1 found with event generation 2 for gfid 54645d52-46e1-49a9-b595-0d693088defa. (Possible split-brain)
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-35: remote operation failed. Path: <gfid:54645d52-46e1-49a9-b595-0d693088defa> (54645d52-46e1-49a9-b595-0d693088defa) [No such file or directory]" repeated 2 times between [2017-03-17 14:23:36.843876] and [2017-03-17 14:23:36.847542]
[2017-03-17 14:23:36.847963] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:54645d52-46e1-49a9-b595-0d693088defa>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:23:36.848042] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 54645d52-46e1-49a9-b595-0d693088defa [Stale file handle]
[2017-03-17 14:23:36.852826] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 45f08e6f-7687-4aa3-9e19-a8bd416ca419. (Possible split-brain)
[2017-03-17 14:23:36.855367] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:45f08e6f-7687-4aa3-9e19-a8bd416ca419>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:23:36.855414] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 45f08e6f-7687-4aa3-9e19-a8bd416ca419 [Stale file handle]
[2017-03-17 14:23:36.885169] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-13/cache=vmware2-replicate-13) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-13)
[2017-03-17 14:23:36.892803] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-27: remote operation failed [No such file or directory]
[2017-03-17 14:23:36.893291] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:3ffd4dc6-83c4-483b-bffe-80b411cfd848> (3ffd4dc6-83c4-483b-bffe-80b411cfd848) [No such file or directory]
[2017-03-17 14:23:36.893360] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:3ffd4dc6-83c4-483b-bffe-80b411cfd848> (3ffd4dc6-83c4-483b-bffe-80b411cfd848) [No such file or directory]
[2017-03-17 14:23:36.893821] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 3ffd4dc6-83c4-483b-bffe-80b411cfd848. (Possible split-brain)
[2017-03-17 14:23:36.894296] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:3ffd4dc6-83c4-483b-bffe-80b411cfd848> (3ffd4dc6-83c4-483b-bffe-80b411cfd848) [No such file or directory]
[2017-03-17 14:23:36.894420] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:3ffd4dc6-83c4-483b-bffe-80b411cfd848> (3ffd4dc6-83c4-483b-bffe-80b411cfd848) [No such file or directory]
[2017-03-17 14:23:36.896066] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:3ffd4dc6-83c4-483b-bffe-80b411cfd848> (3ffd4dc6-83c4-483b-bffe-80b411cfd848) [No such file or directory]
[2017-03-17 14:23:36.896199] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:3ffd4dc6-83c4-483b-bffe-80b411cfd848> (3ffd4dc6-83c4-483b-bffe-80b411cfd848) [No such file or directory]
[2017-03-17 14:23:36.896923] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:3ffd4dc6-83c4-483b-bffe-80b411cfd848>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:23:36.896995] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 3ffd4dc6-83c4-483b-bffe-80b411cfd848 [Stale file handle]
[2017-03-17 14:23:36.961014] I [MSGID: 109066] [dht-rename.c:1569:dht_rename] 17-vmware2-dht: renaming <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx~ (hash=vmware2-replicate-13/cache=vmware2-replicate-13) => <gfid:04b8b83a-25db-46d5-bdb8-08e2be1efe53>/New Virtual Machine 5.vmx (hash=vmware2-replicate-3/cache=vmware2-replicate-13)
[2017-03-17 14:23:36.967930] W [MSGID: 114031] [client-rpc-fops.c:504:client3_3_stat_cbk] 17-vmware2-client-26: remote operation failed [No such file or directory]
[2017-03-17 14:23:36.968359] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:74d49288-8452-40d4-893e-ff4672557ff9> (74d49288-8452-40d4-893e-ff4672557ff9) [No such file or directory]
[2017-03-17 14:23:36.968471] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:74d49288-8452-40d4-893e-ff4672557ff9> (74d49288-8452-40d4-893e-ff4672557ff9) [No such file or directory]
[2017-03-17 14:23:36.968883] W [MSGID: 108008] [afr-read-txn.c:228:afr_read_txn] 17-vmware2-replicate-13: Unreadable subvolume -1 found with event generation 2 for gfid 74d49288-8452-40d4-893e-ff4672557ff9. (Possible split-brain)
[2017-03-17 14:23:36.969331] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:74d49288-8452-40d4-893e-ff4672557ff9> (74d49288-8452-40d4-893e-ff4672557ff9) [No such file or directory]
[2017-03-17 14:23:36.969419] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:74d49288-8452-40d4-893e-ff4672557ff9> (74d49288-8452-40d4-893e-ff4672557ff9) [No such file or directory]
[2017-03-17 14:23:36.970875] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-27: remote operation failed. Path: <gfid:74d49288-8452-40d4-893e-ff4672557ff9> (74d49288-8452-40d4-893e-ff4672557ff9) [No such file or directory]
[2017-03-17 14:23:36.970884] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-26: remote operation failed. Path: <gfid:74d49288-8452-40d4-893e-ff4672557ff9> (74d49288-8452-40d4-893e-ff4672557ff9) [No such file or directory]
[2017-03-17 14:23:36.971435] E [MSGID: 109040] [dht-helper.c:1198:dht_migration_complete_check_task] 17-vmware2-dht: <gfid:74d49288-8452-40d4-893e-ff4672557ff9>: failed to lookup the file on vmware2-dht [Stale file handle]
[2017-03-17 14:23:36.971501] E [MSGID: 133014] [shard.c:1253:shard_common_stat_cbk] 17-vmware2-shard: stat failed: 74d49288-8452-40d4-893e-ff4672557ff9 [Stale file handle]
The message "W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 17-vmware2-client-34: remote operation failed. Path: <gfid:54645d52-46e1-49a9-b595-0d693088defa> (54645d52-46e1-49a9-b595-0d693088defa) [No such file or directory]" repeated 2 times between [2017-03-17 14:23:36.843832] and [2017-03-17 14:23:36.847417]
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users