Re: Input/output error on FUSE log

[Matt Waymack <mwaymack@xxxxxxxxx>]

Attached are the logs from when a failure occurred with diagnostics set to trace.

 

Thank you!

 

From: Raghavendra Gowdappa <rgowdapp@xxxxxxxxxx>
Sent: Saturday, January 5, 2019 8:32 PM
To: Matt Waymack <mwaymack@xxxxxxxxx>
Cc: gluster-users@xxxxxxxxxxx List <gluster-users@xxxxxxxxxxx>
Subject: Re: [Gluster-users] Input/output error on FUSE log

 

 

 

On Sun, Jan 6, 2019 at 7:58 AM Raghavendra Gowdappa <rgowdapp@xxxxxxxxxx> wrote:

 

 

On Sun, Jan 6, 2019 at 4:19 AM Matt Waymack <mwaymack@xxxxxxxxx> wrote:

Hi all,

 

I'm having a problem writing to our volume.  When writing files larger than about 2GB, I get an intermittent issue where the write will fail and return Input/Output error.  This is also shown in the FUSE log of the client (this is affecting all clients).  A snip of a client log is below:

[2019-01-05 22:39:44.581371] W [fuse-bridge.c:2474:fuse_writev_cbk] 0-glusterfs-fuse: 51040978: WRITE => -1 gfid=82a0b5c4-7ef3-43c2-ad86-41e16673d7c2 fd=0x7f949839a368 (Input/output error)

[2019-01-05 22:39:44.598392] W [fuse-bridge.c:1441:fuse_err_cbk] 0-glusterfs-fuse: 51040979: FLUSH() ERR => -1 (Input/output error)

[2019-01-05 22:39:47.420920] W [fuse-bridge.c:2474:fuse_writev_cbk] 0-glusterfs-fuse: 51041266: WRITE => -1 gfid=0e8e1e13-97a5-478a-bc58-e81ddf3698a3 fd=0x7f949809b7f8 (Input/output error)

[2019-01-05 22:39:47.433377] W [fuse-bridge.c:1441:fuse_err_cbk] 0-glusterfs-fuse: 51041267: FLUSH() ERR => -1 (Input/output error)

[2019-01-05 22:39:50.441531] W [fuse-bridge.c:2474:fuse_writev_cbk] 0-glusterfs-fuse: 51041548: WRITE => -1 gfid=0e8e1e13-97a5-478a-bc58-e81ddf3698a3 fd=0x7f949839a368 (Input/output error)

[2019-01-05 22:39:50.451914] W [fuse-bridge.c:1441:fuse_err_cbk] 0-glusterfs-fuse: 51041549: FLUSH() ERR => -1 (Input/output error)

The message "W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1311504267" repeated 1721 times between [2019-01-05 22:39:33.906241] and [2019-01-05 22:39:44.598371]

The message "E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null" repeated 1714 times between [2019-01-05 22:39:33.925981] and [2019-01-05 22:39:50.451862]

The message "W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1137142622" repeated 1707 times between [2019-01-05 22:39:39.636552] and [2019-01-05 22:39:50.451895]

 

This looks to be a DHT issue. Some questions:

* Are all subvolumes of DHT up and client is connected to them? Particularly the subvolume which contains the file in question.

* Can you get all extended attributes of parent directory of the file from all bricks?

* set diagnostics.client-log-level to TRACE, capture these errors again and attach the client log file.

 

I spoke a bit early. dht_writev doesn't search hashed subvolume as its already been looked up in lookup. So, these msgs looks to be of a different issue - not  writev failure.

 

 

This is intermittent for most files, but eventually if a file is large enough it will not write.  The workflow is SFTP tot he client which then writes to the volume over FUSE.  When files get to a certain point,w e can no longer write to them.  The file sizes are different as well, so it's not like they all get to the same size and just stop either.  I've ruled out a free space issue, our files at their largest are only a few hundred GB and we have tens of terrabytes free on each brick.  We are also sharding at 1GB.

 

I'm not sure where to go from here as the error seems vague and I can only see it on the client log.  I'm not seeing these errors on the nodes themselves.  This is also seen if I mount the volume via FUSE on any of the nodes as well and it is only reflected in the FUSE log.

 

Here is the volume info:

Volume Name: gv1

Type: Distributed-Replicate

Volume ID: 1472cc78-e2a0-4c3f-9571-dab840239b3c

Status: Started

Snapshot Count: 0

Number of Bricks: 8 x (2 + 1) = 24

Transport-type: tcp

Bricks:

Brick1: tpc-glus4:/exp/b1/gv1

Brick2: tpc-glus2:/exp/b1/gv1

Brick3: tpc-arbiter1:/exp/b1/gv1 (arbiter)

Brick4: tpc-glus2:/exp/b2/gv1

Brick5: tpc-glus4:/exp/b2/gv1

Brick6: tpc-arbiter1:/exp/b2/gv1 (arbiter)

Brick7: tpc-glus4:/exp/b3/gv1

Brick8: tpc-glus2:/exp/b3/gv1

Brick9: tpc-arbiter1:/exp/b3/gv1 (arbiter)

Brick10: tpc-glus4:/exp/b4/gv1

Brick11: tpc-glus2:/exp/b4/gv1

Brick12: tpc-arbiter1:/exp/b4/gv1 (arbiter)

Brick13: tpc-glus1:/exp/b5/gv1

Brick14: tpc-glus3:/exp/b5/gv1

Brick15: tpc-arbiter2:/exp/b5/gv1 (arbiter)

Brick16: tpc-glus1:/exp/b6/gv1

Brick17: tpc-glus3:/exp/b6/gv1

Brick18: tpc-arbiter2:/exp/b6/gv1 (arbiter)

Brick19: tpc-glus1:/exp/b7/gv1

Brick20: tpc-glus3:/exp/b7/gv1

Brick21: tpc-arbiter2:/exp/b7/gv1 (arbiter)

Brick22: tpc-glus1:/exp/b8/gv1

Brick23: tpc-glus3:/exp/b8/gv1

Brick24: tpc-arbiter2:/exp/b8/gv1 (arbiter)

Options Reconfigured:

performance.cache-samba-metadata: on

performance.cache-invalidation: off

features.shard-block-size: 1000MB

features.shard: on

transport.address-family: inet

nfs.disable: on

cluster.lookup-optimize: on

 

I'm a bit stumped on this, any help is appreciated.  Thank you!

 

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users

[2019-01-07 19:16:37.015300] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned 0
[2019-01-07 19:16:37.015305] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa0b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:37.015309] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned 0
[2019-01-07 19:16:37.015321] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned 0
[2019-01-07 19:16:37.015330] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned 0
[2019-01-07 19:16:37.015338] T [MSGID: 0] [dht-inode-read.c:312:dht_attr_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned 0
[2019-01-07 19:16:37.015347] T [MSGID: 0] [defaults.c:1385:default_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-shard returned 0
[2019-01-07 19:16:37.015424] T [md-cache.c:491:mdc_inode_iatt_set_validate] (-->/lib64/libglusterfs.so.0(default_stat_cbk+0x15b) [0x7f6767cc172b] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0xc3ae) [0x7f67588c53ae] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0x8286) [0x7f67588c1286] ) 0-md-cache: Updated iatt(00000000-0000-0000-0000-000000000001) time:1546888597 
[2019-01-07 19:16:37.015453] T [fuse-bridge.c:847:fuse_attr_cbk] 0-glusterfs-fuse: 1369: STAT() / => 1
[2019-01-07 19:16:37.015469] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 120/120 
[2019-01-07 19:16:44.793308] T [fuse-bridge.c:946:fuse_getattr_resume] 0-glusterfs-fuse: 1370: GETATTR 1 (/)
[2019-01-07 19:16:44.793432] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [io-stats.c:2109:io_stats_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned 0" repeated 4 times between [2019-01-07 19:15:35.383925] and [2019-01-07 19:16:37.015452]
[2019-01-07 19:16:44.793431] T [MSGID: 0] [fuse-bridge.c:949:fuse_getattr_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from fuse to meta-autoload
[2019-01-07 19:16:44.793478] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from meta-autoload to gv1
[2019-01-07 19:16:44.793489] T [MSGID: 0] [io-stats.c:2803:io_stats_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:44.793501] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: STAT scheduled as fast fop
[2019-01-07 19:16:44.793524] T [MSGID: 0] [defaults.c:2034:default_stat_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:44.793555] T [MSGID: 0] [md-cache.c:519:mdc_inode_iatt_get] 0-md-cache: iatt cache not valid for (00000000-0000-0000-0000-000000000001)
[2019-01-07 19:16:44.793567] T [MSGID: 0] [md-cache.c:1269:mdc_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:44.793577] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:44.793590] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:44.793602] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:44.793610] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:44.793618] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:44.793628] T [MSGID: 0] [write-behind.c:2060:wb_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:44.793639] T [MSGID: 0] [shard.c:1578:shard_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.793653] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.793689] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-0: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.793702] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.793720] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.793750] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.793763] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.793778] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-1: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.793788] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.793798] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.793817] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x957 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.793828] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.793842] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-2: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.793852] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.793862] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.793881] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x953 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.793891] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.793906] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-3: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.793915] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.793925] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.793947] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.793958] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.793976] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-4: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.793985] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.793983] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.793995] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.794028] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned 0
[2019-01-07 19:16:44.794041] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x887 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.794056] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.794058] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned 0
[2019-01-07 19:16:44.794075] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-5: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.794087] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.794098] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.794105] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x953 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.794123] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.794123] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned 0
[2019-01-07 19:16:44.794138] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.794157] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-6: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.794155] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned 0
[2019-01-07 19:16:44.794175] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.794193] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.794192] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa0b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.794224] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned 0
[2019-01-07 19:16:44.794233] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa70 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.794247] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x887 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.794273] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned 0
[2019-01-07 19:16:44.794272] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.794327] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned 0
[2019-01-07 19:16:44.794335] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-7: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:44.794353] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.794354] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned 0
[2019-01-07 19:16:44.794328] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x957 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.794374] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 116, payload: 28, rpc hdr: 88
[2019-01-07 19:16:44.794403] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.794429] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned 0
[2019-01-07 19:16:44.794440] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.794430] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned 0
[2019-01-07 19:16:44.794447] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned 0
[2019-01-07 19:16:44.794469] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned 0
[2019-01-07 19:16:44.794492] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa70 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.794505] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned 0
[2019-01-07 19:16:44.794519] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned 0
[2019-01-07 19:16:44.794640] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa0c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.794654] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned 0
[2019-01-07 19:16:44.794664] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned 0
[2019-01-07 19:16:44.794672] T [MSGID: 0] [dht-inode-read.c:312:dht_attr_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned 0
[2019-01-07 19:16:44.794733] T [md-cache.c:491:mdc_inode_iatt_set_validate] (-->/lib64/libglusterfs.so.0(default_stat_cbk+0x15b) [0x7f6767cc172b] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0xc3ae) [0x7f67588c53ae] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0x8286) [0x7f67588c1286] ) 0-md-cache: Updated iatt(00000000-0000-0000-0000-000000000001) time:1546888604 
[2019-01-07 19:16:44.794760] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [defaults.c:1385:default_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-threads returned 0" repeated 4 times between [2019-01-07 19:15:51.137455] and [2019-01-07 19:16:44.794758]
[2019-01-07 19:16:44.794760] T [MSGID: 0] [io-stats.c:2109:io_stats_stat_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned 0
[2019-01-07 19:16:44.794786] T [fuse-bridge.c:847:fuse_attr_cbk] 0-glusterfs-fuse: 1370: STAT() / => 1
[2019-01-07 19:16:44.794800] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 120/120 
[2019-01-07 19:16:44.794979] T [fuse-bridge.c:676:fuse_lookup_resume] 0-glusterfs-fuse: 1371: LOOKUP /test.Big(aa3ef10e-95e0-40d3-9464-133d72fa8a95)
[2019-01-07 19:16:44.794998] T [MSGID: 0] [fuse-bridge.c:689:fuse_lookup_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from fuse to meta-autoload
[2019-01-07 19:16:44.795012] T [MSGID: 0] [defaults.c:2716:default_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from meta-autoload to gv1
[2019-01-07 19:16:44.795022] T [MSGID: 0] [io-stats.c:2790:io_stats_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:44.795033] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: LOOKUP scheduled as fast fop
[2019-01-07 19:16:44.795069] T [MSGID: 0] [defaults.c:1974:default_lookup_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:44.795117] T [MSGID: 0] [md-cache.c:519:mdc_inode_iatt_get] 0-md-cache: iatt cache not valid for (aa3ef10e-95e0-40d3-9464-133d72fa8a95)
[2019-01-07 19:16:44.795166] T [MSGID: 0] [md-cache.c:1200:mdc_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:44.795188] T [MSGID: 0] [defaults.c:2716:default_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:44.795206] T [MSGID: 0] [quick-read.c:514:qr_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:44.795216] T [MSGID: 0] [io-cache.c:298:ioc_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:44.795227] T [MSGID: 0] [defaults.c:2716:default_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:44.795237] T [MSGID: 0] [defaults.c:2716:default_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:44.795246] T [MSGID: 0] [write-behind.c:2442:wb_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:44.795260] T [MSGID: 0] [shard.c:1386:shard_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.795279] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for test.Big
[2019-01-07 19:16:44.795315] D [MSGID: 0] [dht-common.c:3393:dht_lookup] 0-gv1-dht: calling revalidate lookup for /test.Big at gv1-replicate-0
[2019-01-07 19:16:44.795325] T [MSGID: 0] [dht-common.c:3397:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.795358] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.795383] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 1044, payload: 956, rpc hdr: 88
[2019-01-07 19:16:44.795414] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.795431] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.795455] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 1044, payload: 956, rpc hdr: 88
[2019-01-07 19:16:44.795481] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.795498] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.795520] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 1044, payload: 956, rpc hdr: 88
[2019-01-07 19:16:44.795541] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.795766] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.795874] T [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned 0
[2019-01-07 19:16:44.796037] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.796071] T [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned 0
[2019-01-07 19:16:44.796617] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.201:49159: ping timer event already removed
[2019-01-07 19:16:44.796646] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.796652] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.796671] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x409 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.796684] T [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned 0
[2019-01-07 19:16:44.796723] T [MSGID: 0] [afr-common.c:2507:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned 0
[2019-01-07 19:16:44.796743] D [MSGID: 0] [dht-common.c:1593:dht_revalidate_cbk] 0-gv1-dht: revalidate lookup of /test.Big returned with op_ret 0
[2019-01-07 19:16:44.796764] T [MSGID: 0] [dht-common.c:1909:dht_revalidate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned 0
[2019-01-07 19:16:44.796780] T [MSGID: 0] [shard.c:1329:shard_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-shard returned 0
[2019-01-07 19:16:44.796795] T [MSGID: 0] [write-behind.c:2396:wb_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-write-behind returned 0
[2019-01-07 19:16:44.796809] T [MSGID: 0] [io-cache.c:207:ioc_inode_update] 0-gv1-io-cache: locked inode(0x7f6744961190)
[2019-01-07 19:16:44.796824] T [MSGID: 0] [io-cache.c:216:ioc_inode_update] 0-gv1-io-cache: unlocked inode(0x7f6744961190)
[2019-01-07 19:16:44.796836] T [MSGID: 0] [io-cache.c:224:ioc_inode_update] 0-gv1-io-cache: locked table(0x7f6754085360)
[2019-01-07 19:16:44.796848] T [MSGID: 0] [io-cache.c:229:ioc_inode_update] 0-gv1-io-cache: unlocked table(0x7f6754085360)
[2019-01-07 19:16:44.796862] T [MSGID: 0] [io-cache.c:268:ioc_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-cache returned 0
[2019-01-07 19:16:44.796882] T [MSGID: 0] [quick-read.c:473:qr_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-quick-read returned 0
[2019-01-07 19:16:44.796920] T [md-cache.c:491:mdc_inode_iatt_set_validate] (-->/usr/lib64/glusterfs/4.1.6/xlator/performance/quick-read.so(+0x5185) [0x7f6758ce9185] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0xbf12) [0x7f67588c4f12] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0x8286) [0x7f67588c1286] ) 0-md-cache: Updated iatt(00000000-0000-0000-0000-000000000001) time:1546888604 
[2019-01-07 19:16:44.796948] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x409 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.796954] T [md-cache.c:491:mdc_inode_iatt_set_validate] (-->/usr/lib64/glusterfs/4.1.6/xlator/performance/quick-read.so(+0x5185) [0x7f6758ce9185] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0xbf2b) [0x7f67588c4f2b] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0x8286) [0x7f67588c1286] ) 0-md-cache: Updated iatt(aa3ef10e-95e0-40d3-9464-133d72fa8a95) time:1546888604 
[2019-01-07 19:16:44.796970] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-2: Ping latency is 0ms
[2019-01-07 19:16:44.796992] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.open-fd-count doesn't satisfy caching requirements
[2019-01-07 19:16:44.797009] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.open-fd-count doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.796992] and [2019-01-07 19:16:44.797007]
[2019-01-07 19:16:44.797009] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key trusted.glusterfs.shard.file-size doesn't satisfy caching requirements
[2019-01-07 19:16:44.797039] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key trusted.glusterfs.shard.file-size doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797009] and [2019-01-07 19:16:44.797038]
[2019-01-07 19:16:44.797039] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key trusted.glusterfs.shard.block-size doesn't satisfy caching requirements
[2019-01-07 19:16:44.797064] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key trusted.glusterfs.shard.block-size doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797039] and [2019-01-07 19:16:44.797063]
[2019-01-07 19:16:44.797064] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.parent-entrylk doesn't satisfy caching requirements
[2019-01-07 19:16:44.797088] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.parent-entrylk doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797064] and [2019-01-07 19:16:44.797087]
[2019-01-07 19:16:44.797088] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.entrylk-count doesn't satisfy caching requirements
[2019-01-07 19:16:44.797112] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.entrylk-count doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797088] and [2019-01-07 19:16:44.797111]
[2019-01-07 19:16:44.797112] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.inodelk-count doesn't satisfy caching requirements
[2019-01-07 19:16:44.797139] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key glusterfs.inodelk-count doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797112] and [2019-01-07 19:16:44.797138]
[2019-01-07 19:16:44.797139] T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key link-count doesn't satisfy caching requirements
[2019-01-07 19:16:44.797164] D [logging.c:1983:_gf_msg_internal] 0-logging-infra: Buffer overflow of a buffer whose size limit is 5. About to flush least recently used log message to disk
The message "T [MSGID: 0] [md-cache.c:575:is_mdc_key_satisfied] 0-md-cache: xattr key link-count doesn't satisfy caching requirements" repeated 7 times between [2019-01-07 19:16:44.797139] and [2019-01-07 19:16:44.797161]
[2019-01-07 19:16:44.797164] T [MSGID: 0] [md-cache.c:685:mdc_inode_xatt_set] 0-md-cache: xatt cache set for (aa3ef10e-95e0-40d3-9464-133d72fa8a95) time:1546888604
[2019-01-07 19:16:44.797182] T [MSGID: 0] [md-cache.c:1130:mdc_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-md-cache returned 0
[2019-01-07 19:16:44.797190] T [MSGID: 0] [defaults.c:1328:default_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-threads returned 0
[2019-01-07 19:16:44.797198] T [MSGID: 0] [io-stats.c:2278:io_stats_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned 0
[2019-01-07 19:16:44.797205] T [fuse-bridge.c:518:fuse_entry_cbk] 0-glusterfs-fuse: 1371: LOOKUP() /test.Big => 10692692569927355029
[2019-01-07 19:16:44.797217] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 144/144 
[2019-01-07 19:16:44.797429] T [fuse-bridge.c:2327:fuse_open_resume] 0-glusterfs-fuse: 1372: OPEN /test.Big
[2019-01-07 19:16:44.797445] T [MSGID: 0] [fuse-bridge.c:2330:fuse_open_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from fuse to meta-autoload
[2019-01-07 19:16:44.797455] T [MSGID: 0] [defaults.c:2537:default_open] 0-stack-trace: stack-address: 0x7f6748006778, winding from meta-autoload to gv1
[2019-01-07 19:16:44.797463] T [MSGID: 0] [io-stats.c:2963:io_stats_open] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:44.797473] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: OPEN scheduled as fast fop
[2019-01-07 19:16:44.797493] T [MSGID: 0] [defaults.c:1828:default_open_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:44.797512] T [MSGID: 0] [md-cache.c:1924:mdc_open] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:44.797529] T [MSGID: 0] [open-behind.c:290:ob_open_behind] 0-stack-trace: stack-address: 0x7f6748006778, gv1-open-behind returned 0
[2019-01-07 19:16:44.797540] T [MSGID: 0] [md-cache.c:1901:mdc_open_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-md-cache returned 0
[2019-01-07 19:16:44.797549] T [MSGID: 0] [defaults.c:1199:default_open_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-threads returned 0
[2019-01-07 19:16:44.797560] T [MSGID: 0] [io-stats.c:2098:io_stats_open_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned 0
[2019-01-07 19:16:44.797567] T [fuse-bridge.c:1091:fuse_fd_cbk] 0-glusterfs-fuse: 1372: OPEN() /test.Big => 0x7f6748005418
[2019-01-07 19:16:44.797579] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 32/32 
[2019-01-07 19:16:44.797601] T [fuse-bridge.c:3120:fuse_fallocate_resume] 0-glusterfs-fuse: 1373: FALLOCATE (0x7f6748005418, flags=0, size=214748364800, offset=0)
[2019-01-07 19:16:44.797613] T [MSGID: 0] [fuse-bridge.c:3128:fuse_fallocate_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from fuse to meta-autoload
[2019-01-07 19:16:44.797622] T [MSGID: 0] [defaults.c:2748:default_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from meta-autoload to gv1
[2019-01-07 19:16:44.797629] T [MSGID: 0] [io-stats.c:3512:io_stats_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:44.797638] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: FALLOCATE scheduled as slow fop
[2019-01-07 19:16:44.797651] T [MSGID: 0] [defaults.c:2000:default_fallocate_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:44.797662] T [MSGID: 0] [md-cache.c:2730:mdc_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:44.797671] T [MSGID: 0] [open-behind.c:207:ob_fd_wake] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:44.797680] T [MSGID: 0] [quick-read.c:742:qr_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:44.797690] T [MSGID: 0] [io-cache.c:842:ioc_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:44.797698] T [MSGID: 0] [defaults.c:2537:default_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:44.797706] T [MSGID: 0] [read-ahead.c:190:ra_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:44.797713] T [MSGID: 0] [defaults.c:2537:default_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:44.797720] T [MSGID: 0] [shard.c:3459:shard_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.797734] T [MSGID: 0] [dht-inode-read.c:139:dht_open] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.797754] T [MSGID: 0] [afr-open.c:138:afr_open_continue] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.797767] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 120, payload: 32, rpc hdr: 88
[2019-01-07 19:16:44.797790] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.797801] T [MSGID: 0] [afr-open.c:138:afr_open_continue] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.797811] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 120, payload: 32, rpc hdr: 88
[2019-01-07 19:16:44.797834] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.797843] T [MSGID: 0] [afr-open.c:138:afr_open_continue] 0-stack-trace: stack-address: 0x7f674c0233e8, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.797853] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 120, payload: 32, rpc hdr: 88
[2019-01-07 19:16:44.797867] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.798094] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.798114] T [MSGID: 0] [client-rpc-fops_v2.c:276:client4_0_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-client-1 returned 0
[2019-01-07 19:16:44.798217] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.798211] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 11) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.798240] T [MSGID: 0] [client-rpc-fops_v2.c:276:client4_0_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-client-2 returned 0
[2019-01-07 19:16:44.798253] T [MSGID: 0] [client-rpc-fops_v2.c:276:client4_0_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-client-0 returned 0
[2019-01-07 19:16:44.798267] T [MSGID: 0] [afr-open.c:103:afr_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-replicate-0 returned 0
[2019-01-07 19:16:44.798282] T [MSGID: 0] [dht-inode-read.c:66:dht_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-dht returned 0
[2019-01-07 19:16:44.798291] T [MSGID: 0] [shard.c:3450:shard_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-shard returned 0
[2019-01-07 19:16:44.798312] T [MSGID: 0] [read-ahead.c:101:ra_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-read-ahead returned 0
[2019-01-07 19:16:44.798321] T [MSGID: 0] [io-cache.c:579:ioc_open_cbk] 0-gv1-io-cache: locked table(0x7f6754085360)
[2019-01-07 19:16:44.798328] T [MSGID: 0] [io-cache.c:584:ioc_open_cbk] 0-gv1-io-cache: unlocked table(0x7f6754085360)
[2019-01-07 19:16:44.798335] T [MSGID: 0] [io-cache.c:586:ioc_open_cbk] 0-gv1-io-cache: locked inode(0x7f6744961190)
[2019-01-07 19:16:44.798341] T [MSGID: 0] [io-cache.c:594:ioc_open_cbk] 0-gv1-io-cache: unlocked inode(0x7f6744961190)
[2019-01-07 19:16:44.798348] T [MSGID: 0] [io-cache.c:609:ioc_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-io-cache returned 0
[2019-01-07 19:16:44.798354] T [MSGID: 0] [defaults.c:1199:default_open_cbk] 0-stack-trace: stack-address: 0x7f674c0233e8, gv1-quick-read returned 0
[2019-01-07 19:16:44.798365] T [MSGID: 0] [defaults.c:2000:default_fallocate_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:44.798373] T [MSGID: 0] [quick-read.c:706:qr_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:44.798381] T [MSGID: 0] [defaults.c:2748:default_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:44.798388] T [MSGID: 0] [defaults.c:2748:default_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:44.798407] T [MSGID: 0] [defaults.c:2748:default_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:44.798419] T [MSGID: 0] [write-behind.c:2620:wb_fallocate] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:44.798446] D [MSGID: 0] [shard.c:1468:shard_lookup_base_file] 0-gv1-shard: Skipping lookup on base file: aa3ef10e-95e0-40d3-9464-133d72fa8a95Serving prebuf off the inode ctx cache
[2019-01-07 19:16:44.798465] T [MSGID: 0] [shard.c:4458:shard_common_inode_write_post_lookup_handler] 0-gv1-shard: FALLOCATE: gfid=aa3ef10e-95e0-40d3-9464-133d72fa8a95 first_block=0 last_block=204 num_blocks=205 offset=0 total_size=214748364800 flags=0
[2019-01-07 19:16:44.798487] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 1 already present. gfid=e9d8d28c-cf70-40fd-adb9-59463d2abf93. Saving inode for future.
[2019-01-07 19:16:44.798510] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 3 already present. gfid=6e9b581f-fe29-4039-9789-c08fcb9d26af. Saving inode for future.
[2019-01-07 19:16:44.798528] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 4 already present. gfid=2cb4185d-9f39-44cb-9f21-3c24ac123888. Saving inode for future.
[2019-01-07 19:16:44.798553] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 6 already present. gfid=26ef307b-0857-4311-9c0f-ef2dcc044048. Saving inode for future.
[2019-01-07 19:16:44.798570] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 7 already present. gfid=03355c43-26d5-418e-ab8f-f0b254a45acb. Saving inode for future.
[2019-01-07 19:16:44.798586] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 8 already present. gfid=efe89301-5eb4-4cdd-9225-90a36db433da. Saving inode for future.
[2019-01-07 19:16:44.798606] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 10 already present. gfid=0bc3064b-214d-468e-ac2c-570ba4b1dc5b. Saving inode for future.
[2019-01-07 19:16:44.798624] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 11 already present. gfid=b19ed2a4-0a12-4c5d-b328-8ccba5e97759. Saving inode for future.
[2019-01-07 19:16:44.798641] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 12 already present. gfid=8aa37d0d-8e18-4d3e-9a4c-5570b5041396. Saving inode for future.
[2019-01-07 19:16:44.798659] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 13 already present. gfid=20013878-944f-4523-bb3f-6496039a438e. Saving inode for future.
[2019-01-07 19:16:44.798677] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 14 already present. gfid=2db14d0d-a1ea-46f8-8c9f-b91408408b42. Saving inode for future.
[2019-01-07 19:16:44.798694] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 15 already present. gfid=751edbbe-a4bc-438d-b8cd-c933eeb5b232. Saving inode for future.
[2019-01-07 19:16:44.798711] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 16 already present. gfid=ba8a25df-9aac-47d1-b599-29a02777722e. Saving inode for future.
[2019-01-07 19:16:44.798728] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 17 already present. gfid=d13aa81e-d1ff-4fa7-b85d-05fc998eaeae. Saving inode for future.
[2019-01-07 19:16:44.798743] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 18 already present. gfid=47c4938e-6fc2-42c4-853d-623776d09468. Saving inode for future.
[2019-01-07 19:16:44.798760] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 19 already present. gfid=5acb2262-f37c-4791-b0d6-9d614a15fdff. Saving inode for future.
[2019-01-07 19:16:44.798778] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 20 already present. gfid=37730223-8d8b-4fbc-b7b3-ac7d9620f926. Saving inode for future.
[2019-01-07 19:16:44.798798] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 22 already present. gfid=944da85a-f60c-4754-89a8-fc4e20ba7ef2. Saving inode for future.
[2019-01-07 19:16:44.798815] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 23 already present. gfid=aa57b991-697b-466f-bf09-794320feea85. Saving inode for future.
[2019-01-07 19:16:44.798833] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 24 already present. gfid=5a93724c-9427-49c9-a7c5-d0e9cd3302b4. Saving inode for future.
[2019-01-07 19:16:44.798849] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 25 already present. gfid=7eabbcd6-d263-4ce2-ac55-f0e42a43df6d. Saving inode for future.
[2019-01-07 19:16:44.798866] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 26 already present. gfid=d4c6c4f6-aef4-4bdf-ab07-ab35e45f5a4e. Saving inode for future.
[2019-01-07 19:16:44.798883] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 27 already present. gfid=37fae235-363a-4689-9edb-e72615350bc8. Saving inode for future.
[2019-01-07 19:16:44.798900] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 28 already present. gfid=0e0dbb81-7b5d-402e-9373-426177ccaed5. Saving inode for future.
[2019-01-07 19:16:44.798922] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 30 already present. gfid=4404ba1e-178c-45dc-aea8-6585a76ca34c. Saving inode for future.
[2019-01-07 19:16:44.798943] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 32 already present. gfid=78ae325f-0bd3-46e6-8656-2b163e1d3eea. Saving inode for future.
[2019-01-07 19:16:44.798961] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 33 already present. gfid=5754dae2-aff1-48d0-a203-6a7720bc93aa. Saving inode for future.
[2019-01-07 19:16:44.798978] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 34 already present. gfid=0d95e301-7cad-4276-9449-9fd2c6c89db1. Saving inode for future.
[2019-01-07 19:16:44.798998] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 36 already present. gfid=8cf0e3cb-c39c-4325-b716-cebb487a9539. Saving inode for future.
[2019-01-07 19:16:44.799015] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 37 already present. gfid=fb1738c2-702f-45ee-8bfc-1e879fe5d300. Saving inode for future.
[2019-01-07 19:16:44.799029] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 38 already present. gfid=4a433e4f-368e-459a-91f2-7156bde6cba3. Saving inode for future.
[2019-01-07 19:16:44.799046] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 39 already present. gfid=dd538fdf-3bab-4038-a961-3a25bdd1d08d. Saving inode for future.
[2019-01-07 19:16:44.799064] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 41 already present. gfid=07ea2519-b445-4539-a3cb-affcf65a4d5a. Saving inode for future.
[2019-01-07 19:16:44.799080] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 42 already present. gfid=276ce4ad-b77c-4f2e-a344-24416d911854. Saving inode for future.
[2019-01-07 19:16:44.799096] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 43 already present. gfid=c8c67105-ef41-43b3-8c28-60371a3d8614. Saving inode for future.
[2019-01-07 19:16:44.799108] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 44 already present. gfid=8c6ad80e-b6ab-4f22-abaf-9a6f49caa199. Saving inode for future.
[2019-01-07 19:16:44.799125] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 45 already present. gfid=1a496b18-e3de-40fa-97c8-59fe3fd7555e. Saving inode for future.
[2019-01-07 19:16:44.799141] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 46 already present. gfid=bf2abbf4-5fb5-4d1c-9214-70257728b0f7. Saving inode for future.
[2019-01-07 19:16:44.799157] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 47 already present. gfid=41842ebb-97c2-4a2d-82e1-a5c0e3624bf5. Saving inode for future.
[2019-01-07 19:16:44.799175] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 48 already present. gfid=0d81d667-b230-41e1-8c28-d2a056797cf6. Saving inode for future.
[2019-01-07 19:16:44.799192] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 49 already present. gfid=8b155c3c-9cab-4cd6-8ca7-d132a4f75bb2. Saving inode for future.
[2019-01-07 19:16:44.799207] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 50 already present. gfid=914ce9bc-fc7d-49bc-b80e-ffd8993a1df3. Saving inode for future.
[2019-01-07 19:16:44.799222] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 51 already present. gfid=bfb436c6-6921-425d-a5b6-3a4936cb0f45. Saving inode for future.
[2019-01-07 19:16:44.799236] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 52 already present. gfid=f1feb40f-e64d-4e2c-b132-a67493f87372. Saving inode for future.
[2019-01-07 19:16:44.799253] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 53 already present. gfid=a9382161-252b-41d0-87f2-53c4057736eb. Saving inode for future.
[2019-01-07 19:16:44.799273] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 54 already present. gfid=df4968a0-7c97-4ac2-a17a-423e1fd96146. Saving inode for future.
[2019-01-07 19:16:44.799291] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 55 already present. gfid=8179a7e5-4cde-40bb-9388-6eeddeea0daa. Saving inode for future.
[2019-01-07 19:16:44.799308] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 56 already present. gfid=bc3e4454-bd99-4446-8f76-de1b995ecd77. Saving inode for future.
[2019-01-07 19:16:44.799325] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 57 already present. gfid=eae201b5-7158-4f88-a104-62f18fd31578. Saving inode for future.
[2019-01-07 19:16:44.799342] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 58 already present. gfid=6e8a6e46-6b29-4bfa-b199-e5d19292d3a8. Saving inode for future.
[2019-01-07 19:16:44.799360] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 59 already present. gfid=808b69cc-a179-4a53-b8f5-3495517200ea. Saving inode for future.
[2019-01-07 19:16:44.799377] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 60 already present. gfid=33a43c1e-2c8b-4c8f-88e8-5bfd5ef4ab64. Saving inode for future.
[2019-01-07 19:16:44.799398] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 61 already present. gfid=d6328725-50ec-44f2-b03b-baaace38f76f. Saving inode for future.
[2019-01-07 19:16:44.799419] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 62 already present. gfid=1d5d28ed-7974-48fd-ae7f-59c221d6ce6e. Saving inode for future.
[2019-01-07 19:16:44.799435] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 63 already present. gfid=ebdd91f1-5633-4e3b-a34e-776668f00521. Saving inode for future.
[2019-01-07 19:16:44.799453] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 64 already present. gfid=8e8eecfe-a8a6-4dfe-9bf6-bf4a2a11e72e. Saving inode for future.
[2019-01-07 19:16:44.799471] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 65 already present. gfid=d0a05d2a-1c71-434d-a652-636c319d1e1f. Saving inode for future.
[2019-01-07 19:16:44.799488] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 66 already present. gfid=93af9de0-19d7-49fb-b8d1-f295c9250402. Saving inode for future.
[2019-01-07 19:16:44.799506] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 67 already present. gfid=e51ddb91-b594-463f-b303-985bd13a7760. Saving inode for future.
[2019-01-07 19:16:44.799527] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 69 already present. gfid=10179039-d866-4165-b2b1-f7d32d1eae89. Saving inode for future.
[2019-01-07 19:16:44.799544] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 70 already present. gfid=88cacf3b-863e-40f6-b72e-eb264e3a4bdb. Saving inode for future.
[2019-01-07 19:16:44.799561] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 71 already present. gfid=790c07eb-2b67-4564-8c11-a5938e6c2644. Saving inode for future.
[2019-01-07 19:16:44.799578] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 72 already present. gfid=ca73ea1d-a79d-4e2e-a4a4-854994398f2e. Saving inode for future.
[2019-01-07 19:16:44.799595] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 73 already present. gfid=c5a0ebf1-9fa4-4e2c-838e-977a47c9c77a. Saving inode for future.
[2019-01-07 19:16:44.799613] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 74 already present. gfid=d9d066f6-bf54-497e-9b86-eefbac56a28e. Saving inode for future.
[2019-01-07 19:16:44.799630] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 75 already present. gfid=2048b5cd-a330-400b-b431-9874c07f9509. Saving inode for future.
[2019-01-07 19:16:44.799648] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 76 already present. gfid=6d528cb9-6eaf-4ed3-96f8-19bb4f880a83. Saving inode for future.
[2019-01-07 19:16:44.799669] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 77 already present. gfid=d72f4da5-d366-44df-be4b-3e81de506ef1. Saving inode for future.
[2019-01-07 19:16:44.799686] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 78 already present. gfid=079276e0-5d72-43f7-864f-bc19747b5e8d. Saving inode for future.
[2019-01-07 19:16:44.799704] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 79 already present. gfid=09755526-ae2e-4454-aec1-3e9462b81d45. Saving inode for future.
[2019-01-07 19:16:44.799722] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 80 already present. gfid=c869911f-afb0-45b8-a8fe-4a0033c58cb9. Saving inode for future.
[2019-01-07 19:16:44.799739] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 81 already present. gfid=da2998d3-8acb-4fa1-bb75-8f34624797f0. Saving inode for future.
[2019-01-07 19:16:44.799756] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 82 already present. gfid=e627cac2-3f6e-4c99-bbca-1ec6c601eb1f. Saving inode for future.
[2019-01-07 19:16:44.799774] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 83 already present. gfid=331d0595-7640-4e0c-bf1e-0b62a705ee80. Saving inode for future.
[2019-01-07 19:16:44.799791] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 84 already present. gfid=1cff125a-7cdc-4c6d-892c-31fe7226c832. Saving inode for future.
[2019-01-07 19:16:44.799809] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 85 already present. gfid=bbff4702-ea16-4070-94f7-b50cdce82ef5. Saving inode for future.
[2019-01-07 19:16:44.799827] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 86 already present. gfid=2d63f899-45ce-4b6b-b9ff-b3d688bdcccd. Saving inode for future.
[2019-01-07 19:16:44.799845] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 87 already present. gfid=99a77c9a-0742-4736-9dc7-18c43cbda9c7. Saving inode for future.
[2019-01-07 19:16:44.799896] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 88 already present. gfid=c399eb36-0df7-493f-b2f8-3cd1f062c0e4. Saving inode for future.
[2019-01-07 19:16:44.799914] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 89 already present. gfid=ee03721d-6d28-4916-b809-145d8c019542. Saving inode for future.
[2019-01-07 19:16:44.799932] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 90 already present. gfid=0c29b886-29a3-45da-aad0-01087bd58b3a. Saving inode for future.
[2019-01-07 19:16:44.799949] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 91 already present. gfid=b232f060-47ec-4c02-a547-db3cb79fbbaa. Saving inode for future.
[2019-01-07 19:16:44.799966] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 92 already present. gfid=82825f38-4654-4c4a-8a09-3ef802a7e33f. Saving inode for future.
[2019-01-07 19:16:44.799985] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 93 already present. gfid=c707bc8b-a650-4ccb-a607-6e7a7753ae8c. Saving inode for future.
[2019-01-07 19:16:44.800003] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 94 already present. gfid=57d73c33-0041-436f-8dea-ee6a6998eae7. Saving inode for future.
[2019-01-07 19:16:44.800020] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 95 already present. gfid=5c30f4a3-c6a5-41cc-babc-1950058784cf. Saving inode for future.
[2019-01-07 19:16:44.800038] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 96 already present. gfid=792743e2-69dc-4537-8b9f-d244ce540a68. Saving inode for future.
[2019-01-07 19:16:44.800056] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 97 already present. gfid=f495af4e-9eb3-4eff-80c7-bfe8b7db5f1d. Saving inode for future.
[2019-01-07 19:16:44.800077] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 98 already present. gfid=cec65637-2b91-442f-b67b-e2fe3357d8e8. Saving inode for future.
[2019-01-07 19:16:44.800095] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 99 already present. gfid=efff1212-0db3-477d-a01f-1211615bd31b. Saving inode for future.
[2019-01-07 19:16:44.800112] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 100 already present. gfid=df2c86a7-467d-4242-9e33-b470264ce841. Saving inode for future.
[2019-01-07 19:16:44.800129] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 101 already present. gfid=806aadfa-df54-447f-9ec9-1c3b4c620cd9. Saving inode for future.
[2019-01-07 19:16:44.800147] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 102 already present. gfid=ae7c18f2-7578-4549-9f28-e4ce09432ce8. Saving inode for future.
[2019-01-07 19:16:44.800164] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 103 already present. gfid=58a971b7-c283-4de4-bcf3-7f15d3dab726. Saving inode for future.
[2019-01-07 19:16:44.800181] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 104 already present. gfid=4910e7f4-2bf3-4ff6-9db9-1edf211ec779. Saving inode for future.
[2019-01-07 19:16:44.800196] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 105 already present. gfid=4dba8fd4-992f-408b-b7d1-796e5ce5d7f3. Saving inode for future.
[2019-01-07 19:16:44.800213] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 106 already present. gfid=f5a0351e-467d-4745-8e31-fd994be3a814. Saving inode for future.
[2019-01-07 19:16:44.800231] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 107 already present. gfid=f560e811-49b5-468f-8a03-9d45525bc8de. Saving inode for future.
[2019-01-07 19:16:44.800248] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 108 already present. gfid=f0c7f965-6419-4c46-819c-d600d4c4fd74. Saving inode for future.
[2019-01-07 19:16:44.800259] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 109 already present. gfid=b912e7bd-a80d-4f05-93ca-0ddf7602e60f. Saving inode for future.
[2019-01-07 19:16:44.800276] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 110 already present. gfid=4e05ec71-d790-43c4-91ec-8719cf0b9e02. Saving inode for future.
[2019-01-07 19:16:44.800293] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 111 already present. gfid=271371d1-b39f-469a-ad42-d9813d07d3eb. Saving inode for future.
[2019-01-07 19:16:44.800310] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 112 already present. gfid=55a5c209-6795-4a21-a760-8729a89c8968. Saving inode for future.
[2019-01-07 19:16:44.800328] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 113 already present. gfid=65e435ab-f2f0-4490-b1f1-465d2c52201d. Saving inode for future.
[2019-01-07 19:16:44.800344] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 114 already present. gfid=e6f45f8f-58cb-4d8d-a011-7bbad73528fc. Saving inode for future.
[2019-01-07 19:16:44.800357] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 115 already present. gfid=6c4cf62d-3645-45d1-ab83-b429a4bd4d41. Saving inode for future.
[2019-01-07 19:16:44.800375] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 116 already present. gfid=ae08c54d-c21c-4741-9f5a-e34929a2c11c. Saving inode for future.
[2019-01-07 19:16:44.800389] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 117 already present. gfid=91f51ae2-7a67-49fb-ac26-7b4b7747c931. Saving inode for future.
[2019-01-07 19:16:44.800414] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 118 already present. gfid=6a605369-c010-4211-b173-3967d71a70f6. Saving inode for future.
[2019-01-07 19:16:44.800431] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 119 already present. gfid=31cb7b78-6a38-40b7-b554-0fa447c5a0a7. Saving inode for future.
[2019-01-07 19:16:44.800450] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 120 already present. gfid=b8576114-cc10-4511-899d-2f866c676014. Saving inode for future.
[2019-01-07 19:16:44.800468] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 121 already present. gfid=9147f132-1736-40a2-bc39-bbd7f4e710d3. Saving inode for future.
[2019-01-07 19:16:44.800486] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 122 already present. gfid=cd1d5a7f-191f-4840-ab6b-9ce482b30205. Saving inode for future.
[2019-01-07 19:16:44.800504] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 123 already present. gfid=df997038-aa04-4095-8668-ccb65f3ab312. Saving inode for future.
[2019-01-07 19:16:44.800517] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 125 already present. gfid=ae7330b4-fee0-4b78-866d-fe2ddcf46039. Saving inode for future.
[2019-01-07 19:16:44.800533] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 126 already present. gfid=fce7b2f9-fb76-453e-aa45-229a4ad3b6ce. Saving inode for future.
[2019-01-07 19:16:44.800550] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 127 already present. gfid=c6864cd5-d548-450d-b728-83cface585fd. Saving inode for future.
[2019-01-07 19:16:44.800571] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 128 already present. gfid=cc24308e-995e-41a0-8285-efa172e5bb1c. Saving inode for future.
[2019-01-07 19:16:44.800589] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 129 already present. gfid=7cb305f1-f87b-42e8-9f3b-8d66679353ca. Saving inode for future.
[2019-01-07 19:16:44.800609] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 131 already present. gfid=45b1e9b8-ed0d-4d85-aa7f-b94b8732c707. Saving inode for future.
[2019-01-07 19:16:44.800626] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 132 already present. gfid=89d274ac-700b-44ac-91d7-ac6ad6053ced. Saving inode for future.
[2019-01-07 19:16:44.800643] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 133 already present. gfid=c1f12116-0b64-441a-840c-c7bb67935888. Saving inode for future.
[2019-01-07 19:16:44.800660] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 134 already present. gfid=3a3422d7-0384-4e97-b708-4f75067996f2. Saving inode for future.
[2019-01-07 19:16:44.800676] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 135 already present. gfid=b095c598-9d5a-4c98-a110-055e9c4e3202. Saving inode for future.
[2019-01-07 19:16:44.800693] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 136 already present. gfid=9baeb644-7218-4bb1-9cb6-4eca5444ca88. Saving inode for future.
[2019-01-07 19:16:44.800709] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 137 already present. gfid=fb1fd885-5c95-4a48-b6ce-5282d887e1da. Saving inode for future.
[2019-01-07 19:16:44.800727] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 138 already present. gfid=d1f63ea3-ed69-442a-8761-9f4625e9e4f8. Saving inode for future.
[2019-01-07 19:16:44.800747] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 140 already present. gfid=411b3820-b88f-41dc-a783-f0d85fb71ab7. Saving inode for future.
[2019-01-07 19:16:44.800764] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 141 already present. gfid=45cf512b-bf67-48a5-9b7d-39bb772eb70c. Saving inode for future.
[2019-01-07 19:16:44.800780] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 142 already present. gfid=9c250fb4-114c-41b5-a431-fc42885e6747. Saving inode for future.
[2019-01-07 19:16:44.800797] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 143 already present. gfid=5c5f994a-9030-4070-98cd-c2cea80257e8. Saving inode for future.
[2019-01-07 19:16:44.800817] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 144 already present. gfid=2b2e0752-44c3-4bdf-8f52-86316e99d5c7. Saving inode for future.
[2019-01-07 19:16:44.800834] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 145 already present. gfid=a5ea9929-2362-4055-84ea-e004dcde8650. Saving inode for future.
[2019-01-07 19:16:44.800848] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 146 already present. gfid=b2d75b34-cf5d-4f8b-8531-8ada36809ccb. Saving inode for future.
[2019-01-07 19:16:44.800882] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 147 already present. gfid=59aca59f-b177-4ee8-90fd-c128cbba6f27. Saving inode for future.
[2019-01-07 19:16:44.800897] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 148 already present. gfid=6250a5db-4cbb-4d4b-8478-6a02c3ee1212. Saving inode for future.
[2019-01-07 19:16:44.800912] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 149 already present. gfid=640ec298-1c21-417c-8712-a848b50fb9c2. Saving inode for future.
[2019-01-07 19:16:44.800927] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 150 already present. gfid=59150000-263a-48da-b856-c79227f270cc. Saving inode for future.
[2019-01-07 19:16:44.800944] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 151 already present. gfid=9276f1b2-cb95-42b0-b439-f024585ca0b0. Saving inode for future.
[2019-01-07 19:16:44.800963] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 153 already present. gfid=db4182e2-af65-4b7a-8085-be629917b3d7. Saving inode for future.
[2019-01-07 19:16:44.800980] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 154 already present. gfid=a9990b31-fb3d-41bb-a88a-3a6ab2fe464c. Saving inode for future.
[2019-01-07 19:16:44.800997] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 155 already present. gfid=b49f4978-eaee-4884-9331-87047d5354d5. Saving inode for future.
[2019-01-07 19:16:44.801014] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 156 already present. gfid=92d9f487-46ca-420c-86a3-0129f4a6df28. Saving inode for future.
[2019-01-07 19:16:44.801030] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 157 already present. gfid=c3310ecc-5df6-4f26-9d99-c3da11c83f1e. Saving inode for future.
[2019-01-07 19:16:44.801045] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 158 already present. gfid=a11f1142-1dcd-4214-9111-e9537348decd. Saving inode for future.
[2019-01-07 19:16:44.801062] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 159 already present. gfid=22bcf1a8-8ee0-4b5e-ba95-6d08c0231bc6. Saving inode for future.
[2019-01-07 19:16:44.801078] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 160 already present. gfid=ade16de3-c448-45da-81dc-5365df4620e1. Saving inode for future.
[2019-01-07 19:16:44.801096] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 161 already present. gfid=2d409bc9-073e-47ad-9046-46183be0adc0. Saving inode for future.
[2019-01-07 19:16:44.801114] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 162 already present. gfid=c2995ff6-8630-40aa-ad9a-09b5c1cb79ff. Saving inode for future.
[2019-01-07 19:16:44.801130] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 163 already present. gfid=ee9d37f3-60c6-4909-85cf-15243d3500e4. Saving inode for future.
[2019-01-07 19:16:44.801147] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 164 already present. gfid=a77f60fb-ee43-4588-9905-717ed2404970. Saving inode for future.
[2019-01-07 19:16:44.801162] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 165 already present. gfid=b80ed048-8f40-45ef-ad1a-1f013721fde9. Saving inode for future.
[2019-01-07 19:16:44.801183] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 166 already present. gfid=8f080cc8-249a-4c54-8783-0bd4b976ef5e. Saving inode for future.
[2019-01-07 19:16:44.801200] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 167 already present. gfid=c87a1cb0-a831-4d4b-b0d8-9a8bc9fa56c7. Saving inode for future.
[2019-01-07 19:16:44.801218] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 169 already present. gfid=75d12dc9-0118-494a-8a15-057c88132497. Saving inode for future.
[2019-01-07 19:16:44.801232] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 171 already present. gfid=f4a2dabc-d4ef-4499-8db3-1c8e8785f3bb. Saving inode for future.
[2019-01-07 19:16:44.801248] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 172 already present. gfid=eff6ec95-c2ea-49b0-9a05-b81fcc2e7fde. Saving inode for future.
[2019-01-07 19:16:44.801271] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 174 already present. gfid=8e395322-2fe0-413a-b34e-6427bb80ae95. Saving inode for future.
[2019-01-07 19:16:44.801287] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 175 already present. gfid=a312e7ff-e652-4271-83b4-f3142df75692. Saving inode for future.
[2019-01-07 19:16:44.801299] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 176 already present. gfid=3757b73e-0bce-4dc8-a860-1bf9818f0b56. Saving inode for future.
[2019-01-07 19:16:44.801315] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 177 already present. gfid=5656e6c2-ae7a-474f-9394-d799d5d9e161. Saving inode for future.
[2019-01-07 19:16:44.801331] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 178 already present. gfid=307d5323-40d1-4b6b-901a-3d16f5558564. Saving inode for future.
[2019-01-07 19:16:44.801347] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 179 already present. gfid=7eafd8cc-1761-4cbf-b6b0-3c63182bc30c. Saving inode for future.
[2019-01-07 19:16:44.801363] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 180 already present. gfid=d1c84c0c-423a-4b95-9740-664d79980b45. Saving inode for future.
[2019-01-07 19:16:44.801383] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 182 already present. gfid=2dbee6fa-42ad-4abe-ac91-f8501f33e651. Saving inode for future.
[2019-01-07 19:16:44.801430] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 186 already present. gfid=f5752290-9a59-4636-865e-126843caf606. Saving inode for future.
[2019-01-07 19:16:44.801453] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 187 already present. gfid=f0aede4c-d1fb-4c2f-ae34-ede8f8726e25. Saving inode for future.
[2019-01-07 19:16:44.801470] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 188 already present. gfid=8dcebbcc-afa9-4231-bdd9-4c4f4f58ca0d. Saving inode for future.
[2019-01-07 19:16:44.801485] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 189 already present. gfid=36485bdd-0a8f-4051-b61b-e7765348074b. Saving inode for future.
[2019-01-07 19:16:44.801501] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 190 already present. gfid=b3050a5b-b6c0-4968-9ddf-a5d5b88f93dc. Saving inode for future.
[2019-01-07 19:16:44.801516] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 191 already present. gfid=9fe1be20-91b8-4077-8a92-bd8d2b690329. Saving inode for future.
[2019-01-07 19:16:44.801531] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 192 already present. gfid=020f1206-2c0a-40c7-b99e-e4b23641c35f. Saving inode for future.
[2019-01-07 19:16:44.801547] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 193 already present. gfid=c525999e-dc72-4290-83ad-e45e44f26754. Saving inode for future.
[2019-01-07 19:16:44.801565] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 194 already present. gfid=626a7f5c-a970-4788-9156-91c91f3efbcf. Saving inode for future.
[2019-01-07 19:16:44.801589] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 195 already present. gfid=4f9b7607-5ead-4ec1-81c1-68c4b492c77b. Saving inode for future.
[2019-01-07 19:16:44.801608] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 196 already present. gfid=056fe284-9bed-4732-998c-847ffb8aa9cd. Saving inode for future.
[2019-01-07 19:16:44.801623] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 197 already present. gfid=ad5b1f62-aea4-439a-8875-289803bd5d8f. Saving inode for future.
[2019-01-07 19:16:44.801640] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 198 already present. gfid=42639461-bbc6-41fd-a821-f81de5e52267. Saving inode for future.
[2019-01-07 19:16:44.801655] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 199 already present. gfid=70b671d1-e10e-4be2-b347-84cb9652ffa8. Saving inode for future.
[2019-01-07 19:16:44.801666] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 200 already present. gfid=dac1924f-c077-4f8b-9e5a-c49622e182dc. Saving inode for future.
[2019-01-07 19:16:44.801682] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 201 already present. gfid=6a1d24cf-c59d-4eaa-a20c-5a651191ace2. Saving inode for future.
[2019-01-07 19:16:44.801698] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 202 already present. gfid=de9a4b9a-032d-4d6c-ac95-dd9d65c90856. Saving inode for future.
[2019-01-07 19:16:44.801708] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 203 already present. gfid=1e532b8e-68bb-4034-8a44-b59b9de91b88. Saving inode for future.
[2019-01-07 19:16:44.801724] D [MSGID: 0] [shard.c:888:shard_common_resolve_shards] 0-gv1-shard: Shard 204 already present. gfid=422c196e-63a2-4802-8f49-de69dd82d8aa. Saving inode for future.
[2019-01-07 19:16:44.801974] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.801999] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.2
[2019-01-07 19:16:44.802016] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1223621823
[2019-01-07 19:16:44.802027] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2
[2019-01-07 19:16:44.802045] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.802058] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.802069] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.802113] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.802142] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802174] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.802190] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.802217] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802243] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.802262] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.802288] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802328] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.802344] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.802385] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.802423] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802448] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.802463] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.802479] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802500] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x958 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.802510] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.802526] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802532] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.802543] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.802555] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.802572] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.802680] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6bd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.802695] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.802706] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.802727] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.802741] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.802750] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.802762] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.201:49158: ping timer event already removed
[2019-01-07 19:16:44.802785] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.802807] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34e Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.802823] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.802867] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.802886] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802905] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6b9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.802917] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.802923] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.802932] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.802948] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.802959] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x954 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.802964] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.802968] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.802992] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803007] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.803009] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803039] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x349 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.803040] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x958 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.803059] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.803070] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803093] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x34d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.803104] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.803118] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803130] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803141] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.803178] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x34e Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.803189] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-5: Ping latency is 0ms
[2019-01-07 19:16:44.803205] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.201:49157: ping timer event already removed
[2019-01-07 19:16:44.803226] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.803243] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x954 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.803249] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34a Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.803262] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.803264] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.803280] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803317] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.803343] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803369] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x771 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.803380] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.803405] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803401] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6b9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.803436] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.803433] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.803449] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803467] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.803490] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803509] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x401 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.803531] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34a Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.803541] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-8: Ping latency is 0ms
[2019-01-07 19:16:44.803638] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x349 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.803652] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.803662] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803675] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803669] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.201:49156: ping timer event already removed
[2019-01-07 19:16:44.803700] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.803705] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.803736] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x402 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.803751] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa0c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.803753] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.803770] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.803787] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803802] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.803845] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803858] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x771 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.803878] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x888 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.803880] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.803907] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.803924] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.803959] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x402 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.803969] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.803980] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-11: Ping latency is 0ms
[2019-01-07 19:16:44.804001] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5ed Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.804014] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.804031] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804050] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x27d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.804064] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x888 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.804081] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.804095] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804117] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x401 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.804129] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.804138] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804151] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804164] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.804216] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5ed Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.804228] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.804239] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804263] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.202:49153: ping timer event already removed
[2019-01-07 19:16:44.804291] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.804315] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x27e Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.804330] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.804372] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.804399] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804430] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.804444] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x27d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.804445] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.804467] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.804479] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804495] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804480] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804507] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.804531] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.804549] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.804557] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x27e Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.804570] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-14: Ping latency is 0ms
[2019-01-07 19:16:44.804574] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804596] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x361 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.804690] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x96c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.804713] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.804724] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804746] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.804759] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.804772] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.804773] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.202:49160: ping timer event already removed
[2019-01-07 19:16:44.804794] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.804813] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x362 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.804828] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.804872] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.804891] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804916] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa71 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.804926] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.804942] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.804959] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7d6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.804971] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.804985] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805001] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x466 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.805079] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x361 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.805100] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.805115] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805130] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805142] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.805152] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.202:49158: ping timer event already removed
[2019-01-07 19:16:44.805173] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.805176] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x362 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.805186] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-17: Ping latency is 0ms
[2019-01-07 19:16:44.805202] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa71 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.805207] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x467 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.805213] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.805228] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.805228] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805259] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7d6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.805271] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.805272] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.805291] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805290] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805321] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.805332] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.805347] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805365] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x772 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.805374] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.805388] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805419] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x402 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.805469] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x467 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.805480] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-20: Ping latency is 0ms
[2019-01-07 19:16:44.805546] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa0d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.805559] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.805573] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805583] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.202:49159: ping timer event already removed
[2019-01-07 19:16:44.805593] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x466 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.805612] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:44.805615] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.805632] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805637] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x403 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.805653] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805666] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.805681] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.805694] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x772 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.805721] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.5
[2019-01-07 19:16:44.805722] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.805741] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805756] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1087134360
[2019-01-07 19:16:44.805767] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5
[2019-01-07 19:16:44.805779] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.805786] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.805794] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.805816] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x403 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.805819] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.805828] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-23: Ping latency is 0ms
[2019-01-07 19:16:44.805848] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805869] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.805878] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.805893] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805908] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.805917] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.805930] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.805937] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x402 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.805944] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.805953] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.805954] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.805974] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805988] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.805991] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.806009] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806027] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6be Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.806019] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.806043] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.806044] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.806064] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806053] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806095] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x959 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.806109] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.806133] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806153] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.806162] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.806188] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.806195] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.806205] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806216] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.806225] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6ba Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.806226] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806236] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.806251] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806252] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6be Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.806264] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.806267] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x955 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.806284] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.806287] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806300] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806316] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.806317] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.806331] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.806331] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.806350] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806365] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.806375] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.806381] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806401] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.806414] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x772 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.806416] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806435] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806426] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.806459] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.806464] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806496] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x959 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.806499] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.806509] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.806511] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.806524] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806530] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806547] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x403 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.806547] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x34f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.806557] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.806569] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.806581] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806586] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.806603] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806600] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806619] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.806631] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x889 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.806642] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.806649] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6ba Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.806659] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806661] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.806671] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806675] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5ee Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.806685] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.806689] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.806699] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806700] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.806715] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806717] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x27f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.806732] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.806737] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x955 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.806747] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.806757] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.806759] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806773] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806786] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806798] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.806810] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.806812] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.806828] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806861] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x772 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.806852] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.806888] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.806888] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.806919] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806926] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.806952] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa0d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.806953] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x363 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.806967] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.806979] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.806980] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.807001] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5ee Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.807010] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.807011] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.807030] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807031] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807050] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x889 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.807060] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.807063] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa72 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.807068] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807081] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.807088] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x403 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.807097] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807126] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.807137] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807145] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7d7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.807149] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807162] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.807162] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.807181] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807198] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x96d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.807198] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x468 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.807218] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.807218] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.807233] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807247] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.807252] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.807262] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807263] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.807278] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807288] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.807296] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x27f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.807298] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.807315] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.807320] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807334] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807342] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x773 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.807347] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807353] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.807358] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.807368] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807384] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x404 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.807391] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa72 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.807419] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.807425] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.807445] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.9
[2019-01-07 19:16:44.807445] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807456] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1085903184
[2019-01-07 19:16:44.807474] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9
[2019-01-07 19:16:44.807487] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x363 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.807512] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.807515] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.807521] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.807531] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.807533] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807546] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807557] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.807559] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.807588] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807590] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7d7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.807602] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.807607] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.807611] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807624] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.807631] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa0e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.807639] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807642] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.807651] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807655] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.807664] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.807668] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x468 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.807679] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807678] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.807695] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807696] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.807713] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.807714] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807731] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.807740] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.807758] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807757] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x773 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.807779] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.807780] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.807790] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807802] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.807818] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807833] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.807842] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.807849] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x404 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.807875] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807890] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.807900] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807909] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x350 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.807911] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807922] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.807934] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.807944] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.807949] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.807952] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.807965] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.807983] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.807988] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.807992] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.808001] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.808013] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808016] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808028] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x956 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.808037] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.808042] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.808050] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808053] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.808062] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808068] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.808077] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.808080] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.808096] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.808104] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.808105] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808136] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808136] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808153] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.808155] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x773 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.808171] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.808180] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6bf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.808188] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808200] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.808206] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.808238] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.808239] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808256] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808266] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.808273] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x404 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.808277] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.808288] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808289] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.808309] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x350 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.808319] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.808320] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.808331] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808342] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808344] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808354] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.808366] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.808376] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.808377] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x956 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.808399] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.808402] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808421] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808429] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5ef Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.808442] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6bb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.808444] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.808453] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.808462] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808463] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808478] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x280 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.808486] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa0e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.808487] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.808504] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.808515] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808520] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.808533] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.808545] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.808536] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808554] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808579] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.808585] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.808589] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.808600] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808614] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808625] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.808649] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x773 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.808659] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.808600] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.808674] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808699] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808708] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x404 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.808722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.808725] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.808742] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.808742] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808761] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808772] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808778] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x364 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.808793] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.808791] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.808830] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.808846] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808863] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5ef Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.808880] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.808883] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa73 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.808896] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.808898] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808913] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808921] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x96e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.808940] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.808941] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7d8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.808954] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.808966] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.808975] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x280 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.808982] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.808993] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.809000] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x469 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.809003] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809020] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.809022] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809036] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.809048] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.809066] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809065] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.809084] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.809091] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.809095] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809114] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.809129] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa73 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.809131] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809147] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.809156] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x774 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.809157] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809167] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.809188] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809187] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x364 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.809204] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x405 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.809205] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.809221] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809232] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809240] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.809242] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.809257] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.21
[2019-01-07 19:16:44.809268] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1223600709
[2019-01-07 19:16:44.809274] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7d8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.809276] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21
[2019-01-07 19:16:44.809293] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.809297] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.809303] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809307] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.809320] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.809324] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa0f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.809334] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.809343] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809345] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.809363] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x774 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.809366] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809383] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.809399] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.809401] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809422] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.809431] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x469 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.809438] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809448] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.809457] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.809458] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809474] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.809476] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809492] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809490] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.809510] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.809520] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.809561] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.809576] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809593] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.809602] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.809615] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809629] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.809637] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.809654] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809669] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x351 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.809669] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.809701] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.809702] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.809718] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809727] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.809738] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x405 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.809742] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809757] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.809766] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.809767] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809784] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.809786] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809805] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809804] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.809819] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.809823] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x957 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.809833] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.809834] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809847] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809865] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.809877] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.809884] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.809923] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.809932] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809944] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.809952] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.809961] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.809962] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.809972] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.809978] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x774 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.810004] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.810016] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.810026] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.810032] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810047] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810055] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa0f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.810067] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810067] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.810087] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.810092] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810113] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x405 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.810116] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x351 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.810128] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.810129] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.810144] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810158] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.810165] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.810173] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810176] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.810185] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810197] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.810196] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810215] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.810215] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.810240] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810247] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6bc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.810257] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.810258] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.810274] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810274] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.810294] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.810304] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.810308] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810319] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810327] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x281 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.810337] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.810339] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x957 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.810357] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.810362] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.810385] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810382] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810419] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.810433] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.810434] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810447] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.810451] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810483] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa0f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.810495] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.810497] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.810504] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810522] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.810530] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.810542] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810541] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.810558] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810561] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x365 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.810575] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.810577] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x774 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.810593] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.810603] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.810603] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810625] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810648] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa74 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.810650] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.810659] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.810668] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.810676] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810679] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810697] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x405 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.810702] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7d9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.810714] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.810715] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.810729] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810731] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810747] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810757] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.810759] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.810775] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.810791] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x96f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.810802] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.810811] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810807] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.810830] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x281 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.810836] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810852] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.810862] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810867] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa10 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.810879] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.810880] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810895] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810893] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.810912] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x775 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.810922] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.810926] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.810939] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.810938] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.810955] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x406 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.810954] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.810981] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa74 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.810990] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.810991] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.811007] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811007] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.29
[2019-01-07 19:16:44.811024] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1240589523
[2019-01-07 19:16:44.811028] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x365 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.811031] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29
[2019-01-07 19:16:44.811047] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.811049] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.811059] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811064] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.811075] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.811076] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811087] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.811101] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.811114] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7d9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.811124] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811125] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.811141] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811147] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.811157] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.811159] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa10 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.811172] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811173] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.811183] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811187] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.811196] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.811200] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x775 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.811210] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811210] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.811228] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811239] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.811248] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.811250] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.811268] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.811278] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811283] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.811296] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811300] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811329] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.811337] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.811347] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.811358] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x406 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.811364] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811371] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.811379] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.811380] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811389] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.811411] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811414] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811431] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.811439] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x352 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.811441] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.811460] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811461] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.811502] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.811505] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.811519] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811518] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.811535] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811538] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.811548] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.811562] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811576] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x958 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.811584] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.811599] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811603] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.811613] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.811622] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.811622] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.811643] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811656] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.811686] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.811697] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.811690] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811705] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x775 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.811723] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811733] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.811746] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.811750] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811771] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa10 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.811780] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.811789] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.811801] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.811807] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811810] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811823] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x406 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.811829] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.811832] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.811844] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.811855] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811859] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.811873] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811873] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x352 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.811889] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.811896] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.811898] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811908] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.811924] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.811929] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811946] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.811952] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.811962] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.811974] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x958 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.811979] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.811990] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.811996] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x282 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.811999] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812011] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.812019] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.812029] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.812037] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.812038] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812052] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812058] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6bd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.812068] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.812074] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x970 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.812075] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812107] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812112] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.812125] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.812133] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812155] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.812158] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.812174] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.812184] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812175] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.812204] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa10 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.812214] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.812217] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812229] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812237] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x366 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.812248] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.812250] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.812260] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.812269] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812278] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.812293] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812292] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x406 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.812310] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.812317] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa75 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.812320] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812338] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.812355] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812356] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x775 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.812375] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7da Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.812374] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.812386] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.812397] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812413] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812423] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812423] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.812443] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.812455] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x970 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.812456] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.812475] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.812486] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812497] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.812506] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x282 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.812513] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812524] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.812533] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812537] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa11 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.812549] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.812550] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812565] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812563] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.812585] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x776 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.812595] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.812598] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.812609] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.812611] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812620] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812633] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x407 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.812638] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa75 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.812648] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.812656] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812665] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.812674] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x366 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.812685] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.31
[2019-01-07 19:16:44.812686] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.812702] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812703] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1319971604
[2019-01-07 19:16:44.812718] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31
[2019-01-07 19:16:44.812720] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812733] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.812733] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.812748] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.812756] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.812761] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7da Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.812782] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.812792] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812795] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.812810] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa11 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.812812] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812828] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.812838] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812838] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.812855] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.812857] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x776 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.812871] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.812879] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812871] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812896] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.812902] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.812906] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.812916] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812918] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.812936] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.812941] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.812955] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.812963] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.812976] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.813003] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.813017] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813033] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.813041] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.813054] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813068] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.813076] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.813089] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813102] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x353 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.813110] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.813132] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.813145] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813155] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.813159] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6be Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.813175] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.813177] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.813194] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813198] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813217] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x407 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.813219] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x959 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.813237] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.813238] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.813253] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813267] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813267] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813287] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x34f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.813297] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.813284] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.813327] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.813323] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.813346] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813347] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813363] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x776 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.813384] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.813402] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.813421] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813422] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.813440] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813443] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa11 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.813463] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.813466] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.813478] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813478] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.813490] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813494] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x407 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.813504] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.813509] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.813526] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.813536] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.813536] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813556] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813567] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813567] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.813593] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.813599] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x353 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.813603] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.813612] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.813622] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813622] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813640] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.813649] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.813650] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.813663] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813663] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.813678] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813680] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x283 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.813696] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.813695] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813718] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.813734] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.813753] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813752] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6be Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.813770] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.813777] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x971 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.813780] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813805] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x959 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.813796] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.813816] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.813826] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813844] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x34f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.813847] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813862] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.813875] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.813876] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813899] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.813903] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.813916] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.813924] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.813933] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x367 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.813946] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.813964] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.813990] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.813993] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.814005] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814017] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814031] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa76 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.814041] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.814050] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x776 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.814056] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814062] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.814072] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814082] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7db Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.814090] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.814092] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.814102] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.814111] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814112] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814126] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.814130] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x407 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.814147] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.814156] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814159] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.814174] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa11 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.814185] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.814185] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.814208] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814214] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814221] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814233] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.814238] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.814252] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.814258] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x971 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.814269] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814269] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.814285] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814288] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x777 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.814298] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.814303] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x283 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.814312] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814314] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.814325] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814328] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.814336] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814346] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.814361] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.814370] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.814382] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.35
[2019-01-07 19:16:44.814383] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.814417] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814417] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1413476598
[2019-01-07 19:16:44.814432] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35
[2019-01-07 19:16:44.814440] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa76 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.814443] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.814456] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.814458] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.814469] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814470] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.814495] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x367 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.814502] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.814515] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.814523] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814533] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814544] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.814556] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.814558] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814571] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814588] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.814584] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.814598] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.814620] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814634] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7db Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.814639] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.814659] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.814660] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.814683] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814699] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.814705] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.814714] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814716] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.814727] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814733] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.814743] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.814745] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x777 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.814759] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814760] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.814775] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814777] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.814792] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.814797] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.814808] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814808] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.814823] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814825] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x354 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.814843] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.814844] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814876] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.814886] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.814902] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.814906] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814921] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.814930] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6bf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.814940] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.814931] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.814954] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.814969] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.814969] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.814979] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.814988] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.814995] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815005] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815011] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x350 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.815020] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.815026] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.815037] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.815049] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815050] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.815074] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815091] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x777 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.815100] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.815113] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815128] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.815128] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.815136] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.815155] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.815160] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815178] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815184] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.815194] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.815195] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815210] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.815240] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.815241] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.815255] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.815266] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815266] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815292] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.815292] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.815311] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.815311] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.815329] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815329] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815350] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.815357] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x354 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.815360] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.815369] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.815377] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815379] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815397] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x284 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.815391] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815419] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.815419] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.815450] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.815455] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.815466] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815466] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.815492] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x972 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.815504] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.815505] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815523] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815533] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.815525] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815577] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.815586] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815591] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.815605] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.815634] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815650] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x368 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.815654] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x350 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.815674] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.815677] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.815699] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.815695] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815716] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815735] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6bf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.815741] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa77 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.815759] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.815757] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.815778] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815781] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815795] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7dc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.815807] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815815] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.815826] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.815833] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815858] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.815867] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa12 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.815868] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.815910] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.815926] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.815929] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.815960] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.815975] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.815982] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.816004] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.816005] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.816016] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816043] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816052] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x777 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.816060] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.816063] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.816074] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816075] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.816095] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x408 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.816097] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816112] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.816119] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x409 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.816121] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816135] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816153] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.816170] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.816173] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.40
[2019-01-07 19:16:44.816191] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1252560672
[2019-01-07 19:16:44.816199] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40
[2019-01-07 19:16:44.816201] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.816217] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.816217] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.816228] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.816242] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.816243] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816269] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.816270] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x972 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.816285] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816285] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.816300] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816306] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.816323] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x284 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.816321] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.816343] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.816356] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816357] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816374] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816379] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.816385] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.816414] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.816430] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.816462] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.816470] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816462] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816487] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa77 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.816497] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.816498] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.816511] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.816513] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816534] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7dc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.816537] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.816548] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.816554] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816569] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816577] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.816587] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.816587] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.816621] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.816626] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816637] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816643] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.816656] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x368 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.816656] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.816675] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.816680] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816691] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816698] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x355 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.816709] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.816710] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816723] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.816734] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.816749] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.816751] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816767] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.816776] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.816777] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816792] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.816815] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.816808] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816833] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.816840] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.816843] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816856] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.816857] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816874] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816877] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.816890] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x351 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.816900] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.816932] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.816945] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.816960] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.816957] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.816978] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.816979] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.816987] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.817009] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x409 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.817013] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817031] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.817040] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.817051] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.817053] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817065] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817074] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817079] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x409 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.817112] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.817113] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.817145] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.817156] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817157] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.817183] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817203] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.817208] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x88f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.817226] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.817224] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.817246] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817248] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817262] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.817272] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.817285] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817280] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.817299] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x285 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.817300] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.817315] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817315] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.817330] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817342] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.817345] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.817382] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817381] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.817410] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x973 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.817438] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.817438] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.817464] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817462] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817491] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.817501] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x355 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.817515] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.817515] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.817534] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817533] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817553] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x369 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.817559] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x95f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.817571] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.817571] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.817586] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817598] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817598] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.817619] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.817624] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817649] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.817660] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.817664] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa78 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.817668] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817684] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.817692] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.817702] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817703] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.817718] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7dd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.817738] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.817742] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x351 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.817753] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817753] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.817768] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817773] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.817784] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.817784] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817801] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.817811] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.817826] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa13 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.817831] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817842] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.817852] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817856] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.817867] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.817870] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x88f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.817882] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817883] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.817900] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.817898] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817911] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.817931] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.817932] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x409 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.817943] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.817946] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.817963] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.817983] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x778 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.817988] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.817993] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.818003] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818003] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.68
[2019-01-07 19:16:44.818022] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818021] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1394291141
[2019-01-07 19:16:44.818038] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.818049] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68
[2019-01-07 19:16:44.818063] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.818071] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.818072] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.818080] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.818099] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.818116] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818127] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.818142] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x973 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.818156] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.818146] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818165] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818183] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.818183] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x285 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.818199] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.818200] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.818209] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818221] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818222] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818232] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.818236] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.818252] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.818261] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.818268] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818299] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.818305] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.818310] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818315] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.818334] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa78 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.818348] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.818351] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.818389] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818445] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.818461] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.818483] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818500] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818511] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x960 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.818525] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.818536] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7dd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.818549] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.818550] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818572] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818580] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x356 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.818594] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.818596] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.818614] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.818624] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818638] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.818643] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x369 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.818662] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.818665] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818671] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818692] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818698] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.818724] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.818725] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.818768] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818772] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.818793] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.818808] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.818810] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.818830] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.818832] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.818862] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.818865] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x352 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.818918] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.818958] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.818982] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819009] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.819024] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.819047] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819073] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.819088] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.819112] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819107] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.819145] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.819146] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819174] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.819179] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819200] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.819223] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.819251] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819256] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.819282] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.819286] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x890 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.819312] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.819313] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819357] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.819342] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819385] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.819405] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.819411] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819451] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.819468] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.819475] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819486] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.819501] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x286 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.819524] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.819566] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.819590] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819502] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819618] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x974 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.819629] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.819630] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819647] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819653] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.819669] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.819679] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.819697] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819697] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.819711] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.819726] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.819728] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.819745] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819751] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.819764] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819778] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819783] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.819796] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.819809] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819811] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa79 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.819848] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.819873] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.819886] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819888] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.819901] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819913] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7de Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.819925] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x356 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.819930] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.819936] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.819960] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.819978] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.819989] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x46f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.820005] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.820010] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x960 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.820029] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.820044] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820048] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.820091] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820080] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820119] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.820126] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.820145] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.820169] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.820197] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.820176] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820214] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820245] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.820261] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.820265] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.820283] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.820286] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820300] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820318] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.820327] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x352 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.820344] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.820357] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820366] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.820386] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820390] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.124
[2019-01-07 19:16:44.820429] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.820430] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1080394250
[2019-01-07 19:16:44.820456] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124
[2019-01-07 19:16:44.820475] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.820477] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa14 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.820490] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.820508] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.820509] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.820529] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820570] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.820575] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.820591] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820600] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.820613] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.820615] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820630] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.820641] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x779 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.820648] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820660] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.820668] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.820671] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820689] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.820693] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820706] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820707] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.820722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.820732] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.820735] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.820747] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.820760] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820760] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.820782] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x890 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.820785] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820800] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.820809] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.820810] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820821] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.820837] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820836] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x286 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.820852] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.820853] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x961 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.820863] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820874] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.820876] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820890] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820888] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.820920] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x357 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.820921] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6d9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.820940] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.820940] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.820955] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.820970] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.820973] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x974 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.820991] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.820992] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.821006] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821011] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.821023] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.821025] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa79 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.821037] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821036] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.821052] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821054] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.821070] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.821073] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.821084] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.821085] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821101] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821108] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x353 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.821119] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.821119] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821135] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.821145] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.821162] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7de Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.821165] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821196] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.821205] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.821206] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821224] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.821233] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.821241] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821244] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.821253] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821257] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.821267] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.821271] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.821281] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.821286] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821296] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821303] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.821313] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.821315] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x46f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.821331] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.821340] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821340] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.821371] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821377] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821388] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.821415] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x891 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.821427] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.821431] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.821443] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.821445] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821454] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821466] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821469] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.821480] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.821481] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.821496] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821496] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.821510] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821513] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x287 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.821528] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.821550] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.821561] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.821562] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.821579] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821579] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821603] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.821605] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x975 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.821622] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.821622] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.821643] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821661] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821664] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.821675] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.821679] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6da Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.821687] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821688] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.821706] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821712] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821723] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.821730] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.821745] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.821751] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x357 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.821762] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.821771] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821772] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.821791] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.821796] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821807] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.821817] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821822] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.821837] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x961 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.821838] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.821858] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.821869] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821882] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821892] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.821868] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821915] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.821926] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.821928] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7df Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.821934] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821946] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.821954] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x353 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.821964] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.821966] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.821975] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.821989] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x470 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.821993] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.822011] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.822020] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822016] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.822043] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822054] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.822064] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.822083] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa15 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.822094] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.822095] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822124] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822141] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.822143] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x891 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.822162] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.822163] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.822176] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822184] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822196] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.822203] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.822206] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.822221] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822223] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.822241] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.822244] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822258] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.822267] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.822268] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822285] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822297] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.822311] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.822325] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.822328] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.130
[2019-01-07 19:16:44.822343] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.822344] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1277578615
[2019-01-07 19:16:44.822355] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822359] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130
[2019-01-07 19:16:44.822372] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.822375] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x975 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.822386] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.822410] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.822438] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.822398] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.822454] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822463] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822475] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.822488] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6da Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.822490] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.822508] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.822514] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822524] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822535] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.822544] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.822546] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.822565] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.822570] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822581] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822588] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.822599] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.822600] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x287 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.822617] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.822626] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822627] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.822646] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822651] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822666] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.822675] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.822686] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.822694] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7df Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.822703] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822704] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.822719] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x962 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.822733] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.822738] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.822751] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.822761] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822766] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822778] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.822788] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.822792] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x358 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.822797] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822816] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822827] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.822851] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.822861] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.822863] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.822869] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822894] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x470 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.822903] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.822905] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.822920] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822923] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822939] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.822949] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.822950] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.822965] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.822977] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.822984] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.822991] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.823000] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.823001] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823011] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.823023] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823028] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823042] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.823049] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x354 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.823052] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.823063] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823064] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.823107] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.823118] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.823120] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.823135] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823141] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823155] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.823160] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.823171] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.823171] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.823185] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823191] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823204] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.823219] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.823228] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823226] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.823249] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823251] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.823267] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.823279] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823294] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.823305] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.823305] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.823323] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823325] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.823343] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x358 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.823353] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.823361] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823372] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.823379] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x962 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.823389] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.823391] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823418] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823431] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823436] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x892 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.823453] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.823453] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.823483] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823491] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.823501] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.823502] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.823512] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.823517] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823528] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823537] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x354 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.823542] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x288 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.823555] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.823556] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.823574] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823590] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.823594] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.823605] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.823606] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823616] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823627] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823637] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.823638] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x976 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.823662] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa16 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.823661] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.823679] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.823689] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823700] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823710] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6db Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.823721] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.823723] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x892 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.823738] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823739] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.823750] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823755] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.823769] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.823780] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.823764] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.823788] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823807] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.823819] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.823819] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.823836] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823843] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823854] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823865] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.823871] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.823887] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.823894] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.823911] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.823914] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823927] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823937] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.823946] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x976 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.823950] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.823962] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.823968] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.823986] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.823992] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x471 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.824008] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.824018] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x288 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.824033] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.824036] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.824050] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824051] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824074] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.824070] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824086] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.824106] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.824111] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824140] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.824142] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6db Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.824152] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.824161] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.824170] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824171] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824188] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.824195] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.824205] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.824214] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824225] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.824231] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.824240] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.139
[2019-01-07 19:16:44.824241] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.824259] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824260] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1342729215
[2019-01-07 19:16:44.824276] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139
[2019-01-07 19:16:44.824278] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824289] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.824290] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.824305] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.824320] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.824324] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.824333] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.824343] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824354] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.824363] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.824371] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824380] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.824401] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.824403] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824428] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.824444] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.824448] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824476] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.824489] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.824493] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824514] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.824530] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x471 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.824540] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824552] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.824562] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x416 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.824563] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824582] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.824583] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824622] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.824617] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.824641] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824661] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.824662] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.824685] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.824687] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.824704] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824701] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824722] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x963 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.824723] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824731] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.824754] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824750] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.824770] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x359 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.824767] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.824790] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.824791] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824823] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.824840] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824865] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.824872] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.824889] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.824890] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.824910] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824913] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.824927] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x95f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.824937] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.824945] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.824951] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.824965] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.824975] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x355 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.824986] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.824988] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825012] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.825020] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x416 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.825027] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825037] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.825045] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.825047] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825060] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.825062] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825077] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825076] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.825095] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.825105] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.825108] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.825125] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825147] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.825156] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.825158] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825171] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.825181] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x963 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.825197] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.825197] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.825212] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825214] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825233] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x359 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.825245] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x893 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.825249] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.825256] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.825273] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825270] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825289] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.825290] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825299] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.825316] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.825322] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825366] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x289 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.825369] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x95f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.825390] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.825397] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.825410] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825437] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.825448] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.825447] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.825462] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825481] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825487] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x893 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.825505] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.825516] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x977 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.825519] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825538] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.825553] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa17 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.825563] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825587] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6dc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.825587] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.825610] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825611] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.825645] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825644] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.825670] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.825668] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.825697] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825698] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.825731] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x355 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.825745] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.825744] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.825768] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825769] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825785] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825799] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.825795] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.825827] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.825849] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.825857] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825873] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.825884] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.825889] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825906] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.825916] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.825923] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.825934] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.825942] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x472 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.825944] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825952] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.825970] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.825987] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.825989] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.826004] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826031] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.826031] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x289 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.826045] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.826056] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.826062] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826080] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826087] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.826102] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826103] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.826127] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826126] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.826144] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.826181] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x977 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.826197] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.826200] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.826219] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.152
[2019-01-07 19:16:44.826220] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826242] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1288142377
[2019-01-07 19:16:44.826253] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152
[2019-01-07 19:16:44.826255] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6dc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.826269] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.826269] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.826287] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.826287] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826297] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.826313] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.826323] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.826324] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.826339] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826342] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826361] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.826368] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.826381] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.826383] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.826403] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826407] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826419] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.826421] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826429] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.826437] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.826444] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826468] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x417 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.826472] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.826477] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.826490] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.826505] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826509] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.826525] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826525] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.826543] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.826559] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.826561] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.826582] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826587] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826606] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.826608] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x964 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.826625] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.826625] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.826639] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826642] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826660] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x472 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.826663] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.826678] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.826679] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.826700] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826715] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826716] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.826734] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.826739] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826759] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.826763] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.826776] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.826779] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.826794] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826792] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826812] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x960 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.826811] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826829] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.826829] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.826846] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826845] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.826860] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826876] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x356 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.826890] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.826901] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.826914] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.826927] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.826927] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826952] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.826961] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x417 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.826970] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.826972] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.826980] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.826985] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.826995] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827010] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.827019] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.827022] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.827032] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827043] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.827049] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.827054] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827059] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.827070] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827084] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.827086] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.827112] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827119] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.827131] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.827135] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x894 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.827140] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827158] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.827166] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6c9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.827174] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827177] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.827187] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827191] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5f9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.827201] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.827208] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x964 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.827219] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.827218] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827230] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827242] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827245] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.827257] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.827257] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.827286] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.827298] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x960 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.827302] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827317] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.827327] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827333] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x978 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.827343] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.827346] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x356 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.827366] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827367] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.827388] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6dd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.827389] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827417] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.827434] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827433] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.827447] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.827451] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.827462] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.827462] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827483] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827492] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.827495] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.827508] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827522] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x894 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.827531] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.827533] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.827550] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827550] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.827571] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa18 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.827579] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827599] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.827610] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827613] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.827630] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5f9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.827629] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.827652] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.827665] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827666] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827693] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.827693] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x473 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.827710] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.827712] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.827719] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827739] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.827751] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.827751] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.827768] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827771] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827779] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827790] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.827798] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.827809] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.827815] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x978 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.827837] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827840] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.827850] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827857] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.827869] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.827873] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.827885] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.827891] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.827894] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827906] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827908] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.827925] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.827950] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6dd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.827949] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.827968] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.827973] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.168
[2019-01-07 19:16:44.827979] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.827984] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1374046734
[2019-01-07 19:16:44.827993] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168
[2019-01-07 19:16:44.827998] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.828010] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.828010] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.828033] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.828035] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828042] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.828057] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.828068] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.828068] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.828083] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828086] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828101] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.828106] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x787 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.828117] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.828117] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.828132] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828134] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828150] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828157] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa22 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.828161] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.828166] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.828191] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828193] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.828204] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.828206] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x418 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.828215] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828230] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.828239] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.828249] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.828255] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.828270] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828273] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828294] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x473 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.828297] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6ca Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.828311] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.828311] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.828325] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828328] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828342] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828349] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x965 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.828353] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.828359] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.828378] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828389] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.828398] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.828419] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.828419] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.828436] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828449] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828451] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.828470] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.828475] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828486] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.828495] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828495] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.828512] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.828535] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828551] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x961 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.828559] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.828573] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828566] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x418 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.828586] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x357 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.828626] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.828626] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.828653] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.828654] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828668] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828686] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.828691] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa22 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.828695] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.828712] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.828718] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828740] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.828755] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.828758] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828770] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828791] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.828796] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x787 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.828800] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.828817] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.828834] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.828835] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828852] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828862] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.828876] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x895 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.828888] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.828888] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.828905] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828921] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5fa Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.828929] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.828936] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6ca Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.828946] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.828961] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.828969] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.828982] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.828983] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829007] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.829024] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829024] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x965 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.829041] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.829046] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x979 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.829069] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.829073] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829086] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829112] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6de Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.829115] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x961 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.829121] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.829136] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.829144] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829162] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829167] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x36f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.829182] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.829187] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.829198] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.829208] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829208] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.829228] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829228] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.829244] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.829252] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.829256] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829268] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.829270] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829285] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829286] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.829301] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.829312] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.829321] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x895 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.829328] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829340] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.829349] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x474 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.829361] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.829363] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829386] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.829404] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.829410] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829424] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.829441] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.829439] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829454] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.829469] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829476] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x357 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.829484] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.829502] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.829504] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.829518] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829521] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829534] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.829546] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829567] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.829564] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.829586] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.170
[2019-01-07 19:16:44.829603] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1111239957
[2019-01-07 19:16:44.829611] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170
[2019-01-07 19:16:44.829612] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa19 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.829629] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.829631] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.829638] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.829649] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.829652] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829675] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.829684] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x40f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.829691] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829702] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.829715] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x788 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.829717] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829725] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.829741] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829748] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829774] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa23 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.829768] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.829784] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.829806] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829821] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x419 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.829818] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5fa Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.829830] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.829842] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.829858] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.829858] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829881] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829898] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.829899] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x979 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.829916] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.829919] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.829931] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829934] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.829947] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x966 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.829956] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.829963] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.829970] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.829985] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.829994] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.830006] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.830008] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830028] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830033] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.830058] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830055] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.830075] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.830085] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.830097] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6de Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.830102] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830116] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.830125] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x962 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.830135] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.830135] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830156] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830171] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x358 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.830180] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.830173] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.830209] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.830218] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.830240] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830236] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830257] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.830267] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.830268] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.830281] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830292] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.830297] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.830318] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.830321] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830334] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830349] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.830352] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.830358] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.830375] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.830402] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.830389] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830421] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830444] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x36f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.830447] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x896 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.830466] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.830464] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.830484] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830487] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830503] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5fb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.830518] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.830521] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830534] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830542] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.830551] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.830563] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.830584] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.830591] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.830611] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.830618] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830631] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830645] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.830657] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.830664] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x474 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.830672] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830687] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.830696] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6df Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.830707] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.830710] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830721] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830730] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830736] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x370 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.830761] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.830763] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.830796] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.830813] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830819] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.830835] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa7f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.830837] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.830856] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.830856] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830874] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830883] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830891] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.830907] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.830908] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.830922] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.830934] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.830940] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x475 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.830956] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.830958] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.830987] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.831012] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831032] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa23 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.831036] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.831058] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.831060] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.831074] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831075] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831093] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.831106] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x419 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.831107] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.831125] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.831135] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831146] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831153] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.831178] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x788 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.831192] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.831194] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.831207] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.173
[2019-01-07 19:16:44.831210] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831217] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1135612244
[2019-01-07 19:16:44.831230] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173
[2019-01-07 19:16:44.831232] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831242] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.831250] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.831250] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.831258] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.831293] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6cb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.831299] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.831313] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.831321] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831339] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831347] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x789 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.831361] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.831370] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.831376] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831389] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.831405] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa24 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.831422] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.831425] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831437] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831453] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.831462] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.831461] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x966 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.831485] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.831489] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.831516] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831512] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831532] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.831534] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831543] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.831560] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.831568] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831592] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x967 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.831597] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.831602] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.831611] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.831622] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831637] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831645] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.831657] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.831659] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x962 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.831671] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.831680] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831682] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.831698] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.831700] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831717] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.831725] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.831747] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831758] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.831773] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831775] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x77f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.831789] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x963 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.831834] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.831790] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.831850] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831869] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831876] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x359 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.831892] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.831902] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x896 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.831918] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.831922] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.831944] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.831941] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.831961] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.831971] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.831972] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x358 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.831985] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832000] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.832008] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.832007] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.832029] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832030] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832045] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.832056] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.832060] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832083] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.832106] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.832118] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832143] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x897 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.832145] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5fb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.832159] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.832170] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.832176] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832193] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832200] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5fc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.832214] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.832223] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.832228] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832240] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.832248] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.832275] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.832277] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832300] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.832306] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x410 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.832316] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832322] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.832336] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832340] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.832363] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.832366] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832380] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832397] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.832404] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.832424] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.832441] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832444] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6df Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.832459] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x371 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.832460] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.832477] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.832478] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832506] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.832513] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.832522] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832530] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.832545] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa80 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.832546] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832563] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.832567] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832580] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832598] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.832593] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.832608] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.832635] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832647] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa7f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.832651] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x476 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.832669] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.832666] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.832689] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832696] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.832717] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832721] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.832739] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.832737] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.832751] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.832760] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832768] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832792] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x370 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.832794] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.832812] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.832811] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.832829] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.832832] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832845] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.832857] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832879] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.832878] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.832908] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.181
[2019-01-07 19:16:44.832920] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1353889563
[2019-01-07 19:16:44.832931] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181
[2019-01-07 19:16:44.832933] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.832942] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.832950] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.832951] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.832958] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.832977] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.832997] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.833013] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833011] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.833032] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.833030] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.833044] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.833060] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833054] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833075] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa25 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.833084] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.833091] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.833098] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833111] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.833120] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.833142] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.833145] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833169] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.833167] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833185] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833187] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.833201] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.833213] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.833231] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833230] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa24 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.833246] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x968 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.833257] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.833259] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.833272] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833289] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.833285] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833308] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.833325] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x789 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.833342] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.833344] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.833370] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833369] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833404] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6c9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.833419] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.833419] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x475 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.833434] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833443] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.833451] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x964 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.833470] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.833471] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833487] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833499] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833505] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.833519] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.833541] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.833542] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.833582] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.833585] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833598] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833616] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.833628] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.833648] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833644] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.833664] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.833673] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.833673] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.833690] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833722] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833728] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.833746] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.833746] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833772] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.833778] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.833810] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833819] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6cc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.833838] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x898 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.833836] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.833850] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.833870] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833863] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833887] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5fd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.833896] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.833900] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.833911] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833918] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.833926] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.833938] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.833939] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.833964] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.833984] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.833974] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x967 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.834013] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.834013] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.834047] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.834048] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834064] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834083] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.834073] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834093] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.834109] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.834116] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834140] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x372 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.834150] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.834169] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x963 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.834176] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.834196] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.834201] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834213] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834226] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa81 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.834236] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.834237] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.834252] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834254] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.834270] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834273] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.834288] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.834295] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x359 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.834303] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834306] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.834316] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834319] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x477 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.834329] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.834328] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834347] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.834359] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.834377] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834376] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.834401] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.834408] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.834421] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.834426] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834437] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834460] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.834464] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x897 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.834469] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.834495] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834491] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.834513] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.834514] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834555] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.834570] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.183
[2019-01-07 19:16:44.834572] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x780 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.834579] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1394995644
[2019-01-07 19:16:44.834593] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183
[2019-01-07 19:16:44.834593] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.834613] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.834614] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834624] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.834639] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.834650] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.834664] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.834676] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.834681] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834703] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834710] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.834726] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.834735] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.834745] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834761] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.834772] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa26 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.834780] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834788] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.834809] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5fc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.834814] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834828] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.834839] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.834843] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834850] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.834864] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834882] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.834882] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.834899] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834919] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6ce Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.834925] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x411 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.834929] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.834944] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.834958] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.834977] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.834983] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x969 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.834999] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.834998] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835015] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835017] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.835035] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.835045] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.835075] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.835083] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.835090] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835105] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.835115] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6ca Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.835120] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835125] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.835148] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835151] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa80 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.835163] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x965 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.835167] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.835172] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.835196] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835192] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835211] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.835221] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.835224] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.835241] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.835247] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.835269] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835267] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835287] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.835297] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.835306] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x371 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.835312] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835325] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.835335] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.835340] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835345] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.835361] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835368] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835388] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.835400] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.835429] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.835442] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.835458] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.835459] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.835474] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835474] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835498] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x899 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.835502] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.835508] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.835518] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.835527] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835550] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835556] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5fe Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.835573] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.835583] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa25 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.835588] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835602] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.835611] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x28f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.835622] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.835624] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835651] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.835654] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.835671] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835670] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.835694] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835700] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.835717] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.835716] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835738] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835742] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.835756] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.835766] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.835783] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835784] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x476 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.835797] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x373 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.835807] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.835835] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.835806] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.835850] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835868] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835880] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa82 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.835892] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.835894] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835907] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835915] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.835923] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.835939] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.835941] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.835954] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.835956] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.835970] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x478 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.835980] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.836017] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.836047] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836059] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.836079] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.836083] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.836106] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.836107] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836131] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836141] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.836156] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.836158] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.836174] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.836181] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836195] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836204] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836217] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.836221] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.836269] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6cd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.836268] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.836294] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.836301] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.184
[2019-01-07 19:16:44.836310] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836317] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1373794485
[2019-01-07 19:16:44.836331] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184
[2019-01-07 19:16:44.836341] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.836348] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.836363] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.836364] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.836378] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.836404] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836429] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.836441] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x968 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.836455] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836457] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.836480] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836487] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.836505] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.836506] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836531] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836531] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.836557] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa27 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.836573] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.836577] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x964 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.836594] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.836598] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836612] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836633] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.836641] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6c9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.836648] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.836666] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.836682] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836703] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.836731] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836737] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x35a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.836758] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.836763] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.836774] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.836793] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836819] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836802] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836848] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.836860] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.836877] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.836889] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.836901] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.836904] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.836919] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836930] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x360 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.836938] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x898 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.836951] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.836952] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.836960] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.836981] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x781 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.836992] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.836995] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.837003] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837017] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837023] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.837036] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.837035] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.837048] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.837056] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837068] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837084] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x966 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.837086] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.837101] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.837119] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837114] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.837136] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.837138] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837146] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.837168] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5fd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.837174] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.837186] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.837204] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837223] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837240] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.837287] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x412 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.837305] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.837318] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837336] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837352] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.837191] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837390] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.837391] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.837426] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.837426] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.837452] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837464] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837490] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.837489] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa81 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.837501] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.837512] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.837517] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837535] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837541] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.837558] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.837566] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.837582] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.837585] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.837613] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837609] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837641] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.837645] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x372 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.837651] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.837662] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.837668] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837687] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837693] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x5ff Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.837710] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.837708] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837725] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837727] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.837741] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x290 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.837751] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.837774] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.837781] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.837791] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.837803] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837806] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837829] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.837834] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.837840] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.837857] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.837866] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837884] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837890] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.837906] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.837916] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa26 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.837921] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.837935] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.837944] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x374 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.837956] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.837957] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.837982] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.837988] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.837997] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838004] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.838019] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa83 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.838019] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838037] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.838040] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838053] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838065] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.838071] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.838094] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.838108] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x477 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.838121] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838139] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x479 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.838149] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.838147] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.838171] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838180] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.838195] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838202] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838216] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.838233] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.838232] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.838245] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.838258] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838266] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838288] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.838298] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.838320] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838321] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.838335] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.838338] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.838353] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838369] T [MSGID: 0] [shard.c:2195:shard_common_lookup_shards] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.838383] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.838388] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.185
[2019-01-07 19:16:44.838412] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.838415] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1285124113
[2019-01-07 19:16:44.838433] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185
[2019-01-07 19:16:44.838445] D [MSGID: 0] [dht-common.c:3441:dht_lookup] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185, checking on all the subvols to see if it is a directory
[2019-01-07 19:16:44.838438] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838463] D [MSGID: 0] [dht-common.c:3455:dht_lookup] 0-gv1-dht: Found null hashed subvol. Calling lookup on all nodes.
[2019-01-07 19:16:44.838472] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.838468] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838494] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.838502] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.838528] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838542] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6ce Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.838562] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.838548] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.838576] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838593] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.838613] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838617] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x35f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.838633] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa28 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.838637] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.838643] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.838669] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838687] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.838687] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838696] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.838722] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x969 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.838728] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.838741] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.838749] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838765] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838773] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.838785] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.838784] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838805] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838807] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.838821] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.838831] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-1 to gv1-client-5
[2019-01-07 19:16:44.838849] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838848] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x965 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.838863] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x361 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.838865] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.838874] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.838888] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.838907] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.838924] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.838922] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6ca Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.838964] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.838965] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.838981] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.838982] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839000] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839016] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x967 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.839019] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x35b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.839025] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-2 to gv1-client-8
[2019-01-07 19:16:44.839040] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.839046] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839063] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839069] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x35d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.839087] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.839086] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839113] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.839123] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.839144] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839161] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.839159] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.839171] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.839181] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.839187] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839201] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839207] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.839217] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-3 to gv1-client-11
[2019-01-07 19:16:44.839225] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x899 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.839232] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839243] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.839249] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.839253] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839258] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.839273] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x782 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.839282] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.839283] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.839300] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839300] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839324] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x28f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.839325] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.839341] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.839342] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.839358] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839357] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839376] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x600 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.839383] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.839385] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-4 to gv1-client-14
[2019-01-07 19:16:44.839424] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.839430] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839450] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839457] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x291 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.839472] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.839481] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5fe Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.839498] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.839504] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.839528] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839525] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839547] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839553] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x97f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.839570] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.839571] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.839587] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839604] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.839612] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-5 to gv1-client-17
[2019-01-07 19:16:44.839614] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x413 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.839628] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839631] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.839643] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x375 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.839646] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839652] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.839673] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839690] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.839692] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.839708] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839731] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa84 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.839733] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.839749] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.839752] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.839767] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839767] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839784] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7e9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.839793] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-6 to gv1-client-20
[2019-01-07 19:16:44.839806] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839815] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa82 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.839821] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x47a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.839835] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.839838] T [MSGID: 0] [dht-common.c:3462:dht_lookup] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.839859] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839877] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.839895] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839891] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e7 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.839915] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.839918] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.839939] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.839942] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.839958] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.839972] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x373 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.839980] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.839990] T [MSGID: 0] [afr-common.c:3348:afr_lookup_do] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-7 to gv1-client-23
[2019-01-07 19:16:44.839988] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.840006] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 728, payload: 640, rpc hdr: 88
[2019-01-07 19:16:44.840009] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840022] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x416 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) to rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.840034] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840052] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.840062] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.840078] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.840091] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840091] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.840114] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa27 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.840114] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.840132] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.840133] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840143] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840164] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x478 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.840166] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.840187] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.840190] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.840208] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840208] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840234] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840239] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840260] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.840279] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.840292] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.840300] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840302] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.840319] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.840333] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840339] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.840358] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.840363] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6cf Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.840367] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840382] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.840386] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840412] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.840414] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840440] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x360 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.840451] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.840450] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.840462] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840475] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.840486] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x966 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.840491] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840497] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.840511] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840512] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840535] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.840543] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6cb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.840559] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.840570] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840589] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x35c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.840598] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.840598] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.840606] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840625] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.840626] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840652] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.840654] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840680] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.840686] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x783 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.840691] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.840705] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.840707] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840729] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840736] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x290 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.840753] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.840762] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840761] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.840781] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x5ff Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.840784] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.840791] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.840809] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840809] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840835] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840843] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x414 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.840845] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.840860] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.840876] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e3 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.840875] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840895] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.840900] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840906] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840922] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.840931] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa83 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.840943] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.840953] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.840961] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e8 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.840970] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x374 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.840980] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.840978] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.840992] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841010] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841010] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841021] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.841045] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.841043] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.841065] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.841068] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.841074] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841089] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841096] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa28 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.841118] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-1: not found on remote node
[2019-01-07 19:16:44.841127] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841125] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x479 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.841146] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.841150] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.841156] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.841173] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841175] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841199] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841202] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841229] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.841249] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.841267] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.841270] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.841290] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-0: not found on remote node
[2019-01-07 19:16:44.841289] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841309] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841341] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.841356] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-2: not found on remote node
[2019-01-07 19:16:44.841366] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841371] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6d0 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.841390] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-3: not found on remote node
[2019-01-07 19:16:44.841388] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841417] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-0 returned error [No such file or directory]
[2019-01-07 19:16:44.841418] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841447] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-5: received rpc message (RPC XID: 0x361 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-5)
[2019-01-07 19:16:44.841448] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.841466] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-5: not found on remote node
[2019-01-07 19:16:44.841467] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-4: not found on remote node
[2019-01-07 19:16:44.841483] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841484] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841502] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-1 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841504] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x967 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.841521] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-7: not found on remote node
[2019-01-07 19:16:44.841521] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-1 returned error [No such file or directory]
[2019-01-07 19:16:44.841534] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841556] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6cc Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.841565] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-6: not found on remote node
[2019-01-07 19:16:44.841564] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-8: received rpc message (RPC XID: 0x35d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-8)
[2019-01-07 19:16:44.841576] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841589] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-8: not found on remote node
[2019-01-07 19:16:44.841599] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa1f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.841604] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-8 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841621] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-10: not found on remote node
[2019-01-07 19:16:44.841625] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-2 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841631] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-10 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841651] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-2 returned error [No such file or directory]
[2019-01-07 19:16:44.841658] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.841676] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-12: not found on remote node
[2019-01-07 19:16:44.841688] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-12 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841693] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x784 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.841706] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-14: received rpc message (RPC XID: 0x291 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-14)
[2019-01-07 19:16:44.841709] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-9: not found on remote node
[2019-01-07 19:16:44.841716] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-14: not found on remote node
[2019-01-07 19:16:44.841726] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-9 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841726] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-14 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841760] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x600 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.841760] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x97f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.841776] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-13: not found on remote node
[2019-01-07 19:16:44.841779] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-15: not found on remote node
[2019-01-07 19:16:44.841785] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-13 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841800] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-4 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841801] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-15 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841811] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-4 returned error [No such file or directory]
[2019-01-07 19:16:44.841840] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-11: received rpc message (RPC XID: 0x415 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-11)
[2019-01-07 19:16:44.841845] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e4 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.841859] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-11: not found on remote node
[2019-01-07 19:16:44.841862] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-16: not found on remote node
[2019-01-07 19:16:44.841880] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-16 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841882] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-11 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841898] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa84 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.841903] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-3 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841908] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-18: not found on remote node
[2019-01-07 19:16:44.841920] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-18 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841922] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-3 returned error [No such file or directory]
[2019-01-07 19:16:44.841939] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7e9 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.841950] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-19: not found on remote node
[2019-01-07 19:16:44.841960] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-19 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.841962] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-17: received rpc message (RPC XID: 0x375 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-17)
[2019-01-07 19:16:44.841977] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.841978] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-17: not found on remote node
[2019-01-07 19:16:44.841994] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-21: not found on remote node
[2019-01-07 19:16:44.841995] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-17 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842004] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-21 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842022] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-5 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842028] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.842048] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-22: not found on remote node
[2019-01-07 19:16:44.842049] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-5 returned error [No such file or directory]
[2019-01-07 19:16:44.842057] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-22 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842081] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-20: received rpc message (RPC XID: 0x47a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-20)
[2019-01-07 19:16:44.842091] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-20: not found on remote node
[2019-01-07 19:16:44.842090] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-23: received rpc message (RPC XID: 0x416 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 27) from rpc-transport (gv1-client-23)
[2019-01-07 19:16:44.842107] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-20 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842109] T [MSGID: 0] [client-rpc-fops_v2.c:2543:client4_0_lookup_cbk] 0-gv1-client-23: not found on remote node
[2019-01-07 19:16:44.842126] D [MSGID: 0] [client-rpc-fops_v2.c:2548:client4_0_lookup_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-23 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842126] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-6 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842148] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-6 returned error [No such file or directory]
[2019-01-07 19:16:44.842147] D [MSGID: 0] [afr-common.c:2512:afr_lookup_done] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-7 returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.842173] D [MSGID: 0] [dht-common.c:1378:dht_lookup_dir_cbk] 0-gv1-dht: lookup of /.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185 on gv1-replicate-7 returned error [No such file or directory]
[2019-01-07 19:16:44.842189] E [MSGID: 101046] [dht-common.c:1502:dht_lookup_dir_cbk] 0-gv1-dht: dict is null
[2019-01-07 19:16:44.842201] D [MSGID: 0] [dht-common.c:1505:dht_lookup_dir_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: No such file or directory [No such file or directory]
[2019-01-07 19:16:44.843510] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.843587] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.843712] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.843729] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843754] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.843770] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.843782] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843798] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa29 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.843808] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:44.843820] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:44.843830] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843844] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.843853] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:44.843863] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843877] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.843885] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:44.843897] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:44.843907] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843921] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.843930] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:44.843939] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.843953] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x968 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.843965] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:44.843977] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:44.843986] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844002] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.844010] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:44.844020] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844033] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.844044] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:44.844058] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:44.844068] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844093] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.844102] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:44.844113] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844128] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x601 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.844137] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:44.844131] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa29 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.844157] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:44.844158] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-1 returned 0
[2019-01-07 19:16:44.844169] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844184] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.844191] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x980 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.844194] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-0 returned 0
[2019-01-07 19:16:44.844215] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-0 returned 0
[2019-01-07 19:16:44.844216] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:44.844234] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844253] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.844259] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-0': avail_percent is: 22.00 and avail_space is: 9153228115968 and avail_inodes is: 99.00
[2019-01-07 19:16:44.844267] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:44.844285] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:44.844293] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6d1 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:44.844302] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844305] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-3 returned 0
[2019-01-07 19:16:44.844334] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:44.844336] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa85 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.844350] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-12 returned 0
[2019-01-07 19:16:44.844357] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:44.844374] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844379] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6cd Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:44.844391] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7ea Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.844398] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-6 returned 0
[2019-01-07 19:16:44.844414] T [MSGID: 0] [dht-diskusage.c:207:dht_get_du_info] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:44.844429] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:44.844435] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:44.844439] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844460] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-4 returned 0
[2019-01-07 19:16:44.844472] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.844475] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-1 returned 0
[2019-01-07 19:16:44.844482] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f67443f1578, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:44.844495] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 152, payload: 64, rpc hdr: 88
[2019-01-07 19:16:44.844499] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-1': avail_percent is: 23.00 and avail_space is: 9570127962112 and avail_inodes is: 99.00
[2019-01-07 19:16:44.844510] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.844524] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.2
[2019-01-07 19:16:44.844532] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x968 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:44.844535] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1223621823
[2019-01-07 19:16:44.844552] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2
[2019-01-07 19:16:44.844556] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-7 returned 0
[2019-01-07 19:16:44.844560] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.2
[2019-01-07 19:16:44.844589] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.844589] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-2 returned 0
[2019-01-07 19:16:44.844601] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 2 failed: Input/output error
[2019-01-07 19:16:44.844620] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-2': avail_percent is: 23.00 and avail_space is: 9299665338368 and avail_inodes is: 99.00
[2019-01-07 19:16:44.844645] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.844649] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa20 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:44.844664] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.5
[2019-01-07 19:16:44.844665] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-10 returned 0
[2019-01-07 19:16:44.844683] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1087134360
[2019-01-07 19:16:44.844692] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5
[2019-01-07 19:16:44.844694] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x785 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:44.844699] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.5
[2019-01-07 19:16:44.844715] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.844717] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-9 returned 0
[2019-01-07 19:16:44.844734] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 5 failed: Input/output error
[2019-01-07 19:16:44.844734] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-3 returned 0
[2019-01-07 19:16:44.844757] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-3': avail_percent is: 23.00 and avail_space is: 9231467974656 and avail_inodes is: 99.00
[2019-01-07 19:16:44.844764] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.844778] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.9
[2019-01-07 19:16:44.844786] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1085903184
[2019-01-07 19:16:44.844786] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x980 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:44.844793] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9
[2019-01-07 19:16:44.844807] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.9
[2019-01-07 19:16:44.844810] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-15 returned 0
[2019-01-07 19:16:44.844816] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.844837] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 9 failed: Input/output error
[2019-01-07 19:16:44.844844] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x601 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:44.844862] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.844859] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-13 returned 0
[2019-01-07 19:16:44.844873] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.21
[2019-01-07 19:16:44.844888] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1223600709
[2019-01-07 19:16:44.844891] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-4 returned 0
[2019-01-07 19:16:44.844895] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21
[2019-01-07 19:16:44.844912] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.21
[2019-01-07 19:16:44.844919] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-4': avail_percent is: 19.00 and avail_space is: 7846502244352 and avail_inodes is: 99.00
[2019-01-07 19:16:44.844920] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.844943] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 21 failed: Input/output error
[2019-01-07 19:16:44.844955] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e5 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:44.844968] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.844969] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-16 returned 0
[2019-01-07 19:16:44.844978] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.29
[2019-01-07 19:16:44.844989] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1240589523
[2019-01-07 19:16:44.845000] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29
[2019-01-07 19:16:44.844989] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-5 returned 0
[2019-01-07 19:16:44.845007] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.29
[2019-01-07 19:16:44.845022] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845027] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-5': avail_percent is: 19.00 and avail_space is: 7783764815872 and avail_inodes is: 99.00
[2019-01-07 19:16:44.845031] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 29 failed: Input/output error
[2019-01-07 19:16:44.845058] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa85 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:44.845065] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845085] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.31
[2019-01-07 19:16:44.845087] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-18 returned 0
[2019-01-07 19:16:44.845117] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7ea Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:44.845125] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1319971604
[2019-01-07 19:16:44.845127] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-19 returned 0
[2019-01-07 19:16:44.845159] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-6 returned 0
[2019-01-07 19:16:44.845160] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31
[2019-01-07 19:16:44.845177] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.31
[2019-01-07 19:16:44.845183] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-6': avail_percent is: 22.00 and avail_space is: 9016154447872 and avail_inodes is: 99.00
[2019-01-07 19:16:44.845187] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845199] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 31 failed: Input/output error
[2019-01-07 19:16:44.845211] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:44.845222] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-21 returned 0
[2019-01-07 19:16:44.845232] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845240] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:44.845243] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.35
[2019-01-07 19:16:44.845255] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1413476598
[2019-01-07 19:16:44.845257] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-client-22 returned 0
[2019-01-07 19:16:44.845268] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f67443f1578, gv1-replicate-7 returned 0
[2019-01-07 19:16:44.845269] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35
[2019-01-07 19:16:44.845279] D [MSGID: 0] [dht-diskusage.c:97:dht_du_info_cbk] 0-gv1-dht: subvolume 'gv1-replicate-7': avail_percent is: 22.00 and avail_space is: 9128420376576 and avail_inodes is: 99.00
[2019-01-07 19:16:44.845282] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.35
[2019-01-07 19:16:44.845293] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845311] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 35 failed: Input/output error
[2019-01-07 19:16:44.845338] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845347] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.40
[2019-01-07 19:16:44.845355] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1252560672
[2019-01-07 19:16:44.845361] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40
[2019-01-07 19:16:44.845367] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.40
[2019-01-07 19:16:44.845375] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845384] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 40 failed: Input/output error
[2019-01-07 19:16:44.845416] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845429] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.68
[2019-01-07 19:16:44.845437] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1394291141
[2019-01-07 19:16:44.845443] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68
[2019-01-07 19:16:44.845450] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.68
[2019-01-07 19:16:44.845457] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845466] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 68 failed: Input/output error
[2019-01-07 19:16:44.845490] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845499] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.124
[2019-01-07 19:16:44.845507] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1080394250
[2019-01-07 19:16:44.845513] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124
[2019-01-07 19:16:44.845520] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.124
[2019-01-07 19:16:44.845527] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845536] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 124 failed: Input/output error
[2019-01-07 19:16:44.845558] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845566] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.130
[2019-01-07 19:16:44.845574] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1277578615
[2019-01-07 19:16:44.845580] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130
[2019-01-07 19:16:44.845589] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.130
[2019-01-07 19:16:44.845597] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845605] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 130 failed: Input/output error
[2019-01-07 19:16:44.845627] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845636] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.139
[2019-01-07 19:16:44.845643] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1342729215
[2019-01-07 19:16:44.845650] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139
[2019-01-07 19:16:44.845656] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.139
[2019-01-07 19:16:44.845663] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845671] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 139 failed: Input/output error
[2019-01-07 19:16:44.845693] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845701] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.152
[2019-01-07 19:16:44.845708] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1288142377
[2019-01-07 19:16:44.845715] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152
[2019-01-07 19:16:44.845721] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.152
[2019-01-07 19:16:44.845728] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845737] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 152 failed: Input/output error
[2019-01-07 19:16:44.845758] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845766] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.168
[2019-01-07 19:16:44.845773] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1374046734
[2019-01-07 19:16:44.845780] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168
[2019-01-07 19:16:44.845786] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.168
[2019-01-07 19:16:44.845793] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845802] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 168 failed: Input/output error
[2019-01-07 19:16:44.845823] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845834] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.170
[2019-01-07 19:16:44.845847] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1111239957
[2019-01-07 19:16:44.845854] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170
[2019-01-07 19:16:44.845860] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.170
[2019-01-07 19:16:44.845867] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845876] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 170 failed: Input/output error
[2019-01-07 19:16:44.845902] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845911] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.173
[2019-01-07 19:16:44.845918] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1135612244
[2019-01-07 19:16:44.845925] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173
[2019-01-07 19:16:44.845931] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.173
[2019-01-07 19:16:44.845939] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.845949] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 173 failed: Input/output error
[2019-01-07 19:16:44.845971] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.845980] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.181
[2019-01-07 19:16:44.845987] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1353889563
[2019-01-07 19:16:44.845996] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181
[2019-01-07 19:16:44.846002] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.181
[2019-01-07 19:16:44.846009] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846018] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 181 failed: Input/output error
[2019-01-07 19:16:44.846039] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.846048] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.183
[2019-01-07 19:16:44.846055] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1394995644
[2019-01-07 19:16:44.846061] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183
[2019-01-07 19:16:44.846067] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.183
[2019-01-07 19:16:44.846074] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846085] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 183 failed: Input/output error
[2019-01-07 19:16:44.846107] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.846115] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.184
[2019-01-07 19:16:44.846123] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1373794485
[2019-01-07 19:16:44.846129] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184
[2019-01-07 19:16:44.846135] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.184
[2019-01-07 19:16:44.846142] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846151] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 184 failed: Input/output error
[2019-01-07 19:16:44.846172] T [MSGID: 0] [shard.c:3763:shard_common_resume_mknod] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.846180] T [MSGID: 0] [dht-hashfn.c:94:dht_hash_compute] 0-gv1-dht: trying regex for aa3ef10e-95e0-40d3-9464-133d72fa8a95.185
[2019-01-07 19:16:44.846187] W [MSGID: 109011] [dht-layout.c:163:dht_layout_search] 0-gv1-dht: no subvolume for hash (value) = 1285124113
[2019-01-07 19:16:44.846194] D [MSGID: 0] [dht-helper.c:969:dht_subvol_get_hashed] 0-gv1-dht: No hashed subvolume for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185
[2019-01-07 19:16:44.846200] D [MSGID: 0] [dht-common.c:7631:dht_mknod] 0-gv1-dht: no subvolume in layout for path=/.shard/aa3ef10e-95e0-40d3-9464-133d72fa8a95.185
[2019-01-07 19:16:44.846207] D [MSGID: 0] [dht-common.c:7712:dht_mknod] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846215] D [MSGID: 0] [shard.c:3645:shard_common_mknod_cbk] 0-gv1-shard: mknod of shard 185 failed: Input/output error
[2019-01-07 19:16:44.846223] D [MSGID: 0] [shard.c:720:shard_common_inode_write_failure_unwind] 0-stack-trace: stack-address: 0x7f6748006778, gv1-shard returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846234] D [MSGID: 0] [defaults.c:1352:default_fallocate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-quick-read returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846244] D [MSGID: 0] [defaults.c:1352:default_fallocate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-open-behind returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846254] D [MSGID: 0] [md-cache.c:2715:mdc_fallocate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-md-cache returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846264] D [MSGID: 0] [defaults.c:1352:default_fallocate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-threads returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846274] D [MSGID: 0] [io-stats.c:2528:io_stats_fallocate_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned -1 error: Input/output error [Input/output error]
[2019-01-07 19:16:44.846284] W [fuse-bridge.c:1441:fuse_err_cbk] 0-glusterfs-fuse: 1373: FALLOCATE() ERR => -1 (Input/output error)
[2019-01-07 19:16:44.846298] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 16/16 
[2019-01-07 19:16:44.846630] T [fuse-bridge.c:2652:fuse_flush] 0-glusterfs-fuse: 1374: FLUSH 0x7f6748005418
[2019-01-07 19:16:44.846652] T [MSGID: 0] [fuse-bridge.c:2631:fuse_flush_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from fuse to meta-autoload
[2019-01-07 19:16:44.846669] T [MSGID: 0] [defaults.c:2407:default_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from meta-autoload to gv1
[2019-01-07 19:16:44.846679] T [MSGID: 0] [io-stats.c:3049:io_stats_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:44.846689] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: FLUSH scheduled as normal fop
[2019-01-07 19:16:44.846710] T [MSGID: 0] [defaults.c:1722:default_flush_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:44.846735] T [MSGID: 0] [defaults.c:2407:default_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:44.846753] T [MSGID: 0] [defaults.c:1722:default_flush_resume] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:44.846765] T [MSGID: 0] [defaults.c:2407:default_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:44.846777] T [MSGID: 0] [defaults.c:2407:default_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:44.846792] T [MSGID: 0] [defaults.c:2407:default_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:44.846805] T [MSGID: 0] [read-ahead.c:602:ra_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:44.846819] T [MSGID: 0] [write-behind.c:1969:wb_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:44.846832] T [MSGID: 0] [shard.c:4603:shard_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:44.846847] T [MSGID: 0] [dht-inode-read.c:836:dht_flush] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:44.846869] T [MSGID: 0] [afr-common.c:3645:afr_flush_wrapper] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:44.846890] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 124, payload: 36, rpc hdr: 88
[2019-01-07 19:16:44.846920] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x78f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.846936] T [MSGID: 0] [afr-common.c:3645:afr_flush_wrapper] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:44.846953] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 124, payload: 36, rpc hdr: 88
[2019-01-07 19:16:44.846977] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa2a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.846991] T [MSGID: 0] [afr-common.c:3645:afr_flush_wrapper] 0-stack-trace: stack-address: 0x7f6748006778, winding from gv1-replicate-0 to gv1-client-2
[2019-01-07 19:16:44.847007] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 124, payload: 36, rpc hdr: 88
[2019-01-07 19:16:44.847030] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x41f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.847175] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa2a Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.847193] T [MSGID: 0] [client-lk.c:385:delete_granted_locks_owner] 0-gv1-client-1: Number of locks cleared=0
[2019-01-07 19:16:44.847205] T [MSGID: 0] [client-rpc-fops_v2.c:707:client4_0_flush_cbk] 0-gv1-client-1: deleting locks of owner (ac824f7f6618b134) returned 0
[2019-01-07 19:16:44.847213] T [MSGID: 0] [client-rpc-fops_v2.c:721:client4_0_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-1 returned 0
[2019-01-07 19:16:44.847236] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x78f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.847256] T [MSGID: 0] [client-lk.c:385:delete_granted_locks_owner] 0-gv1-client-0: Number of locks cleared=0
[2019-01-07 19:16:44.847268] T [MSGID: 0] [client-rpc-fops_v2.c:707:client4_0_flush_cbk] 0-gv1-client-0: deleting locks of owner (ac824f7f6618b134) returned 0
[2019-01-07 19:16:44.847276] T [MSGID: 0] [client-rpc-fops_v2.c:721:client4_0_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-0 returned 0
[2019-01-07 19:16:44.847278] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x41f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 15) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.847296] T [MSGID: 0] [client-lk.c:385:delete_granted_locks_owner] 0-gv1-client-2: Number of locks cleared=0
[2019-01-07 19:16:44.847308] T [MSGID: 0] [client-rpc-fops_v2.c:707:client4_0_flush_cbk] 0-gv1-client-2: deleting locks of owner (ac824f7f6618b134) returned 0
[2019-01-07 19:16:44.847315] T [MSGID: 0] [client-rpc-fops_v2.c:721:client4_0_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-client-2 returned 0
[2019-01-07 19:16:44.847323] T [MSGID: 0] [afr-common.c:3622:afr_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-replicate-0 returned 0
[2019-01-07 19:16:44.847332] T [MSGID: 0] [dht-inode-read.c:770:dht_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-dht returned 0
[2019-01-07 19:16:44.847339] T [MSGID: 0] [shard.c:4595:shard_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-shard returned 0
[2019-01-07 19:16:44.847347] T [MSGID: 0] [defaults.c:1109:default_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-write-behind returned 0
[2019-01-07 19:16:44.847355] T [MSGID: 0] [read-ahead.c:565:ra_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-read-ahead returned 0
[2019-01-07 19:16:44.847362] T [MSGID: 0] [defaults.c:1109:default_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-open-behind returned 0
[2019-01-07 19:16:44.847370] T [MSGID: 0] [defaults.c:1109:default_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1-io-threads returned 0
[2019-01-07 19:16:44.847379] T [MSGID: 0] [io-stats.c:2356:io_stats_flush_cbk] 0-stack-trace: stack-address: 0x7f6748006778, gv1 returned 0
[2019-01-07 19:16:44.847386] T [fuse-bridge.c:1431:fuse_err_cbk] 0-glusterfs-fuse: 1374: FLUSH() ERR => 0
[2019-01-07 19:16:44.847403] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 16/16 
[2019-01-07 19:16:44.847424] T [fuse-bridge.c:2690:fuse_release] 0-glusterfs-fuse: 1375: RELEASE 0x7f6748005418
[2019-01-07 19:16:44.847446] T [MSGID: 0] [client-rpc-fops.c:3194:client_fdctx_destroy] 0-gv1-client-1: sending release on fd
[2019-01-07 19:16:44.847460] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 108, payload: 28, rpc hdr: 80
[2019-01-07 19:16:44.847482] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa2b Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.847495] T [MSGID: 0] [client-rpc-fops.c:3194:client_fdctx_destroy] 0-gv1-client-2: sending release on fd
[2019-01-07 19:16:44.847505] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 108, payload: 28, rpc hdr: 80
[2019-01-07 19:16:44.847520] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x420 Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) to rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.847530] T [MSGID: 0] [client-rpc-fops.c:3194:client_fdctx_destroy] 0-gv1-client-0: sending release on fd
[2019-01-07 19:16:44.847539] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 108, payload: 28, rpc hdr: 80
[2019-01-07 19:16:44.847557] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x790 Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:44.847573] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 16/16 
[2019-01-07 19:16:44.847707] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa2b Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:44.847770] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x420 Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) from rpc-transport (gv1-client-2)
[2019-01-07 19:16:44.847863] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x790 Program: GlusterFS 3.3, ProgVers: 330, Proc: 41) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:48.643531] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-0: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643566] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-1: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643581] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-3: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643592] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-4: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643600] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-6: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643616] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-7: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643627] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-9: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643640] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-10: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643651] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-12: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643660] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-13: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643669] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-15: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643677] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-16: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643688] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-18: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643699] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-19: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643711] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-21: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:48.643722] D [rpc-clnt-ping.c:335:rpc_clnt_start_ping] 0-gv1-client-22: returning as transport is already disconnected OR there are no frames (0 || 0)
[2019-01-07 19:16:52.769158] T [fuse-bridge.c:3273:fuse_statfs_resume] 0-glusterfs-fuse: 1376: STATFS
[2019-01-07 19:16:52.769190] T [MSGID: 0] [fuse-bridge.c:3276:fuse_statfs_resume] 0-stack-trace: stack-address: 0x7f6748005f58, winding from fuse to meta-autoload
[2019-01-07 19:16:52.769209] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from meta-autoload to gv1
[2019-01-07 19:16:52.769224] T [MSGID: 0] [io-stats.c:3035:io_stats_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:52.769239] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: STATFS scheduled as fast fop
[2019-01-07 19:16:52.769264] T [MSGID: 0] [defaults.c:2275:default_statfs_resume] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:52.769296] T [MSGID: 0] [md-cache.c:1080:mdc_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:52.769312] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:52.769325] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:52.769336] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:52.769348] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:52.769362] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:52.769372] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:52.769383] T [MSGID: 0] [defaults.c:3089:default_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:52.769407] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:52.769432] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-0 to gv1-client-0
[2019-01-07 19:16:52.769450] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.769477] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x791 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:52.770375] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.4:49156: ping timer event already removed
[2019-01-07 19:16:52.770407] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.770414] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x791 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:52.770440] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x792 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-0)
[2019-01-07 19:16:52.770445] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-0 returned 0
[2019-01-07 19:16:52.770455] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:52.770484] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.770506] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa2c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.770674] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.2:49156: ping timer event already removed
[2019-01-07 19:16:52.770687] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-0: received rpc message (RPC XID: 0x792 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-0)
[2019-01-07 19:16:52.770699] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.770712] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-0: Ping latency is 0ms
[2019-01-07 19:16:52.770733] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa2d Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.770747] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:52.770769] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-1 to gv1-client-3
[2019-01-07 19:16:52.770782] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.770798] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa2c Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.770805] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:52.770844] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-1 returned 0
[2019-01-07 19:16:52.770859] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-0 returned 0
[2019-01-07 19:16:52.770867] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa2d Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.770901] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-1: Ping latency is 0ms
[2019-01-07 19:16:52.770978] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.2:49157: ping timer event already removed
[2019-01-07 19:16:52.770999] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.771025] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6d3 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-3)
[2019-01-07 19:16:52.771039] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:52.771053] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.771076] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.771127] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6d2 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:52.771150] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-3 returned 0
[2019-01-07 19:16:52.771185] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-3: received rpc message (RPC XID: 0x6d3 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-3)
[2019-01-07 19:16:52.771209] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-3: Ping latency is 0ms
[2019-01-07 19:16:52.771216] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.4:49157: ping timer event already removed
[2019-01-07 19:16:52.771236] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.771260] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96e Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.771273] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:52.771288] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-2 to gv1-client-6
[2019-01-07 19:16:52.771305] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.771326] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6ce Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:52.771391] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96e Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.771451] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-4: Ping latency is 0ms
[2019-01-07 19:16:52.771486] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.771490] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.4:49158: ping timer event already removed
[2019-01-07 19:16:52.771520] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-4 returned 0
[2019-01-07 19:16:52.771522] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.771544] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-1 returned 0
[2019-01-07 19:16:52.771555] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6cf Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-6)
[2019-01-07 19:16:52.771569] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:52.771581] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.771600] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x969 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.771603] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6ce Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:52.771617] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-6 returned 0
[2019-01-07 19:16:52.771694] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-6: received rpc message (RPC XID: 0x6cf Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-6)
[2019-01-07 19:16:52.771704] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-6: Ping latency is 0ms
[2019-01-07 19:16:52.771744] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.2:49158: ping timer event already removed
[2019-01-07 19:16:52.771764] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.771787] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96a Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.771799] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:52.771814] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-3 to gv1-client-9
[2019-01-07 19:16:52.771829] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.771848] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:52.771932] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x96a Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.771949] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-7: Ping latency is 0ms
[2019-01-07 19:16:52.771983] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.4:49159: ping timer event already removed
[2019-01-07 19:16:52.772002] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.772003] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x969 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.772023] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-7 returned 0
[2019-01-07 19:16:52.772028] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x787 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-9)
[2019-01-07 19:16:52.772039] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-2 returned 0
[2019-01-07 19:16:52.772052] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:52.772071] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.772097] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.772159] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x786 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:52.772182] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-9 returned 0
[2019-01-07 19:16:52.772242] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.2:49159: ping timer event already removed
[2019-01-07 19:16:52.772263] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.772289] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa22 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.772293] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-9: received rpc message (RPC XID: 0x787 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-9)
[2019-01-07 19:16:52.772329] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:52.772326] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-9: Ping latency is 0ms
[2019-01-07 19:16:52.772356] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:52.772375] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.772405] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa21 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.772419] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.772437] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-10 returned 0
[2019-01-07 19:16:52.772456] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-3 returned 0
[2019-01-07 19:16:52.772481] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa22 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.772494] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-10: Ping latency is 0ms
[2019-01-07 19:16:52.772601] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.1:49156: ping timer event already removed
[2019-01-07 19:16:52.772627] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.772633] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89d Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.772647] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-12 returned 0
[2019-01-07 19:16:52.772661] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89e Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.772678] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-4 to gv1-client-13
[2019-01-07 19:16:52.772698] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.772711] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89e Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.772727] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-12: Ping latency is 0ms
[2019-01-07 19:16:52.772727] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x602 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:52.772915] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.3:49156: ping timer event already removed
[2019-01-07 19:16:52.772947] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.772949] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x602 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:52.772976] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-13 returned 0
[2019-01-07 19:16:52.772981] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x603 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-13)
[2019-01-07 19:16:52.772997] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-4 returned 0
[2019-01-07 19:16:52.773008] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:52.773032] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:52.773052] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.773057] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-13: received rpc message (RPC XID: 0x603 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-13)
[2019-01-07 19:16:52.773072] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-13: Ping latency is 0ms
[2019-01-07 19:16:52.773094] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x981 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.773284] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.1:49157: ping timer event already removed
[2019-01-07 19:16:52.773312] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.773316] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x981 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.773337] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-15 returned 0
[2019-01-07 19:16:52.773348] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x982 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.773367] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-5 to gv1-client-16
[2019-01-07 19:16:52.773398] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.773410] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x982 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.773426] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-15: Ping latency is 0ms
[2019-01-07 19:16:52.773431] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:52.773617] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.3:49157: ping timer event already removed
[2019-01-07 19:16:52.773645] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.773652] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e6 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:52.773674] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x6e7 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-16)
[2019-01-07 19:16:52.773674] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-16 returned 0
[2019-01-07 19:16:52.773701] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-5 returned 0
[2019-01-07 19:16:52.773702] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:52.773730] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:52.773750] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.773762] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-16: received rpc message (RPC XID: 0x6e7 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-16)
[2019-01-07 19:16:52.773776] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-16: Ping latency is 0ms
[2019-01-07 19:16:52.773788] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa86 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.773971] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.1:49158: ping timer event already removed
[2019-01-07 19:16:52.773999] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.774001] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa86 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.774023] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-18 returned 0
[2019-01-07 19:16:52.774036] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa87 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.774055] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-6 to gv1-client-19
[2019-01-07 19:16:52.774073] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.774101] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa87 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.774101] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7eb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:52.774112] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-18: Ping latency is 0ms
[2019-01-07 19:16:52.774312] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.3:49158: ping timer event already removed
[2019-01-07 19:16:52.774338] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7eb Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:52.774339] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.774353] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-19 returned 0
[2019-01-07 19:16:52.774368] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-6 returned 0
[2019-01-07 19:16:52.774379] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x7ec Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-19)
[2019-01-07 19:16:52.774405] T [MSGID: 0] [dht-common.c:6262:dht_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:52.774431] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:52.774450] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.774480] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-19: received rpc message (RPC XID: 0x7ec Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-19)
[2019-01-07 19:16:52.774493] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-19: Ping latency is 0ms
[2019-01-07 19:16:52.774489] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa22 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.774681] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.1:49159: ping timer event already removed
[2019-01-07 19:16:52.774706] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.774711] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa22 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.774736] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa23 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.774734] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-21 returned 0
[2019-01-07 19:16:52.774764] T [MSGID: 0] [afr-common.c:4473:afr_statfs] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-7 to gv1-client-22
[2019-01-07 19:16:52.774793] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.774823] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x787 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:52.774819] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa23 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.774844] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-21: Ping latency is 0ms
[2019-01-07 19:16:52.774976] D [rpc-clnt-ping.c:99:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13b)[0x7f6767c372eb] (--> /lib64/libgfrpc.so.0(+0x1256b)[0x7f6767a0656b] (--> /lib64/libgfrpc.so.0(+0x12d31)[0x7f6767a06d31] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x4b1)[0x7f6767a03571] (--> /usr/lib64/glusterfs/4.1.6/xlator/protocol/client.so(+0x13fa1)[0x7f6759ebbfa1] ))))) 0-: 10.17.102.3:49159: ping timer event already removed
[2019-01-07 19:16:52.774998] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 80, payload: 0, rpc hdr: 80
[2019-01-07 19:16:52.775002] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x787 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 14) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:52.775016] T [MSGID: 0] [client-rpc-fops_v2.c:608:client4_0_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-22 returned 0
[2019-01-07 19:16:52.775023] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x788 Program: GF-DUMP, ProgVers: 1, Proc: 2) to rpc-transport (gv1-client-22)
[2019-01-07 19:16:52.775036] T [MSGID: 0] [afr-common.c:4433:afr_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-7 returned 0
[2019-01-07 19:16:52.775048] T [MSGID: 0] [dht-common.c:6205:dht_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-dht returned 0
[2019-01-07 19:16:52.775058] T [MSGID: 0] [md-cache.c:1041:mdc_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-md-cache returned 0
[2019-01-07 19:16:52.775066] T [MSGID: 0] [defaults.c:1608:default_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-io-threads returned 0
[2019-01-07 19:16:52.775075] T [MSGID: 0] [io-stats.c:2415:io_stats_statfs_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1 returned 0
[2019-01-07 19:16:52.775089] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 96/96 
[2019-01-07 19:16:52.775102] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-22: received rpc message (RPC XID: 0x788 Program: GF-DUMP, ProgVers: 1, Proc: 2) from rpc-transport (gv1-client-22)
[2019-01-07 19:16:52.775116] T [fuse-bridge.c:946:fuse_getattr_resume] 0-glusterfs-fuse: 1377: GETATTR 1 (/)
[2019-01-07 19:16:52.775120] D [rpc-clnt-ping.c:210:rpc_clnt_ping_cbk] 0-gv1-client-22: Ping latency is 0ms
[2019-01-07 19:16:52.775153] T [MSGID: 0] [fuse-bridge.c:949:fuse_getattr_resume] 0-stack-trace: stack-address: 0x7f6748005f58, winding from fuse to meta-autoload
[2019-01-07 19:16:52.775174] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from meta-autoload to gv1
[2019-01-07 19:16:52.775189] T [MSGID: 0] [io-stats.c:2803:io_stats_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1 to gv1-io-threads
[2019-01-07 19:16:52.775203] D [MSGID: 0] [io-threads.c:372:iot_schedule] 0-gv1-io-threads: STAT scheduled as fast fop
[2019-01-07 19:16:52.775225] T [MSGID: 0] [defaults.c:2034:default_stat_resume] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-io-threads to gv1-md-cache
[2019-01-07 19:16:52.775253] T [MSGID: 0] [md-cache.c:519:mdc_inode_iatt_get] 0-md-cache: iatt cache not valid for (00000000-0000-0000-0000-000000000001)
[2019-01-07 19:16:52.775265] T [MSGID: 0] [md-cache.c:1269:mdc_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-md-cache to gv1-open-behind
[2019-01-07 19:16:52.775274] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-open-behind to gv1-quick-read
[2019-01-07 19:16:52.775283] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-quick-read to gv1-io-cache
[2019-01-07 19:16:52.775292] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-io-cache to gv1-readdir-ahead
[2019-01-07 19:16:52.775299] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-readdir-ahead to gv1-read-ahead
[2019-01-07 19:16:52.775312] T [MSGID: 0] [defaults.c:2791:default_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-read-ahead to gv1-write-behind
[2019-01-07 19:16:52.775322] T [MSGID: 0] [write-behind.c:2060:wb_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-write-behind to gv1-shard
[2019-01-07 19:16:52.775332] T [MSGID: 0] [shard.c:1578:shard_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-shard to gv1-dht
[2019-01-07 19:16:52.775345] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-0
[2019-01-07 19:16:52.775359] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-0: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775372] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-0 to gv1-client-1
[2019-01-07 19:16:52.775388] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775421] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa2e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.775433] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-1
[2019-01-07 19:16:52.775451] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-1: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775463] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-1 to gv1-client-4
[2019-01-07 19:16:52.775473] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775500] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.775516] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-2
[2019-01-07 19:16:52.775538] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-2: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775552] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-2 to gv1-client-7
[2019-01-07 19:16:52.775568] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775593] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.775609] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-3
[2019-01-07 19:16:52.775635] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-3: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775650] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-3 to gv1-client-10
[2019-01-07 19:16:52.775665] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775690] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa23 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.775705] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-4
[2019-01-07 19:16:52.775727] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-4: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775741] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-4 to gv1-client-12
[2019-01-07 19:16:52.775742] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-1: received rpc message (RPC XID: 0xa2e Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-1)
[2019-01-07 19:16:52.775759] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775774] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-1 returned 0
[2019-01-07 19:16:52.775788] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-0 returned 0
[2019-01-07 19:16:52.775795] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x89f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.775814] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-5
[2019-01-07 19:16:52.775837] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-5: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.775850] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-5 to gv1-client-15
[2019-01-07 19:16:52.775856] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-4: received rpc message (RPC XID: 0x96f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-4)
[2019-01-07 19:16:52.775866] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.775883] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-4 returned 0
[2019-01-07 19:16:52.775941] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-12: received rpc message (RPC XID: 0x89f Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-12)
[2019-01-07 19:16:52.775904] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x983 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.775979] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-6
[2019-01-07 19:16:52.775984] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-12 returned 0
[2019-01-07 19:16:52.775981] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-1 returned 0
[2019-01-07 19:16:52.776018] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-4 returned 0
[2019-01-07 19:16:52.776020] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-6: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.776036] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-7: received rpc message (RPC XID: 0x96b Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-7)
[2019-01-07 19:16:52.776053] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-6 to gv1-client-18
[2019-01-07 19:16:52.776057] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-15: received rpc message (RPC XID: 0x983 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-15)
[2019-01-07 19:16:52.776072] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.776092] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-7 returned 0
[2019-01-07 19:16:52.776094] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-15 returned 0
[2019-01-07 19:16:52.776108] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa88 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.776124] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-2 returned 0
[2019-01-07 19:16:52.776122] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-5 returned 0
[2019-01-07 19:16:52.776155] T [MSGID: 0] [dht-inode-read.c:368:dht_stat] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-dht to gv1-replicate-7
[2019-01-07 19:16:52.776181] D [MSGID: 0] [afr-read-txn.c:252:afr_read_txn] 0-gv1-replicate-7: 00000000-0000-0000-0000-000000000001: generation now vs cached: 3, 3
[2019-01-07 19:16:52.776195] T [MSGID: 0] [afr-inode-read.c:234:afr_stat_wind] 0-stack-trace: stack-address: 0x7f6748005f58, winding from gv1-replicate-7 to gv1-client-21
[2019-01-07 19:16:52.776209] T [rpc-clnt.c:1494:rpc_clnt_record_build_header] 0-rpc-clnt: Request fraglen 112, payload: 28, rpc hdr: 84
[2019-01-07 19:16:52.776160] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-10: received rpc message (RPC XID: 0xa23 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-10)
[2019-01-07 19:16:52.776233] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-10 returned 0
[2019-01-07 19:16:52.776242] T [rpc-clnt.c:1784:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0xa24 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) to rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.776246] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-3 returned 0
[2019-01-07 19:16:52.776271] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-18: received rpc message (RPC XID: 0xa88 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-18)
[2019-01-07 19:16:52.776290] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-18 returned 0
[2019-01-07 19:16:52.776307] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-6 returned 0
[2019-01-07 19:16:52.776348] T [rpc-clnt.c:673:rpc_clnt_reply_init] 0-gv1-client-21: received rpc message (RPC XID: 0xa24 Program: GlusterFS 4.x v1, ProgVers: 400, Proc: 1) from rpc-transport (gv1-client-21)
[2019-01-07 19:16:52.776364] T [MSGID: 0] [client-rpc-fops_v2.c:333:client4_0_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-client-21 returned 0
[2019-01-07 19:16:52.776376] T [MSGID: 0] [afr-inode-read.c:210:afr_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-replicate-7 returned 0
[2019-01-07 19:16:52.776388] T [MSGID: 0] [dht-inode-read.c:312:dht_attr_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-dht returned 0
[2019-01-07 19:16:52.776419] T [MSGID: 0] [defaults.c:1385:default_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-shard returned 0
[2019-01-07 19:16:52.776435] T [MSGID: 0] [defaults.c:1385:default_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-write-behind returned 0
[2019-01-07 19:16:52.776494] T [md-cache.c:491:mdc_inode_iatt_set_validate] (-->/lib64/libglusterfs.so.0(default_stat_cbk+0x15b) [0x7f6767cc172b] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0xc3ae) [0x7f67588c53ae] -->/usr/lib64/glusterfs/4.1.6/xlator/performance/md-cache.so(+0x8286) [0x7f67588c1286] ) 0-md-cache: Updated iatt(00000000-0000-0000-0000-000000000001) time:1546888612 
[2019-01-07 19:16:52.776506] T [MSGID: 0] [md-cache.c:1231:mdc_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-md-cache returned 0
[2019-01-07 19:16:52.776519] T [MSGID: 0] [defaults.c:1385:default_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1-io-threads returned 0
[2019-01-07 19:16:52.776534] T [MSGID: 0] [io-stats.c:2109:io_stats_stat_cbk] 0-stack-trace: stack-address: 0x7f6748005f58, gv1 returned 0
[2019-01-07 19:16:52.776548] T [fuse-bridge.c:847:fuse_attr_cbk] 0-glusterfs-fuse: 1377: STAT() / => 1
[2019-01-07 19:16:52.776566] T [fuse-bridge.c:278:send_fuse_iov] 0-glusterfs-fuse: writev() result 120/120

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users