Attached, bricks log. Where could I find the fuse client log? On Fri, Nov 18, 2016 at 2:22 AM, Krutika Dhananjay <kdhananj@xxxxxxxxxx> wrote: > Could you attach the fuse client and brick logs? > > -Krutika > > On Fri, Nov 18, 2016 at 6:12 AM, Olivier Lambert <lambert.olivier@xxxxxxxxx> > wrote: >> >> Okay, used the exact same config you provided, and adding an arbiter >> node (node3) >> >> After halting node2, VM continues to work after a small "lag"/freeze. >> I restarted node2 and it was back online: OK >> >> Then, after waiting few minutes, halting node1. And **just** at this >> moment, the VM is corrupted (segmentation fault, /var/log folder empty >> etc.) >> >> dmesg of the VM: >> >> [ 1645.852905] EXT4-fs error (device xvda1): >> htree_dirblock_to_tree:988: inode #19: block 8286: comm bash: bad >> entry in directory: rec_len is smaller than minimal - offset=0(0), >> inode=0, rec_len=0, name_len=0 >> [ 1645.854509] Aborting journal on device xvda1-8. >> [ 1645.855524] EXT4-fs (xvda1): Remounting filesystem read-only >> >> And got a lot of " comm bash: bad entry in directory" messages then... >> >> Here is the current config with all Node back online: >> >> # gluster volume info >> >> Volume Name: gv0 >> Type: Replicate >> Volume ID: 5f15c919-57e3-4648-b20a-395d9fe3d7d6 >> Status: Started >> Snapshot Count: 0 >> Number of Bricks: 1 x (2 + 1) = 3 >> Transport-type: tcp >> Bricks: >> Brick1: 10.0.0.1:/bricks/brick1/gv0 >> Brick2: 10.0.0.2:/bricks/brick1/gv0 >> Brick3: 10.0.0.3:/bricks/brick1/gv0 (arbiter) >> Options Reconfigured: >> nfs.disable: on >> performance.readdir-ahead: on >> transport.address-family: inet >> features.shard: on >> features.shard-block-size: 16MB >> network.remote-dio: enable >> cluster.eager-lock: enable >> performance.io-cache: off >> performance.read-ahead: off >> performance.quick-read: off >> performance.stat-prefetch: on >> performance.strict-write-ordering: off >> cluster.server-quorum-type: server >> cluster.quorum-type: auto >> cluster.data-self-heal: on >> >> >> # gluster volume status >> Status of volume: gv0 >> Gluster process TCP Port RDMA Port Online >> Pid >> >> ------------------------------------------------------------------------------ >> Brick 10.0.0.1:/bricks/brick1/gv0 49152 0 Y >> 1331 >> Brick 10.0.0.2:/bricks/brick1/gv0 49152 0 Y >> 2274 >> Brick 10.0.0.3:/bricks/brick1/gv0 49152 0 Y >> 2355 >> Self-heal Daemon on localhost N/A N/A Y >> 2300 >> Self-heal Daemon on 10.0.0.3 N/A N/A Y >> 10530 >> Self-heal Daemon on 10.0.0.2 N/A N/A Y >> 2425 >> >> Task Status of Volume gv0 >> >> ------------------------------------------------------------------------------ >> There are no active volume tasks >> >> >> >> On Thu, Nov 17, 2016 at 11:35 PM, Olivier Lambert >> <lambert.olivier@xxxxxxxxx> wrote: >> > It's planned to have an arbiter soon :) It was just preliminary tests. >> > >> > Thanks for the settings, I'll test this soon and I'll come back to you! >> > >> > On Thu, Nov 17, 2016 at 11:29 PM, Lindsay Mathieson >> > <lindsay.mathieson@xxxxxxxxx> wrote: >> >> On 18/11/2016 8:17 AM, Olivier Lambert wrote: >> >>> >> >>> gluster volume info gv0 >> >>> >> >>> Volume Name: gv0 >> >>> Type: Replicate >> >>> Volume ID: 2f8658ed-0d9d-4a6f-a00b-96e9d3470b53 >> >>> Status: Started >> >>> Snapshot Count: 0 >> >>> Number of Bricks: 1 x 2 = 2 >> >>> Transport-type: tcp >> >>> Bricks: >> >>> Brick1: 10.0.0.1:/bricks/brick1/gv0 >> >>> Brick2: 10.0.0.2:/bricks/brick1/gv0 >> >>> Options Reconfigured: >> >>> nfs.disable: on >> >>> performance.readdir-ahead: on >> >>> transport.address-family: inet >> >>> features.shard: on >> >>> features.shard-block-size: 16MB >> >> >> >> >> >> >> >> When hosting VM's its essential to set these options: >> >> >> >> network.remote-dio: enable >> >> cluster.eager-lock: enable >> >> performance.io-cache: off >> >> performance.read-ahead: off >> >> performance.quick-read: off >> >> performance.stat-prefetch: on >> >> performance.strict-write-ordering: off >> >> cluster.server-quorum-type: server >> >> cluster.quorum-type: auto >> >> cluster.data-self-heal: on >> >> >> >> Also with replica two and quorum on (required) your volume will become >> >> read-only when one node goes down to prevent the possibility of >> >> split-brain >> >> - you *really* want to avoid that :) >> >> >> >> I'd recommend a replica 3 volume, that way 1 node can go down, but the >> >> other >> >> two still form a quorum and will remain r/w. >> >> >> >> If the extra disks are not possible, then a Arbiter volume can be setup >> >> - >> >> basically dummy files on the third node. >> >> >> >> >> >> >> >> -- >> >> Lindsay Mathieson >> >> >> >> _______________________________________________ >> >> Gluster-users mailing list >> >> Gluster-users@xxxxxxxxxxx >> >> http://www.gluster.org/mailman/listinfo/gluster-users >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users@xxxxxxxxxxx >> http://www.gluster.org/mailman/listinfo/gluster-users > >
[2016-11-17 23:39:02.213316] I [MSGID: 100030] [glusterfsd.c:2454:main] 0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version 3.8.5 (args: /usr/sbin/glusterfsd -s 10.0.0.2 --volfile-id gv0.10.0.0.2.bricks-brick1-gv0 -p /var/lib/glusterd/vols/gv0/run/10.0.0.2-bricks-brick1-gv0.pid -S /var/run/gluster/005a9f9af69ac8abebb1340413388f36.socket --brick-name /bricks/brick1/gv0 -l /var/log/glusterfs/bricks/bricks-brick1-gv0.log --xlator-option *-posix.glusterd-uuid=e2230f1e-a4a1-4e20-87e8-6e6c4b115510 --brick-port 49152 --xlator-option gv0-server.listen-port=49152) [2016-11-17 23:39:02.219663] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2016-11-17 23:39:02.240366] I [MSGID: 101173] [graph.c:269:gf_add_cmdline_options] 0-gv0-server: adding option 'listen-port' for volume 'gv0-server' with value '49152' [2016-11-17 23:39:02.240400] I [MSGID: 101173] [graph.c:269:gf_add_cmdline_options] 0-gv0-posix: adding option 'glusterd-uuid' for volume 'gv0-posix' with value 'e2230f1e-a4a1-4e20-87e8-6e6c4b115510' [2016-11-17 23:39:02.240607] I [MSGID: 115034] [server.c:398:_check_for_auth_option] 0-gv0-decompounder: skip format check for non-addr auth option auth.login./bricks/brick1/gv0.allow [2016-11-17 23:39:02.240617] I [MSGID: 115034] [server.c:398:_check_for_auth_option] 0-gv0-decompounder: skip format check for non-addr auth option auth.login.d9b06a73-97bc-45a4-a7a7-18285b048f4a.password [2016-11-17 23:39:02.240619] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2016-11-17 23:39:02.241551] I [rpcsvc.c:2214:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured rpc.outstanding-rpc-limit with value 64 [2016-11-17 23:39:02.241601] W [MSGID: 101002] [options.c:954:xl_opt_validate] 0-gv0-server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction [2016-11-17 23:39:02.246104] I [MSGID: 121050] [ctr-helper.c:259:extract_ctr_options] 0-gfdbdatastore: CTR Xlator is disabled. [2016-11-17 23:39:02.246126] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-pagesize from params.Assigning default value: 4096 [2016-11-17 23:39:02.246137] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-journalmode from params.Assigning default value: wal [2016-11-17 23:39:02.246144] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-sync from params.Assigning default value: off [2016-11-17 23:39:02.246150] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-autovacuum from params.Assigning default value: none [2016-11-17 23:39:02.250234] I [trash.c:2414:init] 0-gv0-trash: no option specified for 'eliminate', using NULL [2016-11-17 23:39:02.250857] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-glusterfs' is not recognized [2016-11-17 23:39:02.250878] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-unix' is not recognized [2016-11-17 23:39:02.250893] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-null' is not recognized [2016-11-17 23:39:02.250920] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'auth-path' is not recognized [2016-11-17 23:39:02.250938] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-quota: option 'timeout' is not recognized [2016-11-17 23:39:02.250977] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-trash: option 'brick-path' is not recognized Final graph: +------------------------------------------------------------------------------+ 1: volume gv0-posix 2: type storage/posix 3: option glusterd-uuid e2230f1e-a4a1-4e20-87e8-6e6c4b115510 4: option directory /bricks/brick1/gv0 5: option volume-id 5f15c919-57e3-4648-b20a-395d9fe3d7d6 6: end-volume 7: 8: volume gv0-trash 9: type features/trash 10: option trash-dir .trashcan 11: option brick-path /bricks/brick1/gv0 12: option trash-internal-op off 13: subvolumes gv0-posix 14: end-volume 15: 16: volume gv0-changetimerecorder 17: type features/changetimerecorder 18: option db-type sqlite3 19: option hot-brick off 20: option db-name gv0.db 21: option db-path /bricks/brick1/gv0/.glusterfs/ 22: option record-exit off 23: option ctr_link_consistency off 24: option ctr_lookupheal_link_timeout 300 25: option ctr_lookupheal_inode_timeout 300 26: option record-entry on 27: option ctr-enabled off 28: option record-counters off 29: option ctr-record-metadata-heat off 30: option sql-db-cachesize 1000 31: option sql-db-wal-autocheckpoint 1000 32: subvolumes gv0-trash 33: end-volume 34: 35: volume gv0-changelog 36: type features/changelog 37: option changelog-brick /bricks/brick1/gv0 38: option changelog-dir /bricks/brick1/gv0/.glusterfs/changelogs 39: option changelog-barrier-timeout 120 40: subvolumes gv0-changetimerecorder 41: end-volume 42: 43: volume gv0-bitrot-stub 44: type features/bitrot-stub 45: option export /bricks/brick1/gv0 46: subvolumes gv0-changelog 47: end-volume 48: 49: volume gv0-access-control 50: type features/access-control 51: subvolumes gv0-bitrot-stub 52: end-volume 53: 54: volume gv0-locks 55: type features/locks 56: subvolumes gv0-access-control 57: end-volume 58: 59: volume gv0-worm 60: type features/worm 61: option worm off 62: option worm-file-level off 63: subvolumes gv0-locks 64: end-volume 65: 66: volume gv0-read-only 67: type features/read-only 68: option read-only off 69: subvolumes gv0-worm 70: end-volume 71: 72: volume gv0-leases 73: type features/leases 74: option leases off 75: subvolumes gv0-read-only 76: end-volume 77: 78: volume gv0-upcall 79: type features/upcall 80: option cache-invalidation off 81: subvolumes gv0-leases 82: end-volume 83: 84: volume gv0-io-threads 85: type performance/io-threads 86: subvolumes gv0-upcall 87: end-volume 88: 89: volume gv0-marker 90: type features/marker 91: option volume-uuid 5f15c919-57e3-4648-b20a-395d9fe3d7d6 92: option timestamp-file /var/lib/glusterd/vols/gv0/marker.tstamp 93: option quota-version 0 94: option xtime off 95: option gsync-force-xtime off 96: option quota off 97: option inode-quota off 98: subvolumes gv0-io-threads 99: end-volume 100: 101: volume gv0-barrier 102: type features/barrier 103: option barrier disable 104: option barrier-timeout 120 105: subvolumes gv0-marker 106: end-volume 107: 108: volume gv0-index 109: type features/index 110: option index-base /bricks/brick1/gv0/.glusterfs/indices 111: option xattrop-dirty-watchlist trusted.afr.dirty 112: option xattrop-pending-watchlist trusted.afr.gv0- 113: subvolumes gv0-barrier 114: end-volume 115: 116: volume gv0-quota 117: type features/quota 118: option volume-uuid gv0 119: option server-quota off 120: option timeout 0 121: option deem-statfs off 122: subvolumes gv0-index 123: end-volume 124: 125: volume /bricks/brick1/gv0 126: type debug/io-stats 127: option log-level INFO 128: option latency-measurement off 129: option count-fop-hits off 130: subvolumes gv0-quota 131: end-volume 132: 133: volume gv0-decompounder 134: type performance/decompounder 135: subvolumes /bricks/brick1/gv0 136: end-volume 137: 138: volume gv0-server 139: type protocol/server 140: option transport.socket.listen-port 49152 141: option rpc-auth.auth-glusterfs on 142: option rpc-auth.auth-unix on 143: option rpc-auth.auth-null on 144: option rpc-auth-allow-insecure on 145: option transport-type tcp 146: option transport.address-family inet 147: option auth.login./bricks/brick1/gv0.allow d9b06a73-97bc-45a4-a7a7-18285b048f4a 148: option auth.login.d9b06a73-97bc-45a4-a7a7-18285b048f4a.password 0c6d4cdf-bbf0-4900-aefb-3d75c41d4a23 149: option auth-path /bricks/brick1/gv0 150: option auth.addr./bricks/brick1/gv0.allow * 151: subvolumes gv0-decompounder 152: end-volume 153: +------------------------------------------------------------------------------+ [2016-11-17 23:39:06.130307] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:39:06.130338] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan1-2328-2016/11/17-23:39:02:99217-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:39:06.287053] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:39:06.287072] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan2-2395-2016/11/17-23:39:02:260600-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:39:06.291732] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:39:06.291748] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan3-2374-2016/11/17-23:39:02:263531-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:39:29.354185] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.359295] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.359964] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.359981] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.403073] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.408522] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.409089] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.409111] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.453385] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.458643] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.460224] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.460245] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.504572] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.509923] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.511047] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.511070] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.556583] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.561947] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.562990] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.563024] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.607373] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.612855] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.613537] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.613560] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.659175] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.664786] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.665782] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.665807] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.710811] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.716577] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.717190] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.717213] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.763300] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.768854] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.770043] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.770063] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.817589] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.823159] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.824490] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.824514] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.870336] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.875872] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.877608] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.877629] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.923612] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.929270] I [glusterfsd-mgmt.c:54:mgmt_cbk_spec] 0-mgmt: Volume file changed [2016-11-17 23:39:29.930065] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:39:29.930093] I [glusterfsd-mgmt.c:1596:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2016-11-17 23:41:50.329300] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:41:50.329330] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan1-2765-2016/11/17-23:41:50:274723-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:42:38.585904] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:42:38.585922] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan2-2777-2016/11/17-23:42:38:539985-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:44:01.162871] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan1-2765-2016/11/17-23:41:50:274723-gv0-client-1-0-0 [2016-11-17 23:44:01.162911] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan1-2765-2016/11/17-23:41:50:274723-gv0-client-1-0-0 [2016-11-17 23:44:18.629630] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan1-2328-2016/11/17-23:39:02:99217-gv0-client-1-0-0 [2016-11-17 23:44:18.629661] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan1-2328-2016/11/17-23:39:02:99217-gv0-client-1-0-0 [2016-11-17 23:44:24.572329] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:44:24.572357] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-1132-2016/11/17-23:44:23:893895-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:44:26.627306] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan2-2395-2016/11/17-23:39:02:260600-gv0-client-1-0-0 [2016-11-17 23:44:26.627340] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan2-2395-2016/11/17-23:39:02:260600-gv0-client-1-0-0 [2016-11-17 23:44:26.864422] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan3-2374-2016/11/17-23:39:02:263531-gv0-client-1-0-0 [2016-11-17 23:44:26.864451] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan3-2374-2016/11/17-23:39:02:263531-gv0-client-1-0-0 [2016-11-17 23:44:30.599411] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:44:30.599429] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-5173-2016/11/17-23:44:27:547143-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:44:31.648189] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:44:31.648205] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan2-2821-2016/11/17-23:44:27:628936-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:44:31.886027] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:44:31.886051] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan3-2684-2016/11/17-23:44:27:865379-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:45:09.700134] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan2-2777-2016/11/17-23:42:38:539985-gv0-client-1-0-0 [2016-11-17 23:45:09.700173] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan2-2777-2016/11/17-23:42:38:539985-gv0-client-1-0-0 [2016-11-17 23:45:09.750575] W [socket.c:590:__socket_rwv] 0-glusterfs: readv on 10.0.0.2:24007 failed (No data available) [2016-11-17 23:45:09.750645] E [glusterfsd-mgmt.c:1902:mgmt_rpc_notify] 0-glusterfsd-mgmt: failed to connect with remote-host: 10.0.0.2 (No data available) [2016-11-17 23:45:09.750660] I [glusterfsd-mgmt.c:1919:mgmt_rpc_notify] 0-glusterfsd-mgmt: Exhausted all volfile servers [2016-11-17 23:45:10.488638] W [glusterfsd.c:1327:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7dc5) [0x7f5703b73dc5] -->/usr/sbin/glusterfsd(glusterfs_sigwaiter+0xe5) [0x7f5705207cd5] -->/usr/sbin/glusterfsd(cleanup_and_exit+0x6b) [0x7f5705207b4b] ) 0-: received signum (15), shutting down [2016-11-17 23:45:10.613024] E [socket.c:3087:socket_connect] 0-glusterfs: connection attempt on 10.0.0.2:24007 failed, (Network is unreachable) [2016-11-17 23:45:10.613063] I [socket.c:3391:socket_submit_request] 0-glusterfs: not connected (priv->connected = 0) [2016-11-17 23:45:10.613071] W [rpc-clnt.c:1640:rpc_clnt_submit] 0-glusterfs: failed to submit rpc-request (XID: 0x1b Program: Gluster Portmap, ProgVers: 1, Proc: 5) to rpc-transport (glusterfs) [2016-11-17 23:45:10.613264] E [glusterfsd-mgmt.c:1902:mgmt_rpc_notify] 0-glusterfsd-mgmt: failed to connect with remote-host: 10.0.0.2 (Transport endpoint is not connected) [2016-11-17 23:45:10.613279] I [glusterfsd-mgmt.c:1919:mgmt_rpc_notify] 0-glusterfsd-mgmt: Exhausted all volfile servers [2016-11-17 23:45:31.794763] I [MSGID: 100030] [glusterfsd.c:2454:main] 0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version 3.8.5 (args: /usr/sbin/glusterfsd -s 10.0.0.2 --volfile-id gv0.10.0.0.2.bricks-brick1-gv0 -p /var/lib/glusterd/vols/gv0/run/10.0.0.2-bricks-brick1-gv0.pid -S /var/run/gluster/005a9f9af69ac8abebb1340413388f36.socket --brick-name /bricks/brick1/gv0 -l /var/log/glusterfs/bricks/bricks-brick1-gv0.log --xlator-option *-posix.glusterd-uuid=e2230f1e-a4a1-4e20-87e8-6e6c4b115510 --brick-port 49152 --xlator-option gv0-server.listen-port=49152) [2016-11-17 23:45:31.822423] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2016-11-17 23:45:31.859327] I [MSGID: 101173] [graph.c:269:gf_add_cmdline_options] 0-gv0-server: adding option 'listen-port' for volume 'gv0-server' with value '49152' [2016-11-17 23:45:31.859354] I [MSGID: 101173] [graph.c:269:gf_add_cmdline_options] 0-gv0-posix: adding option 'glusterd-uuid' for volume 'gv0-posix' with value 'e2230f1e-a4a1-4e20-87e8-6e6c4b115510' [2016-11-17 23:45:31.860475] I [MSGID: 115034] [server.c:398:_check_for_auth_option] 0-gv0-decompounder: skip format check for non-addr auth option auth.login./bricks/brick1/gv0.allow [2016-11-17 23:45:31.860500] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2016-11-17 23:45:31.860490] I [MSGID: 115034] [server.c:398:_check_for_auth_option] 0-gv0-decompounder: skip format check for non-addr auth option auth.login.d9b06a73-97bc-45a4-a7a7-18285b048f4a.password [2016-11-17 23:45:31.865203] I [rpcsvc.c:2214:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured rpc.outstanding-rpc-limit with value 64 [2016-11-17 23:45:31.865268] W [MSGID: 101002] [options.c:954:xl_opt_validate] 0-gv0-server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction [2016-11-17 23:45:31.910039] I [MSGID: 121050] [ctr-helper.c:259:extract_ctr_options] 0-gfdbdatastore: CTR Xlator is disabled. [2016-11-17 23:45:31.910062] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-pagesize from params.Assigning default value: 4096 [2016-11-17 23:45:31.910071] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-journalmode from params.Assigning default value: wal [2016-11-17 23:45:31.910079] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-sync from params.Assigning default value: off [2016-11-17 23:45:31.910085] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-autovacuum from params.Assigning default value: none [2016-11-17 23:45:31.916956] I [trash.c:2414:init] 0-gv0-trash: no option specified for 'eliminate', using NULL [2016-11-17 23:45:31.929100] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-glusterfs' is not recognized [2016-11-17 23:45:31.929134] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-unix' is not recognized [2016-11-17 23:45:31.929151] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-null' is not recognized [2016-11-17 23:45:31.929180] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'auth-path' is not recognized [2016-11-17 23:45:31.929198] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-quota: option 'timeout' is not recognized [2016-11-17 23:45:31.929246] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-trash: option 'brick-path' is not recognized [2016-11-17 23:45:31.931428] W [MSGID: 113026] [posix.c:1487:posix_mkdir] 0-gv0-posix: mkdir (/.trashcan/): gfid (00000000-0000-0000-0000-000000000005) is already associated with directory (/bricks/brick1/gv0/.glusterfs/00/00/00000000-0000-0000-0000-000000000001/.trashcan). Hence, both directories will share same gfid and this can lead to inconsistencies. [2016-11-17 23:45:31.931454] E [MSGID: 113027] [posix.c:1594:posix_mkdir] 0-gv0-posix: mkdir of /bricks/brick1/gv0/.trashcan/ failed [File exists] [2016-11-17 23:45:31.931520] W [MSGID: 113026] [posix.c:1487:posix_mkdir] 0-gv0-posix: mkdir (/.trashcan/internal_op): gfid (00000000-0000-0000-0000-000000000006) is already associated with directory (/bricks/brick1/gv0/.glusterfs/00/00/00000000-0000-0000-0000-000000000005/internal_op). Hence, both directories will share same gfid and this can lead to inconsistencies. [2016-11-17 23:45:31.931539] E [MSGID: 113027] [posix.c:1594:posix_mkdir] 0-gv0-posix: mkdir of /bricks/brick1/gv0/.trashcan/internal_op failed [File exists] Final graph: +------------------------------------------------------------------------------+ 1: volume gv0-posix 2: type storage/posix 3: option glusterd-uuid e2230f1e-a4a1-4e20-87e8-6e6c4b115510 4: option directory /bricks/brick1/gv0 5: option volume-id 5f15c919-57e3-4648-b20a-395d9fe3d7d6 6: end-volume 7: 8: volume gv0-trash 9: type features/trash 10: option trash-dir .trashcan 11: option brick-path /bricks/brick1/gv0 12: option trash-internal-op off 13: subvolumes gv0-posix 14: end-volume 15: 16: volume gv0-changetimerecorder 17: type features/changetimerecorder 18: option db-type sqlite3 19: option hot-brick off 20: option db-name gv0.db 21: option db-path /bricks/brick1/gv0/.glusterfs/ 22: option record-exit off 23: option ctr_link_consistency off 24: option ctr_lookupheal_link_timeout 300 25: option ctr_lookupheal_inode_timeout 300 26: option record-entry on 27: option ctr-enabled off 28: option record-counters off 29: option ctr-record-metadata-heat off 30: option sql-db-cachesize 1000 31: option sql-db-wal-autocheckpoint 1000 32: subvolumes gv0-trash 33: end-volume 34: 35: volume gv0-changelog 36: type features/changelog 37: option changelog-brick /bricks/brick1/gv0 38: option changelog-dir /bricks/brick1/gv0/.glusterfs/changelogs 39: option changelog-barrier-timeout 120 40: subvolumes gv0-changetimerecorder 41: end-volume 42: 43: volume gv0-bitrot-stub 44: type features/bitrot-stub 45: option export /bricks/brick1/gv0 46: subvolumes gv0-changelog 47: end-volume 48: 49: volume gv0-access-control 50: type features/access-control 51: subvolumes gv0-bitrot-stub 52: end-volume 53: 54: volume gv0-locks 55: type features/locks 56: subvolumes gv0-access-control 57: end-volume 58: 59: volume gv0-worm 60: type features/worm 61: option worm off 62: option worm-file-level off 63: subvolumes gv0-locks 64: end-volume 65: 66: volume gv0-read-only 67: type features/read-only 68: option read-only off 69: subvolumes gv0-worm 70: end-volume 71: 72: volume gv0-leases 73: type features/leases 74: option leases off 75: subvolumes gv0-read-only 76: end-volume 77: 78: volume gv0-upcall 79: type features/upcall 80: option cache-invalidation off 81: subvolumes gv0-leases 82: end-volume 83: 84: volume gv0-io-threads 85: type performance/io-threads 86: subvolumes gv0-upcall 87: end-volume 88: 89: volume gv0-marker 90: type features/marker 91: option volume-uuid 5f15c919-57e3-4648-b20a-395d9fe3d7d6 92: option timestamp-file /var/lib/glusterd/vols/gv0/marker.tstamp 93: option quota-version 0 94: option xtime off 95: option gsync-force-xtime off 96: option quota off 97: option inode-quota off 98: subvolumes gv0-io-threads 99: end-volume 100: 101: volume gv0-barrier 102: type features/barrier 103: option barrier disable 104: option barrier-timeout 120 105: subvolumes gv0-marker 106: end-volume 107: 108: volume gv0-index 109: type features/index 110: option index-base /bricks/brick1/gv0/.glusterfs/indices 111: option xattrop-dirty-watchlist trusted.afr.dirty 112: option xattrop-pending-watchlist trusted.afr.gv0- 113: subvolumes gv0-barrier 114: end-volume 115: 116: volume gv0-quota 117: type features/quota 118: option volume-uuid gv0 119: option server-quota off 120: option timeout 0 121: option deem-statfs off 122: subvolumes gv0-index 123: end-volume 124: 125: volume /bricks/brick1/gv0 126: type debug/io-stats 127: option log-level INFO 128: option latency-measurement off 129: option count-fop-hits off 130: subvolumes gv0-quota 131: end-volume 132: 133: volume gv0-decompounder 134: type performance/decompounder 135: subvolumes /bricks/brick1/gv0 136: end-volume 137: 138: volume gv0-server 139: type protocol/server 140: option transport.socket.listen-port 49152 141: option rpc-auth.auth-glusterfs on 142: option rpc-auth.auth-unix on 143: option rpc-auth.auth-null on 144: option rpc-auth-allow-insecure on 145: option transport-type tcp 146: option transport.address-family inet 147: option auth.login./bricks/brick1/gv0.allow d9b06a73-97bc-45a4-a7a7-18285b048f4a 148: option auth.login.d9b06a73-97bc-45a4-a7a7-18285b048f4a.password 0c6d4cdf-bbf0-4900-aefb-3d75c41d4a23 149: option auth-path /bricks/brick1/gv0 150: option auth.addr./bricks/brick1/gv0.allow * 151: subvolumes gv0-decompounder 152: end-volume 153: +------------------------------------------------------------------------------+ [2016-11-17 23:45:32.063450] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:45:32.063492] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-1120-2016/11/17-23:45:31:877988-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:45:37.235263] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:45:37.235290] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-10178-2016/11/17-23:45:33:721214-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:45:37.491421] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:45:37.491443] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-1132-2016/11/17-23:44:23:893895-gv0-client-1-0-1 (version: 3.8.5) [2016-11-17 23:45:38.176878] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:45:38.176897] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-4918-2016/11/17-23:45:35:140882-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:45:39.574718] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-17 23:45:39.574742] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan3-2706-2016/11/17-23:45:35:882821-gv0-client-1-0-0 (version: 3.8.5) [2016-11-17 23:57:10.158750] E [MSGID: 113002] [posix.c:252:posix_lookup] 0-gv0-posix: buf->ia_gfid is null for /bricks/brick1/gv0/.shard/90548459-452d-4129-80f5-b5670b073440.23 [No data available] [2016-11-18 00:09:17.857708] E [MSGID: 113002] [posix.c:252:posix_lookup] 0-gv0-posix: buf->ia_gfid is null for /bricks/brick1/gv0/.shard/90548459-452d-4129-80f5-b5670b073440.272 [No data available] [2016-11-18 00:09:53.605789] E [MSGID: 113002] [posix.c:252:posix_lookup] 0-gv0-posix: buf->ia_gfid is null for /bricks/brick1/gv0/.shard/90548459-452d-4129-80f5-b5670b073440.279 [No data available] [2016-11-18 00:10:12.006504] E [MSGID: 113002] [posix.c:252:posix_lookup] 0-gv0-posix: buf->ia_gfid is null for /bricks/brick1/gv0/.shard/90548459-452d-4129-80f5-b5670b073440.373 [No data available] [2016-11-18 00:14:40.872997] E [MSGID: 113002] [posix.c:252:posix_lookup] 0-gv0-posix: buf->ia_gfid is null for /bricks/brick1/gv0/.shard/90548459-452d-4129-80f5-b5670b073440.598 [No data available] [2016-11-18 00:27:16.215497] I [MSGID: 100030] [glusterfsd.c:2454:main] 0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version 3.8.5 (args: /usr/sbin/glusterfsd -s 10.0.0.2 --volfile-id gv0.10.0.0.2.bricks-brick1-gv0 -p /var/lib/glusterd/vols/gv0/run/10.0.0.2-bricks-brick1-gv0.pid -S /var/run/gluster/005a9f9af69ac8abebb1340413388f36.socket --brick-name /bricks/brick1/gv0 -l /var/log/glusterfs/bricks/bricks-brick1-gv0.log --xlator-option *-posix.glusterd-uuid=e2230f1e-a4a1-4e20-87e8-6e6c4b115510 --brick-port 49152 --xlator-option gv0-server.listen-port=49152) [2016-11-18 00:27:16.223588] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2016-11-18 00:27:16.244396] I [MSGID: 101173] [graph.c:269:gf_add_cmdline_options] 0-gv0-server: adding option 'listen-port' for volume 'gv0-server' with value '49152' [2016-11-18 00:27:16.244421] I [MSGID: 101173] [graph.c:269:gf_add_cmdline_options] 0-gv0-posix: adding option 'glusterd-uuid' for volume 'gv0-posix' with value 'e2230f1e-a4a1-4e20-87e8-6e6c4b115510' [2016-11-18 00:27:16.244602] I [MSGID: 115034] [server.c:398:_check_for_auth_option] 0-gv0-decompounder: skip format check for non-addr auth option auth.login./bricks/brick1/gv0.allow [2016-11-18 00:27:16.244610] I [MSGID: 115034] [server.c:398:_check_for_auth_option] 0-gv0-decompounder: skip format check for non-addr auth option auth.login.d9b06a73-97bc-45a4-a7a7-18285b048f4a.password [2016-11-18 00:27:16.244636] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2016-11-18 00:27:16.245635] I [rpcsvc.c:2214:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured rpc.outstanding-rpc-limit with value 64 [2016-11-18 00:27:16.245687] W [MSGID: 101002] [options.c:954:xl_opt_validate] 0-gv0-server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction [2016-11-18 00:27:16.247645] I [MSGID: 121050] [ctr-helper.c:259:extract_ctr_options] 0-gfdbdatastore: CTR Xlator is disabled. [2016-11-18 00:27:16.247666] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-pagesize from params.Assigning default value: 4096 [2016-11-18 00:27:16.247675] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-journalmode from params.Assigning default value: wal [2016-11-18 00:27:16.247682] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-sync from params.Assigning default value: off [2016-11-18 00:27:16.247689] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-autovacuum from params.Assigning default value: none [2016-11-18 00:27:16.250353] I [trash.c:2414:init] 0-gv0-trash: no option specified for 'eliminate', using NULL [2016-11-18 00:27:16.251116] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-glusterfs' is not recognized [2016-11-18 00:27:16.251151] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-unix' is not recognized [2016-11-18 00:27:16.251168] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-null' is not recognized [2016-11-18 00:27:16.251196] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'auth-path' is not recognized [2016-11-18 00:27:16.251214] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-quota: option 'timeout' is not recognized [2016-11-18 00:27:16.251257] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-trash: option 'brick-path' is not recognized [2016-11-18 00:27:16.253870] W [MSGID: 113026] [posix.c:1487:posix_mkdir] 0-gv0-posix: mkdir (/.trashcan/): gfid (00000000-0000-0000-0000-000000000005) is already associated with directory (/bricks/brick1/gv0/.glusterfs/00/00/00000000-0000-0000-0000-000000000001/.trashcan). Hence, both directories will share same gfid and this can lead to inconsistencies. [2016-11-18 00:27:16.253913] E [MSGID: 113027] [posix.c:1594:posix_mkdir] 0-gv0-posix: mkdir of /bricks/brick1/gv0/.trashcan/ failed [File exists] [2016-11-18 00:27:16.253983] W [MSGID: 113026] [posix.c:1487:posix_mkdir] 0-gv0-posix: mkdir (/.trashcan/internal_op): gfid (00000000-0000-0000-0000-000000000006) is already associated with directory (/bricks/brick1/gv0/.glusterfs/00/00/00000000-0000-0000-0000-000000000005/internal_op). Hence, both directories will share same gfid and this can lead to inconsistencies. [2016-11-18 00:27:16.254003] E [MSGID: 113027] [posix.c:1594:posix_mkdir] 0-gv0-posix: mkdir of /bricks/brick1/gv0/.trashcan/internal_op failed [File exists] Final graph: +------------------------------------------------------------------------------+ 1: volume gv0-posix 2: type storage/posix 3: option glusterd-uuid e2230f1e-a4a1-4e20-87e8-6e6c4b115510 4: option directory /bricks/brick1/gv0 5: option volume-id 5f15c919-57e3-4648-b20a-395d9fe3d7d6 6: end-volume 7: 8: volume gv0-trash 9: type features/trash 10: option trash-dir .trashcan 11: option brick-path /bricks/brick1/gv0 12: option trash-internal-op off 13: subvolumes gv0-posix 14: end-volume 15: 16: volume gv0-changetimerecorder 17: type features/changetimerecorder 18: option db-type sqlite3 19: option hot-brick off 20: option db-name gv0.db 21: option db-path /bricks/brick1/gv0/.glusterfs/ 22: option record-exit off 23: option ctr_link_consistency off 24: option ctr_lookupheal_link_timeout 300 25: option ctr_lookupheal_inode_timeout 300 26: option record-entry on 27: option ctr-enabled off 28: option record-counters off 29: option ctr-record-metadata-heat off 30: option sql-db-cachesize 1000 31: option sql-db-wal-autocheckpoint 1000 32: subvolumes gv0-trash 33: end-volume 34: 35: volume gv0-changelog 36: type features/changelog 37: option changelog-brick /bricks/brick1/gv0 38: option changelog-dir /bricks/brick1/gv0/.glusterfs/changelogs 39: option changelog-barrier-timeout 120 40: subvolumes gv0-changetimerecorder 41: end-volume 42: 43: volume gv0-bitrot-stub 44: type features/bitrot-stub 45: option export /bricks/brick1/gv0 46: subvolumes gv0-changelog 47: end-volume 48: 49: volume gv0-access-control 50: type features/access-control 51: subvolumes gv0-bitrot-stub 52: end-volume 53: 54: volume gv0-locks 55: type features/locks 56: subvolumes gv0-access-control 57: end-volume 58: 59: volume gv0-worm 60: type features/worm 61: option worm off 62: option worm-file-level off 63: subvolumes gv0-locks 64: end-volume 65: 66: volume gv0-read-only 67: type features/read-only 68: option read-only off 69: subvolumes gv0-worm 70: end-volume 71: 72: volume gv0-leases 73: type features/leases 74: option leases off 75: subvolumes gv0-read-only 76: end-volume 77: 78: volume gv0-upcall 79: type features/upcall 80: option cache-invalidation off 81: subvolumes gv0-leases 82: end-volume 83: 84: volume gv0-io-threads 85: type performance/io-threads 86: subvolumes gv0-upcall 87: end-volume 88: 89: volume gv0-marker 90: type features/marker 91: option volume-uuid 5f15c919-57e3-4648-b20a-395d9fe3d7d6 92: option timestamp-file /var/lib/glusterd/vols/gv0/marker.tstamp 93: option quota-version 0 94: option xtime off 95: option gsync-force-xtime off 96: option quota off 97: option inode-quota off 98: subvolumes gv0-io-threads 99: end-volume 100: 101: volume gv0-barrier 102: type features/barrier 103: option barrier disable 104: option barrier-timeout 120 105: subvolumes gv0-marker 106: end-volume 107: 108: volume gv0-index 109: type features/index 110: option index-base /bricks/brick1/gv0/.glusterfs/indices 111: option xattrop-dirty-watchlist trusted.afr.dirty 112: option xattrop-pending-watchlist trusted.afr.gv0- 113: subvolumes gv0-barrier 114: end-volume 115: 116: volume gv0-quota 117: type features/quota 118: option volume-uuid gv0 119: option server-quota off 120: option timeout 0 121: option deem-statfs off 122: subvolumes gv0-index 123: end-volume 124: 125: volume /bricks/brick1/gv0 126: type debug/io-stats 127: option log-level INFO 128: option latency-measurement off 129: option count-fop-hits off 130: subvolumes gv0-quota 131: end-volume 132: 133: volume gv0-decompounder 134: type performance/decompounder 135: subvolumes /bricks/brick1/gv0 136: end-volume 137: 138: volume gv0-server 139: type protocol/server 140: option transport.socket.listen-port 49152 141: option rpc-auth.auth-glusterfs on 142: option rpc-auth.auth-unix on 143: option rpc-auth.auth-null on 144: option rpc-auth-allow-insecure on 145: option transport-type tcp 146: option transport.address-family inet 147: option auth.login./bricks/brick1/gv0.allow d9b06a73-97bc-45a4-a7a7-18285b048f4a 148: option auth.login.d9b06a73-97bc-45a4-a7a7-18285b048f4a.password 0c6d4cdf-bbf0-4900-aefb-3d75c41d4a23 149: option auth-path /bricks/brick1/gv0 150: option auth.addr./bricks/brick1/gv0.allow * 151: subvolumes gv0-decompounder 152: end-volume 153: +------------------------------------------------------------------------------+ [2016-11-18 00:27:17.578964] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 00:27:17.578985] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-1132-2016/11/17-23:44:23:893895-gv0-client-1-0-2 (version: 3.8.5) [2016-11-18 00:27:18.934912] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 00:27:18.934932] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-1131-2016/11/18-00:27:14:896596-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 00:27:20.606028] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 00:27:20.606046] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-2303-2016/11/18-00:27:17:562817-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 00:27:21.658489] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 00:27:21.658518] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-18053-2016/11/18-00:27:17:930148-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 00:27:21.990286] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 00:27:21.990306] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan3-3233-2016/11/18-00:27:18:239944-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 00:35:01.735889] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan-18053-2016/11/18-00:27:17:930148-gv0-client-1-0-0 [2016-11-18 00:35:01.735942] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan-18053-2016/11/18-00:27:17:930148-gv0-client-1-0-0 [2016-11-18 00:35:11.335897] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan-1132-2016/11/17-23:44:23:893895-gv0-client-1-0-2 [2016-11-18 00:35:11.335951] I [MSGID: 115013] [server-helpers.c:293:do_fd_cleanup] 0-gv0-server: fd cleanup on <gfid:90548459-452d-4129-80f5-b5670b073440> [2016-11-18 00:35:11.338060] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan-1132-2016/11/17-23:44:23:893895-gv0-client-1-0-2 [2016-11-18 00:37:12.615988] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 00:37:12.616008] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-1102-2016/11/18-00:37:11:788116-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 00:37:12.696175] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan-2303-2016/11/18-00:27:17:562817-gv0-client-1-0-0 [2016-11-18 00:37:12.696201] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan-2303-2016/11/18-00:27:17:562817-gv0-client-1-0-0 [2016-11-18 00:37:14.382673] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan3-3233-2016/11/18-00:27:18:239944-gv0-client-1-0-0 [2016-11-18 00:37:14.382733] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan3-3233-2016/11/18-00:27:18:239944-gv0-client-1-0-0 [2016-11-18 00:37:17.726775] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 00:37:17.726792] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-2424-2016/11/18-00:37:13:702446-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 00:37:18.116346] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 00:37:18.116372] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-2299-2016/11/18-00:37:14:826644-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 00:37:19.404291] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 00:37:19.404320] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan3-10529-2016/11/18-00:37:15:384681-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 01:25:10.606405] E [MSGID: 113002] [posix.c:252:posix_lookup] 0-gv0-posix: buf->ia_gfid is null for /bricks/brick1/gv0/.shard/90548459-452d-4129-80f5-b5670b073440.224 [No data available] [2016-11-18 01:35:58.751094] I [MSGID: 100030] [glusterfsd.c:2454:main] 0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version 3.8.5 (args: /usr/sbin/glusterfsd -s 10.0.0.2 --volfile-id gv0.10.0.0.2.bricks-brick1-gv0 -p /var/lib/glusterd/vols/gv0/run/10.0.0.2-bricks-brick1-gv0.pid -S /var/run/gluster/005a9f9af69ac8abebb1340413388f36.socket --brick-name /bricks/brick1/gv0 -l /var/log/glusterfs/bricks/bricks-brick1-gv0.log --xlator-option *-posix.glusterd-uuid=e2230f1e-a4a1-4e20-87e8-6e6c4b115510 --brick-port 49152 --xlator-option gv0-server.listen-port=49152) [2016-11-18 01:35:58.758344] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1 [2016-11-18 01:35:58.793706] I [MSGID: 101173] [graph.c:269:gf_add_cmdline_options] 0-gv0-server: adding option 'listen-port' for volume 'gv0-server' with value '49152' [2016-11-18 01:35:58.793732] I [MSGID: 101173] [graph.c:269:gf_add_cmdline_options] 0-gv0-posix: adding option 'glusterd-uuid' for volume 'gv0-posix' with value 'e2230f1e-a4a1-4e20-87e8-6e6c4b115510' [2016-11-18 01:35:58.793914] I [MSGID: 115034] [server.c:398:_check_for_auth_option] 0-gv0-decompounder: skip format check for non-addr auth option auth.login./bricks/brick1/gv0.allow [2016-11-18 01:35:58.793925] I [MSGID: 115034] [server.c:398:_check_for_auth_option] 0-gv0-decompounder: skip format check for non-addr auth option auth.login.d9b06a73-97bc-45a4-a7a7-18285b048f4a.password [2016-11-18 01:35:58.795048] I [rpcsvc.c:2214:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured rpc.outstanding-rpc-limit with value 64 [2016-11-18 01:35:58.795104] W [MSGID: 101002] [options.c:954:xl_opt_validate] 0-gv0-server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction [2016-11-18 01:35:58.796971] I [MSGID: 121050] [ctr-helper.c:259:extract_ctr_options] 0-gfdbdatastore: CTR Xlator is disabled. [2016-11-18 01:35:58.796991] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-pagesize from params.Assigning default value: 4096 [2016-11-18 01:35:58.796999] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-journalmode from params.Assigning default value: wal [2016-11-18 01:35:58.797007] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-sync from params.Assigning default value: off [2016-11-18 01:35:58.797014] W [MSGID: 101105] [gfdb_sqlite3.h:234:gfdb_set_sql_params] 0-gv0-changetimerecorder: Failed to retrieve sql-db-autovacuum from params.Assigning default value: none [2016-11-18 01:35:58.799579] I [trash.c:2414:init] 0-gv0-trash: no option specified for 'eliminate', using NULL [2016-11-18 01:35:58.800323] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-glusterfs' is not recognized [2016-11-18 01:35:58.800347] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-unix' is not recognized [2016-11-18 01:35:58.800364] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'rpc-auth.auth-null' is not recognized [2016-11-18 01:35:58.800392] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-server: option 'auth-path' is not recognized [2016-11-18 01:35:58.800409] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-quota: option 'timeout' is not recognized [2016-11-18 01:35:58.800450] W [MSGID: 101174] [graph.c:360:_log_if_unknown_option] 0-gv0-trash: option 'brick-path' is not recognized [2016-11-18 01:35:58.802345] W [MSGID: 113026] [posix.c:1487:posix_mkdir] 0-gv0-posix: mkdir (/.trashcan/): gfid (00000000-0000-0000-0000-000000000005) is already associated with directory (/bricks/brick1/gv0/.glusterfs/00/00/00000000-0000-0000-0000-000000000001/.trashcan). Hence, both directories will share same gfid and this can lead to inconsistencies. [2016-11-18 01:35:58.802378] E [MSGID: 113027] [posix.c:1594:posix_mkdir] 0-gv0-posix: mkdir of /bricks/brick1/gv0/.trashcan/ failed [File exists] [2016-11-18 01:35:58.802438] W [MSGID: 113026] [posix.c:1487:posix_mkdir] 0-gv0-posix: mkdir (/.trashcan/internal_op): gfid (00000000-0000-0000-0000-000000000006) is already associated with directory (/bricks/brick1/gv0/.glusterfs/00/00/00000000-0000-0000-0000-000000000005/internal_op). Hence, both directories will share same gfid and this can lead to inconsistencies. [2016-11-18 01:35:58.802448] E [MSGID: 113027] [posix.c:1594:posix_mkdir] 0-gv0-posix: mkdir of /bricks/brick1/gv0/.trashcan/internal_op failed [File exists] Final graph: +------------------------------------------------------------------------------+ 1: volume gv0-posix 2: type storage/posix 3: option glusterd-uuid e2230f1e-a4a1-4e20-87e8-6e6c4b115510 4: option directory /bricks/brick1/gv0 5: option volume-id 5f15c919-57e3-4648-b20a-395d9fe3d7d6 6: end-volume 7: 8: volume gv0-trash 9: type features/trash 10: option trash-dir .trashcan 11: option brick-path /bricks/brick1/gv0 12: option trash-internal-op off 13: subvolumes gv0-posix 14: end-volume 15: 16: volume gv0-changetimerecorder 17: type features/changetimerecorder 18: option db-type sqlite3 19: option hot-brick off 20: option db-name gv0.db 21: option db-path /bricks/brick1/gv0/.glusterfs/ 22: option record-exit off 23: option ctr_link_consistency off 24: option ctr_lookupheal_link_timeout 300 25: option ctr_lookupheal_inode_timeout 300 26: option record-entry on 27: option ctr-enabled off 28: option record-counters off 29: option ctr-record-metadata-heat off 30: option sql-db-cachesize 1000 31: option sql-db-wal-autocheckpoint 1000 32: subvolumes gv0-trash 33: end-volume 34: 35: volume gv0-changelog 36: type features/changelog 37: option changelog-brick /bricks/brick1/gv0 38: option changelog-dir /bricks/brick1/gv0/.glusterfs/changelogs 39: option changelog-barrier-timeout 120 40: subvolumes gv0-changetimerecorder 41: end-volume 42: 43: volume gv0-bitrot-stub 44: type features/bitrot-stub 45: option export /bricks/brick1/gv0 46: subvolumes gv0-changelog 47: end-volume 48: 49: volume gv0-access-control 50: type features/access-control 51: subvolumes gv0-bitrot-stub 52: end-volume 53: 54: volume gv0-locks 55: type features/locks 56: subvolumes gv0-access-control 57: end-volume 58: 59: volume gv0-worm 60: type features/worm 61: option worm off 62: option worm-file-level off 63: subvolumes gv0-locks 64: end-volume 65: 66: volume gv0-read-only 67: type features/read-only 68: option read-only off 69: subvolumes gv0-worm 70: end-volume 71: 72: volume gv0-leases 73: type features/leases 74: option leases off 75: subvolumes gv0-read-only 76: end-volume 77: 78: volume gv0-upcall 79: type features/upcall 80: option cache-invalidation off 81: subvolumes gv0-leases 82: end-volume 83: 84: volume gv0-io-threads 85: type performance/io-threads 86: subvolumes gv0-upcall 87: end-volume 88: 89: volume gv0-marker 90: type features/marker 91: option volume-uuid 5f15c919-57e3-4648-b20a-395d9fe3d7d6 92: option timestamp-file /var/lib/glusterd/vols/gv0/marker.tstamp 93: option quota-version 0 94: option xtime off 95: option gsync-force-xtime off 96: option quota off 97: option inode-quota off 98: subvolumes gv0-io-threads 99: end-volume 100: 101: volume gv0-barrier 102: type features/barrier 103: option barrier disable 104: option barrier-timeout 120 105: subvolumes gv0-marker 106: end-volume 107: 108: volume gv0-index 109: type features/index 110: option index-base /bricks/brick1/gv0/.glusterfs/indices 111: option xattrop-dirty-watchlist trusted.afr.dirty 112: option xattrop-pending-watchlist trusted.afr.gv0- 113: subvolumes gv0-barrier 114: end-volume 115: 116: volume gv0-quota 117: type features/quota 118: option volume-uuid gv0 119: option server-quota off 120: option timeout 0 121: option deem-statfs off 122: subvolumes gv0-index 123: end-volume 124: 125: volume /bricks/brick1/gv0 126: type debug/io-stats 127: option log-level INFO 128: option latency-measurement off 129: option count-fop-hits off 130: subvolumes gv0-quota 131: end-volume 132: 133: volume gv0-decompounder 134: type performance/decompounder 135: subvolumes /bricks/brick1/gv0 136: end-volume 137: 138: volume gv0-server 139: type protocol/server 140: option transport.socket.listen-port 49152 141: option rpc-auth.auth-glusterfs on 142: option rpc-auth.auth-unix on 143: option rpc-auth.auth-null on 144: option rpc-auth-allow-insecure on 145: option transport-type tcp 146: option transport.address-family inet 147: option auth.login./bricks/brick1/gv0.allow d9b06a73-97bc-45a4-a7a7-18285b048f4a 148: option auth.login.d9b06a73-97bc-45a4-a7a7-18285b048f4a.password 0c6d4cdf-bbf0-4900-aefb-3d75c41d4a23 149: option auth-path /bricks/brick1/gv0 150: option auth.addr./bricks/brick1/gv0.allow * 151: subvolumes gv0-decompounder 152: end-volume 153: +------------------------------------------------------------------------------+ [2016-11-18 01:35:58.805779] I [MSGID: 101190] [event-epoll.c:628:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2 [2016-11-18 01:36:00.230596] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 01:36:00.230620] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-1102-2016/11/18-00:37:11:788116-gv0-client-1-0-1 (version: 3.8.5) [2016-11-18 01:36:01.893675] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 01:36:01.893691] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-1112-2016/11/18-01:35:57:854303-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 01:36:04.188610] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 01:36:04.188640] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-3081-2016/11/18-01:36:01:60764-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 01:36:05.781152] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 01:36:05.781168] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-2313-2016/11/18-01:36:01:751807-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 01:36:06.175732] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 01:36:06.175753] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan3-11181-2016/11/18-01:36:02:892703-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 01:37:48.191798] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan-3081-2016/11/18-01:36:01:60764-gv0-client-1-0-0 [2016-11-18 01:37:48.191842] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan-3081-2016/11/18-01:36:01:60764-gv0-client-1-0-0 [2016-11-18 01:39:10.429903] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 01:39:10.429925] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-1138-2016/11/18-01:39:09:921906-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 01:39:10.941267] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan3-11181-2016/11/18-01:36:02:892703-gv0-client-1-0-0 [2016-11-18 01:39:10.941300] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan3-11181-2016/11/18-01:36:02:892703-gv0-client-1-0-0 [2016-11-18 01:39:12.701831] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan-2313-2016/11/18-01:36:01:751807-gv0-client-1-0-0 [2016-11-18 01:39:12.701872] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan-2313-2016/11/18-01:36:01:751807-gv0-client-1-0-0 [2016-11-18 01:39:15.962085] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 01:39:15.962113] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan3-11229-2016/11/18-01:39:11:942270-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 01:39:16.513277] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 01:39:16.513296] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-2303-2016/11/18-01:39:13:489293-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 01:39:17.722179] I [login.c:76:gf_auth] 0-auth/login: allowed user names: d9b06a73-97bc-45a4-a7a7-18285b048f4a [2016-11-18 01:39:17.722194] I [MSGID: 115029] [server-handshake.c:692:server_setvolume] 0-gv0-server: accepted client from xosan-2362-2016/11/18-01:39:13:703248-gv0-client-1-0-0 (version: 3.8.5) [2016-11-18 01:39:22.668788] I [MSGID: 115036] [server.c:548:server_rpc_notify] 0-gv0-server: disconnecting connection from xosan-1102-2016/11/18-00:37:11:788116-gv0-client-1-0-1 [2016-11-18 01:39:22.668837] I [MSGID: 115013] [server-helpers.c:293:do_fd_cleanup] 0-gv0-server: fd cleanup on <gfid:90548459-452d-4129-80f5-b5670b073440> [2016-11-18 01:39:22.668919] I [MSGID: 101055] [client_t.c:415:gf_client_unref] 0-gv0-server: Shutting down connection xosan-1102-2016/11/18-00:37:11:788116-gv0-client-1-0-1
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
