Hello.
Version:
[root@glusterfs1 ~]# rpm -qa | grep gluster
glusterfs-client-xlators-3.7.4-2.el7.x86_64
glusterfs-cli-3.7.4-2.el7.x86_64
glusterfs-3.7.4-2.el7.x86_64
glusterfs-geo-replication-3.7.4-2.el7.x86_64
glusterfs-fuse-3.7.4-2.el7.x86_64
glusterfs-server-3.7.4-2.el7.x86_64
glusterfs-libs-3.7.4-2.el7.x86_64
glusterfs-api-3.7.4-2.el7.x86_64
Configuration of volume:
[root@glusterfs2 ~]# gluster volume info ovirt-data
Volume Name: ovirt-data
Type: Replicate
Volume ID: a1faaa53-c53e-4696-a078-0e1f4e2444a2
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.0.102.6:/export/vdb1/ovirt-data
Brick2: 10.0.102.7:/export/vdb1/ovirt-data
Brick3: 10.0.102.8:/export/vdb1/ovirt-data
Options Reconfigured:
server.allow-insecure: on
storage.owner-gid: 36
storage.owner-uid: 36
performance.readdir-ahead: on
geo-replication.indexing: on
geo-replication.ignore-pid-check: on
changelog.changelog: on
cluster.quorum-count: 1
cluster.quorum-type: fixed
network.ping-timeout: 1
features.bitrot: on
features.scrub: Active
features.scrub-freq: hourly
features.scrub-throttle: aggressive
On glusterfs1:
mount -t glusterfs 10.0.102.6:/ovirt-data /mnt
dd if=/dev/urandom of=/mnt/file bs=1M count=100
md5sum /export/vdb1/ovirt-data/file
0b093e9f1524cde2d64ce0a105237190 /export/vdb1/ovirt-data/file
On glusterfs3:
dd if=/dev/urandom of=/export/vdb1/ovirt-data/file bs=1M count=100
md5sum /export/vdb1/ovirt-data/file
af0fee68511176b7fcac6cbbd3137431 /export/vdb1/ovirt-data/file
Logs of scrub and bitd attached.
Question: why changes was not detected and how should be process of
detection of broken files (denied access/read-only mode etc) ?
--
С уважением, Попов О.В. / Best regards, Popov V Oleg
tel: +7 981 783-90-13
skype: livelace
[2015-10-04 12:17:31.609138] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/38adcbd719c0032e9f8696e58da61efd.socket --global-timer-wheel)
[2015-10-04 12:17:31.645017] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 12:17:31.661619] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 12:17:31.661794] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 12:17:31.662283] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-0
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 12:17:31.667006] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 12:17:31.670134] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 12:17:31.670645] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 12:17:31.671839] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 12:17:31.671865] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 12:17:31.674970] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 12:17:31.680085] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 12:17:33.736977] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 12:17:33.737076] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 12:17:33.742953] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/dom_md/inbox [GFID: bed21cec-7710-44b9-8fe1-180b0803c0d7 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.744536] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/dom_md/outbox [GFID: 2ebeb9c5-e8ef-44b5-9c7e-7612ed1ee714 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.745621] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/dom_md/leases [GFID: b0f33088-354d-4c9f-8861-ec89fdc8332d | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.747042] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/dom_md/ids [GFID: ebab48ed-01d8-4b92-b266-b5d7b06b82a7 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.748340] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/dom_md/metadata [GFID: b3b5c4c4-9da9-4791-9e60-483cbd3beb66 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.751789] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/images/d2b1e816-add5-478e-b4d0-88677c2e9caf/0773e895-e8b1-4bf8-9d0a-8b87087fed89 [GFID: 2051b98e-4002-4f78-87f0-767045266403 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.753667] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/images/d2b1e816-add5-478e-b4d0-88677c2e9caf/0773e895-e8b1-4bf8-9d0a-8b87087fed89.meta [GFID: e6e10a4a-bb6f-4678-9e5f-c48787cbfa8f | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.755420] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/images/d2b1e816-add5-478e-b4d0-88677c2e9caf/0773e895-e8b1-4bf8-9d0a-8b87087fed89.lease [GFID: f5cbb4d8-03fc-4441-857f-f32a6c0a7a87 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.757314] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 12:18:33.683902] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:18:33.686599] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:18:33.762008] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:18:33.765380] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:20:24.904011] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:20:24.907254] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:20:24.929476] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:20:24.933100] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:20.993749] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.029236] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.034782] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.042741] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:21.046552] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:21.050885] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.434479] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.475153] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.483468] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.494764] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.494804] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.494820] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.444244] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.477568] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.483324] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.492673] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.492706] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.492721] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:00:06.037879] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:06.041604] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:06.068953] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:00:06.070922] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:00:21.795141] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:21.817480] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:06:06.078433] I [MSGID: 100011] [glusterfsd.c:1291:reincarnate] 0-glusterfsd: Fetching the volume file from server...
[2015-10-04 13:06:06.080453] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:27:50.372738] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/38adcbd719c0032e9f8696e58da61efd.socket --global-timer-wheel)
[2015-10-04 13:27:50.384549] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:27:51.431955] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 13:27:51.435736] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 13:27:51.436070] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-0
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 13:27:51.440091] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 13:27:51.443740] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:27:51.444054] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:27:51.444072] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:27:51.446140] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 13:27:51.446950] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 13:27:51.447867] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:27:53.447477] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:27:53.447596] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 13:27:53.467500] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 13:28:02.992509] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:28:03.032915] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:28:18.101537] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/38adcbd719c0032e9f8696e58da61efd.socket --global-timer-wheel)
[2015-10-04 13:28:18.112739] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:28:19.123804] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 13:28:19.124214] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 13:28:19.124531] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-0
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 13:28:19.139412] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:28:19.150470] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 13:28:19.154044] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:28:19.154263] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:28:19.154277] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:28:19.155679] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 13:28:19.156037] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 13:28:21.156539] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:28:21.156598] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 13:28:21.173092] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 13:40:16.003147] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/38adcbd719c0032e9f8696e58da61efd.socket --global-timer-wheel)
[2015-10-04 13:40:16.015144] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:40:17.016719] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 13:40:17.020013] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 13:40:17.020757] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-0
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 13:40:17.025283] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 13:40:17.039897] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:40:17.040223] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:40:17.040241] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:40:17.053672] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:40:17.053744] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 13:40:17.054289] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 13:40:19.054994] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:40:19.055081] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 13:40:19.075752] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 13:40:27.053789] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:40:27.202280] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:41:32.476878] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/38adcbd719c0032e9f8696e58da61efd.socket --global-timer-wheel)
[2015-10-04 13:41:32.487886] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:41:33.507569] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 13:41:33.507817] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 13:41:33.508283] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
[2015-10-04 13:41:33.508699] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-0
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 13:41:33.513808] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 13:41:33.527555] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:41:33.528662] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:41:33.528692] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:41:33.529873] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 13:41:33.530304] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 13:41:35.531573] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:41:35.531633] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 13:41:35.555140] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 13:56:50.000090] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/38adcbd719c0032e9f8696e58da61efd.socket --global-timer-wheel)
[2015-10-04 13:56:50.011663] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:56:51.009252] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 13:56:51.012574] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 13:56:51.013136] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-0
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 13:56:51.026222] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 13:56:51.029911] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:56:51.030124] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:56:51.030138] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:56:51.031314] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 13:56:51.031634] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:56:51.041995] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 13:56:53.042552] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:56:53.042642] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 13:56:53.059071] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 15:26:41.912335] W [socket.c:588:__socket_rwv] 0-glusterfs: readv on 127.0.0.1:24007 failed (No data available)
[2015-10-04 15:26:48.959107] W [glusterfsd.c:1219:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7df5) [0x7f7103a67df5] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x7f71050d1785] -->/usr/sbin/glusterfs(cleanup_and_exit+0x69) [0x7f71050d1609] ) 0-: received signum (15), shutting down
[2015-10-04 15:28:08.252831] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/38adcbd719c0032e9f8696e58da61efd.socket --global-timer-wheel)
[2015-10-04 15:28:08.265450] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 15:28:08.731768] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 15:28:08.731912] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 15:28:08.733121] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-0
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 15:28:08.745807] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 15:28:08.754763] E [MSGID: 114058] [client-handshake.c:1524:client_query_portmap_cbk] 0-ovirt-data-client-0: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running.
[2015-10-04 15:28:08.754834] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-ovirt-data-client-0: disconnected from ovirt-data-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2015-10-04 15:28:12.270510] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 15:28:12.276446] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 15:28:12.276631] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 15:28:12.276642] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 15:28:12.277581] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 15:28:12.278065] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 15:28:14.278583] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 15:28:14.278783] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 15:28:14.306850] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 12:17:31.617445] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/1bb0aed2b4ab42fad6a0f3488347d596.socket --global-timer-wheel)
[2015-10-04 12:17:31.644963] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 12:17:31.661700] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 1]
[2015-10-04 12:17:31.666054] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: biweekly, Throttle: lazy]
[2015-10-04 12:17:31.666113] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 12:17:31.666943] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle lazy
15: option scrub-freq biweekly
16: subvolumes ovirt-data-client-0
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 12:17:31.675127] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 12:17:31.675615] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 12:17:31.679250] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 12:17:31.680500] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 12:17:31.680521] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 12:17:31.684889] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 12:17:31.687900] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-18 12:17:31
[2015-10-04 12:17:31.687930] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 12:18:33.683982] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:18:33.686856] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:18:33.762517] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: lazy]
[2015-10-04 12:18:33.762556] I [MSGID: 118038] [bit-rot-scrub.c:859:br_fsscan_reschedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data rescheduled to run at 2015-10-04 13:18:33
[2015-10-04 12:18:33.765442] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:20:24.904124] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:20:24.907635] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:20:24.927804] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: normal]
[2015-10-04 12:20:24.934053] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:20.993948] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.029632] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.034641] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.042502] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:21.047106] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:21.051478] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.434381] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.475235] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.485874] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.490217] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.493507] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.495312] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.443887] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.477639] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.483427] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.487704] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.490966] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.492856] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:00:06.038343] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:06.041766] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:06.070360] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [1 => 2]
[2015-10-04 13:00:06.070437] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:00:06.073269] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:00:21.795065] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:21.814461] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:06:06.069000] I [MSGID: 100011] [glusterfsd.c:1291:reincarnate] 0-glusterfsd: Fetching the volume file from server...
[2015-10-04 13:06:06.070087] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:18:29.127463] I [MSGID: 118044] [bit-rot-scrub.c:571:br_fsscanner_log_time] 0-ovirt-data-bit-rot-0: Scrubbing "/export/vdb1/ovirt-data" started at 2015-10-04 13:18:29
[2015-10-04 13:19:05.355021] I [MSGID: 118045] [bit-rot-scrub.c:575:br_fsscanner_log_time] 0-ovirt-data-bit-rot-0: Scrubbing "/export/vdb1/ovirt-data" finished at 2015-10-04 13:19:05
[2015-10-04 13:19:05.355553] I [MSGID: 118038] [bit-rot-scrub.c:815:br_fsscan_activate] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data rescheduled to run at 2015-10-04 14:19:05
[2015-10-04 13:27:51.381058] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/1bb0aed2b4ab42fad6a0f3488347d596.socket --global-timer-wheel)
[2015-10-04 13:27:51.392730] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:27:51.431922] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 13:27:51.431989] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:27:51.435555] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 13:27:51.435954] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-0
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 13:27:51.447917] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:27:51.448021] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 13:27:51.451709] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:27:51.451920] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:27:51.451934] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:27:51.453204] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 13:27:51.453815] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 14:27:51
[2015-10-04 13:27:51.453837] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:28:02.992403] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:28:03.032388] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:28:19.107204] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/1bb0aed2b4ab42fad6a0f3488347d596.socket --global-timer-wheel)
[2015-10-04 13:28:19.121809] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:28:19.123763] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 13:28:19.123824] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:28:19.123860] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 13:28:19.124190] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-0
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 13:28:19.154203] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:28:19.154271] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 13:28:19.157973] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:28:19.158229] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:28:19.158243] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:28:19.159673] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 13:28:19.159943] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 14:28:19
[2015-10-04 13:28:19.159961] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:40:17.008628] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/1bb0aed2b4ab42fad6a0f3488347d596.socket --global-timer-wheel)
[2015-10-04 13:40:17.070492] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:40:17.080533] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 13:40:17.080603] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:40:17.080634] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 13:40:17.081262] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-0
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 13:40:17.085889] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 13:40:17.089010] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:40:17.089387] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:40:17.089423] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:40:17.090568] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 13:40:17.096050] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:40:17.100383] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 14:40:17
[2015-10-04 13:40:17.100447] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:40:27.054011] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:40:27.204318] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:41:33.479995] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/1bb0aed2b4ab42fad6a0f3488347d596.socket --global-timer-wheel)
[2015-10-04 13:41:33.497491] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:41:33.514835] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 13:41:33.514899] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:41:33.515172] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 13:41:33.515592] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-0
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 13:41:33.520208] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 13:41:33.523489] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:41:33.523785] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:41:33.523800] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:41:33.525288] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 13:41:33.525373] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:41:33.531258] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 14:41:33
[2015-10-04 13:41:33.531281] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:56:51.005830] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/1bb0aed2b4ab42fad6a0f3488347d596.socket --global-timer-wheel)
[2015-10-04 13:56:51.034491] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:56:51.037053] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 13:56:51.037117] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:56:51.037149] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 13:56:51.037500] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-0
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 13:56:51.041535] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 13:56:51.045177] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:56:51.045347] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:56:51.045591] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:56:51.045609] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:56:51.050987] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 13:56:51.051743] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 14:56:51
[2015-10-04 13:56:51.051774] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 14:56:02.712660] I [MSGID: 118044] [bit-rot-scrub.c:571:br_fsscanner_log_time] 0-ovirt-data-bit-rot-0: Scrubbing "/export/vdb1/ovirt-data" started at 2015-10-04 14:56:02
[2015-10-04 14:56:40.779426] I [MSGID: 118045] [bit-rot-scrub.c:575:br_fsscanner_log_time] 0-ovirt-data-bit-rot-0: Scrubbing "/export/vdb1/ovirt-data" finished at 2015-10-04 14:56:40
[2015-10-04 14:56:40.779466] I [MSGID: 118038] [bit-rot-scrub.c:815:br_fsscan_activate] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data rescheduled to run at 2015-10-04 15:56:40
[2015-10-04 15:26:41.912487] W [socket.c:588:__socket_rwv] 0-glusterfs: readv on 127.0.0.1:24007 failed (No data available)
[2015-10-04 15:26:48.959631] W [glusterfsd.c:1219:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7df5) [0x7fade12fbdf5] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x7fade2965785] -->/usr/sbin/glusterfs(cleanup_and_exit+0x69) [0x7fade2965609] ) 0-: received signum (15), shutting down
[2015-10-04 15:28:08.262049] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/1bb0aed2b4ab42fad6a0f3488347d596.socket --global-timer-wheel)
[2015-10-04 15:28:08.273629] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 15:28:08.724693] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 15:28:08.724772] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 15:28:08.724807] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 15:28:08.725713] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-0: parent translators are ready, attempting connect on transport
[2015-10-04 15:28:08.726890] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-0
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.6
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-0
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 15:28:08.745412] E [MSGID: 114058] [client-handshake.c:1524:client_query_portmap_cbk] 0-ovirt-data-client-0: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running.
[2015-10-04 15:28:08.745572] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-ovirt-data-client-0: disconnected from ovirt-data-client-0. Client process will keep trying to connect to glusterd until brick's port is available
[2015-10-04 15:28:12.278544] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-0: changing port to 49158 (from 0)
[2015-10-04 15:28:12.281365] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 15:28:12.281534] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-0: Connected to ovirt-data-client-0, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 15:28:12.281546] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-0: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 15:28:12.282553] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-0: Server lk version = 1
[2015-10-04 15:28:12.282759] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 16:28:12
[2015-10-04 15:28:12.282777] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 12:17:31.607625] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/37e1f9695d9a78bb559f3ca9f82f6a31.socket --global-timer-wheel)
[2015-10-04 12:17:31.647120] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 12:17:31.659091] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 12:17:31.659631] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 12:17:31.659942] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-2
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 12:17:31.663664] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 12:17:31.666499] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 12:17:31.666680] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 12:17:31.667521] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 12:17:31.667539] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 12:17:31.668552] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 12:17:31.671672] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 12:17:33.672243] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 12:17:33.672298] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 12:17:33.675855] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/dom_md/inbox [GFID: bed21cec-7710-44b9-8fe1-180b0803c0d7 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.676790] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/dom_md/outbox [GFID: 2ebeb9c5-e8ef-44b5-9c7e-7612ed1ee714 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.677693] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/dom_md/leases [GFID: b0f33088-354d-4c9f-8861-ec89fdc8332d | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.678576] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/dom_md/ids [GFID: ebab48ed-01d8-4b92-b266-b5d7b06b82a7 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.679348] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/dom_md/metadata [GFID: b3b5c4c4-9da9-4791-9e60-483cbd3beb66 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.681337] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/images/d2b1e816-add5-478e-b4d0-88677c2e9caf/0773e895-e8b1-4bf8-9d0a-8b87087fed89 [GFID: 2051b98e-4002-4f78-87f0-767045266403 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.682724] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/images/d2b1e816-add5-478e-b4d0-88677c2e9caf/0773e895-e8b1-4bf8-9d0a-8b87087fed89.meta [GFID: e6e10a4a-bb6f-4678-9e5f-c48787cbfa8f | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.683907] I [MSGID: 118016] [bit-rot.c:1060:bitd_oneshot_crawl] 0-ovirt-data-bit-rot-0: Triggering signing for /48317cb5-e33e-476a-9624-4c44b9705999/images/d2b1e816-add5-478e-b4d0-88677c2e9caf/0773e895-e8b1-4bf8-9d0a-8b87087fed89.lease [GFID: f5cbb4d8-03fc-4441-857f-f32a6c0a7a87 | Brick: /export/vdb1/ovirt-data]
[2015-10-04 12:17:33.685038] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 12:18:33.685242] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:18:33.687748] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:18:33.759344] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:18:33.763888] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:20:24.903553] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:20:24.907137] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:20:24.929461] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:20:24.929497] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:20.998022] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.031969] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.036996] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.047723] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:21.047773] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:21.047804] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.437360] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.476834] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.485124] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.490522] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.490553] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.493461] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.447374] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.478292] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.484010] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.489842] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.489924] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.491410] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:00:06.073724] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:06.076577] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:06.095548] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:00:06.095576] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:00:21.820530] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:21.837827] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:23:02.154655] I [MSGID: 100011] [glusterfsd.c:1291:reincarnate] 0-glusterfsd: Fetching the volume file from server...
[2015-10-04 13:23:02.155573] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:27:50.351012] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/37e1f9695d9a78bb559f3ca9f82f6a31.socket --global-timer-wheel)
[2015-10-04 13:27:50.362627] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:27:51.414761] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 13:27:51.426429] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 13:27:51.426800] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-2
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 13:27:51.429951] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 13:27:51.433395] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:27:51.433609] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:27:51.433622] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:27:51.436163] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 13:27:51.436248] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:27:51.436861] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 13:27:53.440304] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:27:53.440505] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 13:27:53.464115] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 13:28:02.915262] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:28:02.951114] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:28:18.043919] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/37e1f9695d9a78bb559f3ca9f82f6a31.socket --global-timer-wheel)
[2015-10-04 13:28:18.054742] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:28:19.057362] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 13:28:19.057949] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 13:28:19.058293] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-2
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 13:28:19.062030] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 13:28:19.065063] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:28:19.065313] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:28:19.065327] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:28:19.088072] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 13:28:19.088520] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:28:19.088810] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 13:28:21.089566] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:28:21.089658] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 13:28:21.110680] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 13:40:16.014447] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/37e1f9695d9a78bb559f3ca9f82f6a31.socket --global-timer-wheel)
[2015-10-04 13:40:16.025708] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:40:17.031529] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 13:40:17.031672] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 13:40:17.032035] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-2
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 13:40:17.048741] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:40:17.059343] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 13:40:17.063231] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:40:17.063486] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:40:17.063500] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:40:17.064798] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 13:40:17.079343] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 13:40:19.082946] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:40:19.083010] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 13:40:19.106452] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 13:40:27.056793] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:40:27.379098] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:41:32.488683] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/37e1f9695d9a78bb559f3ca9f82f6a31.socket --global-timer-wheel)
[2015-10-04 13:41:32.499250] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:41:33.505417] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 13:41:33.505592] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 13:41:33.505930] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-2
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 13:41:33.523193] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 13:41:33.526789] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:41:33.539017] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:41:33.552983] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:41:33.553012] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:41:33.568060] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 13:41:33.568648] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 13:41:35.569237] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:41:35.569315] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 13:41:35.587416] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 13:56:49.962369] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/37e1f9695d9a78bb559f3ca9f82f6a31.socket --global-timer-wheel)
[2015-10-04 13:56:49.974016] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:56:50.980822] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 13:56:50.981061] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 13:56:50.981545] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-2
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 13:56:50.991263] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 13:56:50.996406] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:56:51.001358] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:56:51.001714] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:56:51.001734] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:56:51.006588] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 13:56:51.007915] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 13:56:53.008712] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:56:53.008806] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 13:56:53.028371] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 15:26:42.556350] W [socket.c:588:__socket_rwv] 0-glusterfs: readv on 127.0.0.1:24007 failed (No data available)
[2015-10-04 15:26:49.140222] W [glusterfsd.c:1219:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7df5) [0x7f9365056df5] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x7f93666c0785] -->/usr/sbin/glusterfs(cleanup_and_exit+0x69) [0x7f93666c0609] ) 0-: received signum (15), shutting down
[2015-10-04 15:28:08.331019] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/bitd -p /var/lib/glusterd/bitd/run/bitd.pid -l /var/log/glusterfs/bitd.log -S /var/run/gluster/37e1f9695d9a78bb559f3ca9f82f6a31.socket --global-timer-wheel)
[2015-10-04 15:28:08.352865] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 15:28:08.736170] I [MSGID: 118037] [bit-rot.c:1738:br_rate_limit_signer] 0-ovirt-data-bit-rot-0: [Rate Limit Info] "FULL THROTTLE"
[2015-10-04 15:28:08.736844] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SIGNER" mode
[2015-10-04 15:28:08.737260] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option brick-count 1
14: option expiry-time 120
15: subvolumes ovirt-data-client-2
16: end-volume
17:
18: volume bitd
19: type debug/io-stats
20: subvolumes ovirt-data-bit-rot-0
21: end-volume
22:
+------------------------------------------------------------------------------+
[2015-10-04 15:28:08.748986] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 15:28:08.758820] E [MSGID: 114058] [client-handshake.c:1524:client_query_portmap_cbk] 0-ovirt-data-client-2: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running.
[2015-10-04 15:28:08.758879] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-ovirt-data-client-2: disconnected from ovirt-data-client-2. Client process will keep trying to connect to glusterd until brick's port is available
[2015-10-04 15:28:12.348476] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 15:28:12.353476] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 15:28:12.353780] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 15:28:12.353802] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 15:28:12.355436] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 15:28:12.355797] I [gf-changelog.c:542:gf_changelog_register_generic] 0-gfchangelog: Registering brick: /export/vdb1/ovirt-data [notify filter: 16]
[2015-10-04 15:28:14.356602] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 15:28:14.356994] I [MSGID: 118018] [bit-rot.c:1093:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Crawling brick [/export/vdb1/ovirt-data], scanning for unsigned objects
[2015-10-04 15:28:14.403377] I [MSGID: 118025] [bit-rot.c:1102:br_oneshot_signer] 0-ovirt-data-bit-rot-0: Completed crawling brick [/export/vdb1/ovirt-data]
[2015-10-04 12:17:31.608047] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/3b1fc63756e72e649dc513d477ae4a68.socket --global-timer-wheel)
[2015-10-04 12:17:31.647666] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 12:17:31.659650] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 1]
[2015-10-04 12:17:31.659689] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: biweekly, Throttle: lazy]
[2015-10-04 12:17:31.659713] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 12:17:31.660023] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle lazy
15: option scrub-freq biweekly
16: subvolumes ovirt-data-client-2
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 12:17:31.663542] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 12:17:31.666542] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 12:17:31.666571] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 12:17:31.668614] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 12:17:31.668631] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 12:17:31.670099] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 12:17:31.671692] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-18 12:17:31
[2015-10-04 12:17:31.671717] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 12:18:33.684742] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:18:33.688080] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:18:33.760636] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: lazy]
[2015-10-04 12:18:33.760691] I [MSGID: 118038] [bit-rot-scrub.c:859:br_fsscan_reschedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data rescheduled to run at 2015-10-04 13:18:33
[2015-10-04 12:18:33.764201] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:20:24.903659] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:20:24.907057] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:20:24.928634] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: normal]
[2015-10-04 12:20:24.928730] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:20.998413] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.031877] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.037410] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:21.041589] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:21.041615] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:21.045729] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.437476] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.476747] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.485072] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:37.490611] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.490646] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:37.493500] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.447263] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.477998] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.483713] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 12:51:43.490134] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.490178] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 12:51:43.491367] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:00:06.073655] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:06.076644] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:06.095282] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [1 => 2]
[2015-10-04 13:00:06.095351] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:00:06.095435] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:00:21.820847] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:00:21.836948] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:18:29.223618] I [MSGID: 118044] [bit-rot-scrub.c:571:br_fsscanner_log_time] 0-ovirt-data-bit-rot-0: Scrubbing "/export/vdb1/ovirt-data" started at 2015-10-04 13:18:29
[2015-10-04 13:19:02.805776] I [MSGID: 118045] [bit-rot-scrub.c:575:br_fsscanner_log_time] 0-ovirt-data-bit-rot-0: Scrubbing "/export/vdb1/ovirt-data" finished at 2015-10-04 13:19:02
[2015-10-04 13:19:02.805820] I [MSGID: 118038] [bit-rot-scrub.c:815:br_fsscan_activate] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data rescheduled to run at 2015-10-04 14:19:02
[2015-10-04 13:23:02.154920] I [MSGID: 100011] [glusterfsd.c:1291:reincarnate] 0-glusterfsd: Fetching the volume file from server...
[2015-10-04 13:23:02.155254] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:27:51.356170] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/3b1fc63756e72e649dc513d477ae4a68.socket --global-timer-wheel)
[2015-10-04 13:27:51.367403] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:27:51.415664] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 13:27:51.415772] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:27:51.415819] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 13:27:51.416598] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-2
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 13:27:51.420670] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 13:27:51.423971] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:27:51.424292] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:27:51.424309] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:27:51.425515] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 13:27:51.425660] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:27:51.440509] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 14:27:51
[2015-10-04 13:27:51.440542] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:28:02.914741] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:28:02.951310] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:28:19.047769] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/3b1fc63756e72e649dc513d477ae4a68.socket --global-timer-wheel)
[2015-10-04 13:28:19.118010] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:28:19.120110] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 13:28:19.120188] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:28:19.120257] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 13:28:19.120609] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-2
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 13:28:19.124705] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 13:28:19.127963] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:28:19.128300] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:28:19.128319] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:28:19.147271] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 13:28:19.147501] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:28:19.148469] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 14:28:19
[2015-10-04 13:28:19.148511] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:40:17.019030] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/3b1fc63756e72e649dc513d477ae4a68.socket --global-timer-wheel)
[2015-10-04 13:40:17.086291] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:40:17.088459] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 13:40:17.091596] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:40:17.091647] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 13:40:17.092014] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-2
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 13:40:17.096087] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 13:40:17.106661] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:40:17.106740] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:40:17.107401] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:40:17.107435] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:40:17.109862] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 13:40:17.114366] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 14:40:17
[2015-10-04 13:40:17.114413] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:40:27.056929] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed
[2015-10-04 13:40:27.378656] I [glusterfsd-mgmt.c:1512:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2015-10-04 13:41:33.492625] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/3b1fc63756e72e649dc513d477ae4a68.socket --global-timer-wheel)
[2015-10-04 13:41:33.549217] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:41:33.563166] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 13:41:33.563289] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:41:33.563380] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 13:41:33.564045] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-2
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 13:41:33.581586] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 13:41:33.584998] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:41:33.585319] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:41:33.585372] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:41:33.588727] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 13:41:33.590387] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 13:41:33.601736] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 14:41:33
[2015-10-04 13:41:33.601774] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:56:50.966468] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/3b1fc63756e72e649dc513d477ae4a68.socket --global-timer-wheel)
[2015-10-04 13:56:51.015425] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 13:56:51.017968] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 13:56:51.018088] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 13:56:51.018165] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 13:56:51.018817] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-2
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 13:56:51.023973] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 13:56:51.027521] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 13:56:51.027845] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 13:56:51.027864] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 13:56:51.029289] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 13:56:51.030451] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 14:56:51
[2015-10-04 13:56:51.030476] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
[2015-10-04 13:56:51.030592] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 14:56:04.033137] I [MSGID: 118044] [bit-rot-scrub.c:571:br_fsscanner_log_time] 0-ovirt-data-bit-rot-0: Scrubbing "/export/vdb1/ovirt-data" started at 2015-10-04 14:56:04
[2015-10-04 14:56:40.690437] I [MSGID: 118045] [bit-rot-scrub.c:575:br_fsscanner_log_time] 0-ovirt-data-bit-rot-0: Scrubbing "/export/vdb1/ovirt-data" finished at 2015-10-04 14:56:40
[2015-10-04 14:56:40.690485] I [MSGID: 118038] [bit-rot-scrub.c:815:br_fsscan_activate] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data rescheduled to run at 2015-10-04 15:56:40
[2015-10-04 15:26:42.556155] W [socket.c:588:__socket_rwv] 0-glusterfs: readv on 127.0.0.1:24007 failed (No data available)
[2015-10-04 15:26:49.140733] W [glusterfsd.c:1219:cleanup_and_exit] (-->/lib64/libpthread.so.0(+0x7df5) [0x7f556925adf5] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x7f556a8c4785] -->/usr/sbin/glusterfs(cleanup_and_exit+0x69) [0x7f556a8c4609] ) 0-: received signum (15), shutting down
[2015-10-04 15:28:08.341255] I [MSGID: 100030] [glusterfsd.c:2301:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.7.4 (args: /usr/sbin/glusterfs -s localhost --volfile-id gluster/scrub -p /var/lib/glusterd/scrub/run/scrub.pid -l /var/log/glusterfs/scrub.log -S /var/run/gluster/3b1fc63756e72e649dc513d477ae4a68.socket --global-timer-wheel)
[2015-10-04 15:28:08.354523] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 1
[2015-10-04 15:28:08.742672] I [MSGID: 118035] [bit-rot-scrub.c:1071:br_scrubber_scale_up] 0-ovirt-data-bit-rot-0: Scaling up scrubbers [0 => 2]
[2015-10-04 15:28:08.743330] I [MSGID: 118048] [bit-rot-scrub.c:1318:br_scrubber_log_option] 0-ovirt-data-bit-rot-0: SCRUB TUNABLES:: [Frequency: hourly, Throttle: aggressive]
[2015-10-04 15:28:08.743371] I [MSGID: 118031] [bit-rot.c:1919:init] 0-ovirt-data-bit-rot-0: bit-rot xlator loaded in "SCRUBBER" mode
[2015-10-04 15:28:08.744249] I [MSGID: 114020] [client.c:2118:notify] 0-ovirt-data-client-2: parent translators are ready, attempting connect on transport
Final graph:
+------------------------------------------------------------------------------+
1: volume ovirt-data-client-2
2: type protocol/client
3: option ping-timeout 42
4: option remote-host 10.0.102.8
5: option remote-subvolume /export/vdb1/ovirt-data
6: option transport-type socket
7: option username 4fdd6008-ec0f-4c99-a832-39c0e62ae727
8: option password a654927b-4f89-4ce6-94f2-d5971fef0d02
9: end-volume
10:
11: volume ovirt-data-bit-rot-0
12: type features/bit-rot
13: option scrubber true
14: option scrub-throttle aggressive
15: option scrub-freq hourly
16: subvolumes ovirt-data-client-2
17: end-volume
18:
19: volume scrub
20: type debug/io-stats
21: subvolumes ovirt-data-bit-rot-0
22: end-volume
23:
+------------------------------------------------------------------------------+
[2015-10-04 15:28:08.777505] E [MSGID: 114058] [client-handshake.c:1524:client_query_portmap_cbk] 0-ovirt-data-client-2: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running.
[2015-10-04 15:28:08.777597] I [MSGID: 114018] [client.c:2042:client_rpc_notify] 0-ovirt-data-client-2: disconnected from ovirt-data-client-2. Client process will keep trying to connect to glusterd until brick's port is available
[2015-10-04 15:28:08.790835] I [MSGID: 101190] [event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread with index 2
[2015-10-04 15:28:12.361208] I [rpc-clnt.c:1851:rpc_clnt_reconfig] 0-ovirt-data-client-2: changing port to 49158 (from 0)
[2015-10-04 15:28:12.366062] I [MSGID: 114057] [client-handshake.c:1437:select_server_supported_programs] 0-ovirt-data-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2015-10-04 15:28:12.366336] I [MSGID: 114046] [client-handshake.c:1213:client_setvolume_cbk] 0-ovirt-data-client-2: Connected to ovirt-data-client-2, attached to remote volume '/export/vdb1/ovirt-data'.
[2015-10-04 15:28:12.366354] I [MSGID: 114047] [client-handshake.c:1224:client_setvolume_cbk] 0-ovirt-data-client-2: Server and Client lk-version numbers are not same, reopening the fds
[2015-10-04 15:28:12.367889] I [MSGID: 114035] [client-handshake.c:193:client_set_lk_version_cbk] 0-ovirt-data-client-2: Server lk version = 1
[2015-10-04 15:28:12.368239] I [MSGID: 118038] [bit-rot-scrub.c:778:br_fsscan_schedule] 0-ovirt-data-bit-rot-0: Scrubbing for /export/vdb1/ovirt-data scheduled to run at 2015-10-04 16:28:12
[2015-10-04 15:28:12.368265] I [MSGID: 118039] [bit-rot.c:1291:br_child_enaction] 0-ovirt-data-bit-rot-0: Connected to brick /export/vdb1/ovirt-data..
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
