Bradley,
You're not the only one that's happened to ;)
It looks like you have some inconsistencies between the 2 peered nodes.
The following will give you some more insight
> gluster volume heal help
Try to see if its attempting to heal files
> gluster volume heal NewRHS info
also if any heals have failed
> gluster volume heal NewRHS info heal-failed
or you're in a split-brain situation (files exist on both peers, but
differ).
> gluster volume heal NewRHS split-brain
On 08/12/14 09:53, Koby, Bradley wrote:
Hello,
I am new to Gluster, having recently been made admin of this after a
sudden departure of a fellow co-worker. There does not seem to be any
visible degradation of Gluster, and replication seems to be happening as
well. However, there are two log files that are filling up so fast,
that the previous administrator had added some cron jobs to purge the
files every 20min.
The running version is 3.3.0.
Any help or ideas would be greatly appreciated!
Thanks.
millions of these type of entries every 20 min in glustershd.log
[2014-08-12 09:28:23.309811] E
[afr-self-heald.c:287:_remove_stale_index] 0-NewRHS-replicate-0:
fd1abea4-472d-4db2-8ffd-13e4cb4325fe: Failed to remove index on
NewRHS-client-0 - No such file or directory
[2014-08-12 09:28:23.310745] E
[afr-self-heald.c:287:_remove_stale_index] 0-NewRHS-replicate-0:
d8e4419f-1ba5-4a42-9c4d-1058e4bcbaf3: Failed to remove index on
NewRHS-client-0 - No such file or directory
[2014-08-12 09:28:23.311651] E
[afr-self-heald.c:287:_remove_stale_index] 0-NewRHS-replicate-0:
7de3320a-eff7-43f6-ab88-67ce765df6a4: Failed to remove index on
NewRHS-client-0 - No such file or directory
and
about 4 million of these type of entries every 20min in
mnt-gluster-NewRHS-start.log
[2014-08-12 09:34:54.492495] E [posix.c:223:posix_stat] 0-NewRHS-posix:
lstat on
/mnt/gluster/NewRHS/start/.glusterfs/a0/79/a079b4de-2998-48ea-9dca-835858ae2ff1
failed: No such file or directory
[2014-08-12 09:34:54.492528] I [server3_1-fops.c:1085:server_unlink_cbk]
0-NewRHS-server: 715465019: UNLINK
<gfid:7c0d26c5-07b2-4ff4-8b8c-e2dc2e6eba47>/34c985ca-2273-4c00-ae42-3c2df7ecd40a
(a079b4de-2998-48ea-9dca-835858ae2ff1) ==> -1 (No such file or directory)
[2014-08-12 09:34:54.493213] W
[marker-quota.c:2047:mq_inspect_directory_xattr] 0-NewRHS-marker: cannot
add a new contribution node
[2014-08-12 09:34:54.493322] E [posix.c:223:posix_stat] 0-NewRHS-posix:
lstat on
/mnt/gluster/NewRHS/start/.glusterfs/11/74/117433f7-ea58-486f-9305-fdebc98e0f0a
failed: No such file or directory
[2014-08-12 09:34:54.493355] I [server3_1-fops.c:1085:server_unlink_cbk]
0-NewRHS-server: 715465021: UNLINK
<gfid:7c0d26c5-07b2-4ff4-8b8c-e2dc2e6eba47>/79493dd2-a267-4dcb-b813-a9de10833de7
(117433f7-ea58-486f-9305-fdebc98e0f0a) ==> -1 (No such file or directory)
Some basic info.......
gluster> volume info NewRHS
Volume Name: NewRHS
Type: Replicate
Volume ID: 6a63d285-d14c-4ac8-8405-3db771d25e34
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: mualglup01.mcs.muohio.edu:/mnt/gluster/NewRHS/start
Brick2: mualglup02.mcs.muohio.edu:/mnt/gluster/NewRHS/start
Options Reconfigured:
auth.allow:
10.0.72.135,10.0.93.*,192.168.251.*,127.0.0.1,10.0.72.132,10.0.72.133
nfs.disable: off
features.quota: on
nfs.register-with-portmap: 1
diagnostics.client-log-level: ERROR
gluster> peer status
Number of Peers: 1
Hostname: mualglup02
Uuid: 33ab9b77-b49b-43af-8edd-f7c39b19841b
State: Peer in Cluster (Connected)
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users
--
John J. Hoffman, CISSP
Computer Engineer, AFRL/RCMI
AFRL DoD Supercomputing Resource Center (DSRC)
Air Force Research Laboratory
2435 Fifth Street
Area B, Bldg 676, Rm 205
Wright-Patterson AFB, OH 45433-7802
john.hoffman@xxxxxxxxxxxx
jhoffman@xxxxxxxxxxxx
(937) 904-5129
Caution: This message may contain competitive, sensitive or other
non-public information not intended for disclosure outside official
government channels. Do not disseminate this message without the
approval of the Office of the Assistant Secretary of the Air Force for
Acquisition. If you received this message in error, please notify the
sender by reply e-mail and delete all copies of this message.
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users
