Hi everyone.
So, a few days ago, I installed another gluster server to our cluster to
prevent split-brains. I told the server to do a self-heal operation, and
sat back and waited while the performance of the cluster dropped
dramatically and our customers all lost patience with us over the course
of several days.
Now I see that the disk on the new node has filled somewhat, but
apparently the self-heal process has stalled. This is what I see when I
run the "volume heal statistics heal-count" command:
root@nfs3:/home/ernied# date
Thu Apr 14 13:14:00 PDT 2016
root@nfs3:/home/ernied# gluster volume heal gv2 statistics heal-count
Gathering count of entries to be healed on volume gv2 has been
successful
Brick nfs1:/brick1/gv2
Number of entries: 475
Brick nfs2:/brick1/gv2
Number of entries: 190
Brick nfs3:/brick1/gv2
Number of entries: 36
root@nfs3:/home/ernied# date
Thu Apr 14 14:35:00 PDT 2016
root@nfs3:/home/ernied# gluster volume heal gv2 statistics heal-count
Gathering count of entries to be healed on volume gv2 has been
successful
Brick nfs1:/brick1/gv2
Number of entries: 475
Brick nfs2:/brick1/gv2
Number of entries: 190
Brick nfs3:/brick1/gv2
Number of entries: 36
After an hour and 20 minutes, I see zero progress. How do I give this
thing a kick in the pants to get moving?
Also, after reading a bit about Gluster tuning, I suspect I may have
made a mistake in creating the bricks. I hear about how we should have
pairs of bricks for faster access, but we've only got 1 brick replicated
over 3 servers. Or maybe that's 3 bricks all named the same thing, I'm
not sure. Here's what the "volume info" command shows:
root@nfs1:/home/ernied# gluster volume info
Volume Name: gv2
Type: Replicate
Volume ID: 3969e9cc-a2bf-4819-8c02-bf51ec0c905f
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: nfs1:/brick1/gv2
Brick2: nfs2:/brick1/gv2
Brick3: nfs3:/brick1/gv2
Options Reconfigured:
cluster.server-quorum-type: none
cluster.server-quorum-ratio: 51
We currently have about 618 GB of data shared on 3 6 TB RAID arrays. The
data is nearly all e-mail, so a lot of small files and IMAP doing a lot
of random read/write operations. Customers are not pleased with the
speed of our webmail right now. Would creating a larger number of
smaller bricks speed up our backend performance? Is there a way to do
that non-destructively?
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
