Dear all,
first of all thanks to Stefan (Kania) for the feedback. So
detaching the peer, removing brick and downgrading to replica 2
until you add a new brick as you would do with any otherwise
faulty node.
So here is how we proceeded:
- check mounts where the faulty (ex-filled-up) node is present and remove it from there
- detach "bad" peer with: gluster peer detach <your_faulty_gluster_node>
- remove brick: gluster volume remove-brick <vol> replica 2 <your_faulty_gluster_node>:<the_brick_path> force
So in our case we are ready to add a new brick to restore our
replica 3.
This should be safer (and in the end as simple) than just copying
the missing files probably bringing along meta-data
inconsistencies. Which was also the doubt expressed on this post.
Am 08.01.24 um 11:13 schrieb Ilias
Chasapakis forumZFD:
Dear all,
we have a replica 3 configuration an issue after that the file system filled up. The gluster daemon is not starting anymore on the affected node and the inconsistency that we noticed is that the /var/lib/glusterd/peers does contain only one "good" node and the other one is missing.
Now what we would like to try is to probe from the affected peer the missing one. We also thought about just copying the missing node config file /var/lib/glusterd/peers here but we think this might be more inconsistent because perhaps of the time passed between finding out the issue and when the fill up had occurred (if this has relevance). We assume probing will fill that missing entry in the affected gluster and will sync to it. What we want to avoid is that this happens the other way around.
So in brief the question is if in our situation probing is too risky and we should try other methods (removing-adding brick) or anything that you might want to suggest from your experience.
Many thanks in advance
Ilias
________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
-- forumZFD Entschieden für Frieden | Committed to Peace Ilias Chasapakis Referent IT | IT Consultant Forum Ziviler Friedensdienst e.V. | Forum Civil Peace Service Am Kölner Brett 8 | 50825 Köln | Germany Tel 0221 91273243 | Fax 0221 91273299 | http://www.forumZFD.de Vorstand nach § 26 BGB, einzelvertretungsberechtigt|Executive Board: Alexander Mauz, Sonja Wiekenberg-Mlalandle, Jens von Bargen VR 17651 Amtsgericht Köln Spenden|Donations: IBAN DE90 4306 0967 4103 7264 00 BIC GENODEM1GLS
Attachment:
OpenPGP_signature.asc
Description: OpenPGP digital signature
________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
