Fwd: repmgr won't update witness after failover

[Aviel Buskila <aviel33@xxxxxxxxx>]

---------- Forwarded message ----------
From: Aviel Buskila <aviel33@xxxxxxxxx>
Date: 2015-08-13 15:43 GMT+03:00
Subject: Re: repmgr won't update witness after failover
To: Jony Cohen ‫‎<jony.cohenjo@xxxxxxxxx>‎‬

Hey,

I have just tried to start the repmgrd on the new standby after I have fixed it as a standby and still this goes the same way.

from the message given in the repmgrd log in the witness server it seems that he is not able to elect a new master because he can't see anyone .

I have check in the repl_nodes table in the witness and it shows:

witness    node3

master     node2

master     node1

is there a way update the witness after the first failover?

2015-08-13 15:06 GMT+03:00 Jony Cohen <jony.cohenjo@xxxxxxxxx>:

Hi Aviel,

you can use the 'show cluster' command to see the repmgr state before you do the 2nd failover - make sure the node1 is indeed marked as replica.

After a failover the Master doesn't automatically attach to the new master - you need to point him as a slave (standby follow - if possible...)

did you start the repmgrd on node1 after making it a replica of the new master? (it needs 2 daemons to decide what to promote)

Regards,

 - Jony

On Thu, Aug 13, 2015 at 1:29 PM, Aviel Buskila <aviel33@xxxxxxxxx> wrote:

Hey,

I have set up three nodes of postgresql 9.4 with repmgr in this way:

1. master - node1

2. standby - node2

3. witness - node3

Now I have set up the replication and the witness as it says here:
https://github.com/2ndQuadrant/repmgr/blob/master/FAILOVER.rst

Now when I do 'kill -9 $(pidof postmaster)' The witness detects that something went wrong and fails over from node1 to node2

But when I setup the replication now to work from node2 to node1 and I kill the postgresql process it doesn't failover and the repmgrd log shows the following  message:

unable to determine a valid master server; waiting 10 seconds to retry...

it seems that the witness doesn't know about the new standby server..

Has anyone got any idea about what am I doing wrong here?

Best regards,

Aviel Buskila