Re: Client vs Server Quorum

[Ravishankar N <ravishankar@xxxxxxxxxx>]

On 10/11/2015 03:53 AM, Lindsay Mathieson wrote:

I'm unclear as to the distinction between the two, but my guess:

- Quorum is only relevant to replica volumes

Client quorum is applicable only to replica volumes. Server quorum is applicable to all volumes.

- It is p[referable to have a odd number of replicas and a minimum of 3

Right.

Client Quorum;

- Client is the fuse client or gfapi driver

True. Also in the gluster nfs server when you use nfs mounts. Client quorum is implemented in the AFR translator. So it comes into play wherever this xlator is loaded.

- Quorum check is performed by the client

More specifically the AFR xlator.

- if the check fails, the client will fail writes

Right. AFR will fail the writes with EROFS.

   * is there a timeout involved or does it fail immediately?

If an ongoing write did not succeed on the necessary number of bricks due to brick disconnect, that FOP will fail with EROFS. Also, subsequent writes will also fail with the same error until quorum is restored (i.e. client connects to the bricks again). So yes, it is pretty much immediate.

- Quorum is determined by how many active bricks the client can see

  * As determined by quorum-type and/or quorum-count

- Bricks remain up

True. Unless the brick process actually went down.

- Datastore remains up

Not sure if you are referring to other bricks, in which case yes.

Server Quorum:

- Server is the brick process (glusterd?)

Not glusterd but the brick process i.e glusterfsd.
The implementation of server quorum is in glusterd though.

- enabled with server-quorum-type=server

- Quorum check is performed by the server

Right.

- Quorum is determined by how many active bricks the server can see

It's more like how many peers the glusterd on each node can see.  Have a look at "How to Test" section in http://www.gluster.org/community/documentation/index.php/Features/Server-quorum

- If quorum fails
  * brick is brought down

Correct. By glusterd.

  * datastore remains up

In both cases bricks which remain part of a quorum can still be written to, whereas bricks which are isolated are readonly, or down altogether and will be healed once quorums returns. In theory this will prevent split brain problems.

Yes.

Question:

- Which is better, server or client quorums?

You can still end up in  split-brain of the files stored in the volume  if sever quorum is enabled. Sever quorum is more useful to avoid conflicts in volume configuration since it also disallows volume set commands, peer probe etc when not in quorum. Client quorum is better if you want to avoid split-brains of files present in the volume.

- Can you safely enable both? recommended?

IMHO, client-quorum is enough. In case of dist-rep volumes, it acts on only those replica sets where quorum is not met making only that replica pair EROFS. Server quorum outright kills the bricks, not even allowing read access. But yes, you can enable both. 

thanks,

--

Lindsay

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users