If you look at the red hat docs for their RH storage, you will see quite a few more options in regards to this quorum. Those 3.2 docs were missing quite a few options. I'm not at my PC right now, but if I recall correctly there was a brick count option and a percentage based option (along with a many others)..
In your case, your situation does sound correct because the quorum is met. If one of the bricks drops your other one which is connected to the other arbator peer which is in the higher peer count so can continue to write. If the other were to drop due to split network, then it will be denied writes and resync when it reconnects.
I remember reading a presentation where they mentioned, gluster is not aware of any network boundaries. This concerns me, as if you were to have redundant paths to nodes, could that not lead to issues during a partial network split?
I'm yet to properly test this theory, but I plan to do so in the coming weeks..
On Mar 16, 2014 12:37 PM, "Steve Dainard" <sdainard@xxxxxxxxxxxxx> wrote:
How sure are we that this is having the intended effect?I have two gluster nodes with bricks which are also ovirt hosts, and the replicated volume configured as such:gluster volume info rep1Volume Name: rep1Type: ReplicateVolume ID: 5876746d-5977-4fa9-a829-69f44b3364ecStatus: StartedNumber of Bricks: 1 x 2 = 2Transport-type: tcpBricks:Brick1: 10.0.10.2:/mnt/storage/lv-storage-domain/rep1Brick2: 10.0.10.3:/mnt/storage/lv-storage-domain/rep1Options Reconfigured:diagnostics.client-log-level: ERRORstorage.owner-gid: 36storage.owner-uid: 36server.allow-insecure: oncluster.quorum-type: fixednfs.trusted-sync: offnfs.trusted-write: onperformance.quick-read: offperformance.read-ahead: offperformance.io-cache: offperformance.stat-prefetch: offcluster.eager-lock: enablenetwork.remote-dio: enablecluster.server-quorum-type: serverI also set the bricks to 1 (cluster.quorum-count 1, although its not showing above) as this option seems related to storage bricks, not gluster peers.I peered with a 3rd gluster node which doesn't have any volumes configured. I've tested this setup, and the storage doesn't go down when one of the nodes is rebooted.There still seems to be a high potential for split brain here.
cluster.quorum-type Method used for quorum enforcement. "None" means no quorum enforcement, which is the historical behavior. "Auto" means quorum is set to be more than half of the bricks in a subvolume, or exactly half if that includes the first listed brick. "Fixed" means quorum is set to a value specified by cluster.quorum-count. If quorum is not met, then modifing operations such as write will fail with EROFS. This prevents most cases of "split brain" which result from conflicting writes to different bricks. None cluster.quorum-count Number of subvolumes that must be present to achieve quorum, as described for cluster.quorum-type. This value is not used unless cluster.quorum-type is "fixed". 0 We don't seem able to configure "allow 50% of the bricks to be active, as long as 51% of the gluster peers are connected".I also don't understand why 'subvolumes' are referred to here. Is this old terminology for 'volumes'?Thoughts?Steve Dainard
IT Infrastructure Manager
Miovision | Rethink Traffic
Blog | LinkedIn | Twitter | Facebook
Miovision Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener, ON, Canada | N2C 1L3
This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.On Mon, Mar 10, 2014 at 7:58 AM, James <purpleidea@xxxxxxxxx> wrote:A quick search reveals:
# Sets the quorum percentage for the trusted storage pool.
cluster.server-quorum-ratio
# If set to server, enables the specified volume to
participate in quorum.
cluster.server-quorum-type
# If quorum-type is "fixed" only allow writes if this many
bricks or present. Other quorum types will OVERWRITE this value.
cluster.quorum-count
# If value is "fixed" only allow writes if quorum-count bricks
are present. If value is "auto" only allow writes if more than half of
bricks, or exactly half including the first, are present.
cluster.quorum-type
I took these from my previous "notes" (code) in:
https://github.com/purpleidea/puppet-gluster/blob/master/manifests/volume/property/data.pp#L18
You can get newer values or appropriate values for your version by
running something like:
gluster volume set help ( i think )
Cheers,
James
On Mon, Mar 10, 2014 at 2:43 AM, Andrew Lau <andrew@xxxxxxxxxxxxxx> wrote:
> Thanks again James!
>
> So far so good, I plan to test this a little more in a few days but so far
> it seems the only volume setting I need is:
> cluster.server-quorum-type: server
>
> Default cluster.server-quorum-ratio >50%
> So 2 is greater than 1.5.. which should allow writes.
>
> On Thu, Mar 6, 2014 at 5:00 PM, James <purpleidea@xxxxxxxxx> wrote:
>>
>> Top posting sorry:
>>
>> Yes, you can add a third "arbiter" node, that exists to help with the
>> quorum issues. AFAICT, all you do is peer is with the cluster (as you
>> did with the other hosts) but don't add any storage for example.
>>
>> Then you set the cluster.quorum* style volume settings that you're
>> interested. I don't have a list of exactly which ones off the top of
>> my head, but if you make a list, let me know!
>>
>> Cheers,
>> James
>>
>>
>> On Wed, Mar 5, 2014 at 10:51 PM, Andrew Lau <andrew@xxxxxxxxxxxxxx> wrote:
>> > Hi,
>> >
>> > I'm looking for an option to add an arbiter node to the gluster
>> > cluster, but the leads I've been following seem to lead to
>> > inconclusive results.
>> >
>> > The scenario is, a 2 node replicated cluster. What I want to do is
>> > introduce a fake host/arbiter node which would set the cluster to a 3
>> > node meaning, we can meet the conditions of allow over 50% to write
>> > (ie. 2 can write, 1 can not).
>> >
>> > elyograg from IRC gave me a few links [1], [2]
>> > But these appear to be over a year old, and still under review.
>> >
>> > Gluster 3.2 volume options (I'm running 3.4, but there doesn't seem to
>> > be an updated page) [3]
>> > seem to state the that the cluster quorum is identified by active
>> > peers. This also backs up the statement in [2] in regards to a patch
>> > for active volumes rather than cluster peers.
>> >
>> > Has anyone gone down this path, or could they confirm any of these
>> > leads? (ie. does a host w/o any volumes get considered as a peer
>> > within the cluster)
>> >
>> > Thanks,
>> > Andrew
>> >
>> > [1] https://bugzilla.redhat.com/show_bug.cgi?id=914804
>> > [2] http://review.gluster.org/#/c/4363/
>> > [3]
>> > http://gluster.org/community/documentation/index.php/Gluster_3.2:_Setting_Volume_Options#cluster.quorum-type
>> > _______________________________________________
>> > Gluster-users mailing list
>> > Gluster-users@xxxxxxxxxxx
>> > http://supercolony.gluster.org/mailman/listinfo/gluster-users
>
>
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://supercolony.gluster.org/mailman/listinfo/gluster-users
