Re: geo-replication vs replicated volumes

Gabriel Kuri <gkuri@xxxxxxxx> · Wed, 10 Jun 2015 14:29:55 -0700

Thanks for the replies, it helps to better understand the architecture.

Is master-master planned for geo-replication?

What is the downfall or problems associated with attempting to run basic replicated volume (synchronous) across servers in separate data centers with latency <75ms and about 1Gbps of bandwidth between them? The write load would not be high at all, I'm talking about occasional writes here and there, less than a megabyte or two per write and maybe 3 or 4 a minute, on average.

Thanks ...

On Wed, Jun 10, 2015 at 1:14 PM, M S Vishwanath Bhat <msvbhat@xxxxxxxxx> wrote:

On 10 June 2015 at 22:38, Aravinda <avishwan@xxxxxxxxxx> wrote:

    On 06/10/2015 09:43 PM, Gabriel Kuri
      wrote:

          > glusterfs doesn't support master-master yet. In your
            case, one of the servers (A or B or C) should be a master
            and your client should write to only that volume. 

            > Other two volumes should be read-only till volume in
            server-A fails for some reason.

          So the writes from the client will go directly to whichever
          server is the master, even though the client has mounted the
          volume on one of the slaves? What about the reads, do they
          still hit the server (ie slave) the client is connected to or
          do the reads hit the master as well?

    To be specific, Gluster Geo-rep doesn't support Master-Master. That
    means no automatic failover when master Volume goes down. Gluster
    replicated volumes support Master-Master within the Volume.

    Replicated Volumes:

    --------------------------

    The replication is synchronous, all writes on the mount will be
    copied to multiple bricks(replica count) synchronously. If one node
    is down during the write, other nodes takes care of syncing data
    when node comes online. This is automatic using self-heal. 

    Replication is between bricks of single volume.

    Geo-replicated Volumes:

    --------------------------------

    Asynchronous replication of whole Gluster Volume. That means their
    will be delay in syncing data from Master Volume to Slave Volume.
    Volume topology does not matter Geo-replication works even if Master
    and Slave Volume types are different.

    Geo-replication is mainly used as disaster recovery mecanism like
    backups. Manual failover failback is also supported, if Master
    Volume goes down Slave Volume can become Master and can establish
    connection back. It is not supported to have Georep running in both
    ways at same time.

        In the case of geo-rep, how is split-brain handled? If the
        network is down between server A (master) and server B (slave)
        and the client has mounted to server B, I assume server B will
        then become the master and writes will then be committed
        directly to server B, but if writes were also committed to
        server A by other clients while the network was down, what
        happens when the network is back up between server A and B, does
        it just figure out which files had the most recent time stamp
        and commit those changes across all the servers?

    Since Master-Master is not supported in Geo-rep, Split brain is not
    handled.  I think there is some confusion between replicated volume
    and geo-rep. Replicated Volume replicates data within Volume. For
    example, Create a Gluster Volume with two bricks with replica count
    as 2. Bricks/Nodes cannot be across data centers. In case of
    Geo-rep, replication is between two Gluster Volumes.

Yes, I think you are confused between replication and geo-replication.

Just to add to what Aravinda mentioned, in AFR (Automatic File Replication, that's what it's called in glusterfs) the replication happens between the bricks of the same volume. The bricks are expected to be part of same network. And the replication is synchronous. You can read more at http://gluster.readthedocs.org/en/latest/Features/afr-v1/index.html?highlight=afr

And glusterfs geo-replication is between two (or more) glusterfs volumes. These volumes in turn can have replicated setup internally. And volumes are generally in different networks. And it's asynchronous one way replication. It's mainly used for disaster recovery. 

I found below link which is very old. But the content seems to be valid still

http://www.gluster.org/community/documentation/index.php/Gluster_3.2:_Replicated_Volumes_vs_Geo-replication

HTH

Greetings,
Vishwanath

              >> If it's not master-master, how does one get
              master-master replication working over a WAN?

          > AFAIK, there is no work around as of now, at least I
            am not aware of it

          Does the basic replicated volume work in this fashion,
            reads and writes to all servers? The only problem is it's
            meant for a low latency network environment?

          Thanks ...

      _______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users

    -- 
regards
Aravinda

_______________________________________________

Gluster-users mailing list

Gluster-users@xxxxxxxxxxx

http://www.gluster.org/mailman/listinfo/gluster-users

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users