Re: replication client or server side

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Vincent Régnard wrote:
Hi all,

I am presently re-thinking the way we are using glusterfs 1.3.5 here. We
are doing replication (*3 with 3 bricks) on client side to produce a
small HA cluster. We are planning to extend the brick number. Drawing
that again and looking at some examples on the wiki doing it another way
(Kritical's tutorial), we are wondering wether doing the replication
(AFR) on the server side (glusterfsd) would be more suitable than doing
it on the client side ? Have you any experience or remark on that ? Does
this have performance impact in your opinion ?

If replication is transfered to server side, we'll have to use
unification on client side to achive HA (and then obtain active
self-heal?). Is this latter configuration reasonable ?


Present configuration:

Client stack:    FUSE
        PERFORMANCE TRANSLATORS (write-b/io-cache/io-thread)
        AFR
        CLIENT TRANSPORT

Server stack:    SERVER TRANSPORT
        PERFORMANCE TRANSLATOR (io-thread)
        POSIX LOCKS FEATURE
        POSIX STORAGE


Planned configuration:

Client stack:    FUSE
        PERFORMANCE TRANSLATORS (write-b/io-cache/io-thread)
        UNIFY
        CLIENT TRANSPORT

Server stack:    SERVER TRANSPORT
        PERFORMANCE TRANSLATOR (io-thread)
        AFR
        POSIX LOCKS FEATURE
        POSIX STORAGE

Vincent

I find using AFR and Unify from the client yields a more robust config with respect to high availability, but using unify on the client complicates the configs and file storage (it necessitates splitting the share between a main and AFR split per server). It may be possible to overload the AFR definitions to get around this, I haven't tried that yet. It's also possible that tweaking the timeout values for the client and server to make the server timeout before the client might yield a more stable config.

Performance wise, moving AFR to the server side will allow you structure the network for more performance, such as implementing a secondary network to handle all the AFR traffic. As it is now (with you doing everything on the client), your writes are constrained to 1/3 of the total available network bandwidth, since you have to write each file 3 times. By moving the AFR to the server and implementing a second network to carry the AFR traffic, you could increase your theoretical network performance by 50% (if the AFR network is the same speed as the client network connection, and you want data stored on 3 servers).

It seems like every other day I think of a new way to set up glusterfs. I have to say this is the most fun I've had with a software product in some time. ;)

--

-Kevan Benson
-A-1 Networks




[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux