Can I bring a development idea to Dev's attention?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



 Hi Glusterfs Dev's,

I might be trying to teach you to suck eggs, but I was reading through the publications on http://www.xtreemfs.org/publications.php and one of the ideas there seemed very relevant to gluster

Please have a look at their FatLease papers and the Paxos lease negotiation algorithm. It was new to me, but seems like an elegant and robust (google use it) way of managing a distributed lock scenario?

In particular it would seem to be a very interesting way to introduce the idea of optimistic locking to the gluster type infrastructure. What I'm thinking here is the situation where you have a client talking to a single brick in a replicated set, that it can effectively optimistically lock a localised set of files/directories on that brick and so further reads/modifications can be lazily written out to other servers (without compromising coherency). If a read/write goes to one of the other replicas then of course it must first break the other servers optimistic lock before continuing and effectively you temporarily revert back to the current situation where every file access causes a network access to all other servers.

The win is clearly where you end up with clients localising their access for certain files to certain servers and that server will then benefit from holding an optimistic lock, since no network activity is generated for further read access and lazy writes become possible without split brain risk.

eg consider a master/master setup with serveral webservers. Especially where each physical machine touches only a subset of the files (eg each machine usually only serves a subset of data), then that machine can start to access the gluster share at basically native disk speeds, having acquired an optimistic lock on that subset of data (no need to check with other bricks since we now have a lock on the data). This would seem to be a huge win for a large class of problems?

(Thinking about it another way, it's a bit like a standard optimistic locking architecture, where one server gets promoted to become the master reader/writer at a time (for a subset of files) and no other server can read/write those files without first breaking the lock by requesting to become the master. The difference is that Paxos allows this to be done robustly and efficiently without a centralised lock server)

The clever part seems to be the fairly stateless method that is used to bootstrap a system with stateful leases (read the publications). I hadn't come across the Fatlease/Paxos idea before and it seems extremely clever

Cheers

Ed W



[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux