Re: Client side AFR race conditions?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



gordan@xxxxxxxxxx wrote:
On Wed, 7 May 2008, Anand Avati wrote:


The only way I see to ensure data integrity is to have some arbiter vet
all writes.  You can try to make that arbiter redundant, but good luck
making it actually distributed.


I've seen the distributed arbiter done in proprietary software, so it
must be possible.  The design is pretty clear to me, but I have no idea
where to start integrating the idea into glusterfs, though gluster's the
closest thing to what I need that I've seen in open source.

Can you give some details/links? We would be interested to learn about it.

I suspect what was referred to was a system where the locks are notified to every host, not an actually load sharing system. DLM (RHCS/GFS) does it by multicasting, presumably with acknowledgements being returned from each connected node. I've not looked at the DLM protocol in great detail, so I don't know what the details are.

Actually, I was thinking of WANdisco's Multi-site CVS/SVN/MySQL mirroring software. It's not generalized to the point of being a disk load sharing system, exactly, but I think the concept and the problems are the same. They use a quorum locking model and basically journal the transaction with whichever server they are wrapping for later replay on the other servers.

There used to be a white-paper on WANdisco's protocol online (I haven't looked recently). I didn't know much about DLM (and, after reading what documentation I could find online just now, I don't feel like I know much more), but it sounds like DLM uses a similar quorum model for locking.

As for the versioning (and perhaps this is relevant to the discussion taking place in another thread), I don't see how this can be done without meta-data journaling, so why not make things even simpler and share a unique version number between all entities changed in a transaction? So, for any server to acquire an implicit write lock, the quorum must agree to increment a global transaction ID (which could also be attached in the FS as a directory and/or file's version number). Then, as long as any given system knew that its journal/replay was up-to-date with the latest transaction ID according to the quorum, then it could trust a file's content without consulting a file-specific revision number.

If a server was not completely up-to-date, then it would at least have to synchronize the meta-data journal and consult it to find if a requested file had any pending writes and decide whether it needed to synchronize the file before serving it.

Regards,

Derek
--
Derek R. Price
Solutions Architect
Ximbiot, LLC <http://ximbiot.com>
Get CVS and Subversion Support from Ximbiot!

v: +1 248.835.1260
f: +1 248.246.1176




[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux