Re: Crawling and indexing hardware

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



gordan@xxxxxxxxxx wrote:
On Fri, 9 May 2008, Daniel Maher wrote:

Do you plan to do any AFR (automatic file replication) ?  If so,
consider that even a one-byte change to your "big index files" will
cause the /entire/ file to be AFR'd between all participating
nodes.

Marcus, what do you mean by this?

You've mis-attributed the quote, Krishna.  What i meant was exactly
what i said : if you have a 5GB file on an AFR cluster, and a client
makes a one-byte change to that file, then the entire 5GB file is
re-copied to all members of the AFR cluster.

At least, that's my understanding of the situation - please correct me
if i'm wrong. :)

In a word - no. If the nodes are connected, then only the actual 1 byte change will be replicated. If the nodes are disconnected, then the "self heal" will copy the entire 5GB file when the nodes connect.

This can be mitigated with striping, if the striping AFR bug was ever fixed (?).

It does increase the complexity of the configuration and the complexity of manual recovery (if ever needed) though.

--

-Kevan Benson
-A-1 Networks




[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux