Re: Re; Load balancing ...

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, 1 May 2008, Martin Fick wrote:

Sounds like a good idea. The next question is where
to keep the log. 1 log per file? 1 log per
directory?
How to store them? Shadow files? Separate
shadow volume? A shadow volume might be a good idea
because it keeps the  main source mounted directory
exactly the same as a normal directory.

I would start as simple as possible and adapt as
necessary if you run into a performance problem.  The
simplest design would probably be a shadow volume with
one log per file with the a sparse mirrored directory
structure.

Indeed, that's exactly what I was thinking. You would effectively need a container, like a namespace, to unify the two.

Logs could be 24(?) bytes concatenated one
after another making appending easy and reliable.  Or
at a minor space cost (but potential added
portability/extendability), each log file could even
be a colon delimited line based ascii file (please
don't anyone suggest an xml file!)

 version1:start2:span2
 version2:start2:span2
 ...

If it's fixed length pointers (or in fact fixed length records), I'd go with packed binary format for efficiency and speed. These will have to be written to on every write. There would also need to be a header that states where the roll-over point is. Effectively, the log would be an RRD.

Having a separate log file for each real file also
makes it easy to code up some optimizations, for
example: it would be easy to lookup the size of the
log and the size of the real file.  As soon as the log
becomes bigger than the real file it is no longer
worth keeping as is!  It also makes it real easy to
just delete the log if the real file is deleted.

Maybe have the default log be about 0.5% of the file in powers of 2, and not used for files below a certain size. Maybe grow/shrink it when it would exceed one step in powers of 2 from it's intended 0.5% size. This would mean that as the file grows, the log increases, but the log extention gets exponentially more rare. log truncation could be left until a suitable roll-over point. If you are syncing inodes, then that is typically 4KB, and a log entry would be, as you said, 24 bytes. That makes a log entry for a changed inode block about 1/170, which is about right for the 0.5% ball park.

Another nice optimizer could make intelligent
decisions about which log files to delete when the
shadow volume starts to fill up.  By simply examining
the size of each log versus the size of the real file
one can set an upper bounds on how much transfer data
the log could be saving (a real estimate would require
adding all the spans together in the log file taking
into account overlapping sections).

Sure - if you want to keep volumes separate. Or you could just maue sure that your log volume is always at least 1/170 of the data volume it's shadowing. Possibly a bit more for a safety margin with the lazy log resizing - around 1% ought to suffice for most sane cases.

Finally, it would
allow an admin to prune the shadow volume manually of
whichever logs he chooses to prune.  An ascii file
would make it easy to script various pruners.

I think that starts getting potentially dangerous. I think just having the logs volume at about 1% of the data volume would be better. Of course, if you keep both on the same physical volume, it won't matter.

It would be nice to design the shadow volume so that
it can be removed from the picture at any time without
corrupting anything.

You already covered that with the sparse shadow volume tree. If there's no log, you resync the whole file.

It would also be nice to ensure
that the journal translator can handle an out of space
condition.  This way each server is not required to
even have the same size journal volume if any at all.

Note that this gets into the chicken and the egg problem - the log files would still need to be syncable directly using the current method - or you'd need a journal for your journalling volume. But if the journal is typically < 1% of the file, that's probably cheap enough that it won't matter too much. You could also probably set the upper limit on the volume size, because past a certain point the file changes will be limited by disk speeds, so from there on a bigger file doesn't imply more log space is required.

A (shadow volume) log should, ideally, also keep
additional sanity check information such as file
metadata (timestamps, size) for cross-check of
whether something went weird and the file was
changed underneath GlusterFS, and if it has, flush
out the log and force a full resync on the file.

Hmm, this seems like an additional layer that might be
nice (and perhaps an XML log would be appropriate
here), but I would put it an separate inline
translator so that it is not required.  The nice part
is that if the protocol is extended to handle the
journal layer, adding another separate layer like this
would probably be easy!

For the sake of an extra few bytes in the log entry (8 byte time stamp + 8 byte file size), I think it is probably worthwhile having it for crosscheck.

Thanks again for your patience, I know it's not easy
listening to back seat designers :)

I second that apology. :-)

Gordan




[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux