John R Pierce wrote:
Now you are telling me that somehow you have code that makes your
database stuff its journal on your RAID controller's cache. Cool, mind
sharing it with the rest of us?
fsync(handle);
If we -dont- do this after processing each event, and the system fails
catastrophically, a thousand or so events (a couple seconds worth of
realtime data) are lost in the operating systems buffering. I feel
like I'm repeating myself.
Oh, I thought you meant that you might have some special code to put for
example postgresql's database journal on the raid cache.
If the aggregate queues are up to 10GB, I really wonder wonder how
much faster your hardware raid makes things unless of course your
cache is much larger than 2GB. Just on the basis of the inadequate
size of your cache I would give software raid + RAM card the benefit
of the doubt.
the combined queue files average a few to 10GB total under a normal
workload. if a downstream subscriber backs up, they can grow quite a
bit, up to an arbitrarily set 100GB limit.. its these queue files that
we are flushing with fsync(). each fsync is writing a few K to a few
100K bytes out, one 'event' worth of data which has been appended to one
or another of the queues, from where it will eventually be forwarded to
some number of downstream subscribers. What we're calling a journal is
just the index/state of these queues, stored in a couple seperate very
small files, that also get fsync() on writes, it has NOTHING to do with
the file system.
Yes it does if you have a journaling filesystem. For example,
fsync/fsyncdata calls get special treatment on filesystems like ext3.
When the filesystem containing the files on which fsync is called and it
is mounted data=journal, those writes hit the filesystem journal first
after which the fsync gets to say OK. After that the kernel will write
from the journal to the rest of the disk at its leisure.
to store these queues on a ramcard, we'd need 100GB to handle the backup
cases, which, I hope you can agree, is ludicrious.
Which is not I would do too. I would just put the filesystem's journal
on a ramcard with data journaling which will achieve the same effect of
what your hardware raid writeback cached controller does. Data hits
ramcard, fsync says OK, kernel writes to disk from ramcard at its
leisure, just like the RAID card.
Throughput under test load (incoming streams free running as fast as
they can be processed)
no fsync - 1000 events/second
fsync w/ direct connect disk - 50-80 events/second
fsync w/ hardware writeback cached raid - 800/second
seems like a clear win to me.
Yeah, with your paltry journal files, they would fit in the raid cache.
I would imagine that 'fsync w/ direct connect disk + filesystem journal
on ramcard' would give you the same results as 'fsync w/ hardware
writeback cached raid'
The performance therefore comes not from the RAID processing being done
on a processor on the card but from its cache. So if you have such a
card, you could get away with ext2 since they should not be any
filesystem corruption due to power loss or otherwise.
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos
