On 11/11/2009 06:32 PM, Ben DJ wrote:
> Hi,
>
> I didn't want to hijack the thread, so a new one here.
>
> On Wed, Nov 11, 2009 at 2:10 PM, Robin Hill <robin@xxxxxxxxxxxxxxx> wrote:
>> It's true for RAID-10, yes. You can't physically grow the array, but
>> you can definitely add/remove the bitmap.
>
> Thanks for clearing that up. The manpage is a bit unclear to my read.
>
> I've just been reading threads about proper sizing. Large
> bitmap-chunk seems good, larger than "1 million bits" -- not, and an
> old bug (resolved?) if bitmap-chunk is smaller than raid10 chunk size.
>
> I've two arrays --
>
> /dev/md0 (RAID-1, across 4x 160MB partitions)
>
> /dev/md1 (RAID-10/f2 , across 4x ~1TB partitions, --chunk=256 )
>
> The first array is so small, that resync takes just a few seconds
> anyway. Is there any advantage to still installing an internal
> write-intent bitmap on it?
Not in my opinion. I skip bitmaps on boot arrays and other smallish
arrays like that.
> The second array takes a few hours to resync from scratch, and so the
> bitmap has performance value. What's the right size for
> --bitmap-chunk for an internal bitmap? Iiuc, the default that "is
> automatically deteremined to make best use of available space" results
> in 2x-4x (some say 10%) write-performance slowdowns.
It makes for noticeable slowdowns anyway. How bad is dependent on your
data writing patterns. Lots of random writes will actually show a
larger slowdown than more sequential writing. The main thing to
understand is that a bitmap like this is useless if the raid stack
doesn't stop any write going to the disk unless the bit for that write's
sector is set to dirty. So, when a new write is initiated on a clean
array, the write is help up until the bitmap write to dirty the proper
bits completes, and only then can the normal write proceed. So, with
lots of random I/O, or even with sequential I/O on very small bitmap
chunk sizes, you end up spending a significant amount of time holding up
writes as you dirty the bits on disk. Picking a larger bitmap chunk
helps to increase the likelihood that more writes will stream without
having to wait on a new bitmap dirty write.
Given that the only real benefit to the bitmap is reduced resync time in
the event something happens, and given that as you said a 160MB section
of array can resync in a relatively short time, and given that a smaller
bitmap chunk hurts performance *all* the time versus only helping in
rare circumstances, bigger is better in my opinion.
I haven't done specific testing of performance differences with
different size bitmap chunks, but my seat of the pants review puts the
32768 area as a good starting point. Any given chunk will resync in
just a second or so, but it doesn't cause as much performance slowdown
as the default chunk size.
--
Doug Ledford <dledford@xxxxxxxxxx>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford
Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband
Attachment:
signature.asc
Description: OpenPGP digital signature
