On Wed, Jan 02 2019, Richard Alloway wrote: > > I've actually got the slabinfo data from the array with 23 devices. The object size is 8872 bytes there. That's pretty good evidence that we need to rethink the allocation of the stripe_head data structure. > > raid6-md0 actually consumes about 3.1 MB additional RAM for each > check, on average (based on 4 checks). Since the first check consumes > the most RAM, excluding it results in an average of 1.8 MB of > additional RAM per check for checks 2 through 4. I still don't understand why the usage grows ... maybe it is just extra fragmentation, but it seems odd... > A quick-and-dirty hack might be to change both kmem_cache_create() > calls in raid5.c to use roundup_pow_of_two() on their second argument. > That would waste some memory, but should cause it to use the same slabs > that kmalloc() uses, so that there is more chance that memory freed by > raid5 can be used by something else. > It would also mean it would use 1-page slabs instead of 8-page slabs. > > This being an easy thing to change and test, I did this and ran the exact same set of tests (also before the holidays). > > Now, instead of the first consistency check leaving ~7 MB of RAM allocated and further checks allocating ~1.8 MB more RAM, each check leaves approx. 30 KB of additional RAM allocated! This is really only approximate, though, since I am taking all of the single page kmalloc slabs into account and cannot isolate the raid6-mdX slabs. It's possible that no additional RAM is left allocated after each check using this modification. > > Since the wasted memory seems to be much less and the retained allocations disappear, what would the negatives be, if any, of using this in an ongoing manner? Besides the overhead of keeping the kernel up-to-date, of course. No, I cannot think of any negatives beyond what you have noted. Thanks for testing and reporting. NeilBrown > > Thanks! > > -Rich > > > NeilBrown
Attachment:
signature.asc
Description: PGP signature
