On 2014-10-03 14:48, Mike Snitzer wrote:
On Fri, Oct 03 2014 at 9:23am -0400,
Mike Snitzer <snitzer@xxxxxxxxxx> wrote:
On Fri, Oct 03 2014 at 7:48am -0400,
Junichi Nomura <j-nomura@xxxxxxxxxxxxx> wrote:
This series of patches reduce the memory overhead of device-mapper
device mainly by removing unused mempool for bio vecs.
DM creates per-device bioset to ensure forward progress under
low memory situation and bioset always includes mempool for bvec.
However, with the introduction of immutable bvec by Kent Overstreet,
dm core now uses bio_clone_fast for creating a clone bio, and no longer
needs bvecs for it.
For example, when you create 10,000 bio-based DM devices and 1,000
request-based DM devices, memory usage of biovec under no load is:
# grep biovec /proc/slabinfo
biovec-256 418068 418068 4096 ...
biovec-128 0 0 2048 ...
biovec-64 0 0 1024 ...
biovec-16 0 0 256 ...
With this patch series applied, the usage becomes:
# grep biovec /proc/slabinfo
biovec-256 116 116 4096 ...
biovec-128 0 0 2048 ...
biovec-64 0 0 1024 ...
biovec-16 0 0 256 ...
So 4096 * (418068 - 116) = 1.6GB of memory is saved in this example.
Very nice Junichi!
Kent definitely left us some low hanging fruit that offers a huge win
(and fixes the really long-standing problem of excessive memory reserves
for each request-based DM device!)
Jens, this is obviously an awkward patchset given it touches both block
and DM (block more so than DM). I'd be fine with you pulling it into
block (or vice-versa if you're OK with me staging it for 3.18 via
linux-dm.git with your Acked-by).
I picked up this patchset, refactored the patches based on my earlier
suggested s/bioset_nobvec_create/bioset_create_nobvec/ rename and
tweaked headers a bit. I've staged the result in the for-next branch of
linux-dm.git for v3.18 inclusion, see:
https://git.kernel.org/cgit/linux/kernel/git/device-mapper/linux-dm.git/log/?h=for-next
b83bb63 dm: remove nr_iovecs parameter from alloc_tio()
b875215 block: use bio_clone_fast() in blk_rq_prep_clone()
cd595c9 block: add bioset_create_nobvec()
3611594 dm: use bioset_create_nobvec()
Jens, I can easily yield this patchset to you if you care. Or I can
backfill your Acked-by in the above 2 block commits (I obviously won't
just send block changes upstream without your OK!).
My only worry is the switch to fast clones, apart from that, it's pretty
basic. But we are pretty close to the 3.18 merge window, and as such,
I'd prefer to let it simmer for a release.
--
Jens Axboe
--
dm-devel mailing list
dm-devel@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/dm-devel
