Hi,
TL;DR - Add object migration (SMO) to the SLUB allocator and implement
object migration for the XArray and the dcache.
Thanks for you patience with all the RFC's of this patch set. Here it
is, ready for prime time.
Internal fragmentation can occur within pages used by the slub
allocator. Under some workloads large numbers of pages can be used by
partial slab pages. This under-utilisation is bad simply because it
wastes memory but also because if the system is under memory pressure
higher order allocations may become difficult to satisfy. If we can
defrag slab caches we can alleviate these problems.
In order to be able to defrag slab chaches we need to be able to migrate
objects to a new slab. Slab object migration is the core functionality
added by this patch series.
Internal slab fragmentation is a long known problem. This series does
not claim to completely _fix_ the issue. Instead we are adding core
code to the SLUB allocator to enable users of the allocator to help
mitigate internal fragmentation. Object migration is on a per cache
basis, with each cache being able to take advantage of object migration
to varying degrees depending on the nature of the objects stored in the
cache.
Series includes test modules and test code that can be used to verify the
claimed behaviour.
Patch #1 - Adds the callbacks used to enable SMO for a particular cache.
Patch #2 - Updates the slabinfo tool to show operations related to SMO.
Patch #3 - Sorts the cache list putting migratable slabs at front.
Patch #4 - Adds the SMO infrastructure. This is the core patch of the
series.
Patch #5, #6 - Further update slabinfo tool for information just added.
Patch #7 - Add a module for testing SMO.
Patch #8 - Add unit test suite in Python utilising test module from #7.
Patch #9 - Add a new slab cache for the XArray (separate from radix tree).
Patch #10 - Implement SMO for the XArray.
Patch #11 - Add module for testing XArray SMO implementation.
Patch #12 - Add a dentry constructor.
Patch #13 - Use SMO to attempt to reduce fragmentation of the dcache by
selectively freeing dentry objects.
Patch #14 - Add functionality to move slab objects to a specific NUMA node.
Patch #15 - Add functionality to balance slab objects across all NUMA nodes.
The last RFC (RFCv5 and discussion on it) included code to conditionally
exclude SMO for the dcache. This has been removed. IMO it is now not
needed. Al sufficiently bollock'ed me during development that I believe
the dentry code is good and does not negatively effect the dcache. If
someone would like to prove me wrong simply remove the call to
kmem_cache_setup_mobility(dentry_cache, d_isolate, d_partial_shrink);
Testing:
The series has been tested to verify that objects are moved using bare
metal (core i5) and also Qemu. This has not been tested on big metal or
on NUMA hardware.
I have no measurements on performance gains achievable with this set, I
have just verified that the migration works and does not appear to break
anything.
Patch #14 and #15 depend on
CONFIG_SLBU_DEBUG_ON or boot with 'slub_debug'
Thanks for taking the time to look at this.
Tobin
Tobin C. Harding (15):
slub: Add isolate() and migrate() methods
tools/vm/slabinfo: Add support for -C and -M options
slub: Sort slab cache list
slub: Slab defrag core
tools/vm/slabinfo: Add remote node defrag ratio output
tools/vm/slabinfo: Add defrag_used_ratio output
tools/testing/slab: Add object migration test module
tools/testing/slab: Add object migration test suite
lib: Separate radix_tree_node and xa_node slab cache
xarray: Implement migration function for xa_node objects
tools/testing/slab: Add XArray movable objects tests
dcache: Provide a dentry constructor
dcache: Implement partial shrink via Slab Movable Objects
slub: Enable moving objects to/from specific nodes
slub: Enable balancing slabs across nodes
Documentation/ABI/testing/sysfs-kernel-slab | 14 +
fs/dcache.c | 105 ++-
include/linux/slab.h | 71 ++
include/linux/slub_def.h | 10 +
include/linux/xarray.h | 3 +
init/main.c | 2 +
lib/radix-tree.c | 2 +-
lib/xarray.c | 109 ++-
mm/Kconfig | 7 +
mm/slab_common.c | 2 +-
mm/slub.c | 827 ++++++++++++++++++--
tools/testing/slab/Makefile | 10 +
tools/testing/slab/slub_defrag.c | 567 ++++++++++++++
tools/testing/slab/slub_defrag.py | 451 +++++++++++
tools/testing/slab/slub_defrag_xarray.c | 211 +++++
tools/vm/slabinfo.c | 51 +-
16 files changed, 2339 insertions(+), 103 deletions(-)
create mode 100644 tools/testing/slab/Makefile
create mode 100644 tools/testing/slab/slub_defrag.c
create mode 100755 tools/testing/slab/slub_defrag.py
create mode 100644 tools/testing/slab/slub_defrag_xarray.c
--
2.21.0
