Sometimes the kernel doesn't take full advantage of system memory bandwidth, leading to a single CPU spending excessive time in initialization paths where the data scales with memory size. Multithreading naturally addresses this problem, and this series is the first step. It extends padata, a framework that handles many parallel singlethreaded jobs, to handle multithreaded jobs as well by adding support for splitting up the work evenly, specifying a minimum amount of work that's appropriate for one helper thread to do, load balancing between helpers, and coordinating them. More documentation in patches 4 and 7. The first user is deferred struct page init, a large bottleneck in kernel boot--actually the largest for us and likely others too. This path doesn't require concurrency limits, resource control, or priority adjustments like future users will (vfio, hugetlb fallocate, munmap) because it happens during boot when the system is otherwise idle and waiting on page init to finish. This has been tested on a variety of x86 systems and speeds up kernel boot by 6% to 49% by making deferred init 63% to 91% faster. Patch 6 has detailed numbers. Test results from other systems appreciated. This series is based on v5.6 plus these three from mmotm: mm-call-touch_nmi_watchdog-on-max-order-boundaries-in-deferred-init.patch mm-initialize-deferred-pages-with-interrupts-enabled.patch mm-call-cond_resched-from-deferred_init_memmap.patch All of the above can be found in this branch: git://oss.oracle.com/git/linux-dmjordan.git padata-mt-definit-v1 https://oss.oracle.com/git/gitweb.cgi?p=linux-dmjordan.git;a=shortlog;h=refs/heads/padata-mt-definit-v1 The future users and related features are available as work-in-progress here: git://oss.oracle.com/git/linux-dmjordan.git padata-mt-wip-v0.3 https://oss.oracle.com/git/gitweb.cgi?p=linux-dmjordan.git;a=shortlog;h=refs/heads/padata-mt-wip-v0.3 Thanks to everyone who commented on the last version of this[0], including Alex Williamson, Jason Gunthorpe, Jonathan Corbet, Michal Hocko, Pavel Machek, Peter Zijlstra, Randy Dunlap, Robert Elliott, Tejun Heo, and Zi Yan. RFC v4 -> padata v1: - merged with padata (Peter) - got rid of the 'task' nomenclature (Peter, Jon) future work branch: - made lockdep-aware (Jason, Peter) - adjust workqueue worker priority with renice_or_cancel() (Tejun) - fixed undo problem in VFIO (Alex) The remaining feedback, mainly resource control awareness (cgroup etc), is TODO for later series. [0] https://lore.kernel.org/linux-mm/20181105165558.11698-1-daniel.m.jordan@xxxxxxxxxx/ Daniel Jordan (7): padata: remove exit routine padata: initialize earlier padata: allocate work structures for parallel jobs from a pool padata: add basic support for multithreaded jobs mm: move zone iterator outside of deferred_init_maxorder() mm: parallelize deferred_init_memmap() padata: document multithreaded jobs Documentation/core-api/padata.rst | 41 +++-- include/linux/padata.h | 43 ++++- init/main.c | 2 + kernel/padata.c | 277 ++++++++++++++++++++++++------ mm/Kconfig | 6 +- mm/page_alloc.c | 118 ++++++------- 6 files changed, 355 insertions(+), 132 deletions(-) base-commit: 7111951b8d4973bda27ff663f2cf18b663d15b48 prerequisite-patch-id: 4ad522141e1119a325a9799dad2bd982fbac8b7c prerequisite-patch-id: 169273327e56f5461101a71dfbd6b4cfd4570cf0 prerequisite-patch-id: 0f34692c8a9673d4c4f6a3545cf8ec3a2abf8620 -- 2.26.2
