On 24 Nov 2016, at 18:59, Balbir Singh wrote: > On 23/11/16 03:25, Zi Yan wrote: >> From: Zi Yan <zi.yan@xxxxxxxxxxxxxx> >> >> Hi all, >> >> This patchset boosts the hugepage migration throughput and helps THP migration >> which is added by Naoya's patches: https://lwn.net/Articles/705879/. >> >> Motivation >> =============================== >> >> In x86, 4KB page migrations are underutilizing the memory bandwidth compared >> to 2MB THP migrations. I did some page migration benchmarking on a two-socket >> Intel Xeon E5-2640v3 box, which has 23.4GB/s bandwidth, and discover >> there are big throughput gap, ~3x, between 4KB and 2MB page migrations. >> >> Here are the throughput numbers for different page sizes and page numbers: >> | 512 4KB pages | 1 2MB THP | 1 4KB page >> x86_64 | 0.98GB/s | 2.97GB/s | 0.06GB/s >> >> As Linux currently use single-threaded page migration, the throughput is still >> much lower than the hardware bandwidth, 2.97GB/s vs 23.4GB/s. So I parallelize >> the copy_page() part of THP migration with workqueue and achieve 2.8x throughput. >> >> Here are the throughput numbers of 2MB page migration: >> | single-threaded | 8-thread >> x86_64 2MB | 2.97GB/s | 8.58GB/s >> > > Whats the impact on CPU utilization? Is there a huge impact? > > Balbir Singh. It depends on the throughput we can achieve. For single-threaded copy, the current routine, it takes one CPU 2MB/(2.97GB/s) = 657.6 us to copy one 2MB page. For 8-thread copy, it take 8 CPUs 2MB/(8.58GB/s) = 227.6 us to copy one 2MB page. If we have 8 idle CPUs, I think it worths using them. I am going to add code to check idle_cpu() in the system before doing the copy. If no idle CPUs are present, I can fall back to single-threaded copy. -- Best Regards Yan Zi
Attachment:
signature.asc
Description: OpenPGP digital signature