> On Sep 11, 2019, at 12:34 PM, Waiman Long <longman@xxxxxxxxxx> wrote:
>
> On 9/11/19 5:01 PM, Qian Cai wrote:
>>
>>> On Sep 11, 2019, at 11:05 AM, Waiman Long <longman@xxxxxxxxxx> wrote:
>>>
>>> When allocating a large amount of static hugepages (~500-1500GB) on a
>>> system with large number of CPUs (4, 8 or even 16 sockets), performance
>>> degradation (random multi-second delays) was observed when thousands
>>> of processes are trying to fault in the data into the huge pages. The
>>> likelihood of the delay increases with the number of sockets and hence
>>> the CPUs a system has. This only happens in the initial setup phase
>>> and will be gone after all the necessary data are faulted in.
>>>
>>> These random delays, however, are deemed unacceptable. The cause of
>>> that delay is the long wait time in acquiring the mmap_sem when trying
>>> to share the huge PMDs.
>>>
>>> To remove the unacceptable delays, we have to limit the amount of wait
>>> time on the mmap_sem. So the new down_write_timedlock() function is
>>> used to acquire the write lock on the mmap_sem with a timeout value of
>>> 10ms which should not cause a perceivable delay. If timeout happens,
>>> the task will abandon its effort to share the PMD and allocate its own
>>> copy instead.
>>>
>>> When too many timeouts happens (threshold currently set at 256), the
>>> system may be too large for PMD sharing to be useful without undue delay.
>>> So the sharing will be disabled in this case.
>>>
>>> Signed-off-by: Waiman Long <longman@xxxxxxxxxx>
>>> ---
>>> include/linux/fs.h | 7 +++++++
>>> mm/hugetlb.c | 24 +++++++++++++++++++++---
>>> 2 files changed, 28 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/include/linux/fs.h b/include/linux/fs.h
>>> index 997a530ff4e9..e9d3ad465a6b 100644
>>> --- a/include/linux/fs.h
>>> +++ b/include/linux/fs.h
>>> @@ -40,6 +40,7 @@
>>> #include <linux/fs_types.h>
>>> #include <linux/build_bug.h>
>>> #include <linux/stddef.h>
>>> +#include <linux/ktime.h>
>>>
>>> #include <asm/byteorder.h>
>>> #include <uapi/linux/fs.h>
>>> @@ -519,6 +520,12 @@ static inline void i_mmap_lock_write(struct address_space *mapping)
>>> down_write(&mapping->i_mmap_rwsem);
>>> }
>>>
>>> +static inline bool i_mmap_timedlock_write(struct address_space *mapping,
>>> + ktime_t timeout)
>>> +{
>>> + return down_write_timedlock(&mapping->i_mmap_rwsem, timeout);
>>> +}
>>> +
>>> static inline void i_mmap_unlock_write(struct address_space *mapping)
>>> {
>>> up_write(&mapping->i_mmap_rwsem);
>>> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
>>> index 6d7296dd11b8..445af661ae29 100644
>>> --- a/mm/hugetlb.c
>>> +++ b/mm/hugetlb.c
>>> @@ -4750,6 +4750,8 @@ void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma,
>>> }
>>> }
>>>
>>> +#define PMD_SHARE_DISABLE_THRESHOLD (1 << 8)
>>> +
>>> /*
>>> * Search for a shareable pmd page for hugetlb. In any case calls pmd_alloc()
>>> * and returns the corresponding pte. While this is not necessary for the
>>> @@ -4770,11 +4772,24 @@ pte_t *huge_pmd_share(struct mm_struct *mm, unsigned long addr, pud_t *pud)
>>> pte_t *spte = NULL;
>>> pte_t *pte;
>>> spinlock_t *ptl;
>>> + static atomic_t timeout_cnt;
>>>
>>> - if (!vma_shareable(vma, addr))
>>> - return (pte_t *)pmd_alloc(mm, pud, addr);
>>> + /*
>>> + * Don't share if it is not sharable or locking attempt timed out
>>> + * after 10ms. After 256 timeouts, PMD sharing will be permanently
>>> + * disabled as it is just too slow.
>> It looks like this kind of policy interacts with kernel debug options like KASAN (which is going to slow the system down
>> anyway) could introduce tricky issues due to different timings on a debug kernel.
>
> With respect to lockdep, down_write_timedlock() works like a trylock. So
> a lot of checking will be skipped. Also the lockdep code won't be run
> until the lock is acquired. So its execution time has no effect on the
> timeout.
No only lockdep, but also things like KASAN, debug_pagealloc, page_poison, kmemleak, debug
objects etc that all going to slow down things in huge_pmd_share(), and make it tricky to get a
right timeout value for those debug kernels without changing the previous behavior.
