The patch titled From: Konstantin Khlebnikov <khlebnikov@xxxxxxxxxx> has been added to the -mm tree. Its filename is proc-report-file-anon-bit-in-proc-pid-pagemap.patch Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/SubmitChecklist when testing your code *** The -mm tree is included into linux-next and is updated there every 3-4 working days ------------------------------------------------------ From: Konstantin Khlebnikov <khlebnikov@xxxxxxxxxx> Subject: proc: report file/anon bit in /proc/pid/pagemap This is an implementation of Andrew's proposal to extend the pagemap file bits to report what is missing about tasks' working set. The problem with the working set detection is multilateral. In the criu (checkpoint/restore) project we dump the tasks' memory into image files and to do it properly we need to detect which pages inside mappings are really in use. The mincore syscall I though could help with this did not. First, it doesn't report swapped pages, thus we cannot find out which parts of anonymous mappings to dump. Next, it does report pages from page cache as present even if they are not mapped, and it doesn't make that has not been cow-ed. Note, that issue with swap pages is critical -- we must dump swap pages to image file. But the issues with file pages are optimization -- we can take all file pages to image, this would be correct, but if we know that a page is not mapped or not cow-ed, we can remove them from dump file. The dump would still be self-consistent, though significantly smaller in size (up to 10 times smaller on real apps). Andrew noticed, that the proc pagemap file solved 2 of 3 above issues -- it reports whether a page is present or swapped and it doesn't report not mapped page cache pages. But, it doesn't distinguish cow-ed file pages from not cow-ed. I would like to make the last unused bit in this file to report whether the page mapped into respective pte is PageAnon or not. [comment stolen from Pavel Emelyanov's v1 patch] Signed-off-by: Konstantin Khlebnikov <khlebnikov@xxxxxxxxxx> Cc: Pavel Emelyanov <xemul@xxxxxxxxxxxxx> Cc: Matt Mackall <mpm@xxxxxxxxxxx> Cc: Hugh Dickins <hughd@xxxxxxxxxx> Cc: Rik van Riel <riel@xxxxxxxxxx> Acked-by: KOSAKI Motohiro <kosaki.motohiro@xxxxxxxxxxxxxx> Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> --- Documentation/vm/pagemap.txt | 2 - fs/proc/task_mmu.c | 46 ++++++++++++++++++++------------- 2 files changed, 30 insertions(+), 18 deletions(-) diff -puN Documentation/vm/pagemap.txt~proc-report-file-anon-bit-in-proc-pid-pagemap Documentation/vm/pagemap.txt --- a/Documentation/vm/pagemap.txt~proc-report-file-anon-bit-in-proc-pid-pagemap +++ a/Documentation/vm/pagemap.txt @@ -16,7 +16,7 @@ There are three components to pagemap: * Bits 0-4 swap type if swapped * Bits 5-54 swap offset if swapped * Bits 55-60 page shift (page size = 1<<page shift) - * Bit 61 reserved for future use + * Bit 61 page is file-page or shared-anon * Bit 62 page swapped * Bit 63 page present diff -puN fs/proc/task_mmu.c~proc-report-file-anon-bit-in-proc-pid-pagemap fs/proc/task_mmu.c --- a/fs/proc/task_mmu.c~proc-report-file-anon-bit-in-proc-pid-pagemap +++ a/fs/proc/task_mmu.c @@ -700,6 +700,7 @@ struct pagemapread { #define PM_PRESENT PM_STATUS(4LL) #define PM_SWAP PM_STATUS(2LL) +#define PM_FILE PM_STATUS(1LL) #define PM_NOT_PRESENT PM_PSHIFT(PAGE_SHIFT) #define PM_END_OF_BUFFER 1 @@ -733,22 +734,33 @@ static int pagemap_pte_hole(unsigned lon return err; } -static u64 swap_pte_to_pagemap_entry(pte_t pte) +static void pte_to_pagemap_entry(pagemap_entry_t *pme, + struct vm_area_struct *vma, unsigned long addr, pte_t pte) { - swp_entry_t e = pte_to_swp_entry(pte); - return swp_type(e) | (swp_offset(e) << MAX_SWAPFILES_SHIFT); -} + u64 frame, flags; + struct page *page = NULL; -static void pte_to_pagemap_entry(pagemap_entry_t *pme, pte_t pte) -{ - if (is_swap_pte(pte)) - *pme = make_pme(PM_PFRAME(swap_pte_to_pagemap_entry(pte)) - | PM_PSHIFT(PAGE_SHIFT) | PM_SWAP); - else if (pte_present(pte)) - *pme = make_pme(PM_PFRAME(pte_pfn(pte)) - | PM_PSHIFT(PAGE_SHIFT) | PM_PRESENT); - else + if (pte_present(pte)) { + frame = pte_pfn(pte); + flags = PM_PRESENT; + page = vm_normal_page(vma, addr, pte); + } else if (is_swap_pte(pte)) { + swp_entry_t entry = pte_to_swp_entry(pte); + + frame = swp_type(entry) | + (swp_offset(entry) << MAX_SWAPFILES_SHIFT); + flags = PM_SWAP; + if (is_migration_entry(entry)) + page = migration_entry_to_page(entry); + } else { *pme = make_pme(PM_NOT_PRESENT); + return; + } + + if (page && !PageAnon(page)) + flags |= PM_FILE; + + *pme = make_pme(PM_PFRAME(frame) | PM_PSHIFT(PAGE_SHIFT) | flags); } #ifdef CONFIG_TRANSPARENT_HUGEPAGE @@ -815,7 +827,7 @@ static int pagemap_pte_range(pmd_t *pmd, if (vma && (vma->vm_start <= addr) && !is_vm_hugetlb_page(vma)) { pte = pte_offset_map(pmd, addr); - pte_to_pagemap_entry(&pme, *pte); + pte_to_pagemap_entry(&pme, vma, addr, *pte); /* unmap before userspace copy */ pte_unmap(pte); } @@ -869,11 +881,11 @@ static int pagemap_hugetlb_range(pte_t * * For each page in the address space, this file contains one 64-bit entry * consisting of the following: * - * Bits 0-55 page frame number (PFN) if present + * Bits 0-54 page frame number (PFN) if present * Bits 0-4 swap type if swapped - * Bits 5-55 swap offset if swapped + * Bits 5-54 swap offset if swapped * Bits 55-60 page shift (page size = 1<<page shift) - * Bit 61 reserved for future use + * Bit 61 page is file-page or shared-anon * Bit 62 page swapped * Bit 63 page present * _ Subject: From: Konstantin Khlebnikov <khlebnikov@xxxxxxxxxx> Patches currently in -mm which might be from khlebnikov@xxxxxxxxxx are linux-next.patch proc-pagemap-correctly-report-non-present-ptes-and-holes-between-vmas.patch mm-remove-swap-token-code.patch mm-vmscan-remove-lumpy-reclaim.patch mm-vmscan-do-not-stall-on-writeback-during-memory-compaction.patch mm-vmscan-remove-reclaim_mode_t.patch mm-correctly-synchronize-rss-counters-at-exit-exec.patch kernel-cgroup-push-rcu-read-locking-from-css_is_ancestor-to-callsite.patch mm-memcg-count-pte-references-from-every-member-of-the-reclaimed-hierarchy.patch mm-memcg-count-pte-references-from-every-member-of-the-reclaimed-hierarchy-fix.patch mm-memcg-scanning_global_lru-means-mem_cgroup_disabled.patch mm-memcg-move-reclaim_stat-into-lruvec.patch mm-push-lru-index-into-shrink_active_list.patch mm-push-lru-index-into-shrink_active_list-fix.patch mm-mark-mm-inline-functions-as-__always_inline.patch mm-remove-lru-type-checks-from-__isolate_lru_page.patch mm-memcg-kill-mem_cgroup_lru_del.patch mm-memcg-use-vm_swappiness-from-target-memory-cgroup.patch proc-report-file-anon-bit-in-proc-pid-pagemap.patch proc-smaps-carefully-handle-migration-entries.patch proc-smaps-show-amount-of-nonlinear-ptes-in-vma.patch proc-smaps-show-amount-of-hwpoison-pages.patch fork-call-complete_vfork_done-after-clearing-child_tid-and-flushing-rss-counters.patch c-r-prctl-add-ability-to-set-new-mm_struct-exe_file-update-after-mm-num_exe_file_vmas-removal.patch -- To unsubscribe from this list: send the line "unsubscribe mm-commits" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html