On 10/18/22 4:11 PM, Michał Mirosław wrote:
On Tue, 18 Oct 2022 at 12:36, Muhammad Usama Anjum
<usama.anjum@xxxxxxxxxxxxx> wrote:
[...]
I've included the masks which the CRIU developers have specified.
max_out_page is another new optional variable which is needed to
terminate the operation without visiting all the pages after finding the
max_out_page number of desired pages. There is no way to terminate the
operation without this variable.
How does the interface looks now? Please comment.
/* PAGEMAP IOCTL */
#define PAGEMAP_GET _IOWR('f', 16, struct pagemap_sd_args)
#define PAGEMAP_CLEAR _IOWR('f', 17, struct pagemap_sd_args)
#define PAGEMAP_GET_AND_CLEAR _IOWR('f', 18, struct pagemap_sd_args)
Why are three IOCTLs needed? Could CLEAR be a flag (like the
PAGEMAP_NO_REUSED_REGIONS) or 'cmask' and GET be implied when vec !=
NULL?
Makes sense. Yes, this can be done.
/* Bits are set in the bitmap of the page_region and masks in
pagemap_sd_args */
#define PAGE_IS_SD 1 << 0
#define PAGE_IS_FILE 1 << 1
#define PAGE_IS_PRESENT 1 << 2
#define PAGE_IS_SWAPED 1 << 3
/**
* struct page_region - Page region with bitmap flags
* @start: Start of the region
* @len: Length of the region
* bitmap: Bits sets for the region
*/
struct page_region {
__u64 start;
__u64 len;
__u64 bitmap;
};
Could you explain what units start and len are using? Are they bytes
or pages (what size)?
These are page sizes. Start must be the starting address of the page.
Length don't need to be the exact multiple of the page size. All the
pages in the length are included just like mincore() syscall.
/**
* struct pagemap_sd_args - Soft-dirty IOCTL argument
Nit: it's not soft-dirty-specific anymore. Maybe "pagemap_scan_args"?
* @start: Starting address
* @len: Length of the region
* @vec: Output page_region struct array
* @vec_len: Length of the page_region struct array
* @max_out_page: Optional max output pages (It must be less than
vec_len if specified)
Why is it required to be less than vec_len? vec_len effectively
specifies max number of ranges to find, and this new additional field
counts pages, I suppose?
BTW, if we count pages, then what size of them? Maybe using bytes
(matching start/len fields) would be more consistent?
Yes, it if for counting pages. As the regions can have multiple pages,
user cannot specify through the number of regions that how many pages
does he need. Page size is used here as well like the start and len.
This is optional argument as this is only needed to emulate the Windows
syscall getWriteWatch.
* @flags: Special flags for the IOCTL
* @rmask: Special flags for the IOCTL
* @amask: Special flags for the IOCTL
* @emask: Special flags for the IOCTL
* @__reserved: Reserved member to preserve data alignment. Must be 0.
*/
struct pagemap_sd_args {
__u64 __user start;
__u64 len;
__u64 __user vec; // page_region
__u64 vec_len; // sizeof(page_region)
__u32 flags; // special flags
__u32 rmask;
__u32 amask;
__u32 emask;
__u32 max_out_page;
__u32 __reserved;
};
/* Special flags */
#define PAGEMAP_NO_REUSED_REGIONS 0x1
What does this flag do?
Some non-dirty pages get marked as dirty because of the kernel's
internal activity. The dirty bit of the pages is stored in the VMA flags
and in the per page flags. If any of these two bits are set, the page is
considered to be dirty. Suppose you have cleared the dirty bit of half
of VMA which will be done by splitting the VMA and clearing dirty flag
in the half VMA and the pages in it. Now kernel may decide to merge the
VMAs again as dirty bit of VMAs isn't considered if the VMAs should be
merged. So the half VMA becomes dirty again. This splitting/merging
costs performance. The application receives a lot of pages which aren't
dirty in reality but marked as dirty. Performance is lost again here.
This PAGEMAP_NO_REUSED_REGIONS flag is used to don't depend on the dirty
flag in the VMA flags. It only depends on the individual page dirty bit.
With doing this, the new memory regions which are just created, doesn't
look like dirty when seen with the IOCTL, but look dirty when seen from
pagemap. This seems okay as the user of this flag know the implication
of using it.
Best Regards
Michał Mirosław
