Hello, > >> > Hi Atsushi san, > >> > > >> > We tried makedumpfile 1.5.0 on a 1TB machine and it seems to regress > >> > badly. We reserved 192MB of memory and following are test results. > >> > > >> > #1. makedumpfile-1.4.2 -E --message-level 1 -d 31 > >> > real 3m47.520s > >> > user 0m56.543s > >> > sys 2m41.631s > >> > > >> > #2. makedumpfile-1.5.0 -E --message-level 1 -d 31 > >> > real 52m25.262s > >> > user 32m51.310s > >> > sys 18m53.265s > >> > > >> > #3. makedumpfile-1.4.2 -c --message-level 1 -d 31 > >> > real 8m49.107s > >> > user 4m34.180s > >> > sys 4m8.691s > >> > > >> > #4. makedumpfile-1.5.0 -c --message-level 1 -d 31 > >> > real 46m48.985s > >> > user 29m35.203s > >> > sys 16m43.149s > >> > > >> > >> Hello Vivek, > >> > >> On v1.5.0 we cannot filter free pages in constant space becuase we > >> have yet to test it. Instead, the existing method is used here, which > >> repeats walking on a whole page frames the number of cycles times. > >> > >> As Kumagai-san explains, the number of cycles can be calculated by the > >> following expression: > >> > >> N = physical memory size / (page size * bit per byte(8) * BUFSIZE_CYCLIC) > >> > >> So, > >> > >> N = 2TB / (4KB * 8 * 1MB) = 64 cycles. > >> > >> I guess on this environment, it took about 50 seconds to filter free > >> pages in one cycle. > > > > Ok, so once we have your walking through page struct patches in, hopefully > > this problem will be gone? > Yes, then free page filtering is integrated in other mem_map array > logics. You can see how it takes from the following message. > > Excluding unnecessary pages : [100 %] STEP [Excluding unnecessary pages] : 0.204574 seconds > > New logic takes equal to or quicker than the total time indicated by > the above kind of messages now. > > > If that's going to take time, can we make using of new logic conditional > > on a command line option. So that user has the option of using old > > logic. > > > > Kumagai-san should decide this. I'm not planning to make the cyclic mode optional. However, I think the performance issue should be improved even without mem_map array logic. So, I will try to reduce the number of cycles as few times as possible for v1.5.1, the performance issue will be improved. To make sure of it, would you re-test with --cyclic-buffer 32768 (32MB), Vivek ? Then the result of v1.5.0 is still too bad, I will consider using the old logic as default logic. > BTW, Kumagai-san, in what version do you plan release of this new > mem_map logic? Around v1.5.2? I want to re-send a new patch set to > which I add small changes and comments for maintainability. Yes, I plan to merge it in v1.5.2. Would you re-send the new patch set based on v1.5.1 ? Additionally, I'm afraid that I haven't reviewed the most of patches for v1.5.1 yet. So I can't tell you when v1.5.2 will be released. Thanks Atsushi Kumagai