On 04/04/16 at 05:46am, Atsushi Kumagai wrote: > Hello, > > Let me get this straight. > >> >The major memory is consumed by makedumpfile from the test result. > >> >crashkernel=auto doesn't work any more, if option --num-threads is > >> >set. Even more, there is no warning to let user enlarge the reserved > >> >memory. > >> > > >> > >> Yes, we should remind user if they want to use too much threads. > > After all, the ENOMEM issue isn't peculiar to a specific patch but > just a matter of course "more threads consume more memory" ? > (At least I expect so.) Hi, Atsushi. Yes, it is, because ENOMEM never happen once the reserved memory is enlarged manually from my testing. Maybe there should be a generic formular to estimate how much memory will be consumed by specific makedumpfile config. > > >This new feature multi-threads will consume more memory during dumping > >vmcore in 2nd kernel. Is it possible to improve it? > > makedumpfile is designed to work in limited memory space, but the extra > memory consumption for each thread isn't considered. > I think it can be estimated by checking code and measured by actual test. > > e.g. If we find that each thread requires 100KB more memory, 100KB * num-thread > should be reserved before filtering processing. > As I said to Zhou, the logic should be inserted into the cyclic buffer > allocation method (calculate_cyclic_buffer_size()). Agree. > > >> >>And I think it will also work if the reserved memory is set to 1G. > >> > > >> >Yes, makedumpfile can work well under 1GB reserved memory. > >> > > >> >> > >> >>>The cache should be dropped before testing, otherwise makedumpfile will > >> >>>fail to dump vmcore. > >> >>>echo 3 > /proc/sys/vm/drop_caches > >> >>>Maybe there is something cleanup we can do to avoid this. > >> >>> > >> >>>Following is the result with different parameter for option > >> >>>--num-threads. > >> >>> > >> >>>makedumpfile -l --num-threads 128 --message-level 1 -d 31 /proc/vmcore a.128 > >> >>>real 5m34.116s > >> >>>user 103m42.531s > >> >>>sys 86m12.586s > >> >[ snip ] > >> >>>makedumpfile -l --num-threads 0 --message-level 1 -d 31 /proc/vmcore a.0 > >> >>>real 3m46.531s > >> >>>user 3m29.371s > >> >>>sys 0m16.909s > >> >>> > >> >>>makedumpfile.back -l --message-level 1 -d 31 /proc/vmcore a > >> >>>real 3m55.712s > >> >>>user 3m39.254s > >> >>>sys 0m16.287s > >> >>> > >> >>>Once the reserved memory is enlarged, makedumpfile works well with or > >> >>>without this increaseing patch. > >> >>> > >> >>>But there is an another issue I found during testing. makedumpfile may > >> >>>hang in about 24%. And with option --num-threads 64, this issue is also > >> >>>occured. > >> >>> > >> >> > >> >>Will it occur with patch v3? > >> >>If it not occurs, then neither of the previous two increasing patches will work? > >> >> > >> >>And did you test it with or without the increasing patch? > >> > > >> >without this increasing patch, v4 works well. > >> > > >> > >> Do you mean makedumpfile won't hang without the increasing patch? > > > >Seem that, but I cann't confirm it, since this issue occurs very > >occasionally. > > I want to know how many times were the two cases (v4 and v4+inc) tested > and how often did them hang. I have test it more than twenty times with patch v4, and there is no hang during testing. Appending option --num-threads 32 or bigger thread number, makedumpfile will hang in about every five times with increasing patch based on patch v4. Thanks Minfei > > > Thanks, > Atsushi Kumagai > > >Thanks > >Minfei
