* Dave Hansen <dave.hansen@xxxxxxxxx> [2016-08-03 11:17:43]: > On 08/02/2016 11:38 PM, Srikar Dronamraju wrote: > > * Dave Hansen <dave.hansen@xxxxxxxxx> [2016-08-02 11:09:21]: > >> On 08/02/2016 06:19 AM, Srikar Dronamraju wrote: > >>> Kernels compiled with CONFIG_DEFERRED_STRUCT_PAGE_INIT will initialise > >>> only certain size memory per node. The certain size takes into account > >>> the dentry and inode cache sizes. However such a kernel when booting a > >>> secondary kernel will not be able to allocate the required amount of > >>> memory to suffice for the dentry and inode caches. This results in > >>> crashes like the below on large systems such as 32 TB systems. > >> > >> What's a "secondary kernel"? > >> > > I mean the kernel thats booted to collect the crash, On fadump, the > > first kernel acts as the secondary kernel i.e the same kernel is booted > > to collect the crash. > > OK, but I'm still not seeing what the problem is. You've said that it > crashes and that it crashes during inode/dentry cache allocation. > > But, *why* does the same kernel image crash in when it is used as a > "secondary kernel"? > I guess you already got it. But let me try to explain it again. Lets say we have a 32 TB system with 16 nodes each node having 2T of memory. We are assuming deferred page initialisation is configured. When the regular kernel boots, 1. It reserves 5% of the memory for fadump. 2. It initializes 8GB per node, i.e 128GB 3. It allocated dentry/inode cache which is around 16GB. 4. It then kicks the parallel page struct initialization. Now lets say kernel crashed and fadump was triggered. 1. The same kernel boots in the 5% reserved space which is 1600GB 2. It reserves the rest 95% memory. 3. It tries to initialize 8GB per node but can only initialize 8GB. (since except for 1st node the rest nodes are all reserved) 4. It tries to allocate dentry/inode cache of 16GB but fails. (tries to reclaim but reclaim needs spinlock and spinlock is not yet initialized.) -- Thanks and Regards Srikar Dronamraju -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>