On Thu, May 12, 2011 at 03:22:44PM -0700, Eric W. Biederman wrote: > "K.Prasad" <prasad at linux.vnet.ibm.com> writes: > > > Hi All, > > We've been trying to study and improve the kdump behaviour when > > a panic is triggered due to an unrecoverable memory error causing a > > machine check exception (MCE) followed by a kernel panic. > > > > In this context we foresee a few issues in capturing kdump and would > > like to receive comments about the ways to handle them. > > > > Probable Issues when capturing coredump through kdump following a memory > > error > > --------------------------- > > - First, a coredump of the memory from the crashing kernel isn't really > > helpful in debugging the crash that was caused due to a faulty memory. > > Collecting the same has some of the problems illustrated below. It should > > therefore suffice to let the user know the reason of the crash > > rather than provide a complete dump of the memory. > > > > For this, a 'slim' yet crash-tool readable coredump containing: > > - message about the cause (such as crash due to unrecoverable memory error) > > in the coredump's elf-note section. > > - and no data from the memory of the 'crashing' kernel (their elf > > sections can be reduced to zero length). > > may be suitable. > > > > - Alternatively, if the kdump kernel decides to capture the coredump, > > its attempts to read the faulty memory location may lead to subsequent > > faults in the context of kdump kernel with fatal consequences. This > > may either be avoided by: > > > > a) Pass the address of the corrupt memory location to the kdump kernel > > and skip reading that location while creating the vmcore. This needs > > an instance of 'struct mce' (from the 'crashing' kernel), which > > already contains the faulty memory address (in the physical address > > form, which should be confirmed using the IA32_MCi_MISC[8:6] bits stored > > in 'misc' field of 'struct mce') to be populated inside the elf > > (-notes?) section. > > > > b) Use modified copy applications (such as a modified 'cp' command) > > that can map the /dev/oldmem into user-space and then initiate the > > creation of vmcore. In this method, the user-space process performing > > the copy will receive a SIGBUS while consuming the faulty memory (through > > INT18 -> do_machine_check) but it must be modified to be resilient to the > > signal, while intelligently skipping to the subsequent memory location > > for further copying. Meanwhile the data for the faulty memory location > > can be represented using 'zero-ed' data and the vmcore enhanced to > > indicate the cause of the crash as one resulting from a fatal MCE. > > > > Any thoughts/suggestions? > > In practice this all works for me. > > I have received several crash dumps where there was an mce error. > > I admit I have my userspace configured to just grab the dmesg from the > kernel log and not do a full crash dump. So in that sense I am already > a slim crash dump. > > But in practice with real hardware errors it is working today without > kernel changes. > The problem with the existing kernel code is that it allows for the old kernel's memory regions to be read (through read_vmcore function), although intelligent userspace tools may avoid such a possibility (like the one you mentioned). Given that the system can experience recursive MCE faults while reading the corrupt memory region, a 'slim' vmcore region presented by the kernel to the user-space would be a safe option. We could also use such a dump to include more relevant information such as address of corrupt memory, type of memory error, etc. Thanks, K.Prasad
