On Mon, Dec 08, 2008 at 09:26:16PM +0530, Chandru wrote:
> On Tuesday 07 October 2008 21:29:51 Bob Montgomery wrote:
> > On Tue, 2008-10-07 at 13:24 +0000, Vivek Goyal wrote:
> > > On Tue, Oct 07, 2008 at 06:21:52PM +0530, Chandru wrote:
> > > > kdump on a quad core Opteron blade machine doesn't give a complete
> > > > vmcore on the system. All works well until we attempt to copy
> > > > /proc/vmcore to some target place ( disk , n/w ). The system
> > > > immediately resets without any OS messages after having copied few mb's
> > > > of vmcore file. Problem also occurs with 2.6.27-rc8 and latest
> > > > kexec-tools. If we pass 'mem=4G' as boot parameter to the first
> > > > kernel, then kdump succeeds in copying a readable vmcore to /var/crash.
> > >
> > > Hi Chandru,
> > >
> > > How much memory this system has got. Can you also paste the output of
> > > /proc/iomem of first kernel.
> > >
> > > Does this system has GART? So looks like we are accessing some memory
> > > area which platform does not like. (We saw issues with GART in the past.)
> > >
> > > Can you also provide /proc/vmcore ELF header (readelf output), in both
> > > the cases (mem=4G and without that).
> > >
> > > You can try putting some printk in /proc/vmcore code and see which
> > > physical memory area you are accessing when system goes bust. If in all
> > > the failure cases it is same physical memory area, then we can try to
> > > find what's so special about it.
> >
> > Or you can assume this is pretty much exactly the problem I ran into in
> > August. I've attached the patch that I'm using with our 2.6.18 kernel
> > to disable CPU-side access by the GART, which prevents the problem on
> > our Family 10H systems. You'll need to fix the directory name for
> > kernels newer than the arch/x86_64 merge.
> >
> > Now that someone else has seen the problem, if this fixes it, I'll
> > submit the patch upstream.
> >
> > Here's the README for the patch:
> >
> > This patch changes the initialization of the GART (in
> > pci-gart.c:init_k8_gatt) to set the DisGartCpu bit in the GART Aperture
> > Control Register. Setting the bit Disables requests from the CPUs from
> > accessing the GART. In other words, CPU memory accesses within the
> > range of addresses in the aperture will not cause the GART to perform an
> > address translation. The aperture area was already being unmapped at
> > the kernel level with clear_kernel_mapping() to prevent accesses from
> > the CPU, but that kernel level unmapping is not in effect in the kexec'd
> > kdump kernel. By disabling the CPU-side accesses within the GART, which
> > does persist through the kexec of the kdump kernel, the kdump kernel is
> > prevented from interacting with the GART during accesses to the dump
> > memory areas which include the address range of the GART aperture.
> > Although the patch can be applied to the kdump kernel, it is not
> > exercised there because the kdump kernel doesn't attempt to initialize
> > the GART.
> >
> > Bob Montgomery
> > working at HP
>
> Hi Bob,
>
> This problem was recently reported on a LS42 blade and the patch given by you
> also resolved the issue here too. However I made couple of changes to
> kexec-tools to ignore GART memory region and not have elf headers created to
> it. This patch also seemed to work on a LS21.
>
> Thanks,
> Chandru
>
> Signed-off-by: Chandru S <chandru at in.ibm.com>
> ---
Hi Chandru,
So this patch will solve the issue (at least for /proc/vmcore) even if
we don't make any changes on kernel side?
Thanks
Vivek
>
> --- kexec-tools/kexec/arch/x86_64/crashdump-x86_64.c.orig 2008-12-08
> 01:50:41.000000000 -0600
> +++ kexec-tools/kexec/arch/x86_64/crashdump-x86_64.c 2008-12-08
> 03:02:45.000000000 -0600
> @@ -47,7 +47,7 @@ static struct crash_elf_info elf_info =
> };
>
> /* Forward Declaration. */
> -static int exclude_crash_reserve_region(int *nr_ranges);
> +static int exclude_region(int *nr_ranges, uint64_t start, uint64_t end);
>
> #define KERN_VADDR_ALIGN 0x100000 /* 1MB */
>
> @@ -164,10 +164,11 @@ static struct memory_range crash_reserve
> static int get_crash_memory_ranges(struct memory_range **range, int *ranges)
> {
> const char *iomem= proc_iomem();
> - int memory_ranges = 0;
> + int memory_ranges = 0, gart = 0;
> char line[MAX_LINE];
> FILE *fp;
> unsigned long long start, end;
> + uint64_t gart_start = 0, gart_end = 0;
>
> fp = fopen(iomem, "r");
> if (!fp) {
> @@ -219,6 +220,10 @@ static int get_crash_memory_ranges(struc
> type = RANGE_ACPI;
> } else if(memcmp(str,"ACPI Non-volatile Storage\n",26) == 0 ) {
> type = RANGE_ACPI_NVS;
> + } else if (memcmp(str, "GART\n", 5) == 0) {
> + gart_start = start;
> + gart_end = end;
> + gart = 1;
> } else {
> continue;
> }
> @@ -233,8 +238,14 @@ static int get_crash_memory_ranges(struc
> memory_ranges++;
> }
> fclose(fp);
> - if (exclude_crash_reserve_region(&memory_ranges) < 0)
> + if (exclude_region(&memory_ranges, crash_reserved_mem.start,
> + crash_reserved_mem.end) < 0)
> return -1;
> + if (gart) {
> + /* exclude GART region if the system has one */
> + if (exclude_region(&memory_ranges, gart_start, gart_end) < 0)
> + return -1;
> + }
> *range = crash_memory_range;
> *ranges = memory_ranges;
> #ifdef DEBUG
> @@ -252,32 +263,27 @@ static int get_crash_memory_ranges(struc
> /* Removes crash reserve region from list of memory chunks for whom elf
> program
> * headers have to be created. Assuming crash reserve region to be a single
> * continuous area fully contained inside one of the memory chunks */
> -static int exclude_crash_reserve_region(int *nr_ranges)
> +static int exclude_region(int *nr_ranges, uint64_t start, uint64_t end)
> {
> int i, j, tidx = -1;
> - unsigned long long cstart, cend;
> struct memory_range temp_region;
>
> - /* Crash reserved region. */
> - cstart = crash_reserved_mem.start;
> - cend = crash_reserved_mem.end;
> -
> for (i = 0; i < (*nr_ranges); i++) {
> unsigned long long mstart, mend;
> mstart = crash_memory_range[i].start;
> mend = crash_memory_range[i].end;
> - if (cstart < mend && cend > mstart) {
> - if (cstart != mstart && cend != mend) {
> + if (start < mend && end > mstart) {
> + if (start != mstart && end != mend) {
> /* Split memory region */
> - crash_memory_range[i].end = cstart - 1;
> - temp_region.start = cend + 1;
> + crash_memory_range[i].end = start - 1;
> + temp_region.start = end + 1;
> temp_region.end = mend;
> temp_region.type = RANGE_RAM;
> tidx = i+1;
> - } else if (cstart != mstart)
> - crash_memory_range[i].end = cstart - 1;
> + } else if (start != mstart)
> + crash_memory_range[i].end = start - 1;
> else
> - crash_memory_range[i].start = cend + 1;
> + crash_memory_range[i].start = end + 1;
> }
> }
> /* Insert split memory region, if any. */
