----- Original Message ----- > Hello Dave, > > When issuing "kmem -s" with crash 6.0.2 (same with 6.0.0) on a s390x > linux-3.2 kernel dump, I get the following: > > ... > 1ff0a600 signal_cache 1104 112 150 6 32k > 1ff0a540 sighand_cache 2088 82 98 7 32k > > kmem: invalid structure member offset: page_lru > FILE: memory.c LINE: 15856 FUNCTION: count_partial() > > [./crash] error trace: 80099674 => 80097156 => 80083c40 => 801206ba > > 801206ba: OFFSET_verify+138 > 80083c40: get_kmem_cache_slub_data+1028 > 80097156: dump_kmem_cache_slub+902 > 80099674: cmd_kmem+6636 > > kmem: invalid structure member offset: page_lru > FILE: memory.c LINE: 15856 FUNCTION: count_partial() > > > > I think the reason is that MEMBER_OFFSET_INIT(page_lru, "page", "lru") fails. > See the following gdb session: > > > Breakpoint 1, vm_init () at memory.c:375 > 375 MEMBER_OFFSET_INIT(page_lru, "page", "lru"); > (gdb) s > datatype_info (name=0x80415c4a "page", member=0x803ce36c "lru", > dm=0x0) > at symbols.c:4919 > 4919 if (dm == ANON_MEMBER_OFFSET_REQUEST) > (gdb) n > 4922 strcpy(buf, name); > (gdb) n > 4924 req = (struct gnu_request *)GETBUF(sizeof(struct > gnu_request)); > (gdb) n > 4925 req->command = GNU_GET_DATATYPE; > (gdb) n > 4926 req->flags |= GNU_RETURN_ON_ERROR; > (gdb) n > 4927 req->name = buf; > (gdb) n > 4928 req->member = member; > (gdb) n > 4929 req->fp = pc->nullfp; > (gdb) n > 4931 gdb_interface(req); > (gdb) n > 4932 if (req->flags & GNU_COMMAND_FAILED) { > (gdb) print *req > $1 = {command = 5, buf = 0x0, fp = 0x8060d540, addr = 0, addr2 = 0, > count = 0, > flags = 8, name = 0x3ffffffda94 "page", length = 56, typecode = 3, > typename = 0x0, target_typename = 0x0, target_length = 0, > target_typecode = 0, is_typedef = 0, member = 0x803ce36c "lru", > member_offset = -1, member_length = 0, member_typecode = 0, value = > 0, > tagname = 0x0, pc = 0, sp = 0, ra = 0, curframe = 0, frame = 0, > prevsp = 0, > prevpc = 0, lastsp = 0, task = 0, debug = 0, hookp = 0x0} > > ---> member_offset = -1 > > (gdb) n > 4937 if (!req->typecode) { > (gdb) n > 4942 if (!req->typecode) { > (gdb) n > 4947 member_typecode = TYPE_CODE_UNDEF; > (gdb) n > 4948 member_size = 0; > (gdb) n > 4949 type_found = 0; > (gdb) n > 4951 if (CRASHDEBUG(2)) { > (gdb) n > 4965 switch (req->typecode) > (gdb) n > 4968 type_found = STRUCT_REQUEST; > (gdb) n > 4969 size = req->length; > (gdb) n > 4970 if (req->member_offset >= 0) { > (gdb) n > 4975 offset = -1; > (gdb) n > 4976 member_size = 0; > > Also crash can't evaluate the offset of the "lru" member: > > crash> struct page > struct page { > long unsigned int flags; > struct address_space *mapping; > struct { > union {...}; > union {...}; > }; > union { > struct list_head lru; > struct {...}; > }; > union { > long unsigned int private; > spinlock_t ptl; > struct kmem_cache *slab; > struct page *first_page; > }; > } > SIZE: 56 > > crash> struct page.mapping > struct page { > [8] struct address_space *mapping; > } > > crash> struct page.lru > struct: invalid data structure reference: page.lru > > Any idea why we do not get the offset from gdb? > Perhaps a problem with the anonymous union? Right -- it's due to commit 49e2258586b423684f03c278149ab46d8f8b6700, which moved the page.lru field into an anonymous structure: /* Third double word block */ - struct list_head lru; /* Pageout list, eg. active_list + union { + struct list_head lru; /* Pageout list, eg. active_list * protected by zone->lru_lock ! */ + struct { /* slub per cpu partial pages */ + struct page *next; /* Next partial slab */ +#ifdef CONFIG_64BIT + int pages; /* Nr of partial slabs left */ + int pobjects; /* Approximate # of objects */ +#else + short int pages; + short int pobjects; +#endif + }; + }; And see https://www.redhat.com/archives/crash-utility/2012-January/msg00023.html for the gdb-related details. Can you try a patch like this to vm_init(): MEMBER_OFFSET_INIT(page_lru, "page", "lru"); + if (INVALID_MEMBER(page_lru)) + ANON_MEMBER_OFFSET_INIT(page_lru, "page", "lru"); Thanks, Dave -- Crash-utility mailing list Crash-utility@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/crash-utility