On 05/01/2014 12:46 AM, Srivatsa S. Bhat wrote:
> On 04/29/2014 03:29 PM, Srivatsa S. Bhat wrote:
>> On 04/29/2014 03:55 AM, Linus Torvalds wrote:
>>> On Mon, Apr 28, 2014 at 3:14 PM, Davidlohr Bueso <davidlohr@xxxxxx> wrote:
>>>>
>>>> I think that returning some stale/bogus vma is causing those segfaults
>>>> in udev. It shouldn't occur in a normal scenario. What puzzles me is
>>>> that it's not always reproducible. This makes me wonder what else is
>>>> going on...
>>>
>>> I've replaced the BUG_ON() with a WARN_ON_ONCE(), and made it be
>>> unconditional (so you don't have to trigger the range check).
>>>
>>> That might make it show up earlier and easier (and hopefully closer to
>>> the place that causes it). Maybe that makes it easier for Srivatsa to
>>> reproduce this. It doesn't make *my* machine do anything different,
>>> though.
>>>
>>> Srivatsa? It's in current -git.
>>>
>>
>> I tried this, but still nothing so far. I rebooted 10-20 times, and also
>> tried multiple runs of multi-threaded ebizzy and kernel compilations,
>> but none of this hit the warning.
>>
>
> I tried to recall the *exact* steps that I had carried out when I first
> hit the bug. I realized that I had actually used kexec to boot the new
> kernel. I had originally booted into a 3.7.7 kernel that happens to be
> on that machine, and then kexec()'ed 3.15-rc3 on it. And that had caused
> the kernel crash. Fresh boots of 3.15-rc3, as well as kexec from 3.15+
> to itself, seems to be pretty robust and has never resulted in any bad
> behavior (this is why I couldn't reproduce the issue earlier, since I was
> doing fresh boots of 3.15-rc).
>
> So I tried the same recipe again (boot into 3.7.7 and kexec into 3.15-rc3+)
> and I got totally random crashes so far, once in sys_kill and two times in
> exit_mmap. So I guess the bug is in 3.7.x and probably 3.15-rc is fine after
> all...
>
>
> Here is the crash around sys_kill:
>
>
And here are the exit_mmap related ones:
1.
mpt2sas0: port enable: SUCCESS
scsi 0:1:0:0: Direct-Access LSI Logical Volume 3000 PQ: 0 ANSI: 6
scsi 0:1:0:0: RAID0: handle(0x00b5), wwid(0x02c5d3368a5aef06), pd_count(1), type(SSP)
scsi 0:1:0:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
scsi 0:0:0:0: Direct-Access IBM-ESXS ST9500620SS BD2C PQ: 0 ANSI: 6
scsi 0:0:0:0: SSP: handle(0x0005), sas_addr(0x5000c500559ffab5), phy(0), device_name(0x5000c500559ffab4)
scsi 0:0:0:0: SSP: enclosure_logical_id(0x5005076056434d90), slot(0)
scsi 0:0:0:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1)
sd 0:1:0:0: [sda] 974608384 512-byte logical blocks: (498 GB/464 GiB)
sd 0:1:0:0: [sda] Write Protect is off
sd 0:1:0:0: [sda] Mode Sense: 03 00 00 08
sd 0:1:0:0: [sda] No Caching mode page found
sd 0:1:0:0: [sda] Assuming drive cache: write through
sda: sda1 sda2 sda3 sda4
sd 0:1:0:0: [sda] Attached SCSI disk
EXT4-fs (sda2): INFO: recovery required on readonly filesystem
EXT4-fs (sda2): write access will be enabled during recovery
random: nonblocking pool is initialized
EXT4-fs (sda2): recovery complete
EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
dracut: Mounted root filesystem /dev/sda2
dracut: Switching root
Welcome to Red Hat Enterprise Linux Server
Starting udev: udev: starting version 147
WARNING! power/level is deprecated; use power/control instead
cat[11602]: segfault at 0 ip (null) sp 00007fff85a583f0 error 14traps: kdump[5307] general protection ip:3c8d22a8db sp:7fff03d1b418 error:0 in libc-2.12.so[3c8d200000+18a000]
in cat[400000+b000]
udevd-work[1304]: '/etc/init.d/kdump restart' unexpected exit with status 0x000b
plymouth[12452]: segfault at 0 ip (null) sp 00007fff49a9e570 error 14 in plymouth[400000+7000]
------------[ cut here ]------------
WARNING: CPU: 13 PID: 12452 at mm/mmap.c:2741 exit_mmap+0x157/0x170()
Modules linked in: acpi_cpufreq(+) ext4(E) jbd2(E) mbcache(E) sd_mod(E) crc_t10dif(E) crct10dif_common(E) mpt2sas(E) scsi_transport_sas(E) raid_class(E)
CPU: 13 PID: 12452 Comm: plymouth Tainted: G E 3.15.0-rc3-mmdbg #1
Hardware name: IBM -[8737R2A]-/00AE502, BIOS -[B2E120QUS-1.20]- 11/14/2012
BUG: Bad page map in process kdump pte:1e00000005f98701 pmd:1031489067
addr:0000003c9e01f000 vm_flags:00100073 anon_vma:ffff88103a654550 mapping:ffff88103e91fb28 index:1f
vma->vm_ops->fault: filemap_fault+0x0/0x450
vma->vm_file->f_op->mmap: ext4_file_mmap+0x0/0x60 [ext4]
0000000000000ab5 ffff88202fe45bb8 ffffffff815a9b38 0000000000000ab5BUG: Bad page map in process kdump pte:1e00000005f98701 pmd:103420b067
addr:0000003c9e01f000 vm_flags:00100073 anon_vma:ffff88102d55ce58 mapping:ffff88103e91fb28 index:1f
vma->vm_ops->fault: filemap_fault+0x0/0x450
vma->vm_file->f_op->mmap: ext4_file_mmap+0x0/0x60 [ext4]
0000000000000000 ffff88202fe45bf8 ffffffff81050f2c 0000000000000000
ffff881032ca0140 0000000000000039 ffff881032ca0140 ffff881032ca01d8
Call Trace:
[<ffffffff815a9b38>] dump_stack+0x51/0x71
[<ffffffff81050f2c>] warn_slowpath_common+0x8c/0xc0
[<ffffffff81050f7a>] warn_slowpath_null+0x1a/0x20
[<ffffffff8116e8e7>] exit_mmap+0x157/0x170
[<ffffffff811ee740>] ? exit_aio+0xb0/0x100
[<ffffffff8104ead3>] mmput+0x73/0x110
[<ffffffff81052434>] exit_mm+0x164/0x1d0
[<ffffffff815af470>] ? _raw_spin_unlock_irq+0x30/0x40
[<ffffffff81052feb>] do_exit+0x15b/0x490
[<ffffffff8105337e>] do_group_exit+0x5e/0xd0
[<ffffffff810654be>] get_signal_to_deliver+0x22e/0x470
[<ffffffff81081c18>] ? finish_task_switch+0x48/0x120
[<ffffffff810039ab>] do_signal+0x4b/0x140
[<ffffffff810a4d9d>] ? trace_hardirqs_on_caller+0xfd/0x1c0
[<ffffffff815a9ab2>] ? printk+0x4d/0x4f
[<ffffffff81081c55>] ? finish_task_switch+0x85/0x120
[<ffffffff81081c18>] ? finish_task_switch+0x48/0x120
[<ffffffff815afb8d>] ? retint_signal+0x11/0x84
[<ffffffff81003b05>] do_notify_resume+0x65/0x90
[<ffffffff812aeb9e>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[<ffffffff815afbc2>] retint_signal+0x46/0x84
---[ end trace b8be17f8a0dd8372 ]---
CPU: 2 PID: 9649 Comm: kdump Tainted: G E 3.15.0-rc3-mmdbg #1
Hardware name: IBM -[8737R2A]-/00AE502, BIOS -[B2E120QUS-1.20]- 11/14/2012
0000003c9e01f000 ffff881031705b40 ffffffff815a9b38 0000000000000001
ffff8810335f28b8 ffff881031705b90 ffffffff811660b3 0000000000000000
ffff88102fe67668 0000000000000000 0000003c9e01f000 0000000000000002
Call Trace:
[<ffffffff815a9b38>] dump_stack+0x51/0x71
[<ffffffff811660b3>] print_bad_pte+0x193/0x260
[<ffffffff811661de>] vm_normal_page+0x5e/0x70
[<ffffffff811693d7>] copy_pte_range+0x217/0x5d0
[<ffffffff81169a0a>] copy_page_range+0x27a/0x4b0
[<ffffffff8104e58f>] dup_mmap+0x24f/0x3f0
[<ffffffff8104ec3c>] dup_mm+0xcc/0x170
[<ffffffff8105009c>] copy_process+0x122c/0x1260
[<ffffffff810a67f4>] ? __lock_release+0x84/0x180
[<ffffffff81050581>] do_fork+0x61/0x220
[<ffffffff811654ef>] ? might_fault+0xaf/0xc0
[<ffffffff811654a6>] ? might_fault+0x66/0xc0
[ 8118] 0 8118 27161 807 17 0 -1000 kdump
/sbin/start_udev[14046] 0 14046 1011 65 7 0 -1000 logger
[ 1003] 0 1003 2854 556 10 0 -1000 udevd
: line 204: 116[ 6046] 0 6046 3084 796 10 0 -1000 udevd
6 Killed [14313] 0 14313 3612 1298 11 0 -1000 udevd
[ 1618] 0 1618 3018 716 10 0 -1000 udevd
/sbin/[13741] 0 13741 1014 65 8 0 -1000 logger
CPU: 12 PID: 8596 Comm: udevd Tainted: G B E 3.15.0-rc3-mmdbg #1
[ 1453] 0 1453 2985 685 10 0 -1000 udevd
Wait timeout. W[14270] 0 14270 4338 159 20 0 -1000 multipath
ill continue in swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
the background.swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
[<ffffffff811a7780>] ? search_binary_handler+0x1c0/0x1c0
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
swap_dup: Bad swap file entry 002a0000
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>