On Mon, Dec 12, 2011 at 7:42 AM, NeilBrown <neilb@xxxxxxx> wrote:
> On Mon, 12 Dec 2011 07:22:17 +0200 Anssi Hannula <anssi.hannula@xxxxxx> wrote:
>
>> On Mon, Dec 12, 2011 at 5:01 AM, NeilBrown <neilb@xxxxxxx> wrote:
>> > On Sun, 11 Dec 2011 09:03:14 +0200 Anssi Hannula <anssi.hannula@xxxxxx> wrote:
>> >
>> >> Hi!
>> >>
>> >> After I rebooted during a raid6 rebuild, the rebuild didn't start again.
>> >> Instead, there is a flood of "RAID conf printout"s that seemingly happen
>> >> on array activity.
>> >>
>> >> All the devices show up properly in --detail and two devices are marked
>> >> as "spare rebuilding", and I can access the contents of the array just
>> >> fine, but the rebuild doesn't actually start. Is this a bug or am I
>> >> missing something? :)
>> >>
>> >> I was initially on 2.6.38.8, but also tried 3.1.4 which seems to have
>> >> the same issue. mdadm is 3.1.5.
>> >>
>> >> I'm not using start_ro and writing to the array doesn't trigger a
>> >> rebuild either.
>> >>
>> >> Attached are --examine outputs before assembly, kernel log output on
>> >> assembly, /proc/mdstat and --detail after assembly (on 3.1.4).
>> >>
>> >
>> > Thank you for the very detailed problem report.
>>
>> Thanks for the quick response :)
>>
>> > Unfortunately it is a complete mystery to me what is happening.
>> >
>> > The repeated "RAID conf printout" messages are almost certainly coming from
>> > the end of raid5_remove_disk.
>> > It is being called from remove_and_add_spares for each of the two devices
>> > that are being rebuilt. raid5_remove_disk declines to remove them because it
>> > can keep rebuilding them.
>> >
>> > remove_and_add_spares then counts them and notes there are 2.
>> > md_check_recovery notes that this is > 0, so it should create a thread to run
>> > md_do_sync.
>> >
>> > md_do_sync should then print out a message like
>> > md: recovery of RAID array md0
>> >
>> > but it doesn't. So something went wrong.
>> > There are three reasons that md_do_sync might not print a message:
>> >
>> > 1/ MD_RECOVERY_DONE is set. As only md_do_sync ever sets it, that is
>> > unlikely, and in any case md_check_recovery clears it.
>> > 2/ mddev->ro != 0. It is only ever set to 0, 1, or 2. If it is 1 or 2
>> > then we would be able to see that in /proc/mdstat as a "(readonly)"
>> > status. But we don't.
>> > 3/ MD_RECOVERY_INTR is set. Again, md_check_recovery clears this. It does
>> > get set if kthread_should_stop() returns 'true', but that should only
>> > happen if kthread_stop() was called. That is only called by
>> > md_unregister_thread and I cannot see any way that could be call.
>> >
>> > So. No idea.
>> >
>> > Are you compiling these kernels yourself?
>>
>> Nope (used Mageia kernels), but I did now (3.1.5).
>>
>> > If so, could you:
>> > - put a printk in the top of md_do_sync to report the values of
>> > mddev->recovery and mddev->ro
>> > - print a message whenever md_unregister_thread is called
>> > - in md_check_recovery, in the
>> > if (mddev->ro) {
>> > /* Only thing we do on a ro array is remove
>> > * failed devices.
>> > */
>> > mdk_rdev_t *rdev;
>> >
>> > in statement, print the value of mddev->ro.
>> >
>> > Then see which of those printk's fire, and what they tell us.
>>
>> Only the last one does, and mddev->ro == 0.
>>
>> For reference, attached is the used patch and resulting log output.
>>
>
> Thanks.
>
> So it isn't running md_do_sync at all. Odd.
>
> Could please add:
> - call "WARN_ON(1);" in print_raid5_conf() so we get a stack trace and can
> see who is calling it.
> - print the value that remove_and_add_spares is going to return.
Attached. As you can see, remove_and_add_spare returns 0.
--
Anssi Hannula
[ 44.270890] md: md0 stopped.
[ 44.276951] md: bind<sdq1>
[ 44.277373] md: bind<sdo1>
[ 44.277504] md: bind<sdh1>
[ 44.278088] md: bind<sdk1>
[ 44.278816] md: bind<sdp1>
[ 44.282215] md: bind<sdn1>
[ 44.282377] md: bind<sdf1>
[ 44.282558] md: bind<sdm1>
[ 44.282741] md: bind<sdl1>
[ 44.282948] md: bind<sdi1>
[ 44.283085] md: bind<sdg1>
[ 44.283249] md: bind<sdd1>
[ 44.283449] md: bind<sdj1>
[ 44.288946] async_tx: api initialized (async)
[ 44.465572] raid6: int64x1 2753 MB/s
[ 44.637492] raid6: int64x2 3533 MB/s
[ 44.805245] raid6: int64x4 2396 MB/s
[ 44.975134] raid6: int64x8 2491 MB/s
[ 45.146107] raid6: sse2x1 7311 MB/s
[ 45.314810] raid6: sse2x2 8670 MB/s
[ 45.484662] raid6: sse2x4 9630 MB/s
[ 45.484665] raid6: using algorithm sse2x4 (9630 MB/s)
[ 45.491202] xor: automatically using best checksumming function: generic_sse
[ 45.534618] generic_sse: 12819.200 MB/sec
[ 45.534623] xor: using function: generic_sse (12819.200 MB/sec)
[ 45.543401] md: raid6 personality registered for level 6
[ 45.543402] md: raid5 personality registered for level 5
[ 45.543403] md: raid4 personality registered for level 4
[ 45.543961] md/raid:md0: device sdj1 operational as raid disk 0
[ 45.543963] md/raid:md0: device sdl1 operational as raid disk 9
[ 45.543964] md/raid:md0: device sdm1 operational as raid disk 8
[ 45.543966] md/raid:md0: device sdn1 operational as raid disk 6
[ 45.543967] md/raid:md0: device sdp1 operational as raid disk 5
[ 45.543968] md/raid:md0: device sdk1 operational as raid disk 4
[ 45.543969] md/raid:md0: device sdo1 operational as raid disk 2
[ 45.543970] md/raid:md0: device sdq1 operational as raid disk 1
[ 45.544667] md/raid:md0: allocated 10592kB
[ 45.544806] md/raid:md0: raid level 6 active with 8 out of 10 devices, algorithm 2
[ 45.544907] ------------[ cut here ]------------
[ 45.544913] WARNING: at drivers/md/raid5.c:5030 print_raid5_conf+0x37/0x110 [raid456]()
[ 45.544920] Hardware name: System Product Name
[ 45.544922] Modules linked in: raid456 async_pq async_xor xor async_memcpy async_raid6_recov raid6_pq async_tx coretemp cpufreq_ondemand cpufreq_conservative cpufreq_powersave acpi_cpufreq freq_table mperf pcspkr nvram netconsole configfs kvm_intel kvm snd_hda_codec_hdmi tda18271c2dd joydev ir_lirc_codec rc_imon_pad lirc_dev ir_mce_kbd_decoder ir_sony_decoder ir_jvc_decoder uas ir_rc6_decoder ir_rc5_decoder imon ir_nec_decoder usb_storage rc_core snd_hda_codec_realtek snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_hda_intel snd_hda_codec snd_hwdep snd_pcm i2c_i801 drxk snd_timer snd_mixer_oss e1000e ddbridge evdev mxm_wmi snd dvb_core firewire_ohci sr_mod serio_raw iTCO_wdt cxd2099(C) firewire_core soundcore iTCO_vendor_support i2c_core i7core_edac crc_itu_t snd_page_alloc edac_core asus_atk0110 wmi processor button sg xhci_hcd ide_generic ide_core mptsas mptscsih mptbase mpt2sas scsi_transport_sas raid_class ata_generic pata_jmicron cciss pata_amd sata_sil24 sata_sil sata_via dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod shpchp pci_hotplug ata_piix ahci libahci libata sd_mod scsi_mod crc_t10dif raid1 ext4 jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd usbhid hid usbcore [last unloaded: scsi_wait_scan]
[ 45.544994] Pid: 848, comm: mdadm Tainted: G C 3.1.5 #5
[ 45.544995] Call Trace:
[ 45.545000] [<ffffffff8106487f>] warn_slowpath_common+0x7f/0xc0
[ 45.545002] [<ffffffff810648da>] warn_slowpath_null+0x1a/0x20
[ 45.545004] [<ffffffffa0675607>] print_raid5_conf+0x37/0x110 [raid456]
[ 45.545007] [<ffffffffa067e2ba>] run+0x2aa/0x750 [raid456]
[ 45.545010] [<ffffffff811908a1>] ? bioset_create+0x1c1/0x280
[ 45.545012] [<ffffffff8133f3eb>] md_run+0x36b/0x8a0
[ 45.545014] [<ffffffff8134060e>] ? add_new_disk+0x2ee/0x550
[ 45.545016] [<ffffffff8133f93e>] do_md_run+0x1e/0xd0
[ 45.545017] [<ffffffff81341223>] md_ioctl+0x9b3/0x1230
[ 45.545021] [<ffffffff8121d260>] blkdev_ioctl+0x230/0x740
[ 45.545023] [<ffffffff81191a31>] block_ioctl+0x41/0x50
[ 45.545026] [<ffffffff8116f867>] do_vfs_ioctl+0x97/0x4f0
[ 45.545028] [<ffffffff8116fd51>] sys_ioctl+0x91/0xa0
[ 45.545031] [<ffffffff8143cd92>] system_call_fastpath+0x16/0x1b
[ 45.545032] ---[ end trace 8bf0f462f72ecc2c ]---
[ 45.545033] RAID conf printout:
[ 45.545034] --- level:6 rd:10 wd:8
[ 45.545035] disk 0, o:1, dev:sdj1
[ 45.545036] disk 1, o:1, dev:sdq1
[ 45.545037] disk 2, o:1, dev:sdo1
[ 45.545038] disk 3, o:1, dev:sdh1
[ 45.545039] disk 4, o:1, dev:sdk1
[ 45.545040] disk 5, o:1, dev:sdp1
[ 45.545041] disk 6, o:1, dev:sdn1
[ 45.545042] disk 7, o:1, dev:sdf1
[ 45.545043] disk 8, o:1, dev:sdm1
[ 45.545044] disk 9, o:1, dev:sdl1
[ 45.545192] created bitmap (15 pages) for device md0
[ 45.967725] md0: bitmap initialized from disk: read 1/1 pages, set 258 of 29809 bits
[ 46.420406] md0: detected capacity change from 0 to 16003169779712
[ 46.421264] md0: in md_check_recovery(), mddev->ro = 0
[ 46.421270] ------------[ cut here ]------------
[ 46.421279] WARNING: at drivers/md/raid5.c:5030 print_raid5_conf+0x37/0x110 [raid456]()
[ 46.421283] Hardware name: System Product Name
[ 46.421285] Modules linked in: raid456 async_pq async_xor xor async_memcpy async_raid6_recov raid6_pq async_tx coretemp cpufreq_ondemand cpufreq_conservative cpufreq_powersave acpi_cpufreq freq_table mperf pcspkr nvram netconsole configfs kvm_intel kvm snd_hda_codec_hdmi tda18271c2dd joydev ir_lirc_codec rc_imon_pad lirc_dev ir_mce_kbd_decoder ir_sony_decoder ir_jvc_decoder uas ir_rc6_decoder ir_rc5_decoder imon ir_nec_decoder usb_storage rc_core snd_hda_codec_realtek snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_hda_intel snd_hda_codec snd_hwdep snd_pcm i2c_i801 drxk snd_timer snd_mixer_oss e1000e ddbridge evdev mxm_wmi snd dvb_core firewire_ohci sr_mod serio_raw iTCO_wdt cxd2099(C) firewire_core soundcore iTCO_vendor_support i2c_core i7core_edac crc_itu_t snd_page_alloc edac_core asus_atk0110 wmi processor button sg xhci_hcd ide_generic ide_core mptsas mptscsih mptbase mpt2sas scsi_transport_sas raid_class ata_generic pata_jmicron cciss pata_amd sata_sil24 sata_sil sata_via dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod shpchp pci_hotplug ata_piix ahci libahci libata sd_mod scsi_mod crc_t10dif raid1 ext4 jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd usbhid hid usbcore [last unloaded: scsi_wait_scan]
[ 46.421387] Pid: 912, comm: md0_raid6 Tainted: G WC 3.1.5 #5
[ 46.421389] Call Trace:
[ 46.421396] [<ffffffff8106487f>] warn_slowpath_common+0x7f/0xc0
[ 46.421398] [<ffffffff810648da>] warn_slowpath_null+0x1a/0x20
[ 46.421402] [<ffffffffa0675607>] print_raid5_conf+0x37/0x110 [raid456]
[ 46.421405] [<ffffffffa0675ff7>] raid5_remove_disk+0x47/0x120 [raid456]
[ 46.421408] [<ffffffff8133dd45>] md_check_recovery+0x3c5/0x690
[ 46.421412] [<ffffffffa067be63>] raid5d+0x43/0x780 [raid456]
[ 46.421415] [<ffffffff810749ea>] ? del_timer_sync+0x3a/0x60
[ 46.421419] [<ffffffff81433057>] ? schedule_timeout+0x177/0x2e0
[ 46.421422] [<ffffffff81073660>] ? cascade+0xa0/0xa0
[ 46.421426] [<ffffffff81338f96>] md_thread+0x116/0x150
[ 46.421430] [<ffffffff81086f40>] ? wake_up_bit+0x40/0x40
[ 46.421433] [<ffffffff81338e80>] ? md_rdev_init+0x130/0x130
[ 46.421435] [<ffffffff810869a6>] kthread+0x96/0xa0
[ 46.421439] [<ffffffff8143eef4>] kernel_thread_helper+0x4/0x10
[ 46.421442] [<ffffffff81086910>] ? kthread_worker_fn+0x1a0/0x1a0
[ 46.421444] [<ffffffff8143eef0>] ? gs_change+0x13/0x13
[ 46.421446] ---[ end trace 8bf0f462f72ecc2d ]---
[ 46.421447] RAID conf printout:
[ 46.421449] --- level:6 rd:10 wd:8
[ 46.421451] disk 0, o:1, dev:sdj1
[ 46.421453] disk 1, o:1, dev:sdq1
[ 46.421454] disk 2, o:1, dev:sdo1
[ 46.421456] disk 3, o:1, dev:sdh1
[ 46.421457] disk 4, o:1, dev:sdk1
[ 46.421459] disk 5, o:1, dev:sdp1
[ 46.421461] disk 6, o:1, dev:sdn1
[ 46.421463] disk 7, o:1, dev:sdf1
[ 46.421464] disk 8, o:1, dev:sdm1
[ 46.421466] disk 9, o:1, dev:sdl1
[ 46.421467] ------------[ cut here ]------------
[ 46.421471] WARNING: at drivers/md/raid5.c:5030 print_raid5_conf+0x37/0x110 [raid456]()
[ 46.421473] Hardware name: System Product Name
[ 46.421474] Modules linked in: raid456 async_pq async_xor xor async_memcpy async_raid6_recov raid6_pq async_tx coretemp cpufreq_ondemand cpufreq_conservative cpufreq_powersave acpi_cpufreq freq_table mperf pcspkr nvram netconsole configfs kvm_intel kvm snd_hda_codec_hdmi tda18271c2dd joydev ir_lirc_codec rc_imon_pad lirc_dev ir_mce_kbd_decoder ir_sony_decoder ir_jvc_decoder uas ir_rc6_decoder ir_rc5_decoder imon ir_nec_decoder usb_storage rc_core snd_hda_codec_realtek snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_hda_intel snd_hda_codec snd_hwdep snd_pcm i2c_i801 drxk snd_timer snd_mixer_oss e1000e ddbridge evdev mxm_wmi snd dvb_core firewire_ohci sr_mod serio_raw iTCO_wdt cxd2099(C) firewire_core soundcore iTCO_vendor_support i2c_core i7core_edac crc_itu_t snd_page_alloc edac_core asus_atk0110 wmi processor button sg xhci_hcd ide_generic ide_core mptsas mptscsih mptbase mpt2sas scsi_transport_sas raid_class ata_generic pata_jmicron cciss pata_amd sata_sil24 sata_sil sata_via dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod shpchp pci_hotplug ata_piix ahci libahci libata sd_mod scsi_mod crc_t10dif raid1 ext4 jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd usbhid hid usbcore [last unloaded: scsi_wait_scan]
[ 46.421531] Pid: 912, comm: md0_raid6 Tainted: G WC 3.1.5 #5
[ 46.421532] Call Trace:
[ 46.421535] [<ffffffff8106487f>] warn_slowpath_common+0x7f/0xc0
[ 46.421537] [<ffffffff810648da>] warn_slowpath_null+0x1a/0x20
[ 46.421540] [<ffffffffa0675607>] print_raid5_conf+0x37/0x110 [raid456]
[ 46.421543] [<ffffffffa067600c>] raid5_remove_disk+0x5c/0x120 [raid456]
[ 46.421546] [<ffffffff8133dd45>] md_check_recovery+0x3c5/0x690
[ 46.421549] [<ffffffffa067be63>] raid5d+0x43/0x780 [raid456]
[ 46.421551] [<ffffffff810749ea>] ? del_timer_sync+0x3a/0x60
[ 46.421554] [<ffffffff81433057>] ? schedule_timeout+0x177/0x2e0
[ 46.421559] [<ffffffff81073660>] ? cascade+0xa0/0xa0
[ 46.421562] [<ffffffff81338f96>] md_thread+0x116/0x150
[ 46.421564] [<ffffffff81086f40>] ? wake_up_bit+0x40/0x40
[ 46.421567] [<ffffffff81338e80>] ? md_rdev_init+0x130/0x130
[ 46.421569] [<ffffffff810869a6>] kthread+0x96/0xa0
[ 46.421572] [<ffffffff8143eef4>] kernel_thread_helper+0x4/0x10
[ 46.421574] [<ffffffff81086910>] ? kthread_worker_fn+0x1a0/0x1a0
[ 46.421577] [<ffffffff8143eef0>] ? gs_change+0x13/0x13
[ 46.421578] ---[ end trace 8bf0f462f72ecc2e ]---
[ 46.421579] RAID conf printout:
[ 46.421581] --- level:6 rd:10 wd:8
[ 46.421582] disk 0, o:1, dev:sdj1
[ 46.421583] disk 1, o:1, dev:sdq1
[ 46.421585] disk 2, o:1, dev:sdo1
[ 46.421586] disk 3, o:1, dev:sdh1
[ 46.421588] disk 4, o:1, dev:sdk1
[ 46.421589] disk 5, o:1, dev:sdp1
[ 46.421590] disk 6, o:1, dev:sdn1
[ 46.421592] disk 7, o:1, dev:sdf1
[ 46.421593] disk 8, o:1, dev:sdm1
[ 46.421594] disk 9, o:1, dev:sdl1
[ 46.421596] ------------[ cut here ]------------
[ 46.421598] WARNING: at drivers/md/raid5.c:5030 print_raid5_conf+0x37/0x110 [raid456]()
[ 46.421600] Hardware name: System Product Name
[ 46.421601] Modules linked in: raid456 async_pq async_xor xor async_memcpy async_raid6_recov raid6_pq async_tx coretemp cpufreq_ondemand cpufreq_conservative cpufreq_powersave acpi_cpufreq freq_table mperf pcspkr nvram netconsole configfs kvm_intel kvm snd_hda_codec_hdmi tda18271c2dd joydev ir_lirc_codec rc_imon_pad lirc_dev ir_mce_kbd_decoder ir_sony_decoder ir_jvc_decoder uas ir_rc6_decoder ir_rc5_decoder imon ir_nec_decoder usb_storage rc_core snd_hda_codec_realtek snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_hda_intel snd_hda_codec snd_hwdep snd_pcm i2c_i801 drxk snd_timer snd_mixer_oss e1000e ddbridge evdev mxm_wmi snd dvb_core firewire_ohci sr_mod serio_raw iTCO_wdt cxd2099(C) firewire_core soundcore iTCO_vendor_support i2c_core i7core_edac crc_itu_t snd_page_alloc edac_core asus_atk0110 wmi processor button sg xhci_hcd ide_generic ide_core mptsas mptscsih mptbase mpt2sas scsi_transport_sas raid_class ata_generic pata_jmicron cciss pata_amd sata_sil24 sata_sil sata_via dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod shpchp pci_hotplug ata_piix ahci libahci libata sd_mod scsi_mod crc_t10dif raid1 ext4 jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd usbhid hid usbcore [last unloaded: scsi_wait_scan]
[ 46.421653] Pid: 912, comm: md0_raid6 Tainted: G WC 3.1.5 #5
[ 46.421655] Call Trace:
[ 46.421657] [<ffffffff8106487f>] warn_slowpath_common+0x7f/0xc0
[ 46.421659] [<ffffffff810648da>] warn_slowpath_null+0x1a/0x20
[ 46.421662] [<ffffffffa0675607>] print_raid5_conf+0x37/0x110 [raid456]
[ 46.421665] [<ffffffffa0675ff7>] raid5_remove_disk+0x47/0x120 [raid456]
[ 46.421668] [<ffffffff8133dd45>] md_check_recovery+0x3c5/0x690
[ 46.421671] [<ffffffffa067be63>] raid5d+0x43/0x780 [raid456]
[ 46.421673] [<ffffffff810749ea>] ? del_timer_sync+0x3a/0x60
[ 46.421676] [<ffffffff81433057>] ? schedule_timeout+0x177/0x2e0
[ 46.421678] [<ffffffff81073660>] ? cascade+0xa0/0xa0
[ 46.421681] [<ffffffff81338f96>] md_thread+0x116/0x150
[ 46.421686] [<ffffffff81086f40>] ? wake_up_bit+0x40/0x40
[ 46.421688] [<ffffffff81338e80>] ? md_rdev_init+0x130/0x130
[ 46.421691] [<ffffffff810869a6>] kthread+0x96/0xa0
[ 46.421693] [<ffffffff8143eef4>] kernel_thread_helper+0x4/0x10
[ 46.421695] [<ffffffff81086910>] ? kthread_worker_fn+0x1a0/0x1a0
[ 46.421698] [<ffffffff8143eef0>] ? gs_change+0x13/0x13
[ 46.421699] ---[ end trace 8bf0f462f72ecc2f ]---
[ 46.421700] RAID conf printout:
[ 46.421701] --- level:6 rd:10 wd:8
[ 46.421703] disk 0, o:1, dev:sdj1
[ 46.421704] disk 1, o:1, dev:sdq1
[ 46.421705] disk 2, o:1, dev:sdo1
[ 46.421707] disk 3, o:1, dev:sdh1
[ 46.421708] disk 4, o:1, dev:sdk1
[ 46.421709] disk 5, o:1, dev:sdp1
[ 46.421711] disk 6, o:1, dev:sdn1
[ 46.421712] disk 7, o:1, dev:sdf1
[ 46.421713] disk 8, o:1, dev:sdm1
[ 46.421715] disk 9, o:1, dev:sdl1
[ 46.421716] ------------[ cut here ]------------
[ 46.421718] WARNING: at drivers/md/raid5.c:5030 print_raid5_conf+0x37/0x110 [raid456]()
[ 46.421720] Hardware name: System Product Name
[ 46.421721] Modules linked in: raid456 async_pq async_xor xor async_memcpy async_raid6_recov raid6_pq async_tx coretemp cpufreq_ondemand cpufreq_conservative cpufreq_powersave acpi_cpufreq freq_table mperf pcspkr nvram netconsole configfs kvm_intel kvm snd_hda_codec_hdmi tda18271c2dd joydev ir_lirc_codec rc_imon_pad lirc_dev ir_mce_kbd_decoder ir_sony_decoder ir_jvc_decoder uas ir_rc6_decoder ir_rc5_decoder imon ir_nec_decoder usb_storage rc_core snd_hda_codec_realtek snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_hda_intel snd_hda_codec snd_hwdep snd_pcm i2c_i801 drxk snd_timer snd_mixer_oss e1000e ddbridge evdev mxm_wmi snd dvb_core firewire_ohci sr_mod serio_raw iTCO_wdt cxd2099(C) firewire_core soundcore iTCO_vendor_support i2c_core i7core_edac crc_itu_t snd_page_alloc edac_core asus_atk0110 wmi processor button sg xhci_hcd ide_generic ide_core mptsas mptscsih mptbase mpt2sas scsi_transport_sas raid_class ata_generic pata_jmicron cciss pata_amd sata_sil24 sata_sil sata_via dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod shpchp pci_hotplug ata_piix ahci libahci libata sd_mod scsi_mod crc_t10dif raid1 ext4 jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd usbhid hid usbcore [last unloaded: scsi_wait_scan]
[ 46.421771] Pid: 912, comm: md0_raid6 Tainted: G WC 3.1.5 #5
[ 46.421772] Call Trace:
[ 46.421774] [<ffffffff8106487f>] warn_slowpath_common+0x7f/0xc0
[ 46.421776] [<ffffffff810648da>] warn_slowpath_null+0x1a/0x20
[ 46.421779] [<ffffffffa0675607>] print_raid5_conf+0x37/0x110 [raid456]
[ 46.421782] [<ffffffffa067600c>] raid5_remove_disk+0x5c/0x120 [raid456]
[ 46.421784] [<ffffffff8133dd45>] md_check_recovery+0x3c5/0x690
[ 46.421788] [<ffffffffa067be63>] raid5d+0x43/0x780 [raid456]
[ 46.421790] [<ffffffff810749ea>] ? del_timer_sync+0x3a/0x60
[ 46.421792] [<ffffffff81433057>] ? schedule_timeout+0x177/0x2e0
[ 46.421795] [<ffffffff81073660>] ? cascade+0xa0/0xa0
[ 46.421797] [<ffffffff81338f96>] md_thread+0x116/0x150
[ 46.421799] [<ffffffff81086f40>] ? wake_up_bit+0x40/0x40
[ 46.421802] [<ffffffff81338e80>] ? md_rdev_init+0x130/0x130
[ 46.421804] [<ffffffff810869a6>] kthread+0x96/0xa0
[ 46.421806] [<ffffffff8143eef4>] kernel_thread_helper+0x4/0x10
[ 46.421809] [<ffffffff81086910>] ? kthread_worker_fn+0x1a0/0x1a0
[ 46.421814] [<ffffffff8143eef0>] ? gs_change+0x13/0x13
[ 46.421815] ---[ end trace 8bf0f462f72ecc30 ]---
[ 46.421816] RAID conf printout:
[ 46.421818] --- level:6 rd:10 wd:8
[ 46.421819] disk 0, o:1, dev:sdj1
[ 46.421821] disk 1, o:1, dev:sdq1
[ 46.421822] disk 2, o:1, dev:sdo1
[ 46.421823] disk 3, o:1, dev:sdh1
[ 46.421825] disk 4, o:1, dev:sdk1
[ 46.421826] disk 5, o:1, dev:sdp1
[ 46.421827] disk 6, o:1, dev:sdn1
[ 46.421829] disk 7, o:1, dev:sdf1
[ 46.421830] disk 8, o:1, dev:sdm1
[ 46.421831] disk 9, o:1, dev:sdl1
[ 46.421833] ------------[ cut here ]------------
[ 46.421835] WARNING: at drivers/md/raid5.c:5030 print_raid5_conf+0x37/0x110 [raid456]()
[ 46.421837] Hardware name: System Product Name
[ 46.421838] Modules linked in: raid456 async_pq async_xor xor async_memcpy async_raid6_recov raid6_pq async_tx coretemp cpufreq_ondemand cpufreq_conservative cpufreq_powersave acpi_cpufreq freq_table mperf pcspkr nvram netconsole configfs kvm_intel kvm snd_hda_codec_hdmi tda18271c2dd joydev ir_lirc_codec rc_imon_pad lirc_dev ir_mce_kbd_decoder ir_sony_decoder ir_jvc_decoder uas ir_rc6_decoder ir_rc5_decoder imon ir_nec_decoder usb_storage rc_core snd_hda_codec_realtek snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_hda_intel snd_hda_codec snd_hwdep snd_pcm i2c_i801 drxk snd_timer snd_mixer_oss e1000e ddbridge evdev mxm_wmi snd dvb_core firewire_ohci sr_mod serio_raw iTCO_wdt cxd2099(C) firewire_core soundcore iTCO_vendor_support i2c_core i7core_edac crc_itu_t snd_page_alloc edac_core asus_atk0110 wmi processor button sg xhci_hcd ide_generic ide_core mptsas mptscsih mptbase mpt2sas scsi_transport_sas raid_class ata_generic pata_jmicron cciss pata_amd sata_sil24 sata_sil sata_via dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod shpchp pci_hotplug ata_piix ahci libahci libata sd_mod scsi_mod crc_t10dif raid1 ext4 jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd usbhid hid usbcore [last unloaded: scsi_wait_scan]
[ 46.421890] Pid: 912, comm: md0_raid6 Tainted: G WC 3.1.5 #5
[ 46.421891] Call Trace:
[ 46.421893] [<ffffffff8106487f>] warn_slowpath_common+0x7f/0xc0
[ 46.421896] [<ffffffff810648da>] warn_slowpath_null+0x1a/0x20
[ 46.421899] [<ffffffffa0675607>] print_raid5_conf+0x37/0x110 [raid456]
[ 46.421902] [<ffffffffa06757b4>] raid5_add_disk+0xd4/0x120 [raid456]
[ 46.421904] [<ffffffff8133dded>] md_check_recovery+0x46d/0x690
[ 46.421907] [<ffffffffa067be63>] raid5d+0x43/0x780 [raid456]
[ 46.421910] [<ffffffff810749ea>] ? del_timer_sync+0x3a/0x60
[ 46.421912] [<ffffffff81433057>] ? schedule_timeout+0x177/0x2e0
[ 46.421915] [<ffffffff81073660>] ? cascade+0xa0/0xa0
[ 46.421917] [<ffffffff81338f96>] md_thread+0x116/0x150
[ 46.421920] [<ffffffff81086f40>] ? wake_up_bit+0x40/0x40
[ 46.421922] [<ffffffff81338e80>] ? md_rdev_init+0x130/0x130
[ 46.421924] [<ffffffff810869a6>] kthread+0x96/0xa0
[ 46.421927] [<ffffffff8143eef4>] kernel_thread_helper+0x4/0x10
[ 46.421929] [<ffffffff81086910>] ? kthread_worker_fn+0x1a0/0x1a0
[ 46.421932] [<ffffffff8143eef0>] ? gs_change+0x13/0x13
[ 46.421933] ---[ end trace 8bf0f462f72ecc31 ]---
[ 46.421934] RAID conf printout:
[ 46.421935] --- level:6 rd:10 wd:8
[ 46.421937] disk 0, o:1, dev:sdj1
[ 46.421938] disk 1, o:1, dev:sdq1
[ 46.421940] disk 2, o:1, dev:sdo1
[ 46.421941] disk 3, o:1, dev:sdh1
[ 46.421942] disk 4, o:1, dev:sdk1
[ 46.421944] disk 5, o:1, dev:sdp1
[ 46.421945] disk 6, o:1, dev:sdn1
[ 46.421946] disk 7, o:1, dev:sdf1
[ 46.421948] disk 8, o:1, dev:sdm1
[ 46.421949] disk 9, o:1, dev:sdl1
[ 46.421950] md0: remove_and_add_spares will return 0
[ 46.434878] md0: unknown partition table
