Null dereference in raid10_size, I/O lockup afterwards

[ValdikSS <iam@xxxxxxxxxxxxxxx>]

Hello raid maintainers,

Fedora 39 with 6.10.11-100.fc39 kernel dereferences NULL in raid10_size 
and locks up with 3-drive raid10 configuration upon its degradation and 
reattachment.

How to reproduce:

1. Get 3 USB flash drives
2. mdadm --create -b internal -l 10 -n 3 -z 1G /dev/md0 /dev/sda 
/dev/sdb /dev/sdc
3. Unplug 2 USB drives
4. Plug one of the drive again

Happens every time, every USB flash reattachment.

Fedora includes udev rules which try to assemble the array as soon as 
the device is plugged it, so it may be race condition reproducible only 
on that distro.
Have not tried more recent kernel, but tried it on Debian 12 with kernel 
6.1 and could not reproduce.

Kernel log snippet with decode_stacktrace.sh (full in the attachment):

md/raid10:md127: not enough operational mirrors.
BUG: kernel NULL pointer dereference, address: 0000000000000050
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: Oops: 0000 [#1] PREEMPT SMP NOPTI
Hardware name: Intel(R) Client Systems NUC13ANKi5/NUC13ANBi5, BIOS 
ANRPL357.0033.2024.0716.1113 07/16/2024
RIP: 0010:raid10_size 
(/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/drivers/md/raid10.c:3768) 
raid10
usb-storage 4-1.2:1.0: USB Mass Storage device detected
scsi host3: usb-storage 4-1.2:1.0
scsi 3:0:0:0: Direct-Access     ASolid   USB                   PQ: 0 ANSI: 6
sd 3:0:0:0: Attached scsi generic sg1 type 0
sd 3:0:0:0: [sdb] 122880001 512-byte logical blocks: (62.9 GB/58.6 GiB)
sd 3:0:0:0: [sdb] Write Protect is off
sd 3:0:0:0: [sdb] Mode Sense: 23 00 00 00
sd 3:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
sd 3:0:0:0: [sdb] Attached SCSI removable disk
md/raid10:md127: not enough operational mirrors.
BUG: kernel NULL pointer dereference, address: 0000000000000050
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: Oops: 0000 [#1] PREEMPT SMP NOPTI
Hardware name: Intel(R) Client Systems NUC13ANKi5/NUC13ANBi5, BIOS ANRPL357.0033.2024.0716.1113 07/16/2024
RIP: 0010:raid10_size (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/drivers/md/raid10.c:3768) raid10
RSP: 0018:ffffc01d05097d10 EFLAGS: 00010246
RAX: ffffffffc2248350 RBX: 0000000000000000 RCX: 0000000080200013
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9af282f0a000
RBP: ffff9af282f0a018 R08: ffff9af2402bac00 R09: 0000000080200013
R10: 0000000080200013 R11: 0000000000000000 R12: 0000000000000001
R13: 0000000000000000 R14: ffff9af282f0a018 R15: ffffffffc2262500
FS:  00007fe7c2507e40(0000) GS:ffff9af9b7300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000050 CR3: 00000001ce6dc000 CR4: 0000000000f50ef0
PKRU: 55555554
Call Trace:
<TASK>
? __die (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/arch/x86/kernel/dumpstack.c:421 /usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/arch/x86/kernel/dumpstack.c:434) 
? page_fault_oops (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/arch/x86/mm/fault.c:715 (discriminator 1)) 
? exc_page_fault (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/./arch/x86/include/asm/paravirt.h:693 /usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/arch/x86/mm/fault.c:1489 /usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/arch/x86/mm/fault.c:1539) 
? asm_exc_page_fault (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/./arch/x86/include/asm/idtentry.h:623) 
? __pfx_raid10_size (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/drivers/md/raid10.c:3763) raid10
? raid10_size (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/drivers/md/raid10.c:3768) raid10
md_run (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/drivers/md/md.c:6147) 
do_md_run (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/drivers/md/md.c:6275) 
array_state_store (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/drivers/md/md.c:4568) 
md_attr_store (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/drivers/md/md.c:5730) 
kernfs_fop_write_iter (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/fs/kernfs/file.c:334) 
vfs_write (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/fs/read_write.c:497 /usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/fs/read_write.c:590) 
ksys_write (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/fs/read_write.c:643) 
do_syscall_64 (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/arch/x86/entry/common.c:52 (discriminator 1) /usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/arch/x86/entry/common.c:83 (discriminator 1)) 
? do_syscall_64 (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/./arch/x86/include/asm/cpufeature.h:178 /usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/arch/x86/entry/common.c:98) 
? exc_page_fault (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/./arch/x86/include/asm/paravirt.h:693 /usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/arch/x86/mm/fault.c:1489 /usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/arch/x86/mm/fault.c:1539) 
entry_SYSCALL_64_after_hwframe (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/arch/x86/entry/entry_64.S:130) 
RIP: 0033:0x7fe7c2640ee4
RSP: 002b:00007ffc9f9c7fa8 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007fe7c2640ee4
RDX: 0000000000000009 RSI: 00005579f82dbcc2 RDI: 0000000000000003
RBP: 00007ffc9f9c8050 R08: 0000000000000073 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000202 R12: 00005579f82dbcc2
R13: 0000000000000000 R14: fffff00000000000 R15: 0000557a2e381ab0
</TASK>
Modules linked in: raid10 uas usb_storage uinput tun rfcomm snd_seq_dummy snd_hrtimer xt_pkttype ipt_REJECT xt_addrtype ip6t_REJECT xt_comment xt_owner nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables vboxnetadp(O) vboxnetflt(O) nfnetlink vboxdrv(O) qrtr bnep sunrpc binfmt_misc snd_sof_pci_intel_tgl snd_sof_pci_intel_cnl snd_sof_intel_hda_generic soundwire_intel soundwire_cadence snd_sof_intel_hda_common snd_sof_intel_hda_mlink snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_hda_codec_hdmi snd_sof_utils snd_soc_hdac_hda snd_soc_acpi_intel_match soundwire_generic_allocation snd_soc_acpi soundwire_bus snd_soc_avs snd_soc_hda_codec iwlmvm snd_hda_ext_core snd_soc_core snd_hda_codec_realtek snd_hda_codec_generic snd_compress snd_hda_scodec_component ac97_bus snd_pcm_dmaengine mac80211 intel_rapl_msr
snd_hda_intel intel_rapl_common snd_intel_dspcfg intel_uncore_frequency intel_uncore_frequency_common snd_intel_sdw_acpi snd_hda_codec x86_pkg_temp_thermal intel_powerclamp snd_hda_core coretemp kvm_intel spi_nor snd_hwdep libarc4 mtd mei_hdcp mei_pxp snd_seq ee1004 snd_seq_device kvm btusb btrtl btintel iwlwifi rapl snd_pcm asus_nb_wmi asus_wmi snd_timer btbcm snd btmtk intel_cstate sparse_keymap cfg80211 platform_profile bluetooth wmi_bmof soundcore i2c_i801 spi_intel_pci spi_intel i2c_smbus joydev pcspkr intel_uncore mei_me vfat mei intel_pmc_core rfkill idma64 fat thunderbolt igen6_edac ov13858 intel_vsec v4l2_fwnode pmt_telemetry pmt_class v4l2_async acpi_tad acpi_pad videodev mc tcp_bbr loop zram dm_crypt hid_logitech_hidpp xe drm_ttm_helper gpu_sched drm_suballoc_helper drm_gpuvm drm_exec hid_logitech_dj i915 crct10dif_pclmul i2c_algo_bit crc32_pclmul drm_buddy crc32c_intel ttm polyval_clmulni polyval_generic nvme drm_display_helper ghash_clmulni_intel nvme_core ucsi_acpi igc sha512_ssse3
sha256_ssse3 typec_ucsi cec sha1_ssse3 wdat_wdt typec nvme_auth video wmi pinctrl_tigerlake ip6_tables ip_tables fuse i2c_dev
CR2: 0000000000000050
---[ end trace 0000000000000000 ]---
RIP: 0010:raid10_size (/usr/src/debug/kernel-6.10.11/linux-6.10.11-100.fc39.x86_64/drivers/md/raid10.c:3768) raid10
RSP: 0018:ffffc01d05097d10 EFLAGS: 00010246
RAX: ffffffffc2248350 RBX: 0000000000000000 RCX: 0000000080200013
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9af282f0a000
RBP: ffff9af282f0a018 R08: ffff9af2402bac00 R09: 0000000080200013
R10: 0000000080200013 R11: 0000000000000000 R12: 0000000000000001
R13: 0000000000000000 R14: ffff9af282f0a018 R15: ffffffffc2262500
FS:  00007fe7c2507e40(0000) GS:ffff9af9b7300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000050 CR3: 00000001ce6dc000 CR4: 0000000000f50ef0
PKRU: 55555554
note: mdadm[3474] exited with irqs disabled

usb-storage 4-1.2:1.0: USB Mass Storage device detected
scsi host3: usb-storage 4-1.2:1.0
scsi 3:0:0:0: Direct-Access     ASolid   USB                   PQ: 0 ANSI: 6
sd 3:0:0:0: Attached scsi generic sg1 type 0
sd 3:0:0:0: [sdb] 122880001 512-byte logical blocks: (62.9 GB/58.6 GiB)
sd 3:0:0:0: [sdb] Write Protect is off
sd 3:0:0:0: [sdb] Mode Sense: 23 00 00 00
sd 3:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
sd 3:0:0:0: [sdb] Attached SCSI removable disk
md/raid10:md127: not enough operational mirrors.
BUG: kernel NULL pointer dereference, address: 0000000000000050
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: Oops: 0000 [#1] PREEMPT SMP NOPTI
CPU: 2 PID: 3474 Comm: mdadm Tainted: G           O       6.10.11-100.fc39.x86_64 #1
Hardware name: Intel(R) Client Systems NUC13ANKi5/NUC13ANBi5, BIOS ANRPL357.0033.2024.0716.1113 07/16/2024
RIP: 0010:raid10_size+0x15/0x70 [raid10]
Code: 00 00 0f 1f 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 0f 1f 44 00 00 53 48 83 ec 08 48 8b 1f 85 d2 75 0b <8b> 53 50 8b 43 28 39 c2 0f 4f d0 48 85 f6 75 07 48 8b b3 80 00 00
RSP: 0018:ffffc01d05097d10 EFLAGS: 00010246
RAX: ffffffffc2248350 RBX: 0000000000000000 RCX: 0000000080200013
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9af282f0a000
RBP: ffff9af282f0a018 R08: ffff9af2402bac00 R09: 0000000080200013
R10: 0000000080200013 R11: 0000000000000000 R12: 0000000000000001
R13: 0000000000000000 R14: ffff9af282f0a018 R15: ffffffffc2262500
FS:  00007fe7c2507e40(0000) GS:ffff9af9b7300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000050 CR3: 00000001ce6dc000 CR4: 0000000000f50ef0
PKRU: 55555554
Call Trace:
 <TASK>
 ? __die+0x23/0x70
 ? page_fault_oops+0x173/0x5c0
 ? exc_page_fault+0x7e/0x180
 ? asm_exc_page_fault+0x26/0x30
 ? __pfx_raid10_size+0x10/0x10 [raid10]
 ? raid10_size+0x15/0x70 [raid10]
 md_run+0x5c7/0xcb0
 do_md_run+0x18/0x110
 array_state_store+0x37e/0x450
 md_attr_store+0x83/0x100
 kernfs_fop_write_iter+0x133/0x1d0
 vfs_write+0x291/0x460
 ksys_write+0x6f/0xf0
 do_syscall_64+0x82/0x160
 ? do_syscall_64+0x8e/0x160
 ? exc_page_fault+0x7e/0x180
 entry_SYSCALL_64_after_hwframe+0x76/0x7e
RIP: 0033:0x7fe7c2640ee4
Code: c7 00 16 00 00 00 b8 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 80 3d 85 74 0d 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 55 48 89 e5 48 83 ec 20 48 89
RSP: 002b:00007ffc9f9c7fa8 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007fe7c2640ee4
RDX: 0000000000000009 RSI: 00005579f82dbcc2 RDI: 0000000000000003
RBP: 00007ffc9f9c8050 R08: 0000000000000073 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000202 R12: 00005579f82dbcc2
R13: 0000000000000000 R14: fffff00000000000 R15: 0000557a2e381ab0
 </TASK>
Modules linked in: raid10 uas usb_storage uinput tun rfcomm snd_seq_dummy snd_hrtimer xt_pkttype ipt_REJECT xt_addrtype ip6t_REJECT xt_comment xt_owner nft_compat nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables vboxnetadp(O) vboxnetflt(O) nfnetlink vboxdrv(O) qrtr bnep sunrpc binfmt_misc snd_sof_pci_intel_tgl snd_sof_pci_intel_cnl snd_sof_intel_hda_generic soundwire_intel soundwire_cadence snd_sof_intel_hda_common snd_sof_intel_hda_mlink snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_hda_codec_hdmi snd_sof_utils snd_soc_hdac_hda snd_soc_acpi_intel_match soundwire_generic_allocation snd_soc_acpi soundwire_bus snd_soc_avs snd_soc_hda_codec iwlmvm snd_hda_ext_core snd_soc_core snd_hda_codec_realtek snd_hda_codec_generic snd_compress snd_hda_scodec_component ac97_bus snd_pcm_dmaengine mac80211 intel_rapl_msr
 snd_hda_intel intel_rapl_common snd_intel_dspcfg intel_uncore_frequency intel_uncore_frequency_common snd_intel_sdw_acpi snd_hda_codec x86_pkg_temp_thermal intel_powerclamp snd_hda_core coretemp kvm_intel spi_nor snd_hwdep libarc4 mtd mei_hdcp mei_pxp snd_seq ee1004 snd_seq_device kvm btusb btrtl btintel iwlwifi rapl snd_pcm asus_nb_wmi asus_wmi snd_timer btbcm snd btmtk intel_cstate sparse_keymap cfg80211 platform_profile bluetooth wmi_bmof soundcore i2c_i801 spi_intel_pci spi_intel i2c_smbus joydev pcspkr intel_uncore mei_me vfat mei intel_pmc_core rfkill idma64 fat thunderbolt igen6_edac ov13858 intel_vsec v4l2_fwnode pmt_telemetry pmt_class v4l2_async acpi_tad acpi_pad videodev mc tcp_bbr loop zram dm_crypt hid_logitech_hidpp xe drm_ttm_helper gpu_sched drm_suballoc_helper drm_gpuvm drm_exec hid_logitech_dj i915 crct10dif_pclmul i2c_algo_bit crc32_pclmul drm_buddy crc32c_intel ttm polyval_clmulni polyval_generic nvme drm_display_helper ghash_clmulni_intel nvme_core ucsi_acpi igc sha512_ssse3
 sha256_ssse3 typec_ucsi cec sha1_ssse3 wdat_wdt typec nvme_auth video wmi pinctrl_tigerlake ip6_tables ip_tables fuse i2c_dev
CR2: 0000000000000050
---[ end trace 0000000000000000 ]---
RIP: 0010:raid10_size+0x15/0x70 [raid10]
Code: 00 00 0f 1f 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 0f 1f 44 00 00 53 48 83 ec 08 48 8b 1f 85 d2 75 0b <8b> 53 50 8b 43 28 39 c2 0f 4f d0 48 85 f6 75 07 48 8b b3 80 00 00
RSP: 0018:ffffc01d05097d10 EFLAGS: 00010246
RAX: ffffffffc2248350 RBX: 0000000000000000 RCX: 0000000080200013
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9af282f0a000
RBP: ffff9af282f0a018 R08: ffff9af2402bac00 R09: 0000000080200013
R10: 0000000080200013 R11: 0000000000000000 R12: 0000000000000001
R13: 0000000000000000 R14: ffff9af282f0a018 R15: ffffffffc2262500
FS:  00007fe7c2507e40(0000) GS:ffff9af9b7300000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000050 CR3: 00000001ce6dc000 CR4: 0000000000f50ef0
PKRU: 55555554
note: mdadm[3474] exited with irqs disabled

Attachment: OpenPGP_signature.asc
Description: OpenPGP digital signature