Hi, we are seeing an issue that can be triggered with relative ease on a server that has been working fine for a few weeks. The regular workload is a backup utility that copies off data from virtual disk images in 4MiB (compressed) chunks from Ceph onto a local NVME-based RAID-6 array that is encrypted using LUKS. Today I started a larger rsync job from another server (that has a couple of million files with around 200-300 gib in total) to migrate data and we’ve seen the server suddenly lock up twice. Any IO that interacts with the mountpoint (/srv/backy) will hang indefinitely. A reset is required to get out of this as the machine will hang trying to unmount the affected filesystem. No other messages than the hung tasks are being presented - I have no indicator for hardware faults at the moment. I’m messaging both dm-devel and linux-raid as I’m suspecting either one or both (or an interaction) might be the cause. Kernel: Linux version 5.15.138 (nixbld@localhost) (gcc (GCC) 12.2.0, GNU ld (GNU Binutils) 2.40) #1-NixOS SMP Wed Nov 8 16:26:52 UTC 2023 See the kernel config attached.
Attachment:
config.gz
Description: GNU Zip compressed data
I can’t say whether there’s an earlier version that worked as this specific config is relatively new: we switched to dm-crypt about half a year ago and haven’t seen any issues. We’ve been using NVMe drives all over the place and we’ve been using MD raid for a long time. MD + dm-crypt has also been running for a while now without this - the new thing we introduced is that the overall combination of MD + dm-crypt + NVMe drives is new.
Here’s the output I’m seeing:
[Aug 6 13:17] INFO: task kworker/u64:4:433 blocked for more than 122 seconds.
[ +0.007829] Not tainted 5.15.138 #1-NixOS
[ +0.005085] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ +0.008801] task:kworker/u64:4 state:D stack: 0 pid: 433 ppid: 2 flags:0x00004000
[ +0.000005] Workqueue: kcryptd/253:4 kcryptd_crypt [dm_crypt]
[ +0.000009] Call Trace:
[ +0.000001] <TASK>
[ +0.000002] __schedule+0x373/0x1580
[ +0.000006] ? sysvec_apic_timer_interrupt+0xa/0x90
[ +0.000002] ? asm_sysvec_apic_timer_interrupt+0x16/0x20
[ +0.000003] schedule+0x5b/0xe0
[ +0.000001] md_bitmap_startwrite+0x177/0x1e0
[ +0.000003] ? finish_wait+0x90/0x90
[ +0.000005] add_stripe_bio+0x449/0x770 [raid456]
[ +0.000005] raid5_make_request+0x1cf/0xbd0 [raid456]
[ +0.000003] ? __bio_clone_fast+0xa5/0xe0
[ +0.000003] ? finish_wait+0x90/0x90
[ +0.000001] ? __blk_queue_split+0x2d0/0x580
[ +0.000003] md_handle_request+0x122/0x1b0
[ +0.000003] md_submit_bio+0x6e/0xb0
[ +0.000001] __submit_bio+0x18f/0x220
[ +0.000002] ? crypt_page_alloc+0x46/0x60 [dm_crypt]
[ +0.000001] submit_bio_noacct+0xbe/0x2d0
[ +0.000002] kcryptd_crypt+0x392/0x550 [dm_crypt]
[ +0.000002] process_one_work+0x1d6/0x360
[ +0.000003] worker_thread+0x4d/0x3b0
[ +0.000001] ? process_one_work+0x360/0x360
[ +0.000001] kthread+0x118/0x140
[ +0.000002] ? set_kthread_struct+0x50/0x50
[ +0.000001] ret_from_fork+0x22/0x30
[ +0.000003] </TASK>
[ +0.000001] INFO: task kworker/u64:6:435 blocked for more than 122 seconds.
[ +0.007823] Not tainted 5.15.138 #1-NixOS
[ +0.005092] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ +0.008799] task:kworker/u64:6 state:D stack: 0 pid: 435 ppid: 2 flags:0x00004000
[ +0.000002] Workqueue: kcryptd/253:4 kcryptd_crypt [dm_crypt]
[ +0.000003] Call Trace:
[ +0.000001] <TASK>
[ +0.000001] __schedule+0x373/0x1580
[ +0.000002] ? sysvec_call_function_single+0xa/0x90
[ +0.000001] ? asm_sysvec_call_function_single+0x16/0x20
[ +0.000002] schedule+0x5b/0xe0
[ +0.000001] md_bitmap_startwrite+0x177/0x1e0
[ +0.000002] ? finish_wait+0x90/0x90
[ +0.000002] add_stripe_bio+0x449/0x770 [raid456]
[ +0.000004] raid5_make_request+0x1cf/0xbd0 [raid456]
[ +0.000002] ? __bio_clone_fast+0xa5/0xe0
[ +0.000002] ? finish_wait+0x90/0x90
[ +0.000001] ? __blk_queue_split+0x2d0/0x580
[ +0.000002] md_handle_request+0x122/0x1b0
[ +0.000001] md_submit_bio+0x6e/0xb0
[ +0.000002] __submit_bio+0x18f/0x220
[ +0.000001] ? crypt_page_alloc+0x46/0x60 [dm_crypt]
[ +0.000001] submit_bio_noacct+0xbe/0x2d0
[ +0.000001] kcryptd_crypt+0x392/0x550 [dm_crypt]
[ +0.000002] process_one_work+0x1d6/0x360
[ +0.000002] worker_thread+0x4d/0x3b0
[ +0.000001] ? process_one_work+0x360/0x360
[ +0.000001] kthread+0x118/0x140
[ +0.000001] ? set_kthread_struct+0x50/0x50
[ +0.000001] ret_from_fork+0x22/0x30
[ +0.000002] </TASK>
[ +0.000000] INFO: task kworker/u64:7:436 blocked for more than 122 seconds.
[ +0.007829] Not tainted 5.15.138 #1-NixOS
[ +0.005095] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ +0.008799] task:kworker/u64:7 state:D stack: 0 pid: 436 ppid: 2 flags:0x00004000
[ +0.000003] Workqueue: kcryptd/253:4 kcryptd_crypt [dm_crypt]
[ +0.000003] Call Trace:
[ +0.000001] <TASK>
[ +0.000001] __schedule+0x373/0x1580
[ +0.000003] ? sysvec_apic_timer_interrupt+0xa/0x90
[ +0.000002] ? asm_sysvec_apic_timer_interrupt+0x16/0x20
[ +0.000002] schedule+0x5b/0xe0
[ +0.000000] md_bitmap_startwrite+0x177/0x1e0
[ +0.000003] ? finish_wait+0x90/0x90
[ +0.000001] add_stripe_bio+0x449/0x770 [raid456]
[ +0.000004] raid5_make_request+0x1cf/0xbd0 [raid456]
[ +0.000002] ? __bio_clone_fast+0xa5/0xe0
[ +0.000002] ? finish_wait+0x90/0x90
[ +0.000002] ? __blk_queue_split+0x2d0/0x580
[ +0.000002] md_handle_request+0x122/0x1b0
[ +0.000002] md_submit_bio+0x6e/0xb0
[ +0.000001] __submit_bio+0x18f/0x220
[ +0.000001] ? crypt_page_alloc+0x46/0x60 [dm_crypt]
[ +0.000002] submit_bio_noacct+0xbe/0x2d0
[ +0.000001] kcryptd_crypt+0x392/0x550 [dm_crypt]
[ +0.000002] process_one_work+0x1d6/0x360
[ +0.000002] worker_thread+0x4d/0x3b0
[ +0.000001] ? process_one_work+0x360/0x360
[ +0.000001] kthread+0x118/0x140
[ +0.000001] ? set_kthread_struct+0x50/0x50
[ +0.000001] ret_from_fork+0x22/0x30
[ +0.000002] </TASK>
[ +0.000001] INFO: task kworker/u64:11:440 blocked for more than 122 seconds.
[ +0.007923] Not tainted 5.15.138 #1-NixOS
[ +0.005093] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ +0.008800] task:kworker/u64:11 state:D stack: 0 pid: 440 ppid: 2 flags:0x00004000
[ +0.000002] Workqueue: kcryptd/253:4 kcryptd_crypt [dm_crypt]
[ +0.000003] Call Trace:
[ +0.000001] <TASK>
[ +0.000001] __schedule+0x373/0x1580
[ +0.000002] schedule+0x5b/0xe0
[ +0.000001] md_bitmap_startwrite+0x177/0x1e0
[ +0.000002] ? finish_wait+0x90/0x90
[ +0.000001] add_stripe_bio+0x449/0x770 [raid456]
[ +0.000004] raid5_make_request+0x1cf/0xbd0 [raid456]
[ +0.000002] ? __bio_clone_fast+0xa5/0xe0
[ +0.000002] ? finish_wait+0x90/0x90
[ +0.000001] ? __blk_queue_split+0x2d0/0x580
[ +0.000003] md_handle_request+0x122/0x1b0
[ +0.000001] md_submit_bio+0x6e/0xb0
[ +0.000002] __submit_bio+0x18f/0x220
[ +0.000001] ? crypt_page_alloc+0x46/0x60 [dm_crypt]
[ +0.000001] submit_bio_noacct+0xbe/0x2d0
[ +0.000001] kcryptd_crypt+0x392/0x550 [dm_crypt]
[ +0.000002] process_one_work+0x1d6/0x360
[ +0.000002] worker_thread+0x4d/0x3b0
[ +0.000001] ? process_one_work+0x360/0x360
[ +0.000001] kthread+0x118/0x140
[ +0.000001] ? set_kthread_struct+0x50/0x50
[ +0.000000] ret_from_fork+0x22/0x30
[ +0.000002] </TASK>
[ +0.000001] INFO: task kworker/u64:13:442 blocked for more than 122 seconds.
[ +0.007919] Not tainted 5.15.138 #1-NixOS
[ +0.005094] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ +0.008799] task:kworker/u64:13 state:D stack: 0 pid: 442 ppid: 2 flags:0x00004000
[ +0.000003] Workqueue: kcryptd/253:4 kcryptd_crypt [dm_crypt]
[ +0.000002] Call Trace:
[ +0.000001] <TASK>
[ +0.000001] __schedule+0x373/0x1580
[ +0.000003] ? sysvec_apic_timer_interrupt+0xa/0x90
[ +0.000002] ? asm_sysvec_apic_timer_interrupt+0x16/0x20
[ +0.000001] schedule+0x5b/0xe0
[ +0.000001] md_bitmap_startwrite+0x177/0x1e0
[ +0.000001] ? finish_wait+0x90/0x90
[ +0.000001] add_stripe_bio+0x449/0x770 [raid456]
[ +0.000004] raid5_make_request+0x1cf/0xbd0 [raid456]
[ +0.000003] ? kmem_cache_alloc_node_trace+0x391/0x3e0
[ +0.000004] ? linear_map+0x44/0x90 [dm_mod]
[ +0.000004] ? finish_wait+0x90/0x90
[ +0.000001] ? __blk_queue_split+0x516/0x580
[ +0.000002] md_handle_request+0x122/0x1b0
[ +0.000002] md_submit_bio+0x6e/0xb0
[ +0.000001] __submit_bio+0x18f/0x220
[ +0.000001] ? crypt_page_alloc+0x46/0x60 [dm_crypt]
[ +0.000001] submit_bio_noacct+0xbe/0x2d0
[ +0.000002] kcryptd_crypt+0x392/0x550 [dm_crypt]
[ +0.000001] process_one_work+0x1d6/0x360
[ +0.000002] worker_thread+0x4d/0x3b0
[ +0.000001] ? process_one_work+0x360/0x360
[ +0.000001] kthread+0x118/0x140
[ +0.000002] ? set_kthread_struct+0x50/0x50
[ +0.000000] ret_from_fork+0x22/0x30
[ +0.000003] </TASK>
[ +0.000004] INFO: task kworker/u64:14:443 blocked for more than 122 seconds.
[ +0.007923] Not tainted 5.15.138 #1-NixOS
[ +0.005096] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ +0.008803] task:kworker/u64:14 state:D stack: 0 pid: 443 ppid: 2 flags:0x00004000
[ +0.000004] Workqueue: kcryptd/253:4 kcryptd_crypt [dm_crypt]
[ +0.000003] Call Trace:
[ +0.000001] <TASK>
[ +0.000001] __schedule+0x373/0x1580
[ +0.000006] schedule+0x5b/0xe0
[ +0.000001] raid5_get_active_stripe+0x50c/0x6c0 [raid456]
[ +0.000004] ? finish_wait+0x90/0x90
[ +0.000002] raid5_make_request+0x18c/0xbd0 [raid456]
[ +0.000002] ? kmem_cache_alloc_node_trace+0x391/0x3e0
[ +0.000002] ? linear_map+0x44/0x90 [dm_mod]
[ +0.000004] ? finish_wait+0x90/0x90
[ +0.000001] ? __blk_queue_split+0x516/0x580
[ +0.000002] md_handle_request+0x122/0x1b0
[ +0.000002] md_submit_bio+0x6e/0xb0
[ +0.000001] __submit_bio+0x18f/0x220
[ +0.000001] ? crypt_page_alloc+0x46/0x60 [dm_crypt]
[ +0.000002] submit_bio_noacct+0xbe/0x2d0
[ +0.000001] kcryptd_crypt+0x392/0x550 [dm_crypt]
[ +0.000002] process_one_work+0x1d6/0x360
[ +0.000001] worker_thread+0x4d/0x3b0
[ +0.000001] ? process_one_work+0x360/0x360
[ +0.000001] kthread+0x118/0x140
[ +0.000001] ? set_kthread_struct+0x50/0x50
[ +0.000001] ret_from_fork+0x22/0x30
[ +0.000002] </TASK>
[ +0.000001] INFO: task kworker/u64:15:444 blocked for more than 123 seconds.
[ +0.007922] Not tainted 5.15.138 #1-NixOS
[ +0.005096] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ +0.008802] task:kworker/u64:15 state:D stack: 0 pid: 444 ppid: 2 flags:0x00004000
[ +0.000003] Workqueue: kcryptd/253:4 kcryptd_crypt [dm_crypt]
[ +0.000002] Call Trace:
[ +0.000001] <TASK>
[ +0.000001] __schedule+0x373/0x1580
[ +0.000003] ? sysvec_apic_timer_interrupt+0xa/0x90
[ +0.000002] ? asm_sysvec_apic_timer_interrupt+0x16/0x20
[ +0.000002] schedule+0x5b/0xe0
[ +0.000001] md_bitmap_startwrite+0x177/0x1e0
[ +0.000002] ? finish_wait+0x90/0x90
[ +0.000002] add_stripe_bio+0x449/0x770 [raid456]
[ +0.000003] raid5_make_request+0x1cf/0xbd0 [raid456]
[ +0.000003] ? kmem_cache_alloc_node_trace+0x391/0x3e0
[ +0.000001] ? linear_map+0x44/0x90 [dm_mod]
[ +0.000003] ? finish_wait+0x90/0x90
[ +0.000001] ? __blk_queue_split+0x516/0x580
[ +0.000003] md_handle_request+0x122/0x1b0
[ +0.000002] md_submit_bio+0x6e/0xb0
[ +0.000001] __submit_bio+0x18f/0x220
[ +0.000001] ? crypt_page_alloc+0x46/0x60 [dm_crypt]
[ +0.000002] submit_bio_noacct+0xbe/0x2d0
[ +0.000001] kcryptd_crypt+0x392/0x550 [dm_crypt]
[ +0.000002] process_one_work+0x1d6/0x360
[ +0.000002] worker_thread+0x4d/0x3b0
[ +0.000001] ? process_one_work+0x360/0x360
[ +0.000001] kthread+0x118/0x140
[ +0.000001] ? set_kthread_struct+0x50/0x50
[ +0.000001] ret_from_fork+0x22/0x30
[ +0.000002] </TASK>
[ +0.000001] INFO: task kworker/u64:17:446 blocked for more than 123 seconds.
[ +0.007922] Not tainted 5.15.138 #1-NixOS
[ +0.005093] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ +0.008800] task:kworker/u64:17 state:D stack: 0 pid: 446 ppid: 2 flags:0x00004000
[ +0.000002] Workqueue: kcryptd/253:4 kcryptd_crypt [dm_crypt]
[ +0.000003] Call Trace:
[ +0.000000] <TASK>
[ +0.000001] __schedule+0x373/0x1580
[ +0.000002] schedule+0x5b/0xe0
[ +0.000001] md_bitmap_startwrite+0x177/0x1e0
[ +0.000001] ? finish_wait+0x90/0x90
[ +0.000002] add_stripe_bio+0x449/0x770 [raid456]
[ +0.000003] raid5_make_request+0x1cf/0xbd0 [raid456]
[ +0.000002] ? kmem_cache_alloc_node_trace+0x391/0x3e0
[ +0.000002] ? linear_map+0x44/0x90 [dm_mod]
[ +0.000004] ? finish_wait+0x90/0x90
[ +0.000001] ? __blk_queue_split+0x516/0x580
[ +0.000002] md_handle_request+0x122/0x1b0
[ +0.000001] md_submit_bio+0x6e/0xb0
[ +0.000002] __submit_bio+0x18f/0x220
[ +0.000001] ? crypt_page_alloc+0x46/0x60 [dm_crypt]
[ +0.000001] submit_bio_noacct+0xbe/0x2d0
[ +0.000001] kcryptd_crypt+0x392/0x550 [dm_crypt]
[ +0.000002] process_one_work+0x1d6/0x360
[ +0.000001] worker_thread+0x4d/0x3b0
[ +0.000001] ? process_one_work+0x360/0x360
[ +0.000001] kthread+0x118/0x140
[ +0.000001] ? set_kthread_struct+0x50/0x50
[ +0.000001] ret_from_fork+0x22/0x30
[ +0.000002] </TASK>
[ +0.000001] INFO: task kworker/u64:18:447 blocked for more than 123 seconds.
[ +0.007919] Not tainted 5.15.138 #1-NixOS
[ +0.005094] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ +0.008803] task:kworker/u64:18 state:D stack: 0 pid: 447 ppid: 2 flags:0x00004000
[ +0.000004] Workqueue: xfs-cil/dm-4 xlog_cil_push_work [xfs]
[ +0.000042] Call Trace:
[ +0.000001] <TASK>
[ +0.000001] __schedule+0x373/0x1580
[ +0.000004] ? dm_submit_bio+0x1e8/0x380 [dm_mod]
[ +0.000004] schedule+0x5b/0xe0
[ +0.000001] xlog_state_get_iclog_space+0x10d/0x320 [xfs]
[ +0.000028] ? wake_up_q+0x90/0x90
[ +0.000002] xlog_write+0x148/0x6c0 [xfs]
[ +0.000024] xlog_cil_push_work+0x37f/0x4c0 [xfs]
[ +0.000025] process_one_work+0x1d6/0x360
[ +0.000001] worker_thread+0x4d/0x3b0
[ +0.000002] ? process_one_work+0x360/0x360
[ +0.000001] kthread+0x118/0x140
[ +0.000001] ? set_kthread_struct+0x50/0x50
[ +0.000001] ret_from_fork+0x22/0x30
[ +0.000002] </TASK>
[ +0.000001] INFO: task kworker/u64:20:449 blocked for more than 123 seconds.
[ +0.007921] Not tainted 5.15.138 #1-NixOS
[ +0.005093] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ +0.008802] task:kworker/u64:20 state:D stack: 0 pid: 449 ppid: 2 flags:0x00004000
[ +0.000003] Workqueue: kcryptd/253:4 kcryptd_crypt [dm_crypt]
[ +0.000003] Call Trace:
[ +0.000001] <TASK>
[ +0.000001] __schedule+0x373/0x1580
[ +0.000003] ? sysvec_apic_timer_interrupt+0xa/0x90
[ +0.000002] ? asm_sysvec_apic_timer_interrupt+0x16/0x20
[ +0.000002] schedule+0x5b/0xe0
[ +0.000001] md_bitmap_startwrite+0x177/0x1e0
[ +0.000002] ? finish_wait+0x90/0x90
[ +0.000003] add_stripe_bio+0x449/0x770 [raid456]
[ +0.000004] raid5_make_request+0x1cf/0xbd0 [raid456]
[ +0.000002] ? kmem_cache_alloc_node_trace+0x391/0x3e0
[ +0.000004] ? linear_map+0x44/0x90 [dm_mod]
[ +0.000003] ? finish_wait+0x90/0x90
[ +0.000001] ? __blk_queue_split+0x516/0x580
[ +0.000003] md_handle_request+0x122/0x1b0
[ +0.000002] md_submit_bio+0x6e/0xb0
[ +0.000001] __submit_bio+0x18f/0x220
[ +0.000001] ? crypt_page_alloc+0x46/0x60 [dm_crypt]
[ +0.000001] submit_bio_noacct+0xbe/0x2d0
[ +0.000002] kcryptd_crypt+0x392/0x550 [dm_crypt]
[ +0.000001] process_one_work+0x1d6/0x360
[ +0.000002] worker_thread+0x4d/0x3b0
[ +0.000001] ? process_one_work+0x360/0x360
[ +0.000001] kthread+0x118/0x140
[ +0.000001] ? set_kthread_struct+0x50/0x50
[ +0.000001] ret_from_fork+0x22/0x30
[ +0.000003] </TASK>
Environment:
CPU:
processor : 0
vendor_id : AuthenticAMD
cpu family : 25
model : 17
model name : AMD EPYC 9124 16-Core Processor
stepping : 1
microcode : 0xa101144
cpu MHz : 3000.000
cache size : 1024 KB
physical id : 0
siblings : 32
core id : 0
cpu cores : 16
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 16
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local avx512_bf16 clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif v_spec_ctrl avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq la57 rdpid overflow_recov succor smca fsrm flush_l1d
bugs : sysret_ss_attrs spectre_v1 spectre_v2 spec_store_bypass srso
bogomips : 6000.04
TLB size : 3584 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 52 bits physical, 57 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]
Modules:
tls 118784 0 - Live 0xffffffffc16af000
dm_crypt 61440 1 - Live 0xffffffffc169f000
cbc 16384 0 - Live 0xffffffffc1697000
encrypted_keys 24576 1 dm_crypt, Live 0xffffffffc168c000
trusted 40960 2 dm_crypt,encrypted_keys, Live 0xffffffffc167c000
asn1_encoder 16384 1 trusted, Live 0xffffffffc1675000
tee 36864 1 trusted, Live 0xffffffffc1666000
tpm 94208 1 trusted, Live 0xffffffffc1646000
af_packet 57344 10 - Live 0xffffffffc161e000
vxlan 81920 0 - Live 0xffffffffc1585000
ip6_udp_tunnel 16384 1 vxlan, Live 0xffffffffc157a000
udp_tunnel 20480 1 vxlan, Live 0xffffffffc1570000
dummy 16384 0 - Live 0xffffffffc1275000
msr 16384 0 - Live 0xffffffffc1188000
ext4 937984 1 - Live 0xffffffffc1476000
crc16 16384 1 ext4, Live 0xffffffffc0f5b000
mbcache 16384 1 ext4, Live 0xffffffffc0ee6000
jbd2 167936 1 ext4, Live 0xffffffffc1146000
edac_mce_amd 36864 0 - Live 0xffffffffc170a000
edac_core 69632 0 - Live 0xffffffffc1253000
intel_rapl_msr 20480 0 - Live 0xffffffffc12ff000
intel_rapl_common 28672 1 intel_rapl_msr, Live 0xffffffffc108c000
kvm_amd 147456 0 - Live 0xffffffffc11e0000
sd_mod 57344 1 - Live 0xffffffffc11d1000
kvm 1056768 1 kvm_amd, Live 0xffffffffc1373000
irqbypass 16384 1 kvm, Live 0xffffffffc1311000
crc32_pclmul 16384 0 - Live 0xffffffffc0f29000
ghash_clmulni_intel 16384 0 - Live 0xffffffffc0f1f000
ipmi_ssif 45056 0 - Live 0xffffffffc12ea000
aesni_intel 380928 2 - Live 0xffffffffc127e000
libaes 16384 1 aesni_intel, Live 0xffffffffc120c000
crypto_simd 16384 1 aesni_intel, Live 0xffffffffc1369000
cryptd 28672 3 ghash_clmulni_intel,crypto_simd, Live 0xffffffffc12f7000
rapl 16384 0 - Live 0xffffffffc0ebb000
ast 57344 0 - Live 0xffffffffc1064000
drm_vram_helper 24576 1 ast, Live 0xffffffffc0ff5000
drm_ttm_helper 16384 2 ast,drm_vram_helper, Live 0xffffffffc0f40000
ttm 86016 2 drm_vram_helper,drm_ttm_helper, Live 0xffffffffc11bb000
mlx5_ib 397312 0 - Live 0xffffffffc10e4000
drm_kms_helper 311296 4 ast,drm_vram_helper, Live 0xffffffffc1097000
uas 32768 1 - Live 0xffffffffc1040000
usb_storage 81920 1 uas, Live 0xffffffffc1077000
ib_uverbs 172032 1 mlx5_ib, Live 0xffffffffc1009000
agpgart 40960 1 ttm, Live 0xffffffffc0ffe000
fb_sys_fops 16384 1 drm_kms_helper, Live 0xffffffffc0ee1000
syscopyarea 16384 1 drm_kms_helper, Live 0xffffffffc0eda000
sysfillrect 16384 1 drm_kms_helper, Live 0xffffffffc0ecf000
ib_core 421888 2 mlx5_ib,ib_uverbs, Live 0xffffffffc0f7a000
sysimgblt 16384 1 drm_kms_helper, Live 0xffffffffc0eca000
ccp 114688 1 kvm_amd, Live 0xffffffffc0f02000
i2c_piix4 28672 0 - Live 0xffffffffc0f72000
rng_core 16384 2 tpm,ccp, Live 0xffffffffc0f69000
acpi_ipmi 20480 0 - Live 0xffffffffc0f63000
ipmi_si 73728 2 - Live 0xffffffffc0f48000
pinctrl_amd 32768 0 - Live 0xffffffffc0ef9000
i2c_designware_platform 16384 0 - Live 0xffffffffc0ec5000
8250_dw 16384 0 - Live 0xffffffffc0ef0000
i2c_designware_core 36864 1 i2c_designware_platform, Live 0xffffffffc0eaf000
acpi_cpufreq 28672 0 - Live 0xffffffffc0ea3000
xt_limit 16384 2 - Live 0xffffffffc0e9b000
xt_CT 16384 6 - Live 0xffffffffc0e7d000
ip6table_nat 16384 1 - Live 0xffffffffc0e78000
iptable_nat 16384 1 - Live 0xffffffffc0e73000
xt_conntrack 16384 2 - Live 0xffffffffc0e6e000
ip6t_rpfilter 16384 1 - Live 0xffffffffc0e66000
ipt_rpfilter 16384 1 - Live 0xffffffffc0e61000
ip6table_raw 16384 1 - Live 0xffffffffc0e5c000
iptable_raw 16384 1 - Live 0xffffffffc0e54000
xt_pkttype 16384 4 - Live 0xffffffffc0e4c000
xt_LOG 20480 4 - Live 0xffffffffc0e43000
nf_log_syslog 24576 4 - Live 0xffffffffc0e39000
ip6t_REJECT 16384 2 - Live 0xffffffffc0e31000
nf_reject_ipv6 20480 1 ip6t_REJECT, Live 0xffffffffc0e29000
ipt_REJECT 16384 2 - Live 0xffffffffc0e1d000
nf_reject_ipv4 16384 1 ipt_REJECT, Live 0xffffffffc0e18000
tcp_bbr 20480 30 - Live 0xffffffffc0e0f000
xt_tcpudp 20480 13 - Live 0xffffffffc0e06000
sch_fq_codel 20480 69 - Live 0xffffffffc0ddf000
ip6table_filter 16384 1 - Live 0xffffffffc0da7000
ip6_tables 36864 5 ip6table_nat,ip6table_raw,ip6table_filter, Live 0xffffffffc0d57000
iptable_filter 16384 1 - Live 0xffffffffc08df000
nf_nat_ftp 20480 0 - Live 0xffffffffc0e00000
nf_conntrack_ftp 24576 1 nf_nat_ftp, Live 0xffffffffc0df9000
nf_nat 57344 3 ip6table_nat,iptable_nat,nf_nat_ftp, Live 0xffffffffc0dea000
nf_conntrack 167936 5 xt_CT,xt_conntrack,nf_nat_ftp,nf_conntrack_ftp,nf_nat, Live 0xffffffffc0db3000
nf_defrag_ipv6 24576 1 nf_conntrack, Live 0xffffffffc0d50000
nf_defrag_ipv4 16384 1 nf_conntrack, Live 0xffffffffc0dae000
atkbd 40960 0 - Live 0xffffffffc0d9c000
libps2 20480 1 atkbd, Live 0xffffffffc0d96000
serio 28672 1 atkbd, Live 0xffffffffc0d8e000
loop 40960 0 - Live 0xffffffffc0d83000
tun 61440 0 - Live 0xffffffffc0d73000
tap 28672 0 - Live 0xffffffffc0d6b000
macvlan 28672 0 - Live 0xffffffffc0d63000
bridge 303104 0 - Live 0xffffffffc0d05000
stp 16384 1 bridge, Live 0xffffffffc0cfc000
llc 16384 2 bridge,stp, Live 0xffffffffc0cf4000
mq_deadline 32768 1 - Live 0xffffffffc0ce7000
ipmi_watchdog 32768 1 - Live 0xffffffffc0cde000
ipmi_devintf 20480 0 - Live 0xffffffffc0cd8000
ipmi_msghandler 73728 5 ipmi_ssif,acpi_ipmi,ipmi_si,ipmi_watchdog,ipmi_devintf, Live 0xffffffffc0747000
rbd 122880 0 - Live 0xffffffffc0841000
libceph 462848 1 rbd, Live 0xffffffffc086b000
drm 598016 6 ast,drm_vram_helper,drm_ttm_helper,ttm,drm_kms_helper, Live 0xffffffffc0c45000
fuse 155648 1 - Live 0xffffffffc070a000
backlight 24576 2 drm_kms_helper,drm, Live 0xffffffffc0703000
configfs 57344 1 - Live 0xffffffffc06ec000
ip_tables 36864 4 iptable_nat,iptable_raw,iptable_filter, Live 0xffffffffc06e2000
x_tables 53248 18 xt_limit,xt_CT,ip6table_nat,iptable_nat,xt_conntrack,ip6t_rpfilter,ipt_rpfilter,ip6table_raw,iptable_raw,xt_pkttype,xt_LOG,ip6t_REJECT,ipt_REJECT,xt_tcpudp,ip6table_filter,ip6_tables,iptable_filter,ip_tables, Live 0xffffffffc06cd000
autofs4 53248 0 - Live 0xffffffffc057f000
xfs 2052096 4 - Live 0xffffffffc0a4f000
raid456 184320 1 - Live 0xffffffffc062e000
async_raid6_recov 24576 1 raid456, Live 0xffffffffc0627000
async_memcpy 20480 2 raid456,async_raid6_recov, Live 0xffffffffc0621000
async_pq 20480 2 raid456,async_raid6_recov, Live 0xffffffffc0618000
async_xor 20480 3 raid456,async_raid6_recov,async_pq, Live 0xffffffffc0612000
xor 24576 1 async_xor, Live 0xffffffffc0607000
async_tx 20480 5 raid456,async_raid6_recov,async_memcpy,async_pq,async_xor, Live 0xffffffffc0601000
raid6_pq 122880 3 raid456,async_raid6_recov,async_pq, Live 0xffffffffc0594000
libcrc32c 16384 5 nf_nat,nf_conntrack,libceph,xfs,raid456, Live 0xffffffffc057a000
crc32c_generic 16384 0 - Live 0xffffffffc0509000
igb 270336 0 - Live 0xffffffffc05be000
xhci_pci 24576 0 - Live 0xffffffffc0573000
i2c_algo_bit 16384 2 ast,igb, Live 0xffffffffc04fd000
xhci_pci_renesas 20480 1 xhci_pci, Live 0xffffffffc04e4000
crc32c_intel 24576 3 - Live 0xffffffffc0864000
mlx5_core 1400832 1 mlx5_ib, Live 0xffffffffc08f8000
xhci_hcd 315392 1 xhci_pci, Live 0xffffffffc07d2000
ahci 49152 0 - Live 0xffffffffc0524000
libahci 49152 1 ahci, Live 0xffffffffc08eb000
usbcore 339968 4 uas,usb_storage,xhci_pci,xhci_hcd, Live 0xffffffffc077e000
libata 307200 2 ahci,libahci, Live 0xffffffffc0681000
i2c_core 106496 9 ipmi_ssif,ast,drm_kms_helper,i2c_piix4,i2c_designware_platform,i2c_designware_core,drm,igb,i2c_algo_bit, Live 0xffffffffc0666000
nvme 49152 13 - Live 0xffffffffc0566000
mlxfw 36864 1 mlx5_core, Live 0xffffffffc0774000
dca 16384 1 igb, Live 0xffffffffc065d000
pci_hyperv_intf 16384 1 mlx5_core, Live 0xffffffffc04f3000
psample 20480 1 mlx5_core, Live 0xffffffffc05b8000
nvme_core 139264 15 nvme, Live 0xffffffffc0543000
ptp 32768 2 igb,mlx5_core, Live 0xffffffffc049b000
pps_core 24576 1 ptp, Live 0xffffffffc0538000
t10_pi 16384 2 sd_mod,nvme_core, Live 0xffffffffc0517000
usb_common 16384 2 xhci_hcd,usbcore, Live 0xffffffffc050f000
crc_t10dif 20480 1 t10_pi, Live 0xffffffffc051e000
crct10dif_generic 16384 0 - Live 0xffffffffc04f8000
crct10dif_pclmul 16384 1 - Live 0xffffffffc04eb000
crct10dif_common 16384 3 crc_t10dif,crct10dif_generic,crct10dif_pclmul, Live 0xffffffffc0496000
rtc_cmos 28672 1 - Live 0xffffffffc041b000
dm_mod 155648 15 dm_crypt, Live 0xffffffffc04bd000
bfq 90112 0 - Live 0xffffffffc04a6000
mpt3sas 348160 0 - Live 0xffffffffc0440000
raid_class 16384 1 mpt3sas, Live 0xffffffffc0438000
scsi_transport_sas 49152 1 mpt3sas, Live 0xffffffffc0425000
megaraid_sas 180224 0 - Live 0xffffffffc03ee000
scsi_mod 274432 8 sd_mod,uas,usb_storage,libata,mpt3sas,raid_class,scsi_transport_sas,megaraid_sas, Live 0xffffffffc03aa000
scsi_common 16384 5 uas,usb_storage,libata,mpt3sas,scsi_mod, Live 0xffffffffc03a5000
Driver/Hardware Info:
0000-02ff : PCI Bus 0000:00
0000-001f : dma1
0020-0021 : pic1
0040-0043 : timer0
0050-0053 : timer1
0060-0060 : keyboard
0061-0061 : PNP0800:00
0064-0064 : keyboard
0070-0071 : rtc0
0080-008f : dma page reg
00a0-00a1 : pic2
00b2-00b2 : APEI ERST
00c0-00df : dma2
00f0-00ff : fpu
0300-03af : PCI Bus 0000:00
03b0-03df : PCI Bus 0000:c0
03c0-03df : vga+
03e0-0cf7 : PCI Bus 0000:00
03f8-03ff : serial
040b-040b : pnp 00:04
04d0-04d1 : pnp 00:04
04d6-04d6 : pnp 00:04
0800-089f : pnp 00:04
0800-0803 : ACPI PM1a_EVT_BLK
0804-0805 : ACPI PM1a_CNT_BLK
0808-080b : ACPI PM_TMR
0820-0827 : ACPI GPE0_BLK
0900-090f : pnp 00:04
0910-091f : pnp 00:04
0a00-0a3f : pnp 00:02
0a40-0a5f : pnp 00:02
0a60-0a6f : pnp 00:02
0a70-0a7f : pnp 00:02
0a80-0a8f : pnp 00:02
0b00-0b0f : pnp 00:04
0b00-0b08 : piix4_smbus
0b20-0b3f : pnp 00:04
0b20-0b28 : piix4_smbus
0c00-0c01 : pnp 00:04
0c14-0c14 : pnp 00:04
0c50-0c51 : pnp 00:04
0c52-0c52 : pnp 00:04
0c6c-0c6c : pnp 00:04
0c6f-0c6f : pnp 00:04
0ca2-0ca2 : IPI0001:00
0ca2-0ca2 : IPMI Address 1
0ca2-0ca2 : ipmi_si
0ca3-0ca3 : IPI0001:00
0ca3-0ca3 : IPMI Address 2
0ca3-0ca3 : ipmi_si
0cd0-0cd1 : pnp 00:04
0cd2-0cd3 : pnp 00:04
0cd4-0cd5 : pnp 00:04
0cd6-0cd7 : pnp 00:04
0cd8-0cdf : pnp 00:04
0cf8-0cff : PCI conf1
1000-2fff : PCI Bus 0000:00
1000-1fff : PCI Bus 0000:08
2000-2fff : PCI Bus 0000:07
3000-3fff : PCI Bus 0000:40
3000-3fff : PCI Bus 0000:48
4000-5fff : PCI Bus 0000:80
4000-4fff : PCI Bus 0000:85
5000-5fff : PCI Bus 0000:84
6000-ffff : PCI Bus 0000:c0
6000-6fff : PCI Bus 0000:c1
7000-7fff : PCI Bus 0000:c2
8000-8fff : PCI Bus 0000:c3
9000-9fff : PCI Bus 0000:c4
e000-efff : PCI Bus 0000:c7
e000-e01f : 0000:c7:00.0
f000-ffff : PCI Bus 0000:c5
f000-ffff : PCI Bus 0000:c6
f000-f07f : 0000:c6:00.0
00000000-00000fff : Reserved
00001000-0008abff : System RAM
0008ac00-0008ffff : RAM buffer
000a0000-000bffff : PCI Bus 0000:c0
000c0000-000dffff : PCI Bus 0000:00
000c0000-000c7fff : Video ROM
000cb000-000cbfff : Adapter ROM
000e0000-000fffff : Reserved
000f0000-000fffff : System ROM
00100000-75b9dfff : System RAM
75b9e000-75b9ffff : RAM buffer
75ba0000-75be3fff : ACPI Non-volatile Storage
75be4000-75c9ffff : System RAM
75ca0000-75ffffff : Reserved
76000000-9df7ffff : System RAM
9df80000-a34e6fff : Reserved
a249d018-a249d019 : APEI ERST
a249d01c-a249d021 : APEI ERST
a249d028-a249d039 : APEI ERST
a249d040-a249d04c : APEI ERST
a249d050-a249f04f : APEI ERST
a34e7000-a35d1fff : ACPI Tables
a35d2000-a3a52fff : ACPI Non-volatile Storage
a3a53000-a63fefff : Reserved
a63ff000-a7f54fff : System RAM
a7f55000-a7ffcfff : RAM buffer
a7ffd000-afffffff : Reserved
b0000000-b22fffff : PCI Bus 0000:40
b0000000-b01fffff : PCI Bus 0000:44
b0200000-b03fffff : PCI Bus 0000:45
b0400000-b05fffff : PCI Bus 0000:46
b0600000-b07fffff : PCI Bus 0000:47
b0800000-b09fffff : PCI Bus 0000:48
b2000000-b20fffff : PCI Bus 0000:43
b2000000-b200ffff : 0000:43:00.0
b2010000-b2017fff : 0000:43:00.0
b2010000-b2017fff : nvme
b2100000-b21fffff : PCI Bus 0000:42
b2100000-b210ffff : 0000:42:00.0
b2110000-b2117fff : 0000:42:00.0
b2110000-b2117fff : nvme
b2200000-b22fffff : PCI Bus 0000:41
b2200000-b220ffff : 0000:41:00.0
b2210000-b2217fff : 0000:41:00.0
b2210000-b2217fff : nvme
b8180000-b8180fff : Reserved
b8180000-b81803ff : IOAPIC 4
b9200000-b92fffff : Reserved
b9410500-b94fffff : AMDI0095:00
b9500000-b9500fff : Reserved
b9500000-b95003ff : IOAPIC 1
ba000000-bddfffff : PCI Bus 0000:c0
ba000000-ba1fffff : PCI Bus 0000:c1
ba200000-ba3fffff : PCI Bus 0000:c2
ba400000-ba5fffff : PCI Bus 0000:c3
ba600000-ba7fffff : PCI Bus 0000:c4
bc000000-bd0fffff : PCI Bus 0000:c5
bc000000-bd0fffff : PCI Bus 0000:c6
bc000000-bcffffff : 0000:c6:00.0
bd000000-bd03ffff : 0000:c6:00.0
bd200000-bd2fffff : PCI Bus 0000:c9
bd200000-bd2007ff : 0000:c9:00.1
bd200000-bd2007ff : ahci
bd201000-bd2017ff : 0000:c9:00.0
bd201000-bd2017ff : ahci
bd300000-bd3fffff : PCI Bus 0000:c8
bd300000-bd3fffff : 0000:c8:00.4
bd300000-bd3fffff : xhci-hcd
bd400000-bd4fffff : PCI Bus 0000:c7
bd400000-bd47ffff : 0000:c7:00.0
bd400000-bd47ffff : igb
bd480000-bd483fff : 0000:c7:00.0
bd480000-bd483fff : igb
e0000000-efffffff : PCI MMCONFIG 0000 [bus 00-ff]
e0000000-efffffff : Reserved
e0000000-efffffff : pnp 00:00
f0000000-f21fffff : PCI Bus 0000:80
f0000000-f01fffff : PCI Bus 0000:82
f0200000-f03fffff : PCI Bus 0000:83
f0400000-f05fffff : PCI Bus 0000:84
f0600000-f07fffff : PCI Bus 0000:85
f2000000-f21fffff : PCI Bus 0000:81
f2000000-f20fffff : 0000:81:00.1
f2100000-f21fffff : 0000:81:00.0
f4180000-f4180fff : Reserved
f4180000-f41803ff : IOAPIC 2
f5180000-f5180fff : Reserved
f5180000-f51803ff : IOAPIC 3
f6000000-f8cfffff : PCI Bus 0000:00
f8000000-f82fffff : PCI Bus 0000:0a
f8000000-f80fffff : 0000:0a:00.5
f8000000-f80fffff : ccp
f8100000-f81fffff : 0000:0a:00.4
f8100000-f81fffff : xhci-hcd
f8200000-f8201fff : 0000:0a:00.5
f8200000-f8201fff : ccp
f8300000-f83fffff : PCI Bus 0000:0b
f8300000-f83007ff : 0000:0b:00.1
f8300000-f83007ff : ahci
f8301000-f83017ff : 0000:0b:00.0
f8301000-f83017ff : ahci
f8400000-f84fffff : PCI Bus 0000:09
f8400000-f840ffff : 0000:09:00.0
f8410000-f8413fff : 0000:09:00.0
f8410000-f8413fff : nvme
f8500000-f85fffff : PCI Bus 0000:08
f8500000-f850ffff : 0000:08:00.0
f8510000-f8517fff : 0000:08:00.0
f8510000-f8517fff : nvme
f8600000-f86fffff : PCI Bus 0000:07
f8600000-f860ffff : 0000:07:00.0
f8610000-f8617fff : 0000:07:00.0
f8610000-f8617fff : nvme
f8700000-f87fffff : PCI Bus 0000:06
f8700000-f870ffff : 0000:06:00.0
f8710000-f8717fff : 0000:06:00.0
f8710000-f8717fff : nvme
f8800000-f88fffff : PCI Bus 0000:05
f8800000-f880ffff : 0000:05:00.0
f8810000-f8817fff : 0000:05:00.0
f8810000-f8817fff : nvme
f8900000-f89fffff : PCI Bus 0000:04
f8900000-f890ffff : 0000:04:00.0
f8910000-f8917fff : 0000:04:00.0
f8910000-f8917fff : nvme
f8a00000-f8afffff : PCI Bus 0000:03
f8a00000-f8a0ffff : 0000:03:00.0
f8a10000-f8a17fff : 0000:03:00.0
f8a10000-f8a17fff : nvme
f8b00000-f8bfffff : PCI Bus 0000:02
f8b00000-f8b0ffff : 0000:02:00.0
f8b10000-f8b17fff : 0000:02:00.0
f8b10000-f8b17fff : nvme
f8c00000-f8cfffff : PCI Bus 0000:01
f8c00000-f8c0ffff : 0000:01:00.0
f8c10000-f8c17fff : 0000:01:00.0
f8c10000-f8c17fff : nvme
fea00000-feafffff : Reserved
fec00000-fec00fff : Reserved
fec00000-fec003ff : IOAPIC 0
fec10000-fec10fff : Reserved
fec10000-fec10fff : pnp 00:04
fed00000-fed00fff : Reserved
fed00000-fed003ff : HPET 0
fed40000-fed44fff : Reserved
fed80000-fed8ffff : Reserved
fed80000-fed814ff : pnp 00:04
fed81500-fed818ff : AMDI0030:00
fed81900-fed8ffff : pnp 00:04
fedc0000-fedc0fff : Reserved
fedc0000-fedc0fff : pnp 00:04
fedc2000-fedc8fff : Reserved
fedc2000-fedc2fff : AMDI0010:00
fedc2000-fedc2fff : AMDI0010:00 AMDI0010:00
fedc3000-fedc3fff : AMDI0010:01
fedc3000-fedc3fff : AMDI0010:01 AMDI0010:01
fedc4000-fedc4fff : AMDI0010:02
fedc4000-fedc4fff : AMDI0010:02 AMDI0010:02
fedc5000-fedc5fff : AMDI0010:03
fedc5000-fedc5fff : AMDI0010:03 AMDI0010:03
fedc7000-fedc7fff : AMDI0020:00
fedc8000-fedc8fff : AMDI0020:01
fedc9000-fedc9fff : AMDI0020:00
fedc9000-fedc901f : serial
fedca000-fedcafff : AMDI0020:01
fedca000-fedca01f : serial
fedcb000-fedcbfff : AMDI0010:05
fedcb000-fedcbfff : AMDI0010:05 AMDI0010:05
fee00000-feefffff : Reserved
fee00000-fee00fff : Local APIC
fee00000-fee00fff : pnp 00:04
ff000000-ffffffff : Reserved
ff000000-ffffffff : pnp 00:04
100000000-184dbbffff : System RAM
7b4200000-7b4e01f81 : Kernel code
7b5000000-7b5745fff : Kernel rodata
7b5800000-7b5a3e7ff : Kernel data
7b5fb5000-7b63fffff : Kernel bss
184dbc0000-184fffffff : Reserved
10021000000-30020ffffff : PCI Bus 0000:c0
10021000000-100211fffff : PCI Bus 0000:c1
10021200000-100213fffff : PCI Bus 0000:c2
10021400000-100215fffff : PCI Bus 0000:c3
10021600000-100217fffff : PCI Bus 0000:c4
30020f00000-30020ffffff : PCI Bus 0000:c8
30020f00000-30020f7ffff : 0000:c8:00.1
30020f80000-30020ffffff : 0000:c8:00.1
30021000000-50020ffffff : PCI Bus 0000:80
30021000000-300211fffff : PCI Bus 0000:82
30021200000-300213fffff : PCI Bus 0000:83
30021400000-300215fffff : PCI Bus 0000:84
30021600000-300217fffff : PCI Bus 0000:85
5001a000000-5001effffff : PCI Bus 0000:81
5001a000000-5001bffffff : 0000:81:00.1
5001a000000-5001bffffff : mlx5_core
5001c000000-5001dffffff : 0000:81:00.0
5001c000000-5001dffffff : mlx5_core
5001e000000-5001e7fffff : 0000:81:00.1
5001e800000-5001effffff : 0000:81:00.0
5001f100000-5001f1fffff : PCI Bus 0000:86
5001f100000-5001f17ffff : 0000:86:00.1
5001f180000-5001f1fffff : 0000:86:00.1
50081000000-70080ffffff : PCI Bus 0000:00
50081000000-500811fffff : PCI Bus 0000:01
50081200000-500813fffff : PCI Bus 0000:02
50081400000-500815fffff : PCI Bus 0000:03
50081600000-500817fffff : PCI Bus 0000:04
50081800000-500819fffff : PCI Bus 0000:05
50081a00000-50081bfffff : PCI Bus 0000:06
50081c00000-50081dfffff : PCI Bus 0000:07
50081e00000-50081ffffff : PCI Bus 0000:08
70080f00000-70080ffffff : PCI Bus 0000:0a
70080f00000-70080f7ffff : 0000:0a:00.1
70080f80000-70080ffffff : 0000:0a:00.1
70081000000-90080ffffff : PCI Bus 0000:40
70081000000-700811fffff : PCI Bus 0000:41
70081200000-700813fffff : PCI Bus 0000:42
70081400000-700815fffff : PCI Bus 0000:43
70081600000-700817fffff : PCI Bus 0000:44
70081800000-700819fffff : PCI Bus 0000:45
70081a00000-70081bfffff : PCI Bus 0000:46
70081c00000-70081dfffff : PCI Bus 0000:47
70081e00000-70081ffffff : PCI Bus 0000:48
90080f00000-90080ffffff : PCI Bus 0000:49
90080f00000-90080f7ffff : 0000:49:00.1
90080f80000-90080ffffff : 0000:49:00.1
3ffc00000000-3ffc03ffffff : Reserved
PCI: see attached
Attachment:
lspci
Description: Binary data
Here’s the block device structure:
nvme1n1 259:0 0 14T 0 disk
└─md127 9:127 0 111.8T 0 raid6
└─vgbackup-backy--crypted 253:0 0 111.8T 0 lvm
└─backy 253:4 0 111.8T 0 crypt /srv/backy
nvme0n1 259:1 0 14T 0 disk
└─md127 9:127 0 111.8T 0 raid6
└─vgbackup-backy--crypted 253:0 0 111.8T 0 lvm
└─backy 253:4 0 111.8T 0 crypt /srv/backy
nvme4n1 259:2 0 14T 0 disk
└─md127 9:127 0 111.8T 0 raid6
└─vgbackup-backy--crypted 253:0 0 111.8T 0 lvm
└─backy 253:4 0 111.8T 0 crypt /srv/backy
nvme2n1 259:3 0 14T 0 disk
└─md127 9:127 0 111.8T 0 raid6
└─vgbackup-backy--crypted 253:0 0 111.8T 0 lvm
└─backy 253:4 0 111.8T 0 crypt /srv/backy
nvme10n1 259:4 0 14T 0 disk
└─md127 9:127 0 111.8T 0 raid6
└─vgbackup-backy--crypted 253:0 0 111.8T 0 lvm
└─backy 253:4 0 111.8T 0 crypt /srv/backy
nvme11n1 259:5 0 14T 0 disk
└─md127 9:127 0 111.8T 0 raid6
└─vgbackup-backy--crypted 253:0 0 111.8T 0 lvm
└─backy 253:4 0 111.8T 0 crypt /srv/backy
nvme6n1 259:6 0 14T 0 disk
└─md127 9:127 0 111.8T 0 raid6
└─vgbackup-backy--crypted 253:0 0 111.8T 0 lvm
└─backy 253:4 0 111.8T 0 crypt /srv/backy
nvme5n1 259:7 0 14T 0 disk
└─md127 9:127 0 111.8T 0 raid6
└─vgbackup-backy--crypted 253:0 0 111.8T 0 lvm
└─backy 253:4 0 111.8T 0 crypt /srv/backy
nvme3n1 259:8 0 14T 0 disk
└─md127 9:127 0 111.8T 0 raid6
└─vgbackup-backy--crypted 253:0 0 111.8T 0 lvm
└─backy 253:4 0 111.8T 0 crypt /srv/backy
nvme7n1 259:9 0 14T 0 disk
└─md127 9:127 0 111.8T 0 raid6
└─vgbackup-backy--crypted 253:0 0 111.8T 0 lvm
└─backy 253:4 0 111.8T 0 crypt /srv/backy
nvme9n1 259:10 0 14T 0 disk
└─md127 9:127 0 111.8T 0 raid6
└─vgbackup-backy--crypted 253:0 0 111.8T 0 lvm
└─backy 253:4 0 111.8T 0 crypt /srv/backy
And the software raid setup:
Personalities : [raid6] [raid5] [raid4]
md127 : active raid6 nvme5n1[5] nvme7n1[7] nvme4n1[4] nvme3n1[3] nvme11n1[10](S) nvme0n1[0] nvme1n1[1] nvme2n1[2] nvme6n1[6] nvme9n1[8] nvme10n1[9]
120006369280 blocks super 1.2 level 6, 512k chunk, algorithm 2 [10/10] [UUUUUUUUUU]
bitmap: 1/112 pages [4KB], 65536KB chunk
unused devices: <none>
Let me know if I can provide any other info that might help diagnosing this,
Thanks and hugs,
Christian
--
Christian Theune · ct@xxxxxxxxxxxxxxx · +49 345 219401 0
Flying Circus Internet Operations GmbH · https://flyingcircus.io
Leipziger Str. 70/71 · 06108 Halle (Saale) · Deutschland
HR Stendal HRB 21169 · Geschäftsführer: Christian Theune, Christian Zagrodnick
