On 7/26/19 2:02 PM, Sedat Dilek wrote: > On Fri, Jul 26, 2019 at 10:38 PM Sedat Dilek <sedat.dilek@xxxxxxxxx> wrote: >> >> Hi Yonghong Song, >> >> On Fri, Jul 26, 2019 at 5:45 PM Yonghong Song <yhs@xxxxxx> wrote: >>> >>> >>> >>> On 7/26/19 1:26 AM, Sedat Dilek wrote: >>>> Hi, >>>> >>>> I have opened a new issue in the ClangBuiltLinux issue tracker. >>> >>> Glad to know clang 9 has asm goto support and now It can compile >>> kernel again. >>> >> >> Yupp. >> >>>> >>>> I am seeing a problem in the area bpf/seccomp causing >>>> systemd/journald/udevd services to fail. >>>> >>>> [Fri Jul 26 08:08:43 2019] systemd[453]: systemd-udevd.service: Failed >>>> to connect stdout to the journal socket, ignoring: Connection refused >>>> >>>> This happens when I use the (LLVM) LLD ld.lld-9 linker but not with >>>> BFD linker ld.bfd on Debian/buster AMD64. >>>> In both cases I use clang-9 (prerelease). >>> >>> Looks like it is a lld bug. >>> >>> I see the stack trace has __bpf_prog_run32() which is used by >>> kernel bpf interpreter. Could you try to enable bpf jit >>> sysctl net.core.bpf_jit_enable = 1 >>> If this passed, it will prove it is interpreter related. >>> >> >> After... >> >> sysctl -w net.core.bpf_jit_enable=1 >> >> I can start all failed systemd services. >> >> systemd-journald.service >> systemd-udevd.service >> haveged.service >> >> This is in maintenance mode. >> >> What is next: Do set a permanent sysctl setting for net.core.bpf_jit_enable? >> > > This is what I did: I probably won't have cycles to debug this potential lld issue. Maybe you already did, I suggest you put enough reproducible details in the bug you filed against lld so they can take a look. > > Jul 26 22:43:06 iniza kernel: BUG: unable to handle page fault for > address: ffffffffa8203370 > Jul 26 22:43:06 iniza kernel: #PF: supervisor read access in kernel mode > Jul 26 22:43:06 iniza kernel: #PF: error_code(0x0000) - not-present page > Jul 26 22:43:06 iniza kernel: PGD 2cfa0e067 P4D 2cfa0e067 PUD > 2cfa0f063 PMD 450829063 PTE 800ffffd30bfc062 > Jul 26 22:43:06 iniza kernel: Oops: 0000 [#3] SMP PTI > Jul 26 22:43:06 iniza kernel: CPU: 3 PID: 436 Comm: systemd-udevd > Tainted: G D 5.3.0-rc1-7-amd64-cbl-asmgoto > #7~buster+dileks1 > Jul 26 22:43:06 iniza kernel: Hardware name: LENOVO > 20HDCTO1WW/20HDCTO1WW, BIOS N1QET83W (1.58 ) 04/18/2019 > Jul 26 22:43:06 iniza kernel: RIP: 0010:___bpf_prog_run+0x40/0x14f0 > Jul 26 22:43:06 iniza kernel: Code: f3 eb 24 48 83 f8 38 0f 84 a9 0c > 00 00 48 83 f8 39 0f 85 8a 14 00 00 0f 1f 00 48 0f bf 43 02 48 8d 1c > c3 48 83 c3 08 0f b6 > 33 <48> 8b 04 f5 10 2e 20 a8 48 83 f8 3b 7f 62 48 83 f8 1e 0f 8f c8 00 > Jul 26 22:43:06 iniza kernel: RSP: 0018:ffffb3cec0327a88 EFLAGS: 00010246 > Jul 26 22:43:06 iniza kernel: RAX: ffffb3cec0327b30 RBX: > ffffb3cec00d1038 RCX: 0000000000000000 > Jul 26 22:43:06 iniza kernel: RDX: ffffb3cec0327b10 RSI: > 00000000000000ac RDI: ffffb3cec0327ab0 > Jul 26 22:43:06 iniza kernel: RBP: ffffb3cec0327aa0 R08: > ffff9b33c94c0a00 R09: 0000000000000000 > Jul 26 22:43:06 iniza kernel: R10: ffff9b33cfe14e00 R11: > ffffffffa77b8210 R12: 0000000000000000 > Jul 26 22:43:06 iniza kernel: R13: ffffb3cec00d1000 R14: > 0000000000000000 R15: ffffb3cec0327ab0 > Jul 26 22:43:06 iniza kernel: FS: 00007f7ac2d28d40(0000) > GS:ffff9b33d2580000(0000) knlGS:0000000000000000 > Jul 26 22:43:06 iniza kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > Jul 26 22:43:06 iniza kernel: CR2: ffffffffa8203370 CR3: > 000000044f3ea006 CR4: 00000000003606e0 > Jul 26 22:43:06 iniza kernel: Call Trace: > Jul 26 22:43:06 iniza kernel: __bpf_prog_run32+0x44/0x70 > Jul 26 22:43:06 iniza kernel: ? security_sock_rcv_skb+0x3f/0x60 > Jul 26 22:43:06 iniza kernel: sk_filter_trim_cap+0xe4/0x220 > Jul 26 22:43:06 iniza kernel: ? __skb_clone+0x2e/0x100 > Jul 26 22:43:06 iniza kernel: netlink_broadcast_filtered+0x2df/0x4f0 > Jul 26 22:43:06 iniza kernel: netlink_sendmsg+0x34f/0x3c0 > Jul 26 22:43:06 iniza kernel: ___sys_sendmsg+0x315/0x330 > Jul 26 22:43:06 iniza kernel: ? seccomp_run_filters+0x54/0x110 > Jul 26 22:43:06 iniza kernel: ? filename_parentat+0x210/0x490 > Jul 26 22:43:06 iniza kernel: ? __seccomp_filter+0xf7/0x6e0 > Jul 26 22:43:06 iniza kernel: ? __d_alloc+0x159/0x1c0 > Jul 26 22:43:06 iniza kernel: ? kmem_cache_free+0x1e/0x5c0 > Jul 26 22:43:06 iniza kernel: ? fast_dput+0x73/0xb0 > Jul 26 22:43:06 iniza kernel: __x64_sys_sendmsg+0x97/0xe0 > Jul 26 22:43:06 iniza kernel: do_syscall_64+0x59/0x90 > Jul 26 22:43:06 iniza kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9 > Jul 26 22:43:06 iniza kernel: RIP: 0033:0x7f7ac3519914 > Jul 26 22:43:06 iniza kernel: Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff > ff ff eb b5 0f 1f 80 00 00 00 00 48 8d 05 e9 5d 0c 00 8b 00 85 c0 75 > 13 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 41 54 41 > 89 d4 55 48 89 f5 53 > Jul 26 22:43:06 iniza kernel: RSP: 002b:00007ffcfb66a478 EFLAGS: > 00000246 ORIG_RAX: 000000000000002e > Jul 26 22:43:06 iniza kernel: RAX: ffffffffffffffda RBX: > 0000561e28ac9390 RCX: 00007f7ac3519914 > Jul 26 22:43:06 iniza kernel: RDX: 0000000000000000 RSI: > 00007ffcfb66a4a0 RDI: 000000000000000d > Jul 26 22:43:06 iniza kernel: RBP: 0000561e28acd210 R08: > 0000561e28990140 R09: 0000000000000002 > Jul 26 22:43:06 iniza kernel: R10: 0000000000000000 R11: > 0000000000000246 R12: 0000000000000000 > Jul 26 22:43:06 iniza kernel: R13: 0000000000000000 R14: > 000000000000005e R15: 00007ffcfb66a490 > Jul 26 22:43:06 iniza kernel: Modules linked in: nfsd auth_rpcgss > nfs_acl lockd grace i2c_dev parport_pc ppdev lp parport sunrpc > efivarfs ip_tables x_tables autofs4 ext4 crc32c_generic mbcache crc16 > jbd2 btrfs zstd_decompress zstd_compress algif_skcipher af_alg sd_mod > uas usb_storage scsi_mod hid_generic usbhid hid dm_crypt dm_mod raid10 > raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor > raid6_pq libcrc32c raid1 raid0 multipath linear md_mod > crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel > aesni_intel i915 intel_lpss_pci nvme aes_x86_64 glue_helper > i2c_algo_bit crypto_simd cryptd xhci_pci psmouse e1000e drm_kms_helper > xhci_hcd i2c_i801 nvme_core intel_lpss drm usbcore thermal wmi video > button > Jul 26 22:43:06 iniza kernel: CR2: ffffffffa8203370 > Jul 26 22:43:06 iniza kernel: ---[ end trace 312670b063bd0391 ]--- > Jul 26 22:43:06 iniza kernel: RIP: 0010:___bpf_prog_run+0x40/0x14f0 > Jul 26 22:43:06 iniza kernel: Code: f3 eb 24 48 83 f8 38 0f 84 a9 0c > 00 00 48 83 f8 39 0f 85 8a 14 00 00 0f 1f 00 48 0f bf 43 02 48 8d 1c > c3 48 83 c3 08 0f b6 33 <48> 8b 04 f5 10 2e 20 a8 48 83 f8 3b 7f 62 48 > 83 f8 1e 0f 8f c8 00 > Jul 26 22:43:06 iniza kernel: RSP: 0018:ffffb3cec0253cb8 EFLAGS: 00010246 > Jul 26 22:43:06 iniza kernel: RAX: ffffb3cec0253d60 RBX: > ffffb3cec00e9038 RCX: 0000000000000002 > Jul 26 22:43:06 iniza kernel: RDX: ffffb3cec0253d40 RSI: > 00000000000000ac RDI: ffffb3cec0253ce0 > Jul 26 22:43:06 iniza kernel: RBP: ffffb3cec0253cd0 R08: > 0000000000000000 R09: ffffb3cec0253f58 > Jul 26 22:43:06 iniza kernel: R10: 0000000000000000 R11: > ffffffffa77b8210 R12: 000000007fff0000 > Jul 26 22:43:06 iniza kernel: R13: ffffb3cec0253eb8 R14: > 0000000000000000 R15: ffffb3cec0253ce0 > Jul 26 22:43:06 iniza kernel: FS: 00007f7ac2d28d40(0000) > GS:ffff9b33d2580000(0000) knlGS:0000000000000000 > Jul 26 22:43:06 iniza kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > Jul 26 22:43:06 iniza kernel: CR2: ffffffffa8203370 CR3: > 000000044f3ea006 CR4: 00000000003606e0 > Jul 26 22:43:06 iniza kernel: BUG: unable to handle page fault for > address: ffffffffa8203370 > Jul 26 22:43:06 iniza kernel: #PF: supervisor read access in kernel mode > Jul 26 22:43:06 iniza kernel: #PF: error_code(0x0000) - not-present page > Jul 26 22:43:06 iniza kernel: PGD 2cfa0e067 P4D 2cfa0e067 PUD > 2cfa0f063 PMD 450829063 PTE 800ffffd30bfc062 > Jul 26 22:43:06 iniza kernel: Oops: 0000 [#4] SMP PTI > Jul 26 22:43:06 iniza kernel: CPU: 0 PID: 437 Comm: systemd-udevd > Tainted: G D 5.3.0-rc1-7-amd64-cbl-asmgoto > #7~buster+dileks1 > Jul 26 22:43:06 iniza kernel: Hardware name: LENOVO > 20HDCTO1WW/20HDCTO1WW, BIOS N1QET83W (1.58 ) 04/18/2019 > Jul 26 22:43:06 iniza kernel: RIP: 0010:___bpf_prog_run+0x40/0x14f0 > Jul 26 22:43:06 iniza kernel: Code: f3 eb 24 48 83 f8 38 0f 84 a9 0c > 00 00 48 83 f8 39 0f 85 8a 14 00 00 0f 1f 00 48 0f bf 43 02 48 8d 1c > c3 48 83 c3 08 0f b6 33 <48> 8b 04 f5 10 2e 20 a8 48 83 f8 3b 7f 62 48 > 83 f8 1e 0f 8f c8 00 > Jul 26 22:43:06 iniza kernel: RSP: 0018:ffffb3cec032fa88 EFLAGS: 00010246 > Jul 26 22:43:06 iniza kernel: RAX: ffffb3cec032fb30 RBX: > ffffb3cec00d1038 RCX: 0000000000000000 > Jul 26 22:43:06 iniza kernel: RDX: ffffb3cec032fb10 RSI: > 00000000000000ac RDI: ffffb3cec032fab0 > Jul 26 22:43:06 iniza kernel: RBP: ffffb3cec032faa0 R08: > ffff9b33cf34b000 R09: 0000000000000000 > Jul 26 22:43:06 iniza kernel: R10: ffff9b33cf3a3400 R11: > ffffffffa77b8210 R12: 0000000000000000 > Jul 26 22:43:06 iniza kernel: R13: ffffb3cec00d1000 R14: > 0000000000000000 R15: ffffb3cec032fab0 > Jul 26 22:43:06 iniza kernel: FS: 00007f7ac2d28d40(0000) > GS:ffff9b33d2400000(0000) knlGS:0000000000000000 > Jul 26 22:43:06 iniza kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > Jul 26 22:43:06 iniza kernel: CR2: ffffffffa8203370 CR3: > 000000044724a001 CR4: 00000000003606f0 > Jul 26 22:43:06 iniza kernel: Call Trace: > Jul 26 22:43:06 iniza kernel: __bpf_prog_run32+0x44/0x70 > Jul 26 22:43:06 iniza kernel: ? prep_new_page+0x47/0x1a0 > Jul 26 22:43:06 iniza kernel: ? security_sock_rcv_skb+0x3f/0x60 > Jul 26 22:43:06 iniza kernel: sk_filter_trim_cap+0xe4/0x220 > Jul 26 22:43:06 iniza kernel: ? __skb_clone+0x2e/0x100 > Jul 26 22:43:06 iniza kernel: netlink_broadcast_filtered+0x2df/0x4f0 > Jul 26 22:43:06 iniza kernel: netlink_sendmsg+0x34f/0x3c0 > Jul 26 22:43:06 iniza kernel: ___sys_sendmsg+0x315/0x330 > Jul 26 22:43:06 iniza kernel: ? seccomp_run_filters+0x54/0x110 > Jul 26 22:43:06 iniza kernel: ? filename_parentat+0x210/0x490 > Jul 26 22:43:06 iniza kernel: ? __seccomp_filter+0xf7/0x6e0 > Jul 26 22:43:06 iniza kernel: ? __d_alloc+0x159/0x1c0 > Jul 26 22:43:06 iniza kernel: ? kmem_cache_free+0x1e/0x5c0 > Jul 26 22:43:06 iniza kernel: ? fast_dput+0x73/0xb0 > Jul 26 22:43:06 iniza kernel: __x64_sys_sendmsg+0x97/0xe0 > Jul 26 22:43:06 iniza kernel: do_syscall_64+0x59/0x90 > Jul 26 22:43:06 iniza kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9 > Jul 26 22:43:06 iniza kernel: RIP: 0033:0x7f7ac3519914 > Jul 26 22:43:06 iniza kernel: Code: 00 f7 d8 64 89 02 48 c7 c0 ff ff > ff ff eb b5 0f 1f 80 00 00 00 00 48 8d 05 e9 5d 0c 00 8b 00 85 c0 75 > 13 b8 2e 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 41 54 41 > 89 d4 55 48 89 f5 53 > Jul 26 22:43:06 iniza kernel: RSP: 002b:00007ffcfb66a478 EFLAGS: > 00000246 ORIG_RAX: 000000000000002e > Jul 26 22:43:06 iniza kernel: RAX: ffffffffffffffda RBX: > 0000561e28aaa600 RCX: 00007f7ac3519914 > Jul 26 22:43:06 iniza kernel: RDX: 0000000000000000 RSI: > 00007ffcfb66a4a0 RDI: 000000000000000e > Jul 26 22:43:06 iniza kernel: RBP: 0000561e28aaaac0 R08: > 0000561e28990140 R09: 0000000000000002 > Jul 26 22:43:06 iniza kernel: R10: 0000000000000000 R11: > 0000000000000246 R12: 0000000000000000 > Jul 26 22:43:06 iniza kernel: R13: 0000000000000000 R14: > 000000000000005d R15: 00007ffcfb66a490 > Jul 26 22:43:06 iniza kernel: Modules linked in: nfsd auth_rpcgss > nfs_acl lockd grace i2c_dev parport_pc ppdev lp parport sunrpc > efivarfs ip_tables x_tables autofs4 ext4 crc32c_generic mbcache crc16 > jbd2 btrfs zstd_decompress zstd_compress algif_skcipher af_alg sd_mod > uas usb_storage scsi_mod hid_generic usbhid hid dm_crypt dm_mod raid10 > raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor > raid6_pq libcrc32c raid1 raid0 multipath linear md_mod > crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel > aesni_intel i915 intel_lpss_pci nvme aes_x86_64 glue_helper > i2c_algo_bit crypto_simd cryptd xhci_pci psmouse e1000e drm_kms_helper > xhci_hcd i2c_i801 nvme_core intel_lpss drm usbcore thermal wmi video > button > Jul 26 22:43:06 iniza kernel: CR2: ffffffffa8203370 > Jul 26 22:43:06 iniza kernel: ---[ end trace 312670b063bd0392 ]--- > > Full `journalctl -xb` attached. > > - Sedat - >
