Chris, I see NO I/O messages. I used less to look at history.sqlite. The file looks like a log file around the time of a system crash. LOTS of unprintable characters. Some text at the end that looks like they might be from dnf. @^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ 91652 ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ 91652 ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@<81>^M<82><F8>5^F^@^B <82>^W^T<82>warning: file 91652 /usr/src/kernels/5.12.0-0.rc1.162.fc35.x86_64/include/config/video/tveeprom.h: remove failed: No such file or dire 91652 ctory<81>^L<82><F8>4^F^@^B <82>^U^T<82>warning: file /usr/src/kernels/5.12.0-0.rc1.162.fc35.x86_64/include/co 91652 nfig/video/tvp5150.h: remove failed: No such file or directory<81>^K<82><F8>3^F^@^B <82>^S^T<82>warning: file 91652 /usr/src/kernels/5.12.0-0.rc1.162.fc35.x86_64/include/config/video/tw2804.h: remove failed: No such file or direct 91652 ory<81>^K<82><F8>2^F^@^B <82>^S^T<82>warning: file /usr/src/kernels/5.12.0-0.rc1.162.fc35.x86_64/include/co 91652 nfig/video/tw686x.h: remove failed: No such file or directory<81>^K<82><F8>1^F^@^B <82>^S^T<82>warning: file 91652 /usr/src/kernels/5.12.0-0.rc1.162.fc35.x86_64/include/config/video/tw9903.h: remove failed: No such file or direct 91652 ory<81>^K<82><F8>0^F^@^B <82>^S^T<82>warning: file /usr/src/kernels/5.12.0-0.rc1.162.fc35.x86_64/include/co 91652 nfig/video/tw9906.h: remove failed: No such file or directory<81>^L<82><F8>/^F^@^B <82>^U^T<82>warning: file 91652 /usr/src/kernels/5.12.0-0.rc1.162.fc35.x86_64/include/config/video/uda1342.h: remove failed: No such file or direc 91652 tory<81>^N<82><F8>.^F^@^B <82>^Y^T<82>warning: file /usr/src/kernels/5.12.0-0.rc1.162.fc35.x86_64/include/co 91652 nfig/video/upd64031a.h: remove failed: No such file or directory<81>^M<82><F8>-^F^@^B <82>^W^T<82>warning: file 91652 /usr/src/kernels/5.12.0-0.rc1.162.fc35.x86_64/include/config/video/upd64083.h: remove failed: No such file or dire 91652 ctory<81> 91653 <82><F8>,^F^@^B <82>^Q^T<82>warning: file /usr/src/kernels/5.12.0-0.rc1.162.fc35.x86_64/include/config/video/usbtv 91653 .h: remove failed: No such file or directory<81>^G<82><F8>+^F^@^B <82>^K^T<82>warning: file /usr/src/kernels 91653 /5.12.0-0.rc1.162.fc35.x86_64/include/config/video/v4l2: remove failed: No such file or directory Scrub is clean. Odd messages from journalctl though: -- Journal begins at Wed 2021-02-24 19:05:12 PST. -- Mar 13 17:32:57 fc35 kernel: BTRFS warning (device sda5): csum failed root 5 ino 2239235 off 1218498560 csum 0x694ef7e2 expected csum 0x204f8b8d mirror 1 Mar 13 17:32:57 fc35 kernel: BTRFS error (device sda5): bdev /dev/sda5 errs: wr 0, rd 0, flush 0, corrupt 82, gen 0 Mar 13 17:32:57 fc35 kernel: BTRFS warning (device sda5): csum failed root 5 ino 2239235 off 1218506752 csum 0x302a9a33 expected csum 0x204f8b8d mirror 1 Mar 13 17:32:57 fc35 kernel: BTRFS error (device sda5): bdev /dev/sda5 errs: wr 0, rd 0, flush 0, corrupt 83, gen 0 Mar 13 17:32:57 fc35 kernel: BTRFS warning (device sda5): csum failed root 5 ino 2239235 off 1218445312 csum 0x9ee6da83 expected csum 0x204f8b8d mirror 1 Mar 13 17:32:57 fc35 kernel: BTRFS error (device sda5): bdev /dev/sda5 errs: wr 0, rd 0, flush 0, corrupt 84, gen 0 Mar 13 17:32:57 fc35 kernel: BTRFS warning (device sda5): csum failed root 5 ino 2239235 off 1218445312 csum 0x9ee6da83 expected csum 0x204f8b8d mirror 1 Mar 13 17:32:57 fc35 kernel: BTRFS error (device sda5): bdev /dev/sda5 errs: wr 0, rd 0, flush 0, corrupt 85, gen 0 Mar 13 17:33:47 fc35 kernel: BTRFS info (device sda5): scrub: started on devid 1 Mar 13 17:34:39 fc35 kernel: BTRFS info (device sda5): scrub: finished on devid 1 with status: 0 ---------------------------------------------------------------------- Date: Sat, 13 Mar 2021 11:20:19 -0700 From: Chris Murphy <lists@xxxxxxxxxxxxxxxxx> Subject: Re: Is there a procedure to rebuild /var/lib/dnf/history.sqlite? To: For testing and quality assurance of Fedora releases <test@xxxxxxxxxxxxxxxxxxxxxxx> Message-ID: <CAJCQCtSUEbUAnPyxNXvkN1CTgJrbvhh4SjziZXQpj7c1+FOJ+Q@xxxxxxxxxxxxxx> Content-Type: text/plain; charset="UTF-8" On Sat, Mar 13, 2021 at 10:13 AM George R Goffe via test <test@xxxxxxxxxxxxxxxxxxxxxxx> wrote: > > Chris, > > I goofed and sent this with the wrong subject. Sorry about that. > > > Thanks for responding. > > I converted /var to btrfs about 2 weeks ago so this isn't necessarily related to that (I hope). > > Here's what "btrfs filesystem usage /var" says about /var: (it is NOT mounted ro) > > > fc35-bash 5.1 ~# btrfs filesystem usage /var > Overall: > Device size: 64.00GiB > Device allocated: 63.94GiB > Device unallocated: 65.68MiB > Device missing: 0.00B > Used: 50.30GiB > Free (estimated): 13.01GiB (min: 13.01GiB) > Free (statfs, df): 13.01GiB > Data ratio: 1.00 > Metadata ratio: 1.00 > Global reserve: 61.03MiB (used: 0.00B) > Multiple profiles: no > > Data,single: Size:63.15GiB, Used:50.20GiB (79.49%) > /dev/sda5 63.15GiB > > Metadata,single: Size:776.00MiB, Used:105.33MiB (13.57%) > /dev/sda5 776.00MiB > > System,single: Size:32.00MiB, Used:16.00KiB (0.05%) > /dev/sda5 32.00MiB > > Unallocated: > /dev/sda5 65.68MiB > > > > > dmesg buffer, recent boot: yeah what I'm looking for are the kernel messages at the time of the I/O error you're getting from dnf. Do this: journalctl -fk And in another terminal shell, run the same dnf history command that triggers the error, switch back to the first shell and see what messages have appeared. >You will note what looks like a kernel dump related to duplicate lock requests. This and other kernel oops like events happen on a regular basis on "my" systems. I got tired of reporting them and getting NO response to the bug reports. Sigh. It's an older 5.11.0-rc6 debug kernel. You don't need to report the lockdep warnings. They come and go in kernel rc development. Pretty sure that particular lock dep warning was fixed a while ago. The current Fedora kernel is 5.11.5 with 5.11.6 heading to stable soon, so you should update. -- Chris Murphy ------------------------------ _______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure