----- Original Message ----- > From: "Ming Lei" <ming.lei@xxxxxxxxxx> > To: "CKI Project" <cki-project@xxxxxxxxxx> > Cc: linux-block@xxxxxxxxxxxxxxx, axboe@xxxxxxxxx, "Changhui Zhong" <czhong@xxxxxxxxxx> > Sent: Friday, September 4, 2020 3:02:33 AM > Subject: Re: 💥 PANICKED: Test report for kernel 5.9.0-rc3-020ad03.cki (block) > > On Thu, Sep 03, 2020 at 05:07:57PM -0000, CKI Project wrote: > > > > Hello, > > > > We ran automated tests on a recent commit from this kernel tree: > > > > Kernel repo: > > https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git > > Commit: 020ad0333b03 - Merge branch 'for-5.10/block' into > > for-next > > > > The results of these automated tests are provided below. > > > > Overall result: FAILED (see details below) > > Merge: OK > > Compile: OK > > Tests: PANICKED > > > > All kernel binaries, config files, and logs are available for download > > here: > > > > https://cki-artifacts.s3.us-east-2.amazonaws.com/index.html?prefix=datawarehouse/2020/09/02/613166 > > > > One or more kernel tests failed: > > > > ppc64le: > > 💥 storage: software RAID testing > > > > aarch64: > > 💥 storage: software RAID testing > > > > x86_64: > > 💥 storage: software RAID testing > > > > We hope that these logs can help you find the problem quickly. For the full > > detail on our testing procedures, please scroll to the bottom of this > > message. > > > > Please reply to this email if you have any questions about the tests that > > we > > ran or if you have any suggestions on how to make future tests more > > effective. > > > > ,-. ,-. > > ( C ) ( K ) Continuous > > `-',-.`-' Kernel > > ( I ) Integration > > `-' > > ______________________________________________________________________________ > > > > Compile testing > > --------------- > > > > We compiled the kernel for 4 architectures: > > > > aarch64: > > make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg > > > > ppc64le: > > make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg > > > > s390x: > > make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg > > > > x86_64: > > make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg > > > > > > > > Hardware testing > > ---------------- > > We booted each kernel and ran the following tests: > > > > aarch64: > > Host 1: > > ✅ Boot test > > ✅ ACPI table test > > ✅ LTP > > ✅ Loopdev Sanity > > ✅ Memory function: memfd_create > > ✅ AMTU (Abstract Machine Test Utility) > > ✅ Ethernet drivers sanity > > ✅ storage: SCSI VPD > > 🚧 ✅ CIFS Connectathon > > 🚧 ✅ POSIX pjd-fstest suites > > > > Host 2: > > > > ⚡ Internal infrastructure issues prevented one or more tests (marked > > with ⚡⚡⚡) from running on this architecture. > > This is not the fault of the kernel that was tested. > > > > ⚡⚡⚡ Boot test > > ⚡⚡⚡ xfstests - ext4 > > ⚡⚡⚡ xfstests - xfs > > ⚡⚡⚡ storage: software RAID testing > > ⚡⚡⚡ stress: stress-ng > > 🚧 ⚡⚡⚡ xfstests - btrfs > > 🚧 ⚡⚡⚡ Storage blktests > > > > Host 3: > > ✅ Boot test > > ✅ xfstests - ext4 > > ✅ xfstests - xfs > > 💥 storage: software RAID testing > > ⚡⚡⚡ stress: stress-ng > > 🚧 ⚡⚡⚡ xfstests - btrfs > > 🚧 ⚡⚡⚡ Storage blktests > > > > ppc64le: > > Host 1: > > ✅ Boot test > > 🚧 ✅ kdump - sysrq-c > > > > Host 2: > > ✅ Boot test > > ✅ xfstests - ext4 > > ✅ xfstests - xfs > > 💥 storage: software RAID testing > > 🚧 ⚡⚡⚡ xfstests - btrfs > > 🚧 ⚡⚡⚡ Storage blktests > > > > Host 3: > > > > ⚡ Internal infrastructure issues prevented one or more tests (marked > > with ⚡⚡⚡) from running on this architecture. > > This is not the fault of the kernel that was tested. > > > > ✅ Boot test > > ⚡⚡⚡ LTP > > ⚡⚡⚡ Loopdev Sanity > > ⚡⚡⚡ Memory function: memfd_create > > ⚡⚡⚡ AMTU (Abstract Machine Test Utility) > > ⚡⚡⚡ Ethernet drivers sanity > > 🚧 ⚡⚡⚡ CIFS Connectathon > > 🚧 ⚡⚡⚡ POSIX pjd-fstest suites > > > > s390x: > > Host 1: > > ✅ Boot test > > ✅ stress: stress-ng > > 🚧 ✅ Storage blktests > > > > Host 2: > > ✅ Boot test > > ✅ LTP > > ✅ Loopdev Sanity > > ✅ Memory function: memfd_create > > ✅ AMTU (Abstract Machine Test Utility) > > ✅ Ethernet drivers sanity > > 🚧 ✅ CIFS Connectathon > > 🚧 ✅ POSIX pjd-fstest suites > > > > x86_64: > > Host 1: > > ✅ Boot test > > ✅ Storage SAN device stress - qedf driver > > > > Host 2: > > ⏱ Boot test > > ⏱ Storage SAN device stress - mpt3sas_gen1 > > > > Host 3: > > ✅ Boot test > > ✅ xfstests - ext4 > > ✅ xfstests - xfs > > 💥 storage: software RAID testing > > ⚡⚡⚡ stress: stress-ng > > 🚧 ⚡⚡⚡ xfstests - btrfs > > 🚧 ⚡⚡⚡ Storage blktests > > > > Host 4: > > ✅ Boot test > > ✅ Storage SAN device stress - lpfc driver > > > > Host 5: > > ✅ Boot test > > 🚧 ✅ kdump - sysrq-c > > > > Host 6: > > ✅ Boot test > > ✅ ACPI table test > > ✅ LTP > > ✅ Loopdev Sanity > > ✅ Memory function: memfd_create > > ✅ AMTU (Abstract Machine Test Utility) > > ✅ Ethernet drivers sanity > > ✅ kernel-rt: rt_migrate_test > > ✅ kernel-rt: rteval > > ✅ kernel-rt: sched_deadline > > ✅ kernel-rt: smidetect > > ✅ storage: SCSI VPD > > 🚧 ✅ CIFS Connectathon > > 🚧 ✅ POSIX pjd-fstest suites > > > > Host 7: > > ✅ Boot test > > ✅ kdump - sysrq-c - megaraid_sas > > > > Host 8: > > ✅ Boot test > > ✅ Storage SAN device stress - qla2xxx driver > > > > Host 9: > > ⏱ Boot test > > ⏱ kdump - sysrq-c - mpt3sas_gen1 > > > > Test sources: https://gitlab.com/cki-project/kernel-tests > > Hello, > Hi Ming, first the good news: Both issues detected by LTP and RAID test are officially gone after the revert. There's some x86_64 testing still running but the results look good so far! > Can you share us the exact commands for setting up xfstests over > 'software RAID testing' from the above tree? > It's this test (which seeing your @redhat email, you can also trigger via internal Brew testing if you use the "stor" test set): https://gitlab.com/cki-project/kernel-tests/-/tree/master/storage/swraid/trim The important part of the test is: https://gitlab.com/cki-project/kernel-tests/-/blob/master/storage/swraid/trim/main.sh#L27 The test maintainer (Changhui) is cced on this thread in case you need any help or have questions about the test. I'll just quickly mention, please be careful if you're planning on testing LTP/msgstress04 on ppc64le in Beaker, as the conserver overload is causing issues to lab owners. Let us know if we can help you with something else, Veronika > BTW I can't reproduce it by running xfstest generic/551 on my simple raid10 > settings, include raid stop/remove steps. > > Thanks, > Ming > >
