[STATUS UPDATE] test-media regression script status

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi all,

As you all know I have been working on creating a regression test script
to help catch regressions before they get into released kernels.

The test-media script is in v4l-utils, contrib/test. It is currently testing
with vivid, vim2m and vimc. Once vicodec is stable enough it will be added
to the test suite as well.

It is already run as part of the daily build using a kernel config with many
debug options enabled.

The daily build summary looks like this:

	virtme: OK: Final Summary: 1981, Succeeded: 1981, Failed: 0, Warnings: 14

	<snip>

	Detailed regression test results are available here:

	http://www.xs4all.nl/~hverkuil/logs/Wednesday-test-media.log
	http://www.xs4all.nl/~hverkuil/logs/Wednesday-test-media-dmesg.log

The daily build is currently only checking for failures in the compliance tests
and ignores warnings. It also ignores issues in the kernel log.

The main problems that I am trying to solve are kernel oopses (should be solved
once all my pending 5.2 pull requests are merged) and vivid warnings (12 of the
warnings are known and work is in progress to fix this, the remainder of the
warnings I need to look at and are likely caused by v4l2-compliance itself).

Note that there is one known remaining kernel oops when testing vimc, but resolving
that takes a lot more time (low-level media life-cycle issues) and the test has
been disabled for now in test-media.

The test can be run on a system (or VM) running the kernel you want to test with
by simply running it as root.

Usually for V4L2/MC changes it is enough to run it as:

sudo test-media mc

If you want the script to unload all media modules first, then add the -unload
option. If you want the script to scan for memory leaks (and the corresponding
debug option is enabled in the kernel), then add the -kmemleak option. To dump
the kernel log when done, add the -dmesg option.

Run test-media without options to see a help text.

Note that 'mc' is shorthand for 'vivid vim2m vimc'.

There are also cec compliance tests and the 'all' test which tests everything
(used in the daily build). The cec tests are slow (especially the cec-pwr tests),
so unless you changed something in the cec subsystem you want to avoid running
those.

The test script assumes that the very latest v4l-utils are installed.

The test can also be run using virtme. I made a virtme.sh script to run the tests.
It assumes that it is run from the top of the kernel tree. You can find the script
here: https://hverkuil.home.xs4all.nl/virtme/

The start of the script documents where to get virtme and how to install it.

This also contains a kernel config that enables various kernel debug options.

I strongly recommend that testing is done using virtme since this allows you
to enable many kernel debug options that you don't want to enable on the host.
Delayed kobject release is a particularly nasty debug option :-)

Note: running virtme when a VMWare instance is also running on the same host
does not work reliably. I don't know why exactly, but the only way I can use virtme
is from inside a VMWare linux guest. Unfortunately, I need to use VMWare Workstation,
so I'm stuck with it.

I hope that during the 5.2 development cycle we can fix the last remaining issues
and this can become part of the regular development process.

Future plans (besides adding vicodec support) is adding support for other drivers
like uvc and perhaps au0828 (for testing the upcoming Media Device Allocator API).
But those tests require real hardware to be connected, and there may be different
variants of the same hardware. We'll see how that will work out.

Regards,

	Hans



[Index of Archives]     [Linux Input]     [Video for Linux]     [Gstreamer Embedded]     [Mplayer Users]     [Linux USB Devel]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [Yosemite Backpacking]

  Powered by Linux