On 11/04/2014 12:22 PM, Kees Cook wrote: > On Tue, Nov 4, 2014 at 9:10 AM, Shuah Khan <shuahkh@xxxxxxxxxxxxxxx> wrote: >> This patch series adds a new kselftest_install make target >> to enable selftest install. When make kselftest_install is >> run, selftests are installed on the system. A new install >> target is added to selftests Makefile which will install >> targets for the tests that are specified in INSTALL_TARGETS. >> During install, a script is generated to run tests that are >> installed. This script will be installed in the selftest install >> directory. Individual test Makefiles are changed to add to the >> script. This will allow new tests to add install and run test >> commands to the generated kselftest script. > > I'm all for making the self tests more available, but I don't think > this is the right approach. My primary objection is that it creates a > second way to run tests, and that means any changes and additions need > to be updated in two places. I'd much rather just maintain the single > "make" targets instead. Having "make" available on the target device > doesn't seem too bad to me. Is there a reason that doesn't work for > your situation? Kees, My primary objective is to provide a way to install selftests for a specific kernel release. This will allow developers to run tests for a specific release and look for regressions. Adding an install target will also help support local execution of tests in a virtualized environments. In some cases such as qemu, it is not practical to expect the target to have support for "make". Once tests are installed to be run outside the git environment, we need a master script that can run the tests. Hence the need for a master script that can run tests. We have the ability to run all tests via make kselftest target or run a specific test using the individual test's run_tests target. Both of above are necessary to support running tests from the tree. Embedding run_tests logic in the makefiles doesn't work very well in the long run. We also need a way to run them outside tree. I agree with you that the way I added the script generation, duplicates the code in individual run_tests targets and that changes/updates need to be made in both places. Would you be ok with the approach if I fixed the duplicating problem? I can address the duplication concern easily. > > I would, however, like to see some better standardization of the test > "framework" that we've got in there already. (For example, some > failures fail the "make", some don't, there are various reporting > methods for success/failure depending on the test, etc.) This is being addressed and I have the framework in linux-kselftest git next branch at the moment. I do think the above work is part of addressing the larger framework issues such as being able to run tests on a target system that might not have "make" support and makes it easier to use. thanks, -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Samsung Research America (Silicon Valley) shuahkh@xxxxxxxxxxxxxxx | (970) 217-8978 -- To unsubscribe from this list: send the line "unsubscribe linux-api" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
