On Tue, Nov 4, 2014 at 3:45 PM, Shuah Khan <shuahkh@xxxxxxxxxxxxxxx> wrote: > On 11/04/2014 12:22 PM, Kees Cook wrote: >> On Tue, Nov 4, 2014 at 9:10 AM, Shuah Khan <shuahkh@xxxxxxxxxxxxxxx> wrote: >>> This patch series adds a new kselftest_install make target >>> to enable selftest install. When make kselftest_install is >>> run, selftests are installed on the system. A new install >>> target is added to selftests Makefile which will install >>> targets for the tests that are specified in INSTALL_TARGETS. >>> During install, a script is generated to run tests that are >>> installed. This script will be installed in the selftest install >>> directory. Individual test Makefiles are changed to add to the >>> script. This will allow new tests to add install and run test >>> commands to the generated kselftest script. >> >> I'm all for making the self tests more available, but I don't think >> this is the right approach. My primary objection is that it creates a >> second way to run tests, and that means any changes and additions need >> to be updated in two places. I'd much rather just maintain the single >> "make" targets instead. Having "make" available on the target device >> doesn't seem too bad to me. Is there a reason that doesn't work for >> your situation? > > Kees, > > My primary objective is to provide a way to install selftests for a > specific kernel release. This will allow developers to run tests for > a specific release and look for regressions. Adding an install target > will also help support local execution of tests in a virtualized > environments. In some cases such as qemu, it is not practical to > expect the target to have support for "make". Once tests are installed > to be run outside the git environment, we need a master script that > can run the tests. Hence the need for a master script that can run > tests. > > We have the ability to run all tests via make kselftest target or > run a specific test using the individual test's run_tests target. > Both of above are necessary to support running tests from the tree. > Embedding run_tests logic in the makefiles doesn't work very well > in the long run. > > We also need a way to run them outside tree. I agree with you that > the way I added the script generation, duplicates the code in individual > run_tests targets and that changes/updates need to be made in both > places. > > Would you be ok with the approach if I fixed the duplicating > problem? I can address the duplication concern easily. Yeah, getting rid of duplication would be much preferred. Thanks! -Kees > >> >> I would, however, like to see some better standardization of the test >> "framework" that we've got in there already. (For example, some >> failures fail the "make", some don't, there are various reporting >> methods for success/failure depending on the test, etc.) > > This is being addressed and I have the framework in linux-kselftest > git next branch at the moment. I do think the above work is part of > addressing the larger framework issues such as being able to run tests > on a target system that might not have "make" support and makes it > easier to use. > > thanks, > -- Shuah > > > -- > Shuah Khan > Sr. Linux Kernel Developer > Samsung Research America (Silicon Valley) > shuahkh@xxxxxxxxxxxxxxx | (970) 217-8978 -- Kees Cook Chrome OS Security -- To unsubscribe from this list: send the line "unsubscribe linux-api" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
