On Fri, Nov 4, 2022 at 12:20 PM Daniel Latypov <dlatypov@xxxxxxxxxx> wrote: > rmoar@xxxxxxxxxx was working on them and should hopefully be able to > send them out real soon. > You should get CC'd on those. > > I think the follow-up work is just crafting an example parser input > file and iterating until > $ ./tools/testing/kunit/kunit.py parse < /tmp/example_input > produces our desired results. I briefly played around with this on top of https://lore.kernel.org/linux-kselftest/20221104194705.3245738-1-rmoar@xxxxxxxxxx/ But instead, I tried to make it so kunit.py parse could be just given a directory as input so we don't have to do `find | xargs cat`. I can't get module tests to work locally, so I'll replicate the setup with some regular files. Say we have /tmp/results with two files: file1 contains # Subtest: suite1 KTAP version 1 1..1 ok 1 test ok 1 suite1 and file2 contains # Subtest: suite2 KTAP version 1 1..1 ok 1 test ok 1 suite2 I think this should match what module output in debugfs would look like. A key thing to note: the top level test counter resets between modules, iirc. The patch below makes this almost work $ ./tools/testing/kunit/kunit.py parse /tmp/results/ ============================================================ ==================== suite2 (1 subtest) ==================== [PASSED] test ===================== [PASSED] suite2 ====================== ==================== suite1 (1 subtest) ==================== [PASSED] test [ERROR] Test: suite1: Expected test number 2 but found 1 ===================== [PASSED] suite1 ====================== ============================================================ Testing complete. Ran 2 tests: passed: 2, errors: 1 But there's a few issues a. the error about multiple top level tests with "test number 1" b. how do we make this handle kernel output with prefixes (timestamps) c. and what happens if files each have different length prefixes? diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index 4d4663fb578b..c5b2eb416c2d 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -189,6 +189,14 @@ def _map_to_overall_status(test_status: kunit_parser.TestStatus) -> KunitStatus: return KunitStatus.SUCCESS return KunitStatus.TEST_FAILURE +def _kernel_output_from_dir(dir: str) -> Iterable[str]: + yield 'KTAP version 1' + for root, dirs, files in os.walk(dir): + for sub in files: + with open(os.path.join(root, sub), errors='backslashreplace') as f: + for line in f: + yield line + def parse_tests(request: KunitParseRequest, metadata: kunit_json.Metadata, input_data: Iterable[str]) -> Tuple[KunitResult, kunit_parser.Test]: parse_start = time.time() @@ -496,6 +504,8 @@ def main(argv): if cli_args.file is None: sys.stdin.reconfigure(errors='backslashreplace') # pytype: disable=attribute-error kunit_output = sys.stdin + elif os.path.isdir(cli_args.file): + kunit_output = _kernel_output_from_dir(cli_args.file) else: with open(cli_args.file, 'r', errors='backslashreplace') as f: kunit_output = f.read().splitlines()
