> -----Original Message----- > From: Shuah Khan on Friday, September 15, 2017 7:14 AM > On 09/15/2017 01:17 AM, Paul Elder wrote: > > On 09/14/2017 08:55 PM, Bird, Timothy wrote: > >> > >> > >>> -----Original Message----- > >>> From: Paul on Thursday, September 14, 2017 4:09 PM > >>> On 08/18/2017 11:57 AM, Shuah Khan wrote: > >>>> On 08/18/2017 12:48 PM, Bird, Timothy wrote: > >>>>>> -----Original Message----- > >>>>>> From: Shuah Khan [mailto:shuahkh@xxxxxxxxxxxxxxx] > >>>>>> As I am converting tests to TAP13, I am encountering cases where > >>>>>> a test invokes other tests. For example, say test1 invokes test2 > >>>>>> and test3, the output will have nested TAP 13 headers and footers. > >>>>>> > >>>>>> Example: > >>>>>> > >>>>>> TAP version 13 > >>>>>> > >>>>>> test1 output > >>>>>> > >>>>>> TAP version 13 > >>>>>> test2 output > >>>>>> 1..1 > >>>>>> > >>>>>> TAP version 13 > >>>>>> test3 output > >>>>>> 1..1 > >>>>>> > >>>>>> 1..1 > >>>>>> > >>>>>> > >>>>>> This is the case with timers tests. Some tests invoke 3 to 4 tests. > >>>>>> As these tests can be run independently, they all will need to be > >>>>>> converted. So we can't avoid nested TAP headers and footers. > >>>>>> > >>>>>> Do you see any problems with nesting? > >>>>> > >>>>> It's not in the TAP13 protocol itself, but I note that the TAP plugin > >>>>> for Jenkins (tap4j) supports this kind of nesting. I'd have to do some > >>> testing > >>>>> to see exactly how they handle this. For example, the example on > the > >>>>> plugin page shows > >>>>> nesting similar to what you're doing, but it omits the header line > indicating > >>>>> the TAP protocol used. Your example, by the way, doesn't have any > >>>>> actual test lines (I assume that "test 2 output" is a placeholder for the > >>>>> actual test output lines.) > >>>>> > >>>>> If one program is calling another program, how is the indentation > >>> handled? > >>>>> Would the combined output really look like the above (with subtest > >>> output > >>>>> indented?) I believe the Jenkins TAP plugin relies on indentation to > >>> determine > >>>>> where the subtest output ends. > >>>>> > >>>>> Another question: is the subtest considered a test case of the parent > >>> test? > >>>>> That is, does the following output make sense: > >>>>> > >>>>> TAP version 13 > >>>>> # doing timer tests > >>>>> ok 1 first test > >>>>> # executing sub-test timer-sub1 > >>>>> TAP version 13 > >>>>> ok 1 sub-test one > >>>>> 1..1 > >>>>> ok 2 execute sub-test timer-sub1 > >>>>> ok 3 some other test > >>>>> 1..3 > >>>>> ------------------ > >>>> > >>>> Yes it does make sense, however, it isn't possible to print > >>>> the above indented text from the second test, as this test can > >>>> also be run all by itself. > >>> What if we added a parameter to the ksft_* output functions that could > >>> take an indentation level? Then when the parent test executes the sub > test > >>> program, it could provide an int to notify the sub test that it is being run > >>> as a sub test, and at what sub level (to accommodate sub-sub tests and > sub- > >>> sub-sub > >>> tests, etc). > >> > >> > >> Or just just pass a string on the command line: > >> $ someprog --output_indent " " > >> > >> Any program that is both called by another program, and calls another > >> program can just add to the string. Either a string or an int would work. > > This would require changing individual tests. We need a solution that can be > implemented in the kselftest framework. I thought of a few off-the wall ideas that I'll list here. These are just by way of brainstorming. = fork wrapper / stdout augmentation = One idea is to have the kselftest framework provide a fork wrapper, that grabs the stdout stream and modifies it for nested programs, as needed. Programs that executed other programs (with nested output) would need to use the fork wrapper instead of regular fork. The parent program would read the output from the child, reformat it with indents and output it as part of its output stream. This would be transparent to the child, and the only modification would be to the calling program (to change to use something like ksft_fork()). = parent-checking = Have ksft functions keep track of pids during execution, and check if their parent pid is in a list of currently executing ksft programs. If so, then use an appropriate indent. I'm imagining an external, persistent file with lines of tuples: pid, indent You could also use some memory mapped area for the data. The list of currently active pids should fit on one page, so this would be a fairly small overhead. This requires that there be consistent use of ksft function calls that can be used to determine start (or at least one call to ksft before the program begins forking) and exit of programs (we might be able to register something with an exit handler). Use of an exit handler assumes that we're doing C. Are there selftests is other languages? Anyway - just tossing ideas out there. -- Tim ��.n��������+%������w��{.n�����{��K ����n�r������&��z�ޗ�zf���h���~����������_��+v���)ߣ�
