> -----Original Message----- > From: Paul on Thursday, September 14, 2017 4:09 PM > On 08/18/2017 11:57 AM, Shuah Khan wrote: > > On 08/18/2017 12:48 PM, Bird, Timothy wrote: > >>> -----Original Message----- > >>> From: Shuah Khan [mailto:shuahkh@xxxxxxxxxxxxxxx] > >>> As I am converting tests to TAP13, I am encountering cases where > >>> a test invokes other tests. For example, say test1 invokes test2 > >>> and test3, the output will have nested TAP 13 headers and footers. > >>> > >>> Example: > >>> > >>> TAP version 13 > >>> > >>> test1 output > >>> > >>> TAP version 13 > >>> test2 output > >>> 1..1 > >>> > >>> TAP version 13 > >>> test3 output > >>> 1..1 > >>> > >>> 1..1 > >>> > >>> > >>> This is the case with timers tests. Some tests invoke 3 to 4 tests. > >>> As these tests can be run independently, they all will need to be > >>> converted. So we can't avoid nested TAP headers and footers. > >>> > >>> Do you see any problems with nesting? > >> > >> It's not in the TAP13 protocol itself, but I note that the TAP plugin > >> for Jenkins (tap4j) supports this kind of nesting. I'd have to do some > testing > >> to see exactly how they handle this. For example, the example on the > >> plugin page shows > >> nesting similar to what you're doing, but it omits the header line indicating > >> the TAP protocol used. Your example, by the way, doesn't have any > >> actual test lines (I assume that "test 2 output" is a placeholder for the > >> actual test output lines.) > >> > >> If one program is calling another program, how is the indentation > handled? > >> Would the combined output really look like the above (with subtest > output > >> indented?) I believe the Jenkins TAP plugin relies on indentation to > determine > >> where the subtest output ends. > >> > >> Another question: is the subtest considered a test case of the parent > test? > >> That is, does the following output make sense: > >> > >> TAP version 13 > >> # doing timer tests > >> ok 1 first test > >> # executing sub-test timer-sub1 > >> TAP version 13 > >> ok 1 sub-test one > >> 1..1 > >> ok 2 execute sub-test timer-sub1 > >> ok 3 some other test > >> 1..3 > >> ------------------ > > > > Yes it does make sense, however, it isn't possible to print > > the above indented text from the second test, as this test can > > also be run all by itself. > What if we added a parameter to the ksft_* output functions that could > take an indentation level? Then when the parent test executes the sub test > program, it could provide an int to notify the sub test that it is being run > as a sub test, and at what sub level (to accommodate sub-sub tests and sub- > sub-sub > tests, etc). Or just just pass a string on the command line: $ someprog --output_indent " " Any program that is both called by another program, and calls another program can just add to the string. Either a string or an int would work. I thought about doing something like this. I think only a few programs would need to take an indentation argument through their command line. > Then at the beginning of the sub test, it could set a variable that > represents its sub-level or indentation level, and then plug that variable into > every ksft_* output call. If it doesn't receive any indentation level then it is > being run as a top-level test and can then set the indentation level to 0. > > I considered a global variable but that's a problem when we get into > multithreaded > tests. > > I know it's not a good idea to change the API, but there are fewer calls to > replace > now than later :) Maybe we should just add new calls that print with indent, and only use them where nesting is needed. Again, I suspect nesting is not used widely. I looked at the possibility of using variadic macros to mask the difference in arguments, but we're already using variadic functions, so this won't work, unless we get tricky. -- Tim ��.n��������+%������w��{.n�����{��K ����n�r������&��z�ޗ�zf���h���~����������_��+v���)ߣ�
