Hi folks, Summary: I propose a new macro (currently named AX_AT_CHECK_PATTERN()) which is like AT_CHECK() except that the expected stdout and stderr arguments may contain awk regular expressions, to allow for output where certain parts are expected to change each run. Any thoughts or comments about this idea? If it was considered "useful" I wouldn't mind if it was incorporated into the standard suite of autotest macros. I've attached a tar file containing the macro and a test harness. Details: I've been using autotest for a few years. Not long after I started using it I had a need to be able to use AT_CHECK where certain parts of the output were expected to change each run (e.g, timestamps). I had experimented with saving the output of AT_CHECK (with [stdout]) and post-processing the output (e.g, using grep to filter out the lines that we expected to change), but found that to be a bit unwieldy. The solution used for a few years was a custom python script which line-by-line compares the saved output against an expected pattern file, and only failing if there was a pattern match. This generally sufficed, but it has the following limitations: - depends upon python (might not be available everywhere) - because of the line-by-line nature, the addition of an extra line of output resulted in a lot of false positive matching because the output and pattern checking got "out of sync". Recently I had the idea to rework the implementation to postprocess the output of diff, and elide from the output "change" blocks where the left-hand output and the right-hand pattern lines matched up. This results in much less output to wade through in testsuite.log if there's only a couple of lines added or removed (a la the traditional AT_CHECK output). E.g, given file 'pattern': 1 2 a? 2 [b-e]+ 3 and file 'output': 1 2 a 2 b 3 4 then 'diff pattern output' is: 2,3c2,3 < 2 a? < 2 [b-e]+ --- > 2 a > 2 b 4a5 > 4 but my diff post-processing just results in: 4a5 > 4 As part of the rework, I wrote the diff-postprocessing code in awk only because I felt that it might be more available than python (or perl), and there's other macros in autoconf and autotest that depend upon awk. What I've ended up with is a macro - AX_AT_CHECK_PATTERN() - which takes arguments as per AT_CHECK() except that the expected stdout and stderr arguments may contain awk regular expressions. This macro depends upon a shell function ax_at_diff_pattern which prerequisite macro defines. The macro works by abusing the implementation of AT_CHECK() by temporarily redefining the variable $at_diff to use my shell function ax_at_diff_pattern(). There's still some outstanding issues: 1. The required quoting is a bit ugly if the expected regex contains square brackets, because m4 wants to use those. 2. I'm not 100% happy with the way I temporarily change $at_diff. Arguably the _AT_CHECK() helper could optionally take an extra argument and do this for us instead; I'm not fussed. regards, Luke.
Attachment:
at_check_pattern-1.tar.gz
Description: GNU Zip compressed data
Attachment:
pgpQXGaCgnIeA.pgp
Description: PGP signature
_______________________________________________ Autoconf mailing list Autoconf@xxxxxxx https://lists.gnu.org/mailman/listinfo/autoconf
