Hi,
there's been some discussion around extending our unit testing framework
to avoid duplication when declaring test functions. Right now, a testing
function has to be declared and then wired up via the test's main
function, which can be a bit annoying. In the thread, René proposes an
alternative that gets rid of this duplication by using macros. And while
that does solve the issue, there were some concerns about things being
too much "magic" while at the same time not being flexible enough.
Part of the discussion revolved around whether we maybe want to have a
proper unit testing framework in our codebase instead of reinventing the
wheel. As I quite liked the "clar" [2] testing framework from back when
I was still developing libgit2 I proposed it as a possible alternative.
This patch series wires up the clar framework as a proof of concept and
converts the strvec test suite to use it.
The magic to avoid the above code duplication is quite self-contained in
a "generate.py" file. This script extracts function declarations from
all unit test suites and then writes those into a "clar.suite" header
file. All that one needs to do is thus to declare a function with a
specific name "test_<suite>__<name>" and then everything else gets wired
up automatically.
Whether this is better than the solution proposed by René is probably a
matter of taste. While I'm not a huge fan of the macro-based solution,
I don't want to veto it either (not that I'd have that power anyway). So
please, you should rather read this as a proof of concept to see how
alternatives could look like such that we have a better picture of where
we want to end up.
Some random thoughts:
- The mandated Python dependency is suboptimal in my opinion.
Rewriting the script in e.g. Perl should be easy enough though, it's
no rocket science.
- I prefer that the proposed solution results in a single binary as
compared to one binary per test system.
- The clar gives us the ability to pick which tests to run via command
line parameters, which I personally like more than picking the
specific binary to run.
- The clar replaces some test assertions that we already have. They
feel a bit more mature, but overall there aren't all that many
assertions available. If we wanted to pick it up, then we'd likely
have to add some more wrappers.
- The clar uses longjmp instead of manually having to `return` from
functions in case there was an assertion failure. This is easier to
work with in my opinion.
Also, note that I only tested this on my Linux machine. I have no clue
whether this works as-is on Windows, but I do know that libgit2 tests
run on Linux, macOS and Windows. So it should work in theory, it's just
a matter of polishing this series.
I'm happy to hear your thoughts on this, even if it ultimately ends up
being shot down.
Patrick
[1]: <85b6b8a9-ee5f-42ab-bcbc-49976b30ef33@xxxxxx>
[2]: https://github.com/clar-test/clar
Patrick Steinhardt (3):
t: import the clar unit testing framework
Makefile: wire up the clar unit testing framework
t/unit-tests: convert strvec tests to use clar
.gitignore | 1 +
Makefile | 36 +-
t/Makefile | 1 +
t/unit-tests/.gitignore | 3 +
t/unit-tests/clar/.github/workflows/ci.yml | 23 +
t/unit-tests/clar/COPYING | 15 +
t/unit-tests/clar/README.md | 329 ++++++++
t/unit-tests/clar/clar.c | 842 +++++++++++++++++++++
t/unit-tests/clar/clar.h | 173 +++++
t/unit-tests/clar/clar/fixtures.h | 50 ++
t/unit-tests/clar/clar/fs.h | 522 +++++++++++++
t/unit-tests/clar/clar/print.h | 211 ++++++
t/unit-tests/clar/clar/sandbox.h | 154 ++++
t/unit-tests/clar/clar/summary.h | 143 ++++
t/unit-tests/clar/generate.py | 267 +++++++
t/unit-tests/clar/test/.gitignore | 5 +
t/unit-tests/clar/test/Makefile | 39 +
t/unit-tests/clar/test/clar_test.h | 16 +
t/unit-tests/clar/test/main.c | 40 +
t/unit-tests/clar/test/main.c.sample | 27 +
t/unit-tests/clar/test/resources/test/file | 1 +
t/unit-tests/clar/test/sample.c | 84 ++
t/unit-tests/{t-strvec.c => strvec.c} | 124 ++-
t/unit-tests/unit-test.c | 16 +
t/unit-tests/unit-test.h | 3 +
25 files changed, 3041 insertions(+), 84 deletions(-)
create mode 100644 t/unit-tests/clar/.github/workflows/ci.yml
create mode 100644 t/unit-tests/clar/COPYING
create mode 100644 t/unit-tests/clar/README.md
create mode 100644 t/unit-tests/clar/clar.c
create mode 100644 t/unit-tests/clar/clar.h
create mode 100644 t/unit-tests/clar/clar/fixtures.h
create mode 100644 t/unit-tests/clar/clar/fs.h
create mode 100644 t/unit-tests/clar/clar/print.h
create mode 100644 t/unit-tests/clar/clar/sandbox.h
create mode 100644 t/unit-tests/clar/clar/summary.h
create mode 100755 t/unit-tests/clar/generate.py
create mode 100644 t/unit-tests/clar/test/.gitignore
create mode 100644 t/unit-tests/clar/test/Makefile
create mode 100644 t/unit-tests/clar/test/clar_test.h
create mode 100644 t/unit-tests/clar/test/main.c
create mode 100644 t/unit-tests/clar/test/main.c.sample
create mode 100644 t/unit-tests/clar/test/resources/test/file
create mode 100644 t/unit-tests/clar/test/sample.c
rename t/unit-tests/{t-strvec.c => strvec.c} (54%)
create mode 100644 t/unit-tests/unit-test.c
create mode 100644 t/unit-tests/unit-test.h
--
2.46.0.dirty
Attachment:
signature.asc
Description: PGP signature
