On Mon, Oct 23, 2023 at 2:41 PM Christian Couder <christian.couder@xxxxxxxxx> wrote: > > On Fri, Oct 20, 2023 at 3:16 PM Achu Luma <ach.lumap@xxxxxxxxx> wrote: > > > > Dear Git Community and Mentors, > > > > I hope this email finds you well. As a follow-up to my previous application, I'd like to provide additional details on the process of migrating existing unit tests from the t/helper/ directory to the new Git unit test framework. > > Thanks for these details! > > > -- Identify Target Unit Tests: Start by identifying the specific unit tests in the t/helper/ directory that we want to port to the new Git unit test framework. Ensure that the tests are suitable for migration and that the benefits of doing so outweigh the effort(By avoiding integration tests). The following points have been developed with on going work on the unit-tests framework visible here: > > > > 1- https://lore.kernel.org/git/0169ce6fb9ccafc089b74ae406db0d1a8ff8ac65.1688165272.git.steadmon@xxxxxxxxxx/ > > 2- https://github.com/steadmon/git/blob/unit-tests-asciidoc/Documentation/technical/unit-tests.adoc > > Maybe if you have time you could add some descriptions or comments > related to the above emails and documents. For example you could tell > what the new unit test framework will be like, how the unit tests will > look like, etc. Maybe a short overview would be nice. > sure, 1- https://lore.kernel.org/git/0169ce6fb9ccafc089b74ae406db0d1a8ff8ac65.1688165272.git.steadmon@xxxxxxxxxx/ : The emails highlight the significant milestones achieved in defining and testing the custom TAP framework for writing git unit tests. It also contains some examples of implementation such as that of STRBUF_INIT with output: ok 1 - static initialization works 1..1 2- https://github.com/steadmon/git/blob/unit-tests-asciidoc/Documentation/technical/unit-tests.adoc: From this technical doc, the new unit test framework in the Git project represents a significant enhancement, introducing a systematic and efficient approach to unit testing. The custom git TAP implementation was selected from several alternatives based on strict criteria as the most suitable test framework for porting the unit tests. The unit tests are written in pure C, eliminating the need for the previous shell/test-tool helper setup, simplifying test configuration, data handling, and reducing testing runtime. Each unit test is encapsulated as a function and employs a range of predefined check functions for validation. These checks can evaluate conditions, compare integers or characters, validate strings, and more, providing comprehensive coverage for test scenarios. When a test is run using the TEST() macro, it undergoes a series of checks, and if any check fails, a diagnostic message is printed to aid in debugging. This diagnostic output includes information about the specific check that failed, the file and line number where it occurred, and a clear comparison of the expected and actual values. Such detailed reporting simplifies the identification and resolution of issues, contributing to codebase stability. Additionally, the framework supports features like skipping tests with explanations, sending custom diagnostic messages using test_msg(), and marking known-to-fail checks using TEST_TODO(). This flexibility allows developers to tailor their tests to specific scenarios while ensuring a comprehensive testing suite. > You could also try to apply the patches in the series that adds the > test framework, or alternatively use the 'seen' branch where the > series has been merged, and start playing with it by writing, or > porting, a small example test. > ok, I think I can push a patch for one. > > -- Create a New C Test File: For each unit test I plan to migrate, create a new C source file (.c) in the Git project's test suite directory(t/unit-tests). Name it appropriately to reflect the purpose of the test. > > Could you provide an example of what the new name would be for an > existing test that is worth porting? > Sure... let's consider an existing unit test in t/helper directory such as t/helper/test-date.c or its shell named t0006-date.sh, which is part of the current shell-based test suite. In the context of the new unit testing framework, this test could be reimagined and renamed as "t-date.c". The "t-" prefix is typically used for test program files in Git, and "date" is retained to reflect the nature of the tests within this suite. > > -- Include Necessary Headers:In the new C test file, include the necessary Git unit test framework headers. Typically, this includes headers like "test-lib.h" and others relevant to the specific test. > > #include "test-lib.h" > > Maybe you could continue the above example and tell which headers > would be needed for it? > > > -- Convert Test Logic: Refactor the test logic from the original Shell script into the new C-based test format. Use the testing macros provided by the Git unit test framework, such as test_expect_success, test_expect_failure, etc., to define the tests. > > test_expect_success("simple progress display", "{ > > // Test logic here... > > }"); > > Ok, a simple example would be nice too. > we can continue with the example used for naming: test-date.c. a typical t-date.c unit test would look like the following: -- #include "test-lib.h" #include "date.h" -- date.h here is a necessary header file. Now refactoring the test logic from the original shell script: -- #include "test-lib.h" #include "date.h" static void test_parse_dates(void) { const char *dates[] = { "invalid_date", "2023-10-17 10:00:00 +0200", NULL }; for (const char **argv = dates; *argv; argv++) { check_int(parse_dates((const char *[]){ *argv, NULL }), 0); } } -- > > -- Add Test Descriptions: Provide clear and informative descriptions for each test using the testing macros. These descriptions will help in identifying the purpose of each test when the test suite is run. > > This would seem to be part of the previous step, as you would have to > provide a description when using the testing macro. But Ok. > > > -- Define a Test Entry Point: Create a cmd_main function as the entry point for the C-based tests. Inside this function, include the test functions using the testing macros. > > int cmd_main(int argc, const char **argv) { > > // Test functions... > > return test_done(); > > } > > Yeah, continuing an example would be nice. > Continuing, we can add a test entrance as follows: -- #include "test-lib.h" #include "date.h" static void test_parse_dates(void) { const char *dates[] = { "invalid_date", "2023-10-17 10:00:00 +0200", NULL }; for (const char **argv = dates; *argv; argv++) { check_int(parse_dates((const char *[]){ *argv, NULL }), 0); } } int main(int argc UNUSED, const char **argv UNUSED) { TEST(test_parse_dates, "Test date parsing"); return test_done(); } -- A typical unit tests with the custom TAP framework would look something like above. This might run in theory but I have not yet run it as I used it here just for demonstration. The unit tests can be built using "make unit-tests." Additionally, Makefile can be modified to add the file to the build: -- UNIT_TEST_PROGRAMS += t-date -- > > -- Ensure TAP Format Output: Ensure that the C-based tests produce output in the Test Anything Protocol (TAP) format. This format includes the test name, status (ok or not ok), and any diagnostic information. > > That means using TEST* macros in the cmd_main() function, as they > should do the right thing or is there more to be done here? > > > -- Test Interaction: Ensure that the migrated tests interact correctly with the new Git unit test framework and any other tests that may be relevant. Consider dependencies and interactions with other parts of the Git project. > > I am not sure what work would be needed here. Is there more to do than > compiling the test files? Having an example would be nice. > > > -- Test Execution: Run the migrated tests to verify that they produce the expected results when executed as part of the Git project's test suite. Use the Git testing framework's test runners to execute the tests. > > Ok. > > > -- Documentation Update: Update the Git project's documentation to reflect the changes made during the migration. Include a reference to the original unit tests in the t/helper/ directory and indicate that these tests have been ported to the new Git unit test framework. > > I am not sure that we would want that. I think we might instead want > to document things in t/helper/ that we don't want to port to the new > unit test framework and why. > Ok noted. > > By following these points, I think I can successfully port existing unit tests from the t/helper/ directory to use the new Git unit test framework. This migration helps standardize and streamline the testing process within the Git project, improving code quality and maintainability. > > Yeah! > > > Next Steps: > > > > I am eager to discuss these suggestions and collaborate with the Git community to ensure the success of this project. I will continue to engage with the community, seek guidance, and refine my proposal as per your suggestions. > > I look forward to the opportunity to contribute to the Git project and help make it even more robust and reliable. > > Thanks for this application and sorry for the late answer! > > Best, > Christian. I look forward to feedback and better understanding of the new testing framework. BR, Achu Luma.
