On Mon, Nov 20, 2023 at 10:05 AM Andrii Nakryiko <andrii@xxxxxxxxxx> wrote: > > Reduce verboseness of test_progs' output in reg_bounds set of tests with > two changes. > > First, instead of each different operator (<, <=, >, ...) being it's own > subtest, combine all different ops for the same (x, y, init_t, cond_t) > values into single subtest. Instead of getting 6 subtests, we get one > generic one, e.g.: > > #192/53 reg_bounds_crafted/(s64)[0xffffffffffffffff; 0] (s64)<op> 0xffffffff00000000:OK > > Second, for random generated test cases, treat all of them as a single > test to eliminate very verbose output with random values in them. So now > we'll just get one line per each combination of (init_t, cond_t), > instead of 6 x 25 = 150 subtests before this change: > > #225 reg_bounds_rand_consts_s32_s32:OK > > Given we reduce verboseness so much, it makes sense to do a bit more > random testing, so we also bump default number of random tests to 100, > up from 25. This doesn't increase runtime significantly, especially in > parallelized mode. > > With all the above changes we still make sure that we have all the > information necessary for reproducing test case if it happens to fail. > That includes reporting random seed and specific operator that is > failing. Those will only be printed to console if related test/subtest > fails, so it doesn't have any added verboseness implications. Thanks for the quick fix. Applied. I also noticed: #200 reg_bounds_gen_consts_s64_u64:SKIP #201 reg_bounds_gen_consts_u32_s32:SKIP #202 reg_bounds_gen_consts_u32_s64:SKIP #203 reg_bounds_gen_consts_u32_u32:SKIP #204 reg_bounds_gen_consts_u32_u64:SKIP what is the reason for SKIP ?
