On Mon, Nov 20, 2023 at 12:57 PM Alexei Starovoitov <alexei.starovoitov@xxxxxxxxx> wrote: > > On Mon, Nov 20, 2023 at 10:05 AM Andrii Nakryiko <andrii@xxxxxxxxxx> wrote: > > > > Reduce verboseness of test_progs' output in reg_bounds set of tests with > > two changes. > > > > First, instead of each different operator (<, <=, >, ...) being it's own > > subtest, combine all different ops for the same (x, y, init_t, cond_t) > > values into single subtest. Instead of getting 6 subtests, we get one > > generic one, e.g.: > > > > #192/53 reg_bounds_crafted/(s64)[0xffffffffffffffff; 0] (s64)<op> 0xffffffff00000000:OK > > > > Second, for random generated test cases, treat all of them as a single > > test to eliminate very verbose output with random values in them. So now > > we'll just get one line per each combination of (init_t, cond_t), > > instead of 6 x 25 = 150 subtests before this change: > > > > #225 reg_bounds_rand_consts_s32_s32:OK > > > > Given we reduce verboseness so much, it makes sense to do a bit more > > random testing, so we also bump default number of random tests to 100, > > up from 25. This doesn't increase runtime significantly, especially in > > parallelized mode. > > > > With all the above changes we still make sure that we have all the > > information necessary for reproducing test case if it happens to fail. > > That includes reporting random seed and specific operator that is > > failing. Those will only be printed to console if related test/subtest > > fails, so it doesn't have any added verboseness implications. > > Thanks for the quick fix. Applied. > > I also noticed: > #200 reg_bounds_gen_consts_s64_u64:SKIP > #201 reg_bounds_gen_consts_u32_s32:SKIP > #202 reg_bounds_gen_consts_u32_s64:SKIP > #203 reg_bounds_gen_consts_u32_u32:SKIP > #204 reg_bounds_gen_consts_u32_u64:SKIP > > what is the reason for SKIP ? Those are "slow tests". If they don't see `SLOW_TESTS=1` envvar, they will mark themselves as skipped. This patch didn't change this behavior, it was like that before.
