Greg Smith <greg@xxxxxxxxxxxxxxx> writes: > They do have a regression test suite: > http://dev.mysql.com/doc/refman/5.0/en/mysql-test-suite.html > But it's not really clear that they run it on every platform, i.e. > http://ourdelta.org/hidden-tests-of-the-mysql-testsuite They definitely don't run it on every combination of allegedly-supported options. I had to turn on --with-big-tables in the Red Hat build awhile ago, which is probably a good thing anyway (though if so, why isn't it default?); the reason I had to do it was the regression tests started showing obvious failures without it, proving that they don't bother to run any internal tests without it. I'm not sure how thorough our buildfarm coverage is for different option combinations, but the fact that their test suite takes circa four hours to run is *not* an advantage for them in the comparison. They clearly haven't got the resources to run all the cases they ought to. (BTW, that's 4 hours for standard "make check", not any of the optional tests referred to in the above-cited blog entry.) > This supports the rumors I've heard that the development on the database > regularly cheats by just disabling tests that don't work right in some > situations, just so they can ship saying "there's no know issues!". Oh, absolutely. They actually have a standard mechanism built into the test harness for disabling tests that are currently failing, and the set that are so disabled changes with every update. Compare the contents of mysql-test/t/disabled.def in various releases sometime. regards, tom lane -- Sent via pgsql-general mailing list (pgsql-general@xxxxxxxxxxxxxx) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
