On Wed, Feb 20, 2019 at 11:35:00AM -0800, Omar Sandoval wrote: > > It should fail on older cli versions, we want latest cli/tools to be > > used when running these testcases so that all the latest tools are > > also been tested for the compliance, else we might be running tests > > with any bugs with older version(s) which are fixed already. > > > > Actually I'd like to add quick nvme-cli version check at the start of > > each test that will certainly help/force user to have a latest version. > > Perfect, I was going to suggest adding a check for the minimum version > needed for each test. It would be great if we could document what is (a) the minimum version for things to work at all, and (b) the ideal version to use so that tests aren't being skipped due to version constraints and so we are testing the kernel as much as possible. Ideally, tests should either be designed primarily to test userspace, or to test the kernel, so that people who want to use blktests aren't having to build the latest versions of userspace. (As an aside, Bart has told me that the failures I'm seeing with srp/002 and srp/011 are due to the fact that I'm using a version of multipathd which is too old. Apparently both the version of multipathd in Debian stable *and* Debian unstable are still too old, since they are still failing. Next stop, building multipathd from the git repo.... Grrr......) - Ted
