Hi, Bart In this way, we may end up modifying probably most of the common tools, which in long run can create a bunch of the code for the tests. If everyone (test contributors and tools maintainers) agrees to have such a "test" mode to all the tools, we can go for this approach. -Chaitanya From: linux-block-owner@xxxxxxxxxxxxxxx <linux-block-owner@xxxxxxxxxxxxxxx> on behalf of Bart Van Assche <bvanassche@xxxxxxx> Sent: Thursday, February 14, 2019 3:26 PM To: Johannes Thumshirn; Theodore Y. Ts'o Cc: lsf-pc@xxxxxxxxxxxxxxxxxxxxxxxxxx; linux-block@xxxxxxxxxxxxxxx; linux-fsdevel@xxxxxxxxxxxxxxx Subject: Re: [LSF/MM TOPIC] improving storage testing On Thu, 2019-02-14 at 11:55 +0100, Johannes Thumshirn wrote: > On Wed, Feb 13, 2019 at 01:07:54PM -0500, Theodore Y. Ts'o wrote: > > Also, there are expectations about minimum versions of bash that can > > be supported; but there aren't necessarily for other components such > > as nvme-cli, and I suspect that it is due to the use of a overly new > > version of nvme-cli from its git tree. Is that supposed to work, or > > should I constrain myself to whatever version is being shipped in > > Fedora or some other reference distribution? More generally, what is > > the overall expectations that should be expected? xfstests has some > > extremely expansive set of sed scripts to normalize shell script > > output to make xfstests extremely portable; will patches along similar > > lines something that we should be doing for blktests? > > I think this is the root cause of the problems you've sent out mails for this > week. A lot of blktests test need filtering. See [1] as an example. > > [1] https://github.com/osandov/blktests/pull/34 https://avatars0.githubusercontent.com/u/5319409?s=400&v=4 Add filter function for nvme discover by frankenmichl · Pull Request #34 · osandov/blktests github.com Several NVMe tests (002, 016, 017) used a pipe to a sed call filtering the output. This call is moved to a new filter function nvme/rc and the calls to sed are replaced by this function. Additional... Hi Johannes, Output of tools like nvme-cli is not an ABI although an ABI is what is required to make blktests work reliably. One possible approach is to modify nvme-cli such that it has two output formats: one output format that is intended for humans and another that is easy to parse by software. I think we should consider that approach and compare it to using sed scripts. Bart.
