On Thu, Apr 5, 2018 at 3:09 AM, Willem Jan Withagen <wjw@xxxxxxxxxxx> wrote: > On 4-4-2018 09:55, Gregory Farnum wrote: >> >> Teuthology is not without problems. The next one we discussed was >> difficulty even running tests for smaller contributors without access >> to an existing lab. This extended a bit to the difficulty of simply >> *developing* tests. There isn’t an immediate solution to this problem, >> but there are some steps people have taken independently. >> >> John wrote a “vstart_runner” framework which supports a reasonable >> subset of the full teuthology commands and is used throughout the >> CephFS test suite (and I assume the manager as well?); this lets you >> run tests against a vstart instance with pretty good fidelity to what >> happens in the lab. Radoslaw and Orit have a similar thing for RGW >> which hasn’t been merged but which that team sometimes uses in test >> development: https://github.com/ceph/ceph/pull/20536 . We should unify >> these efforts if possible and extend them into more tests whenever >> possible. This helps a lot with test development. We’ll of course >> never be able to support running a full rados suite on a single >> machine (just due to the machine time required), but if we switch >> enough tests into a framework that runs in multiple environments >> individual contributors can at least run smoke suites on their own, as >> well as any new tests they create for their contributions. >> PROBLEM TOPIC: unify “tests-without-full-teuthology” frameworks and >> use them more extensively going forward. > > > Hi Greg, > > I'd be interested in that approach for running more FreeBSD tests. > Now I'm just running the 145 tests that come with running 'ctest -j 16'. > > Wanting to go further I boldly attacked teutology, finding out that the > porting threshold was even bigger than porting Ceph. :( > > So any change to run more elaborate tests would be great. > I don't perse have a lab, but I can allocate quite some (virtual) hardware > to run the bigger tests. But then some things need to be less Linux tied > than they are now. > > Now I haven't searched for it, for a long time, but is there any > documentation on how to do/use/run some of the tests that are between > ctest and teutology? Right now I think it's limited to that PR I mentioned and the doc strings in ceph/qa/tasks/vstart_runner.py. I was hoping one of the authors would speak up with a bit more friendly of an introduction, TBH. ;) But keep in mind these are mostly set up for running a single test, or a small set of them, as they are targeted at a literal vstart instance running in the folder you invoke the commands from. If we get teuthology set up to work with the local init system (one of the other threads) that may eventually make things easier to port properly to FreeBSD (I'm not sure what pain points you ran into)? -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
