On Wed, Apr 13, 2016 at 11:46 AM, John Spray <jspray@xxxxxxxxxx> wrote: > On Wed, Apr 13, 2016 at 7:02 PM, Vasu Kulkarni <vakulkar@xxxxxxxxxx> wrote: >> >> >> On Wed, Apr 13, 2016 at 3:52 AM, John Spray <jspray@xxxxxxxxxx> wrote: >>> >>> Lately, we've instances where ceph.git and ceph-qa-suite.git got >>> slightly out of sync, as we were adding new stuff and interface >>> changes to ceph (especially in cephfs). >>> >>> We used to have two repos (ceph and teuthology), now we have three >>> (ceph, ceph-qa-suite and teuthology). >> >> >> we also have s3 suites in its own repo, we have big files in ceph.com/qa >> which i believe is the right place. > > I'm a bit ignorant of the s3 tests, is there a particular reason they > live separately to the main ceph-qa-suite? Not sure, probably yehuda wanted to make it easier to test with Amazon::S3 and Ceph > > As for the stuff in ceph.com/qa, I of course am not suggesting putting > those bit binaries into the main git repo. It is sort of annoying > that they live out there on a web server, but it's a separate issue > from the ceph/ceph-qa-suite one. > >>> Splitting tests out of >>> teuthology was a good thing, but maybe they should have gone into the >>> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >>> pretty much mirror what we do with ceph, with master vs jewel etc. >>> >>> Historically we had a comparatively static set of workloads in the qa >>> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >>> much with ceph changes. But these days we're adding much more >>> detailed tests, so there's more effort to keep the two in sync. >>> >>> I would personally love to be able to have a single PR that contained >>> my code and the tests for it. What if after Jewel we pulled all of >>> ceph-qa-suite into the ceph repo? >> >> >> I understand the advantage of having same sha between builds and tests, but >> how are we going to guarantee ceph-builds in different places(one built on >> other than ceph.com) will have same sha? > > The thing that needs to match is the sha1 of the ceph code you're > running, and the sha1 you use when pulling the code to test it with. > > I believe you currently have a situation where you have a built RPM > but you're not sure what ceph sha1 it came from, so you're pulling > e.g. the jewel branch of ceph-qa-suite rather than a specific sha1. > That needs fixing independently of ceph/ceph-qa-suite separation: you > have to run the right revision of the tests, irrespective of where > they're getting pulled from. > > Basically, anyone generating packages with versions that don't map to > a ceph sha1 will need to have a way of remembering what ceph sha1 > their packages correspond to. This is not a new requirement, it's the > same issue that was discussed on IRC recently. > >> why not move ceph tests from ceph.git into ceph-qa-suite, Ideally this is >> not an issue with any major release, jewel workunits are supposed to work >> with jewel ceph, > > That would not help, because you still have the same issue of changes > in ceph being out of step with changes in ceph-qa-suite. > > I think your misconception here is to assume that running tests from > the same branch as the code is sufficient. You need to run tests from > the same *revision*. Otherwise, when I add a new test to reproduce a > bug, and fix the bug at the same time, then you will get failures if > you run the newer commit against the older code. This one I think figured out, I agree it needs to match for those rare cases, but the option exists in teuthology with overrides, just need to have right override. So hopefully it shoudn't be an issue anymore. > > John > >> only in rare cases it may cause some issue but easily fixable. All the dev >> runs can point to any qa-sha using the cli option so much less of an issue >> in rare case. > > > >>> >>> We could still enable folks running test changes without having to >>> rebuild ceph packages: the suite sha1 selected when running a >>> teuthology suite could still be different from that used for >>> installing ceph, it's just that it would fetch that sha1 from the ceph >>> repo instead of from a separate repo. >>> >>> John >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
