Hey folks! I just realized I hadn't let anyone know about this yet, so I probably should! If all the robots are working right, from now on, it should be possible for nightly validation events to be created at any point in the cycle. Up till F23, this didn't happen. Nightly validation only happened up until Alpha TC1; after that point we only validated TCs and RCs. Now that nightly composes have a full set of deliverables, it seemed to me that this probably wasn't the best way to go any more. It turns out we can't get rid of TCs and RCs yet but we *do* want to reduce the need to do them, so ideally we would push blocker/FE fixes stable promptly and test nightlies without needing to do candidate composes until late in the milestone cycle when all blockers are fixed. So the new thing that does validation event creation (it's a fedmsg consumer that listens out for new composes and applies a bunch of rules to decide whether to create a validation event for the new compose) *will* now create nightly events even when the current event is a 'candidate' event, according to the same rules as before: * If it's been < 3 days since the last event, don't create a new one * If it's been > 14 days since the last event, create a new one for sure * If it's been > 2 but < 14 days since the last event, create a new one if interesting packages have changed There's a couple of potential issues here: 1) Things may look a bit weird if we get a new nightly event after an 'RC' candidate compose - we would not consider shipping the nightly as the milestone release, but the nightly event would show up as the "current" event, not the RC candidate. My expectation is that this is quite unlikely to happen as we will create RCs quite frequently and the 'interesting package change' check is unlikely to trigger, but if it happens we might need to see how weird it looks to everyone. Also, see 2... 2) The 'interesting package change' check is at present quite dumb and only checks if there's any difference. This means we might actually create a nightly event four days after a candidate event because a package in the nightly compose is *older* than a package in the candidate compose, because we haven't pushed a blocker/FE fix stable yet. This seems wrong, and I think I'll try to make the check a bit more sophisticated (i.e. check that the changed package is *newer*). Doing that entirely correctly is quite difficult, but doing it to a 'good enough' level probably isn't. So yeah, just thought I'd let everyone know not to be surprised if you see nightly validation events start popping up again after the Alpha release, it's expected. If one pops up while we're still finalizing Alpha it's unfortunate, but not entirely unexpected =) If anyone was thinking: "will that screw up testcase_stats?", here's a gold star for smart thinking! The answer is that yes it certainly would have, only I fixed it. The old wikitcms sort code for events cannot sort nightly events mixed in with 'candidate' events - it assumes nightly events are always before candidate events. But testcase_stats now sorts first by page creation date, then by wikitcms sort as a tie- breaker (so it sorts two events created on the same day correctly, or probably does). This is a bit hacky and requires some wiki roundtrips, but hey, it's Wikitcms. Of course it's hacky! If anyone's interested in the code of the event creator, here it is: https://git.fedorahosted.org/cgit/fedora-qa.git/tree/relvalconsumer -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@xxxxxxxxxxxxxxxxxxxxxxx
