On 29/08/2015 07:46, Abhishek Varshney wrote:> Hi Loic, > > How about marking the PRs which pass integration/upgrade tests with a prefix PT (Passed Tests) or something in the title after they are merged into the stable branch. This is probably how we can do it: > > 1. While preparing an integration branch, also get all the PRs which are merged but do not have PT as prefix. These are essentially the PRs which have bypassed tests. Nathan had a similar idea and asked that the "Integration Passed" status is added to the Backport tracker (see http://tracker.ceph.com/issues/11824). We should start using it. > 2. Perform integration/update tests on all the other open PRs in the integration branch. > 3. If the integration branch passes all the tests, mark all the PRs as PT, including the ones which had bypassed tests earlier. Now, we know that the PRs which had bypassed tests are bug free. That makes senses. We should amend http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_populate_the_integration_branch to also select merged commits that have not seen tests. Note that the hammer branch also is tested, automatically (the test plan is documented at http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_monitor_the_automated_tests_AKA_nightlies). So, once a PR is merged, it will eventually be tested in this way. > 4. If the integration branch encounters failures, we know it could be because of the PRs which had bypassed tests and we know what those PRs are from step 1. This is going in the right direction :-) That's a lot of manual updating though and it would be great of teuthology could update the issues / PRs with test results so we don't have to manually maintain that inventory. Cheers > > Thanks for the clarification on this scenario. > > Regards > Abhishek > > On Sat, Aug 29, 2015 at 2:13 AM, Josh Durgin <jdurgin@xxxxxxxxxx <mailto:jdurgin@xxxxxxxxxx>> wrote: > > On 08/28/2015 12:16 PM, Loic Dachary wrote: > > Hi Abhishek, > > We've just had an example of a backport merged into hammer although it did not follow the procedure : https://github.com/ceph/ceph/pull/5691 > > It's a key aspect of backports : we're bound to follow procedure, but developers are allowed to bypass it entirely. It may seem like something leading to chaos and frustration but it turns out to be exactly the opposite. In a nutshell, it would be constant source of frustration for developers to learn and obey the rules documented at http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO because it would not benefit them significantly. It would also be a problem for us, backporters, because developers would not be as interested in backporting and our workload would significantly increase. > > When a developer prepares a backport on his / her own, we update the pull request and the issues to obey the procedure so the (s)he does not have to. Sure, it's a little tedious but it's a small price to pay for the benefit of having a backport being dealt with. That's what I did for https://github.com/ceph/ceph/pull/5691 : updaging the corresponding issues, adding cross references to the pull request. > > Samuel Just felt confident enough about the backport that it did not need a rados run to verify it does the right thing. Since it's ultimately Sam's responsibility, that's also ok. The only thing we need to keep in mind when analyzing the next rados run is that this backport did not pass yet. We don't have a way to mark commits that bypassed tests just yet, if you have ideas let us know :-) > > > That was me merging it based on my local testing. I'll keep an eye out > for any fallout in the hammer runs. > > Thanks for keeping everything updated Loic! > Josh > > > > ------------------------------------------------------------------------------------------------------------------------------------------ > > This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. Although Flipkart has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments > -- Loïc Dachary, Artisan Logiciel Libre
Attachment:
signature.asc
Description: OpenPGP digital signature
