On Sun, Jul 22, 2018 at 7:04 AM Sage Weil <sweil@xxxxxxxxxx> wrote: > > Backports are always hard, but I think there are two main categories of > reasons why they can break things: > > (1) is the pattern we saw with 12.2.6, where a feature is introduced with > a bug, a bugfix follows a bit later when it is discovered and fixed, but > the feature backport doesn't realize there is a later fix. > (2) is when a feature is backported completely but there is some > functional second order interaction with the rest of the system that > prevents it from working correctly. > > The latter scenario will always require either a developer who is paying > close attention or a lot of testing to prevent, which is why we should > always be very mindful with backports, especially non-bugfix ones, and > especially ones in complicated code. The first case, though, is IMO a > failure in tooling and process. > > A few possible solutions come to mind: > > 1- Always reference the commit that introduced the bug/regression in the > bugfix commit. Then, when backporting, always search git for the sha1 of > the patch being cherry-pick -x'd. This should turn up the follow-on fix. > (Perhaps some tooling could streamline this?) > > 2- Always mention the follow-on fix as a comment on the pull request that > originally introduced the new code/feature. This requires checking the > original pull request when doing the backport. > > 3- Always open a tracker ticket, even for recent, just-introduced bugfixes > in master. Reference that... somewhere? > > I think the core problem is that when doing feature backports there is not > always a (feature) ticket or other single reference to look at that > captures everything about that feature. Even #2 and using the PR doesn't > seem like a complete solution because often there are multiple pull > requests to introduce a feature. > > It seems like there are two main options: lean on the git history, or lean > on external feature tickets. > > I'm worried about the latter because we frequently have a series of > incremental pieces for a single feature, or related features that build on > one another, so there isn't always a 1:1 mapping between the PRs and a > (hypothetic) feature ticket. We also don't always maintain feature > tickets, so this would be a net increase of process overhead. > > This makes me lean toward leaning on git, and being more pedantic about > ensure any bug fix references the git commit where the issue was > reproduced. The kernel uses a simple > > Fixes: <sha1> > > tag in commits. With that, I think, the missing piece is some > tooling/help to make sure we pay attention to these tags. I see two > scenarios: > > - You are backport foo and need to notice commit bar that has a Fixes: > foo line. > - You already backported foo. Later you commit bar with Fixes: foo, and > need to noticed that foo was already backported and the backport branch > also needs bar. I think you've just introduced a giant knowledge dependency in exchange for avoiding an external tool (with auditing capabilities!). If somebody fixes a bug, what's to say they know what commit introduced it? What if there are two fixes that chain on each other and they don't notice that it's all tied back into a specific feature change, or the backporter doesn't check every single one of those chained backport hashes to see if it's referenced later in the history? What about when one cherry-picks for backports and the hashes no longer have any useful relationship to the source? I understand that having to create issues in tracker.ceph.com can be annoying when you spot a bug two hours after merging something. Nonetheless, bug trackers are a powerful tool for coordinating teams and are MUCH better at helping you relate patches and issues to each other. Plus the backports team already uses Redmine extensively in their workflow, and so do the dev teams. Why put effort into dodging it? On Mon, Jul 23, 2018 at 5:48 AM Alfredo Deza <adeza@xxxxxxxxxx> wrote: > > I do think the two can go hand in hand (is it a requirement to exclude > tooling vs. git commit additions?), but certainly what we lack today > is an issue tracker > that can easily give an overview of something that spawns multiple > issues/backports at once I'm not entirely sure what your standards are here, but Redmine (even as we've quite lazily not customized it) is reasonably capable here. You can create parent tasks (though perhaps not multi-level ancestors?), move issues around, and specify a number of "Related to" fields between otherwise unrelated issues. If we need more specific capabilities to support a particular workflow, we could certainly put more effort into configuring and streamlining that stuff than the nothing we've invested so far. -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
