Ævar Arnfjörð Bjarmason <avarab@xxxxxxxxx> writes: > ... I.e. as a mechanism for > mitigating mistakes and catching obscure edge cases just being more > careful or having things sit in 'next' for longer has, I think, proved > itself to not be an effective method (not just in this case, but a few > similar cases). I tend to agree. Given that people do not discover possible regression that will affect even after a change hits 'master', cooking in 'next' alone would not be all that effective. But that does not mean we shouldn't cook in 'next' at all. Especially the previous cycle, I was experimenting with a tweak in my workflow to have topics in 'next' to cook for one week and have them graduate to 'master' unless we saw regression in a week. Previously, I tended to keep topics on the larger side in 'next' and we did see "oops we found this after the topic hit 'next' and here is a fix-up" to them, which I think helped to catch bugs before it broke 'master'. > I'm not sure what the solution is exactly, but I'm pretty sure it > involves more controlled exposure to the wild (e.g. shipping certain > things as feature flags first), not deferring that exposure for long > periods, which is what having things sit it "next" for longer amounts > to. To be fair, "is the hook invoked with its standard output stream connected to the original standard output of the main process?" is not something either of you cared while working on and reviewing the changes, and releases based on 'next' $BIGCOMPANY have its users use internally wouldn't have helped to catch this particular regression, as the primary reason we didn't think of it as a problem is because internal users of $BIGCOMPANY tend to be more monoculture than folks in the wild. So the fundamental solution would be to find a way to involve those who found the regression after a release was done in the development process at a much earlier stage. I do not offhand know how to get there. It may be very hard for us to do with end-users, who typically have only one instance of Git they use and a single valuable repository they cannot subject to "experiments". They may depend on certain aspects of the behaviour of the current version, but they lack an environment to try out our new version to see if we broke them. They probably may not even be aware of what they are relying on, just as we are unaware of their dependence. But for toolsmiths who integrate their gears with Git, there may be something we can do. Perhaps we can start giving "works with Git" badge out (we need to control its use with some kind of trademark registration), to those who "maintain X that enhances Git" when they regularly test their ware with 'next' or much earlier. And when they stop us before unleashing a possible regression by reporting bugs early, give them a "star", so that their "works with Git *****" logo can boast how much contribution in testing they are maing upstream. Or something like that, perhaps?
