On 04/11/2016 04:09 PM, Greg KH wrote: > On Mon, Apr 11, 2016 at 02:58:37PM -0400, Sasha Levin wrote: >> On 04/11/2016 02:41 PM, Greg KH wrote: >>> On Mon, Apr 11, 2016 at 01:53:41PM -0400, Sasha Levin wrote: >>>> Quite a few users of the stable trees pointed out that on complex deployments, >>>> where validation is non-trivial, there is little incentive to follow the >>>> stable tree after the product has been deployed to production. There is no >>>> interest in "random" kernel fixes and the only requirements are to keep up >>>> with security vulnerabilities. >>> >>> "random" is in the eye of the beholder :) >> >> It's not "random" for us building the tree, but it's very much "random" >> for the users for see fixes for drivers they've never even heard of before. > > Then why would it matter if they pulled the tree or not? How are you > going to judge which driver fixes to take and which not to? Why not > take them all if they fix bugs? Because some fixes introduce bug on their own? Take a look at how many commits in the stable tree have a "Fixes:" tag that points to a commit that's also in the stable tree. Look at the opposite side of this question: why would anyone take a commit that fixes a bug he doesn't care about? Are the benefits really worth it considering the risks? [snip] >>> Define "important". Now go and look at the tty bug we fixed that people >>> only realized was "important" 1 1/2 years later and explain if you >>> would, or would not have, taken that patch in this tree. >> >> Probably not, but I would have taken it after it received a CVE number. >> >> Same applies to quite a few commits that end up in stable - no one thinks >> they're stable material at first until someone points out it's crashing >> his production boxes for the past few months. > > Yes, but those are rare, what you are doing here is suddenly having to > judge if a bug is a "security" issue or not. You are now in the > position of trying to determine "can this be exploited or not", for > every commit, and that's a very hard call, as is seen by this specific > issue. The stable stuff isn't rare as you might think, even more: the amount of actual CVE fixes that are not in the stable tree might surprise you. This usually happens for the reason you described, we look at a commit that has an innocent subject/description, not CC'ed to stable@ and we figure that it's not stable material until someone shows how to exploit it and requests a CVE. This is not rare. > If you only take things you "know" are issues, well, you miss lots of > fixes that were really security things but only found out later, so > people have to scamble to fix their kernels much faster than they needed > to. I don't only take known issues. I don't claim to have a 100% success rate, but this is the same story as the stable tree and the patch selection there. > Putting up a tree like this isn't going to cause people to update their > trees any quicker. If they haven't been doing it now, they aren't going > to do it with this tree, as their workloads don't allow them to take > updated kernel trees. > > In short, it's not the fact that we have stable trees that are "too big" > for them to take, it's the fact that they just can't take and test > updates properly. They need to fix their processes, it's not a > deficiency on our side from everyone I have talked to, including the > example you gave me off-list :) Pulling 100+ "random" (yes, I used it again) commits would require a very different review process than cherry picking ~5 commits that you can more easily review and validate. This is actually what happens now; projects get to the point they don't want to update their whole kernel tree anymore so that just freezes because they don't want to re-validate the whole thing over and over, but they still cherry pick upstream and out-of-tree commits that they care about. If they added a handful of security commits to cherry pick and carefully review their security will be much better than what happens now. Thanks, Sasha -- To unsubscribe from this list: send the line "unsubscribe stable" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html