On Tue, Feb 16, 2016 at 10:44 PM, Daniel Vetter <daniel@xxxxxxxx> wrote: > On Tue, Feb 16, 2016 at 11:44:52AM +1000, Dave Airlie wrote: >> On 15 February 2016 at 20:06, Daniel Vetter <daniel.vetter@xxxxxxxx> wrote: >> > Hi all, >> > >> > I've already chatted with some of you in private, here's the entire idea with a >> > bit more thought. My motiviation for group maintainership of drm-misc was that I >> > got a bit a guilty feeling the last few vacations/conferences when folks pinged >> > about reviewed/tested pretty patches not landing. But also just increasing the >> > bus factor and sharing the load better is good. And finally a shared misc drm >> > tree would allow fringe drivers to get faster into Dave's drm-next by >> > piggy-packing on top of the one pull request train. And it would also reduce a >> > bit tree proliferation (at one point we had drm-misc/-bridge/-panel/-trivial and >> > may more even). And at least everyone I chatted with seems to like the idea in >> > principle. >> > >> > But what's still open is how to do it exactly. One big change with group >> > maintainership is that you can't rebase a tree anymore. And right now I need to >> > rebase drm-misc fairly often to throw out bad apples again. I think solving that >> > is the important bit to make a shared drm-misc work. A few ideas: >> >> This is kinda what I don't like. I don't want a tree that can't rebase >> out bad stuff >> coming to me that often. If we are being too over eager in merging >> stuff the answer >> is to merge less not try and merge more. If we have a tree where things >> are thrown until they stick I'm not sure the history will ever be nice >> to pull from. >> >> I'd rather consider a staging tree where everything from patchwork can >> get thrown >> into, CIed, but that we cherry-pick fixes things to go back to me from the what >> works category. > > I fully agree on your concern that drm-misc could become a mess - that's > why I think everything bigger than e.g. 2 patches or touching drivers > needs to soak first in a topic branch. We could still get it into drm-misc > with cherry-picks to avoid too many merges. > > The other part where I often had to squash in fixups is build fail on arm. > But I fixed myself with some decent cross-compile toolchain, so I'm > positive that won't happen any more. > > I think a good idea would be that I test-drive this process a bit without > adding more maintainers, i.e. > - no more rebases for drm-misc > - topic branches for everything big > - dutifully testing arm before pushing > > I'll do that over the next few weeks, then we can see how bad it would > look and whether we need more to avoid bad history. > >> I'm finding with i915 for example there is a massive latency in the pipeline now >> waiting for fixes, and the pipeline to the end of drm-intel-next is >> very long and >> hard to figure out what fixes should be pulled back, and how much of the >> driver has been rewritten between the -next and the -fixes pulls. >> >> > - I think CI is super-important. We're starting to finally roll that out for >> > real for i915, and it's catching an awful lot of stuff already. Not yet ready >> > for prime-time on public mailing lists, and for misc we probably can't test >> > every patch before they land like we do for i915. But CI should have veto >> > power before a pull request goes to Dave imo. >> > >> > For non-i915 Daniel Stone and others are working on ARM CI using the generic >> > igt testcases for kms. And I'm open to merging driver-specific tests into igt >> > too, if it makes sense, and e.g. Eric has already pushed some vc4 tests. >> > >> > - Stuff needs to at least compile cleanly before pushing. I've been really bad >> > at that wrt arm drivers with my own drm-misc, but turns out it's fairly easy >> > to get this right: http://blog.ffwll.ch/2016/02/arm-kernel-cross-compiling.html >> > >> > - Bad apples need to be kicked out with reverts, not rebases. I think that's >> > fine for simple patches, and hence those can go directly to drm-misc. But a >> > bunch of subsystem-wide refactorings go in through mis trees, and for those >> > constantly mass-reverting until it's all solid is silly. And ime you need some >> > soak time in a shared tree to iron out bugs with those kind of endeavours. We >> > can address that with ad-hoc&short-lived topic branches which are then again >> > owned by a single maintainer, but automatically pulled into an integration >> > tree. After some soaking time to give CI systems time to crunch through those >> > topic branches can then be merged into the main drm-misc and removed. >> >> I think integration trees are probably the way forward, but I >> understand they come >> with a massive overhead for constant merging, I like the idea of topic >> branches until >> it becomes my turn to spaghetti merge 3-4 features at once. At that >> point I still >> fallback to the slow things down, pick one feature merge it, back the >> others off, >> and I think this is something we should do more off, the pile everything in and >> hope that magical CI will stabilise things doesn't seem to be a responsible >> process to me. > > My idea is that you won't see the integration tree mess ever, but that > tested stuff would land in drm-misc first to resolve conflicts. But really > there never should be, since when a series takes longer than 1-2 weeks to > get ready it's just not ready for merging. And the entire topic branch > should be kicked out. > > Also integration tree mangling is why I suggested we start out using dim > scripts - we're integrating 5-10 branches for drm-intel-nightly since > years, since a few months with multiple maintainers pushing conflict > resolutions, and it all works really well. > > Also agreed that CI won't magically make stuff stabilise - if CI > complaints the rule needs to be to revert or drop the topic branch, not > try to wrestle it until it's less unhappy. That indeed just doesn't work, > and we're learning all that with drm-intel right now (before we had no CI > but ran on hope, which works even less). > >> so I'd like CI to be happening but I'd like it to be happening before the git >> history is baked into stone. > > Yeah that's what we do for drm-intel. At least right now we don't even > have anyone doing arm CI, much less CI before merging. Daniel Stone is > working on it though, so I'm hopeful. But yeah for my own stuff or patches > from intel folks we already throw them all at CI before they land > anywhere. > >> > - Also we can't roll forward to latest drm-next easily with rebases any more. So >> > that needs to happen either right after the pull request lands (when no patch >> > has been merged meanwhile). Or with backmerges, which then need a short commit >> > message as to way the backmerge is needed ("Backmerge because we need >> > $feature" is what I usually type), plus sob line from the committer. And of >> > course the backmerged treed must be stable/non-rebasing and preferrably the >> > backmerge should be a release/pull-request tag. >> > >> > - For tooling I suggest we just go with drm-intel maintainer-tools for a start. >> > Picking dim has a few reasons: >> > * Well tested, documented and fairly complete (includes e.g. bash-completion). >> > * Integrated support for topic branches, including support for git worktree. >> > * Well-exercised and robust integration tree support. >> > Short-term we could add some convenience functions (e.g. for creating/merging >> > drm-misc topic branches). Long-term we might or might not want to have this >> > separated from drm-intel - that should be possible but a bit of work. Also, >> > this way drm-misc would stay at it's current location while we figure things >> > out. Longer term we might also want to look into adding other big drivers >> > into an over drm integration. >> > >> > Of course group maintainership needs an initial group. My experience from >> > drm-intel is that a bigger group of maintainer has benefits: It's clear that >> > part-time maintaining is ok too, with maintainers focusing on their area of >> > interest/expertise and only helping out in other places when there's a gap (due >> > to e.g. vacations). Anyway, here's my thoughts for the starting group: >> > Archit, Jani, Thierry & me as existing maintainers of drm-misc/bridge/panel, >> > Alex&Rob as maintainers of big drivers and engaged in core drm stuff, >> > Daniel Stone so that he has no more excuses to stall on arm drm ci. >> > >> > I think if this goes well we can extend it to more driver maintainers, e.g. >> > Patrick would like to just push gma500 patches to some tree and not fiddle with >> > pull requests all the time himself. >> >> Is pull request sending a big issue? I don't really want a misc tree >> as a way to avoid sending distinct pull requests for drivers. If we do >> this I want >> it to be drm core patches and subsequent fallout only and if we want >> an unmaintained >> drivers patches tree to do that separately. > > Eric and Patrick mentioned in private that pull reqs for drivers is > somewhat a nuisance for them. And iirc Thierry also said he's drowning, so > often only gets around to stuff very late. I don't mind sending pull requests. It just felt overkill to do a PR for a few trivial fixes. If keeping gma500 stuff separated from drm-misc is easier for you and Dave I don't mind doing that. -Patrik > > But really driver trees for small drivers is an entire different topic > that I'd like to maybe keep in mind. But certainly not for initial. > >> I don't really like the misc tree replacing panel/bridge tree either, >> those were nicely >> compartmentalised in my head as trees I pulled but didn't really care >> about as Thierry >> did a good job and I don't have ARM hw that I personally care about. > > Hm, I mentioned that since I've heard complaints from ARM folks about > tree proliferation, so figured we could try something to get things > together a bit more. But since I care about x86 first I don't mind at all > if we keep drm-misc restricted to what I pulled in thus far if you don't > want this. > >> It's important to me that stuff still comes in some sort of >> compartments, as it decides >> for each pull request how much time/effort I put into testing it on my >> end, I'm pretty much >> at the mercy of ARM hw maintainers, but with x86 I still like to smoke >> test it on some >> hardware as much as possible, also with DRM core stuff. > > If we keep bridge/panel out I don't expect stuff will change at all in > that regard for you. drm-misc will still be misc random patches, plus the > oddball treewide refactor. And I'll still try to pace the bigger refactors > reasonable. That needs some coordination among maintainers, but I think > the bigger upside here is to get the oddball ones in with less overhead - > big patch series are much harder to forget about ;-) > >> So I'm still not sure where I stand on this, personally I don't care >> if you take holidays >> and stuff stalls, delays aren't the big enemy here, merging crap is >> much worse than >> delaying. > > One part of my motivation is that this went much, much better than I ever > hoped for in drm-intel. There's teething issues for sure, but it helped > massively in increasing the bus factor, and in reducing people pinging me > directly for every little tiny patch. So much more time for me to look at > patches that are actually important for me to look at wrt "should this go > in or not". > > Anyway no rush at all here for drm-misc, and I think next step is for me > to test-run the "no rebasing" to figure that part out. But still would > like comments/ideas from others, too. > > Cheers, Daniel > -- > Daniel Vetter > Software Engineer, Intel Corporation > http://blog.ffwll.ch > _______________________________________________ > dri-devel mailing list > dri-devel@xxxxxxxxxxxxxxxxxxxxx > https://lists.freedesktop.org/mailman/listinfo/dri-devel _______________________________________________ dri-devel mailing list dri-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/dri-devel
