On Fri, Mar 07, 2025 at 08:42:46AM -0800, Rob Clark wrote: > On Tue, Sep 24, 2024 at 5:27 AM Vignesh Raman > <vignesh.raman@xxxxxxxxxxxxx> wrote: > > > > Hi, > > > > On 12/09/24 11:18, Dmitry Baryshkov wrote: > > > On Mon, Sep 09, 2024 at 07:34:04AM GMT, Rob Clark wrote: > > >> On Mon, Sep 9, 2024 at 2:54 AM Dmitry Baryshkov > > >> <dmitry.baryshkov@xxxxxxxxxx> wrote: > > >>> > > >>> On Mon, 9 Sept 2024 at 10:50, Maxime Ripard <mripard@xxxxxxxxxx> wrote: > > >>>> > > >>>> Hi, > > >>>> > > >>>> On Tue, Jul 09, 2024 at 01:27:51AM GMT, Dmitry Baryshkov wrote: > > >>>>> On Mon, 8 Jul 2024 at 21:38, Rob Clark <robdclark@xxxxxxxxx> wrote: > > >>>>>> > > >>>>>> On Mon, Jul 8, 2024 at 1:52 AM Daniel Vetter <daniel.vetter@xxxxxxxx> wrote: > > >>>>>>> > > >>>>>>> On Fri, Jul 05, 2024 at 12:31:36PM -0700, Rob Clark wrote: > > >>>>>>>> On Fri, Jul 5, 2024 at 3:36 AM Daniel Vetter <daniel.vetter@xxxxxxxx> wrote: > > >>>>>>>>> > > >>>>>>>>> On Thu, Jul 04, 2024 at 08:40:26AM -0700, Rob Clark wrote: > > >>>>>>>>>> On Thu, Jul 4, 2024 at 7:08 AM Daniel Vetter <daniel.vetter@xxxxxxxx> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>> On Tue, Jul 02, 2024 at 05:32:39AM -0700, Rob Clark wrote: > > >>>>>>>>>>>> On Fri, Jun 28, 2024 at 10:44 AM Daniel Vetter <daniel@xxxxxxxx> wrote: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> On Thu, Jun 27, 2024 at 11:51:37AM -0700, Rob Clark wrote: > > >>>>>>>>>>>>>> On Wed, Jun 26, 2024 at 10:47 AM Daniel Vetter <daniel@xxxxxxxx> wrote: > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> On Wed, Jun 26, 2024 at 11:38:30AM +0300, Dmitry Baryshkov wrote: > > >>>>>>>>>>>>>>>> On Wed, Jun 26, 2024 at 09:32:44AM GMT, Daniel Vetter wrote: > > >>>>>>>>>>>>>>>>> On Mon, Jun 24, 2024 at 10:25:25AM -0300, Helen Koike wrote: > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> On 24/06/2024 02:34, Vignesh Raman wrote: > > >>>>>>>>>>>>>>>>>>> Hi, > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> On 15/03/24 22:50, Rob Clark wrote: > > >>>>>>>>>>>>>>>>>>>> Basically, I often find myself needing to merge CI patches on top of > > >>>>>>>>>>>>>>>>>>>> msm-next in order to run CI, and then after a clean CI run, reset HEAD > > >>>>>>>>>>>>>>>>>>>> back before the merge and force-push. Which isn't really how things > > >>>>>>>>>>>>>>>>>>>> should work. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> This sounds more like you want an integration tree like drm-tip. Get msm > > >>>>>>>>>>>>>>>>> branches integrated there, done. Backmerges just for integration testing > > >>>>>>>>>>>>>>>>> are not a good idea indeed. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> But AFAIU this doesn't help for pre-merge testing, ie. prior to a > > >>>>>>>>>>>>>> patch landing in msm-next > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> My idea was to have a drm-ci-next managed similar to drm-misc-next, if > > >>>>>>>>>>>>>> we have needed drm/ci patches we could push them to drm-ci-next, and > > >>>>>>>>>>>>>> then merge that into the driver tree (along with a PR from drm-ci-next > > >>>>>>>>>>>>>> to Dave). > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> I guess I'm confused about what kind of pre-merge testing we're talking > > >>>>>>>>>>>>> about then ... Or maybe this just doesn't work too well with the linux > > >>>>>>>>>>>>> kernel. The model is that you have some pile of trees, they're split up, > > >>>>>>>>>>>>> and testing of all the trees together is done in integration trees like > > >>>>>>>>>>>>> linux-next or drm-tip. > > >>>>>>>>>>>> > > >>>>>>>>>>>> pre-merge: for msm we've been collecting up patches from list into a > > >>>>>>>>>>>> fast-forward MR which triggers CI before merging to msm-next/msm-fixes > > >>>>>>>>>>>> > > >>>>>>>>>>>> Ideally drm-misc and other trees would do similar, we'd catch more > > >>>>>>>>>>>> regressions that way. For example, in msm-next the nodebugfs build is > > >>>>>>>>>>>> currently broken, because we merged drm-misc-next at a time when > > >>>>>>>>>>>> komeda was broken: > > >>>>>>>>>>>> > > >>>>>>>>>>>> https://gitlab.freedesktop.org/drm/msm/-/jobs/60575681#L9520 > > >>>>>>>>>>>> > > >>>>>>>>>>>> If drm-misc was using pre-merge CI this would have been caught and the > > >>>>>>>>>>>> offending patch dropped. > > >>>>>>>>>>> > > >>>>>>>>>>> That sounds more like we should push on the drm-misc pre-merge CI boulder > > >>>>>>>>>>> to move it uphill, than add even more trees to make it even harder to get > > >>>>>>>>>>> there long term ... > > >>>>>>>>>>> > > >>>>>>>>>>> Short term it helps locally to have finer trees, but only short term and > > >>>>>>>>>>> only very locally. > > >>>>>>>>>> > > >>>>>>>>>> The path to have fewer trees (ideally only one for all of drm) is to > > >>>>>>>>>> use gitlab MRs to land everything :-) > > >>>>>>>>>> > > >>>>>>>>>> drm-ci-next is only a stop-gap.. but one that we need. The > > >>>>>>>>>> ${branchname}-external-fixes trick covers _most_ cases where we need > > >>>>>>>>>> unrelated patches (ie. to fix random ToT breakage outside of drm or in > > >>>>>>>>>> core drm), but it doesn't help when the needed changes are yml > > >>>>>>>>>> (because gitlab processes all the yml before merging the > > >>>>>>>>>> -external-fixes branch). This is where we need drm-ci-next, otherwise > > >>>>>>>>>> we are having to create a separate MR which cherry-picks drm/ci > > >>>>>>>>>> patches for doing the CI. This is a rather broken process. > > >>>>>>>>> > > >>>>>>>>> So what I don't get is ... if we CI drm-misc, how does that not help > > >>>>>>>>> improve the situation here? Step one would be post-merge (i.e. just enable > > >>>>>>>>> CI in the repo), then get MRs going. > > >>>>>>>> > > >>>>>>>> I guess post-merge is better than nothing.. but pre-merge is better. > > >>>>>>>> > > >>>>>>>> post-merge can work if you have a "sheriff" system where someone > > >>>>>>>> (perhaps on a rotation) is actively monitoring results and "revert and > > >>>>>>>> ask questions later" when something breaks. Pre-merge ensures the > > >>>>>>>> interested party is involved in the process ;-) > > >>>>>>> > > >>>>>>> So ... make that happen? And it doesn't have to be for all of drm-misc, > > >>>>>>> mesa after all switched over to MR also on a bit a driver/area basis. So > > >>>>>>> agreeing among all drm-ci folks to use gitlab MR in drm-misc for pre-merge > > >>>>>>> testing shouldn't be that hard to make happen. And unlike a separate > > >>>>>>> branch it's not some kind of detour with a good chance to get stuck in a > > >>>>>>> local optimum. > > >>>>>> > > >>>>>> Tree vs branch doesn't really have much in the way of distinction, > > >>>>>> modulo gitlab permissions. In that it doesn't do much good if drm/ci > > >>>>>> patches are landing on a different branch. > > >>>>>> > > >>>>>> I guess what you are suggesting is that we have a single tree/branch > > >>>>>> that drm/ci + drm/msm + (plus whoever else wants to get in on the > > >>>>>> drm/ci, so probably at least vkms) lands patches into via gitlab MRs? > > >>>>> > > >>>>> This again brings a separate CI-enabled tree. I think it has been > > >>>>> suggested to start with enabling DRM CI for drm-misc branches. Then we > > >>>>> can possibly start landing MRs with CI testing (probably starting with > > >>>>> vkms). > > >>>> > > >>>> It's something we've discussed with Sima for a while, but enabling CI on > > >>>> drm-misc at some point will make sense so we could just as well start > > >>>> now. > > >>>> > > >>>> The biggest unknown at the moment to start doing so is how we could keep > > >>>> drm-tip and the rerere repo with MR. > > >>> > > >>> Let's do a slow start and begin with post-merge testing. At least this > > >>> gives us an idea of how stable it is (and what does it take to keep it > > >>> green). Maybe we can perform post-merge testing for both drm-misc and > > >>> drm-tip. > > >> > > >> The one thing is that currently the runtime for igt is quite long > > >> (~1hr) because you can't really parallelize kms tests. So maybe > > >> nightly scheduled runs would be a better idea. > > > > > > SGTM. So, the question would be, who can set it up? > > > > > > > We will test the nightly pipelines in a forked repo and then will > > set it up for drm-misc and drm-tip. > > > > Revisiting this old thread... > > It's becoming increasingly clear that landing drm/ci changes via drm-misc > (where gitlab CI is not used) isn't working out. Why? Also, tbf, drm-misc doesn't use gitlab CI because nobody solved the drm-tip issue. And because the mail I sent 6 months ago about didn't get any feedback. > On the drm/msm side, we pretty regularly end up needing a 2nd dummy MR > with additional drm/ci, etc patches on top for running our CI > pipelines, which is really _not_ the way this is supposed to work. > > So I think we want a single tree to merge drm/ci, drm/msm, and changes for > any other driver that wants to participate in the CI process. We could > call it drm-gitlab or drm-ci or whatever. The rules would be: > > * _Only_ land changes via MR with passing CI pipeline. We should configure > the gitlab tree to disallow MRs without a green pipeline. > > * All drm/ci changes go thru this tree. > > * When we need to backmerge drm-next/files or drm-misc-next/fixes, that > goes via an MR into this shared tree, just like any other change. > > If there are expectation updates (tests start to fail or pass, we make > the fails/flakes/skips changes on the same MR, no questions asked. > (But would be polite to tag the associated driver maintainer on the > MR for visibility.) > > * Once we've done this, we could conceivable use similar file-path rules > like mesa does to only run applicable jobs. Ie. if the MR only touches > drm/msm there is no need to run i915 CI jobs. So we could optimize > the CI runner utilization this way. > > * Only ff-merges. At least to start with it would be only driver > maintainers submitting MRs with patches collected up from list, so > it would be few enough people that this shouldn't be a problem to > coordinate. > > * I'd be open to the idea of allowing drm core and cross-driver changes > thru this tree. These are especially the sorts of things that we'd > like to see have a clean CI pipeline. But not sure how this would > conflict with drm-misc. One possible future is that it replaces > drm-misc eventually. And we get back to the drm-tip discussion here. If you want to do any meaningful cross-driver or core changes, you need to have an integration tree like drm-tip. And if we solve that, afaict, we solve the only obstacle to using gitlab ci in drm-misc so we might just take drm/msm in instead. Maxime
Attachment:
signature.asc
Description: PGP signature
