Hi John,
It's great to see some traction on this front!
Earlier this year I started this PR to bring in a containerized build pipeline to our Jenkins CI. The build stage worked perfectly (20 mins including the checkout and configure stages). It failed in the test/check stage since some tests required sudo to open privileged ports (I wanted to keep a non-sudo build environment so that it can run securely both in root and rootless container mode).
Kind Regards,
Ernesto
On Fri, Aug 12, 2022 at 5:20 PM John Mulligan <phlogistonjohn@xxxxxxxxxxxxx> wrote:
On Thursday, August 11, 2022 7:17:19 PM EDT John Mulligan wrote:
> On Thursday, August 11, 2022 5:34:40 PM EDT Josh Durgin wrote:
> > I think it's a great idea - it's related to ideas in this thread:
> > https://lists.ceph.io/hyperkitty/list/dev@xxxxxxx/thread/UD43OL6YBN5A2QHLK
> > RU QLYQRXMHM5FKJ/
>
> Indeed, I remember participating a little in that thread.
>
> > The main idea there is to make it simple to update containers so you can
> > run teuthology tests against code you just changed with very little
> > overhead (no need to wait for hours for package + container complete
> > rebuilds).
>
> Great, I've been thinking about this as well - I've been poking at cpatch
> here-and-there as well and had started working on a python version that I
> believe will be easier to hack on further. I need to polish it up a little
> and share it.
>
I spent some time today to make my branch presentable and filed an RFC PR for
the python version of cpatch I mentioned above. Posting it here for context:
https://github.com/ceph/ceph/pull/47573
> > Zack Cerza has made a lot of progress on the teuthology side of this -
> > running the tests locally using an existing container image. The 2nd half,
> > of making it easy and fast to update a container image, is still TBD.
>
> That's great! I only got Teuthology in Sepia access recently and I would
> love to try out this version too. Is there a link to a WIP PR or something
> along that line? I'd be interested in trying it out a bit.
>
>
> In the short term, I'll try to put together a proof-of-concept PR for some
> of the build container ideas I'm thinking of. It seems like there's a fair
> amount of interest.
>
> Thanks!
>
> > Josh
> >
> > On Thu, Aug 11, 2022 at 12:09 PM Tom R <precision.automobilia@xxxxxxxxx>
> >
> > wrote:
> > > John
> > >
> > > I think your proposal to separate the build process such that the user
> > > can
> > > select an os flavor of their liking is a fantastic idea.
> > >
> > > I'm not familiar with the process to assist but would love to follow if
> > > this proposal is accepted.
> > >
> > >
> > >
> > > On Thu, Aug 11, 2022, 1:37 PM John Mulligan
> > > <phlogistonjohn@xxxxxxxxxxxxx>
> > >
> > > wrote:
> > >> On the user's list one thread about the packages took a turn into
> > >> discussing
> > >> building in containers [1]. This is a topic that I have had some idle
> > >> conversations about with Adam King and so I figured that I would raise
> > >> this to
> > >> a wider audience.
> > >>
> > >> My thought is to use container images specifically for building ceph -
> > >> and not
> > >> just it's container images. The builds may continue to produce
> > >> packages,
> > >> but a
> > >> container would be used as an abstraction between the actual OS and the
> > >> build
> > >> process of (do_cmake.sh, etc.).
> > >>
> > >> Builder images would be be available for use both by the build system
> > >> (jenkins
> > >> builders) as well as individual users. One advantage of this is that
> > >> the
> > >> user
> > >> can build the packages for distros that don't match the local distro.
> > >> I've
> > >> also found it advantageous for my own builds to use the container to
> > >> limit
> > >> memory and CPU for the build.
> > >>
> > >> I'm curious if anyone has discussed this before. Does it interest
> > >> anyone? I
> > >> am willing to volunteer some time to help as well.
> > >>
> > >> Thanks for reading!
> > >>
> > >>
> > >> [1] https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/message/
> > >> VR3ZKP4T2PLZ6BJ23GPZAG3KBV6AI3LA/
> > >> <https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/message/VR3ZK
> > >> P4
> > >> T2PLZ6BJ23GPZAG3KBV6AI3LA/>
> > >>
> > >>
> > >> _______________________________________________
> > >> Dev mailing list -- dev@xxxxxxx
> > >> To unsubscribe send an email to dev-leave@xxxxxxx
> > >
> > > _______________________________________________
> > > Dev mailing list -- dev@xxxxxxx
> > > To unsubscribe send an email to dev-leave@xxxxxxx
>
> _______________________________________________
> Dev mailing list -- dev@xxxxxxx
> To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
