reviving an old thread, but i'm still very interested in this idea. has any other progress been made here? are there simple steps we could take to start experimenting with this? On Thu, Aug 18, 2022 at 6:19 AM Ernesto Puerta <epuertat@xxxxxxxxxx> wrote: > > Hi John, > >> I'm not very jenkins savvy so I can't speak much to that part. I'll dig into >> your dockerfile a bit. One thing I note is that there's only a dockerfile for >> centos (stream). The build infrastructure I'm imagining let's one build ubuntu >> binaries & packages and/or centos binaries & packages regardless of the "real" >> OS/distro . What do you think about supporting multiple different dockefiles for >> each supported distro and selecting between them hypothetically based on a >> (distro, release, flavor, arch) style tuple like used in the builds? > > > Yeah, that was my idea too. We currently support 2 different distros in our CI (CentOS and Ubuntu), but we randomly test a PR only on one of them. With containerized (reproducible) builds we could start using a distributed ccache, and with the savings in build times we could afford running the PR pipeline for 2 different distros. > > Regarding the Dockerfile, if you check it, there's only 1 line of code that is CentOS-specific (the "dnf install" for the EPEL repo). If we move that to the install-deps.sh, we could have a distro-neutral Dockerfile (the base layer would still have to be set via DISTRO build arg). > >> >> > Kind Regards, >> > Ernesto >> > >> > >> > On Fri, Aug 12, 2022 at 5:20 PM John Mulligan <phlogistonjohn@xxxxxxxxxxxxx> >> > wrote: >> > > On Thursday, August 11, 2022 7:17:19 PM EDT John Mulligan wrote: >> > > > On Thursday, August 11, 2022 5:34:40 PM EDT Josh Durgin wrote: >> > > > > I think it's a great idea - it's related to ideas in this thread: >> > > https://lists.ceph.io/hyperkitty/list/dev@xxxxxxx/thread/UD43OL6YBN5A2QHLK >> > > >> > > > > RU QLYQRXMHM5FKJ/ >> > > > >> > > > Indeed, I remember participating a little in that thread. >> > > > >> > > > > The main idea there is to make it simple to update containers so you >> > > >> > > can >> > > >> > > > > run teuthology tests against code you just changed with very little >> > > > > overhead (no need to wait for hours for package + container complete >> > > > > rebuilds). >> > > > >> > > > Great, I've been thinking about this as well - I've been poking at >> > > > cpatch >> > > > here-and-there as well and had started working on a python version that >> > > > I >> > > > believe will be easier to hack on further. I need to polish it up a >> > > >> > > little >> > > >> > > > and share it. >> > > >> > > I spent some time today to make my branch presentable and filed an RFC PR >> > > for >> > > the python version of cpatch I mentioned above. Posting it here for >> > > context: >> > > https://github.com/ceph/ceph/pull/47573 >> > > >> > > > > Zack Cerza has made a lot of progress on the teuthology side of this - >> > > > > running the tests locally using an existing container image. The 2nd >> > > >> > > half, >> > > >> > > > > of making it easy and fast to update a container image, is still TBD. >> > > > >> > > > That's great! I only got Teuthology in Sepia access recently and I >> > > > would >> > > > love to try out this version too. Is there a link to a WIP PR or >> > > >> > > something >> > > >> > > > along that line? I'd be interested in trying it out a bit. >> > > > >> > > > >> > > > In the short term, I'll try to put together a proof-of-concept PR for >> > > >> > > some >> > > >> > > > of the build container ideas I'm thinking of. It seems like there's a >> > > >> > > fair >> > > >> > > > amount of interest. >> > > > >> > > > Thanks! >> > > > >> > > > > Josh >> > > > > >> > > > > On Thu, Aug 11, 2022 at 12:09 PM Tom R < >> > > >> > > precision.automobilia@xxxxxxxxx> >> > > >> > > > > wrote: >> > > > > > John >> > > > > > >> > > > > > I think your proposal to separate the build process such that the >> > > >> > > user >> > > >> > > > > > can >> > > > > > select an os flavor of their liking is a fantastic idea. >> > > > > > >> > > > > > I'm not familiar with the process to assist but would love to follow >> > > >> > > if >> > > >> > > > > > this proposal is accepted. >> > > > > > >> > > > > > >> > > > > > >> > > > > > On Thu, Aug 11, 2022, 1:37 PM John Mulligan >> > > > > > <phlogistonjohn@xxxxxxxxxxxxx> >> > > > > > >> > > > > > wrote: >> > > > > >> On the user's list one thread about the packages took a turn into >> > > > > >> discussing >> > > > > >> building in containers [1]. This is a topic that I have had some >> > > >> > > idle >> > > >> > > > > >> conversations about with Adam King and so I figured that I would >> > > >> > > raise >> > > >> > > > > >> this to >> > > > > >> a wider audience. >> > > > > >> >> > > > > >> My thought is to use container images specifically for building >> > > >> > > ceph - >> > > >> > > > > >> and not >> > > > > >> just it's container images. The builds may continue to produce >> > > > > >> packages, >> > > > > >> but a >> > > > > >> container would be used as an abstraction between the actual OS and >> > > >> > > the >> > > >> > > > > >> build >> > > > > >> process of (do_cmake.sh, etc.). >> > > > > >> >> > > > > >> Builder images would be be available for use both by the build >> > > >> > > system >> > > >> > > > > >> (jenkins >> > > > > >> builders) as well as individual users. One advantage of this is >> > > > > >> that >> > > > > >> the >> > > > > >> user >> > > > > >> can build the packages for distros that don't match the local >> > > >> > > distro. >> > > >> > > > > >> I've >> > > > > >> also found it advantageous for my own builds to use the container >> > > > > >> to >> > > > > >> limit >> > > > > >> memory and CPU for the build. >> > > > > >> >> > > > > >> I'm curious if anyone has discussed this before. Does it interest >> > > > > >> anyone? I >> > > > > >> am willing to volunteer some time to help as well. >> > > > > >> >> > > > > >> Thanks for reading! >> > > > > >> >> > > > > >> >> > > > > >> [1] >> > > >> > > https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/message/ >> > > >> > > > > >> VR3ZKP4T2PLZ6BJ23GPZAG3KBV6AI3LA/ >> > > > > >> < >> > > >> > > https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/message/VR3ZK >> > > >> > > > > >> P4 >> > > > > >> T2PLZ6BJ23GPZAG3KBV6AI3LA/> >> > > > > >> >> > > > > >> >> > > > > >> _______________________________________________ >> > > > > >> Dev mailing list -- dev@xxxxxxx >> > > > > >> To unsubscribe send an email to dev-leave@xxxxxxx >> > > > > > >> > > > > > _______________________________________________ >> > > > > > Dev mailing list -- dev@xxxxxxx >> > > > > > To unsubscribe send an email to dev-leave@xxxxxxx >> > > > >> > > > _______________________________________________ >> > > > Dev mailing list -- dev@xxxxxxx >> > > > To unsubscribe send an email to dev-leave@xxxxxxx >> > > >> > > _______________________________________________ >> > > Dev mailing list -- dev@xxxxxxx >> > > To unsubscribe send an email to dev-leave@xxxxxxx >> >> >> > _______________________________________________ > Dev mailing list -- dev@xxxxxxx > To unsubscribe send an email to dev-leave@xxxxxxx _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
