On Wed, Sep 5, 2018 at 2:28 AM Gregory Farnum <gfarnum@xxxxxxxxxx> wrote: > > On Tue, Sep 4, 2018 at 8:29 AM, kefu chai <tchaikov@xxxxxxxxx> wrote: > > On Tue, Sep 4, 2018 at 10:20 PM Alfredo Deza <adeza@xxxxxxxxxx> wrote: > >> > >> On Tue, Sep 4, 2018 at 10:15 AM, kefu chai <tchaikov@xxxxxxxxx> wrote: > >> > hi guys, > >> > > >> > we build boost every time when building ceph. i don't think it's > >> > efficient, hence http://tracker.ceph.com/issues/25186 is filed to > >> > track this issue. > >> > > >> > to cache the built libboost for xenial and bionic, we can setup a PPA > >> > repo or host the repo by ourselves. if this approach works fine, we > >> > can also build libboost for centos. and expand the coverage to other > >> > dependencies, like dpdk, rocksdb and seastar. > >> > > >> > what do you think? > >> > >> I worry about the maintenance burden. Repositories are somewhat easy > >> with our infrastructure. If we are able to HTTP POST the binary to > >> chacra.ceph.com > >> > >> Who will be maintaining this package? > > > > i will. in the short-term, i will > > > > 1. setup repo(s) hosting the "debian/" directory. so the downstream > > changes can be tracked. > > 2. build the deb packages manually. > > 3. upload the source/binary packages to chacra > > > > in long-term, i think i will try to setup a jenkins job to build and > > upload the deb files for different branches when they gets updated. > > for instance, we will have xenial, bionic branches for libboost 1.67, > > once the xenial branch is updated, we should built the new recipe and > > push the built packages to chacra. > > > > my concern is, we might want to host different versions of dependency > > packages, for instance, rocksdb x.y.z adds a new API, and we picks it > > up and uses it in master. apparently, we also bump up the version # of > > it in install-deps.sh in master, where we will install the pre-built > > packages. but the Ceph versions which use rocksdb version x.y.$(z-1) > > should still build after this change in master. and their > > install-deps.sh still point to rocksdb x.y.$(z-1). so if we cannot > > support multiple versions in one repo. it's a no-go. > > Sounds like this will work nicely for our Jenkins jobs, but it won't > do much for development builds as I think we're mostly on Fedora or > other Ubuntus. :( > Is that expected? i don't follow. what do you mean by "development builds"? are they the builds of pull requests? all of our building jobs are performed by jenkins. if jenkins slaves are able to pull from chacra repo, we are good, i think. as long as different debian derivatives are able to pull from different repos, and they should, the problem is solved. BTW, reprepro now supports multiple versions support. but the change (https://github.com/profitbricks/reprepro) has not been accepted upstream. really? are we building on fedora also on the jenkins builders? i think we only build Ceph on centos, ubuntu and debian. but i don't think it's a show-stopper, as we can always setup a rpm repo for fedora anyway. it just adds more maintenance load. > -Greg > > > > >> > >> We have done something similar in the past, where Vagrant doesn't have > >> a repository (but does have packages) and we've set them up at > >> https://chacra.ceph.com/r/vagrant/latest/HEAD/ > >> so that our tooling can just consume from there. > >> > >> > >> > > >> > -- > >> > Regards > >> > Kefu Chai > > > > > > > > -- > > Regards > > Kefu Chai -- Regards Kefu Chai
