On Thu, Sep 6, 2018 at 4:06 AM Alfredo Deza <adeza@xxxxxxxxxx> wrote: > > On Tue, Sep 4, 2018 at 11:29 AM, kefu chai <tchaikov@xxxxxxxxx> wrote: > > On Tue, Sep 4, 2018 at 10:20 PM Alfredo Deza <adeza@xxxxxxxxxx> wrote: > >> > >> On Tue, Sep 4, 2018 at 10:15 AM, kefu chai <tchaikov@xxxxxxxxx> wrote: > >> > hi guys, > >> > > >> > we build boost every time when building ceph. i don't think it's > >> > efficient, hence http://tracker.ceph.com/issues/25186 is filed to > >> > track this issue. > >> > > >> > to cache the built libboost for xenial and bionic, we can setup a PPA > >> > repo or host the repo by ourselves. if this approach works fine, we > >> > can also build libboost for centos. and expand the coverage to other > >> > dependencies, like dpdk, rocksdb and seastar. > >> > > >> > what do you think? > >> > >> I worry about the maintenance burden. Repositories are somewhat easy > >> with our infrastructure. If we are able to HTTP POST the binary to > >> chacra.ceph.com > >> > >> Who will be maintaining this package? > > > > i will. in the short-term, i will > > > > 1. setup repo(s) hosting the "debian/" directory. so the downstream > > changes can be tracked. > > 2. build the deb packages manually. > > 3. upload the source/binary packages to chacra > > I think that you would need #2 and #3 only, the repos are "free" once > they get to a chacra instance please note, by "/debian" directory, i mean the downstream changes for packaging, see https://github.com/tchaikov/boost/tree/master/debian , without which, we can hardly *package* upstream dist tarball properly. unless you assume we already have read-to-use source packages. > > > > > in long-term, i think i will try to setup a jenkins job to build and > > upload the deb files for different branches when they gets updated. > > for instance, we will have xenial, bionic branches for libboost 1.67, > > once the xenial branch is updated, we should built the new recipe and > > push the built packages to chacra. > > That makes sense > > > > > my concern is, we might want to host different versions of dependency > > packages, for instance, rocksdb x.y.z adds a new API, and we picks it > > up and uses it in master. apparently, we also bump up the version # of > > it in install-deps.sh in master, where we will install the pre-built > > packages. but the Ceph versions which use rocksdb version x.y.$(z-1) > > should still build after this change in master. and their > > install-deps.sh still point to rocksdb x.y.$(z-1). so if we cannot > > support multiple versions in one repo. it's a no-go. > > you could make the version part of the url (although that would mean > different repos). If that is not an option then > we would need to figure out what to do about reprepro. currently, i encode the version number in the package name, like ceph-libboost-regex1.67-dev . so it's fine at this moment. > > It might not be that hard to swap reprepro for another thing, the > commands/flags used are almost in a single function: > > https://github.com/ceph/chacra/blob/master/chacra/util.py#L270-L296 > > > > >> > >> We have done something similar in the past, where Vagrant doesn't have > >> a repository (but does have packages) and we've set them up at > >> https://chacra.ceph.com/r/vagrant/latest/HEAD/ > >> so that our tooling can just consume from there. Alfredo, is there any way i can upload the built packages to chacra? i'd like to upload the xenial build of ceph-libboost-*.deb first and update install-deps.sh locally. if this works fine. will push the change to master. and then expand it to more distros. > >> > >> > >> > > >> > -- > >> > Regards > >> > Kefu Chai > > > > > > > > -- > > Regards > > Kefu Chai -- Regards Kefu Chai
