Re: cache libboost (and more) for speeding up the build

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Tue, Sep 4, 2018 at 8:29 AM, kefu chai <tchaikov@xxxxxxxxx> wrote:
> On Tue, Sep 4, 2018 at 10:20 PM Alfredo Deza <adeza@xxxxxxxxxx> wrote:
>>
>> On Tue, Sep 4, 2018 at 10:15 AM, kefu chai <tchaikov@xxxxxxxxx> wrote:
>> > hi guys,
>> >
>> > we build boost every time when building ceph. i don't think it's
>> > efficient, hence http://tracker.ceph.com/issues/25186 is filed to
>> > track this issue.
>> >
>> > to cache the built libboost for xenial and bionic, we can setup a PPA
>> > repo or host the repo by ourselves. if this approach works fine, we
>> > can also build libboost for centos. and expand the coverage to other
>> > dependencies, like dpdk, rocksdb and seastar.
>> >
>> > what do you think?
>>
>> I worry about the maintenance burden. Repositories are somewhat easy
>> with our infrastructure. If we are able to HTTP POST the binary to
>> chacra.ceph.com
>>
>> Who will be maintaining this package?
>
> i will. in the short-term, i will
>
> 1. setup repo(s) hosting the "debian/" directory. so the downstream
> changes can be tracked.
> 2. build the deb packages manually.
> 3. upload the source/binary packages to chacra
>
> in long-term, i think i will try to setup a jenkins job to build and
> upload the deb files for different branches when they gets updated.
> for instance, we will have xenial, bionic branches for libboost 1.67,
> once the xenial branch is updated, we should built the new recipe and
> push the built packages to chacra.
>
> my concern is, we might want to host different versions of dependency
> packages, for instance,  rocksdb x.y.z adds a new API, and we picks it
> up and uses it in master. apparently, we also bump up the version # of
> it in install-deps.sh in master, where we will install the pre-built
> packages. but the Ceph versions which use rocksdb version x.y.$(z-1)
> should still build after this change in master. and their
> install-deps.sh still point to rocksdb x.y.$(z-1). so if we cannot
> support multiple versions in one repo. it's a no-go.

Sounds like this will work nicely for our Jenkins jobs, but it won't
do much for development builds as I think we're mostly on Fedora or
other Ubuntus. :(
Is that expected?
-Greg

>
>>
>> We have done something similar in the past, where Vagrant doesn't have
>> a repository (but does have packages) and we've set them up at
>> https://chacra.ceph.com/r/vagrant/latest/HEAD/
>> so that our tooling can just consume from there.
>>
>>
>> >
>> > --
>> > Regards
>> > Kefu Chai
>
>
>
> --
> Regards
> Kefu Chai



[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux