i was using boost and rocksdb as examples for this cmake stuff, so i went back to see how we handle their defaults. it turns out that no other WITH_SYSTEM_ cmake variable in ceph defaults to ON, so i think it was wrong to do that with
WITH_SYSTEM_UTF8PROC. i've opened https://github.com/ceph/ceph/pull/45696 to turn it OFF as the fix for https://tracker.ceph.com/issues/55114
unfortunately this means that everybody's going to be building this submodule by default, even if install-deps.sh installed the package for you. that's the part i was trying to avoid. but if you aren't familiar with these cmake variables, especially WITH_SYSTEM_BOOST and WITH_SYSTEM_ROCKSDB, try turning them on - you probably have them installed already
On Tue, Mar 29, 2022 at 4:24 PM Laura Flores <lflores@xxxxxxxxxx> wrote:
If there is no way around this, there should at least be some documentation added about -DWITH_RADOSGW_SELECT_PARQUET=OFF and -DWITH_SYSTEM_UTF8PROC=OFF for RHEL/CentOS users in the documentation, like the main README file.On Tue, Mar 29, 2022 at 2:31 PM Casey Bodley <cbodley@xxxxxxxxxx> wrote:On Tue, Mar 29, 2022 at 3:04 PM Kaleb Keithley <kkeithle@xxxxxxxxxx> wrote:
>
> I'm not sure I understand the question.
>
> rhel8 and CentOS Stream 8 (or even CentOS 8) have ut8proc-2.1.1 in the AppStream repo; EPEL provides utf8proc-devel since RHEL (and CentOS) does not provide it in CodeReadyBuilder.
>
> Note that EPEL is not permitted to distribute any package (at any version) that exists in RHEL.
>
> Over in the CentOS Storage SIG utf8proc-2.6.1 for CentOS Stream 8, CentOS 8, and RHEL8 is provided in the CentOS-Ceph-Quincy repository.
>
> (RHEL9 and CentOS Stream 9 have utf8proc- and utf8proc-devel in AppStream and CodeReadyBuilder respectively.)
>
> I believe for RHEL builds you need to use -DWITH_SYSTEM_UTF8PROC:BOOL=False.
arrow requires at least v2.2.0, which is what motivated the submodule
approach for centos/rhel 8
to restate the question: what should our cmake do by default?
currently WITH_SYSTEM_UTF8PROC=ON is the default, which means that any
developers still running centos/rhel will have to override that
default to build successfully. i would guess that centos/rhel users
are in the minority here, so i tend to think i chose a sensible
default. but since this is a regression, i'm open to suggestions here
>
> On Tue, Mar 29, 2022 at 2:01 PM Casey Bodley <cbodley@xxxxxxxxxx> wrote:
>>
>> in https://tracker.ceph.com/issues/55114, Laura pointed out a
>> regression for cmake builds on centos/rhel involving the new
>> 'utf8proc' dependency. for these distros, the Arrow submodule PR
>> https://github.com/ceph/ceph/pull/44696 added an extra submodule for
>> utf8proc, along with the cmake variable WITH_SYSTEM_UTF8PROC to
>> control its use
>>
>> the required utf8proc v2.2.0 is available on other distros, so i chose
>> WITH_SYSTEM_UTF8PROC=ON as the default. for centos/rhel, ceph.spec.in
>> conditionally disables that flag when building the RPMs - so all of
>> our CI should handle this correctly
>>
>> for local cmake builds, how would developers like this to work?
>>
>> we might consider a cmake script that auto-detects the presence of a
>> compatible utf8proc, and only builds the submodule when necessary.
>> however, package maintainers seem to prefer cmake options like
>> WITH_SYSTEM_UTF8PROC for explicit control over bundled dependencies
>>
>> in the meantime, anyone that's blocked on cmake failures about
>> utf8proc can work around this issue, either by disabling the arrow
>> dependency entirely via -DWITH_RADOSGW_SELECT_PARQUET=OFF, or by
>> selecting the utf8proc submodule via -DWITH_SYSTEM_UTF8PROC=OFF
>>
>>
>> On Wed, Mar 23, 2022 at 3:48 PM Casey Bodley <cbodley@xxxxxxxxxx> wrote:
>> >
>> > fyi, the Arrow submodule just merged to master with
>> > https://github.com/ceph/ceph/pull/44696. please let me know if this
>> > causes any build issues on master asap! i'm planning to cherry-pick
>> > this for the quincy release
>> >
>> > On Thu, Feb 10, 2022 at 5:16 PM Kaleb Keithley <kkeithle@xxxxxxxxxx> wrote:
>> > >
>> > >
>> > > For those not in the RGW standups.
>> > >
>> > > liborc (Apache ORC) passed package review for Fedora and packages are built for f37/rawhide. It is also updated to the latest version, 1.7.3. (The ugly patch I used to get it to build shlibs in 1.6.6 is much prettier — if that's possible — in 1.7.3.) If anyone has any contacts at Apache that can get them to stop bundling dependencies that's be a big win in my book.
>> > >
>> > > I'm now preparing libarrow (Apache Arrow) for Fedora package review. As Casey noted in another email version 4.0.1 does not build with newer compilers, e.g. gcc-12 at least, maybe gcc-11 too, so I am working on packaging 7.0.0.
>> > >
>> > > Once I get libarrow submitted to package review I will circle back and update the versions of liborc and libarrow in CentOS Stream 8 (and Stream 9 too) to the latest.
>> > >
>> > > And longer term will get them and the other dependencies built in EPEL8 and EPEL9.
>> > >
>> > >
>> > > On Fri, Jan 28, 2022 at 1:22 PM Casey Bodley <cbodley@xxxxxxxxxx> wrote:
>> > >>
>> > >> after discussion with our Debian/Ubuntu maintainers, we've decided
>> > >> that Ceph will need to build arrow from a submodule for the quincy
>> > >> release. both distros do provide the dependencies necessary for a
>> > >> minimal build of arrow/parquet. i'm working through the cmake
>> > >> integration of the arrow submodule in
>> > >> https://github.com/ceph/ceph/pull/44696
>> > >>
>> > >> for Centos, Kaleb has packaged arrow and its missing dependencies and
>> > >> made them available in the Centos Storage SIG
>> > >> (https://wiki.centos.org/SpecialInterestGroup/Storage). i'm a bit wary
>> > >> about requiring that our Centos users install this additional repo via
>> > >> 'yum install centos-release-ceph', though. the Storage SIG is also
>> > >> going to host Centos' ceph release packages, so it may not a good idea
>> > >> to mix those repos with our upstream download.ceph.com repo
>> > >>
>> > >> getting this stuff into EPEL would be ideal - even if it was just
>> > >> enough to enable us to build the arrow submodule - but it sounds like
>> > >> some obstacles remain here. what do people think about requiring this
>> > >> extra centos-release-ceph repo for quincy? could that be acceptable as
>> > >> a last resort?
>> > >>
>> > >
>> > >
>> > > --
>> > >
>> > > Kaleb
>>
>
>
> --
>
> Kaleb
--Laura Flores
She/Her/Hers
Associate Software Engineer, Ceph Storage
La Grange Park, IL
lflores@xxxxxxxxxx
M: +17087388804
![]()
_______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
