Hi Oliver
Thanks for following up. I just picked this up again today and it was indeed librados2...the package wasn't installed! It's working now, haven't tested much but I haven't noticed any problems yet. This is with nfs-ganesha-2.6.1-0.1.el7.x86_64, libcephfs2-12.2.5-0.el7.x86_64 and librados2-12.2.5-0.el7.x86_64. Thanks for the pointer on that.On Wed, May 16, 2018 at 4:14 PM, Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx> wrote:
Hi David,
did you already manage to check your librados2 version and manage to pin down the issue?
Cheers,
Oliver
> http://lists.ceph.com/
Am 11.05.2018 um 17:15 schrieb Oliver Freyermuth:
> Hi David,
>
> Am 11.05.2018 um 16:55 schrieb David C:
>> Hi Oliver
>>
>> Thanks for the detailed reponse! I've downgraded my libcephfs2 to 12.2.4 and still get a similar error:
>>
>> load_fsal :NFS STARTUP :CRIT :Could not dlopen module:/usr/lib64/ganesha/libfsalceph.so Error:/lib64/libcephfs.so.2: undefined symbol: _Z14common_ preinitRK18CephInitParameters1 8code_environment_ti
>> load_fsal :NFS STARTUP :MAJ :Failed to load module (/usr/lib64/ganesha/libfsalceph.so) because: Can not access a needed shared library
>>
>> I'm on CentOS 7.4, using the following package versions:
>>
>> # rpm -qa | grep ganesha
>> nfs-ganesha-2.6.1-0.1.el7.x86_64
>> nfs-ganesha-vfs-2.6.1-0.1.el7.x86_64
>> nfs-ganesha-ceph-2.6.1-0.1.el7.x86_64
>>
>> # rpm -qa | grep ceph
>> libcephfs2-12.2.4-0.el7.x86_64
>> nfs-ganesha-ceph-2.6.1-0.1.el7.x86_64
>
> Mhhhm - that sounds like a messup in the dependencies.
> The symbol you are missing should be provided by
> librados2-12.2.4-0.el7.x86_64
> which contains
> /usr/lib64/ceph/ceph/libcephfs-common.so.0
> Do you have a different version of librados2 installed? If so, I wonder how yum / rpm allowed that ;-).
>
> Thinking again, it might also be (if you indeed have a different version there) that this is the cause also for the previous error.
> If the problematic symbol is indeed not exposed, but can be resolved only if both libraries (libcephfs-common and libcephfs) are loaded in unison with matching versions,
> it might be that also 12.2.5 works fine...
>
> First thing, in any case, is to checkout which version of librados2 you are using ;-).
>
> Cheers,
> Oliver
>
>>
>> I don't have the ceph user space components installed, assuming they're not nesscary apart from libcephfs2? Any idea why it's giving me this error?
>>
>> Thanks,
>>
>> On Fri, May 11, 2018 at 2:17 AM, Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxx-bonn.de >> wrote:
>>
>> Hi David,
>>
>> for what it's worth, we are running with nfs-ganesha 2.6.1 from Ceph repos on CentOS 7.4 with the following set of versions:
>> libcephfs2-12.2.4-0.el7.x86_64
>> nfs-ganesha-2.6.1-0.1.el7.x86_64
>> nfs-ganesha-ceph-2.6.1-0.1.el7.x86_64
>> Of course, we plan to upgrade to 12.2.5 soon-ish...
>>
>> Am 11.05.2018 um 00:05 schrieb David C:
>> > Hi All
>> >
>> > I'm testing out the nfs-ganesha-2.6.1-0.1.el7.x86_64.rpm package from http://download.ceph.com/nfs- ganesha/rpm-V2.6-stable/ <http://download.ceph.com/nfs-luminous/x86_64/ ganesha/rpm-V2.6-stable/ >luminous/x86_64/
>> >
>> > It's failing to load /usr/lib64/ganesha/libfsalceph.so
>> >
>> > With libcephfs-12.2.1 installed I get the following error in my ganesha log:
>> >
>> > load_fsal :NFS STARTUP :CRIT :Could not dlopen module:/usr/lib64/ganesha/libfsalceph.so Error:
>> > /usr/lib64/ganesha/libfsalceph.so: undefined symbol: ceph_set_deleg_timeout
>> > load_fsal :NFS STARTUP :MAJ :Failed to load module (/usr/lib64/ganesha/libfsalceph.so) because
>> > : Can not access a needed shared library
>>
>> That looks like an ABI incompatibility, probably the nfs-ganesha packages should block this libcephfs2-version (and older ones).
>>
>> >
>> >
>> > With libcephfs-12.2.5 installed I get:
>> >
>> > load_fsal :NFS STARTUP :CRIT :Could not dlopen module:/usr/lib64/ganesha/libfsalceph.so Error:
>> > /lib64/libcephfs.so.2: undefined symbol: _ZNK5FSMap10parse_roleEN5boost17basic_string_ viewIcSt11char_ traitsIcEEEP10mds_role_tRSo
>> > load_fsal :NFS STARTUP :MAJ :Failed to load module (/usr/lib64/ganesha/libfsalceph.so) because
>> > : Can not access a needed shared library
>>
>> That looks ugly and makes me fear for our planned 12.2.5-upgrade.
>> Interestingly, we do not have that symbol on 12.2.4:
>> # nm -D /lib64/libcephfs.so.2 | grep FSMap
>> U _ZNK5FSMap10parse_roleERKSsP10mds_role_tRSo
>> U _ZNK5FSMap13print_summaryEPN4ceph9FormatterEPSo
>> and NFS-Ganesha works fine.
>>
>> Looking at:
>> https://github.com/ceph/ceph/blob/v12.2.4/src/mds/FSMap.h <https://github.com/ceph/ceph/blob/v12.2.4/src/mds/FSMap.h >
>> versus
>> https://github.com/ceph/ceph/blob/v12.2.5/src/mds/FSMap.h <https://github.com/ceph/ceph/blob/v12.2.5/src/mds/FSMap.h >
>> it seems this commit:
>> https://github.com/ceph/ceph/commit/ <https://github.com/ceph/ceph/7d8b3c1082b6b870710989773f3cd9 8a472b9a3d commit/ >7d8b3c1082b6b870710989773f3cd9 8a472b9a3d
>> changed libcephfs2 ABI.
>>
>> I've no idea how that's usually handled and whether ABI breakage should occur within point releases (I would not have expected that...).
>> At least, this means either:
>> - ABI needs to be reverted to the old state.
>> - A new NFS Ganesha build is needed. Probably, if this is a common thing, builds should be automated and be synchronized to ceph releases,
>> and old versions should be kept around.
>>
>> I'll hold back our update to 12.2.5 until this is resolved, so many thanks from my side!
>>
>> Let's see who jumps in to resolve it...
>>
>> Cheers,
>> Oliver
>> >
>> >
>> > My cluster is running 12.2.1
>> >
>> > All package versions:
>> >
>> > nfs-ganesha-2.6.1-0.1.el7.x86_64
>> > nfs-ganesha-ceph-2.6.1-0.1.el7.x86_64
>> > libcephfs2-12.2.5-0.el7.x86_64
>> >
>> > Can anyone point me in the right direction?
>> >
>> > Thanks,
>> > David
>> >
>> >
>> > _______________________________________________
>> > ceph-users mailing list
>> > ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxx.com >
>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph. <http://lists.ceph.com/com listinfo.cgi/ceph-users-ceph. >com
>> >
>>
>>
>>
>
>
>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
listinfo.cgi/ceph-users-ceph. com
>
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
