Hi David, did you already manage to check your librados2 version and manage to pin down the issue? Cheers, Oliver Am 11.05.2018 um 17:15 schrieb Oliver Freyermuth: > Hi David, > > Am 11.05.2018 um 16:55 schrieb David C: >> Hi Oliver >> >> Thanks for the detailed reponse! I've downgraded my libcephfs2 to 12.2.4 and still get a similar error: >> >> load_fsal :NFS STARTUP :CRIT :Could not dlopen module:/usr/lib64/ganesha/libfsalceph.so Error:/lib64/libcephfs.so.2: undefined symbol: _Z14common_preinitRK18CephInitParameters18code_environment_ti >> load_fsal :NFS STARTUP :MAJ :Failed to load module (/usr/lib64/ganesha/libfsalceph.so) because: Can not access a needed shared library >> >> I'm on CentOS 7.4, using the following package versions: >> >> # rpm -qa | grep ganesha >> nfs-ganesha-2.6.1-0.1.el7.x86_64 >> nfs-ganesha-vfs-2.6.1-0.1.el7.x86_64 >> nfs-ganesha-ceph-2.6.1-0.1.el7.x86_64 >> >> # rpm -qa | grep ceph >> libcephfs2-12.2.4-0.el7.x86_64 >> nfs-ganesha-ceph-2.6.1-0.1.el7.x86_64 > > Mhhhm - that sounds like a messup in the dependencies. > The symbol you are missing should be provided by > librados2-12.2.4-0.el7.x86_64 > which contains > /usr/lib64/ceph/ceph/libcephfs-common.so.0 > Do you have a different version of librados2 installed? If so, I wonder how yum / rpm allowed that ;-). > > Thinking again, it might also be (if you indeed have a different version there) that this is the cause also for the previous error. > If the problematic symbol is indeed not exposed, but can be resolved only if both libraries (libcephfs-common and libcephfs) are loaded in unison with matching versions, > it might be that also 12.2.5 works fine... > > First thing, in any case, is to checkout which version of librados2 you are using ;-). > > Cheers, > Oliver > >> >> I don't have the ceph user space components installed, assuming they're not nesscary apart from libcephfs2? Any idea why it's giving me this error? >> >> Thanks, >> >> On Fri, May 11, 2018 at 2:17 AM, Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx <mailto:freyermuth@xxxxxxxxxxxxxxxxxx>> wrote: >> >> Hi David, >> >> for what it's worth, we are running with nfs-ganesha 2.6.1 from Ceph repos on CentOS 7.4 with the following set of versions: >> libcephfs2-12.2.4-0.el7.x86_64 >> nfs-ganesha-2.6.1-0.1.el7.x86_64 >> nfs-ganesha-ceph-2.6.1-0.1.el7.x86_64 >> Of course, we plan to upgrade to 12.2.5 soon-ish... >> >> Am 11.05.2018 um 00:05 schrieb David C: >> > Hi All >> > >> > I'm testing out the nfs-ganesha-2.6.1-0.1.el7.x86_64.rpm package from http://download.ceph.com/nfs-ganesha/rpm-V2.6-stable/luminous/x86_64/ <http://download.ceph.com/nfs-ganesha/rpm-V2.6-stable/luminous/x86_64/> >> > >> > It's failing to load /usr/lib64/ganesha/libfsalceph.so >> > >> > With libcephfs-12.2.1 installed I get the following error in my ganesha log: >> > >> > load_fsal :NFS STARTUP :CRIT :Could not dlopen module:/usr/lib64/ganesha/libfsalceph.so Error: >> > /usr/lib64/ganesha/libfsalceph.so: undefined symbol: ceph_set_deleg_timeout >> > load_fsal :NFS STARTUP :MAJ :Failed to load module (/usr/lib64/ganesha/libfsalceph.so) because >> > : Can not access a needed shared library >> >> That looks like an ABI incompatibility, probably the nfs-ganesha packages should block this libcephfs2-version (and older ones). >> >> > >> > >> > With libcephfs-12.2.5 installed I get: >> > >> > load_fsal :NFS STARTUP :CRIT :Could not dlopen module:/usr/lib64/ganesha/libfsalceph.so Error: >> > /lib64/libcephfs.so.2: undefined symbol: _ZNK5FSMap10parse_roleEN5boost17basic_string_viewIcSt11char_traitsIcEEEP10mds_role_tRSo >> > load_fsal :NFS STARTUP :MAJ :Failed to load module (/usr/lib64/ganesha/libfsalceph.so) because >> > : Can not access a needed shared library >> >> That looks ugly and makes me fear for our planned 12.2.5-upgrade. >> Interestingly, we do not have that symbol on 12.2.4: >> # nm -D /lib64/libcephfs.so.2 | grep FSMap >> U _ZNK5FSMap10parse_roleERKSsP10mds_role_tRSo >> U _ZNK5FSMap13print_summaryEPN4ceph9FormatterEPSo >> and NFS-Ganesha works fine. >> >> Looking at: >> https://github.com/ceph/ceph/blob/v12.2.4/src/mds/FSMap.h <https://github.com/ceph/ceph/blob/v12.2.4/src/mds/FSMap.h> >> versus >> https://github.com/ceph/ceph/blob/v12.2.5/src/mds/FSMap.h <https://github.com/ceph/ceph/blob/v12.2.5/src/mds/FSMap.h> >> it seems this commit: >> https://github.com/ceph/ceph/commit/7d8b3c1082b6b870710989773f3cd98a472b9a3d <https://github.com/ceph/ceph/commit/7d8b3c1082b6b870710989773f3cd98a472b9a3d> >> changed libcephfs2 ABI. >> >> I've no idea how that's usually handled and whether ABI breakage should occur within point releases (I would not have expected that...). >> At least, this means either: >> - ABI needs to be reverted to the old state. >> - A new NFS Ganesha build is needed. Probably, if this is a common thing, builds should be automated and be synchronized to ceph releases, >> and old versions should be kept around. >> >> I'll hold back our update to 12.2.5 until this is resolved, so many thanks from my side! >> >> Let's see who jumps in to resolve it... >> >> Cheers, >> Oliver >> > >> > >> > My cluster is running 12.2.1 >> > >> > All package versions: >> > >> > nfs-ganesha-2.6.1-0.1.el7.x86_64 >> > nfs-ganesha-ceph-2.6.1-0.1.el7.x86_64 >> > libcephfs2-12.2.5-0.el7.x86_64 >> > >> > Can anyone point me in the right direction? >> > >> > Thanks, >> > David >> > >> > >> > _______________________________________________ >> > ceph-users mailing list >> > ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx> >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com <http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> >> > >> >> >> > > > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
