Hi Ben,
The issue seems to be that you don't have a ceph keyring available to the
nfs-ganesha server. The upstream doc talks about this. The nfs-ganesha
runtime environment needs to be essentially identical to one (a pod, I
guess) that would run radosgw.
Matt
On Thu, Jan 12, 2023 at 7:27 AM Ruidong Gao <ruidong.gao@xxxxxxxxx> wrote:
> Hi,
>
> This is running Quincy 17.2.5 deployed by rook on k8s. RGW nfs export will
> crash Ganesha server pod. CephFS export works just fine. Here are steps of
> it:
> 1, create export:
> bash-4.4$ ceph nfs export create rgw --cluster-id nfs4rgw --pseudo-path
> /bucketexport --bucket testbk
> {
> "bind": "/bucketexport",
> "path": "testbk",
> "cluster": "nfs4rgw",
> "mode": "RW",
> "squash": "none"
> }
>
> 2, check pods status afterwards:
> rook-ceph-nfs-nfs1-a-679fdb795-82tcx 2/2 Running
> 0 4h3m
> rook-ceph-nfs-nfs4rgw-a-5c594d67dc-nlr42 1/2 Error
> 2 4h6m
>
> 3, check failing pod’s logs:
>
> 11/01/2023 08:11:53 : epoch 63be6f49 :
> rook-ceph-nfs-nfs4rgw-a-5c594d67dc-nlr42 : nfs-ganesha-1[main]
> nfs_start_grace :STATE :EVENT :NFS Server Now IN GRACE, duration 90
> 11/01/2023 08:11:54 : epoch 63be6f49 :
> rook-ceph-nfs-nfs4rgw-a-5c594d67dc-nlr42 : nfs-ganesha-1[main]
> nfs_start_grace :STATE :EVENT :grace reload client info completed from
> backend
> 11/01/2023 08:11:54 : epoch 63be6f49 :
> rook-ceph-nfs-nfs4rgw-a-5c594d67dc-nlr42 : nfs-ganesha-1[main]
> nfs_try_lift_grace :STATE :EVENT :check grace:reclaim complete(0) clid
> count(0)
> 11/01/2023 08:11:57 : epoch 63be6f49 :
> rook-ceph-nfs-nfs4rgw-a-5c594d67dc-nlr42 : nfs-ganesha-1[main]
> nfs_lift_grace_locked :STATE :EVENT :NFS Server Now NOT IN GRACE
> 11/01/2023 08:11:57 : epoch 63be6f49 :
> rook-ceph-nfs-nfs4rgw-a-5c594d67dc-nlr42 : nfs-ganesha-1[main]
> export_defaults_commit :CONFIG :INFO :Export Defaults now
> (options=03303002/00080000 , , , ,
> , , , , expire= 0)
> 2023-01-11T08:11:57.853+0000 7f59dac7c200 -1 auth: unable to find a
> keyring on /var/lib/ceph/radosgw/ceph-admin/keyring: (2) No such file or
> directory
> 2023-01-11T08:11:57.853+0000 7f59dac7c200 -1 AuthRegistry(0x56476817a480)
> no keyring found at /var/lib/ceph/radosgw/ceph-admin/keyring, disabling
> cephx
> 2023-01-11T08:11:57.855+0000 7f59dac7c200 -1 auth: unable to find a
> keyring on /var/lib/ceph/radosgw/ceph-admin/keyring: (2) No such file or
> directory
> 2023-01-11T08:11:57.855+0000 7f59dac7c200 -1 AuthRegistry(0x7ffe4d092c90)
> no keyring found at /var/lib/ceph/radosgw/ceph-admin/keyring, disabling
> cephx
> 2023-01-11T08:11:57.856+0000 7f5987537700 -1 monclient(hunting):
> handle_auth_bad_method server allowed_methods [2] but i only support [1]
> 2023-01-11T08:11:57.856+0000 7f5986535700 -1 monclient(hunting):
> handle_auth_bad_method server allowed_methods [2] but i only support [1]
> 2023-01-11T08:12:00.861+0000 7f5986d36700 -1 monclient(hunting):
> handle_auth_bad_method server allowed_methods [2] but i only support [1]
> 2023-01-11T08:12:00.861+0000 7f59dac7c200 -1 monclient: authenticate NOTE:
> no keyring found; disabled cephx authentication
> failed to fetch mon config (--no-mon-config to skip)
>
> 4, delete the export:
> ceph nfs export delete nfs4rgw /bucketexport
>
> Ganesha servers go back normal:
> rook-ceph-nfs-nfs1-a-679fdb795-82tcx 2/2 Running
> 0 4h30m
> rook-ceph-nfs-nfs4rgw-a-5c594d67dc-nlr42 2/2 Running
> 10 4h33m
>
> Any ideas to make it work?
>
> Thanks
> Ben
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
--
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103
http://www.redhat.com/en/technologies/storage
tel. 734-821-5101
fax. 734-769-8938
cel. 734-216-5309
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
