Hi Jeff
Thanks for the pointer on this. I found some details on this the other day and your link is a big help. I will get this updated in my ansible playbook and test.
Rgds
Lee
On 01/08/2019 17:03, Jeff Layton wrote:
On Sun, 2019-07-28 at 18:20 +0000, Lee Norvall wrote:
Update to this I found that you cannot create a 2nd files system as yet and it is still experimental. So I went down this route:
Added a pool to the existing cephfs and then setfattr -n ceph.dir.layout.pool -v SSD-NFS /mnt/cephfs/ssdnfs/ from a ceph-fuse client.
I then nfs mounted from another box. I can see the files and dir etc from the nfs client but my issue now is that I do not have permission to write, create dir etc. The same goes for the default setup after running the ansible playbook even when setting export to no_root_squash. I am missing a chain of permission? ganesha-nfs is using admin userid, is this the same as the client.admin or is this a user I need to create? Any info appreciated.
Ceph is on CentOS 7 and SELinux is currently off as well.
Copy of the ganesha conf below. Is secType correct or is it missing something?
RADOS_URLS {
ceph_conf = '/etc/ceph/ceph.conf';
userid = "admin";
}
%url rados://cephfs_data/ganesha-export-index
NFSv4 {
RecoveryBackend = 'rados_kv';
}
I your earlier email, you mentioned that you had more than one NFS
server, but rados_kv is not safe in a multi-server configuration. The
servers will be competing to store recovery information in the same
objects, and won't honor each others' grace periods/
You may want to explore using "RecoveryBackend = rados_cluster" instead,
which should handle that situation better. See this writeup, for some
guidelines:
https://jtlayton.wordpress.com/2018/12/10/deploying-an-active-active-nfs-cluster-over-cephfs/
Much of this is already automated too if you use k8s+rook.
RADOS_KV {
ceph_conf = '/etc/ceph/ceph.conf';
userid = "admin";
pool = "cephfs_data";
}
EXPORT
{
Export_id=20133;
Path = "/";
Pseudo = /cephfile;
Access_Type = RW;
Protocols = 3,4;
Transports = TCP;
SecType = sys,krb5,krb5i,krb5p;
Squash = Root_Squash;
Attr_Expiration_Time = 0;
FSAL {
Name = CEPH;
User_Id = "admin";
}
}
On 28/07/2019 12:11, Lee Norvall wrote:
Hi
I am using ceph-ansible to deploy and just looking for best way/tips on
how to export multiple pools/fs.
Ceph: nautilus (14.2.2)
NFS-Ganesha v 2.8
ceph-ansible stable 4.0
I have 3 x osd/NFS gateways running and NFS on the dashboard can see
them in the cluster. I have managed to export for cephfs / and mounted
it on another box.
1) can I add a new pool/fs to the export under that same NFS gateway
cluster, or
2) do I have the to do something like add a new pool to the fs and then
setfattr to make the layout /newfs_dir point to /new_pool? does this
cause issues and false object count?
3) any other better ways...
Rgds
Lee
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx<mailto:ceph-users@xxxxxxxxxxxxxx>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Lee Norvall | CEO / Founder
Mob. +44 (0)7768 201884
Tel. +44 (0)20 3026 8930
Web. www.blocz.io<http://www.blocz.io>
Enterprise Cloud | Private Cloud | Hybrid/Multi Cloud | Cloud Backup
This e-mail (and any attachment) has been sent from a PC belonging to My Mind (Holdings) Limited. If you receive it in error, please tell us by return and then delete it from your system; you may not rely on its contents nor copy/disclose it to anyone. Opinions, conclusions and statements of intent in this e-mail are those of the sender and will not bind My Mind (Holdings) Limited unless confirmed by an authorised representative independently of this message. We do not accept responsibility for viruses; you must scan for these. Please note that e-mails sent to and from blocz IO Limited are routinely monitored for record keeping, quality control and training purposes, to ensure regulatory compliance and to prevent viruses and unauthorised use of our computer systems. My Mind (Holdings) Limited is registered in England & Wales under company number 10186410. Registered office: 1st Floor Offices, 2a Highfield Road, Ringwood, Hampshire, United Kingdom, BH24 1RQ. VAT Registration GB 244 9628 77
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx<mailto:ceph-users@xxxxxxxxxxxxxx>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
[blocz IO Limted]
Lee Norvall | CEO / Founder
Mob. +44 (0)7768 201884
Tel. +44 (0)20 3026 8930
Web. www.blocz.io<http://www.blocz.io>
Enterprise Cloud | Private Cloud | Hybrid/Multi Cloud | Cloud Backup
[blocz IO Limted] [blocz IO Limted] [blocz IO Limted] [blocz IO Limted] [blocz IO Limted] [ESET MSP Partner]
This e-mail (and any attachment) has been sent from a PC belonging to My Mind (Holdings) Limited. If you receive it in error, please tell us by return and then delete it from your system; you may not rely on its contents nor copy/disclose it to anyone. Opinions, conclusions and statements of intent in this e-mail are those of the sender and will not bind My Mind (Holdings) Limited unless confirmed by an authorised representative independently of this message. We do not accept responsibility for viruses; you must scan for these. Please note that e-mails sent to and from blocz IO Limited are routinely monitored for record keeping, quality control and training purposes, to ensure regulatory compliance and to prevent viruses and unauthorised use of our computer systems. My Mind (Holdings) Limited is registered in England & Wales under company number 10186410. Registered office: 1st Floor Offices, 2a Highfield Road, Ringwood, Hampshire, United Kingdom, BH24 1RQ. VAT Registration GB 244 9628 77
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
