Thanks Matt! Really useful configs. I am still on luminous, so I can
forget about this now :( I will try when I am nautilus, I have already
updated my configuration. However it is interesting that in the
configuration nowhere the tenant is specified, so I guess that is being
extracted from the access key/irrelevant.
-----Original Message-----
Subject: Re: NFS
Hi Mark,
Here's an example that should work--userx and usery are RGW users
created in different tenants, like so:
radosgw-admin --tenant tnt1 --uid userx --display-name "tnt1-userx" \
--access_key "userxacc" --secret "test123" user create
radosgw-admin --tenant tnt2 --uid usery --display-name "tnt2-usery" \
--access_key "useryacc" --secret "test456" user create
Remember that to make use of this feature, you need recent librgw and
matching nfs-ganesha. In particular, Ceph should have, among other
changes:
commit 65d0ae733defe277f31825364ee52d5102c06ab9
Author: Matt Benjamin <mbenjamin@xxxxxxxxxx>
Date: Wed Jun 5 07:25:35 2019 -0400
rgw_file: include tenant in hashes of object
Because bucket names are taken as object names in the top
of an export. Make hashing by tenant general to avoid disjoint
hashing of bucket.
Fixes: http://tracker.ceph.com/issues/40118
Signed-off-by: Matt Benjamin <mbenjamin@xxxxxxxxxx>
(cherry picked from commit 8e0fd5fbfa7c770f6b668e79b772179946027bce)
commit 459b6b2b224953655fd0360e8098ae598e41d3b2
Author: Matt Benjamin <mbenjamin@xxxxxxxxxx>
Date: Wed May 15 15:53:32 2019 -0400
rgw_file: include tenant when hashing bucket names
Prevent identical paths from distinct tenants from colliding in
RGW NFS handle cache.
Fixes: http://tracker.ceph.com/issues/40118
Signed-off-by: Matt Benjamin <mbenjamin@xxxxxxxxxx>
(cherry picked from commit b800a9de83dff23a150ed7d236cb61c8b7d971ae)
Signed-off-by: Matt Benjamin <mbenjamin@xxxxxxxxxx>
ganesha.conf.deuxtenant:
EXPORT
{
# Export Id (mandatory, each EXPORT must have a unique Export_Id)
Export_Id = 77;
# Exported path (mandatory)
Path = "/";
# Pseudo Path (required for NFS v4)
Pseudo = "/userx";
# Required for access (default is None)
# Could use CLIENT blocks instead
Access_Type = RW;
SecType = "sys";
Protocols = 3,4;
Transports = UDP,TCP;
#Delegations = Readwrite;
Squash = No_Root_Squash;
# Exporting FSAL
FSAL {
Name = RGW;
User_Id = "userx";
Access_Key_Id = "userxacc";
Secret_Access_Key = "test123";
}
}
EXPORT
{
# Export Id (mandatory, each EXPORT must have a unique Export_Id)
Export_Id = 78;
# Exported path (mandatory)
Path = "/";
# Pseudo Path (required for NFS v4)
Pseudo = "/usery";
# Required for access (default is None)
# Could use CLIENT blocks instead
Access_Type = RW;
SecType = "sys";
Protocols = 3,4;
Transports = UDP,TCP;
#Delegations = Readwrite;
Squash = No_Root_Squash;
# Exporting FSAL
FSAL {
Name = RGW;
User_Id = "usery";
Access_Key_Id = "useryacc";
Secret_Access_Key = "test456";
}
}
#mount at bucket case
EXPORT
{
# Export Id (mandatory, each EXPORT must have a unique Export_Id)
Export_Id = 79;
# Exported path (mandatory)
Path = "/buck5";
# Pseudo Path (required for NFS v4)
Pseudo = "/usery_buck5";
# Required for access (default is None)
# Could use CLIENT blocks instead
Access_Type = RW;
SecType = "sys";
Protocols = 3,4;
Transports = UDP,TCP;
#Delegations = Readwrite;
Squash = No_Root_Squash;
# Exporting FSAL
FSAL {
Name = RGW;
User_Id = "usery";
Access_Key_Id = "useryacc";
Secret_Access_Key = "test456";
}
}
RGW {
ceph_conf = "/home/mbenjamin/ceph-noob/build/ceph.conf";
#init_args = "-d --debug-rgw=16";
init_args = "";
}
NFS_Core_Param {
Nb_Worker = 17;
mount_path_pseudo = true;
}
CacheInode {
Chunks_HWMark = 70000;
Entries_Hwmark = 2000000;
}
NFSV4 {
Graceless = true;
Allow_Numeric_Owners = true;
Only_Numeric_Owners = true;
}
LOG {
Components {
#NFS_READDIR = FULL_DEBUG;
#NFS4 = FULL_DEBUG;
#CACHE_INODE = FULL_DEBUG;
#FSAL = FULL_DEBUG;
}
Facility {
name = FILE;
destination = "/tmp/ganesha-rgw.log";
enable = active;
}
}
On Thu, Oct 3, 2019 at 10:34 AM Marc Roos <M.Roos@xxxxxxxxxxxxxxxxx>
wrote:
>
>
> How should a multi tenant RGW config look like, I am not able get this
> working:
>
> EXPORT {
> Export_ID=301;
> Path = "test:test3";
> #Path = "/";
> Pseudo = "/rgwtester";
>
> Protocols = 4;
> FSAL {
> Name = RGW;
> User_Id = "test$tester1";
> Access_Key_Id = "TESTER";
> Secret_Access_Key = "xxx";
> }
> Disable_ACL = TRUE;
> CLIENT { Clients = 192.168.10.0/24; access_type = "RO"; } }
>
>
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> create_export :FSAL :CRIT :RGW module: librgw init failed (-5)
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> mdcache_fsal_create_export :FSAL :MAJ :Failed to call create_export on
> underlying FSAL RGW
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> fsal_put :FSAL :INFO :FSAL RGW now unused
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> fsal_cfg_commit :CONFIG :CRIT :Could not create export for
(/rgwtester)
> to (test:test3)
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> fsal_cfg_commit :FSAL :F_DBG :FSAL RGW refcount 0
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> config_errs_to_log :CONFIG :CRIT :Config File
> (/etc/ganesha/ganesha.conf:216): 1 validation errors in block FSAL
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> config_errs_to_log :CONFIG :CRIT :Config File
> (/etc/ganesha/ganesha.conf:216): Errors processing block (FSAL)
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> config_errs_to_log :CONFIG :CRIT :Config File
> (/etc/ganesha/ganesha.conf:209): 1 validation errors in block EXPORT
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> config_errs_to_log :CONFIG :CRIT :Config File
> (/etc/ganesha/ganesha.conf:209): Errors processing block (EXPORT)
>
> -----Original Message-----
> Subject: Re: NFS
>
> RGW NFS can support any NFS style of authentication, but users will
have
> the RGW access of their nfs-ganesha export. You can create exports
with
> disjoint privileges, and since recent L, N, RGW tenants.
>
> Matt
>
> On Tue, Oct 1, 2019 at 8:31 AM Marc Roos <M.Roos@xxxxxxxxxxxxxxxxx>
> wrote:
> >
> > I think you can run into problems
> > with a multi user environment of RGW and nfs-ganesha.
> >
> > _______________________________________________
> > ceph-users mailing list
> > ceph-users@xxxxxxxxxxxxxx
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> --
>
> Matt Benjamin
> Red Hat, Inc.
> 315 West Huron Street, Suite 140A
> Ann Arbor, Michigan 48103
>
> http://www.redhat.com/en/technologies/storage
>
> tel. 734-821-5101
> fax. 734-769-8938
> cel. 734-216-5309
>
>
--
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103
http://www.redhat.com/en/technologies/storage
tel. 734-821-5101
fax. 734-769-8938
cel. 734-216-5309
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
