Re: NFS

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Thanks Matt! Really useful configs. I am still on luminous, so I can 
forget about this now :( I will try when I am nautilus, I have already 
updated my configuration. However it is interesting that in the 
configuration nowhere the tenant is specified, so I guess that is being 
extracted from the access key/irrelevant.



-----Original Message-----
Subject: Re:  NFS

Hi Mark,

Here's an example that should work--userx and usery are RGW users 
created in different tenants, like so:

radosgw-admin --tenant tnt1 --uid userx --display-name "tnt1-userx" \
     --access_key "userxacc" --secret "test123" user create

 radosgw-admin --tenant tnt2 --uid usery --display-name "tnt2-usery" \
     --access_key "useryacc" --secret "test456" user create

Remember that to make use of this feature, you need recent librgw and 
matching nfs-ganesha.  In particular, Ceph should have, among other
changes:

commit 65d0ae733defe277f31825364ee52d5102c06ab9
Author: Matt Benjamin <mbenjamin@xxxxxxxxxx>
Date:   Wed Jun 5 07:25:35 2019 -0400

    rgw_file: include tenant in hashes of object

    Because bucket names are taken as object names in the top
    of an export.  Make hashing by tenant general to avoid disjoint
    hashing of bucket.

    Fixes: http://tracker.ceph.com/issues/40118

    Signed-off-by: Matt Benjamin <mbenjamin@xxxxxxxxxx>
    (cherry picked from commit 8e0fd5fbfa7c770f6b668e79b772179946027bce)

commit 459b6b2b224953655fd0360e8098ae598e41d3b2
Author: Matt Benjamin <mbenjamin@xxxxxxxxxx>
Date:   Wed May 15 15:53:32 2019 -0400

    rgw_file: include tenant when hashing bucket names

    Prevent identical paths from distinct tenants from colliding in
    RGW NFS handle cache.

    Fixes: http://tracker.ceph.com/issues/40118

    Signed-off-by: Matt Benjamin <mbenjamin@xxxxxxxxxx>
    (cherry picked from commit b800a9de83dff23a150ed7d236cb61c8b7d971ae)
    Signed-off-by: Matt Benjamin <mbenjamin@xxxxxxxxxx>


ganesha.conf.deuxtenant:


EXPORT
{
    # Export Id (mandatory, each EXPORT must have a unique Export_Id)
    Export_Id = 77;

    # Exported path (mandatory)
    Path = "/";

    # Pseudo Path (required for NFS v4)
    Pseudo = "/userx";

    # Required for access (default is None)
    # Could use CLIENT blocks instead
    Access_Type = RW;

    SecType = "sys";

    Protocols = 3,4;
    Transports = UDP,TCP;

    #Delegations = Readwrite;

    Squash = No_Root_Squash;

    # Exporting FSAL
    FSAL {
        Name = RGW;
        User_Id = "userx";
        Access_Key_Id = "userxacc";
        Secret_Access_Key = "test123";
    }
}

EXPORT
{
    # Export Id (mandatory, each EXPORT must have a unique Export_Id)
    Export_Id = 78;

    # Exported path (mandatory)
    Path = "/";

    # Pseudo Path (required for NFS v4)
    Pseudo = "/usery";

    # Required for access (default is None)
    # Could use CLIENT blocks instead
    Access_Type = RW;

    SecType = "sys";

    Protocols = 3,4;
    Transports = UDP,TCP;

    #Delegations = Readwrite;

    Squash = No_Root_Squash;

    # Exporting FSAL
    FSAL {
        Name = RGW;
        User_Id = "usery";
        Access_Key_Id = "useryacc";
        Secret_Access_Key = "test456";
    }
}

#mount at bucket case
EXPORT
{
    # Export Id (mandatory, each EXPORT must have a unique Export_Id)
    Export_Id = 79;

    # Exported path (mandatory)
    Path = "/buck5";

    # Pseudo Path (required for NFS v4)
    Pseudo = "/usery_buck5";

    # Required for access (default is None)
    # Could use CLIENT blocks instead
    Access_Type = RW;

    SecType = "sys";

    Protocols = 3,4;
    Transports = UDP,TCP;

    #Delegations = Readwrite;

    Squash = No_Root_Squash;

    # Exporting FSAL
    FSAL {
        Name = RGW;
        User_Id = "usery";
        Access_Key_Id = "useryacc";
        Secret_Access_Key = "test456";
    }
}



RGW {
    ceph_conf = "/home/mbenjamin/ceph-noob/build/ceph.conf";
    #init_args = "-d --debug-rgw=16";
    init_args = "";
}

NFS_Core_Param {
        Nb_Worker = 17;
        mount_path_pseudo = true;
}

CacheInode {
    Chunks_HWMark = 70000;
    Entries_Hwmark = 2000000;
}

NFSV4 {
    Graceless = true;
    Allow_Numeric_Owners = true;
    Only_Numeric_Owners = true;
}

LOG {
    Components {
        #NFS_READDIR = FULL_DEBUG;
        #NFS4 = FULL_DEBUG;
        #CACHE_INODE = FULL_DEBUG;
        #FSAL = FULL_DEBUG;
    }
    Facility {
        name = FILE;
        destination = "/tmp/ganesha-rgw.log";
        enable = active;
    }
}

On Thu, Oct 3, 2019 at 10:34 AM Marc Roos <M.Roos@xxxxxxxxxxxxxxxxx> 
wrote:
>
>
> How should a multi tenant RGW config look like, I am not able get this
> working:
>
> EXPORT {
>        Export_ID=301;
>        Path = "test:test3";
>        #Path = "/";
>        Pseudo = "/rgwtester";
>
>        Protocols = 4;
>        FSAL {
>                Name = RGW;
>                User_Id = "test$tester1";
>                Access_Key_Id = "TESTER";
>                Secret_Access_Key = "xxx";
>        }
>        Disable_ACL = TRUE;
>        CLIENT { Clients = 192.168.10.0/24; access_type = "RO"; } }
>
>
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> create_export :FSAL :CRIT :RGW module: librgw init failed (-5)
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> mdcache_fsal_create_export :FSAL :MAJ :Failed to call create_export on
> underlying FSAL RGW
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> fsal_put :FSAL :INFO :FSAL RGW now unused
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> fsal_cfg_commit :CONFIG :CRIT :Could not create export for 
(/rgwtester)
> to (test:test3)
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> fsal_cfg_commit :FSAL :F_DBG :FSAL RGW refcount 0
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> config_errs_to_log :CONFIG :CRIT :Config File
> (/etc/ganesha/ganesha.conf:216): 1 validation errors in block FSAL
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> config_errs_to_log :CONFIG :CRIT :Config File
> (/etc/ganesha/ganesha.conf:216): Errors processing block (FSAL)
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> config_errs_to_log :CONFIG :CRIT :Config File
> (/etc/ganesha/ganesha.conf:209): 1 validation errors in block EXPORT
> 03/10/2019 16:15:37 : epoch 5d8d274c : c01 : ganesha.nfsd-4722[sigmgr]
> config_errs_to_log :CONFIG :CRIT :Config File
> (/etc/ganesha/ganesha.conf:209): Errors processing block (EXPORT)
>
> -----Original Message-----
> Subject: Re:  NFS
>
> RGW NFS can support any NFS style of authentication, but users will 
have
> the RGW access of their nfs-ganesha export.  You can create exports 
with
> disjoint privileges, and since recent L, N, RGW tenants.
>
> Matt
>
> On Tue, Oct 1, 2019 at 8:31 AM Marc Roos <M.Roos@xxxxxxxxxxxxxxxxx>
> wrote:
> >
> >  I think you can run into problems
> > with a multi user environment of RGW and nfs-ganesha.
> >
> > _______________________________________________
> > ceph-users mailing list
> > ceph-users@xxxxxxxxxxxxxx
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> --
>
> Matt Benjamin
> Red Hat, Inc.
> 315 West Huron Street, Suite 140A
> Ann Arbor, Michigan 48103
>
> http://www.redhat.com/en/technologies/storage
>
> tel.  734-821-5101
> fax.  734-769-8938
> cel.  734-216-5309
>
>


-- 

Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-821-5101
fax.  734-769-8938
cel.  734-216-5309


_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux