ACL nightmare on RadosGW for 200 TB dataset

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello there,

Our setup is with Ceph Hammer (latest release).

We want to publish in our Object Storage some Scientific Datasets.
These are collections of around 100K objects and total size of about
200 TB.

For Object Storage we use the RadosGW with S3 API.

For the initial testing we are using a smaller dataset of about 26K
files and 5Tb of data.

Authentication to radosGW is with Keystone integration.

We created a Openstack Tenant to manage the datasets, and with EC2
credentials we upload all the files.
Once the bucket is full lets look at the ACLs:

s3cmd info s3://googlebooks-ngrams-gz/

ACL: TENANTDATASET: FULL_CONTROL

So far so good.

At this point we want to enable a user of a different tenant, to
access this Dataset READ-ONLY.

Given the UUID of the tenant of the user it would be as easy as:

s3cmd setacl --acl-grant=read:<UUID> s3://googlebooks-ngrams-gz/

However this is not enough, the user will be able to list the objects
of the bucket, but not to read them. The read ACL is not inherited for
the Objects from the Bucket. So we must do:

s3cmd setacl --acl-grant=read:<UUID> --recursive s3://googlebooks-ngrams-gz/

But this takes ages on 26K objects. It works but you spend several
hours updating ACLs and we cannot have this procedure everytime a user
wants read access.

Now the painful questions:

There is a way to bulk update the "read acl" on all the objects of a bucket ???

What happens to ACLs when SWIFT and S3 API are used simultaneously ?
>From my test RadosGW ignores the swift client when we try to post
ACLs, however the swift API honors S3 ACLs when reading.

Saverio
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux