Hi,
just a comment (and please correct my if I'm wrong)....
There are no "folders" in S3. A bucket is a plain list of objects. What
you recognize as a folder is an artificial construct, e.g. usual path
delimiter used by S3 access tool to create "folders".
As a result, listing a bucket with 6 million objects in 875 "folders"
does require listing all 6 million objects. You can validate this by
looking at the requests send to the RGW (for example using 's3cmd -d la'):
...
DEBUG: Sending request method_string='GET', uri='/?delimiter=/',
headers={'x-amz-content-sha256': 'XXXXXXX', 'Authorization':
'AWS4-HMAC-SHA256
Credential=XXXXXXX/US/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=XXXXXX',
'x-amz-date': '20190508T073339Z'}, body=(0 bytes)
....
And compare the request URL to the S3 API spec:
https://docs.aws.amazon.com/AmazonS3/latest/API/v2-RESTBucketGET.html
'delimiter=/' is just a convenience parameter for grouping the results.
The implementation still has to enumerate all objects.
Regards,
Burkhard
--
Dr. rer. nat. Burkhard Linke
Bioinformatics and Systems Biology
Justus-Liebig-University Giessen
35392 Giessen, Germany
Phone: (+49) (0)641 9935810
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
