By default, radosgw only returns the first 1000 objects. Looks like radosgw-admin has the same limit.
Looking at the man page, I don't see any way to page through the list. I must be missing something.
The S3 API does have the ability to page through the list. I use the command line tool s3cmd to get the full bucket list. It does require user credentials though, so that might be a pain if you have many users.
On Wed, Mar 11, 2015 at 6:47 PM, Sean Sullivan <seapasulli@xxxxxxxxxxxx> wrote:
I have a single radosgw user with 2 s3 keys and 1 swift key. I have created a few buckets and I can list all of the contents of bucket A and C but not B with either S3 (boto) or python-swiftclient. I am able to list the first 1000 entries using radosgw-admin 'bucket list --bucket=bucketB' without any issues but this doesn't really help.
The odd thing is I can still upload and download objects in the bucket. I just can't list them. I tried setting the bucket canned_acl to private and public but I still can't list the objects inside.
I'm using ceph .87 (Giant) Here is some info about the cluster::
http://pastebin.com/LvQYnXem -- ceph.conf
http://pastebin.com/efBBPCwa -- ceph -s
http://pastebin.com/tF62WMU9 -- radosgw-admin bucket list
http://pastebin.com/CZ8TkyNG -- python list bucket objects script
http://pastebin.com/TUCyxhMD -- radosgw-admin bucket stats --bucketB
http://pastebin.com/uHbEtGHs -- rados -p .rgw.buckets ls | grep default.20283.2 (bucketB marker)
http://pastebin.com/WYwfQndV -- Python Error when trying to list BucketB via boto
I have no idea why this could be happening outside of the acl. Has anyone seen this before? Any idea on how I can get access to this bucket again via s3/swift? Also is there a way to list the full list of a bucket via radosgw-admin and not the first 9000 lines / 1000 entries, or a way to page through them?
EDIT:: I just fixed it (I hope) but the fix doesn't make any sense:
radosgw-admin bucket unlink --uid=user --bucket=bucketB
radosgw-admin bucket link --uid=user --bucket=bucketB --bucket-id=default.20283.2
Now with swift or s3 (boto) I am able to list the bucket contents without issue ^_^
Can someone elaborate on why this works and how it broken in the first place when ceph was health_ok the entire time? With 3 replicas how did this happen? Could this be a bug? sorry for the rambling. I am confused and tired ;p
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
