Thanks Wido ! Could you please explain a bit more on the relationship between user created buckets and the objects within .bucket.index pool ? I am not seeing for each bucket one entry is created within .bucket.index pool. Regards Somnath -----Original Message----- From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Wido den Hollander Sent: Wednesday, November 25, 2015 10:56 PM To: ceph-users@xxxxxxxxxxxxxx Subject: Re: RGW pool contents On 11/24/2015 08:48 PM, Somnath Roy wrote: > Hi Yehuda/RGW experts, > > I have one cluster with RGW up and running in the customer site. > > I did some heavy performance testing on that with CosBench and as a > result written significant amount of data to showcase performance on that. > > Over time, customer also wrote significant amount of data using S3 api > into the cluster. > > Now, I want to remove the buckets/objects created by CosBench and need > some help on that. > > I ran the following command to list the buckets. > > > > "radosgw-admin bucket list" > > > > The output is the following snippet.. > > > > "rgwdef42", > > "rgwdefghijklmnop79", > > "rgwyzabc43", > > "rgwdefgh43", > > "rgwdefghijklm200", > > > > .......................... > > .......................... > > > > My understanding is , cosbench should create containers with > "*mycontainers_*<n>" and objects with format "*myobjects*_<n>" prefix > (?). But, it's not there in the output of the above command. > > Well, if it did, they should show up there. > > Next, I tried to list the contents of the different rgw pools.. > > > > *rados -p .rgw.buckets.index ls* > > > > .dir.default.5407.17 > > .dir.default.6063.24 > > .dir.default.6068.23 > > .dir.default.6046.7 > > .dir.default.6065.44 > > .dir.default.5409.3 > > ........... > > ........... > > > > Nothing with rgw prefix...Shouldn't the bucketindex objects having > similar prefix with bucket names ? > No, there are the internal IDs of the buckets. You can find the actual bucket objects in the ".rgw" pool. > > > > > Now, tried to get the actual objects... > > *rados -p .rgw.buckets ls* > > > > default.6662.5_myobjects57862 > > default.5193.18_myobjects6615 > > default.5410.5_myobjects68518 > > default.6661.8_myobjects7407 > > default.5410.22_myobjects54939 > > default.6651.6_myobjects23790 > > > > ............................ > > ........................... > > > > So, looking at these, it seems cosbench run is creating the > .dir.default.<n>* buckets and the default.<n>_myobjects<n>* objects > (?) > No, again, the .dir.default.X is the internal ID of the bucket. It creates "myobject" object on those buckets. > > > But, these buckets are not listed by the first "radosgw-admin" > command, *why ?* > > > > Next, I listed the contents of the .rgw pool and here is the output.. > > > > *rados -p .rgw ls* > > > > .bucket.meta.rgwdefghijklm78:default.6069.18 > > rgwdef42 > > rgwdefghijklmnop79 > > rgwyzabc43 > > .bucket.meta.rgwdefghijklmnopqr71:default.6655.3 > > rgwdefgh43 > > .bucket.meta.rgwdefghijklm119:default.6066.25 > > rgwdefghijklm200 > > .bucket.meta.rgwxghi2:default.5203.4 > > rgwxjk17 > > rgwdefghijklm196 > > > > ................... > > ................... > > > > It seems this pool has the buckets listed by the radosgw-admin command. > > > > Can anybody explain what is *.rgw pool* supposed to contain ? > > This pool contains only the bucket metadata objects, here it references to the internal IDs. You can fetch this with 'radosgw-admin metadata get bucket:XX' > > Also, what is the difference between .*users.uid and .users pool* ? > > In the .user.uid pool the RGW can do a quick query for users IDs since that is required for matching ACLs which might be on a bucket and/or object. Wido > > > > Appreciate any help on this. > > > > Thanks & Regards > > Somnath > > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Wido den Hollander 42on B.V. Ceph trainer and consultant Phone: +31 (0)20 700 9902 Skype: contact42on _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |