On 11/26/2015 09:03 AM, Somnath Roy wrote: > Thanks Wido ! > Could you please explain a bit more on the relationship between user created buckets and the objects within .bucket.index pool ? > I am not seeing for each bucket one entry is created within .bucket.index pool. > I don't think the bucket index is created when you actually create a bucket. IIRC it creates the index when the first object is written to the bucket. But I could be wrong. > Regards > Somnath > > -----Original Message----- > From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Wido den Hollander > Sent: Wednesday, November 25, 2015 10:56 PM > To: ceph-users@xxxxxxxxxxxxxx > Subject: Re: RGW pool contents > > On 11/24/2015 08:48 PM, Somnath Roy wrote: >> Hi Yehuda/RGW experts, >> >> I have one cluster with RGW up and running in the customer site. >> >> I did some heavy performance testing on that with CosBench and as a >> result written significant amount of data to showcase performance on that. >> >> Over time, customer also wrote significant amount of data using S3 api >> into the cluster. >> >> Now, I want to remove the buckets/objects created by CosBench and need >> some help on that. >> >> I ran the following command to list the buckets. >> >> >> >> "radosgw-admin bucket list" >> >> >> >> The output is the following snippet.. >> >> >> >> "rgwdef42", >> >> "rgwdefghijklmnop79", >> >> "rgwyzabc43", >> >> "rgwdefgh43", >> >> "rgwdefghijklm200", >> >> >> >> .......................... >> >> .......................... >> >> >> >> My understanding is , cosbench should create containers with >> "*mycontainers_*<n>" and objects with format "*myobjects*_<n>" prefix >> (?). But, it's not there in the output of the above command. >> >> > > Well, if it did, they should show up there. > >> >> Next, I tried to list the contents of the different rgw pools.. >> >> >> >> *rados -p .rgw.buckets.index ls* >> >> >> >> .dir.default.5407.17 >> >> .dir.default.6063.24 >> >> .dir.default.6068.23 >> >> .dir.default.6046.7 >> >> .dir.default.6065.44 >> >> .dir.default.5409.3 >> >> ........... >> >> ........... >> >> >> >> Nothing with rgw prefix...Shouldn't the bucketindex objects having >> similar prefix with bucket names ? >> > > No, there are the internal IDs of the buckets. You can find the actual bucket objects in the ".rgw" pool. > >> >> >> >> >> Now, tried to get the actual objects... >> >> *rados -p .rgw.buckets ls* >> >> >> >> default.6662.5_myobjects57862 >> >> default.5193.18_myobjects6615 >> >> default.5410.5_myobjects68518 >> >> default.6661.8_myobjects7407 >> >> default.5410.22_myobjects54939 >> >> default.6651.6_myobjects23790 >> >> >> >> ............................ >> >> ........................... >> >> >> >> So, looking at these, it seems cosbench run is creating the >> .dir.default.<n>* buckets and the default.<n>_myobjects<n>* objects >> (?) >> > > No, again, the .dir.default.X is the internal ID of the bucket. It creates "myobject" object on those buckets. > >> >> >> But, these buckets are not listed by the first "radosgw-admin" >> command, *why ?* >> >> >> >> Next, I listed the contents of the .rgw pool and here is the output.. >> >> >> >> *rados -p .rgw ls* >> >> >> >> .bucket.meta.rgwdefghijklm78:default.6069.18 >> >> rgwdef42 >> >> rgwdefghijklmnop79 >> >> rgwyzabc43 >> >> .bucket.meta.rgwdefghijklmnopqr71:default.6655.3 >> >> rgwdefgh43 >> >> .bucket.meta.rgwdefghijklm119:default.6066.25 >> >> rgwdefghijklm200 >> >> .bucket.meta.rgwxghi2:default.5203.4 >> >> rgwxjk17 >> >> rgwdefghijklm196 >> >> >> >> ................... >> >> ................... >> >> >> >> It seems this pool has the buckets listed by the radosgw-admin command. >> >> >> >> Can anybody explain what is *.rgw pool* supposed to contain ? >> >> > > This pool contains only the bucket metadata objects, here it references to the internal IDs. > > You can fetch this with 'radosgw-admin metadata get bucket:XX' > >> >> Also, what is the difference between .*users.uid and .users pool* ? >> >> > > In the .user.uid pool the RGW can do a quick query for users IDs since that is required for matching ACLs which might be on a bucket and/or object. > > Wido > >> >> >> >> Appreciate any help on this. >> >> >> >> Thanks & Regards >> >> Somnath >> >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > > > -- > Wido den Hollander > 42on B.V. > Ceph trainer and consultant > > Phone: +31 (0)20 700 9902 > Skype: contact42on > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Wido den Hollander 42on B.V. Ceph trainer and consultant Phone: +31 (0)20 700 9902 Skype: contact42on _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |