----- Original Message ----- From: Wido den Hollander <wido@xxxxxxxx> To: Василий Ангапов <angapov@xxxxxxxxx> Date: Tuesday, June 14, 2016, 12:05:51 PM Subject: 40Mil objects in S3 rados pool / how calculate PGs >> Op 14 juni 2016 om 11:00 schreef Василий Ангапов <angapov@xxxxxxxxx>: >> >> >> Is it a good idea to disable scrub and deep-scrub for bucket.index >> pool? What negative consequences it may cause? >> > No, I would not do that. Scrubbing is essential to detect (silent) data corruption. > You should really scrub all your data. Ceph do not protect from silent data corruption at all. You can read this thread http://lists.ceph.com/pipermail/ceph-users-ceph.com/2016-February/007680.html >> 2016-06-14 11:51 GMT+03:00 Wido den Hollander <wido@xxxxxxxx>: >> > >> >> Op 14 juni 2016 om 10:10 schreef Ansgar Jazdzewski <a.jazdzewski@xxxxxxxxxxxxxx>: >> >> >> >> >> >> Hi, >> >> >> >> we are using ceph and radosGW to store images (~300kb each) in S3, >> >> when in comes to deep-scrubbing we facing task timeouts (> 30s ...) >> >> >> >> my questions is: >> >> >> >> in case of that amount of objects/files is it better to calculate the >> >> PGs on a object-bases instant of the volume size? and how it should be >> >> done? >> >> >> > >> > Do you have bucket sharding enabled? >> > >> > And how many objects do you have in a single bucket? >> > >> > If sharding is not enabled for the bucket index you might have large RADOS objects with bucket indexes which are hard to scrub. >> > >> > Wido >> > >> >> thanks >> >> Ansgar >> >> _______________________________________________ >> >> ceph-users mailing list >> >> ceph-users@xxxxxxxxxxxxxx >> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > _______________________________________________ >> > ceph-users mailing list >> > ceph-users@xxxxxxxxxxxxxx >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |