Wido den Hollander wrote: : : > Op 27 februari 2017 om 15:59 schreef Jan Kasprzak <kas@xxxxxxxxxx>: : > : > : > Here is some statistics from our biggest instance of the object storage: : > : > : > : > : > : > objects stored: 100_000_000 : > : > : > < 1024 bytes: 10_000_000 : > : > : > 1k-64k bytes: 80_000_000 : > : > : > 64k-4M bytes: 10_000_000 : > : > : > 4M-256M bytes: 1_000_000 : > : > : >> 256M bytes: 10_000 : > : > : > biggest object: 15 GBytes : > : > : > : > : > : > Would it be feasible to put 100M to 1G objects as a native RADOS objects : > : > : > into a single pool? [...] : > https://github.com/ceph/ceph/blob/master/src/libradosstriper/RadosStriperImpl.cc#L33 : > : > If I understand it correctly, it looks like libradosstriper only splits : > large stored objects into smaller pieces (RADOS objects), but does not : > consolidate more small stored objects into larger RADOS objects. : : Why would you want to do that? Yes, very small objects can be a problem if you have millions of them since it takes a bit more to replicate them and recover them. Yes. This is what I was afraid of. The immutability of my objects would allow to consolidate smaller objects into larger bundles, but if you say is not necessary for the problem of my size, I'll store them into individual RADOS objects. : : But overall I wouldn't bother about it too much. OK, thanks! : > So do you think I am ok with >10M tiny objects (smaller than 1KB) : > and ~100,000,000 to 1,000,000,000 total objects, provided that I split : > huge objects using libradosstriper? -Yenya -- | Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}> | | http://www.fi.muni.cz/~kas/ GPG: 4096R/A45477D5 | Assuming that OpenSSL is written as carefully as Wietse's own code, every 1000 lines introduce one additional bug into Postfix." --TLS_README _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |