> Op 27 februari 2017 om 15:59 schreef Jan Kasprzak <kas@xxxxxxxxxx>: > > > Hello, > > Gregory Farnum wrote: > : On Mon, Feb 20, 2017 at 11:57 AM, Jan Kasprzak <kas@xxxxxxxxxx> wrote: > : > Gregory Farnum wrote: > : > : On Mon, Feb 20, 2017 at 6:46 AM, Jan Kasprzak <kas@xxxxxxxxxx> wrote: > : > : > > : > : > I have been using CEPH RBD for a year or so as a virtual machine storage > : > : > backend, and I am thinking about moving our another subsystem to CEPH: > [...] > : > : > Here is some statistics from our biggest instance of the object storage: > : > : > > : > : > objects stored: 100_000_000 > : > : > < 1024 bytes: 10_000_000 > : > : > 1k-64k bytes: 80_000_000 > : > : > 64k-4M bytes: 10_000_000 > : > : > 4M-256M bytes: 1_000_000 > : > : >> 256M bytes: 10_000 > : > : > biggest object: 15 GBytes > : > : > > : > : > Would it be feasible to put 100M to 1G objects as a native RADOS objects > : > : > into a single pool? > : > : > : > : This is well outside the object size RADOS is targeted or tested with; > : > : I'd expect issues. You might want to look at libradosstriper from the > : > : requirements you've mentioned. > : > > : > OK, thanks! Is there any documentation for libradosstriper? > : > I am looking for something similar to librados documentation: > : > http://docs.ceph.com/docs/master/rados/api/librados/ > : > : Not that I see, and I haven't used it myself, but the header file (see > : ceph/src/libradosstriper) seems to have reasonable function docs. It's > : a fairly thin wrapper around librados AFAIK. > > OK, I have read the docs in the header file and the comment > near the top of RadosStriperImpl.cc: > > https://github.com/ceph/ceph/blob/master/src/libradosstriper/RadosStriperImpl.cc#L33 > > If I understand it correctly, it looks like libradosstriper only splits > large stored objects into smaller pieces (RADOS objects), but does not > consolidate more small stored objects into larger RADOS objects. Why would you want to do that? Yes, very small objects can be a problem if you have millions of them since it takes a bit more to replicate them and recover them. But overall I wouldn't bother about it too much. Wido > > So do you think I am ok with >10M tiny objects (smaller than 1KB) > and ~100,000,000 to 1,000,000,000 total objects, provided that I split > huge objects using libradosstriper? > > Thanks, > > -Yenya > > -- > | Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}> | > | http://www.fi.muni.cz/~kas/ GPG: 4096R/A45477D5 | > Assuming that OpenSSL is written as carefully as Wietse's own code, > every 1000 lines introduce one additional bug into Postfix." --TLS_README > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |