> Op 2 augustus 2017 om 17:55 schreef Marcus Haarmann <marcus.haarmann@xxxxxxxxx>: > > > Hi, > we are doing some tests here with a Kraken setup using bluestore backend (on Ubuntu 64 bit). > We are trying to store > 10 mio very small objects using RADOS. > (no fs, no rdb, only osd and monitors) > > The setup was done with ceph-deploy, using the standard bluestore option, no separate devices > for wal. The test cluster spreads over 3 virtual machines, each with 100GB storage für osd. > > We are now in the following situation (used pool is "test"): > rados df > POOL_NAME USED OBJECTS CLONES COPIES MISSING_ON_PRIMARY UNFOUND DEGRAED RD_OPS RD WR_OPS WR > rbd 0 2 0 6 0 0 0 49452 39618k 855 12358k > test 17983M 595427 0 1786281 0 0 0 29 77824 596426 17985M > > total_objects 595429 > total_used 141G > total_avail 158G > total_space 299G > > ceph osd df > ID WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS > 0 0.09760 1.00000 102298M 50763M 51535M 49.62 1.00 72 > 1 0.09760 1.00000 102298M 50799M 51499M 49.66 1.00 72 > 2 0.09760 1.00000 102298M 50814M 51484M 49.67 1.00 72 > TOTAL 299G 148G 150G 49.65 > MIN/MAX VAR: 1.00/1.00 STDDEV: 0.02 > > As you can see, there are about 18GB data stored in ~595000 objects now. > The actual space consumption is about 150GB, which fills about half of the storage. > Not really. Each OSD uses 50GB, but since you replicate 3 times (default) it's storing 150GB spread out over 3 OSDs. So your data is 18GB, but consumes 50GB. That's still ~2.5x which is a lot, but a lot less then 150GB. > Objects have been added with a test script using the rados command line (put). > > Obviously, the stored objects are counted byte by byte in the rados df command, > but the real space allocation is about factor 8. > As written above, it's ~2.5x, not 8x. > The stored objects are a mixture of 2kb, 10kb, 50kb, 100kb objects. > > Is there any recommended way to configure bluestore with a better suitable > block size for those small objects ? I cannot find any configuration option > which would allow modification of the internal block handling of bluestore. > Is luminous an option which allows more specific configuration ? > Could you try this with the Luminous RC as well? I don't know the answer here, but since Kraken a LOT has been improved to BlueStore. Wido > Thank you all in advance for support. > > Marcus Haarmann > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |