Hi, On Sunday, February 21st, 2021 at 12:39, Loïc Dachary <loic@xxxxxxxxxxx> wrote: > For the record, here is a summary of the key takeaways from this conversation (so far): > > - Ambry[0] is a perfect match and I'll keep exploring it[1]. > - To keep billions of small objects manageable, they must be packed together. > - Immutable & never deleted objects can be grouped together for the purpose of packing them without a central database. For this to work the id of the group to which an object belongs is included in the object ID (e.g. SHA256 + UUID of the group). That's what Ambry does. What about using OMAP for the purpose of "grouping together" small objects? You would store tiny objects as OMAP key/value pairs on a Ceph object. Internally, they're stored in RocksDB so the min allocation size isn't an issue, and retrieving individual objects should be fairly quick. Has anyone tried this? How does RocksDB cope with terabytes of data and hundreds of millions of key-value pairs? Is recovery faster for OMAP compared to the equivalent number of RADOS objects? Cheers, -- Ben _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |