Yes, this is certainly an interesting topic. It also seems
quite relevant for RGW, which has a similar issue handling small
immutable objects. IIRC there were some ideas around batching
small objects together at that layer.
Josh
On 3/25/21 5:23 AM, Loïc Dachary wrote:
Thank for the firm approval :-) I'll prepare a pad then!
On 25/03/2021 10:51, Ilya Dryomov wrote:
On Thu, Mar 25, 2021 at 10:16 AM Loïc Dachary <loic@xxxxxxxxxxx> wrote:
Bonjour Ceph,
(And thanks for the announcement Mike ;-) )
I wonder if the topic "Improving immutable objects use cases" is relevant for the CDS. It is a little vague, reason why I'm hesitant. There has been technical discussions on the benefits of packing large quantities of small immutable objects[0], comparisons with Ambry[1] or Seaweedfs[2] and I'm benchmarking the use of RBD to improve how billions of small objects can be mirrored and packed together[3]. These efforts are however focused on how to workaround Ceph limitations (space amplification, efficiency of listing all objects for mirroring purposes) to address these particular use cases.
Maybe the better solution would be to improve Ceph so that these workarounds for immutable objects are not necessary? Could Ceph treat small immutable and never deleted objects in such a way that there is no space amplification? Could Ceph provide an efficient way to mirror a pool of immutable and never deleted objects? To be more concrete (and probably naive), maybe a specialized OSD could be developed that would only accept immutable and never deleted objects and take advantage of these properties.
Do you think this is a topic worth discussing during the CDS?
Hi Loïc,
Definitely! I think it's worth talking about both the current most
viable solution (implementing something similar to EOS on top of rbd,
IIUC) and also possible object store backend improvements to manage
space amplification. The rbd segment would be a natural fit for the
former but I'd put something on the RADOS agenda as well.
Thanks,
Ilya
Cheers
[0] https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/thread/JSG2TXKNXPXEKZOJZGYF2ZPTQHOB4LHJ/
[1] https://github.com/linkedin/ambry/wiki
[2] https://github.com/chrislusf/seaweedfs/wiki/Components
[3] https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/thread/DRZ6XP45PEXKMYJSSYSDXC2RGJ3BW75H/
On 17/03/2021 05:55, Mike Perez wrote:
Hi everyone,
Mark your calendars for April 6 - 7 for the Ceph Developer Summit!
The plan is to have a virtual meeting style with a loose schedule
based on technical developer-focused discussions for the next software
release, Quincy. Each day (2-3 hours) will have development
discussions focused around a particular Ceph component.
The format for these sessions is primarily discussions, but
presentations are ok for visual diagrams, as an example. Birds of a
feather and other hallway-type tracks are acceptable as well.
Please follow this etherpad and the Ceph Dev mailing list for further
updates on exact start times and meeting link. Session proposals will
also be collected on this etherpad:
https://pad.ceph.com/p/cds-quincy
--
Mike Perez
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
--
Loïc Dachary, Artisan Logiciel Libre
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
