On Fri, 8 Apr 2016, Marcel Lauhoff wrote: > Hi list, > short recap of the dedup topic from the CDM on Wednesday: > > The main change from the original mail is not to add a PG backend, but > rather use Object Redirects (Tiering v2). Let's call this 'Tiering v2', since it won't be based on redirects. (These were extremely problematic in cache tiering because they prevent us from maintaining an ordering; we now proxy ops instead.) Some point soonish we should recreate/update that old blueprint with the new design. > Another backend would have to implement its own replication for > recipes and increase the OSD code base just for dedup. Redirects are > useful beyond deduplication. > > The CAS pool design was refined: An object class should handle the ref > counting and content addressing. cls_refcount might be sufficient here; if not it's probably a starting point. > The pool should also only > allow access through this object class to prevent collisions with > regular objects and support immutable objects. We decided we can just do this via the cephx capabilities by granting the client(s) access to the appropriate class only. > There was also the idea of client-side deduplication by using metadata > that clients like RGW store. This would save the additional round trip > that object redirects add. Presumably the (first) user here is radosgw, which is already writing more or less immutable chunks and could easily dump them in a CAS pool instead of a normal replicated pool. > I'll be working on the CAS pool first, since there > is ongoing refactoring in the ReplicatedPG code base. I'll work out a > more detailed design document for the CAS pool soon. Sounds great. Thanks, Marcel! sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
