Re: Started developing a deduplication feature

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sat, Apr 9, 2016 at 12:01 AM, Marcel Lauhoff <lauhoff@xxxxxxxxxxxx> wrote:
>
> Hi list,
> short recap of the dedup topic from the CDM on Wednesday:
>
>
> The main change from the original mail is not to add a PG backend, but
> rather use Object Redirects (Tiering v2).
> Another backend would have to implement its own replication for
> recipes and increase the OSD code base just for dedup. Redirects are
> useful beyond deduplication.
>
>
> The CAS pool design was refined: An object class should handle the ref
> counting and content addressing. The pool should also only
> allow access through this object class to prevent collisions with
> regular objects and support immutable objects.
>
>
> There was also the idea of client-side deduplication by using metadata
> that clients like RGW store. This would save the additional round trip
> that object redirects add.

Sounds good.

>
>
> I'll be working on the CAS pool first, since there
> is ongoing refactoring in the ReplicatedPG code base. I'll work out a
> more detailed design document for the CAS pool soon.

I will be able to test it against if you don't mind.

>
>
> ~irq0
> --
> Marcel Lauhoff
> Mail: lauhoff@xxxxxxxxxxxx
> XMPP: mlauhoff@xxxxxxxxxxxxxxxxxxx
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Cheers,
Shinobu

-- 
Email:
shinobu@xxxxxxxxx
GitHub:
shinobu-x
Blog:
Life with Distributed Computational System based on OpenSource
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux