Re: Pluggable interface for erasure coding?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



That is great.

Here is an appear.in room we can use for the meeting tomorrow: https://appear.in/erasurecode41

Looking forward to the meeting!

Best,
Per


On Fri, Mar 3, 2017 at 1:15 PM, Xavier Hernandez <xhernandez@xxxxxxxxxx> wrote:
March 8th also works for me.

On 03/03/17 06:18, Pranith Kumar Karampuri wrote:
8th March works for me. 9th I am travelling. Xavi is well versed with
this part of the code, so even if I am not available, you guys should
pick the time which is convenient for Xavi.

The following may be of help for this discussion.
https://github.com/gluster/glusterfs/blob/master/doc/developer-guide/ec-implementation.md

If my memory serves right, Xavi's implementation outperformed
liberasure, that is the reason we didn't integrate with liberasure sooner.


On Fri, Mar 3, 2017 at 7:01 AM, Per Simonsen <per.simonsen@xxxxxxxxxxxxx
<mailto:per.simonsen@memoscale.com>> wrote:

    Hi,

    I forgot to add the time zone: the suggested time is 10 am (GMT+1).

    Best,
    Per

    On Thu, Mar 2, 2017 at 5:19 PM, Per Simonsen
    <per.simonsen@xxxxxxxxxxxxx <mailto:per.simonsen@memoscale.com>> wrote:

        Hi,

        I suggest that we set up an online meeting next week to discuss
        the erasure coding features as well as possible implementations
        of a plugin architecture. We also have some experience
        integrating with the liberasure library mentioned by Prashant
        which we can share.

        Does 10 am on Wednesday(8th of March) or Thursday(9th of March)
        next week work for you guys?

        Best,
        Per Simonsen
        CEO
        MemoScale


        On Thu, Mar 2, 2017 at 12:00 AM, Xavier Hernandez
        <xhernandez@xxxxxxxxxx <mailto:xhernandez@xxxxxxxxxx>> wrote:

            Hi Niels,

            On 02/03/17 07:58, Niels de Vos wrote:

                Hi guys,

                I think this is a topic/question that has come up
                before, but I can not
                find any references or feature requests related to it.
                Because there are
                different libraries for Erasure Coding, it would be
                interesting to be
                able to select alternatives to the bundled
                implementation that Gluster
                has.


            I agree.

                Are there any plans to make the current Erasure Coding
                implementation more pluggable?


            Yes. I've had this in my todo list for a long time. Once I
            even tried to implement the necessary infrastructure but
            didn't finish and now the code has changed too much to reuse it.

                Would this be a possible feature request,
                or would it require a major rewrite of the current
                interface?


            At the time I tried it, it required major changes. Now that
            the code has been considerably restructured to incorporate
            the dynamic code generation feature, maybe it doesn't
            require so many changes, though I'm not sure.


                Here at FAST [0] I have briefly spoken to Per Simonsen
                from MemoScale
                [1]. This company offers a (proprietary) library for
                Erasure Coding,
                optimized for different architectures, and  with some
                unique(?) features
                for recovering a failed fragment/disk. If Gluster allows
                alternative
                implementations for the encoding, it would help
                organisations and
                researchers to get results of their work in a
                distributed filesystem.
                And with that, spread the word about how easy to adapt
                and extend
                Gluster is :-)


            That could be interesting. Is there any place where I can
            find additional information about the features of this library ?

            Xavi



                Thanks,
                Niels


                0. https://www.usenix.org/conference/fast17
                <https://www.usenix.org/conference/fast17>
                1. https://memoscale.com/







--
Pranith


_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-devel

[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux