Re: writing a ceph cliente for MS windows

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Sorry, I mean YOU can have a BP first.

On 27 December 2014 at 13:14, Dong Yuan <yuandong1222@xxxxxxxxx> wrote:
> Why don't you push your impl? Maybe I can have a BP first.
>
> On 27 December 2014 at 01:10, Ketor D <d.ketor@xxxxxxxxx> wrote:
>> Is here any progress on Cephfs Windows Client ?
>>
>> 2013-11-08 22:15 GMT+08:00 Alphe Salas Michels <asalas@xxxxxxxxx>:
>>> Hello malcom and matt thank you for apporting some more information source.
>>> OpenAFS is sure interesting httpfs too.
>>>
>>> I hope it will help us on deciding the best path to follow in our interface
>>> with window.
>>> Actually I still trying to isolate the needed client code in the shortest
>>> way possible.
>>>
>>> Regards.
>>>
>>> Alphe Salas
>>>
>>> El nov 7, 2013 9:11 p.m., "Malcolm Haak" <malcolm@xxxxxxx
>>> <mailto:malcolm@xxxxxxx>> escribió:
>>>
>>>    I'm just going to throw these in there.
>>>
>>>    http://www.acc.umu.se/~bosse/ <http://www.acc.umu.se/%7Ebosse/>
>>>
>>>
>>>    They are GPLv2 some already use sockets and such from inside the
>>>    kernel.  Heck you might even be able to mod the HTTP one to use
>>>    rados gateway. I don't know as I havent sat down and pulled them
>>>    apart enough yet.
>>>
>>>    They might help, but they might be useless. Not sure.
>>>
>>>    On 08/11/13 06:47, Alphe Salas Michels wrote:
>>>
>>>        Hello all I finally finished my first source code extraction
>>>        that starts
>>>        from ceph/src/client/fuse_ll.c
>>>        The result is accurate unlike previous provided results.
>>>        basically the
>>>        script start from a file extract all the private includes
>>>        definitions
>>>        #include "something.h" and recursively extract private includes
>>>        too. the
>>>        best way to know who is related with who.
>>>
>>>        starting from fuse_ll.cc I optain 390 files retreived and 120
>>>        000 lines
>>>        of code !
>>>        involved dirs are : in ceph/src
>>>        objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
>>>        global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
>>>
>>>        probably not a good way to analyse what amount of work it means
>>>        since
>>>        most of those directories are the implementation of servers
>>>        (osd, mon,
>>>        mds) and even if only a tiny bit of them is needed at client
>>>        level. you
>>>        need two structures from ./osd/OSD.h and  my script by relation will
>>>        take into acount the whole directory...
>>>
>>>        I ran the script with libcephfs.cc as start point and got almost the
>>>        same results. 131 000 lines of code and 386 files most of the
>>>        same dirs
>>>        involved.
>>>
>>>
>>>
>>>        I think I will spend alot of time doing the manual source code
>>>        isolation
>>>        and understand way each #include is set in the files I read (what
>>>        purpose they have do they allow to integrate a crucial data type
>>>        or not.
>>>
>>>
>>>        The other way around will be to read src/libcephfs.cc. It seems
>>>        shorter
>>>        but without understanding what part is used for each included
>>>        header I
>>>        can t say anything...
>>>
>>>
>>>
>>>        I will keep reading the source code and take notes. I think in
>>>        the case
>>>        of libcephfs I will gain alot of time.
>>>
>>>        signature
>>>
>>>        *Alphé Salas*
>>>        Ingeniero T.I
>>>
>>>        asalas@xxxxxxxxx <mailto:asalas@xxxxxxxxx>
>>>        *www.kepler.cl <http://www.kepler.cl> <http://www.kepler.cl>*
>>>
>>>
>>>        On 11/07/13 15:02, Alphe Salas Michels wrote:
>>>
>>>            Hello D.Ketor and Matt Benjamin,
>>>            You give me alot to think about and this is great!
>>>            I merged your previous post to make a single reply that
>>>            anyone can
>>>            report to easyly
>>>
>>>            Windows NFS 4.1 is available here:
>>>            http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
>>>
>>>            pnfs is another name for NFS4.X. It is presented as
>>>            alternative to
>>>            ceph and we get known terminology as MDS and OSD but without
>>>            the self
>>>            healing part if I understand well my rapid look on the
>>>            topic. (when I
>>>            say rapid look I mean ... 5 minutes spent in that... which
>>>            is really
>>>            small amount of time to get an accurate view on something)
>>>
>>>
>>>            starting from mount.ceph ... I know that mount.ceph does
>>>            little but it
>>>            is a great hint to know what ceph needs and do things.
>>>            Basically mount.ceph modprobe the ceph driver in the linux
>>>            kernel then
>>>            call mount with the line command passed args and the cephfs
>>>            type as
>>>            argument. Then the kernel does the work I don t understand
>>>            yet what is
>>>            the start calls that are made to the ceph driver but it
>>>            seemed to me
>>>            that is was relatively light. (a first impression compared
>>>            to ceph-fuse.)
>>>
>>>            I think I will do both isolate source code from ceph-client
>>>            kernel
>>>            (cephfs module for linux kernel) and the one pointed by Sage
>>>            starting
>>>            from client/fuse_ll.cc in ceph master branch. The common
>>>            files betwin
>>>            those 2 extractions will be our core set of mandatory features.
>>>
>>>            Then we try to compile with cygwin a cephfs client library .
>>>            Then we
>>>            will try to interface with a modified windows nfs 4.1 client
>>>            or pnfs
>>>            or any other that will accept to be compiled with gcc for
>>>            win32...
>>>
>>>            the fact that windows 8.1 is and windows 2012 are out of
>>>            reach at the
>>>            moment is not a problem to me.
>>>
>>>            Our first concern is to understand what is ceph protocol.
>>>            Then adapt
>>>            it to something that can be used on windows prior windows
>>>            8.1. Dokan
>>>            fs if I remember well use too the WDK (windows driver
>>>            dev-kit ) for it
>>>            s compilation so possibly we will see the same limitations.
>>>
>>>            We need to multiply our source of information by example
>>>            regarding
>>>            ceph-client (kernel or fuse, radosgw is on a different layer
>>>            so I will
>>>            not try anything around it at first.) And we need to
>>>            multiply our
>>>            source of information by example regarding virtual file system
>>>            technologies on windoes OS.
>>>            Alot of work but all of those available source code everyone
>>>            point at
>>>            me will make our best solution. And in the end we will choose
>>>            technologies knowing what we do and what concequencies they
>>>            have.
>>>
>>>            regards,
>>>
>>>
>>>
>>>
>>>            Regards
>>>
>>>            signature
>>>
>>>            *Alphé Salas*
>>>            Ingeniero T.I
>>>
>>>            asalas@xxxxxxxxx <mailto:asalas@xxxxxxxxx>
>>>
>>>
>>>
>>>            On 11/07/13 11:29, Ketor D wrote:
>>>
>>>                Hi Alphe:
>>>                        Yes Callback Filesystem is very expensive and
>>>                can't open source.
>>>                It's not a good choice for ceph4win.
>>>                        Another way for ceph4win maybe develop a
>>>                kernel-mode fs like
>>>                pnfs. pnfs has a kernel-mode windows client. I think you
>>>                can read its
>>>                src code and maybe migrating from ceph kernel client to
>>>                windows kernel
>>>                fs is easier than from userspace ceph fuse client.And a
>>>                kernel-mode fs
>>>                client has greater performance than userspace fs like
>>>                ceph-fuse client
>>>                and ceph kernel client.
>>>
>>>                        Regards.
>>>
>>>            On 11/07/13 11:50, Matt W. Benjamin wrote:
>>>
>>>                Hi,
>>>
>>>                The Window NFS v4.1 client is what we work on, so this
>>>                may be good for
>>>                code sharing.  The license is lgplv2, like Ceph's.
>>>
>>>                Something important to be aware of is that the client
>>>                uses rdbss, which
>>>                is a (partial) fsd abstraction that simplified
>>>                implementation
>>>                quite a bit, kind of like a mini driver.  However,
>>>                Microsoft's support
>>>                for rdbss has been in limbo for a bit.  For example, to
>>>                link with
>>>                the rdbss symbols you can't use the Windows 8 driver
>>>                kit--you'll need
>>>                to use the one for Windows 7.  (There's a private rdbss2
>>>                used internally
>>>                by Microsoft's SMB implemenation.  A the moment, 3rd
>>>                party drivers
>>>                can't use that.)
>>>
>>>                We've been in communication with Microsoft about this
>>>                issue, and know of
>>>                a few other fsds using it, but it could be a good thing
>>>                for that
>>>                lobbying
>>>                effort to have another user--or it could be a dead end :(.
>>>
>>>                There are a couple of other choices if you're looking to
>>>                go this route,
>>>                that I'm aware of (and we may need to take them too, if
>>>                RDBSS has no
>>>                way forward), but the required work could be a lot larger.
>>>
>>>                Matt
>>>
>>>                ----- "Ketor D"<d.ketor@xxxxxxxxx
>>>                <mailto:d.ketor@xxxxxxxxx>>  wrote:
>>>
>>>                    Hi Alphe:
>>>                            Yes Callback Filesystem is very expensive
>>>                    and can't open
>>>                    source.
>>>                    It's not a good choice for ceph4win.
>>>                            Another way for ceph4win maybe develop a
>>>                    kernel-mode fs like
>>>                    pnfs. pnfs has a kernel-mode windows client. I think
>>>                    you can read its
>>>                    src code and maybe migrating from ceph kernel client
>>>                    to windows
>>>                    kernel
>>>                    fs is easier than from userspace ceph fuse
>>>                    client.And a kernel-mode
>>>                    fs
>>>                    client has greater performance than userspace fs
>>>                    like ceph-fuse
>>>                    client
>>>                    and ceph kernel client.
>>>
>>>                            Regards.
>>>
>>>
>>>
>>>                    On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas
>>>                    Michels<asalas@xxxxxxxxx <mailto:asalas@xxxxxxxxx>>
>>>
>>>                    wrote:
>>>
>>>                        Commercial libraries are a pain ...
>>>
>>>                        If we want the more permossive licence offered
>>>                        by callback file
>>>
>>>                    system we
>>>
>>>                        have to buy it for 20.000 usd. Then we will have
>>>                        to provide a
>>>
>>>                    backbox that
>>>
>>>                        we have no control upon and that will kill our
>>>                        product anytime they
>>>
>>>                    want anf
>>>
>>>                        if they decide to stop their commercial activity
>>>                        we will be in the
>>>
>>>                    same
>>>
>>>                        situation that with dokanfs but without having
>>>                        the source code of
>>>
>>>                    the black
>>>
>>>                        box. If i have to spend 20 000 dollars i would
>>>                        prefere paying
>>>
>>>                    someone to
>>>
>>>                        retake dokanfs or to write from scratch a
>>>                        dokanfs fuselike software
>>>
>>>                    make it
>>>
>>>                        all shiny and pumpy fantastic and ready to plug
>>>                        to ceph client.
>>>
>>>                        I would prefere if people have to pay something
>>>                        to get access to
>>>
>>>                    ceph4win
>>>
>>>                        that this money goes in ceph main branch
>>>                        pockets... Or as a gift you
>>>
>>>                    donante
>>>
>>>                        to ceph 10 dollars  you get 2 free registration
>>>                        codes for
>>>
>>>                    ceph4win... or
>>>
>>>                        something like that.
>>>
>>>                        If ceph4win as to be comercial then I would
>>>                        prefer delegate the task
>>>
>>>                    to a
>>>
>>>                        company like south river technologies and their
>>>                        great product
>>>
>>>                    webdrive. I
>>>
>>>                        would mininaly get involved in that project and
>>>                        simply buy the final
>>>
>>>                    product
>>>
>>>                        to sell it together with my ceph based product
>>>                        (which could be a
>>>
>>>                    calxeda
>>>
>>>                        ceph box or something like that).
>>>
>>>                        I m open anyway to any proposition. But I doubt
>>>                        that callback
>>>
>>>                    filesystem
>>>
>>>                        offers us a suitable solution in the way I see
>>>                        ceph4win to be spread
>>>
>>>                    and
>>>
>>>                        used... I m maybe wrong. And anything that will
>>>                        be done around
>>>
>>>                    ceph4win will
>>>
>>>                        be public documented etc... And licensed the way
>>>                        that if someone
>>>
>>>                    want to
>>>
>>>                        build a commercial solution on top of it, that
>>>                        would be a
>>>
>>>                    possibility.
>>>
>>>                        My idea is to giveback somehow to ceph project
>>>                        and at same time
>>>
>>>                    forge a
>>>
>>>                        better knowledge in ceph technologies. Because
>>>                        like many in libre
>>>
>>>                    world I
>>>
>>>                        think the business is in the services around the
>>>                        software more than
>>>
>>>                    on the
>>>
>>>                        software. That the ones writing code should be
>>>                        financed and benefits
>>>
>>>                    from
>>>
>>>                        the one selling and giving support of the
>>>                        software at all levels. I
>>>
>>>                    m
>>>
>>>                        probably too idealistic. And too optimistic
>>>                        after all I m the one
>>>
>>>                    saying I
>>>
>>>                        will do this stuff I have no idea how but well
>>>                        it is interesting and
>>>
>>>                    fun so
>>>
>>>                        lets do it.
>>>
>>>                        Regards,
>>>
>>>                        P.S: using commercial backend libraries appart
>>>                        including their own
>>>
>>>                    cost will
>>>
>>>                        force you to use commercial IDE like MS
>>>                        VisualStudio because their
>>>
>>>                    library
>>>
>>>                        has some kind of drm that only that IDE compiler
>>>                        can use. So alot of
>>>
>>>                    cost
>>>
>>>                        and yet there is nothing done. If I had to open
>>>                        a kickstarter
>>>
>>>                    project saying
>>>
>>>                        we need 60 000 USD to do ceph4win with that
>>>                        monney we will buy the
>>>
>>>                    right to
>>>
>>>                        use and share a commercial copyrighted library
>>>                        but abandonned
>>>
>>>                    punctually to
>>>
>>>                        us in  public domaine and that we will
>>>                        eventually produce something
>>>
>>>                    out of
>>>
>>>                        it. I doubt I will get a dollar.
>>>
>>>                        We still can suggest the idea to Edlos the
>>>                        commercial company that
>>>
>>>                    has the
>>>
>>>                        copyright of Callback FS, Or to buy them their
>>>                        product in a blender
>>>
>>>                    way
>>>
>>>                        (blender was bought with donation before being
>>>                        put opensource and
>>>
>>>                    public
>>>
>>>                        domaine), Or to open source their library. But
>>>                        in commercial minds
>>>                        opensourcing = death of their technical
>>>                        advantage and death of
>>>
>>>                    their
>>>
>>>                        marketing strategy. They will have to invent
>>>                        something more to
>>>
>>>                    retrieve
>>>
>>>                        monney from it.
>>>
>>>                        El nov 6, 2013 11:22 p.m., "Ketor D"
>>>                        <d.ketor@xxxxxxxxx <mailto:d.ketor@xxxxxxxxx>
>>>                        <mailto:d.ketor@xxxxxxxxx
>>>                        <mailto:d.ketor@xxxxxxxxx>>> escribió:
>>>
>>>
>>>                             Hi Alphe,
>>>                                       I think you could try Callback
>>>                        Filesystem dev
>>>
>>>                    framework. It
>>>
>>>                             is a commerical dev framework and is
>>>                        maintained by Edlos today.
>>>                                       I have communicated with Edlos to
>>>                        get a try code for
>>>                             development. To dokan, Callback Filesystem
>>>                        has vary document and
>>>
>>>                    maybe
>>>
>>>                             more stabilize.
>>>
>>>                                       Regards.
>>>
>>>
>>>
>>>                             On Thu, Nov 7, 2013 at 10:00 AM, Alphe
>>>                        Salas <asalas@xxxxxxxxx <mailto:asalas@xxxxxxxxx>
>>>                             <mailto:asalas@xxxxxxxxx
>>>
>>>                        <mailto:asalas@xxxxxxxxx>>> wrote:
>>>                              > Hello ketor thank you for your interest
>>>                        un ceph4win. Since
>>>
>>>                    muy
>>>
>>>                             first mail I
>>>                              > exposed the lacks of dokanfs and that I
>>>                        m far from being a
>>>                             specialist un
>>>                              > filesystems.
>>>                              > I exposed what i like un dokanfs bit I
>>>                        not a fanátic of it.
>>>
>>>                    Muy
>>>
>>>                             goal is to
>>>                              > have something working quickly.
>>>                              >
>>>                              > So I am up to any proposición sure the
>>>                        one with the more docs
>>>
>>>                    and
>>>
>>>                             support
>>>                              > will be the best choice. As for right
>>>                        now what I need is
>>>                             understand what are
>>>                              > the files involved what are the
>>>                        interfaces functions and what
>>>
>>>                    are
>>>
>>>                             the needed
>>>                              > library dependencies and if they exist
>>>                        ported to windows with
>>>                             cygwin. And
>>>                              > all that is retrieved from source code.
>>>                              >
>>>                              > Regards.
>>>                              >
>>>                              > El nov 6, 2013 10:34 p.m., "Ketor D"
>>>                        <d.ketor@xxxxxxxxx <mailto:d.ketor@xxxxxxxxx>
>>>                             <mailto:d.ketor@xxxxxxxxx
>>>
>>>                        <mailto:d.ketor@xxxxxxxxx>>> escribió:
>>>
>>>                              >
>>>                              >> Hi Alphe,
>>>                              >>      We are taking an interest in your
>>>                        work on Ceph Client
>>>
>>>                    for
>>>
>>>                             Windows
>>>                              >> with Dokan.As we know, the performance
>>>                        of Dokan is not very
>>>                             good, and it's
>>>                              >> abandoned 3 years ago.
>>>                              >>      I have learned and used
>>>                        OpenDedup(SDFS) for a long
>>>
>>>                    time.
>>>
>>>                               OpenDedup
>>>                              >> has a Dokan version. And the author of
>>>                        OpenDedup said
>>>                              >>
>>>                              >> The Dokan library is quite flakey  and
>>>                        testing should be
>>>                             performed before
>>>                              >> putting into production
>>>                              >>
>>>                              >>       So what do you think about this?
>>>                        And if there is
>>>
>>>                    another
>>>
>>>                             solution of
>>>                              >> fuse-like filesystem dev framwork on
>>>                        Windows?
>>>                              >>
>>>                              >>        Best Wish!
>>>                              >>
>>>                              >>
>>>                              >>
>>>                              >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe
>>>                        Salas Michels
>>>                             <asalas@xxxxxxxxx <mailto:asalas@xxxxxxxxx>
>>>                        <mailto:asalas@xxxxxxxxx <mailto:asalas@xxxxxxxxx>>>
>>>
>>>                              >> wrote:
>>>                              >>>
>>>                              >>> Hello I created the github repository
>>>                        for this project
>>>                              >>>https://github.com/alphe/Ceph4Win
>>>                              >>>
>>>                              >>> Regards,
>>>                              >>>
>>>                              >>> signature
>>>                              >>>
>>>                              >>> *Alphé Salas*
>>>                              >>> Ingeniero T.I
>>>                              >>>
>>>                              >>>asalas@xxxxxxxxx
>>>                        <mailto:asalas@xxxxxxxxx>
>>>                        <mailto:asalas@xxxxxxxxx <mailto:asalas@xxxxxxxxx>>
>>>
>>>
>>>                              >>> *<http://www.kepler.cl>*
>>>                              >>>
>>>                              >>> On 11/05/13 21:00, Sage Weil wrote:
>>>                              >>>>
>>>                              >>>> Hi Alphe,
>>>                              >>>>
>>>                              >>>> On Tue, 5 Nov 2013, Alphe Salas
>>>                        Michels wrote:
>>>                              >>>>>
>>>                              >>>>> signature *Hi, Sage !
>>>                              >>>>> thank you for you enthousiast reply.
>>>                              >>>>> I sure want to make the best use of
>>>                        everything or
>>>
>>>                    anything
>>>
>>>                             previously
>>>                              >>>>> done to
>>>                              >>>>> tend to
>>>                              >>>>> write ceph cliente for windows.
>>>                              >>>>>
>>>                              >>>>> Apart using libre tools for building
>>>                        the future ceph
>>>
>>>                    cliente
>>>
>>>                             I am open
>>>                              >>>>> to
>>>                              >>>>> anything.
>>>                              >>>>> I would recommand eclipse CDT or
>>>                        Code::BLocks they are
>>>
>>>                    based
>>>
>>>                             on mingwin
>>>                              >>>>> open
>>>                              >>>>> and easyly enhanceable.**
>>>                              >>>>>
>>>                              >>>>> more free tools can be found here:
>>>
>>>>>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>>>                              >>>>>
>>>                              >>>>>
>>>                              >>>>> I will read libcephfs source code
>>>                        and take some notes
>>>
>>>                    about the
>>>
>>>                              >>>>> protocol.
>>>                              >>>>
>>>                              >>>> I think you don't need to worry about
>>>                        hte protocol at all,
>>>
>>>                    since
>>>
>>>                              >>>> libcephs
>>>                              >>>> implements it for you (and will
>>>                        capture any future
>>>
>>>                    changes).
>>>
>>>                              >>>>
>>>                              >>>>> I was more going from what I know
>>>                        and trying to track down
>>>
>>>                    how
>>>
>>>                              >>>>> mount.ceph work
>>>                              >>>>> with the parameters passed to it.
>>>                              >>>>> since it point finally to
>>>                        Kernel/fs/ceph and that I don t
>>>
>>>                    really
>>>
>>>                              >>>>> understand
>>>                              >>>>> how that module work and that it
>>>                        probably points to some
>>>
>>>                    other
>>>
>>>                              >>>>> dependencies
>>>                              >>>>> Reading libcephfs source code could
>>>                        be a big gain of
>>>
>>>                    time.
>>>
>>>                              >>>>
>>>                              >>>> (I would also ignore mount.ceph as
>>>                        everything it does it
>>>                             specific to
>>>                              >>>> how Linux mounts work.)
>>>                              >>>>
>>>                              >>>>> basically on the protocol what is
>>>                        need are:
>>>                              >>>>>
>>>                              >>>>> 1) open and maintain a connection
>>>                        (socket open, auth, etc
>>>
>>>                    )
>>>
>>>                              >>>>> 2) retreive a map of directories and
>>>                        disk Quota (disk
>>>
>>>                    sizing
>>>
>>>                             Y TB free,
>>>                              >>>>> Z TB
>>>                              >>>>> total)
>>>                              >>>>> 3) procedure to send files /
>>>                        directories in a maner that
>>>
>>>                    it
>>>
>>>                             will allow
>>>                              >>>>> our
>>>                              >>>>> client to fit ceph transmission
>>>                        protocols
>>>                              >>>>> (limit bandwith for stability?,
>>>                        limit connection amount?,
>>>                             limit cpu
>>>                              >>>>> use?,
>>>                              >>>>> Cache for preparing data transfer (a
>>>                        FIFO cache)?)
>>>                              >>>>> 4)Procedure to retreive files /
>>>                        directory from ceph
>>>
>>>                    cluster
>>>
>>>                              >>>>> 5) Management copy/move files
>>>                        /Directories, FS stats,
>>>                             Connection Stats.
>>>                              >>>>> logging.
>>>                              >>>>>
>>>                              >>>>> My idea to progress is to take those
>>>                        main bulletpoint in
>>>
>>>                    ceph
>>>
>>>                             protocol
>>>                              >>>>> based
>>>                              >>>>> on general ideas of what ceph file
>>>                        system does and start
>>>                             identifying
>>>                              >>>>> parts
>>>                              >>>>> from libcephfs to match those "needs".
>>>                              >>>>
>>>                              >>>> Instead, I would look at
>>>                        include/cephfs/libcephfs.h, the
>>>                             interface that
>>>                              >>>> libcephfs provides, and try to map
>>>                        that to what the fuse
>>>
>>>                    layer
>>>
>>>                             expects.
>>>                              >>>> There is both a path-based that I
>>>                        suspsect lends itself
>>>
>>>                    well
>>>
>>>                             to the
>>>                              >>>> Windows interface and (very soon now)
>>>                        a handle based API
>>>
>>>                    that is
>>>
>>>                              >>>> targetted
>>>                              >>>> at the Unix-style VFS layers.  I'm
>>>                        mostly guessing,
>>>
>>>                    though,
>>>
>>>                             since I've
>>>                              >>>> never seen any low-level fs code in
>>>                        windows before.
>>>                              >>>>
>>>                              >>>> In this case, the analogous code for
>>>                        Linux should be
>>>                             client/fuse_ll.cc
>>>                              >>>> itself (and not much else), although
>>>                        there will probably be
>>>
>>>                    a
>>>
>>>                             few tricks
>>>                              >>>> necessary to map cleanly onto how the
>>>                        windows interfaces
>>>
>>>                    work.
>>>
>>>                              >>>>
>>>                              >>>> Does that make sense?
>>>                              >>>>
>>>                              >>>> Cheers!
>>>                              >>>> sage
>>>                              >>>>
>>>                              >>>>
>>>                              >>>>> Any suggestion and contributions are
>>>                        welcome.
>>>                              >>>>>
>>>                              >>>>>
>>>                              >>>>> *
>>>                              >>>>> On 11/05/13 11:23, Sage Weil wrote:
>>>                              >>>>>>
>>>                              >>>>>> Hi Alphe,
>>>                              >>>>>>
>>>                              >>>>>> On Mon, 4 Nov 2013, Alphe Salas
>>>                        Michels wrote:
>>>                              >>>>>>>
>>>                              >>>>>>> Good day developers!
>>>                              >>>>>>>
>>>                              >>>>>>> I would like to propose to the one
>>>                        interested  work with
>>>
>>>                    me to
>>>
>>>                              >>>>>>> develop a
>>>                              >>>>>>> ceph
>>>                              >>>>>>> cliente for MS windows world,
>>>                        Basing us on dokanFS.
>>>                              >>>>>>>
>>>                              >>>>>>> My company is a ceph enthousiast
>>>                        that use on a dayly
>>>
>>>                    basis
>>>
>>>                             ceph and
>>>                              >>>>>>> that
>>>                              >>>>>>> need
>>>                              >>>>>>> both transfer speed and big
>>>                        expendable and cheap
>>>
>>>                    storage.
>>>
>>>                              >>>>>>> My company is specialised in data
>>>                        recovery and we want
>>>
>>>                    to
>>>
>>>                             participate
>>>                              >>>>>>> to
>>>                              >>>>>>> ceph
>>>                              >>>>>>> effort by bringing a ceph cliente
>>>                        for windows.
>>>                              >>>>>>
>>>                              >>>>>> Awesome!
>>>                              >>>>>>
>>>                              >>>>>>> Our experience shows us that the
>>>                        best gateway is each
>>>                             clientes being
>>>                              >>>>>>> its
>>>                              >>>>>>> own
>>>                              >>>>>>> gateway, instead of having a
>>>                        bottle neck server or a
>>>
>>>                    cluster of
>>>
>>>                              >>>>>>> bottle
>>>                              >>>>>>> neck
>>>                              >>>>>>> servers as gateway (FTP, samba,
>>>                        SFTP,webdav, s3,
>>>
>>>                    etc..).
>>>
>>>                              >>>>>>>
>>>                              >>>>>>> We already did some research in
>>>                        that domain.
>>>                              >>>>>>>
>>>                              >>>>>>> Dokan FS is an intent  to write an
>>>                        opensource fuse like
>>>                             cliente for
>>>                              >>>>>>> MS
>>>                              >>>>>>> windows.
>>>                              >>>>>>>
>>>                              >>>>>>> More information on DOKANFS can be
>>>                        triggered here
>>>                         >>>>>>>http://dokan-dev.net/en/download/
>>>                              >>>>>>>
>>>                              >>>>>>> Positive points of using DOKANFS.
>>>                              >>>>>>>
>>>                              >>>>>>> - its opensourced and well
>>>                        licenced mit licence, gpl
>>>                             licence and lgpl
>>>                              >>>>>>> licence.
>>>                              >>>>>>>
>>>                              >>>>>>> Negative point of using DOKAN FS.
>>>                              >>>>>>> - unreachable author
>>>                              >>>>>>> - Poor documentation . Dev
>>>                        comments in japanese.
>>>                              >>>>>>> - Work in progress so it is
>>>                        unstable and needs to be
>>>
>>>                    updated,
>>>
>>>                              >>>>>>> debugged and
>>>                              >>>>>>> maintained by a MS Windows file
>>>                        system expert
>>>
>>>                    developper.
>>>
>>>                              >>>>>>
>>>                              >>>>>> I am not very familiar with windows
>>>                        storage APIs, but
>>>                             somebody told me
>>>                              >>>>>> at once point there were several
>>>                        interfaces against which
>>>
>>>                    a
>>>
>>>                             new file
>>>                              >>>>>> system could be implemented,
>>>                        everything from a full
>>>                             in-kernel driver
>>>                              >>>>>> to
>>>                              >>>>>> something that is explorer-based.
>>>                          Are any of those
>>>                             suitable?  Using a
>>>                              >>>>>> potentially abandoned fuse-like
>>>                        layer makes me a bit
>>>
>>>                    nervous.
>>>
>>>                              >>>>>>
>>>                              >>>>>> That said,
>>>                              >>>>>>
>>>                              >>>>>>>
>>>                              >>>>>>> I try past year to do a merge from
>>>                        ceph-fuse to dokanfs
>>>                              >>>>>>> here are what I learnt.
>>>                              >>>>>>> - Ceph-fuse and related source
>>>                        code is around 60 000
>>>
>>>                    lines
>>>
>>>                             of code.
>>>                              >>>>>>> - Ceph protocol isn t documented
>>>                        so it is like trying
>>>
>>>                    to
>>>
>>>                             draw a map
>>>                              >>>>>>> of
>>>                              >>>>>>> america
>>>                              >>>>>>> using only a sextan and a compass.
>>>                              >>>>>>>
>>>                              >>>>>>> Those led me to those conclusions:
>>>                              >>>>>>> - I can t do it alone.
>>>                              >>>>>>> - It is easier to draw down the
>>>                        ceph protocol way to
>>>
>>>                    work from
>>>
>>>                              >>>>>>> kernel/fs/ceph
>>>                              >>>>>>> sources and mount.ceph
>>>                              >>>>>>> - Ceph depending libraries may be
>>>                        unexistant or not up
>>>
>>>                    to
>>>
>>>                             date in
>>>                              >>>>>>> their
>>>                              >>>>>>> port
>>>                              >>>>>>> on MS Windows (cygwin)
>>>                              >>>>>>
>>>                              >>>>>> I think the most sane path should
>>>                        be to make libcephfs
>>>                             sufficiently
>>>                              >>>>>> portable to build on windows (or
>>>                        cygwin).  For the bits
>>>
>>>                    used
>>>
>>>                             by the
>>>                              >>>>>> client-side coe, I don't think
>>>                        there should be much in
>>>
>>>                    the
>>>
>>>                             way of
>>>                              >>>>>> dependencies, and the main
>>>                        challenge would be untangling
>>>
>>>                    the
>>>
>>>                             build for
>>>                              >>>>>> the necessary pieces out from the
>>>                        rest of Ceph.
>>>                              >>>>>>
>>>                              >>>>>> Have you seen the wip-port
>>>                        portability work that is
>>>                             currently underway
>>>                              >>>>>> by
>>>                              >>>>>> Noah and Alan?  That may solve many
>>>                        of the cygwin
>>>
>>>                    problems
>>>
>>>                             you are
>>>                              >>>>>> seeing
>>>                              >>>>>> today.
>>>                              >>>>>>
>>>                              >>>>>>> - MS file system specialist are
>>>                        hard do find in the
>>>
>>>                    "open
>>>
>>>                             source
>>>                              >>>>>>> libre
>>>                              >>>>>>> world"
>>>                              >>>>>>> so I will try in the commercial world.
>>>                              >>>>>>>
>>>                              >>>>>>> The commercial world has some
>>>                        problems too. They need
>>>
>>>                    ceph
>>>
>>>                             protocol
>>>                              >>>>>>> draft
>>>                              >>>>>>> to
>>>                              >>>>>>> implemente it to their own product
>>>                        They will have
>>>
>>>                    licencing
>>>
>>>                              >>>>>>> /commercial
>>>                              >>>>>>> politics that infringe lgpl, and
>>>                        hide that most of the
>>>
>>>                    work
>>>
>>>                             is done
>>>                              >>>>>>> by
>>>                              >>>>>>> people
>>>                              >>>>>>> other than them. They will not
>>>                        participate in a
>>>
>>>                    financial
>>>
>>>                             way to ceph
>>>                              >>>>>>> enhancement and growth.
>>>                              >>>>>>
>>>                              >>>>>> I don't think reimplementing the
>>>                        client code is an
>>>
>>>                    efficient way
>>>
>>>                              >>>>>> forward.
>>>                              >>>>>> Unless the goal is a pure kernel
>>>                        implementation...but a
>>>                             significant
>>>                              >>>>>> ongoing investment in development
>>>                        resources would be
>>>
>>>                    needed
>>>
>>>                             for that
>>>                              >>>>>> going
>>>                              >>>>>> forward.  I suspect that is a
>>>                        challenge for a platform
>>>
>>>                    that
>>>
>>>                             does not
>>>                              >>>>>> typically rally that sort of
>>>                        community effort.
>>>                              >>>>>>
>>>                              >>>>>> The easiest thing is of course just
>>>                        to use CIFS and
>>>
>>>                    Samba
>>>
>>>                             (which works
>>>                              >>>>>> today).  A fuse-like approach is
>>>                        probably a reasonably
>>>                             middle ground
>>>                              >>>>>> (both
>>>                              >>>>>> in initial effort and
>>>                        maintainability going forward)...
>>>                              >>>>>>
>>>                              >>>>>> sage
>>>                              >>>>>>
>>>                              >>>>>>
>>>                              >>>>>
>>>                              >>>
>>>                              >>> --
>>>                              >>> To unsubscribe from this list: send
>>>                        the line "unsubscribe
>>>                             ceph-devel" in
>>>                              >>> the body of a message
>>>                        tomajordomo@xxxxxxxxxxxxxxx
>>>                        <mailto:tomajordomo@xxxxxxxxxxxxxxx>
>>>                             <mailto:majordomo@xxxxxxxxxxxxxxx
>>>                        <mailto:majordomo@xxxxxxxxxxxxxxx>>
>>>
>>>                              >>> More majordomo info at
>>>
>>>                    http://vger.kernel.org/majordomo-info.html
>>>
>>>                              >>
>>>                              >>
>>>                              >
>>>
>>>
>>>
>>>                    --
>>>                    To unsubscribe from this list: send the line
>>>                    "unsubscribe ceph-devel"
>>>                    in
>>>                    the body of a message tomajordomo@xxxxxxxxxxxxxxx
>>>                    <mailto:tomajordomo@xxxxxxxxxxxxxxx>
>>>                    More majordomo info
>>>                    athttp://vger.kernel.org/majordomo-info.html
>>>                    <http://vger.kernel.org/majordomo-info.html>
>>>
>>>
>>>
>>>        --
>>>        To unsubscribe from this list: send the line "unsubscribe
>>>        ceph-devel" in
>>>        the body of a message to majordomo@xxxxxxxxxxxxxxx
>>>        <mailto:majordomo@xxxxxxxxxxxxxxx>
>>>        More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> Dong Yuan
> Email:yuandong1222@xxxxxxxxx



-- 
Dong Yuan
Email:yuandong1222@xxxxxxxxx
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux