Hi Richard, Thanks a lot for you feedback. I've done my replies inline. On Sat, Sep 5, 2015 at 5:46 AM, Richard Wareing <rwareing@xxxxxx> wrote: > Hey Atin (and the wider community), > > This looks interesting, though I have a couple questions: > > 1. Language choice - Why the divergence from Python (which I'm no fan of) which is already heavily used in GlusterFS? It seems a bit strange to me to introduce yet another language into the GlusterFS code base. Doing this will make things less cohesive, harder to test, make it more difficult for contributors to understand the code base and improve their coding skills to be effective contributors. I'm a bit concerned we are setting a precedent that development will switch to the new flavor of the day. If a decision has been made to shift away from Python for the portions of GlusterFS where performance isn't a concern, will the portions currently written in Python be re-written as well? I also question the wisdom of a language will a shallow developer pool, and a less development process (somewhat of an ironic choice IMHO). > One of our aims for GlusterD-2.0 was to switch to a higher level language. While C is good for solving lower level performance critical problems, it isn't best suited for the kind of management tasks we want GlusterD to focus on. The choice of Go over Python as the higher level language, was mainly driven by the following - Go is easier to a hang of for a developer coming from a C background. IMO for a developer new to both Go and Python, it's easier to start producing working code in Go. The structure and syntax of the language and the related tools make it easier. - Go has built into the language support (think goroutines, channels) for easily implementing the concurrency patterns that we have in the current GlusterD codebase. This makes it easier for us think about newer designs based on our understanding of the existing implementation. - We have concerns about the concurrency and threading capabilities of Python. We have faced a lot of problems with doing concurrency and threading in GlusterD (though this is mostly down to bad design). Python has known issues with threading, which doesn't give us confidence as python novices. - Go has a pretty good standard library (possibly the best standard library), which provides us with almost everything required. This reduces the number of dependencies that we pull in. That's not to say Go doesn't have it's drawbacks. The major drawback that I see currently with Go is that it doesn't have a way to do plugins (dynamically load and execute binaries). But there have been recent developments in this area. Go 1.5 landed support for building Go packages as dynamic libraries. There is design ready to support runtime loadability (plugins) ready, which is targeted for inclusion in Go 1.6. As Go follows a 6 month release cycle, 1.6 is scheduled for a Feb 2016 release, we need not wait too long for this to land. In the meantime, we plan to build up the rest of the infrastructure required. > 2. RPC framework - What's the reasoning behind using Protocol Buffers vs Thrift? I'm admittedly biased here (since it's heavily used here at FB), however Thrift supports far more languages, has a larger user-base, features better data structure support, has exceptions and has a more open development process (it's an Apache project). It's mentioned folks are "uncomfortable" with GLib, exactly why? Has anyone done any latency benchmarks on the serialization/de-serialization to ensure we don't shoot ourselves in the foot by moving away from XDR for brick<->client/gNFSd communication? The low latency communication between bricks & clients is to me a _critical_ component to GlusterFS's success; adding weight to the protocol or (worse) making it easier to add weight to me is unwise. > I'd like to clarify that we are not moving away from sunrpc/xdr for brick - client communication. This will remain as is. The new RPC framework is only for communication with GlusterD. We started looking for an alternative rpc mechanism because Go doesn't have an existing sunrpc implementation. Go does have an XDR implementation, but there isn't a stable code generator to generate Go code from .x files. We are looking for certain features in the RPC framework, - Language support for C and Go - Code generation - we didn't want to write message data structures for each language ourselves - Support for SSL transports - thought we wouldn't be using this initially, we would require it later on. - Support for non-blocking I/O - to better integrate with current glusterfsd code, which mainly uses non-blocking I/O for network operations. At a high level thrift ticks all the boxes. But we initially were weary of using thrift because of it's usage of Glib in the C implementation. Glib brings in its own programming patterns and idioms of which we were a little uncomfortable. But we did revisit Thrift later (after my original mail), once we felt more comfortable. We found that the C implementation of thrift is lacking in many features, the major one among them is that it doesn't have SSL transports. Also, the C implementation just has a simple blocking server. It doesn't have either the multi-threaded server or the non-blocking server which are present in the implementations of other languages. As we couldn't find any suitable existing RPC frameworks, we went forward with implementing a simple, easy to use framework ourselves. We chose protobuf over thrift, as the protobuf-c implementation brings in less dependencies and works just as well as thrift for serialization. For the transport, we've tested using libevent for C, which provides non-blocking I/O and support for encryption. We are in the process of completing a usable library with this. We've also started investigating if it's worth using nanomsg for our transports. But, we are still ready to use thrift, if we can get help implementing the missing features. > So far things are moving towards 3-4 languages (Python, C, Go, sprinkle of BASH) and 2 RPC frameworks. No language or RPC mechanism is perfect, but the proficiency of the coder at the keyboard is _far_ more important. IMHO we should focus on 1 low level high-performance language (C) and 1 higher level language for other components where high performance isn't required (geo-rep, glusterd etc), as it will encourage higher proficiency in the chosen languages and less fractured knowledge amongst developers. > I agree with this. But I believe, with what I've said above, that we have valid reasons to justify our decisions. But, even then I'd like to apologize for not having had these discussions and decisions made transparently. We did say that we'd be doing GlusterD-2.0 in the open during the Gluster summit, but we haven't been following our own words. This has been a mistake on our part, and we will correct it. > My 2 cents. > > Richard > Cheers, Kaushal > > ________________________________________ > From: gluster-devel-bounces@xxxxxxxxxxx [gluster-devel-bounces@xxxxxxxxxxx] on behalf of Atin Mukherjee [amukherj@xxxxxxxxxx] > Sent: Monday, August 31, 2015 10:04 PM > To: Gluster Devel > Subject: GlusterD 2.0 status updates > > Here is a quick summary of what we accomplished over last one month: > > 1. The skeleton of GlusterD 2.0 codebase is now available @ [1] and the > same is integrated with gerrithub. > > 2. Rest end points for basic commands like volume > create/start/stop/delete/info/list have been implemented. Needs little > bit of more polishing to strictly follow the heketi APIs > > 3. Team has worked on coming up with a cross language light weight RPC > framework using pbrpc and the same can be found at [2]. The same also > has pbcodec package which provides a protobuf based rpc.ClientCodec and > rpc.ServerCodec that can be used with rpc package in Go's standard library > > 4. We also worked on the first cut of volfile generation and its > integrated in the repository. > > > The plan for next month is as follows: > > 1. Focus on the documentation along with publishing the design document > 2. Unit tests > 3. Come up with the initial design & a basic prototype for transaction > framework. > > [1] https://github.com/kshlm/glusterd2 > [2] https://github.com/kshlm/pbrpc > > Thanks, > Atin > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@xxxxxxxxxxx > https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=eSXH2j44tGdWnHEbaIk1Tg%3D%3D%0A&m=%2Fqy%2B94CzpNhZn9QOWkfc%2FZkbPJPDiR9uYJNVtG%2BgZPA%3D%0A&s=68e118c111403736815ea0ddf1c756a6c66800a66cbc5e1d14e0586c24ceb695 > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@xxxxxxxxxxx > http://www.gluster.org/mailman/listinfo/gluster-devel _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
