Great summary Jeff - thanks!
On Wed, May 4, 2016 at 3:50 AM, Jeff Darcy <jdarcy@xxxxxxxxxx> wrote:
One of my recurring action items at community meetings is to report to
the list on how 4.0 is going. So, here we go.
The executive summary is that 4.0 is on life support. Many features
were proposed - some quite ambitious. Many of those *never* had anyone
available to work on them. Of those that did, many have either been
pulled forward into 3.8 (which is great) or lost what resources they had
(which is bad). Downstream priorities have been the biggest cause of
those resource losses, though other factors such as attrition have also
played a part. Net result is that, with the singular exception of
GlusterD 2.0, progress on 4.0 has all but stopped. I'll provide more
details below. Meanwhile, I'd like to issue a bit of a call to action
here, in two parts.
* Many of the 4.0 sub-projects are still unstaffed. Some of them are
in areas of code where our combined expertise is thin. For example,
"glusterfsd" is where we need to make many brick- and
daemon-management changes for 4.0, but it has no specific maintainer
other than the project architects so nobody touches it. Over the
past year it has been touched by fewer than two patches per month,
mostly side effects of patches which were primarily focused elsewhere
(less than 400 lines changed). It can be challenging to dive into
such a "fallow" area, but it can also be an opportunity to make a big
difference, show off one's skill, and not have to worry much about
conflicts with other developers' changes. Taking on projects like
these is how people get from contributing to leading (FWIW it's how I
did), so I encourage people to make the leap.
* I've been told that some people have asked how 4.0 is going to affect
existing components for which they are responsible. Please note that
only two components are being replaced - GlusterD and DHT. The DHT2
changes are going to affect storage/posix a lot, so that *might* be
considered a third replacement. JBR (formerly NSR) is *not* going to
replace AFR or EC any time soon. In fact, I'm making significant
efforts to create common infrastructure that will also support
running AFR/EC on the server side, with many potential benefits to
them and their developers. However, just about every other component
is going to be affected to some degree, if only to use the 4.0
CLI/volgen plugin interfaces instead of being hard-coded into their
current equivalents. 4.0 tests are also expected to be based on
Distaf rather than TAP (the .t infrastructure) so there's a lot of
catch-up to be done there. In other cases there are deeper issues to
be resolved, and many of those discussions - e.g. regarding quota or
georep - have already been ongoing. There will eventually be a
Gluster 4.0, even if it happens after I'm retired and looks nothing
like what I describe below. If you're responsible for any part of
GlusterFS, you're also responsible for understanding how 4.0 will
affect that part.
With all that said, I'm going to give item-by-item details of where we
stand. I'll use
http://www.gluster.org/community/documentation/index.php/Planning40
as a starting point, even though (as you'll see) in some ways it's out
of date.
* GlusterD 2 is still making good progress, under Atin's and Kaushal's
leadership. There are designs for most of the important pieces, and
a significant amount of code which we should be able to demo soon.
* DHT2 had been making good progress for a while, but has been stalled
recently as its lead developer (Shyam) has been unavailable.
Hopefully we'll get him back soon, and progress will accelerate
again.
* Sharding got pulled forward because of its importance for other
efforts, so it's no longer a 4.0 feature.
* Client-side caching has been dropped for now, though it could still
return with a new design based on the lease infrastructure.
* Data classification (beyond just tiering) has been dropped.
* Multiple-network support and network QoS are very much still part of
the 4.0 plan as far as I'm concerned, but there's still nobody
available to work on them.
* "Better brick management" is also still an un-resourced part of the
4.0 plan. A lot of the higher-level logic will go into Heketi, but
exporting multiple bricks through a single daemon (and port) can't
be.
* Compression/dedup have been dropped.
* Composite operations are already being implemented, either as part of
3.8 or as part of the Samba/Ganesha efforts depending on how you look
at it, so that's not a 4.0 feature any more.
* Stat/xattr caching (on the server) is a bit of a question mark. On
the one hand, it should be pretty simple to implement. On the other
hand, nobody has made even a minimal effort to do so. Recent events
have also raised the issue of needing to do this for correctness
(especially around maintaining ctime across replicas) as well as
performance. This would be a *great* opportunity for a currently
junior/novice Gluster contributor to make their mark.
* Code generation already exists, and is actively being used to
implement other 4.0 features. My only other comment here is that
people should start using it instead of continuing to use macros in
many cases. Every macro we add is another little nugget of technical
debt, causing all sorts of headaches for anyone who has to edit or
debug the code later. Please do your part to stamp out macro abuse.
* Management plugins are part of the GlusterD 2 plan.
* Performance monitoring etc. (last item on list) has been dropped, for
lack of a well defined scope or requirements.
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
