* I've also posted this to my blog at https://kshlm.in/post/volgen-2.0/ * # Designing and prototyping volgen for GD2 I've recently begun working on volgen for GD2. This post gives an overview of what I've been doing till now. ## Some background first One of the requirements for GD2 is to make it easier to modify the volume graphs, in particular make it easier to add new translators into the volume graph. GlusterD right now has hardcoded graph topologies in its code. This makes it very hard to begin modify graphs. This made it hard for features like tiering to do their stuff. For GD2, we are designing a more flexible volgen package, that makes it much easier to modify, extend and combine graphs. We've begun working on a prototype to explore how this design would work in real life. Different approaches to volgen was discussed earlier this year in a meeting during DevConf. The approaches discussed are summarized in the ["Flexible volgen" wiki][1]. Of the discussed approaches, the "SystemD units" style approach was picked as the most suitable. I've begun a prototype implementation of this approach at [kshlm/glusterd2-volgen][2]. I welcome comments and queries on the approach as well as the prototype. ## The SystemD style volgen > NOTE: Most of the names, paths etc. used below are just placeholders. They will all be changed to more decsriptive terms before we reach the end. More information on this can be found on the wiki, but the wiki might now be always up to date. So I'm providing a quick summary of the approach here. This approach to volgen makes use of what I'm currently calling Nodefiles. These are analogus to the systemd unit files. A Nodefile describes a node which can appear in a volume graph. There are two types of nodes/nodefiles, Xlators and Targets ### Xlators Xlators nodefiles describe GlusterFS translators. Nodefiles are text files, which have to follow a defined format, that describe the particular translator. Examples of Nodefiles can be found at [kshlm/glusterd2-volgen/examples/xlator][3 ]. _Note: These examples are just that; examples. They are not the final form._ In addition to describing a translator, Nodefiles specify the dependencies of the translator on other translator. Some of the dependecies currently available are, - Requires, this translator requires the specified translator also to be enabled - Conflicts, this translator cannot work with the specified translator - Before, this translator must appear before the other translator in the volume graph - After, this translator must appear after the specified translator in the volume graph - Parent, this translator can only have the specified translator as its parent - Child, this translator can only have the specified translator as its child(ren). In the prototype, [TOML][4] is used as the description language. The Nodefile is described in code as a the `Node` struct in [volgen/nodefile.go][5]. _Note: This is incomplete right now._ Each translator will provide its own nodefile, all of which will need to be added to a directory known to GD2 (let's call it `XLATORDIR` for now), for eg. `/usr/lib/glusterd/xlators`. To make it easier for authors to create these files, we may provide tools to scrape information from the C code and build the file. But that's a task for later. GD2 will read the Xlator nodefiles from `XLATORDIR` at startup and build an internal translator map, and a transaltor options table. ### Targets Targets are nodes which describe graphs. You can have Targets for FUSE graph, brick graph, self-heal daemon etc. On disc, Targets are directories which have Xlators linked into it. The linked in translators are the translators which will be present in the graph. The Targets decribe themselves with an `info` nodefile in the directory. Specific depe ndencies for the target can also be given in the nodefile. An example target for the brick graph can be found at [kshlm/glusterd2-volgen/examples/brick.target][6]. A `Target` struct is described in [volgen/target.go][7]. Targets can include other targets among themselves. This will make it easier to describe complex graphs. For eg. a `fuse.target` for the FUSE graph can include a `performance.target `and `clients.target` among other translators. The `pe rformance.target` includes all known performance translators, and would provide an easy way to enable/disable performance translators. The `clients.target` would just include the cluster translators, ie the part of the client graph belo w and including DHT. This will make it easier to generate other graphs, like NFS for example, which would now need to only need to include the `clients.target` instead of the whole client graph. Targets are the starting points of building an actual volume graph. Once a target is loaded into GD2, building a graph will be as simple as calling `Target.BuildGraph(volinfo)`. The `BuildGraph` function will first do a dependency resolution of all the translators/targets included in the target and build a intermediate graph. `BuildGraph` is called on any targets in this graph, and the returned graph is inserte d at the targets location. Build graph will next fill in this intermediate graph with information from the volinfo, disable xlators if required and generate the actual volume graph. Calling `graph.Write()` will write out a volfile descr ibing this graph. The protoype contains a [simple app][8] which shows how to make use of this volgen packaged. ### Things left to be done - I still need to design and implement the actual dependency resolver. I expect to take a significant amount of time for this. I'll keep everyone updated on my progress on this. - I still have only a vague picture of how to handle assigning identities and options for client xlators, ie. how to get volgen to assign a client xlator to a brick. - There is a question of how a graph with complex interconnections, such as the one for server side replication, can be described with this. This is something I expect to have an answer as I proceed further with the prototype. ## What Next I'll be continuing my work on the prototype and continue to provide regular updates. At the sametime, I request the community to provide their comments on, and ask questions about this attempt. I'm particularly interested in hearing fro m developers working on features that will do or be doing a lot of volume graph modifications. I hope to make Volgen-2.0 suitable for everyone, but for this I will need to hear from all of you. Thanks, Kaushal [1]: https://github.com/gluster/glusterd2/wiki/Flexible-volgen [2]: https://github.com/kshlm/glusterd2-volgen [3]: https://github.com/kshlm/glusterd2-volgen/tree/volgen-systemd-style/examples/xlators [4]: https://github.com/toml-lang/toml/blob/master/versions/en/toml-v0.4.0.md [5]: https://github.com/kshlm/glusterd2-volgen/blob/volgen-systemd-style/volgen/nodefile.go [6]: https://github.com/kshlm/glusterd2-volgen/tree/volgen-systemd-style/examples/brick.target [7]: https://github.com/kshlm/glusterd2-volgen/blob/volgen-systemd-style/volgen/target.go [8]: https://github.com/kshlm/glusterd2-volgen/blob/volgen-systemd-style/main.go _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
