I've created an issue [1] to request the changes to the xlators. I've also posted a patch for review [2] which adds the new fields to the xlator options. The patch is on the experimental branch for now, but I could just as well post it on master. It doesn't affect any operations of GD1 or xlators yet. ~kaushal [1]: https://github.com/gluster/glusterfs/issues/302 [2]: https://review.gluster.org/18050 On Thu, Aug 17, 2017 at 12:46 PM, Kaushal M <kshlmster@xxxxxxxxx> wrote: > On Thu, Aug 3, 2017 at 2:12 PM, Milind Changire <mchangir@xxxxxxxxxx> wrote: >> >> >> On Thu, Aug 3, 2017 at 12:56 PM, Kaushal M <kshlmster@xxxxxxxxx> wrote: >>> >>> On Thu, Aug 3, 2017 at 2:14 AM, Niels de Vos <ndevos@xxxxxxxxxx> wrote: >>> > On Wed, Aug 02, 2017 at 05:03:35PM +0530, Prashanth Pai wrote: >>> >> Hi all, >>> >> >>> >> The ongoing work on glusterd2 necessitates following non-breaking and >>> >> non-exhaustive list of changes to glusterfs source code: >>> >> >>> >> Port management >>> >> - Remove hard-coding of glusterd's port as 24007 in clients and >>> >> elsewhere. >>> >> Glusterd2 can be configured to listen to clients on any port (still >>> >> defaults to >>> >> 24007 though) >>> >> - Let the bricks and daemons choose any available port and if needed >>> >> report >>> >> the port used to glusterd during the "sign in" process. Prasanna has >>> >> a >>> >> patch >>> >> to do this. >>> >> - Glusterd <--> brick (or any other local daemon) communication should >>> >> always happen over Unix Domain Socket. Currently glusterd and brick >>> >> process communicates over UDS and also port 24007. This will allow us >>> >> to set better authentication and rules for port 24007 as it shall >>> >> only be >>> >> used >>> >> by clients. >>> > >>> > I prefer this last point to be configurable. At least for debugging we >>> > should be able to capture network traces and display the communication >>> > in Wireshark. Defaulting to UNIX Domain Sockets is fine though. >>> >>> This is the communication between GD2 and bricks, of which there is >>> not a lot happening, and not much to capture. >>> But I agree, it will be nice to have this configurable. >>> >> >> Could glusterd start attempting port binding at 24007 and progress on to >> higher port numbers until successful and register the bound port number with >> rpcbind ? This way the setup will be auto-configurable and admins need not >> scratch their heads to decide upon one port number. Gluster clients could >> always talk to rpcbind on the nodes to get glusterd service port whenever a >> reconnect is required. > > 24007 has always been used as the GlusterD port. There was a plan to > have it registered with IANA as well. > Having a well defined port is useful to allow proper firewall rules to be setup. > >> >>> >>> > >>> > >>> >> Changes to xlator options >>> >> - Xlator authors do not have to modify glusterd2 code to expose new >>> >> xlator >>> >> options. IOW, glusterd2 will not contain the "glusterd_volopt_map" >>> >> table. >>> >> Most of its fields will be moved to the xlator itself. Glusterd2 can >>> >> load >>> >> xlator's shared object and read it's volume_options table. This also >>> >> means >>> >> xlators have to adhere to some naming conventions for options. >>> >> - Add following additional fields (names are indicative) to >>> >> volume_option_t: >>> >> - Tag: This is to enable users to list only options having a >>> >> certain >>> >> tag. >>> >> IOW, it allows us to filter "volume set help" like output. >>> >> Example of tags: debug, perf, network etc. >>> >> - Opversion: The minimum (or a range) op-version required by the >>> >> xlator. >>> >> - Configurable: A bool to indicate whether this option is >>> >> user-configurable. >>> >> This may also be clubbed with DOC/NO_DOC >>> >> functionality. >>> > >>> > This is something I have been thinking about to do as well. libgfapi >>> > users would like to list all the valid options before mounting (and >>> > receiving the .vol file) is done. Similar to how many mount options are >>> > set over FUSE, the options should be available through libgfapi. >>> > Hardcoding the options is just wrong, inspecting the available xlators >>> > (.so files) seems to make more sense. Each option would have to describe >>> > if it can be client-side so that we can apply some resonable filters by >>> > default. >>> > >>> >>> Looks like we'd missed this. All the fields available in the vol opt >>> map will move to xlator option tables, including the client flag. >>> >>> > A GitHub Issue with this feature request is at >>> > https://github.com/gluster/glusterfs/issues/263. I appreciate additional >>> > comments and ideas about it :-) >>> > >>> >>> We need to open an issue for our requested changes are well, which >>> will be a superset of this request. We'll make sure to mention this >>> feature request in it. >>> Or we could use a single issue as a tracker for all the xlator option >>> changes, in which case I'd prefer we update the existing issue. >>> >>> > >>> >> - Xlators like AFR, changelog require non-static information such as >>> >> brick >>> >> path >>> >> to be present in it's options in the volfile. Currently, xlator >>> >> authors >>> >> have >>> >> to modify glusterd code to get it. >>> >> This can rather be indicated by the xlator itself using >>> >> templates/placehoders. >>> >> For example, "changelog-dir" can be set in xlator's option as as >>> >> <<brick-path>>/.glusterfs/changelogs and then glusterd2 will ensure >>> >> to >>> >> replace >>> >> <<brick-path>> with actual path during volfile generation. >>> > >>> > I suggest to stick with whatever is a common syntax for other >>> > configuration files that uses placeholders. Maybe just {variable} or >>> > $VARIABLE, the <<variable>> looks a bit awkward. >>> >>> The exact syntax for these variables hasn't been decided yet. But I'm >>> leaning towards '{{ variable }}' used in the Go template package, >>> which is what we'll mostly end up using to implement this >>> functionality. >>> >>> > >>> > >>> >> We'd like to hear your thoughts, suggestions and comments to these >>> >> proposed >>> >> changes. >>> > >>> > Thanks for sharing! >>> > Niels >>> > _______________________________________________ >>> > Gluster-devel mailing list >>> > Gluster-devel@xxxxxxxxxxx >>> > http://lists.gluster.org/mailman/listinfo/gluster-devel >>> _______________________________________________ >>> Gluster-devel mailing list >>> Gluster-devel@xxxxxxxxxxx >>> http://lists.gluster.org/mailman/listinfo/gluster-devel >> >> >> >> >> -- >> Milind >> _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-devel
