On Thu, Aug 3, 2017 at 12:56 PM, Kaushal M <kshlmster@xxxxxxxxx> wrote:
On Thu, Aug 3, 2017 at 2:14 AM, Niels de Vos <ndevos@xxxxxxxxxx> wrote:
> On Wed, Aug 02, 2017 at 05:03:35PM +0530, Prashanth Pai wrote:
>> Hi all,
>>
>> The ongoing work on glusterd2 necessitates following non-breaking and
>> non-exhaustive list of changes to glusterfs source code:
>>
>> Port management
>> - Remove hard-coding of glusterd's port as 24007 in clients and elsewhere.
>> Glusterd2 can be configured to listen to clients on any port (still
>> defaults to
>> 24007 though)
>> - Let the bricks and daemons choose any available port and if needed report
>> the port used to glusterd during the "sign in" process. Prasanna has a
>> patch
>> to do this.
>> - Glusterd <--> brick (or any other local daemon) communication should
>> always happen over Unix Domain Socket. Currently glusterd and brick
>> process communicates over UDS and also port 24007. This will allow us
>> to set better authentication and rules for port 24007 as it shall only be
>> used
>> by clients.
>
> I prefer this last point to be configurable. At least for debugging we
> should be able to capture network traces and display the communication
> in Wireshark. Defaulting to UNIX Domain Sockets is fine though.
This is the communication between GD2 and bricks, of which there is
not a lot happening, and not much to capture.
But I agree, it will be nice to have this configurable.
Could glusterd start attempting port binding at 24007 and progress on to higher port numbers until successful and register the bound port number with rpcbind ? This way the setup will be auto-configurable and admins need not scratch their heads to decide upon one port number. Gluster clients could always talk to rpcbind on the nodes to get glusterd service port whenever a reconnect is required.
>
>
>> Changes to xlator options
>> - Xlator authors do not have to modify glusterd2 code to expose new xlator
>> options. IOW, glusterd2 will not contain the "glusterd_volopt_map" table.
>> Most of its fields will be moved to the xlator itself. Glusterd2 can load
>> xlator's shared object and read it's volume_options table. This also means
>> xlators have to adhere to some naming conventions for options.
>> - Add following additional fields (names are indicative) to volume_option_t:
>> - Tag: This is to enable users to list only options having a certain
>> tag.
>> IOW, it allows us to filter "volume set help" like output.
>> Example of tags: debug, perf, network etc.
>> - Opversion: The minimum (or a range) op-version required by the xlator.
>> - Configurable: A bool to indicate whether this option is
>> user-configurable.
>> This may also be clubbed with DOC/NO_DOC
>> functionality.
>
> This is something I have been thinking about to do as well. libgfapi
> users would like to list all the valid options before mounting (and
> receiving the .vol file) is done. Similar to how many mount options are
> set over FUSE, the options should be available through libgfapi.
> Hardcoding the options is just wrong, inspecting the available xlators
> (.so files) seems to make more sense. Each option would have to describe
> if it can be client-side so that we can apply some resonable filters by
> default.
>
Looks like we'd missed this. All the fields available in the vol opt
map will move to xlator option tables, including the client flag.
> A GitHub Issue with this feature request is at
> https://github.com/gluster/glusterfs/issues/263 . I appreciate additional
> comments and ideas about it :-)
>
We need to open an issue for our requested changes are well, which
will be a superset of this request. We'll make sure to mention this
feature request in it.
Or we could use a single issue as a tracker for all the xlator option
changes, in which case I'd prefer we update the existing issue.
>
>> - Xlators like AFR, changelog require non-static information such as brick
>> path
>> to be present in it's options in the volfile. Currently, xlator authors
>> have
>> to modify glusterd code to get it.
>> This can rather be indicated by the xlator itself using
>> templates/placehoders.
>> For example, "changelog-dir" can be set in xlator's option as as
>> <<brick-path>>/.glusterfs/changelogs and then glusterd2 will ensure to The exact syntax for these variables hasn't been decided yet. But I'm
>> replace
>> <<brick-path>> with actual path during volfile generation.
>
> I suggest to stick with whatever is a common syntax for other
> configuration files that uses placeholders. Maybe just {variable} or
> $VARIABLE, the <<variable>> looks a bit awkward.
leaning towards '{{ variable }}' used in the Go template package,
which is what we'll mostly end up using to implement this
functionality.
>
>
>> We'd like to hear your thoughts, suggestions and comments to these proposed
>> changes.
>
> Thanks for sharing!
> Niels
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel@xxxxxxxxxxx
> http://lists.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-devel
--
Milind
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-devel
