Re: On the command line tools ....

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 5/22/19 9:02 PM, William Brown wrote:
Hi all,

I've had this thought and email in mind for a while, so I think it's time for me to write out some thoughts. I think there is a difference in the command line tools today from what I had envisaged, to what they became.

Now of course, there are lots of reasons for this - Simon and Mark have done awesome work here, and the team picked up after my uhh ... "untimely eviction" we'll say, when I went through a lot of troubling personal issues. So you know, lots of <3 for the team being fantastic and patient with me despite my apparent insanity :)

So I think what happened is that because I didn't expect this, communication didn't happen, I didn't support everyone and didn't communicate what I had in mind. I'm basically going to take this one on myself.

What this has led to is a pretty different approach - where I took a task-oriented approach, Simon and Mark in the design of the new web-console have taken an attribute-setting approach.
This is not entirely true, and I will give examples below.  Yes the approach I took was that you can have fine grained control over all settings.  This was voted on by the entire 389 development team btw.  I still strongly believe we should not hide any configuration in the CLI tools.  The tools should be able to do everything.  That being said we did try to do bundle operations into a task approach which I will show under your example of how the tools currently works...
What does this mean?

Let's take replication as an example: I would have created the tools to look more like:

# Setup a first master
dsconf instance_a replication initialise-topology

# Join a second master, doing a re-init as needed and setting up replication managers etc.
dsconf instance_b replication join-as-master --master instance_a

# Join a third master ...
dsconf instance_c replication join-as-master --master instance_a

# Make sure the agreements between a-b-c are full mesh
dsconf instance_a relication ensure-agreement instance_b
dsconf instance_a relication ensure-agreement instance_c
...


This could have actually been a different tool like dsrepl or something which may even be more expressive, to coordinate replication as a topology rather than per-server focussed.

An attribute-setting approach though is more like (this is made up in my head, not reflective of current actual command, just for the point of ideological discussion):

dsconf instance_a changelog init
dsconf instance_a replication_manager create
dsconf instance_a replica init --rid=10
dsconf instance_b changelog init
dsconf instance_b replication_manager create
dsconf instance_a replica init --rid=11
dsconf instance_a replication_agreement create --to=instance_b

This looks a bit tedious I agree, but it's not how it works. Here are the actual steps to setup MMR replication between two servers using dsconf as it exists today.  It only takes 4 commands to get it running...


Enable replication:

# dsconf instanceA replication enable --role master --rid 1 --bind-dn "cn=replication manager,cn=config" --bind-pw PASSWORD --suffix "dc=example,dc=com"

->  This command above enables replication, creates the changelog, creates the replication manager, and adds the manager to the replication configuration.  Nice!  One command does all these steps :-)  The below command sets up the second master...

# dsconf instanceB replication enable --role master --rid 2 --bind-dn "cn=replication manager,cn=config" --bind-pw PASSWORD --suffix "dc=example,dc=com"


Create the agreements:

# dsconf instanceA repl-agmt create MY_REPL_AGMT_NAME  --host ldapB.example.com --port 636 --conn-protocol LDAPS --bind-method SIMPLE --bind-dn "cn=replication manager manager,cn=config" --bind-pw PASSWORD --init

This creates the agreement and initializes it.  Then create the agreement from B to A which does not need to be inited

# dsconf instanceB repl-agmt create MY_REPL_AGMT_NAME  --host ldapA.example.com --port 636 --conn-protocol LDAPS --bind-method SIMPLE --bind-dn "cn=replication manager manager,cn=config" --bind-pw PASSWORD


Is this really hard, confusing, or inferior?   I don't know how it could be made easier - an Admin needs to choose these settings.  I think what you don't like is that you have to know and use these extra arguments for the connection protocol and bind method?   I think you wanted to hide all of that, but these are really decisions that can not be assumed.


The point here - I have always wanted to make the tools abstract from the gory horrid details that exist in cn=config. Setting attributes by hand wasn't what I "envisaged". The goal of these tools was such that you never had to know about these attributes or objects at all.

I think you are making assumptions again about how the current CLI works.  It's not strictly attribute based settings as you keep insisting (as we saw from my replication example)  I'm sorry but I think you are biased here and you have not really looked at the tools (I'm not biased of course haha).  But let me give you some more examples:

Create backend/suffix:

# dsconf instanceA backend create --suffix dc=my,dc=suffix --be-name userRoot


Configure a backend:

# dsconf instanceA backend suffix set --add-referral REFERRAL --cache-memsize 100000000


Add encrypted attribute:

# dsconf instanceA backend attr-encrypt --add-attr creditCardAttr


Run cleanAllRUV task

# dsconf instanceA repl-tasks cleanallruv --replica-id 1 --suffix dc=my,dc=suffix


Run the monitor

# dsconf instanceA monitor [backend, server, chaining, SNMP, ldbm]


These are pretty basic and I'm not sure how they could be improved, but then there are configurations that require more fine grained control like password policy.  So here we do  require attribute settings, but they use more friendly names:

# dsconf instanceA pwpolicy set --pwdhistorycount 5 --pwdchecksyntax on  --pwdminlowers 3


The issue with what you envisioned is that it is impossible to assume how everything should be configured.   You still need the user to make precise decisions about settings.


Now, people can always go an edit cn=config, I'll never stop that. If someone wants to use ldapmodify to admin their server, absolutely go ahead. But if I use a cli tool like this, I want it to abstract toward tasks that help me get my job done. I don't want to be an expert in remembering every single attribute on cn=replica, I want to just have two servers replicate. (sorry to bash-on replication, it's just a good example of a task that is reasonable complex to configure).

For me this is hard emotionally, because I feel like lib389, and the ds* tools are really creations of my love and passion,
I think some of us who worked countless hours on the initial version of lib389, before you joined the team, feel exactly the same way.
  to improve things in the ways I always wanted 389 to be as the product I want to use, and enjoy using. To make tooling that encapsulates all our amazing knowledge and experience inside of highly accessible commands. Which then leads me to be sad about how things turned out because it's not the way "I wanted" (again, I feel like this failure is my responsibility due to what happened last year, not the fault of anyone else - everyone else has done their absolute best).


I would propose to change this - but I feel we are now wedged on change by two things:

* Red Hat have documented these tools, and it's a big ask to get them to re-document everything.
* The cockpit console uses these tools in a special --json mode ( I would have created a dscockpit command that just handled the --json mode, rather than trying to make a tool do two jobs well.). This means rewriting cockpit integration ....
There was a brief talk about this actually while you were gone.  But Admins could use the json output for these same tasks so we left it in dsconf.  I would not be opposed to doing this though.  It would not be a major change for the UI, but it would be a lot of work to write the tool (alot of redundancy with dsconf).  We also talked about doing bulk data loading, so we could do something like call "ds-cockpit load replication", or load-everything, instead of calling "dsconf get" 20 times
* Time - heaps of time has already gone into these tools now, can we really justify undoing all that effort for this?


Which is a bit why I'm writing this
- Should I stop my task-oriented-vision and complaining that I have been doing, and accept what we have today?
- Do we want to try and pivot to change based on the ideas I have?
- Do I build something in parallel to satisfy my selfish desires, and to supplement what we have?

I really think you should write a new CLI tool that only does your recipe-style tasks for certain things.   Like I said that approach will not cover everything, but it could be used for other tasks.   Let the existing dsconf handle the fine grained configuration (although it's not as bad as you make it sound), then have a tool like "dstask", "dsrecipe" or something, that handles larger replication deployments, or whatever else you wanted to generalize.

To be honest I would like to hear from you which exact configurations you want to change.  Give some good detailed examples.  You keep saying how we should use a different style, but I think when you look closely at these areas of configuration you will find it's not possible to generalize them and to hide the horrid attributes underneath.  IMHO I feel most of the CLI commands are already doing what you kind of want except for a few of them which require fine grained control.  We can also add new arguments to the existing commands, but it's a bit late to rewrite/replace all of the usage.

So I am all for extending/improving the current CLI (it's certainly not perfect), or even you adding another tool for doing recipe tasks, but rewriting all of it doesn't sit well with me.

These are just my (biased) opinions, it would be nice if others from the 389 team (or anyone) would speak up and voice their opinions and perspectives...

Mark


I don't know. But I think it's important to have this discussion as a team, including our emotional investments, concerns, visions and desires so that we can choose whats best. It's not good if we are "in-fighting" over the things we are doing, that helps no one - and neither does people feeling like their work isn't appreciated or respected (and I'm worried I have come off as disrespectful, which I am sorry for). We should be unified in what we want to do and achieve.

So I'd appreciate advice here. What should I/we do? What do we want 389 to look like in the future from a user-experience perspective?

Thanks everyone,

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs
_______________________________________________
389-devel mailing list -- 389-devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-devel@xxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
389-devel mailing list -- 389-devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-devel@xxxxxxxxxxxxxxxxxxxxxxx




[Index of Archives]     [Fedora Directory Announce]     [Fedora Users]     [Older Fedora Users Mail]     [Fedora Advisory Board]     [Fedora Security]     [Fedora Devel Java]     [Fedora Desktop]     [ATA RAID]     [Fedora Marketing]     [Fedora Mentors]     [Fedora Package Review]     [Fedora Art]     [Fedora Music]     [Fedora Packaging]     [CentOS]     [Fedora SELinux]     [Big List of Linux Books]     [KDE Users]     [Fedora Art]     [Fedora Docs]

  Powered by Linux