On 5/22/19 9:02 PM, William Brown wrote:
Hi all,
I've had this thought and email in mind for a while, so I think it's time for me to write out some thoughts. I think there is a difference in the command line tools today from what I had envisaged, to what they became.
Now of course, there are lots of reasons for this - Simon and Mark have done awesome work here, and the team picked up after my uhh ... "untimely eviction" we'll say, when I went through a lot of troubling personal issues. So you know, lots of <3 for the team being fantastic and patient with me despite my apparent insanity :)
So I think what happened is that because I didn't expect this, communication didn't happen, I didn't support everyone and didn't communicate what I had in mind. I'm basically going to take this one on myself.
What this has led to is a pretty different approach - where I took a task-oriented approach, Simon and Mark in the design of the new web-console have taken an attribute-setting approach.
This is not entirely true, and I will give examples below. Yes the
approach I took was that you can have fine grained control over all
settings. This was voted on by the entire 389 development team btw. I
still strongly believe we should not hide any configuration in the CLI
tools. The tools should be able to do everything. That being said we
did try to do bundle operations into a task approach which I will show
under your example of how the tools currently works...
What does this mean?
Let's take replication as an example: I would have created the tools to look more like:
# Setup a first master
dsconf instance_a replication initialise-topology
# Join a second master, doing a re-init as needed and setting up replication managers etc.
dsconf instance_b replication join-as-master --master instance_a
# Join a third master ...
dsconf instance_c replication join-as-master --master instance_a
# Make sure the agreements between a-b-c are full mesh
dsconf instance_a relication ensure-agreement instance_b
dsconf instance_a relication ensure-agreement instance_c
...
This could have actually been a different tool like dsrepl or something which may even be more expressive, to coordinate replication as a topology rather than per-server focussed.
An attribute-setting approach though is more like (this is made up in my head, not reflective of current actual command, just for the point of ideological discussion):
dsconf instance_a changelog init
dsconf instance_a replication_manager create
dsconf instance_a replica init --rid=10
dsconf instance_b changelog init
dsconf instance_b replication_manager create
dsconf instance_a replica init --rid=11
dsconf instance_a replication_agreement create --to=instance_b
This looks a bit tedious I agree, but it's not how it works. Here are
the actual steps to setup MMR replication between two servers using
dsconf as it exists today. It only takes 4 commands to get it running...
Enable replication:
# dsconf instanceA replication enable --role master --rid 1 --bind-dn
"cn=replication manager,cn=config" --bind-pw PASSWORD --suffix
"dc=example,dc=com"
-> This command above enables replication, creates the changelog,
creates the replication manager, and adds the manager to the replication
configuration. Nice! One command does all these steps :-) The below
command sets up the second master...
# dsconf instanceB replication enable --role master --rid 2 --bind-dn
"cn=replication manager,cn=config" --bind-pw PASSWORD --suffix
"dc=example,dc=com"
Create the agreements:
# dsconf instanceA repl-agmt create MY_REPL_AGMT_NAME --host
ldapB.example.com --port 636 --conn-protocol LDAPS --bind-method SIMPLE
--bind-dn "cn=replication manager manager,cn=config" --bind-pw PASSWORD
--init
This creates the agreement and initializes it. Then create the
agreement from B to A which does not need to be inited
# dsconf instanceB repl-agmt create MY_REPL_AGMT_NAME --host
ldapA.example.com --port 636 --conn-protocol LDAPS --bind-method SIMPLE
--bind-dn "cn=replication manager manager,cn=config" --bind-pw PASSWORD
Is this really hard, confusing, or inferior? I don't know how it could
be made easier - an Admin needs to choose these settings. I think what
you don't like is that you have to know and use these extra arguments
for the connection protocol and bind method? I think you wanted to
hide all of that, but these are really decisions that can not be assumed.
The point here - I have always wanted to make the tools abstract from the gory horrid details that exist in cn=config. Setting attributes by hand wasn't what I "envisaged". The goal of these tools was such that you never had to know about these attributes or objects at all.
I think you are making assumptions again about how the current CLI
works. It's not strictly attribute based settings as you keep insisting
(as we saw from my replication example) I'm sorry but I think you are
biased here and you have not really looked at the tools (I'm not biased
of course haha). But let me give you some more examples:
Create backend/suffix:
# dsconf instanceA backend create --suffix dc=my,dc=suffix --be-name
userRoot
Configure a backend:
# dsconf instanceA backend suffix set --add-referral REFERRAL
--cache-memsize 100000000
Add encrypted attribute:
# dsconf instanceA backend attr-encrypt --add-attr creditCardAttr
Run cleanAllRUV task
# dsconf instanceA repl-tasks cleanallruv --replica-id 1 --suffix
dc=my,dc=suffix
Run the monitor
# dsconf instanceA monitor [backend, server, chaining, SNMP, ldbm]
These are pretty basic and I'm not sure how they could be improved, but
then there are configurations that require more fine grained control
like password policy. So here we do require attribute settings, but
they use more friendly names:
# dsconf instanceA pwpolicy set --pwdhistorycount 5 --pwdchecksyntax on
--pwdminlowers 3
The issue with what you envisioned is that it is impossible to assume
how everything should be configured. You still need the user to make
precise decisions about settings.
Now, people can always go an edit cn=config, I'll never stop that. If someone wants to use ldapmodify to admin their server, absolutely go ahead. But if I use a cli tool like this, I want it to abstract toward tasks that help me get my job done. I don't want to be an expert in remembering every single attribute on cn=replica, I want to just have two servers replicate. (sorry to bash-on replication, it's just a good example of a task that is reasonable complex to configure).
For me this is hard emotionally, because I feel like lib389, and the ds* tools are really creations of my love and passion,
I think some of us who worked countless hours on the initial version of
lib389, before you joined the team, feel exactly the same way.
to improve things in the ways I always wanted 389 to be as the product I want to use, and enjoy using. To make tooling that encapsulates all our amazing knowledge and experience inside of highly accessible commands. Which then leads me to be sad about how things turned out because it's not the way "I wanted" (again, I feel like this failure is my responsibility due to what happened last year, not the fault of anyone else - everyone else has done their absolute best).
I would propose to change this - but I feel we are now wedged on change by two things:
* Red Hat have documented these tools, and it's a big ask to get them to re-document everything.
* The cockpit console uses these tools in a special --json mode ( I would have created a dscockpit command that just handled the --json mode, rather than trying to make a tool do two jobs well.). This means rewriting cockpit integration ....
There was a brief talk about this actually while you were gone. But
Admins could use the json output for these same tasks so we left it in
dsconf. I would not be opposed to doing this though. It would not be a
major change for the UI, but it would be a lot of work to write the tool
(alot of redundancy with dsconf). We also talked about doing bulk data
loading, so we could do something like call "ds-cockpit load
replication", or load-everything, instead of calling "dsconf get" 20 times
* Time - heaps of time has already gone into these tools now, can we really justify undoing all that effort for this?
Which is a bit why I'm writing this
- Should I stop my task-oriented-vision and complaining that I have been doing, and accept what we have today?
- Do we want to try and pivot to change based on the ideas I have?
- Do I build something in parallel to satisfy my selfish desires, and to supplement what we have?
I really think you should write a new CLI tool that only does your
recipe-style tasks for certain things. Like I said that approach will
not cover everything, but it could be used for other tasks. Let the
existing dsconf handle the fine grained configuration (although it's not
as bad as you make it sound), then have a tool like "dstask", "dsrecipe"
or something, that handles larger replication deployments, or whatever
else you wanted to generalize.
To be honest I would like to hear from you which exact configurations
you want to change. Give some good detailed examples. You keep saying
how we should use a different style, but I think when you look closely
at these areas of configuration you will find it's not possible to
generalize them and to hide the horrid attributes underneath. IMHO I
feel most of the CLI commands are already doing what you kind of want
except for a few of them which require fine grained control. We can
also add new arguments to the existing commands, but it's a bit late to
rewrite/replace all of the usage.
So I am all for extending/improving the current CLI (it's certainly not
perfect), or even you adding another tool for doing recipe tasks, but
rewriting all of it doesn't sit well with me.
These are just my (biased) opinions, it would be nice if others from the
389 team (or anyone) would speak up and voice their opinions and
perspectives...
Mark
I don't know. But I think it's important to have this discussion as a team, including our emotional investments, concerns, visions and desires so that we can choose whats best. It's not good if we are "in-fighting" over the things we are doing, that helps no one - and neither does people feeling like their work isn't appreciated or respected (and I'm worried I have come off as disrespectful, which I am sorry for). We should be unified in what we want to do and achieve.
So I'd appreciate advice here. What should I/we do? What do we want 389 to look like in the future from a user-experience perspective?
Thanks everyone,
—
Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server
SUSE Labs
_______________________________________________
389-devel mailing list -- 389-devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-devel@xxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
389-devel mailing list -- 389-devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-devel@xxxxxxxxxxxxxxxxxxxxxxx
