On 5/24/19 9:50 AM, Mark Reynolds wrote:
On 5/22/19 9:02 PM, William Brown wrote:
Hi all,
I've had this thought and email in mind for a while, so I think it's
time for me to write out some thoughts. I think there is a difference
in the command line tools today from what I had envisaged, to what
they became.
Now of course, there are lots of reasons for this - Simon and Mark
have done awesome work here, and the team picked up after my uhh ...
"untimely eviction" we'll say, when I went through a lot of troubling
personal issues. So you know, lots of <3 for the team being fantastic
and patient with me despite my apparent insanity :)
So I think what happened is that because I didn't expect this,
communication didn't happen, I didn't support everyone and didn't
communicate what I had in mind. I'm basically going to take this one
on myself.
What this has led to is a pretty different approach - where I took a
task-oriented approach, Simon and Mark in the design of the new
web-console have taken an attribute-setting approach.
This is not entirely true, and I will give examples below. Yes the
approach I took was that you can have fine grained control over all
settings. This was voted on by the entire 389 development team btw.
I still strongly believe we should not hide any configuration in the
CLI tools. The tools should be able to do everything. That being
said we did try to do bundle operations into a task approach which I
will show under your example of how the tools currently works...
What does this mean?
Let's take replication as an example: I would have created the tools
to look more like:
# Setup a first master
dsconf instance_a replication initialise-topology
# Join a second master, doing a re-init as needed and setting up
replication managers etc.
dsconf instance_b replication join-as-master --master instance_a
# Join a third master ...
dsconf instance_c replication join-as-master --master instance_a
# Make sure the agreements between a-b-c are full mesh
dsconf instance_a relication ensure-agreement instance_b
dsconf instance_a relication ensure-agreement instance_c
...
This could have actually been a different tool like dsrepl or
something which may even be more expressive, to coordinate
replication as a topology rather than per-server focussed.
An attribute-setting approach though is more like (this is made up in
my head, not reflective of current actual command, just for the point
of ideological discussion):
dsconf instance_a changelog init
dsconf instance_a replication_manager create
dsconf instance_a replica init --rid=10
dsconf instance_b changelog init
dsconf instance_b replication_manager create
dsconf instance_a replica init --rid=11
dsconf instance_a replication_agreement create --to=instance_b
This looks a bit tedious I agree, but it's not how it works. Here are
the actual steps to setup MMR replication between two servers using
dsconf as it exists today. It only takes 4 commands to get it running...
Enable replication:
# dsconf instanceA replication enable --role master --rid 1 --bind-dn
"cn=replication manager,cn=config" --bind-pw PASSWORD --suffix
"dc=example,dc=com"
-> This command above enables replication, creates the changelog,
creates the replication manager, and adds the manager to the
replication configuration. Nice! One command does all these steps
:-) The below command sets up the second master...
# dsconf instanceB replication enable --role master --rid 2 --bind-dn
"cn=replication manager,cn=config" --bind-pw PASSWORD --suffix
"dc=example,dc=com"
Create the agreements:
# dsconf instanceA repl-agmt create MY_REPL_AGMT_NAME --host
ldapB.example.com --port 636 --conn-protocol LDAPS --bind-method
SIMPLE --bind-dn "cn=replication manager manager,cn=config" --bind-pw
PASSWORD --init
This creates the agreement and initializes it. Then create the
agreement from B to A which does not need to be inited
# dsconf instanceB repl-agmt create MY_REPL_AGMT_NAME --host
ldapA.example.com --port 636 --conn-protocol LDAPS --bind-method
SIMPLE --bind-dn "cn=replication manager manager,cn=config" --bind-pw
PASSWORD
Is this really hard, confusing, or inferior? I don't know how it
could be made easier - an Admin needs to choose these settings. I
think what you don't like is that you have to know and use these extra
arguments for the connection protocol and bind method? I think you
wanted to hide all of that, but these are really decisions that can
not be assumed.
The point here - I have always wanted to make the tools abstract from
the gory horrid details that exist in cn=config. Setting attributes
by hand wasn't what I "envisaged". The goal of these tools was such
that you never had to know about these attributes or objects at all.
I think you are making assumptions again about how the current CLI works.
It's not strictly attribute based settings as you keep insisting (as
we saw from my replication example) I'm sorry but I think you are
biased here and you have not really looked at the tools (I'm not
biased of course haha). But let me give you some more examples:
Create backend/suffix:
# dsconf instanceA backend create --suffix dc=my,dc=suffix --be-name
userRoot
Configure a backend:
# dsconf instanceA backend suffix set --add-referral REFERRAL
--cache-memsize 100000000
Add encrypted attribute:
# dsconf instanceA backend attr-encrypt --add-attr creditCardAttr
Run cleanAllRUV task
# dsconf instanceA repl-tasks cleanallruv --replica-id 1 --suffix
dc=my,dc=suffix
Run the monitor
# dsconf instanceA monitor [backend, server, chaining, SNMP, ldbm]
These are pretty basic and I'm not sure how they could be improved,
but then there are configurations that require more fine grained
control like password policy. So here we do require attribute
settings, but they use more friendly names:
# dsconf instanceA pwpolicy set --pwdhistorycount 5 --pwdchecksyntax
on --pwdminlowers 3
The issue with what you envisioned is that it is impossible to assume
how everything should be configured. You still need the user to make
precise decisions about settings.
Now, people can always go an edit cn=config, I'll never stop that. If
someone wants to use ldapmodify to admin their server, absolutely go
ahead. But if I use a cli tool like this, I want it to abstract
toward tasks that help me get my job done. I don't want to be an
expert in remembering every single attribute on cn=replica, I want to
just have two servers replicate. (sorry to bash-on replication, it's
just a good example of a task that is reasonable complex to configure).
For me this is hard emotionally, because I feel like lib389, and the
ds* tools are really creations of my love and passion,
I think some of us who worked countless hours on the initial version
of lib389, before you joined the team, feel exactly the same way.
to improve things in the ways I always wanted 389 to be as the
product I want to use, and enjoy using. To make tooling that
encapsulates all our amazing knowledge and experience inside of
highly accessible commands. Which then leads me to be sad about how
things turned out because it's not the way "I wanted" (again, I feel
like this failure is my responsibility due to what happened last
year, not the fault of anyone else - everyone else has done their
absolute best).
I would propose to change this - but I feel we are now wedged on
change by two things:
* Red Hat have documented these tools, and it's a big ask to get them
to re-document everything.
* The cockpit console uses these tools in a special --json mode ( I
would have created a dscockpit command that just handled the --json
mode, rather than trying to make a tool do two jobs well.). This
means rewriting cockpit integration ....
There was a brief talk about this actually while you were gone. But
Admins could use the json output for these same tasks so we left it in
dsconf. I would not be opposed to doing this though. It would not be
a major change for the UI, but it would be a lot of work to write the
tool (alot of redundancy with dsconf). We also talked about doing
bulk data loading, so we could do something like call "ds-cockpit load
replication", or load-everything, instead of calling "dsconf get" 20
times
* Time - heaps of time has already gone into these tools now, can we
really justify undoing all that effort for this?
Which is a bit why I'm writing this
- Should I stop my task-oriented-vision and complaining that I have
been doing, and accept what we have today?
- Do we want to try and pivot to change based on the ideas I have?
- Do I build something in parallel to satisfy my selfish desires, and
to supplement what we have?
I really think you should write a new CLI tool that only does your
recipe-style tasks for certain things. Like I said that approach
will not cover everything, but it could be used for other tasks. Let
the existing dsconf handle the fine grained configuration (although
it's not as bad as you make it sound), then have a tool like "dstask",
"dsrecipe" or something, that handles larger replication deployments,
or whatever else you wanted to generalize.
To be honest I would like to hear from you which exact configurations
you want to change. Give some good detailed examples. You keep
saying how we should use a different style, but I think when you look
closely at these areas of configuration you will find it's not
possible to generalize them and to hide the horrid attributes
underneath. IMHO I feel most of the CLI commands are already doing
what you kind of want except for a few of them which require fine
grained control. We can also add new arguments to the existing
commands, but it's a bit late to rewrite/replace all of the usage.
So I am all for extending/improving the current CLI (it's certainly
not perfect), or even you adding another tool for doing recipe tasks,
but rewriting all of it doesn't sit well with me.
These are just my (biased) opinions, it would be nice if others from
the 389 team (or anyone) would speak up and voice their opinions and
perspectives...
I think I've also made some wrong assumptions about what you want to do
with the "recipe" style. I looked at
https://www.port389.org/docs/389ds/design/cli-guide.html, but didn't see
any good examples about what you actually want, and how it relates to
complex tasks that require several required parameters, etc. So like I
said above I want to see explicit examples of how you handle some of
these tasks. For example in my replication examples I have:
# dsconf instanceA replication enable --role master --replica-id 1
--bind-dn "cn=replication manager,cn=config" --bind-pw PASSWORD --suffix
"dc=example,dc=com"
Here, "--role", "--replica-id" and "--suffix" are required. "--bind-dn"
and "--bind-pw" are optional. So is this how you would like to see it?
Does this format bring you joy? :-)
# dsconf instanceA replication enable role master replica-id 1 suffix
"dc=example,dc=com" --bind-dn "cn=replication manager,cn=config"
--bind-pw PASSWORD
Simon pointed out the openshift CLI is what you were trying mimic to
some degree. Well I never used it, never seen it. I think when you
write up some of this stuff you assume the audience has a certain
knowledge/experience level. This is why I like to see detailed examples
of what you are describing, not just abstract logic. So what would be
beneficial to a lot of us on the team would be to see actual examples of
these complex tasks using the CLI style you would prefer. Then we can
better evaluate it and have more efficient discussions about it.
Thanks,
Mark
Mark
I don't know. But I think it's important to have this discussion as a
team, including our emotional investments, concerns, visions and
desires so that we can choose whats best. It's not good if we are
"in-fighting" over the things we are doing, that helps no one - and
neither does people feeling like their work isn't appreciated or
respected (and I'm worried I have come off as disrespectful, which I
am sorry for). We should be unified in what we want to do and achieve.
So I'd appreciate advice here. What should I/we do? What do we want
389 to look like in the future from a user-experience perspective?
Thanks everyone,
—
Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server
SUSE Labs
_______________________________________________
389-devel mailing list -- 389-devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/389-devel@xxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
389-devel mailing list -- 389-devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/389-devel@xxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
389-devel mailing list -- 389-devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to 389-devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/389-devel@xxxxxxxxxxxxxxxxxxxxxxx
