Hi Junio,
On Mon, 1 Nov 2021, Junio C Hamano wrote:
> Derrick Stolee <stolee@xxxxxxxxx> writes:
>
> > Optimization Factors
> > --------------------
> >
> > There are multiple factors involved that we wish to optimize. Here is
> > my understanding of the most-important factors and how they interact:
> >
> > ### Value
> >
> > The main reason to include Scalar within the Git codebase is to make
> > it available to more users. This should have downstream effects based
> > on user feedback about what they like or dislike about the Scalar CLI
> > that can inform future features in core Git that simplify repository
> > management at scale.
>
> In other words, Scalar wants to piggyback on Git's popularity, yet wants
> to retain total control of how its end-user experience should look like,
> eating the cake and having it at the same time?
>
> I do not think it is fair to proceed with this discussion without having
> "Scalar lives in its own tree, uses this mailing list to discuss how it
> should evolve, drawing expertise from the contributors found here, but
> otherwise would stay to be a spearate project" as another option and
> debating its pros-and-cons, next to the spectrum among contrib/scalar,
> scalar/, and scalar.c approaches.
If that was indeed the intention with Scalar, I would totally agree.
We don't want Scalar to piggy-back onto Git. In fact, we don't even want
it to be separate from Git, which is why we are contributing it here. I
firmly believe that Scalar has a lot of value and will benefit the Git
project and community. That is my motivation behind contributing these
patches.
> IIUC, cgit has one-directional coupling to depend on libgit.a just like
> the Scalar wants to link with it---I am not saying Scalar should do the
> same, but just raising it as an example that you do *not* have to live
> in my tree anywhere to be able to link with libgit.a; that would
> probably count as an "**Option 0:** A separate project that borrows from
> Git by having it as a submodule".
I did consider keeping Scalar/C in its own repository, and I, too, had
cgit in mind (and cinnabar). In fact, Scalar/C does live in its own
repository right now, and would remain there if the community doesn’t want
it: in the microsoft/git fork of git/git.
However, just like partial clone, scheduled maintenance, sparse-checkout
cone mode and sparse-index, the remainder of Scalar can benefit Git by
moving its functionality to `libgit.a`, or reimplementing it there.
> I can think of only one obvious advantage going that route, and that is
> why I said "I am not saying Scalar should do the same", which is that
> the Scalar project will then retain total control over how it is built
> and how its end-user experience should stay the same.
>
> I do not think it is an option to refuse to take any input from others
> on this list on parts of what you have already done, if you want to be
> in my tree, anywhere, not even in contrib/.
>
> Just like we say "the plumbing command X has behaved this way forever,
> and we cannot break the backward compatibility" to some possible changes
> to any of our tools, however, "Scalar has been used with that subcommand
> behaving this particular way even before the code was donated to Git
> project, so we cannot lightly change it" would equally be a good input
> when we evaluate when some things in it may need to be changed, so I do
> not think it would pose a great backward compatibility issue (after all,
> we do not change things lightly just for the sake of changing it).
That's a really good point.
> But approaching our developers with a new piece of code, saying "I want
> to piggyback on your popularity. Among the stuff given to you, some are
> out of limit and you may not even discuss about modifying it" upfront,
> is a problematic attitude.
Yes, I agree, that would be a problematic attitude. I apologize for giving
you the impression that that was my attitude. I want to contribute Scalar
to Git because I believe that Git will benefit from it, not because I want
to abuse Git’s popularity to advance Scalar.
> Now, the goal from purely Git project's point of view would be to
> improve Git's user experience, not necessarily Scalar's, by exposing
> more of our users to the way its opinionated (which is *not* necessarily
> a bad thing) decisions work to help its users, and our developers to the
> scalar codebase. Some of our developers may not even agree with the
> design decisions Scalar made and that is expected and is fine; based on
> such a reaction, the lesson Git learns from Scalar to solve a similar
> issue may turn out to be solve it differently.
>
> I do not know how much Options 1-3 would help such cross pollination
> more over the Option 0, offhand, though.
I am still of the opinion that having the code in the same tree will make
cross pollination much easier.
> > Possible approaches to land in our final target
> > -----------------------------------------------
>
> Here I sense the "final target" is somehow Scalar is delivered to
> end-users through my tree and being maintained there, but from my point
> of view, that is merely an early step. Wasn't the final target for
> "git" to learn from Scalar's successes and mistakes and give native
> support to itches Scalar wanted to scratch?
I think we are very much in alignment here, and I believe Stolee's goal
was to reach such an alignment.
> And if that is indeed the goal, when it is achieved, Scalar would become
> a historical curiosity just like Cogito is today. With Cogito, we
> cross-pollinated just fine without having both in the same tree at all,
> and when Cogito outlived its usefulness, we did not have to remove
> anything from our tree. With cgit, we did not cross-pollinate at all,
> unfortunately, and gitweb more or less stays the same way as it was
> years ago, and cgit is still a project with a useful product. We can
> reach both extremes even if we take Option 0. But it would have been
> messy for something like Cogito, which was an early UI and workflow
> experiment, if we took a more integrated approach.
Scalar is more similar to cgit than to gitweb or Cogito in this context:
gitweb and Cogito are scripted users of Git, whereas Scalar and cgit link
to libgit.a, are written in C, and it is easy to refactor code from Scalar
into libgit.a. That was never the case with gitweb nor with Cogito.
> > **Phase 1.** Keep code in contrib/ while we contribute the Scalar
> > features.
> >
> > Since the current patches on the mailing list are not feature
> > complete, it can be beneficial to move forward with the code in
> > `contrib/scalar/` until we reach feature parity. At that point, we
> > could move the source into its final resting place.
> >
> > This leans into the fact that `contrib/` contains "interesting
> > tools... maybe even experimental ones". While Scalar is not feature
> > complete, it can be isolated as experimental here until it is ready
> > for promotion to non-experimental.
>
> I personally have no problem with this approach, but with less scrutiny,
> being in contrib/ longer, with "some stuff are not subject to
> discussion" attitude, would inevitably lower the code quality of the
> product, and it is dubious it would become "ready for promotion" ever.
To be clear, I'm not pushing for promotion of Scalar to a top-level
command delivered via the Git project at all. I _might_ push for it in the
future, _iff_ a wider audience has used Scalar _and_ finds it useful.
In any case, pushing for Scalar to be such a top-level command would be
premature at this stage: to avoid overwhelming reviewers, the Scalar patch
series is not yet feature-complete, there are other patch series I intend
to contribute on top, when the first patch series has settled down.
> The contrib/ hierarchy, especially in early days when we wanted a way to
> grow the ecosystem, as a place to host "these are done externally, and
> we only carry them for convenience", was a successful model, but I do
> not think of anything that successfully used contrib/ as a nursery to
> later graduate to the core and to be seen as the same quality as other
> things that started their life in core. I am pessimistic to the model
> that uses contrib/ as such a place.
After seeing `vcs-svn`'s fate, I understand the trepidation. I, too, would
have loved to see it being cooked to perfection and then graduate.
Even so, I targeted `contrib/` with my patch series because
`contrib/README` says:
The intention is to keep interesting tools around git here, maybe
even experimental ones, to give users an easier access to them,
and to give tools wider exposure, so that they can be improved
faster.
Does this README's intent still apply, or has the purpose of contrib/
changed?
> > Depending on the final goal, we could drop some of the work that is
> > currently built within the `contrib/scalar/` directory, such as
> > `-c`/`-C` parsing or documentation builds. These features would be
> > reimplemented in the new location, so we can prevent that duplicate
> > effort if we have a different final location in mind.
>
> > **Phase 2.** Establish community standards on optional components
> >
> > While the work in `contrib/scalar/` continues to reach feature parity,
> > we can work as a community to set some ground rules about these kinds
> > of optional features (depending on how "optional" we land). This can
> > include standards such as how the files are laid out in the repository
> > and how they interact with the `Makefile`. Whatever we do for Scalar
> > is likely to form an example for a future contribution that we can't
> > predict today. Many of the questions we are asking today can be
> > written for posterity:
> >
> > * What is an appropriate level of coupling with the Git codebase?
> > * Where should code custom to the component live?
> > * How should documentation and tests be organized?
> > * How do we initialize builds of a component?
> > * Who is responsible for supporting the component? Who fixes the bugs?
> > * Who is responsible for reviewing changes to the component?
>
> I am afraid that the answer to the first four are "it depends" (but the
> remaining two would fall out naturally out of them).
Would you like me to organize the code and patches such that they more
clearly fall under your maintenance once the patches are merged? If that
is your preference, I will gladly put in the work.
When I submitted the Scalar patches, I did not want to burden you with
maintaining it, I expected to maintain Scalar going forward, but maybe
that is not your preference? I assumed it to be my maintenance burden, we
aim to support backward compatibility for existing Scalar users, after
all. Absolutely everything else, however, is up for discussion, subject to
the full scrutiny of the community and to the same standards of a
submission to the core of the project.
And I see your point about backward compatibility: The Git project has
plenty of experience with maintaining backward compatibility and _still_
improving on initial designs. I therefore can easily give up my hard “this
command-line interface must not be changed” stance.
> Thanks. My brain is fried with the -rc work, rereading the material
> that went to -rc0 since the last release, so I may have more to add
> later, but for now, the above is what came to my mind.
Thank you for taking the time to respond, especially during the -rc cycle.
I look forward to continuing this thread when you have time.
Ciao,
Dscho
