(Presenter: Emily Shaffer, Notetaker: Taylor Blau)
* (Emily) In the last year, there were a handful of scenarios where we had
issues with backwards compatibility.
* E.g. config based hooks. As a result of this change, text coloration was
broken from user-provided hooks. Missing tests, but still viewed it as
backwards-compatibility-breaking.
* E.g. deleted internal "git submodule--helper" that looked like a plumbing
command, which other projects depended on. Was being used in the wild, even
though we didn't expect it.
* E.g. bare repository embedding. Interested in fixing that as a security
concern
(https://offensi.com/2019/12/16/4-google-cloud-shell-bugs-explained-bug-3/,
https://github.com/justinsteven/advisories/blob/main/2022_git_buried_bare_repos_and_fsmonitor_various_abuses.md).
Weren't able to do so in a widespread fashion, since many projects are
using it for testing setups (i.e. test fixtures).
* (Emily) When do we consider odd behavior as bugs, versus when do we consider
them as something that's part of our backwards compatibility guarantee.
* (Emily) What do we want backwards compatibility to look like for libraries?
How do we want to handle this in the future?
* (Minh) Is there documentation on how this should behave?
* (Emily): Typically try to guarantee backwards compatibility via integration
tests. Have changed documented behavior in the past when it seems "just
wrong". Is the documentation the source of truth?
* (Jonathan Nieder): In the case of browsers, using a specification for
compatibility is a useful tool. What will work at the same time across
different implementations? When there is a single implementation (e.g. git)
it is easier to capture your intention with the implementation instead of a
specification.
* (Jonathan Nieder): Converting that documentation into a specification can
hurt readability or inhibit its other uses.
* (Junio): Tend to ensure that observable behavior is locked in via
integration tests. Tests are the source of truth, along with
implementation. Documentation is often lying. Unlike the browser example,
we don't have an external specification to rely on. Intention from
developers is captured in the log proposed message.
* (Minh) Should we be testing more, then?
* (Emily) That's part of it, but some older behavior (e.g. from the original
implementation) has less information in the commit message as a result of
project culture at the time.
* (Junio) Working-as-designed, but the design outlived its usefulness.
* (Jonathan Nieder) E.g. ‘git-add' versus ‘git add'. Outcry after we changed
behavior, so we rolled it back. Much later we got to a place where people
weren't relying on this behavior as much.
* (Jonathan Nieder) There is another kind of documentation besides
specification. E.g. the kernel has a documented guarantee about compatibility:
"the kernel never breaks userspace". This doesn't mean that we can't have
observable behavior changes. Only that they maintain "depended-upon" behavior
that the kernel is reasonably responsible for providing. Can only determine
this surface area by rolling out changes and seeing if folks complain.
* (Jonathan Nieder) It sometimes feels like we have adopted a similar
philosophy, but the kernel has an easier job since POSIX, System V, etc have
defined the overall shape of the syscall interface.
* (Elijah) Difficult to distinguish between bug fixes and breaking backwards
compatibility. When we break existing test cases, document a rationale for
doing so in the proposed patch message. Cases where documentation was just
wrong. Often comes down to a judgment call.
* (Minh) Is the consensus to keep tests up-to-date, and add more tests when
behavior is unclear?
* (Jonathan Nieder) Problem is that there can be differences of opinion on what
are safe compatibility guarantees to break.
* (Emily) Also the case that there are tests that are in the integration suite
that are enforcing things that weren't meant to be compatibility guarantees.
E.g. enforcing error messages. How do we cope with legacy behavior and legacy
tests when making a judgment call? There is some documentation in general
about backwards compatibility, plumbing commands are frozen, porcelain
commands are not. Should we expand that documentation to clarify how to
decide?
* (brian) This would be helpful, but not sure what it would look like. Kernel's
approach may be too rigid for Git. Sometimes useful to break backwards
compatibility. E.g. "we have it, but it isn't a good choice." Users depend on
those error messages. When we make a change that is overwhelmingly beneficial,
can't please everybody all of the time.
* (Jonathan Nieder) Back to guarantees for library code. Kind of view the
plumbing/porcelain decision as a failed experiment. Of course scripters are
going to use the plumbing. Want a better backwards compatibility guarantee.
People are going to want to add more functionality there and lock in
additional behavior. When people write scripts, they write using the commands
that they understand how to use. End-user commands gain for-scripting
functionality.
* (Junio) Worse is when new features are added only to porcelain, and plumbing
code is left behind.
* (Jonathan Nieder) In a way, we made it harder on ourselves. If porcelains are
written as scripts, you need plumbing commands to expose the functionality
they need. Now porcelains use function calls, so the well maintained interface
is more on the (internal) library side
* Libification moves us in a good direction, since it provides an alternative to
the CLI as a well-defined programmatic access method.
* (Jonathan Nieder) If we succeed at this, the command-line backwards
compatibility guarantee for porcelain commands can break down a bit to the
extent that users start to adopt the library code as their interface to Git.
* (Emily) If we have suitable replacements in the library, can we deprecate the
plumbing variant of that functionality eventually? Freeze a particular
plumbing command instead of adding to it
* (Taylor) Can't break existing behavior, shouldn't have to force users to
upgrade to library code for existing behavior. Apologies if this is what you
were saying.
* (Jakub) Auto-generated CLI shim, like cloud providers often provide for their
APIs?
* (Jonathan Tan) Might be hard to create scriptable interfaces for library
commands. Library allows us to pass pointers and function callbacks, neither
of these we can accomplish via the shell.
* (Minh) Is there an understanding that the library has to implement 100% of the
functionality of plumbing commands?
* (Emily) Not convinced that we need a one-to-one match between the library and
command-line interface. Want to expose the same intent, not necessarily exact
incantations.
* (Jonathan Nieder) Let me try and summarize. Question resonates with people, no
one has a silver bullet. Maybe some agreement for using more tests, but the
general approach to figuring out our compatibility guarantees remains an open
discussion.
* (brian, via chat) One final thought: maybe we could look at what Semantic
Versioning defines a breaking change as, since they've defined this in a very
public way.
* (Phillip, via chat) Thinking back to yesterday there were people saying that
they chose the cli over a library because of concerns about memory leaks and
the library crashing/dying as well as licensing concerns. If we were to add
new functionality only in libraries we'd need to make sure that they were
robust.
