Re: Python extension commands in git - request for policy change

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 11/25/2012 09:53 AM, Felipe Contreras wrote:
> On Sun, Nov 25, 2012 at 3:44 AM, Eric S. Raymond <esr@xxxxxxxxxxx> wrote:
>> 1) In 2012, we can specify a "floor" Python version of 2.6 (shipped in
>> 2008) and be pretty much guaranteed it will be anywhere we want to
>> deploy except Windows.  Windows will remain a problem because Python
>> isn't part of the stock install, but that's an equal or worse problem
>> for shell and Perl - and at least the Python project ships a binary
>> installer for Windows.
> 
> What if my extension only supports python 2.7? Or what if my extension
> wants to support 2.0?

There would obviously have to be a policy like "all Python code in core
git must run on any Python interpreter with 2.6 <= version < 3.0", just
as there are policies about what C and shell features are allowed.  If
you happen to want to support earlier versions of Python, I don't see
why anybody would stop you as long as your code also runs in the
mandated versions.

(In practice, backwards compatibility within Python versions 2.x is very
good and almost any code that runs in Python 2.6 would automatically run
in all later 2.x versions.  Moreover, the Python documentation covering
what is available in each version and the deltas between versions is
high-quality and easily available online.)

There is, of course, the awkward issue of how/when to transition to
Python 3.x, which is *not* backwards compatible with Python 2.x.  I
expect that when the time comes there will be volunteers (myself
included) willing to help adapt Python scripts to the new version, but
the problem shouldn't be minimized.

Of course Perl will have the same problem if Perl6 ever materializes.

>> 2) Python extension commands should test the Python version on startup
>> and die loudly but gracefully in the rare case that they don't find
>> what they need.
> 
> Yes, they should _if_ they know what version they need. In my
> extensions I really have no idea.

Then simply (with the help of the mailing list) ensure that your
extensions run under 2.6 (or whatever the chosen minimum version is) and
everything will be OK.  It is not an error to specify 2.6 as the minimum
version even though your script happens also to run on older versions :-)

>> 3) We should be unconditionally be encouraging extensions to move
>> from shell and Perl to Python.  This would be a clear net gain is
>> portability and maintainability.
> 
> NO! It's up to the developer to choose what language to use, and I
> find it very chauvinist of you to say "python is better, so let's all
> use python". So far you have listed a few advantages of python, but
> you haven't explained so far what is wrong with shell and perl.

Given that some languages are accepted in git-core and others are not,
it's already not "up to the developer to choose what language to use".
At best there is a short list of "blessed" languages, and the developer
can choose among only those.

> In fact, while advancing python you have made clear a problem with
> python; the version requirements. So far I have *never* encountered a
> problem with git because of my bash version, or my perl version. And
> we haven't touched to the python3 mess yet. To me, those are
> advantages of shell and perl.

On the contrary, there is *constant* traffic on the mailing list about
incompatibilities between different shell implementations (sh, dash,
bash, etc), not to mention those in other utilities (sed, grep, etc)
that one is forced to work with in shell scripts.  Compatibility is a
*huge* pain when developing shell code for git.  The fact that users
typically don't encounter such problems is due to the hard work of POSIX
lawyers on the mailing list correcting the compatibility errors of
mortal programmers.

> Actually, I don't care if 'git foo' is written in perl, or shell, or
> c; as long as it *works*. And I would hate it if 'git rebase' ever
> told me that I need a newer version of python, or worst; that I don't
> have python in my system (Arch Linux ships 'python2', not 'python').

The configure script would locate the correct interpreter and the build
would adjust the scripts' shebang lines, just as things are tweaked
within Perl scripts at build time.

>> 4) We should be encouraging C code to move to Python, too.  There's
>> little gain in portability on this path because modern C has cleaned
>> up its act a lot, but the drop in expected bug loads would be well
>> worth the porting effort.  Segfaults are not your friend, and the x2 to
>> x5 drop in line count would do very good things for long-term
>> maintainability.
> 
> Definitely NO! I really really doubt git in python would be able to
> achieve the same performance as git in c, but to show me wrong, it
> wouldn't be very difficult to run a few measurements with python
> dulwich *if* we are even to begin considering this point.
> 
> And are segmentation faults really that different from python's
> exceptions? Not to the user.

There is one huge difference: it C it is all too easy to write code that
leads to a security hole due to buffer overflows and other memory
management errors.  Code written in a scripting language are largely
immune to such problems (except of course for any such bugs in the
interpreter itself, but the testing of the interpreter is shared across
many projects and users).

It would be insane to rewrite performance-critical C code in any
scripting language, but there is a huge penumbra of code that is not
performance critical and that mutates rapidly.  Such code is much easier
to write and maintain in a sane scripting language if the portability
issues can be mastered.

The most important issues to consider when imagining a future with a
hybrid of code in C and some scripting language "X" are:

* Portability: is "X" available on all platforms targeted by git, in
  usable and mutually-compatible versions?

* Startup time: Is the time to start the "X" interpreter prohibitive?
  (On my computer, "python -c pass", which starts the Python
  interpreter and does nothing, takes about 24ms.)  This overhead would
  be incurred by every command that is not pure C.

* Should the scripting language access the C functionality only by
  calling pure-C executables or by dynamically or statically linking to
  a binary module interface?  If the former, then the granularity of
  interactions between "X" and C is necessarily coarse, and "X" cannot
  be used to implement anything but the outermost layer of
  functionality.  If the latter, then the way would be clear to
  implement much more of git in "X" (and lua would also be worth
  considering).

* Learning curve for developers: how difficult is it for a typical git
  developer to become conversant with "X", considering both (1) how
  likely is it that the typical git developer already knows "X" and
  (2) how straightforward and predictable is the language "X"?
  In this category I think that Python has a huge advantage over
  Perl, though certainly opinions will differ and Ruby would also be
  a contender.

Personally, I regret wasting my time programming pointer arithmetic in
git modules that are not performance-critical (and correcting bugs by
others in these areas).  And I'm tired of having an idea to improve a
git feature only to find that it is implemented in shell, where not even
arrays are available.  I would therefore welcome more friendliness
towards a decent scripting language in the git project.

Michael

-- 
Michael Haggerty
mhagger@xxxxxxxxxxxx
http://softwareswirl.blogspot.com/
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]