On 2007-05-21 16:17:22 +0100, Catalin Marinas wrote:
> On 21/05/07, Karl Hasselström <kha@xxxxxxxxxxx> wrote:
>
> > On 2007-05-21 12:15:40 +0200, Karl Hasselström wrote:
> >
> > > We might consider having plumbing written in C or something, and
> > > make sure that the plumbing can be called directly if there's
> > > need, but it's going to complicate things greatly compared to
> > > pure Python.
>
> Yes, it will complicate things. I think StGIT would have progressed at
> a much slower pace if written in C :-).
I agree. C is not a good language to write a whole application in. But
for the hot spots, C is a reasonable choice.
> > What I'm (foggily) envisioning here is to rewrite parts of StGIT
> > (as little as possible) as a C library (libstgit.so?), and call
> > the library both from the Python code, and from a "plumbing" C
> > program (stgit-helper?). We should not try to make the library API
> > stable, just like the current git library.
>
> Apart from the start-up time, I don't see other major slowdowns
> caused by Python.
I haven't, either.
> It would be useful to use a git library directly without invoking
> external applications (I'm not sure what's the state of a "libgit.a"
> or what improvement we would get).
There is no usable git library yet. But once there is, I agree we
should use it.
> As for the start-up time, unless you write most of the commands in C,
> we would still have to load Python modules. If you run stg-prof
> instead of stg for a simple command like 'top', you can see that the
> main function takes about 60-70ms, the rest to 150ms reported by the
> external 'time' is Python start-up and module loading.
My suggestion was to have a small stand-alone C program that could do
some operations that need to be really fast, such as
top/applied/unapplied. It need not have a nice user interface since
it's only going to be called by scripts (bash-completion and the
like), and it should only handle those operations that _must- avoid
the Python startup penalty. And for sanity reasons, it should share
code with stgit.
> I had a quick try at using "freeze.py" to generate a binary (well,
> it includes python bytecodes but it might save time on module
> look-up) but it got confused by my optimisation to only load module
> commands based on the stg arguments. Maybe we should try this first.
I agree that we should try pure-Python optimizations first.
> > There are two kinds of things we'd want to have in the library:
> > (1) things that are too slow to do in Python, and (2) things that
> > need to be available from stgit-helper in order to avoid Python's
> > startup cost, such as top/applied/unapplied for the bash
> > completion and bash prompt.
>
> As you probably guessed, I'm not really in favour of re-writing
> parts of StGIT in C, at least not in the near future, though anyone
> can fork and re-implement it :-).
I wouldn't do it just for fun, either. But if it's a prerequisite to
get good enough performance for something we really want, and all else
has failed, I'd be willing to argue for the introduction of a C
library and helper application.
--
Karl Hasselström, kha@xxxxxxxxxxx
www.treskal.com/kalle
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
