On Tue, Jun 28, 2016 at 3:43 PM, Lars Schneider <larsxschneider@xxxxxxxxx> wrote: > >> On 28 Jun 2016, at 15:14, Johannes Schindelin <Johannes.Schindelin@xxxxxx> wrote: >> >> Hi Duy, >> >> On Tue, 28 Jun 2016, Duy Nguyen wrote: >> >>> On Tue, Jun 28, 2016 at 11:40 AM, Johannes Schindelin >>> <Johannes.Schindelin@xxxxxx> wrote: >>>> >>>> On Mon, 27 Jun 2016, Duy Nguyen wrote: >>>> >>>>> On Mon, Jun 27, 2016 at 7:38 AM, <larsxschneider@xxxxxxxxx> wrote: >>>>>> ## Proposed solution >>>>>> Git LFS caches its objects under .git/lfs/objects. Most of the time >>>>>> Git LFS objects are already available in the cache (e.g. if you >>>>>> switch branches back and forth). I implemented these "cache hits" >>>>>> natively in Git. Please note that this implementation is just a >>>>>> quick and dirty proof of concept. If the Git community agrees that >>>>>> this kind of approach would be acceptable then I will start to work >>>>>> on a proper patch series with cross platform support and unit >>>>>> tests. >>>>> >>>>> Would it be possible to move all this code to a separate daemon? >>>>> Instead of spawning a new process to do the filtering, you send a >>>>> command "convert this" over maybe unix socket and either receive the >>>>> whole result over the socket, or receive a path of the result. >>>> >>>> Unix sockets are not really portable... >>> >>> It's the same situation as index-helper. I expect you guys will >>> replace the transport with named pipe or similar. >> >> Yes, I will have to work on that. But I might need to ask for a change in >> the design if I hit some obstacle there: named pipes are not the same at >> all as Unix sockets. >> >> Read: it will be painful, and not a general solution. So every new Unix >> socket that you introduce will introduce new problems for me. > > Thanks Duy for your suggestion. I considered a daemon, but a daemon makes > it always harder for the user as the user needs to ensure the daemon is > running! Plus, Dscho's concerns regarding Windows. > > I think the core problem is that we invoke the filter for every file: > https://github.com/git/git/blob/master/convert.c#L461-L475 > > Couldn't we start the filter executable at the beginning of the Git process > and communicate with it via stdin/stdout whenever we hit the Git filter > code? Would that work? Yeah if one filter process per one git process still brings significant perf. gain for you, why not, it's simpler than daemon. Though you may want to look at Christian's external odb first in the other mail. Note though that external odb may still spawn process a lot (because the design is you cache objects locally once and you don't have to spawn again). Whether that fits in lfs scheme, I have no idea (I have never used git-lfs myself). > Alternatively, do you see a way to add a "plugin" system to Git? Where Git > could be configured to dynamically load a "filter" library? I don't think plugins as .so files are welcome because it would force us to freeze some ABI. So far all git extension has always been via an external process. -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html