Re: Watchman/inotify support and other ways to speed up git status

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Oct 28, 2015 at 12:54 AM, David Turner <dturner@xxxxxxxxxxxxxxxx> wrote:
>
> On Thu, 2015-10-22 at 07:59 +0200, Christian Couder wrote:
>> Hi everyone,
>>
>> I am starting to investigate ways to speed up git status and other git
>> commands for Booking.com (thanks to AEvar) and I'd be happy to discuss
>> the current status or be pointed to relevant documentation or mailing
>> list threads.
>>
>> From the threads below ([0], [1], [2], [3], [4], [5], [6], [7], [8]) I
>> understand that the status is roughly the following:
>>
>> - instead of working on inotify support it's better to work on using a
>> cross platform tool like Watchman
>>
>> - instead of working on Watchman support it is better to work first on
>> caching information in the index
>>
>> - git update-index --untracked-cache has been developed by Duy and
>> others and merged to master in May 2015 to cache untracked status in
>> the index; it is still considered experimental
>>
>> - git index-helper has been worked on by Duy but its status is not
>> clear (at least to me)
>>
>> Is that correct?
>> What are the possible/planned next steps in this area? improving
>
> We're using Watchman at Twitter.  A week or two ago posted a dump of our
> code to github, but I would advise waiting a day or two to use it, as
> I'm about to pull a large number of bugfixes into it (I'll update this
> thread and provide a link once I do so).

Great, I will have a look at it then!

> It's good, but it's not great.  One major problem is a bug on OS X[1]
> that causes missed updates.  Another is that wide changes end up being
> quite inefficient when querying watchman.  This means that we do some
> hackery to manually update the fs_cache during various large git
> operations.
>
> I agree that in general it would be better to store or all some of this
> information in the index, and the untracked-cache is a good step on
> that. But with it enabled and watchman disabled, there still appears to
> be 1 lstat per file (plus one stat per dir).  The stats per-directory
> alone are a large issue for Twitter because we have a relatively deep
> and bushy directory structure (an average dir has about 3 or 4 entries
> in it).  As a result, git status with watchman is almost twice as fast
> as with the untracked cache (on my particular machine).

Thanks for this detailled description.

> [1] https://github.com/facebook/watchman/issues/172
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]