On Sat, May 3, 2014 at 11:39 AM, David Turner <dturner@xxxxxxxxxxxxxxxx> wrote: >> Index v4 and split index (and the following read-cache daemon, >> hopefully) > > Looking at some of the archives for read-cache daemon, it seems to be > somewhat similar to watchman, right? But I only saw inotify code; what > about Mac OS? Or am I misunderstanding what it is? It's mentioned in [1], the second paragraph, mostly to hide index I/O read cost and the SHA-1 hashing cost in the background. In theory it should work on all platforms that support multiple processes and efficient IPC. It can help load watchman file cache faster too. >> The last line could be a competition between watchman and my coming >> "untracked cache" series. I expect to cut the number in that line at >> least in half without external dependency. > > I hadn't seen the "untracked cached" work (I actually finished these > patches a month or so ago but have been waiting for some internal > reviews before sending them out). Looks interesting. It seems we use a > similar strategy for handling ignores. Yep, mostly the same at the core, except that I exploit directory mtime while you use inotify. Each approach has its own pros and cons, I think. Both should face the same traps in caching (e.g. if you "git rm --cached" a file, that file could be come either untracked, or ignored). >> Patch 2/3 did not seem to make it to the list by the way.. > > Thanks for your comments. I just tried again to send patch 2/3. I do > actually see the CC of it in my @twitter.com mailbox, but I don't see it > in the archives on the web. Do you know if there is a reason the > mailing list would reject it? Probably its size, 131K, which is also an indicator to split it (and the third patch) into smaller patches if you want to merge this feature in master eventually. > At any rate, the contents may be found > at > https://github.com/dturner-tw/git/commit/cf587d54fc72d82a23267348afa2c4b60f14ce51.diff Good enough for me :) > >> initial >> reaction is storing the list of all paths seems too much, but I'll >> need to play with it a bit to understand it. > > I wonder if it would make sense to use the untracked cache as the > storage strategy, but use watchman as the update strategy. I'm afraid not. If a directory mtime is changed, which means files/dirs have been added or deleted, the untracked code would fall back to the opendir/readdir/is_excluded dance again on that directory. If we naively do the same using watchman, we lose its advantage that it knows exactly what files/dirs are added/removed. That kind of knowledge can help speed up the dance, which is not stored anywhere in the untracked cache. We could extend the "read-cache daemon" mentioned above though, to hide all the hard work in the background and present a good view to git: when a file/dir is added, read-cache daemon classifies the new files/dirs as tracked/untracked/ignore and update its untracked cache in memory. When "git status" asks about the index and untracked cache, it will receive the _updated_ cache (not the on disk version any more) with latest dir mtime so git can verify the cache is perfect and skip opendir/.... All git does is to write the index down in the end to make the updated data permanent. It sounds interesting. But I'm not so sure if it's worth the complexity. [1] http://article.gmane.org/gmane.comp.version-control.git/247268 -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html