I have done some tests about reading git-rev-list output with different IPC facilities: - Native Qt QProcess socket based IPC - pipe based: popen() and fread() - redirecting git-rev-list to a file (under tmpfs) and 'block reading' back the file: read() under Qt QFile class. The file, always in memory, is deleted at the end of loading. I have tested with different block sizes and different CPU speed on linux and git trees. The averaged results on linux tree (about 30MB of data) and CPU set at 1.2GHz are: - QProcess 6734ms - pipe and fread() with 64KB blocks 4832ms (38% faster then QProcess) - temporary file and read() with 64KB blocks 4321ms (10% faster then pipe) I have not enough knowledge to understand why temporary file is faster then pipe. My guess is, after reading some docs around, fread() uses a C standard I/O buffer, while read() is unbuffered. To make git-rev-list writing a temporary file I create and run a script more or less like: git rev-list --header --boundary --parents --topo-order HEAD > /tmp/qgit_135902672.txt There is also some additional kludge to get git-rev-list pid, so to use in case of a cancel request arrives while loading. So my request is if it is possible to make git-rev-list write _directly_ to a file, without shell redirection, I would ask if it is possible: - Add a -o, --output option to git-rev-list to specify output file instead of stdout - Use an unbuffered binary block write to be as fastest as possible - Do *not* flush ever the file, it's only a waste under this scenario. This will let me to avoid the shell launching crap and probably gain also some more speed. I understand this could be not exactly a top priority feature for git people, but I would really like to get the best possible interface with the plumbing git and the -o options is also a very common one. Thanks Marco P.S: On another thread I explained why I see problematic linking directly against libgit.a I rewrite here for completeness:
I've looked again to Shawn idea (and code) of linking qgit against libgit.a but I found these two difficult points: - traverse_commit_list(&revs, show_commit, show_object) is blocking, i.e. the GUI will stop responding for few seconds while traversing the list. This is easily and transparently solved by the OS scheduler if an external process is used for git-rev-list. To solve this in qgit I have two ways: 1) call QEventLoop() once in a while from inside show_commit()/ show_object() to process pending events 2) Use a separate thread (QThread class). The first idea is not nice, the second opens a whole a new set of problems and it's a big amount of not trivial new code to add. - traverse_commit_list() having an internal state it's not re-entrant. git-rev-list it's used to load main view data but also file history in another tab, and the two calls _could_ be ran concurrently. With external process I simply run two instances of DataLoader class and consequently two external git-rev-list processes, but If I link against libgit.a that would be a big problem.
- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html