On 2/17/21 9:48 PM, Neeraj K. Singh via GitGitGadget wrote:
From: Neeraj Singh <neerajsi@xxxxxxxxxxxxxxxxxxx>
Writing an index 8K at a time invokes the OS filesystem and caching code
very frequently, introducing noticeable overhead while writing large
indexes. When experimenting with different write buffer sizes on Windows
writing the Windows OS repo index (260MB), most of the benefit came by
bumping the index write buffer size to 64K. I picked 128K to ensure that
we're past the knee of the curve.
With this change, the time under do_write_index for an index with 3M
files goes from ~1.02s to ~0.72s.
[...]
-#define WRITE_BUFFER_SIZE 8192
+#define WRITE_BUFFER_SIZE (128 * 1024)
static unsigned char write_buffer[WRITE_BUFFER_SIZE];
static unsigned long write_buffer_len;
[...]
Very nice.
I can confirm that this gives nice gains on Windows. (I'm using
the Office repo which has a 188MB index file (2.1M files at HEAD).
Running "git status" shows a gain of about 200ms.
We get a smaller gain on Mac of about 50ms (again, using the Office
repo).
So, you may add my sign-off or ACK to this.
Signed-off-by: Jeff Hostetler <jeffhost@xxxxxxxxxxxxx>
FWIW, You might take a look at `t/perf/p0007-write-cache.sh`
Update it as follows:
```
diff --git a/t/perf/p0007-write-cache.sh b/t/perf/p0007-write-cache.sh
index 09595264f0..337280ff1c 100755
--- a/t/perf/p0007-write-cache.sh
+++ b/t/perf/p0007-write-cache.sh
@@ -4,7 +4,8 @@ test_description="Tests performance of writing the index"
. ./perf-lib.sh
-test_perf_default_repo
+test_perf_large_repo
test_expect_success "setup repo" '
if git rev-parse --verify refs/heads/p0006-ballast^{commit}
```
Then you can run it like this:
$ cd t/perf
$ GIT_PERF_LARGE_REPO=/path/to/your/enlistment ./p0007-write-cache
Then you can run it with the small and then with the large buffer and
get times for essentially just the index write in isolation.
Hope this helps,
Jeff
