Dmitry Potapov <dpotapov@xxxxxxxxx> wrote:
> Though Git uses MAP_PRIVATE with mmap, this only marks mapped pages as
> copy-on-write. Because cloning does not change the pack file, all those
> pages should be shared.
I reran the test today. One client is receiving while the other one is
compressing. I have to interrupt both client because the server is
becoming unusable. I am sure you are right about sharing the pages but I
can't test it.
> On 64-bit architecture, you have plenty virtual space, and mapping
> a file to memory should not take much physical memory (only space
> needed for system tables).
What I can tell from the mmap man page is that it should map memory to a
file. I assume it shouldn't take up physical memory. However I am seeing
physical memory being consumed. It might be a feature of the kernel. Is
there a way to turn it off?
> You can reduce core.packedGitWindowSize in the Git configuration to
> see if it helps, but I doubt that it will have any noticeable effect.
It was worth a shot, it didn't help...
Looking some more into it today the bulk of the memory allocation
happens in write_pack_file in the following loop.
for (; i < nr_objects; i++) {
if (!write_one(f, objects + i, &offset))
break;
display_progress(progress_state, written);
}
This eventually calls write_object, here I am wondering if the
unuse_pack function is doing its job. As far as I can tell it writes a
null in memory, that I think is not enough to reclaim memory.
I also looked at the use_pack function where the mmap is
happening. Would it be worth refactoring this function so that it uses
an index withing a file instead of mmap?
Unless I hear of a better idea I'll be trying that tomorrow...
Take care,
--
Ivan Kanis
I can stand brute force, but brute reason is quite unbearable. There
is something unfair about its use. It is hitting below the intellect.
-- Oscar Wilde
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html