Dmitry Potapov <dpotapov@xxxxxxxxx> wrote:
> On Fri, Aug 06, 2010 at 07:23:17PM +0200, Ivan Kanis wrote:
>>
>> I expected the malloc to take 4G but was surprised it didn't. It seems
>> to be mmap taking all the memory. I am not familiar with that function,
>> it looks like it's mapping memory to a file... Is it reasonable to mmap
>> so much memory?
>
> AFAIK, Git does not need to mmap the whole pack to memory, but it
> is more efficient to mmap the whole pack wherever possible, because
> it has a completely random access, so if you store only one sliding
> window, you will have to re-read it many times. Besides, mmap size
> does not mean that so much physical memory is used. Pages should
> be loaded when they are necessary, and if you have more than one
> client cloning the same repo, this memory should be shared by them.
I have clone identical repositories and the system starts to swap. I
think it shows that cloning two repository doesn't share mmap.
I saw this constant defined in git-compat-util.h
/* This value must be multiple of (pagesize * 2) */
#define DEFAULT_PACKED_GIT_WINDOW_SIZE \
(sizeof(void*) >= 8 \
? 1 * 1024 * 1024 * 1024 \
: 32 * 1024 * 1024)
If I read this correctly git is allocating 1G of mmap on 64 bit
architecture. Isn't that a bit much? I am running on a 64 bit server so
I have bumped DEFAULT_PACKED_GIT_WINDOW_SIZE down to 64M but, alas, the
pack command still takes over 4G...
I'll keep investigating today,
Take care,
--
Ivan Kanis
Nothing in life is to be feared. It is only to be understood.
-- Marie Curie
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html