Re: [PATCH] don't use mmap() to hash files

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 02/14/2010 03:42 AM, Dmitry Potapov wrote:
In fact, we know mmap() also may fail for huge files, so can
strbuf_read().

On a 64-bit machine mmap should fail pretty much never, as it is limited
by address space and not available memory.

That said I can reproduce the result here too, and it's actually quite understandable from your "time" results: mmap has a minor page fault for every 4k you read (or at least that's the order of magnitude), read has basically none. Furthermore, it's true that with read you touch the whole memory twice from beginning to end, while with mmap you touch a page twice and move on; however, a file's contents will almost always fit in the L2 cache, so it's not too expensive.

I tried madvise(buf, size, MADV_SEQUENTIAL) and MADV_WILLNEED but it has no effect.

I suspect mmap will be faster only if the data is not in cache (so the cost of a page fault is negligible compared to going to disk) and the average file size is a few megabytes.

Paolo
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]