Re: [PATCH] read-cache: call verify_hdr() in a background thread

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





On 3/24/2017 11:36 AM, Jeff King wrote:
On Fri, Mar 24, 2017 at 01:27:51PM +0000, git@xxxxxxxxxxxxxxxxx wrote:

From: Jeff Hostetler <jeffhost@xxxxxxxxxxxxx>

Teash do_read_index() in read-cache.c to call verify_hdr()
in a background thread while the forground thread parses
the index and builds the_index.

This is a performance optimization to reduce the overall
time required to get the index into memory.

Testing on Windows (using the OpenSSL SHA1 routine) showed
that parsing the index and computing the SHA1 take almost
equal time, so this patch effectively reduces the startup
time by 1/2.

Have you considered just skipping the sha1 check on read (possibly with
an option)?

Its purpose is to detect disk bit-rot. Checking it for every single
operation may be a bit excessive, especially because it gets rewritten a
lot. Nobody really cared until now because they don't have index files
that are hundreds of megabytes.

Yes, we have done that in a version of the client customized
for the virtual file system effort and have been using it
internally without incident for quite a while.

And we can get a patch for that instead (or include it with
this one) if there's interest.

I tried to limit my proposal here to increasing performance while
preserving the existing behavior.

Jeff



[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]