Re: [GSoC] Designing a faster index format - Progress report week 14

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 07/24, Nguyen Thai Ngoc Duy wrote:
> On Tue, Jul 24, 2012 at 2:08 AM, Thomas Gummerer <t.gummerer@xxxxxxxxx> wrote:
>
> Is ls-files improvement not drastic because you do not limit subdir
> like grep? I see Thomas Rast put similar partial loading hack to
> ls-files.c so I assume it can partial load too. Is partial loading
> still fast with when a lot of unmerged entries are present?
> -- 
> Duy

Yes, exactly, the ls-files was to show the overall performance of the
index for a full read. The improvement when limiting it to a subdir
is about the same as with git grep. Here are some more timings. I'm
using update-index instead of ls-files in this tests, with a
--force-rewrite option I added which writes the index even if there
are no changes, to test the performance of both the reader and the
writer.

Test                                        this tree      
-----------------------------------------------------------
0002.2: v[23]: update-index                 0.29(0.21+0.06)
0002.3: v[23]: grep nonexistent -- subdir   0.13(0.12+0.01)
0002.5: v4: update-index                    0.26(0.20+0.05)
0002.6: v4: grep nonexistent -- subdir      0.11(0.08+0.02)
0002.7: v4: ls-files -- subdir              0.10(0.07+0.02)
0002.9: v5: update-index                    0.19(0.11+0.07)
0002.10: v5: grep nonexistent -- subdir     0.01(0.00+0.00)
0002.11: v5: ls-files -- subdir             0.01(0.00+0.00)

Partial loading is still fast with unmerged entries, since we only
need to load files that belong to a specific directory there too.
I've created about 15,000 conflicts on the webkit repository, and
got the following times:

Test                                        this tree      
-----------------------------------------------------------
0002.2: v[23]: update-index                 0.30(0.18+0.10)
0002.3: v[23]: grep nonexistent -- subdir   0.13(0.09+0.04)
0002.5: v4: update-index                    0.26(0.22+0.04)
0002.6: v4: grep nonexistent -- subdir      0.11(0.07+0.03)
0002.7: v4: ls-files -- subdir              0.10(0.09+0.01)
0002.9: v5: update-index                    0.21(0.16+0.05)
0002.10: v5: grep nonexistent -- subdir     0.01(0.00+0.00)
0002.11: v5: ls-files -- subdir             0.01(0.00+0.00)

I could create more conflicts (~180,000 files are  in the index),
but I think 15,000 already is a number that's very unlikely to
be reached.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]