On 8/13/07, Shawn O. Pearce <spearce@xxxxxxxxxxx> wrote: > > On 32b there's windowing code for accessing the packfile since we can > > run out of address space, does this code get turned off for 64b? > > The windowing code you are talking about defaults as follows: > > Parameter 32b 64b > ----------------------------------------- > core.packedGitWindowSize 32M 1G > core.packedGitLimit 256M 8G > > So I doubt you are having issues with the windowing code on a 64b > system, unless your repository is just *huge*. I did not think that > anyone had a Git repository that exceeded 8G, though the window > size of 1G might be a tad too small if there are many packfiles > and they are each larger than 1G. Why use windows on 64b? Default core.packedGitWindowSize equal to core.packedGitLimit I haven't measured it but I suspect the OS calls for moving the windows are are quite slow on a relative basis since they have to rewrite a bunch of page tables. Why is the window so small on 32b? I thought we were up to about a 1GB packfile before running out of address space with Mozilla. Shouldn't the window simply be set as large as possible on 32b, this size being a function of the available address space, not the amount of physical memory? > > > * On the other hand, we could run all try_delta operations for one object > > > parallel. This way, we would need not very much more memory, but > > > require more synchronization (and more complex code). > > > > This solution was my first thought too. Use the main thread to get > > everything needed for the object into RAM, then multi-thread the > > compute bound, in-memory delta search operation. Shared CPU caches > > might make this very fast. > > I have been thinking about doing this, especially now that the > default window size is much larger. I think the default is up as > high as 50, which means we'd keep that shiny new UltraSPARC T2 busy. > Not that I have one... so anyone from Sun is welcome to send me > one if they want. ;-) > > I'm not sure its that complex to run all try_delta calls of the > current window in parallel. Might be a simple enough change that > its actually worth the extra complexity, especially with these > multi-core systems being so readily available. Repacking is the > most CPU intensive operation Git performs, and the one that is also > the easiest to make parallel. > > Maybe someone else will beat me to it, but if not I might give such > a patch a shot in a few weeks. I'll test it. But my assignment this week is to figure out how to program the BestComm DMA engine in a PowerPC chip. It's oh so much fun trying to figure out how to program these "engines" that ASIC designers love to build and never fully document. > > -- > Shawn. > -- Jon Smirl jonsmirl@xxxxxxxxx - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
