"brian m. carlson" <sandals@xxxxxxxxxxxxxxxxxxxx> writes: > On 2023-09-08 at 23:10:18, Eric W. Biederman wrote: >> The v3 pack index file as documented has a lot of complexity making it >> difficult to implement correctly. I worked with bryan's preliminary >> implementation and it took several passes to get the bugs out. >> >> The complexity also requires multiple table look-ups to find all of >> the information that is needed to translate from one kind of oid to >> another. Which can't be good for cache locality. >> >> Even worse coming up with a new index file version requires making >> changes that have the potentialy to break anything that uses the index >> of a pack file. >> >> Instead of continuing to deal with the chance of braking things >> besides the oid mapping functionality, the additional complexity in >> the file format, and worry if the performance would be reasonable I >> stripped down the problem to it's fundamental complexity and came up >> with a file format that is exactly about mapping one kind of oid to >> another, and only supports two kinds of oids. > > I think this is a fine approach, and as I'm sure you noticed from my > series, it's a lot more robust than trying to implement pack v3. I'd be > fine with going with this approach instead of pack v3. I think I got your pack v3 working but it was at a minimum a serious distraction. I worry a little bit that this might leave some performance on the table, with something like a 256 way jump table like we have in the index file. Still I figure we can start simple and when we start optimizing and profiling we can revisit the format if it shows up as a performance issue. Eric
