On Saturday 12 September 2009, Shawn O. Pearce wrote: > Johan Herland <johan@xxxxxxxxxxx> wrote: > > Shawn, do you have any additional defence for the date-based fanout? > > No. > > The only defense I have for it is "it sounds like a nice theory > given access patterns", and the note about memory usage you made, > but which I clipped to keep this email shorter. :-) > > It was only a theory I tossed out there in a back-seat-driver > sort of way. Your results show my hunch was correct, it may help. > But they also say it may not help enough to justify the complexity, > so I now agree with you that SHA-1 fan out may be good enough. Ok, so I guess we can drop the flexible part of notes code. Junio: Feel free to drop the two last patches from the jh/notes series. > > How does the plan for notes usage in your code-review thingy > > compare to my test scenario? > > I think your tests may still have been too low in volume, 115k notes > isn't a lot. Based on the distributions I was looking at before, > I could be seeing a growth of >100k notes/year. Ask me again in > 5 years if 115k notes is a lot. :-) > > But we all know that SHA-1 distributes data quite well, so the SHA-1 > fan-out may just need to change from 2_38 to 2_2_2_34 (or something) > to handle that larger volume. Yes, I expect that the optimal number of entries per tree level is ~256, so if we add an upper threshold at ~300 (where we start using another fanout level), and a lower threshold at ~200 (where we consolidate subtrees and put all into this level), the (still-to-be-written) writing part of the notes code should automatically adjust the notes tree to the optimal layout. With those assumptions, and a growth of 100k notes/year, a 2/2/36 fanout should last you ~150 years, and a 2/2/2/34 fanout should be enough for the next ~40,000 years... ;) Have fun! :) ...Johan -- Johan Herland, <johan@xxxxxxxxxxx> www.herland.net -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html