On Mon, 10 Sep 2007, Jeff King wrote: > On Mon, Sep 10, 2007 at 12:06:09AM -0400, Nicolas Pitre wrote: > > > + /* this should be auto-tuned somehow */ > > + chunk_size = window * 1000; > > How do you come up with this value? It was a wild initial guess. > It seems to work OK for the kernel repository, but not for repos which > consist of few large objects. My (perhaps pathological) repo of photos > has only 2523 objects, but is over a gig packed. Multithreading the > delta search is a big win here, but it doesn't activate with such a > large chunk size. Right. I had no (didn't take the time to find a) better idea at the time, hence the comment. The idea for this value is to have a small enough value so all threads are kept busy until the end. A too large value could create unbalanced work loads where one thread remains active long after all the others are done. A too small value would affect deltification since deltas are not performed across chunks currently, and even if it was done the result would still be suboptimal. Keeping chunks large enough avoid cross chunk deltification complexity (at least for now) and doesn't affect the pack size too much (with the Linux kernel repo at least). Of course this "window * 1000" is rather dumb and certainly could see improvements, possibly with dynamic adjustments. Please feel free to experiment. > Why did you switch from "total work / number of threads"? See "a too large number" above. Nicolas - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
