> Binary pickle format is effectively a memory dump, so once the pickle > gets generated, reading it back in is effectively as fast as your disk- > to-memory transfer speeds. :) I've tested it on a 433 Celeron this > weekend, and if it was taking upwards of 40 seconds to parse the .xml > file with 3000 packages, reading in the pickle takes about 2 seconds. > > The only drawback is that it takes some time to generate and write out > the pickle -- about 30 seconds on that 433 Celeron, so on your initial > run with new metadata things will hurt. However, keep in mind that in > the release situation, the biggest repository will never change, only > the "updates" repository, which is not likely to have upwards of several > thousands of packages. One other drawback - the pickle format is given to break whenever the python version changes. So we'll need to make sure, if there is a problem importing the pickle it falls back properly. -sv
