On Mon, 2004-10-04 at 12:34 -0400, seth vidal wrote:
> > > Icon did some good work this last week to help that problem. In short,
> > > yum will parse the xml files and write out a pickle file (if it is run
> > > as root) then it will mark the pickle with a checksum
> > > So the next time you need to read in that metadata, if the time repodata
> > > hasn't changed it will just read in the pickle which is an order of
> > > magnitude faster than the xml read-in.
> >
> > Nifty! Is that already in 2.1.5 that you just released or will it go in
> > later on?
>
> It will go in later on. I didn't have as much time as I wanted this
> weekend to mess with it and patch it in.
Binary pickle format is effectively a memory dump, so once the pickle
gets generated, reading it back in is effectively as fast as your disk-
to-memory transfer speeds. :) I've tested it on a 433 Celeron this
weekend, and if it was taking upwards of 40 seconds to parse the .xml
file with 3000 packages, reading in the pickle takes about 2 seconds.
The only drawback is that it takes some time to generate and write out
the pickle -- about 30 seconds on that 433 Celeron, so on your initial
run with new metadata things will hurt. However, keep in mind that in
the release situation, the biggest repository will never change, only
the "updates" repository, which is not likely to have upwards of several
thousands of packages.
Cheers,
--
Konstantin ("Icon") Ryabitsev
Duke University Physics Sysadmin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://lists.dulug.duke.edu/pipermail/yum/attachments/20041004/de806b95/attachment.bin
