Hey, Attached is a yum metadata parser written in C. It should produce identical results with the sqlitecache implementation in yum-2.6.1 but should be quite a bit faster. Here are the numbers parsing FC5 core metadata (2207 packages): file this old -------------------- primary 0.75 8.54 filelists 2.55 17.0 other 3.87 19.96 Total 7.17 45.5 I ran the tests 3 times and took the average and deleted *.sqlite after each run. To test it, unpack the attached tarball and build it with standard: python setup.py build sudo python setup.py install To make yum use it, rename the original parser and copy the file from the tarball to it's place: mv /usr/lib/python2.4/site-pakages/yum/sqlitecache.py /usr/lib/python2.4/site-pakages/yum/sqlitecache.py.old cp ./sqlitecache.py /usr/lib/python2.4/site-pakages/yum/sqlitecache.py That's it, the next time yum should use the new and faster parser. Some notes: I updated the dbversion because I modified the SQL schema slightly: When deleting a package from 'packages' table, there are now SQL triggers to delete related rows from other tables (files, 'prco', filelists, changelog). It doesn't work for regular users: The current implementation uses in memory database for that case, which is not possible with this implementation. There's no way to return the sqlite db handle from C to python so the parser currently closes the db and returns the file name and python part re-opens it. I'm thinking about adding per-user sqlite caches somewhere in users' home directories for that. Something like ~/.yum/$reponame/$md_filename.xml.gz.sqlite. The yum logging isn't used. All the output is printed to stdout and stderr. It's quite easy to fix but ... Thoughts? Ideas? Comments? Thanks, Tambet -------------- next part -------------- A non-text attachment was scrubbed... Name: yum-parser.tar.gz Type: application/x-compressed-tar Size: 10621 bytes Desc: not available Url : http://lists.dulug.duke.edu/pipermail/yum/attachments/20060510/a9efc94e/yum-parser.tar.bin
