On Tue, Apr 05, 2005 at 01:28:43PM -0400, Konstantin Ryabitsev wrote: > On Apr 5, 2005 11:35 AM, seth vidal <skvidal@xxxxxxxxxxxx> wrote: > > Icon was working on repoview and decided to try out CelementTree b/c he > > was using kid anyway and it used it. After some preliminary tests it > > showed up as significantly faster parsing the metadata. For > > primary.xml.gz the times went from 21s for 1800ish pkgs to 7s. Then when > > he switched it to use iterparse() the memory footprint dropped below 10M > > for the whole parse. > > That was filelists.xml for development -- 36MB of XML, not > primary.xml. There was a 2.5 times speed improvement with cElementTree > code, compared to old yum code, using libxml2 -- around 20 seconds for > libxml2, and around 7-8 seconds for cElementTree on an AMD Athlon > 2600+. it's not libxml2. you will see that libxml2 by itself on such a box should parse your 36MB of XML in a mere second. Try with xmllint --stream for example. The potential for improvements by solving the python string import would have been huge if you did want to look at it, but it seems obvious you didn't want. Daniel -- Daniel Veillard | Red Hat Desktop team http://redhat.com/ veillard@xxxxxxxxxx | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
