specially interesting to see that pyrpm is alot faster than the yum for fc7 and the yum used in PRE fc7..
yum (pre fc7) vs yum fc7 doesn't realy seem to be much of a speed improvement.. some parts are faster but some parts are slower.
funny to see how those 3 versions differ in speed
i still thinks that a pure c or c++ implementation in combination with mysql will be the fastest.. downside of that is that python and the repository data itself will have to be rebuilded.. and RPM itself will most likely have to get some vital updates.. like bringing in logic in versioning. o well.. that will take time till it gets done i guess.
2007/5/15, Florian Festi <ffesti@xxxxxxxxxx>:
Hi!
Yum traditionally used the rpmlib to resolve the dependecies of the packages
being installed/removed/updated. This has always restricted yum to very
simple algorithms as the interface of the rpmlib doesn't allow much control.
The current yum - that will be shipped with F7 - has now replaced the rpmlib
by a Python only resolver. I did some test runs to find out the current
state of development and to see if the new resolver is an improvement over
the rpmlib.
The candidates:
===============
* yum-3.0.3 as used in FC6
* yum from CVS head (2007/05/07), which is very similar to the yum in F7
* pyrpmyum CVS head (2007/05/07) - a pure Python implementation developed
for testing purposed [1].
The test cases:
===============
Tests are based in FC-5 run on FC-6.x86_64. All operations are aborted by
"pressing n".
Installs:
Try to install into an empty build root. There is a big number of excluded
pkgs to make the install '*' work.
* Normal install: several groups, 856 Pkgs
* Full install: "*", 6057 Pkgs
* Install "[a-k]*", 3052 Pkgs - needs to drag in a lot of
pkgs to satisfy dependencies
Updates:
Before the Updates the "Normal Install" is run and the Pkgs are installed
into the build root.
* Empty update
* Update libgnome: 1 Pkg
* Big update: 346u+12i Pkgs
* Dist upgrade: Enable FC6 repos and do an upgrade; fails for both yum
versions,so don't take results too serious
Erase:
* glibc: 828 Pkgs
Warm ups:
Do a install '*'/upgrade to build the caches. Each program has different
issues with that. yum-3.0.3 needs to download the rpm headers (Test was run
with repos in a NFS mount). All need to build the sqlite databases. pyrpmyum
currently uses a Python only implementation that uses a lot of time. Warm up
operations fail for all programs.
This all make the results of the warm ups a bit unreliable and questionable.
They are included for completeness.
Measured Values:
================
All data is from just one test run - so don't expect too high precision. I
put 3 different test results into tables:
* Real time passed during the test
* Heap peak - maximum of Memory used
* Heap total - amount of heap memory allocated
The last one gives an idea of how much data has been moved around.
Conclusions:
============
Please read the data tables first to get your own impression.
Memory usage:
The new yum saves a lot of memory for big operations (about 50%). The
comparison to pyrpmyum shows that it is possible to save even more memory
and to drop memory usage below 100MB for all cases. In pyrpmyum this is
achieved with dynamic loading of rpm tags. Some are not even kept in the
pkgs but just cached in a least recently used list.
For really small updates the old 3.0.3 yum wins. I guess this is because it
does not do any obsolete handling.
Runtime:
Although the new yum save the very costly rpm header downloads (which take
nearly 3 minutes for the whole repository) it is still significantly slower
for big operations. Things may look a bit different in real world scenarios
where downloading the headers is even more expensive.
Nevertheless the performance of the current yum is poor. Runtime behavior
has dropped from O(#prco * #rpmlib_calls) to O(#prco^2) (prco = Provides,
Requires, Conflicts, Obsoletes). The results of pyrpmyum show that is it
possible to do the depsolving in O(#prco) (Probably just O(#prco * log
#prco) but that doesn't matter much).
There are already plans how to get rid of the quadratic behavior and I hope
we can fix these issues before Fedora 8.
Have fun
Florian Festi
[1] https://hosted.fedoraproject.org/projects/pyrpm
Heap peak (MB)
Operation | 3.0.3 | 05/07 | pyrpm
------------------------------------------------
Warmup: Full install | 475.3 | 226.6 | 128.6
Resolve normal install | 94.6 | 74.1 | 27.0
Resolve full install | 462.4 | 218.5 | 59.8
Resolve install [a-k]* | 278.5 | 137.8 | 52.0
Resolve empty update | 25.9 | 52.9 | 45.2
Resolve update libgnome | 30.3 | 57.0 | 48.8
Resolve big update | 112.5 | 108.2 | 60.3
Resolve erase glibc | 210.0 | 99.8 | 39.6
Warmup: Dist upgrade | 162.7 | 107.3 | 112.0
Resolve dist upgrade | 162.4 | 107.3 | 62.9
Heap total (MB)
This is the total amount of memory requested.
The amount of memory being allocated at one time
is much less!
Operation | 3.0.3 | 05/07 | pyrpm
-----------------------------------------------
Warmup: Full install | 20003 | 90859 | 10829
Resolve normal install | 2088 | 9448 | 639
Resolve full install | 15402 | 84951 | 4987
Resolve install [a-k]* | 17542 | 24805 | 4121
Resolve empty update | 167 | 80 | 747
Resolve update libgnome | 282 | 136 | 1247
Resolve big update | 3497 | 2699 | 2574
Resolve erase glibc | 1732 | 2192 | 3369
Warmup: Dist upgrade | 6203 | 2644 | 7395
Resolve dist upgrade | 5869 | 2644 | 2154
Real time (s)
Operation | 3.0.3 | 07/05/07| pyrpm
----------------------------------------------------
Warmup: Full install | 653.51 | 1064.35 | 147.23
Resolve normal install | 40.59 | 134.11 | 8.04
Resolve full install | 477.67 | 1093.57 | 51.01
Resolve install [a-k]* | 313.56 | 887.40 | 33.52
Resolve empty update | 6.91 | 6.57 | 13.36
Resolve update libgnome | 3.34 | 2.57 | 7.51
Resolve big update | 33.04 | 45.56 | 24.81
Resolve erase glibc | 29.83 | 37.72 | 30.95
Warmup: Dist upgrade | 81.42 | 40.67 | 108.50
Resolve dist upgrade | 52.07 | 40.29 | 27.20
--
fedora-devel-list mailing list
fedora-devel-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/fedora-devel-list
-- fedora-devel-list mailing list fedora-devel-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/fedora-devel-list
