Re: lazy loading of filelists.xml to speed up dnf

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 08/06/2018 04:42 PM, Jeff Johnson wrote:
Whether dnf does what yum does misses the point:

Lazy downloading of file dependencies did not work then and does not work now.

Here's why:

The fundamental decision made by depsolvers is what packages are "newer" and need to be upgraded.

When _ANY_ commonly used package has a file dependency outside of the paths chosen to be delivered in primary.xml, then _ALL_ file dependencies need to be downloaded in order to find the package that contains the path so that the upgrade decision based on "newer" can be determined by the depsolver.
   [[snip]]

What about using pre-processed bloom filters to restrict downloading
to a smaller set of information that still covers the files and packages
that are relevant to each specific depsolve?  Hash each filename to a
bit position < 128, OR together the words for all files in a package.
16 bytes/pkg * 60,000 packages in Fedora is ~1MB.  Partition the
filelist.xml info into a couple hundred groups of a couple hundred packages each.
(Assignment of packages to groups can be improved using learning by experience.)
Download the relevant groups, or perform an online query of a database of
package ==> filelist.  It might be possible to reduce network traffic
by a factor 10: 5MB instead of 47MB.
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx/message/WAMMTGEBFA4HJONXWEJP35BBTH2OIX2F/




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux