On Sun, Feb 17, 2019 at 04:07:46PM +0100, Adrian Reber wrote: > The latest release of MirrorManager2 is still in updates-testing. I did > not want to push this version to updates-released because this is the > first Python3 based released. > > Could someone test the version from updates-testing? Update the > mirrorlist container and test it on one of the proxies? I tested it locally and it does not work. For someone knowing Python better than me it is probably obvious, but the mirrorlist server using Python 3 cannot read the pickle from the MirrorManager backend running Python 2. To be able to use a Python 2 based backend and Python 3 based mirrorlist I added an additional export format using Protobuf. The good news about this is that the file containing all the information for the mirrorlist server using protbuf is smaller than the pickle file. The protbuf is about 2/3 of the pickle file. The bad thing about this change is that the memory required by the mirrorlist server is 5 times higher using the protbuf file than using the pickle file. It is amazing how reading a 32MB file results in a process requiring 2.5GB of memory. From the memory consumption point of view this change seems like a bad idea. Being independent of the pickle format seems to be a good idea as it would be possible to have Python2 and Python3 parts in our MirrorManager deployment. It would also open up the possibility to rewrite the mirrorlist server in something else which does not require to understand Python's pickle format. This is the pull request with all the changes: https://github.com/fedora-infra/mirrormanager2/pull/266 Any recommendations if this is useful or how to convince Python to use less memory. Adrian
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
