Dne 08. 11. 19 v 20:40 Fabio Valentini napsal(a): > On Fri, Nov 8, 2019 at 6:01 PM Charalampos Stratakis > <cstratak@xxxxxxxxxx> wrote: >> >> >> ----- Original Message ----- >>> From: "Vít Ondruch" <vondruch@xxxxxxxxxx> >>> Cc: devel@xxxxxxxxxxxxxxxxxxxxxxx >>> Sent: Friday, November 8, 2019 10:01:47 AM >>> Subject: Re: Fedora 32 System-Wide Change proposal: Build Python 3 to statically link with libpython3.8.a for better >>> performance >>> >>> >>> Dne 07. 11. 19 v 23:08 Florian Weimer napsal(a): >>>> * Vít Ondruch: >>>> >>>>> Dne 07. 11. 19 v 16:05 Tom Hughes napsal(a): >>>>>> On 07/11/2019 14:59, Victor Stinner wrote: >>>>>> >>>>>>> I cannot explain why PLT is needed when a libpython function calls a >>>>>>> libpython function. >>>>>> Because an exported symbol in an ELF shared library is subject to >>>>>> potential interposition using LD_PRELOAD so the calls need to go >>>>>> through the PLT to be resolved. >>>>> Not sure what PLT is (pre load table?), but is it something what could >>>>> be disabled? >>>> Procedure Linkage Table. >>>> >>>> It can be disabled by using hidden symbols. For internal symbols, that >>>> is easy. For symbols that are used externally, I do not think we have >>>> good toolchain support. Link-time optimization can detect truly >>>> internal symbols and make them hidden. Some targets can also perform >>>> relaxation of relocations, eliminating most of the PLT indirection >>>> overhead if it turns out a function is not exported after all and >>>> therefore cannot be interposed. But that needs a version script, and it >>>> can't work for calls to functions that are in fact public. >>>> >>>> In glibc, we create hidden aliases for public functions which should not >>>> be interposed. It's not too bad if you have preprocessor macros for >>>> this task, but you do need to annotate each function declaration and >>>> definition separately. >>>> >>>>> This sounds like the whole system could be 25% faster if we link >>>>> statically. >>>> Not reallly, quite a few system components already do this kind of >>>> optimization. >>>> >>>> Toolchain support for this is quite poor however. Ideally, we would >>>> have a compilation mode that reuses the annotations that Windows uses, >>>> but given that our system headers currently lack __dllimport specifiers >>>> (or whatever they are called), even with that approach, it's quite a lot >>>> of work. I might be mistaken about this, but I think there was a huge >>>> conflict about some intermediate visibility setting (protected?) that >>>> might help with this, basically creating non-interposable function >>>> symbols, but I don't think it's usable for that in its current state. >>> >>> Thx for explanation Florian. >>> >>> >>> Generally, I am against this change proposal, because: >>> >>> 1) Apparently, there is some work which needs to be done on the >>> toolchain. Applying workarounds just hides the issues and we won't move >>> forward ever. >> I think it's more reasonable to do a small SPEC change in Python to achieve a 27% performance boost, than wait for the toolchain to catch up on things that are not well defined yet. I don't see that as a valid reason for not accepting the change, although you might want to elaborate further here. >> >>> 2) I am asking this questions, because I believe that the same issue >>> might suffer Ruby and others are concerned about Perl. Applying this >>> just to Python is not systematic. >> Maybe. Systematic or not when compared to other dynamic languages is not really relevant for this change to take effect. I don't know about Perl's or Ruby's architecture design, but is there a reason to keep them in line in that aspect? Or for any other aspect at all, apart from the general packaging guidelines? >> >>> However, if part of this change proposal was actually collecting the >>> information what have to be done to have similar performance for the >>> dynamically linked libraries comparing to static linking, if there is >>> push to improve the toolchain and if there is generally better >>> understanding of the issue, then I would not mind if this is accepted as >>> temporary measure. Unfortunately, nothing like this is mentioned in the >>> change proposal. >>> >> No this is not intended to be temporal, hence why it's not mentioned as such. The information has been collected as a case for the change. If other languages would like to conduct similar benchmarks and experiments they are free to do so, but the scope of this change is just for Python. Also it's not intended as a push for certain toolchain changes/optimizations, although thet would be more than welcome. In addition this change is not a case for the benefits of static linking in general, as we are not aware or experimented on other things, we outlined a specific case, which is in line with the packaging guidelines, since statically linking a binary against its own library is permitted. >> >> This change intends to speed up Python by a significant margin in the context of the current toolchain, with the outlined effects on compatibility and package changes. We can debate on that, the rest seem irrelevant in the context of accepting it or not. > I agree. The benefit of this change for python (and hence, a big > amount of software that runs on both users' *and* fedora infra > machines) is rather big, while the impact on other packages is either > non-existent, or pretty small. Additionally, I think Miro mentioned > that this change was already done in other distros, so we're also not > the first ones doing this. Now it looks like I am strongly against this change proposal, but what I wanted is this: ~~~ However, if part of this change proposal was actually collecting the information what have to be done to have similar performance for the dynamically linked libraries comparing to static linking, if there is push to improve the toolchain and if there is generally better understanding of the issue, then I would not mind if this is accepted as temporary measure. Unfortunately, nothing like this is mentioned in the change proposal. ~~~ I don't see anything from these points being part of the change proposal. I don't see that Python team really understands the issue. If they do, they don't do good job explaining it. All I see is "others do this and it improves speed, so lets do it as well". It is also surprising that from all these points I got response just the "temporary" word, which is admittedly an easy target to pick upon if my arguments should be shot down. In this context, I don't mind if temporary is "years", but it should not be forgotten why the change was done and it should be reverted as soon as the toolchain improves. Vít > > (Side note: I wouldn't even have objected to this being a > Self-Contained Change, since it basically only affects one package - > albeit an important one (python3) - and the Change owners have offered > to help with adapting other packages, in the very few cases where that > would be necessary. I also trust the members of the Python SIG have > done / will do their homework here, as they have done with both the > python 3.8 transition and python2 retirement, which were / are both > being executed almost flawlessly.) > > Fabio > >>> Vít >>> >>> >>>> Thanks, >>>> Florian >>> _______________________________________________ >>> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx >>> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx >>> Fedora Code of Conduct: >>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >>> List Archives: >>> https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx >>> >> -- >> Regards, >> >> Charalampos Stratakis >> Software Engineer >> Python Maintenance Team, Red Hat >> _______________________________________________ >> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx >> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx >> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >> List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx > _______________________________________________ > devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
