On Mon, 28 Sep 2020 12:31:59 +0200, Mark Wielaard wrote: > If you want to make -fdebug-types-sections the default you really > should work with the upstream GCC developers to figure out why they > don't want that. I haven't seen that, according to Richard Biener from GCC -fdebug-types-section is a normally supported GCC feature: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88878#c6 I am aware only of Jakub Jelinek who is against -fdebug-types-section. I should also state he is the author of DWZ. If GCC is unable to support such a trivial feature as -fdebug-types-section then Fedora should really already switch to the standard compiler. It will come sooner or later anyway. This deviation from standard tools just causes continuous troubles such as: [fesco] Issue #2020: Firefox is switching from gcc to clang/llvm https://pagure.io/fesco/issue/2020#comment-671672 > Trying to override upstream defaults in Fedora without understanding why > upstream decided on the current defaults isn't a good idea IMHO. You know very well Fedora already overrides upstream GCC defaults a lot: -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection And DWZ is even a project unrelated to GCC so calling for upstream defaults would really call for dropping DWZ. > I totally get that it is frustrating if you worked for a long time on a > new feature to support some DWARF constructs for lldb It is in no way a feature as it does not bring any user visible improvement. It is only a compatibility with marginally (0.1%) used file format with disputable benefits. > and your aren't able to get the patches in shape to be accepted upstream. That is a repeated lie I have never even suggested. LLDB is fine accepting my DWZ patches. I understand you are used to difficulty of upstreaming patches in GNU projects but that is not the case of LLVM. In fact LLDB is a completely different world in accepting patches than GDB where it was taking me even 10+ years to get some patches accepted. For GDB you need to learn first how to do the ancient ineffective and bug-prone coding practices, force yourself to really execute them to become a global maintainer and only then you manage to get patches checked in. You are repeatedly trying to make it look as the problem is upstreaming DWZ support. That is not any problem. The problem is the DWZ itself as it just isn't worth the effort of supporting it in all the consumers. As I am repeating again and again I just find DWZ too complicated for both production and consumption for so little gain (size reduction) it achieves. So before I complicate the LLDB codebase by the DWZ support and make it a catch-up game for Red Hat, Fedora and others forever (as Apple+Google is never going to use DWZ and they know why) I am trying by this Change to save a lot of time for everyone. The years of engineering time I have already spent on DWZ and the years of engineering time I will have to spend on its future maintenance and reworks (for clang DWARF) could be better spent on improving the debugging experience. We are no longer living in 80es where few saved bytes of size were critical whether the debugger will be able to run or not. Apparently GNU developers still haven't realized that change. > But I am > happy people now know about your patches and seem to find them useful. "useful" means with my patches they can workaround the Fedora problem of encumbering its debuginfo by DWZ. And this question is not about the existing patches. That waste of engineering time has been already done. It is about the future waste of time maintaining compatibility with the DWZ format almost nobody (0.1%) cares about. > You say it is difficult to support DWARF partial units as generated by > dwz in lldb, but dwz doesn't really do anything non-standard (and GCC > with LTO also generates partial units). "standard" means that the DWZ specification has been accepted into DWARF-5 standard. IMO that was a mistake. It may have happened because the DWZ author is a member of the DWARF standard committee. Apparently nobody has so far implemented any reasonable/effective consumer for DWARF-5 DWZ otherwise they would put some restrictions into the standard. To make DWZ better consumable it needs to have the partial units separately parseable. That way they can be shared at IR level and not just at DWARF level That means: * DW_TAG_partial_unit should have DW_AT_language. * DW_TAG_partial_unit must contain only types (struct/class). Currently they contain for example also static constant variables but when you parse such independent DW_TAG_partial_unit into which dictionary you will register such variable? That makes no sense. Currently DWZ has benefits only with DWZ common file. Without DWZ common files DWZ produces 1.6% files bigger than -fdebug-types-section. Therefore if the DWZ common files saving of 3.3% per Fedora distribution size is worth it then the existing DWZ tool should be dropped anyway as one can use normal -fdebug-types-section and one can just write a simple tool moving DW_UT_type units to the DWZ common file and converting DW_AT_signature declarations to DW_FORM_ref_sup4 and we are done. DWARF standard sometimes makes mistakes, for example .debug_pubnames and .debug_pubtypes were never really usable and DWARF-5 removed them. It may be perfectly possible the DWZ extension of DWARF-5 will be removed in DWARF-6 or future DWARF standards as it turns out it is not worth the format complexity. That some text has been accepted into the DWARF standard does not mean much. It is all about engineering effort. I agree if the support of DWZ was trivial (or there were unlimited engineering resources) then DWZ is really better than -fdebug-types-section (except it would need a DWZ tool with less bugs and better coding practices). But reality shows the DWZ support is not trivial engineering resources for Fedora are very limited and so we have to decide whether the serious effort to support DWZ is better spent on DWZ or on making the debuggers better/really usable. Given how much you propose DWZ apparently the 3.3% Fedora distribution size increase (if DWZ is dropped) is considered as too much. Obviously if the size increase was just let's say 0.1% it would be acceptable to drop DWZ - do you agree? So where is your size limit where the years-effort of supporting DWZ is worth it? I would say my limit would be maybe 20%, far above the 3.3%. For example during Fedora Package Review Process do some packages get rejected because they would make the distribution too large? Not worth of including such new package? I am not aware of such decision and it even sounds funny to me. But that is what you choose here by enforcing DWZ no matter how little savings it has. DWZ is a nice engineering idea and a nice engineering challenge. But it has no meaning for end-users. This is why I have switched from GDB to LLVM as Google&Apple are solving real end-user problems and not artificial engineering challenges just to have some nice coding time. But in the end I end up stuck on another GNU non-sense (this time DWZ) needing to be supported by LLVM in Fedora only for compatibility reasons. > If your complaint is that partial unit DIEs are missing some for your use > case essential attributes, No, my complaint is that DWZ is just not worth it. > I do find your statistics per package useful because they show dwz is > in general effective by producing at least 20% (more) on-disk size > reduction, even though there are some packages where dwz doesn't seem > as effective as it could be. We definitely should investigate those > issues. And does the 20% reduction of installed size when whole *-debuginfo.rpms get installed is really worth the delay of 3.5 years of DWARF-5, delay of 3.5 years of LLDB index (.debug_names), years of incompatible LLVM and years of wasted engineering time? I do not think so. Maybe Fedora/FESCo thinks differently and this is what I am asking by my Change about. > But I don't really understand why you then focus on the zstd compressed > rpms (even if even those favor dwz). I do not see how it favors dwz, I haven't seen and I haven't done a measurement of separate *.debug files compression of DWZ vs. -fdebug-types-section. My guess is there will not be a big difference for DWZ vs. -fdebug-types-section size ratio. > Or why for the debuginfod use case you seem to do the opposite, not take > into account that the http debuginfod server will compress the files before > sending over the network. See the paragraph above. -fdebug-types-section has better compression ratio than DWZ for *-debuginfo.rpm because for *.rpm the compression is applied for all its files together. The compression algorithm then finds the same parts of separate *.debug files similar to what DWZ does. > I don't think either of those later statistics are really relevant with > respect to your proposal. I am not sure what do you exactly mean. You can update the Wiki if you disagree with any numbers there. > Finally I am interested in your proposal to implement a different way > to reduce the size of DIE trees by eliminating "unused" DIEs. It is > hard to predict what effect that would have without seeing an > implementation (in theory GCC with LTO would not actually generate > debuginfo for unused functions). It is fastest to wait a few days for the LLVM presentation: https://whova.com/embedded/session/llvm_202010/1193947/ 2) Fragmenting the DWARF to Enable Dead Debug Data Elimination > But I think that can be done separate > from your proposal and combined with other size reduction techniques. By that LLVM dead DIES reduction talk I wanted to just show apparent nobody cares too much about few percents of the *-debuginfo.rpm size - otherwise it would be already coded/used. Or if there wasn't DWZ Fedora could already switch to DWARF-5 which saves probably more size than DWZ does. A year ago I was talking about DWZ incompatibility with DWARF-5. As nobody has done anything in the past year I was going to compare DWZ + DWARF-4 against -fdebug-types-section + DWARF-5 which would be an easy with for -fdebug-types-section + DWARF-5. Unfortunately when I started proposing to drop DWZ due to it you started to resurrect the DWZ tool by porting it to DWARF-5. I do not find that a good idea longterm. Jan _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
