On Tue, 21 Jul 2020 at 19:59, Dave Love <loveshack@xxxxxxxxxxxxxxxxx> wrote: > > Iñaki Ucar <iucar@xxxxxxxxxxxxxxxxx> writes: > > >> The licence seems to me to > >> rule it out a priori. > > > > The authors are going to add an exception, so the license won't be a > > problem. What problems do you think it produces? > > Obviously GPLv3 is incompatible with GPLv2, for instance, and typically > with other strong copyleft. The "Linked over a controlled interface" exception has been added in version 3.0.1. So GPLv3 incompatibilities don't apply to linking over the BLAS/LAPACK API anymore. > >> The proposal doesn't justify things, including its dismissal of the > >> simple, clean alternative in similar to Debian's, with which I have some > > > > I don't justify that because Debian's alternative has been repeatedly > > dismissed as an option for Fedora for a simple reason: one could end > > up with libblas pointing to one implementation and liblapack pointing > > to another. This problem, that Debian has, is solved with this > > library. > > You'll have to explain; I'd have thought the explanation should have > been in the proposal. I don't know what it means to have "another > implementation" of LAPACK in relation to BLIS for example. I explicitly > want to be able to substitute LAPACK and BLAS implementations, including > serial and OpenMP. So here you are exclusively talking about BLIS, because other libraries implement BLAS + some parts of LAPACK. I may agree that doing the latter is a bad idea given that the reference implementation offers two distinct libraries. But that's the landscape we have unfortunately. Being able to switch LAPACK independently while avoiding symbol collisions is an interesting feature that could be added in FlexiBLAS (at least I'm gonna propose it upstream). That's certainly not in the feature set right now, but it's not a possibility either without FlexiBLAS. E.g., many applications in Fedora are linked against OpenBLAS serial. How would you change BLAS and LAPACK in those? > >> There will be hoops to jump through to get packages to configure when > >> they don't know about the library. > > > > I don't think so, but if there are serious issues, rolling back the > > change is pretty straightforward. > > That sounds like insufficient experience! It's very easy to say "there will be hoops to jump through". That is insufficient experience. I don't have any evidence suggesting that. But I acknowledge that software is very heterogeneous out there and sometimes you find unexpected problems. Those problems cannot be uncovered unless someone tries to make this happen. Following the same logic, many (most?) Change Proposals have insufficient experience. > >> If I want to use a library that's > >> not included, I'm in the same position. > > > > No, you are not. You just need to point to that library using the > > FLEXIBLAS environment variable. > > I'm unclear on exactly how flexiblas currently works, but the above > seems to be saying that I must have a limited choice of BLAS and LAPACK > anyway. I can flip an environment variable already to subvert other > BLASs with BLIS. (I considered dynamically loading different > implementations when I tackled this initially.) Setting the environment > other than explicitly in a batch script isn't necessarily easy or robust > anyway, as opposed to ldconfig on sets of nodes. We have a more limited choice right now. See my comment above about applications linked against OpenBLAS, or ATLAS. With FlexiBLAS, there are less limitations and more flexibility (and I believe the specific use case you have in mind for BLIS could be discussed upstream). > >> It's not clear to me a priori > >> what happens if you try to use just BLIS even, given that OpenBLAS' > >> LAPACK implementation isn't the vanilla one, and depends on OpenBLAS as > >> far as I know. > > > > I don't understand this. You can switch to BLIS easily. The default > > one doesn't matter. > > See above about an independent choice of BLAS and LAPACK. What happens > if I want to use BLIS functions and link against a library that's using > OpenBLAS and don't know about whatever magic I need in the environment > to make it work? I'm not sure I'm following you anymore. You may also want to use 3 functions from OpenBLAS serial, 4 functions from OpenBLAS OpenMP, 1 from ATLAS, 5 from BLIS and the rest from whatever other library. But it doesn't scale. That's obviously a dramatization. My point is that it's almost impossible to cover all the very specialised HPC use cases out there. But I would argue that, if there's a way to cover them all, that could be achieved by adding features to FlexiBLAS, because it's the most general way to solve the issue of the implementation disparity in the BLAS/LAPACK landscape: just exposing a complete API and then internally rewiring those calls to the appropriate libraries given some configuration. > >> The choice of OpenBLAS isn't justified. It's not even > >> obvious that you want the same implementation for serial and > >> multi-threaded as OpenBLAS seems to have had continual problems with > >> threading which BLIS hasn't as far as I know. I expect OpenBLAS is the > >> best serial option on average, at least, but that needs data for > >> different architectures. > > > > This is about having a sane default one. That's it. You can switch > > then to whatever implementation you want. > > So what's the basis for choosing the default? > https://github.com/flame/blis/blob/80e6c10b72d50863b4b64d79f784df7befedfcd1/docs/Performance.md > for instance? An OpenMP default that performs serially if you set > OMP_NUM_THREADS and OMP_PROC_BIND doesn't seem a sane default. Again > this seems to be saying I can switch things at will, despite what it > says above, and that the mechanism is robust. We support a number of architectures in which BLIS doesn't perform well, so I think we would agree that this rules out BLIS as a default. Then, both @jussilehtola and the authors of FlexiBLAS independently agree that the OpenMP version of OpenBLAS would be the best default. For the record, I copy&paste below the author's assessment: | * OpenBLAS / Serial performance comparable to BLIS and MKL, require compilation with the USE_LOCKING=1 flag set to be safe in multithreaded applications. | * OpenBLAS-PThread, fast, flexible and good automatic threading, let OpenMP applications crash if BLAS is called from the inside of a OpenMP parallel region, preferred version on Debian/Ubuntu. Not usable on ppc64le architectures. | * OpenBLAS-OpenMP. around 2 % slower than the Pthread version but less complicated with multithreaded applications, works on all platforms, good automatic threading. Can get in trouble with StarPU parallel applications. but I do not know of any application outside research that uses this. | | So I prefer setting OpenBLAS-OpenMP as default on all systems I have under my control and colleagues do the same with their machines. > >> I realize no-one is going to be running Fedora on HPC systems, where > >> this really matters, but presumably it filters down to RHEL, which is > >> looking less and less like a good HPC platform to me. > > > > Actually, FlexiBLAS is developed by researchers in HPC. > > I know. That doesn't necessarily mean they have substantial experience > of HPC research support, system management, and packaging; that helps > when evaluating any necessary engineering trades-off. (If I listened to > the HPC mainstream on software provision, I wouldn't be a packager and > wouldn't have had a easily manageable cluster.) I'm waiting to be > convinced what I've concluded from R&D is invalid. You are arguing from a narrow niche. And it may be possible that I'm not following you correctly and it's my fault, but from your comments, my impression is that you didn't take the time to read how FlexiBLAS works (there are links in the change proposal; the documentation is extensive, and there are links in their home page to their papers, to a very informative presentation...). -- Iñaki Úcar _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
