On Wed, 2024-02-21 at 12:57 -0500, dalias@xxxxxxxx wrote: > > This feels like it's getting complicated and I fear it may be an > > uphill > > struggle to get such code merged, at least for arm64. My instinct > > is > > that it's going to be much more robust and generally tractable to > > let > > things run to some suitable synchronisation point and then disable > > there, but if we're going to do that then userspace can hopefully > > arrange to do the disabling itself through the standard disable > > interface anyway. Presumably it'll want to notice things being > > disabled > > at some point anyway? TBH that's been how all the prior proposals > > for > > process wide disable I've seen were done. > > If it's possible to disable per-thread rather than per-process, some > things are easier. Disabling on account of using alt stacks only > needs > to be done on the threads using those stacks. However, for dlopen > purposes you need a way to disable shadow stack for the whole > process. > Initially this is only needed for the thread that called dlopen, but > it needs to have propagated to any thread that synchronizes with > completion of the call to dlopen by the time that synchronization > occurs, and since that synchronization can happen in lots of > different > ways that are purely userspace (thanks to futexes being userspace in > the uncontended case), I don't see any way to make it work without > extremely invasive, high-cost checks. For glibc's use, we talked about a couple of options. 1. A mode to start suppressing the #UD's from the shadow stack instructions 2. A mode to start suppressing #CPs (the exception that happens when the shadow stack doesn't match). So the shadow stack instructions continue to operate normally, but if the shadow stack gets mismatched due to lack of support, the ret is emulated. It probably is safer (but still not perfect), but the performance penalty of emulating every RET after things get screwed up would be a significant down side. There also needs to be clean handling of shadow stack #PFs. 3. Per-thread locking that is used around all shadow stack operations that could be sensitive to disabling. This could be maybe exposed to apps in case they want to use shadow stack instructions manually. Then during dlopen() it waits until it can cleanly disable shadow stack for each thread. In each critical sections there are checks for whether shadow stack is still enabled. 3 is the cleanest and safest I think, and it was thought it might not need kernel help, due to a scheme Florian had to direct signals to specific threads. It's my preference at this point. 1 and 2 are POCed here, if you are interested: https://github.com/rpedgeco/linux/commits/shstk_suppress_rfc/ > > If folks on the kernel side are not going to be amenable to doing the > things that are easy for the kernel to make it work without breaking > compatibility with existing interfaces, but that are impossible or > near-impossible for userspace to do, this seems like a dead-end. And > I > suspect an operation to "disable shadow stack, but without making > threads still in SS-critical sections crash" is going to be > necessary.. I think we have to work through all the alternative before we can accuse the kernel of not being amenable. Is there something that you would like to see out of this conversation that is not happening?
