On Fri, May 13, 2022 at 4:26 PM David Laight <David.Laight@xxxxxxxxxx> wrote: > > From: Alexander Potapenko > > Sent: 13 May 2022 13:26 > > > > On Fri, May 13, 2022 at 1:28 PM David Laight <David.Laight@xxxxxxxxxx> wrote: > > > > > > ... > > > > Once we have the possibility to store tags in the pointers, we don't > > > > need redzones for heap/stack objects anymore, which saves quite a bit > > > > of memory. > > > > > > You still need redzones. > > > The high bits are ignored for actual memory accesses. > > > > > > To do otherwise you'd need the high bits to be in the PTE, > > > copied to the TLB and finally get into the cache tag. > > > > > > Then you'd have to use the correct tags for each page. > > > > Sorry, I don't understand how this is relevant to HWASan in the userspace. > > Like in ASan, we have a custom allocator that assigns tags to heap > > objects. The assigned tag is stored in both the shadow memory for the > > object and the pointer returned by the allocator. > > Instrumentation inserted by the compiler checks the pointer before > > every memory access and ensures that its tag matches the tag of the > > object in the shadow memory. > > Doesn't that add so much overhead that the system runs like a sick pig? > I don't see any point adding overhead to a generic kernel to support > such operation. Let me ensure we are on the same page here. Right now we are talking about LAM support for userspace addresses. At this point nobody is going to add instrumentation to a generic kernel - just a prctl (let aside how exactly it works) that makes the CPU ignore certain address bits in a particular userspace process. The whole system is not supposed to run slower because of that - even if one or many processes choose to enable LAM. Now let's consider ASan (https://clang.llvm.org/docs/AddressSanitizer.html). It is a powerful detector of memory corruptions in the userspace, but it comes with a cost: - compiler instrumentation bloats the code (by roughly 50%) and slows down the execution (by up to 2x); - redzones around stack and heap objects, memory quarantine and shadow memory increase the memory consumption (by up to 4x). In short, for each 8 bytes of app memory ASan stores one byte of the metadata (shadow memory) indicating the addressability of those 8 bytes. It then uses compiler instrumentation to verify that every memory access in the program accesses only addressable memory. ASan is widely used for testing and to some extent can be used in production, but for big server-side apps the RAM overhead becomes critical. This is where HWASan (https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html) comes to the rescue. Instead of storing addressability info in the shadow memory, it stores a 1-byte tag for every 16 bytes of app memory (see https://arxiv.org/pdf/1802.09517.pdf for other options). As I mentioned before, the custom userspace memory allocator assigns the tags to memory chunks and returns tagged pointers. Like ASan, HWASan uses compiler instrumentation to verify that every memory access is touching valid memory, but in this case it must ensure that the pointer tag matches the tag stored in the shadow memory. Because of instrumentation, HWASan still has comparable code size and execution overheads, but it uses way less memory (10-35% of the original app memory consumption). This lets us test beefy applications, e.g. feeding real-world queries to production services. Even smaller applications benefit from it, e.g. because of reduced cache pressure. HWASan has been available for Android for a while now, and proved itself useful. > > A tag mismatch is reported as an out-of-bounds or a use-after-free, > > depending on whether the accessed memory is still considered > > allocated. > > Because objects with different tags follow each other, there is no > > need to add extra redzones to the objects to detect buffer overflows. > > (We might need to increase the object alignment though, but that's a > > different story). > > How does all that help if a system call (eg read()) is given > an invalid length. Neither ASan nor HWASan care about what happens in the kernel. Instead, they wrap system calls (along with some important libc functions) and check their arguments to ensure there are no buffer overflows. HTH, Alex -- Alexander Potapenko Software Engineer Google Germany GmbH Erika-Mann-Straße, 33 80636 München Geschäftsführer: Paul Manicle, Liana Sebastian Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Diese E-Mail ist vertraulich. Falls Sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde. This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.
