On Wed, Feb 24 2016, Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx> wrote: > > Typically, a library or application will keep the cpu number > cache in a thread-local storage variable, or other memory > areas belonging to each thread. It is recommended to perform > a volatile read of the cpu number cache to prevent the com‐ > piler from doing load tearing. An alternative approach is to > read the cpu number cache from inline assembly in a single > instruction. > > Each thread is responsible for registering its own cpu number > cache. Only one cpu cache address can be registered per > thread. > > The symbol __getcpu_cache_tls is recommended to be used > across libraries and applications wishing to register a > thread-local getcpu_cache. The attribute "weak" is recom‐ > mended when declaring this variable in libraries. Applica‐ > tions can choose to define their own version of this symbol > without the weak attribute as a performance improvement. > > In a typical usage scenario, the thread registering the cpu > number cache will be performing reads from that cache. It is > however also allowed to read the cpu number cache from other > threads. The cpu number cache updates performed by the kernel > provide single-copy atomicity semantics, which guarantee that > other threads performing single-copy atomic reads of the cpu > number cache will always observe a consistent value. > > Memory registered as cpu number cache should never be deallo‐ > cated before the thread which registered it exits: specifi‐ > cally, it should not be freed, and the library containing the > registered thread-local storage should not be dlclose'd. Maybe spell out the consequence if this is violated - since the SIGSEGV only happens on migration, it may take a while to strike. Random thoughts: The current implementation ensures that getcpu_cache is "idempotent" from within a single thread - once set, it can never get unset nor set to some other pointer. I think that can be useful, since it means a library can reliably use the TLS variable itself (initialized with some negative number) as an indicator of whether getcpu_cache(GETCPU_CACHE_SET) has been called. So if a single test on a fast path where the library would need to load __getcpu_cache_tls anyway is acceptable, it can avoid requiring some library init function to be called in each thread - which can sometimes be hard to arrange. Is this something we want to guarantee - that is, will we never implement GETCPU_CACHE_UNSET or a "force" flag to _SET? Either way, I think we should spend a few words on it to avoid the current behaviour becoming accidental ABI. In another thread: > However, there are other use-cases for having a fast mechanism for > reading the current CPU number, besides restartable sequences. For > instance, it can be used by glibc to implement a faster sched_getcpu. Will glibc do that? It may be a little contentious for glibc to claim a unique resource such as task_struct::cpu_cache for itself, even if everybody is supposed to use the same symbol. Hm, maybe one could say that if an application does define the symbol __getcpu_cache_tls (which is techically in the implementation namespace), that gives glibc (and any other library) license to do getcpu_cache(SET, &&__getcpu_cache_tls) (pseudo-code, of course). If a library initializes its own weak version with -2 it can check whether the application defined __getcpu_cache_tls. Ok, I'm probably overthinking this... Rasmus -- To unsubscribe from this list: send the line "unsubscribe linux-api" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
