On 13/08/2019 19:16, Dan Heinz wrote: > On 09/08/2019 14:33, Dan Heinz wrote: >>> I have a static library using OpenSSL (built as static library with the >>> no-pinshared parameter in the configuration) that is then included in a >>> DLL that gets loaded and unloaded many times by the calling application. >>> Now that the code is in 1.1.1c to allow me to manually shutdown the >>> OpenSSL library, I've run into an issue when loading and unloading my DLL >>> multiple times from the calling application. I am testing this on >>> Windows 10. A summary of what I am doing: >>> >>> 1. Calling executable calls LoadLibrary to load the DLL containing the >>> static library that uses OpenSSL. 2. Calling executable calls a function >>> of the now loaded DLL. 3. The DLL function calls a function in the static >>> library to initialize OpenSSL using: openssl_result = >>> OPENSSL_init_crypto(OPENSSL_INIT_NO_ATEXIT, NULL); 4. The DLL function >>> calls a function in the static library to decrypt some data using the >>> RSA_private_decrypt OpenSSL API. 5. The DLL function calls a function in >>> the static library to shutdown OpenSSL using: OPENSSL_cleanup(); >>> (required) and CRYPTO_cleanup_all_ex_data(); (not sure if this is >>> needed). 6. The calling executable uses FreeLibrary to unload the DLL. 7. >>> The calling executable goes to step 1 to repeat the process. >>> >>> Iterating through the above steps always fails on iteration 1077. OpenSLL >>> reports the error: error:4088003:rsa routines:RSA_setup_binding:BN lib >>> >>> >>> >>> After some debugging, I see that the failure happens in the function >>> CRYPTO_THREAD_init_local in threads_win.c. *key = TlsAlloc(); fails with >>> the error TLS_OUT_OF_INDEXES. >>> >>> Is something not being freed? Is there something additional I need to do >>> at shutdown? > >> For every call of CRYPTO_THREAD_init_local() while the library is running >> you should see a matching call to CRYPTO_THREAD_cleanup_local() to cleanup >> the index (as a result of an OPENSSL_cleanup() call). If you don't see that >> then an >index is leaking. > > > Thanks for the guidance, Matt. I added some code to output the counts of the > calls to TlsAlloc and TlsFree. I see 4 allocations and 4 matching > deallocations for each iteration of my tests. The allocations/deallocations > are for: > > destructor_key.value in OPENSSL_cleanup() private_drbg (from > CRYPTO_THREAD_init_local(&private_drbg, NULL)) public_drbg (from > CRYPTO_THREAD_init_local(&public_drbg, NULL)) err_thread_local > (fromCRYPTO_THREAD_init_local(&err_thread_local, NULL)) > > On the 1076th test iteration, I get the error (TLS_OUT_OF_INDEXES) from > TlsAlloc. > > I checked for any errors from TlsFree during the tests and it always returns > success. One thing I did notice is that the index value (from this line: *key > = TlsAlloc()) is always incremented by 1 after each test iteration. The > destructor_key.value always stayed the same number (7), but the other three > allocations always incremented the value of the key variable. That is quite strange behaviour. I can't think of a good reason why that particular index should always be the same while the others increment. I suppose the 4 deallocation calls that you see are deallocating the correct index? And that the TlsFree calls actually succeed? I'm not really a Windows programmer, so perhaps others who have more knowledge of Windows than I do can make some suggestions. Matt > By the time my > test reaches the 1076th iteration, the value of key is just under 2000. For > example the indices from TlsAlloc might be 10,11, and 12 and the next > iteration they are 11, 12, 13. MS documentation states "The maximum number > of indexes per process is 1,088" which seems to coincide with the number of > my test iterations. None of my code uses thread local storage. Any ideas? > > > Here's some output from my tests (note the key values keep incrementing). > Iteration: 1 *********************************************** Key value: 7 > TlsAlloc called. AllocationCount: 1 Key value: 9 TlsAlloc called. > AllocationCount: 2 Key value: 10 TlsAlloc called. AllocationCount: 3 Key > value: 11 TlsAlloc called. AllocationCount: 4 TlsFree called. > AllocationCount: 3 TlsFree called. AllocationCount: 2 TlsFree called. > AllocationCount: 1 TlsFree called. AllocationCount: 0 Iteration: 2 > *********************************************** Key value: 7 TlsAlloc called. > AllocationCount: 1 Key value: 10 TlsAlloc called. AllocationCount: 2 Key > value: 11 TlsAlloc called. AllocationCount: 3 Key value: 12 TlsAlloc called. > AllocationCount: 4 TlsFree called. AllocationCount: 3 TlsFree called. > AllocationCount: 2 TlsFree called. AllocationCount: 1 TlsFree called. > AllocationCount: 0 > > ..... > > Iteration: 1076 *********************************************** Key value: > 7 TlsAlloc called. AllocationCount: 1 Key value: 1086 TlsAlloc called. > AllocationCount: 2 Key value: 1087 TlsAlloc called. AllocationCount: 3 Key > value: 4294967295 (this is TLS_OUT_OF_INDEXES) > > >
