On Wed, 27 Jun 2018 08:29:00 -0500 richard boaz via gtk-list <gtk-list@xxxxxxxxx> wrote: > hello, > > (not sure if this is a glib question, or a linux kernel question, any > helpful hints appreciated...) > > i have a server-side program that is responsible for managing hundreds > (increasing daily) of IoT connections, where this program listens for a > connection request and then spawns a separate thread to manage the incoming > real-time data feed using g_thread_try_new(). > > i have placed a 30 second timeout on the socket and when this is breached, > the socket is closed and the thread is exited using g_thread_exit(). > > however, with so many connections coming and going over all time, > eventually, spawning a new thread results in g_thread_try_new() failing > with the error: "Error creating thread: Resource temporarily unavailable". > i have upped the max threads per process to 95K, but this is just a > stop-gap measure that merely delays the onset of this problem. > > my question: is g_thread_exit() guaranteed to reduce the kernel resource > pool counter by 1? my experience seems to suggest that it does not, so my > next question is: how to avoid the problem (i.e., not receive this false > error) when the total number of threads existing at any given moment does > not exceed max threads per process? > > or is it the (unfortunate) reality that the max threads per program > represents the total number of threads over all time, exited or not? > > thanks in advance for any pointers to a solution... Are you calling g_thread_join() on the extinct thread to reclaim pthread and glib resources? If that is problematic you might consider using pthreads directly and detaching threads at the outset. On a different issue, one thread per connection is not scalable on most hardware. If your program is glib based, using GIO's asynchronous operations with a glib main loop is likely to give you much better results. GIO uses a dedicated thread pool where it needs to hand work off. This may require you to rewrite your application, and it may involve you dealing with "callback hell" (there is no easy way of implementing await semantics with coroutines if you are using glib), but it will probably prove worth it if you are otherwise trying to run nearly 100,000 threads. Kudos for even getting near that number of native threads on reasonable hardware. Chris _______________________________________________ gtk-list mailing list gtk-list@xxxxxxxxx https://mail.gnome.org/mailman/listinfo/gtk-list
