2011/8/8 Avi Kivity <avi@xxxxxxxxxx>: > On 08/08/2011 03:49 PM, Frediano Ziglio wrote: >> >> 2011/8/8 Avi Kivity<avi@xxxxxxxxxx>: >> > In certain circumstances, posix-aio-compat can incur a lot of latency: >> > - threads are created by vcpu threads, so if vcpu affinity is set, >> > aio threads inherit vcpu affinity. This can cause many aio threads >> > to compete for one cpu. >> > - we can create up to max_threads (64) aio threads in one go; since a >> > pthread_create can take around 30μs, we have up to 2ms of cpu time >> > under a global lock. >> > >> > Fix by: >> > - moving thread creation to the main thread, so we inherit the main >> > thread's affinity instead of the vcpu thread's affinity. >> > - if a thread is currently being created, and we need to create yet >> > another thread, let thread being born create the new thread, >> > reducing >> > the amount of time we spend under the main thread. >> > - drop the local lock while creating a thread (we may still hold the >> > global mutex, though) >> > >> > Note this doesn't eliminate latency completely; scheduler artifacts or >> > lack of host cpu resources can still cause it. We may want >> > pre-allocated >> > threads when this cannot be tolerated. >> > >> > Thanks to Uli Obergfell of Red Hat for his excellent analysis and >> > suggestions. >> > >> > Signed-off-by: Avi Kivity<avi@xxxxxxxxxx> >> >> Why not calling pthread_attr_setaffinity_np (where available) before >> thread creation or shed_setaffinity at thread start instead of telling >> another thread to create a thread for us just to get affinity cleared? >> > > The entire qemu process may be affined to a subset of the host cpus; we > don't want to break that. > > For example: > > taskset 0xf0 qemu .... > (qemu) info cpus > <pin individual vcpu threads to host cpus> > > Just call sched_getaffinity at program start, save to a global variable and then set this affinity for io threads. I didn't use affinity that much but from manual it seems that if you own process you can set affinity as you like. IMHO this patch introduce a delay in io thread creation due to posting thread creation to another thread just to set different affinity. Frediano -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
