Mark Hounschell wrote: > On 03/04/2010 09:04 AM, Luis Claudio R. Goncalves wrote: >> On Thu, Mar 04, 2010 at 01:29:53PM +0100, M. Koehrer wrote: >> | Hi all! >> | >> | I am running the RT_PREEMPT kernel 2.6.31.2-rt13 on a Intel Quad Core CPU. >> | I start my kernel with isolcpus=1-7 option to force all processes to run on CPU core 0 only. >> | Now we have the need for a very fast loop. Within this loop some accesses from/to a PCIe I/O board >> | (mapped in user space) and some additional computation has to be done. >> | For this, I use a real time thread, run this on CPU core 3 and let it run in an endless polling loop. >> | So far everything works fine. >> | This thread is the only running user mode thread on CPU core 3. >> | However, we measure some run time jitters when accessing the I/O board in the range of up to >> | 15 microseconds which are not tolerable by the application. >> | I see that on all cores the "Local Timer Interrupt" occurs 100 times a >> | second (of course this is the timer frequency select in the kernel configuration). >> >> Are CPU0 and CPU3 on the same socket? Are you using a SMP or a NUMA box? I >> would suggest running your application in a CPU on a different socket just >> to ensure you are not having any cache issues. >> >> Have you tried running hwlat_detect or smi-test? Your 15us threshold is pretty >> tight and could easilly be affected by SMI spikes (if present in your >> system). >> >> Luis >> >> | My question is now: >> | Is it possible to disable the "Local Timer Interrupt" for core 3 as it is actually not required here? >> | I want to use the full 100% CPU core time for this single loop. >> | >> | Any help or ideas are welcome! >> | > > This has been a long standing issue with me too. Moving your process to > another socket won't help you. It is not a cache issue. It is the local > timer interrupt just as you suspect. I've played with disabling it on a > core but haven't been successful. This is a problem with both the vanilla > and RT kernels. No matter what you do as far as isolation of tasks and > normal interrupts, the local timer interrupt kills ya. The kernel is broken > in this regard, by design. Your processors aren't yours. The kernel > developers insist on claiming a piece of every one of them for their code. > The kernel people will never change/fix this flaw in it's basic design > because only a few (hard real-time) consider it a problem. Those people are > told to use something else and that Linux wasn't designed for that kind of > thing. Never say never. Given a safe and not too intrusive design, I bet this could become mainline. > > Unfortunately, the instructions (rdmsr and wrmsr) that could be used to > disable/re-enable the local timer interrupt require DOM-0 privileges and > can't be executed from user land. If it were not for that one little thing > a solution would be easy. You wouldn't even need the RT patch set any more. Right, with perfect isolation, you could run user-space-only RT on a PREEMPT_NONE kernel. > > You could probably hack the kernel up such that you could get DOM-0 > privileges in user land but don't expect any help from any kernel people > for that sort of thing. This would kill your box within very short time. Other CPUs will once in a while want to talk to your isolated CPU that then spins with IRQs disabled. So it doesn't react, and the whole system locks up hard. > > Your best bet will be to attempt to use a high speed video cards GPU for > your process. There are methods available (out side of the kernel) for > this. I haven't got there yet but I think NVIDIA's CUDA may be an answer. > > > http://en.wikipedia.org/wiki/CUDA Careful: We currently have bug report open with those guys as the nvidia driver issues RT-killing cash flushes once in a while (wbinvd on all CPUs...). Definitely on certain memory allocations which you may avoid, but it's yet unclear if these are all. Well, binary-only $your_preferred_term, you know... Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux -- To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
