On Mon, Jun 04, 2018 at 10:51:00PM +0300, Ran Shalit wrote:
> Hello,
Hi,
> I currently use Linux without rt patch, yet we have some timing
> requirements, and we consider moving to RT linux.
>
> We tried to measure time in threads, in order to build a better rt application.
>
> I hope I can ask a question about one issue, which we are not sure about:
>
> We have a thread which does:
> ...
> <-- start time measurement
> {
> osteam::write
> ostream::flush
> .}
> <-- end time measurement
> ...
I am unsure, are you using writing to disk as a test for RT performance, or
are you testing the RT performance of disk IO?
> It seems that in general, the write to disk using these functions add
> time for the thread between entrace to main function of thread.
>
> But the time difference of these two commands behaves strangely: we
> see that in of the measurements it's 10msec, and only once in a while
> it gets up to 500msec.
10ms for a simple flush? Do you use rotational media?
The 500ms timeout can be a lot of things, extra data being flushed, once
you write to disk, you run the risk of writing data for other processes as
well. You also do a detour inside the kernel, interrupts can happen,
network softirqs, if you block on IO, then another thread can grab the CPU
delaying you even further etc.
That being said, 500ms is a pretty long delay.
> So, I am not sure why it only sometimes consumes more time then the average.
> Is it that there is some kernel thread responsible for disk which only
> sometimes consumes more time ?
>
> I would assume that the flush should always consume the time for write
> to disk, is it wrong to assume so ? we always write the same size, so
> why is the average different then the maximum ?
You cannot assume anything when you do blocking IO from a thread. The rule
of thumb is: don't write to disk, or do any other blocking syscalls form an
RT thread. Delegate to other helpers, allocate up front etc.
> Will it help to put it in another RT thread with lower priority ?
It will help doing it from /another/ thread yes. It will also help to
shield the core where you run your RT app, make sure no other interrupts
are delivered there etc.
A good starting point is to use the tracing infrastructure in the kernel to
measure delays. Instead of writing to disk, write to the trace_marker and
stop the trace when you hit a 500ms delay. Enable a few events (I like
syscalls, sched and irq as a start)
Have a look at Stevens intro to using ftrace for debugging:
https://lwn.net/Articles/365835/ (part 1)
https://lwn.net/Articles/366796/ (part 2)
Good luck!
--
Henrik Austad
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
