On 8/1/2012 12:55 AM, Jason Newton wrote:Jason, have you considered something like this to solve your problems?
>> Just to figure out for sure what the bottlenecks are and whether they can
> be dealt with rather than looking at it as opaque system and assuming
> nothing can be done. Also as a learning experience.
RAM is cheap. Far cheaper than attacking this problem with any other
hardware type. And you can't easily solve it by rewriting to use AIO,
given the effort involved with that.
You should be able to fit 32GB of RAM on the board. Create a 24GB RAM
disk and use that for writing your 5.7MB frame files in real time. This
eliminates any latency and stutter issues during capture. Treat the RAM
disk as a FIFO, taking each new file and copying it out to SSD after
it's been closed, then delete the original. This gives you in essence a
very fast buffer. If my math is correct, 24,000MB / 300MB/s = roughly
80 seconds of buffer at a 300MB/s streaming capture rate, 40 seconds at
600MB/s.
The system has a single slot of SODIMM. We have an 8GB DDR3 stick in it. I've added a circular buffer for each frame and limit it to some number of frames (so far 2 seconds, I haven't had time to experiment with it yet). The serialization thread is now separate and consumes the circular buffer so we're effectively talking about the same thing sans files. This solves the problem but as I mentioned before... I do have a desire to seek out the sources of the latency and cpu usage... I'm not really sure of how to go about it though.
This should be very easy to implement, and cheaper than all other
alternatives. It should eliminate all possible latency issues, though
it will increase CPU cycles due to the data movement to/from the RAM
disk, though how much I can't guess at this point. 8GB RAM disk will
give you 26 seconds of buffering at 300MB/s, and a 4GB RAM disk will
give you 13 seconds of buffering. If 13 seconds is sufficient, you can
implement this on a machine with only 8GB RAM, assuming you need no more
than 4GB for kernel/user space/application.
Agreed that it's the easiest and cheapest solution. Average performance probably won't change but burst of cpu will as it compensates for the high latency writes in future cycles... this is undesirable but I think OK (the important stuff is at high priority on SCHED_RR, these serialization threads are high priority SCHED_OTHER). Again, I haven't had time to test it as I've been putting out other fires.
-Jason
-Jason
_______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
