On Thu, 2009-08-27 at 10:51 -0600, Chris Friesen wrote:
> On 08/26/2009 03:37 PM, raz ben yehuda wrote:
> >
> > On Wed, 2009-08-26 at 15:15 -0600, Chris Friesen wrote:
>
> >> We gave it as close to a whole cpu as we could using cpu and irq
> >> affinity and we used message queues in shared memory to allow another
> >> cpu to handle I/O. In our case we still had kernel threads running on
> >> the app cpu, but if we'd had a straightforward way to avoid them we
> >> would have used it.
>
> > Chris. I offer myself to help anyone wishes to apply OFFSCHED.
>
> I just went and read the docs. One of the things I noticed is that it
> says that the offlined cpu cannot run userspace tasks. For our
> situation that's a showstopper, unfortunately.
Given that your entire software is T size , and T' is the amount of real
time size, what is the relation T'/T ?
If T'/T << 1 then dissect it, and put the T' in OFFSCHED.
My software T's is about 100MB while the real time section is about 60K.
They communicate through a simple ioctls.
CPU isolation example: a transmission engine.
In the image bellow, I am presenting 4 streaming engines, over 4 Intels
82598EB 10Gbps. A streaming engine is actually a Xeon E5420 2.5GHz.
Each engine has ***full control*** over its own interface. So you can:
1. fully control the processor's usage.
2. know **exactly*** how much each single packet transmission costs. for
example, in this case in processor 3 a single packet average
transmission is 1974tscs, which is ~700ns.
3. know how many packets fails to transmit right **on time** ( the Lates
counter) . and on time in this case means within the 122us jitter.
4. There are 8 cores in this machine. The rest 4 OS cores are ~95% idle.
The only resource these cores share is the bus.
State: kstreamer UP. Started at October 05 05:19:51
******************************************************
CPU 3,63% usage,Sessions 1499,6124301 kbps
CPU 5,77% usage,Sessions 1499,6123859 kbps
CPU 6,78% usage,Sessions 1498,6123709 kbps
CPU 7,73% usage,Sessions 1498,6117766 kbps
Summary: Throughput=24.489Gbps Sessions =5994
******************************************************
Streaming Processor 3
Tx Count : Tot=399990164 Good=399990164 Bad=0 ERR=0(LOC=0,FULL=0)
Time : GoodSendTsc( Max 1565895 Avg 1974) Lates=649
Flow Errors : Underflow (0,0) NotResched=0 GenErr=0
Sessions : Cur 1499(RTP=1499,UDP=0,MCAST=0,ENCRYPT=0) PAUSED=0
CPU : 63% usage
Queue : Max Size 92 Avg 69 Csc=79
Throughput (bps) : Tot 6271285040 MPEG 5988905440
Throughput (Mbps): Tot Mbps 6271 MPEG Mbps 5988
Throughput : Packets/sec 568855
Streaming Processor 5
Tx Count : Tot=399944597 Good=399944595 Bad=2 ERR=2(LOC=0,FULL=2)
Time : GoodSendTsc( Max 1566052 Avg 2464) Lates=5521
Flow Errors : Underflow (0,0) NotResched=0 GenErr=0
Sessions : Cur 1499(RTP=1499,UDP=0,MCAST=0,ENCRYPT=0) PAUSED=0
CPU : 77% usage
Queue : Max Size 95 Avg 69 Csc=79
Throughput (bps) : Tot 6270832416 MPEG 5988473792
Throughput (Mbps): Tot Mbps 6270 MPEG Mbps 5988
Throughput : Packets/sec 568814
Streaming Processor 6
Tx Count : Tot=399898586 Good=399898585 Bad=0 ERR=0(LOC=0,FULL=0)
Time : GoodSendTsc( Max 1661385 Avg 2474) Lates=8064
Flow Errors : Underflow (0,0) NotResched=0 GenErr=0
Sessions : Cur 1498(RTP=1498,UDP=0,MCAST=0,ENCRYPT=0) PAUSED=0
CPU : 78% usage
Queue : Max Size 91 Avg 69 Csc=87
Throughput (bps) : Tot 6270678560 MPEG 5988326400
Throughput (Mbps): Tot Mbps 6270 MPEG Mbps 5988
Throughput : Packets/sec 568800
Streaming Processor 7
Tx Count : Tot=399845166 Good=399845100 Bad=66
ERR=66(LOC=0,FULL=66)
Time : GoodSendTsc( Max 2962620 Avg 2377) Lates=42626
Flow Errors : Underflow (0,0) NotResched=0 GenErr=0
Sessions : Cur 1498(RTP=1498,UDP=0,MCAST=0,ENCRYPT=0) PAUSED=0
CPU : 73% usage
Queue : Max Size 94 Avg 69 Csc=66
Throughput (bps) : Tot 6264592672 MPEG 5982514944
Throughput (Mbps): Tot Mbps 6264 MPEG Mbps 5982
Throughput : Packets/sec 568248
--------------- Reservation Load Balancer ------------------
eth2 : 5994501 kbps
eth3 : 5994501 kbps
eth4 : 5990502 kbps
eth5 : 5990502 kbps
> Chris
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
