On Tue, Jun 17, 2003 at 06:50:03PM +0200, Robert Olsson wrote: > David S. Miller writes: > > > 60.0073 seconds passed, avg forwarding rate: 157557.710 pps > > > To be honest, this isn't half-bad for pure DoS load. > > No thats pretty good and profiles looks as expected. It would interesting > to get the singeflow performance as a comparison. I changed Juno to send from a single IP, but it only spat out about 330000 pps, which the dual Tigon3 Opteron box forwarded completely. In order to do a single flow forwarding test, I need to be able to create more input traffic somehow. Seeing as you wrote pktgen.c, maybe you could help in this department. :) > Also think Simon used only /32 routes... I took "real" Internet-routing > and made a script so it can be used for experiments. I can make it available. Yes, I found that area less interesting since Dave M. fixed the hash buckets. But yes, the prefix scanning will slow it down some. Whoa. Uhm. A lot. I should compare with 2.4 again to see what's going on here. 60.0042 seconds passed, avg forwarding rate: 50759.683 pps 60.0039 seconds passed, avg forwarding rate: 50311.258 pps 60.0046 seconds passed, avg forwarding rate: 50420.562 pps 60.0036 seconds passed, avg forwarding rate: 50399.389 pps 60.0038 seconds passed, avg forwarding rate: 50431.732 pps 60.0041 seconds passed, avg forwarding rate: 50403.777 pps 60.0036 seconds passed, avg forwarding rate: 50210.604 pps 60.0033 seconds passed, avg forwarding rate: 50279.220 pps 60.0036 seconds passed, avg forwarding rate: 50549.291 pps 60.0046 seconds passed, avg forwarding rate: 50437.615 pps Cpu type: Athlon Cpu speed was (MHz estimation) : 1394.26 Counter 0 counted RETIRED_INSNS events (Retired instructions (includes exceptions, interrupts, resyncs)) with a unit mask of 0x00 (No unit mask) count 697000 vma samples % symbol name c02bf730 16019 33.2014 fn_hash_lookup c0292b70 3882 8.04593 ip_route_input_slow c0221710 2335 4.83958 tg3_rx c02bd550 2004 4.15354 fib_validate_source c0290d70 1955 4.05198 rt_hash_code c0294e50 1670 3.46128 ip_rcv c02933a0 1404 2.90997 ip_route_input c01351b0 1349 2.79597 __kmalloc c02885c0 1314 2.72343 netif_receive_skb c02b8040 1168 2.42083 inet_select_addr c0135270 1123 2.32756 kfree c0284620 987 2.04568 alloc_skb c028ec90 900 1.86536 eth_type_trans c0135170 860 1.78246 kmem_cache_alloc c02be8e0 844 1.7493 fib_semantic_match c0135230 812 1.68297 kmem_cache_free c0222330 652 1.35135 tg3_start_xmit c02916b0 648 1.34306 rt_intern_hash c02215c0 542 1.12336 tg3_recycle_rx c010fc40 459 0.951335 do_gettimeofday c028f220 422 0.874648 pfifo_fast_dequeue c02be9b0 419 0.86843 __fib_res_prefsrc c028eb20 417 0.864285 eth_header c0295df0 386 0.800033 ip_forward c028c520 363 0.752363 neigh_resolve_output c0284860 345 0.715056 __kfree_skb c0284840 311 0.644586 kfree_skbmem c02847a0 295 0.611424 skb_release_data c028b530 285 0.590698 neigh_lookup c01adc80 276 0.572044 memcpy c0134ff0 269 0.557536 free_block c0291460 240 0.49743 rt_garbage_collect c0134e20 236 0.489139 cache_alloc_refill c02972d0 216 0.447687 ip_finish_output c0114030 215 0.445614 get_offset_tsc c0128a00 197 0.408307 call_rcu c028ae20 193 0.400017 dst_alloc c0288080 187 0.387581 dev_queue_xmit c028af50 184 0.381363 dst_destroy c028f1a0 175 0.362709 pfifo_fast_enqueue c011f080 170 0.352346 local_bh_enable c0221350 168 0.348201 tg3_tx c0221e90 160 0.33162 tg3_set_txd c0297570 152 0.315039 ip_output c028c3a0 149 0.308821 neigh_hh_init c028eeb0 141 0.29224 qdisc_restart size IN: hit tot mc no_rt bcast madst masrc OUT: hit tot mc GC: tot ignored goal_miss ovrf 17929 13 214343 0 0 0 0 163822 0 0 0 50521 50519 0 0 18296 18 213694 0 0 0 0 163018 0 0 0 50676 50674 1 0 17616 11 214529 0 0 0 0 163993 0 0 0 50536 50534 0 0 17841 12 213816 0 0 0 0 163157 0 0 0 50659 50657 1 0 18272 7 214093 0 0 0 0 163583 0 0 0 50510 50508 1 0 18216 9 214843 0 0 0 0 164214 0 0 0 50629 50627 0 0 18318 16 214976 0 0 0 0 164299 0 0 0 50677 50675 0 0 18099 9 213447 0 0 0 0 162995 0 0 0 50452 50450 1 0 17610 14 216438 0 0 0 0 165408 0 0 0 51030 51028 1 0 17643 14 214638 0 0 0 0 163987 0 0 0 50651 50649 0 0 17516 7 213185 0 0 0 0 163016 0 0 0 50169 50167 1 0 18355 10 213894 0 0 0 0 163564 0 0 0 50330 50328 1 0 17723 11 214477 0 0 0 0 163705 0 0 0 50772 50770 0 0 17915 6 214342 0 0 0 0 163625 0 0 0 50717 50715 0 0 18166 19 213965 0 0 0 0 163521 0 0 0 50444 50442 0 0 17943 19 213417 0 0 0 0 162955 0 0 0 50462 50460 2 0 17515 5 214423 0 0 0 0 163718 0 0 0 50705 50703 0 0 18231 10 213434 0 0 0 0 162919 0 0 0 50515 50513 1 0 17523 8 213856 0 0 0 0 163385 0 0 0 50471 50469 0 0 18217 16 214940 0 0 0 0 164165 0 0 0 50775 50773 0 0 ...recompiling with fibstats... Erm. I can't get fib_stats2.pat to apply against 2.5.71, 2.5.71+davem's join-two-diffs patch, 2.4.21-rc7, or 2.5.71+davem's rtcache changes. What's it supposed to be against? [sroot@debinst:/d/linux-2.5]# patch -p0 --dry < ../fib_stats2.pat patching file include/net/ip_fib.h Hunk #1 succeeded at 139 (offset 4 lines). patching file net/ipv4/fib_hash.c Hunk #3 succeeded at 305 (offset -11 lines). Hunk #4 succeeded at 1110 with fuzz 1 (offset -14 lines). Hunk #5 succeeded at 1166 (offset -14 lines). patching file net/ipv4/route.c Hunk #1 FAILED at 2754. Hunk #2 succeeded at 2760 (offset -6 lines). Hunk #3 succeeded at 2783 (offset -6 lines). Hunk #4 FAILED at 2793. 2 out of 4 hunks FAILED -- saving rejects to file net/ipv4/route.c.rej In any event, here is the profile of the single flow case with the full routing table (probably identical to the empty routing table case). The sender is pushing enough for NAPI to kick in, so there is a lot of tg3 overhead that would be with more traffic: 60.0041 seconds passed, avg forwarding rate: 329808.310 pps Cpu type: Athlon Cpu speed was (MHz estimation) : 1394.26 Counter 0 counted RETIRED_INSNS events (Retired instructions (includes exceptions, interrupts, resyncs)) with a unit mask of 0x00 (No unit mask) count 697000 vma samples % symbol name c0222330 4470 8.51445 tg3_start_xmit c0221710 3760 7.16204 tg3_rx c0294e50 3142 5.98488 ip_rcv c02885c0 2428 4.62485 netif_receive_skb c0295df0 2065 3.93341 ip_forward c028f220 2058 3.92007 pfifo_fast_dequeue c02933a0 2033 3.87245 ip_route_input c01351b0 1987 3.78483 __kmalloc c0290d70 1904 3.62674 rt_hash_code c02972d0 1752 3.33721 ip_finish_output c01adc80 1649 3.14101 memcpy c0134ff0 1626 3.0972 free_block c0284620 1511 2.87815 alloc_skb c0135270 1489 2.83624 kfree c0288080 1461 2.78291 dev_queue_xmit c028ec90 1351 2.57338 eth_type_trans c0135170 1319 2.51243 kmem_cache_alloc c028f1a0 1243 2.36766 pfifo_fast_enqueue c0134e20 1172 2.23242 cache_alloc_refill c0297570 1145 2.18099 ip_output c0135230 1133 2.15814 kmem_cache_free c0221350 1085 2.06671 tg3_tx c0221a50 991 1.88766 tg3_poll c02215c0 893 1.70098 tg3_recycle_rx c0221e90 832 1.58479 tg3_set_txd c0221b60 812 1.5467 tg3_interrupt c028eeb0 755 1.43812 qdisc_restart c010fc40 672 1.28002 do_gettimeofday c011f080 578 1.10097 local_bh_enable c0284840 492 0.937161 kfree_skbmem c010a8b2 426 0.811444 restore_all c02847a0 375 0.714299 skb_release_data c010c6a0 327 0.622869 handle_IRQ_event c010c910 288 0.548582 do_IRQ c0284860 284 0.540963 __kfree_skb c0114030 283 0.539058 get_offset_tsc c021f4b0 270 0.514296 tg3_enable_ints c0115e10 234 0.445723 end_level_ioapic_irq c011f4a0 220 0.419056 cpu_raise_softirq c028eb20 199 0.379055 eth_header c0288500 188 0.358102 net_tx_action c028c520 187 0.356197 neigh_resolve_output c0134b40 179 0.340959 cache_init_objs c0288900 171 0.32572 net_rx_action c0132290 165 0.314292 buffered_rmqueue c01321b0 122 0.232385 free_hot_cold_page If I start two threads on the sender (Xeon w/HT), I'm able to push 420000 pps, which only partially starts to use NAPI on the Opteron box. Going to try 2.4 again for a comparison (note: 2.5 seems to have an opposite PCI scan order from 2.4 for the dual Tigon3s). Simon- [ Simon Kirby ][ Network Operations ] [ sim@netnation.com ][ NetNation Communications Inc. ] [ Opinions expressed are not necessarily those of my employer. ] - : send the line "unsubscribe linux-net" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
