Hello. I've been tracking down an issue and it has led me to a
question regarding fast retransmit behavior. I'd appreciate any input
offered.
Short question: When Linux is using CUBIC, after receiving 3 dup acks
should the network stack always immediately send the "missing" packet?
I believe it should per RFCs. However, I'm not seeing that behavior.
I'm seeing behavior where the "missing" packet is transmitted after a
delay (which keeps increasing). I have a packet trace pasted below
for reference.
The scenario is more or less an upload from a client to a server. The
high level summary is below.
1) TCP connection established
2) Data starts being exchanged
3) packet is dropped, not received by server
4) Client's timer expires and retransmits the packet (Frame #44)
5) Server ACKs this packet (Frame #45)
6) Client sends more data (Frames 46-48)
7) Server starts dup acking (Frames 49-51)
8) After these 3 dup acks, client does NOT retran immediately. Rather
it delays and then retransmits (Frame 52)
Because several packets were dropped, this dup ack behavior repeats for
subsequent packets. The client continues to not immediately retransmit
after the 3 dup acks and the delay keeps getting longer and longer.
Examples: Frame 60, 70, 84.
The raw data is below. Any comments on this behavior are greatly
appreciated.
No. Time Source Destination
Protocol Info
1 0.000000 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [SYN] Seq=0 Win=5808 Len=0 MSS=1452 TSV=13817777 TSER=0
WS=5
2 0.002346 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [SYN, ACK] Seq=0 Ack=1 Win=32768 Len=0 MSS=1380
3 0.007300 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=1 Ack=1 Win=5808 Len=0
4 0.373434 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=1 Ack=1 Win=5808 Len=72
5 0.375432 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=1 Ack=73 Win=32696 Len=0
6 0.377290 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [PSH, ACK] Seq=1 Ack=73 Win=32768 Len=1380
7 0.377299 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [PSH, ACK] Seq=1381 Ack=73 Win=32768 Len=1380
8 0.377411 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [PSH, ACK] Seq=2761 Ack=73 Win=32768 Len=1281
9 0.382224 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=73 Ack=1381 Win=8280 Len=0
10 0.382234 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=73 Ack=2761 Win=11040 Len=0
11 0.382239 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=73 Ack=4042 Win=13800 Len=0
12 0.491684 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=73 Ack=4042 Win=13800 Len=1380
13 0.491753 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=1453 Ack=4042 Win=13800 Len=1380
14 0.493982 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4042 Ack=2833 Win=30008 Len=0
15 0.499642 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=2833 Ack=4042 Win=13800 Len=1380
16 0.499738 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=4213 Ack=4042 Win=13800 Len=694
17 0.501883 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4042 Ack=4907 Win=27934 Len=0
18 0.565408 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=4907 Ack=4042 Win=13800 Len=329
19 0.579907 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [PSH, ACK] Seq=4042 Ack=5236 Win=32768 Len=6
20 0.579958 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [PSH, ACK] Seq=4048 Ack=5236 Win=32768 Len=45
21 0.585457 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=5236 Ack=4048 Win=13800 Len=0
22 0.586295 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=5236 Ack=4093 Win=13800 Len=0
23 0.658677 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [PSH, ACK] Seq=4093 Ack=5236 Win=32768 Len=45
24 0.664273 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=5236 Ack=4138 Win=13800 Len=0
25 *REF* 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=5236 Ack=4138 Win=13800 Len=90
26 0.037211 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [PSH, ACK] Seq=4138 Ack=5326 Win=32768 Len=77
27 0.043571 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=5326 Ack=4215 Win=13800 Len=114
28 0.051457 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=5440 Ack=4215 Win=13800 Len=1380
29 0.053659 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4215 Ack=6820 Win=31274 Len=0
30 0.056332 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=6820 Ack=4215 Win=13800 Len=1380
31 0.058451 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=8200 Ack=4215 Win=13800 Len=780
32 0.060625 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4215 Ack=8980 Win=29114 Len=0
33 0.065555 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=8980 Ack=4215 Win=13800 Len=944
34 0.070753 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=9924 Ack=4215 Win=13800 Len=1380
35 0.072941 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4215 Ack=11304 Win=26790 Len=0
36 0.076995 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=11304 Ack=4215 Win=13800 Len=1346
37 0.085623 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=12650 Ack=4215 Win=13800 Len=1380
38 0.087750 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4215 Ack=14030 Win=24064 Len=0
39 0.089287 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=14030 Ack=4215 Win=13800 Len=1380
40 0.091344 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=15410 Ack=4215 Win=13800 Len=684
41 0.100039 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=16094 Ack=4215 Win=13800 Len=1380
42 0.105600 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=17474 Ack=4215 Win=13800 Len=1380
43 0.110634 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=18854 Ack=4215 Win=13800 Len=1380
44 0.305121 11.11.1.1 22.222.222.22 TCP
[TCP Retransmission] PORT1 > PORT2 [PSH, ACK] Seq=14030 Ack=4215
Win=13800 Len=1380
45 0.497694 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4215 Ack=15410 Win=31232 Len=0
46 0.501600 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=20234 Ack=4215 Win=13800 Len=1380
47 0.502348 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=21614 Ack=4215 Win=13800 Len=1379
48 0.502358 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=22993 Ack=4215 Win=13800 Len=1
49 0.503698 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 45#1] PORT2 > PORT1 [ACK] Seq=4215 Ack=15410 Win=31232
Len=0
50 0.504465 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 45#2] PORT2 > PORT1 [ACK] Seq=4215 Ack=15410 Win=31232
Len=0
51 0.504472 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 45#3] PORT2 > PORT1 [ACK] Seq=4215 Ack=15410 Win=31232
Len=0
52 0.928385 11.11.1.1 22.222.222.22 TCP
[TCP Retransmission] PORT1 > PORT2 [PSH, ACK] Seq=15410 Ack=4215
Win=13800 Len=684
53 0.930646 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4215 Ack=16094 Win=31744 Len=0
54 0.936591 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=22994 Ack=4215 Win=13800 Len=1380
55 0.937366 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=24374 Ack=4215 Win=13800 Len=1379
56 0.937373 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=25753 Ack=4215 Win=13800 Len=1
57 0.938822 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 53#1] PORT2 > PORT1 [ACK] Seq=4215 Ack=16094 Win=31744
Len=0
58 0.939486 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 53#2] PORT2 > PORT1 [ACK] Seq=4215 Ack=16094 Win=31744
Len=0
59 0.939493 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 53#3] PORT2 > PORT1 [ACK] Seq=4215 Ack=16094 Win=31744
Len=0
60 1.789904 11.11.1.1 22.222.222.22 TCP
[TCP Retransmission] PORT1 > PORT2 [ACK] Seq=16094 Ack=4215 Win=13800
Len=1380
61 1.792075 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4215 Ack=17474 Win=31232 Len=0
62 1.798018 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=25754 Ack=4215 Win=13800 Len=1380
63 1.798749 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=27134 Ack=4215 Win=13800 Len=1379
64 1.798757 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=28513 Ack=4215 Win=13800 Len=1
65 1.800281 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 61#1] PORT2 > PORT1 [ACK] Seq=4215 Ack=17474 Win=31232
Len=0
66 1.800957 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 61#2] PORT2 > PORT1 [ACK] Seq=4215 Ack=17474 Win=31232
Len=0
67 1.800965 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 61#3] PORT2 > PORT1 [ACK] Seq=4215 Ack=17474 Win=31232
Len=0
68 1.805485 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=28514 Ack=4215 Win=13800 Len=1380
69 1.807632 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 61#4] PORT2 > PORT1 [ACK] Seq=4215 Ack=17474 Win=31232
Len=0
70 3.501979 11.11.1.1 22.222.222.22 TCP
[TCP Retransmission] PORT1 > PORT2 [ACK] Seq=17474 Ack=4215 Win=13800
Len=1380
71 3.504215 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4215 Ack=18854 Win=31232 Len=0
72 3.510261 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=29894 Ack=4215 Win=13800 Len=1379
73 3.511026 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=31273 Ack=4215 Win=13800 Len=1380
74 3.511035 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=32653 Ack=4215 Win=13800 Len=1
75 3.512409 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 71#1] PORT2 > PORT1 [ACK] Seq=4215 Ack=18854 Win=31232
Len=0
76 3.513136 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 71#2] PORT2 > PORT1 [ACK] Seq=4215 Ack=18854 Win=31232
Len=0
77 3.513144 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 71#3] PORT2 > PORT1 [ACK] Seq=4215 Ack=18854 Win=31232
Len=0
78 3.517030 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=32654 Ack=4215 Win=13800 Len=1379
79 3.517765 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=34033 Ack=4215 Win=13800 Len=1380
80 3.517774 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=35413 Ack=4215 Win=13800 Len=1
81 3.519187 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 71#4] PORT2 > PORT1 [ACK] Seq=4215 Ack=18854 Win=31232
Len=0
82 3.519966 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 71#5] PORT2 > PORT1 [ACK] Seq=4215 Ack=18854 Win=31232
Len=0
83 3.519972 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 71#6] PORT2 > PORT1 [ACK] Seq=4215 Ack=18854 Win=31232
Len=0
84 6.918782 11.11.1.1 22.222.222.22 TCP
[TCP Retransmission] PORT1 > PORT2 [ACK] Seq=18854 Ack=4215 Win=13800
Len=1380
85 6.920996 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4215 Ack=32654 Win=17432 Len=0
86 6.925662 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [PSH, ACK] Seq=35414 Ack=4215 Win=13800 Len=36
87 6.927628 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 85#1] PORT2 > PORT1 [ACK] Seq=4215 Ack=32654 Win=17432
Len=0
88 6.930298 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=35450 Ack=4215 Win=13800 Len=1380
89 6.932473 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 85#2] PORT2 > PORT1 [ACK] Seq=4215 Ack=32654 Win=17432
Len=0
90 6.936266 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=36830 Ack=4215 Win=13800 Len=1380
91 6.938553 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 85#3] PORT2 > PORT1 [ACK] Seq=4215 Ack=32654 Win=17432
Len=0
92 6.945639 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=38210 Ack=4215 Win=13800 Len=1380
93 6.947825 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 85#4] PORT2 > PORT1 [ACK] Seq=4215 Ack=32654 Win=17432
Len=0
94 6.951391 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=39590 Ack=4215 Win=13800 Len=1380
95 6.953579 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 85#5] PORT2 > PORT1 [ACK] Seq=4215 Ack=32654 Win=17432
Len=0
96 6.958442 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=40970 Ack=4215 Win=13800 Len=1380
97 6.960584 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 85#6] PORT2 > PORT1 [ACK] Seq=4215 Ack=32654 Win=17432
Len=0
98 6.965491 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=42350 Ack=4215 Win=13800 Len=1380
99 13.743297 11.11.1.1 22.222.222.22 TCP
[TCP Retransmission] PORT1 > PORT2 [ACK] Seq=32654 Ack=4215 Win=13800
Len=1380
100 13.745543 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4215 Ack=34034 Win=30720 Len=0
101 23.782698 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [PSH, ACK] Seq=4215 Ack=34034 Win=32768 Len=29
102 23.782746 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [FIN, ACK] Seq=4244 Ack=34034 Win=32768 Len=0
103 23.827627 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=43730 Ack=4245 Win=13800 Len=0
104 27.386210 11.11.1.1 22.222.222.22 TCP
[TCP Retransmission] PORT1 > PORT2 [PSH, ACK] Seq=34034 Ack=4245
Win=13800 Len=1380
105 27.388483 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [ACK] Seq=4245 Ack=35414 Win=31388 Len=0
106 27.393457 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=43730 Ack=4245 Win=13800 Len=1380
107 27.394204 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=45110 Ack=4245 Win=13800 Len=1379
108 27.394213 11.11.1.1 22.222.222.22 TCP
PORT1 > PORT2 [ACK] Seq=46489 Ack=4245 Win=13800 Len=1
109 27.395577 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 105#1] PORT2 > PORT1 [ACK] Seq=4245 Ack=35414 Win=31388
Len=0
110 27.396298 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 105#2] PORT2 > PORT1 [ACK] Seq=4245 Ack=35414 Win=31388
Len=0
111 27.396305 22.222.222.22 11.11.1.1 TCP
[TCP Dup ACK 105#3] PORT2 > PORT1 [ACK] Seq=4245 Ack=35414 Win=31388
Len=0
112 27.423284 22.222.222.22 11.11.1.1 TCP
PORT2 > PORT1 [RST, ACK] Seq=4245 Ack=35414 Win=31388 Len=0
This is from a 2.6.24 Kernel. I know it's not the most recent, but
it's what I need to use.
Thanks again.
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
