On Thu, Feb 23, 2017 at 1:30 PM, Sun Paul <paulrbk@xxxxxxxxx> wrote: > does this fixed in RHEL7? yes, I think so. > > On Wed, Feb 22, 2017 at 11:03 AM, Xin Long <lucien.xin@xxxxxxxxx> wrote: >> On Wed, Feb 22, 2017 at 10:29 AM, Sun Paul <paulrbk@xxxxxxxxx> wrote: >>> Hi Xin >>> >>> do you mean we need to patch the kernel? >> Yups, pls comment on that bz if it's really needed for your env. >> A z-stream kernel may be available for that issue soon. >> >> Thanks. >> >>> >>> >>> >>> On Wed, Feb 22, 2017 at 10:00 AM, Xin Long <lucien.xin@xxxxxxxxx> wrote: >>>> On Wed, Feb 22, 2017 at 9:12 AM, Sun Paul <paulrbk@xxxxxxxxx> wrote: >>>>> Hi >>>>> >>>>> the router is actually is a linux running on RHEL6.8 >>>>> (2.6.32-642.4.2.el6.x86_64). it uses iptables to do SNAT aand DNAT >>>>> forward. >>>> https://bugzilla.redhat.com/show_bug.cgi?id=1412038 >>>> >>>> sctp_manip_pkt->sctp_compute_cksum: >>>> struct sctphdr *sh = sctp_hdr(sub); >>>> >>>> But in rhel6, skb->transport_header is not yet set at that time. >>>> This patch should be backported into rhel6. >>>> >>>> commit 21d1196a35f5686c4323e42a62fdb4b23b0ab4a3 >>>> Author: Eric Dumazet <edumazet@xxxxxxxxxx> >>>> Date: Mon Jul 15 20:03:19 2013 -0700 >>>> >>>> ipv4: set transport header earlier >>>> >>>> >>>>> >>>>> On Tue, Feb 21, 2017 at 11:53 PM, Xin Long <lucien.xin@xxxxxxxxx> wrote: >>>>>> On Tue, Feb 21, 2017 at 12:26 PM, Sun Paul <paulrbk@xxxxxxxxx> wrote: >>>>>>> Hi, >>>>>>> >>>>>>> sorry to get back late, the platform is running on KVM. and this >>>>>>> problem is resolved by moving to VMware environment, however, a new >>>>>>> problem is coming out, we noticed that the HB REQ is being ABORT from >>>>>>> client. >>>>>>> >>>>>>> >>>>>>> 03:32:35.233572 IP 10.165.250.22.3868 > 192.168.2.13.40001: sctp (1) >>>>>>> [HB REQ] (from server to sctp router) >>>>>>> 03:32:35.233603 IP 192.168.2.14.3868 > 192.168.2.13.40001: sctp (1) >>>>>>> [HB REQ] (from sctp router to client) >>>>>>> 03:32:35.233852 IP 192.168.2.13.40001 > 192.168.2.14.3868: sctp (1) >>>>>>> [ABORT] (from client to sctp router) >>>>>>> >>>>>>> 03:32:37.928679 IP 10.165.250.22.3868 > 192.168.2.13.40001: sctp (1) [HB REQ] >>>>>>> 03:32:37.928717 IP 192.168.2.14.3868 > 192.168.2.13.40001: sctp (1) [HB REQ] >>>>>>> 03:32:37.929247 IP 192.168.2.13.40001 > 192.168.2.14.3868: sctp (1) [ABORT] >>>>>>> >>>>>>> For the above packet flow, 10.165.250.22 is the server and >>>>>>> 192.168.2.13 is the client, the server 10.165.250.22 sends HB REQ to >>>>>>> client 192.168.2.13 through 192.168.2.14 (the SCTP router), and the >>>>>>> SCTP router change the src address before forward the HB REQ to the >>>>>>> client. >>>>>>> >>>>>>> But somehow the client is ABORT the HB REQ, any idea? is it related to >>>>>>> the HEARTBEAT information? or the checksum again>? >>>>>> The incorrect checksum won't cause ABORT, but the abnormal HB REQ >>>>>> could be, if HB information was modified when calculating the checksum >>>>>> on router, the ABORT may be caused in client process. >>>>>> >>>>>> is your SCTP router linux ? if yes, what's the kernel version ? >>>>>> >>>>>>> >>>>>>> On Fri, Jan 13, 2017 at 9:19 PM, Michael Tuexen >>>>>>> <Michael.Tuexen@xxxxxxxxxxxxxxxxx> wrote: >>>>>>>>> On 13 Jan 2017, at 10:43, Michael Tuexen <Michael.Tuexen@xxxxxxxxxxxxxxxxx> wrote: >>>>>>>>> >>>>>>>>> Your router does NOT change any field in the SCTP packet, but the >>>>>>>>> SCTP checksum was modified from >>>>>>>>> Checksum: 0xbaea49e5 (not verified) >>>>>>>>> to >>>>>>>>> Checksum: 0xa9a86d3f (not verified) >>>>>>>>> At least one of these is wrong. Read the tracefiles in wireshark and >>>>>>>>> enable checksum validation and wireshark will tell you which one is >>>>>>>>> correct. (That is why I asked for .pcap file instead of a .txt). >>>>>>>>> >>>>>>>>> My guess is that the initial checksum is correct and your box middlebox >>>>>>>>> not only changes the destination address and ttl field and header >>>>>>>>> checksum in the IP-header (which is expected) but also incorrectly the >>>>>>>>> SCTP checksum. Since no field in the SCTP packet has changed, the checksum >>>>>>>>> must be the same. >>>>>>>> At the server have a look at the SNMP counters: >>>>>>>> cat /proc/net/sctp/snmp >>>>>>>> You should find a line staring with >>>>>>>> SctpChecksumErrors >>>>>>>> If the number reported there is positive, the node received packets >>>>>>>> with checksum errors. >>>>>>>> >>>>>>>> Best regards >>>>>>>> Michael >>>>>>>>> >>>>>>>>> Best regards >>>>>>>>> Michael >>>>>>>>>> On 13 Jan 2017, at 04:29, Sun Paul <paulrbk@xxxxxxxxx> wrote: >>>>>>>>>> >>>>>>>>>> Frame 2: 98 bytes on wire (784 bits), 98 bytes captured (784 bits) >>>>>>>>>> Encapsulation type: Ethernet (1) >>>>>>>>>> Arrival Time: Jan 6, 2017 16:52:49.662321000 Malay Peninsula Standard Time >>>>>>>>>> [Time shift for this packet: 0.000000000 seconds] >>>>>>>>>> Epoch Time: 1483692769.662321000 seconds >>>>>>>>>> [Time delta from previous captured frame: 0.000179000 seconds] >>>>>>>>>> [Time delta from previous displayed frame: 0.000179000 seconds] >>>>>>>>>> [Time since reference or first frame: 0.000179000 seconds] >>>>>>>>>> Frame Number: 2 >>>>>>>>>> Frame Length: 98 bytes (784 bits) >>>>>>>>>> Capture Length: 98 bytes (784 bits) >>>>>>>>>> [Frame is marked: False] >>>>>>>>>> [Frame is ignored: False] >>>>>>>>>> [Protocols in frame: eth:ethertype:ip:sctp] >>>>>>>>>> Ethernet II, Src: Vmware_81:41:6b (00:50:56:81:41:6b), Dst: >>>>>>>>>> Vmware_81:a6:a3 (00:50:56:81:a6:a3) >>>>>>>>>> Destination: Vmware_81:a6:a3 (00:50:56:81:a6:a3) >>>>>>>>>> Address: Vmware_81:a6:a3 (00:50:56:81:a6:a3) >>>>>>>>>> .... ..0. .... .... .... .... = LG bit: Globally unique >>>>>>>>>> address (factory default) >>>>>>>>>> .... ...0 .... .... .... .... = IG bit: Individual address (unicast) >>>>>>>>>> Source: Vmware_81:41:6b (00:50:56:81:41:6b) >>>>>>>>>> Address: Vmware_81:41:6b (00:50:56:81:41:6b) >>>>>>>>>> .... ..0. .... .... .... .... = LG bit: Globally unique >>>>>>>>>> address (factory default) >>>>>>>>>> .... ...0 .... .... .... .... = IG bit: Individual address (unicast) >>>>>>>>>> Type: IPv4 (0x0800) >>>>>>>>>> Internet Protocol Version 4, Src: 192.168.206.83, Dst: 192.168.206.66 >>>>>>>>>> 0100 .... = Version: 4 >>>>>>>>>> .... 0101 = Header Length: 20 bytes (5) >>>>>>>>>> Differentiated Services Field: 0x02 (DSCP: CS0, ECN: ECT(0)) >>>>>>>>>> 0000 00.. = Differentiated Services Codepoint: Default (0) >>>>>>>>>> .... ..10 = Explicit Congestion Notification: ECN-Capable >>>>>>>>>> Transport codepoint '10' (2) >>>>>>>>>> Total Length: 84 >>>>>>>>>> Identification: 0x0000 (0) >>>>>>>>>> Flags: 0x02 (Don't Fragment) >>>>>>>>>> 0... .... = Reserved bit: Not set >>>>>>>>>> .1.. .... = Don't fragment: Set >>>>>>>>>> ..0. .... = More fragments: Not set >>>>>>>>>> Fragment offset: 0 >>>>>>>>>> Time to live: 63 >>>>>>>>>> Protocol: SCTP (132) >>>>>>>>>> Header checksum: 0x1d3d [validation disabled] >>>>>>>>>> [Good: False] >>>>>>>>>> [Bad: False] >>>>>>>>>> Source: 192.168.206.83 >>>>>>>>>> Destination: 192.168.206.66 >>>>>>>>>> [Source GeoIP: Unknown] >>>>>>>>>> [Destination GeoIP: Unknown] >>>>>>>>>> Stream Control Transmission Protocol, Src Port: 50001 (50001), Dst >>>>>>>>>> Port: 3868 (3868) >>>>>>>>>> Source port: 50001 >>>>>>>>>> Destination port: 3868 >>>>>>>>>> Verification tag: 0x00000000 >>>>>>>>>> [Assocation index: 0] >>>>>>>>>> Checksum: 0xa9a86d3f (not verified) >>>>>>>>>> >>>>>>>>>> INIT chunk (Outbound streams: 3000, inbound streams: 3000) >>>>>>>>>> Chunk type: INIT (1) >>>>>>>>>> 0... .... = Bit: Stop processing of the packet >>>>>>>>>> .0.. .... = Bit: Do not report >>>>>>>>>> Chunk flags: 0x00 >>>>>>>>>> Chunk length: 52 >>>>>>>>>> Initiate tag: 0xe79f40cb >>>>>>>>>> Advertised receiver window credit (a_rwnd): 62464 >>>>>>>>>> Number of outbound streams: 3000 >>>>>>>>>> Number of inbound streams: 3000 >>>>>>>>>> Initial TSN: 176990880 >>>>>>>>>> IPv4 address parameter (Address: 192.168.206.83) >>>>>>>>>> Parameter type: IPv4 address (0x0005) >>>>>>>>>> 0... .... .... .... = Bit: Stop processing of chunk >>>>>>>>>> .0.. .... .... .... = Bit: Do not report >>>>>>>>>> Parameter length: 8 >>>>>>>>>> IP Version 4 address: 192.168.206.83 >>>>>>>>>> IPv4 address parameter (Address: 192.168.1.83) >>>>>>>>>> Parameter type: IPv4 address (0x0005) >>>>>>>>>> 0... .... .... .... = Bit: Stop processing of chunk >>>>>>>>>> .0.. .... .... .... = Bit: Do not report >>>>>>>>>> Parameter length: 8 >>>>>>>>>> IP Version 4 address: 192.168.1.83 >>>>>>>>>> Supported address types parameter (Supported types: IPv6, IPv4) >>>>>>>>>> Parameter type: Supported address types (0x000c) >>>>>>>>>> 0... .... .... .... = Bit: Stop processing of chunk >>>>>>>>>> .0.. .... .... .... = Bit: Do not report >>>>>>>>>> Parameter length: 8 >>>>>>>>>> Supported address type: IPv6 address (6) >>>>>>>>>> Supported address type: IPv4 address (5) >>>>>>>>>> ECN parameter >>>>>>>>>> Parameter type: ECN (0x8000) >>>>>>>>>> 1... .... .... .... = Bit: Skip parameter and continue >>>>>>>>>> processing of the chunk >>>>>>>>>> .0.. .... .... .... = Bit: Do not report >>>>>>>>>> Parameter length: 4 >>>>>>>>>> Forward TSN supported parameter >>>>>>>>>> Parameter type: Forward TSN supported (0xc000) >>>>>>>>>> 1... .... .... .... = Bit: Skip parameter and continue >>>>>>>>>> processing of the chunk >>>>>>>>>> .1.. .... .... .... = Bit: Do report >>>>>>>>>> Parameter length: 4 >>>>>>>>> >>>>>>>>> -- >>>>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in >>>>>>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>>>>> >>>>>>> -- >>>>>>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in >>>>>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-sctp" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
