Re: download.ceph.com unreachable IPv6 [was: v9.1.0 Infernalis release candidate released]

Scott Laird <scott@xxxxxxxxxxx> · Sat, 07 Nov 2015 16:55:29 +0000

Ah, it looks like a pMTU problem.  We negotiate a 1440 byte mss, but that's apparently too large for something along the way; if you look at the tcpdump below, you'll see that the sequence number from download.ceph.com jumps from 1 up to 4285 without the packets inbetween ever arriving.
08:39:23.459684 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 > 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [S], seq 1826359551, win 26820, options [mss 8940,sackOK,TS val 60514795 ecr 0,nop,wscale 7], length 0
08:39:23.577649 IP6 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80 > 2001:470:e959:201:f652:14ff:fe09:530.50714: Flags [S.], seq 3196528483, ack 1826359552, win 28560, options [mss 1440,sackOK,TS val 917313247 ecr 60514795,nop,wscale 7], length 0
08:39:23.577679 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 > 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [.], ack 1, win 210, options [nop,nop,TS val 60514825 ecr 917313247], length 0
08:39:23.577749 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 > 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [P.], seq 1:82, ack 1, win 210, options [nop,nop,TS val 60514825 ecr 917313247], length 81
08:39:23.655644 IP6 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80 > 2001:470:e959:201:f652:14ff:fe09:530.50714: Flags [.], ack 82, win 224, options [nop,nop,TS val 917313267 ecr 60514825], length 0
08:39:23.818794 IP6 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80 > 2001:470:e959:201:f652:14ff:fe09:530.50714: Flags [P.], seq 4285:4764, ack 82, win 224, options [nop,nop,TS val 917313308 ecr 60514825], length 479
08:39:23.818806 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 > 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [.], ack 1, win 218, options [nop,nop,TS val 60514885 ecr 917313267,nop,nop,sack 1 {4285:4764}], length 0
08:39:28.706360 IP6 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80 > 2001:470:e959:201:f652:14ff:fe09:530.50714: Flags [F.], seq 4764, ack 82, win 224, options [nop,nop,TS val 917314529 ecr 60514885], length 0
08:39:28.706384 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 > 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [.], ack 1, win 218, options [nop,nop,TS val 60516107 ecr 917313267,nop,nop,sack 1 {4285:4765}], length 0
08:40:28.709596 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 > 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [.], ack 1, win 218, options [nop,nop,TS val 60531108 ecr 917313267,nop,nop,sack 1 {4285:4765}], length 0
08:40:28.875876 IP6 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80 > 2001:470:e959:201:f652:14ff:fe09:530.50714: Flags [.], ack 82, win 224, options [nop,nop,TS val 917329571 ecr 60516107], length 0

FWIW, it looks like eu.ceph.com uses an MSS of 1420, not 1440.

Scott

On Sat, Nov 7, 2015 at 8:30 AM Scott Laird <scott@xxxxxxxxxxx> wrote:
FWIW, I'm also having problems connecting to download.ceph.com over IPv6, from a HE tunnel.  I can talk to eu.ceph.com just fine.
I'm seeing 100% failures over HTTP.  Here's a traceroute, including my address:

# traceroute6 download.ceph.com
traceroute to download.ceph.com (2607:f298:6050:51f3:f816:3eff:fe50:5ec) from 2001:470:e959:201:f652:14ff:fe09:530, 30 hops max, 24 byte packets
 1  2001:470:e959:201:52c5:8dff:febe:a981 (2001:470:e959:201:52c5:8dff:febe:a981)  1.013 ms  0.923 ms  0.8 ms
 2  2001:470:e959:300:210:db01:2cff:1000 (2001:470:e959:300:210:db01:2cff:1000)  0.313 ms  0.343 ms  0.177 ms
 3  2001:470:b:685::1 (2001:470:b:685::1)  0.728 ms  0.408 ms  0.452 ms
 4  scottlaird-2.tunnel.tserv14.sea1.ipv6.he.net (2001:470:a:685::1)  7.75 ms  7.831 ms  7.487 ms
 5  v225.core1.sea1.he.net (2001:470:0:9b::1)  4.928 ms  15.432 ms  5.014 ms
 6  10ge13-4.core1.sjc2.he.net (2001:470:0:1c7::1)  34.031 ms  24.544 ms  25.762 ms
 7  2001:470:0:34f::2 (2001:470:0:34f::2)  23.92 ms  23.942 ms  24.185 ms
 8  2001:428::205:171:203:158 (2001:428::205:171:203:158)  74.452 ms  74.568 ms  74.53 ms
 9  2001:428:2402:10:0:d:0:2 (2001:428:2402:10:0:d:0:2)  75.029 ms  74.792 ms  75.145 ms
10  border11-bbnet1.wdc002.pnap.net (2600:c08:0:101:0:2:1:11)  160.907 ms  245.192 ms  204.006 ms
11  2600:c08:2002:d::2 (2600:c08:2002:d::2)  78.141 ms  78.102 ms  77.949 ms
12  ip-2607-f298-5-cc01--1.dreamhost.com (2607:f298:5:cc01::1)  79.807 ms  80.133 ms  77.979 ms
13  ip-2607-f298-5-cc08--2.dreamhost.com (2607:f298:5:cc08::2)  76.345 ms  76.229 ms  77.439 ms
14  2607:f298:5:110d:f816:3eff:fe79:ad5d (2607:f298:5:110d:f816:3eff:fe79:ad5d)  84.761 ms  79.603 ms  78.729 ms
15  * * *
16  * * *

On Thu, Oct 15, 2015 at 11:32 PM Corin Langosch <corin.langosch@xxxxxxxxxxx> wrote:
download.ceph.com resolves to 2607:f298:6050:51f3:f816:3eff:fe50:5ec here. Ping seems to be blocked. Connect to port 80

works every few requests, probably 50%. So I assume there's some load-balancer there with a dead backend, which the

load-balancer didn't detect/ kick...just guessing. Best Corin

Am 16.10.2015 um 08:27 schrieb Björn Lässig:

> Getting the same error here.

> With sixxs 4 out of 5 wgets are failing. ping6 is dropped.

>

> We tried from different sites in .at .uk and .de. Tries from uk and de are failing mostly. at is working.

>

> does ''ping6 download.ceph.com'' works for you?

>

> regards

>   Björn Lässig

>

> _______________________________________________

> ceph-users mailing list

> ceph-users@xxxxxxxxxxxxxx

> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________

ceph-users mailing list

ceph-users@xxxxxxxxxxxxxx

http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com