2019-08-11 NetFlix vs IPv6

Update 2020-08-29: So, I found some more issues and dug deeper, and really, the answer is below hidden in the TCP dump output. The negotiated MSS is 1440 bytes, which is too large given I have a PPPoE service, which supports only 1432 byte MSS. Yes, PMTUD black holes still exist with IPv6.

---

So I finally got around to trying to find out why NetFlix seems to behave badly on Apple iOS devices at home. Running tpcdump on my NetBSD router gave me some interesting traces of hung NetFlix app startup on an iPad:

13:12:06.715945 IP6 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260 > 2a00:86c0:2041::1.443: Flags [S], seq 2246225929, win 65535, options [mss 1440,nop,wscale 7,nop,nop,TS val 625696573 ecr 0,sackOK,eol], length 0
13:12:06.716955 IP6 2a00:86c0:2041::1.443 > 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260: Flags [S.], seq 658221981, ack 2246225930, win 65535, options [mss 1440,nop,wscale 9,sackOK,TS val 2595614193 ecr 625696573], length 0
13:12:06.718660 IP6 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260 > 2a00:86c0:2041::1.443: Flags [.], ack 1, win 1026, options [nop,nop,TS val 625696575 ecr 2595614193], length 0
13:12:06.719173 IP6 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260 > 2a00:86c0:2041::1.443: Flags [P.], seq 1:238, ack 1, win 1026, options [nop,nop,TS val 625696575 ecr 2595614193], length 237
13:12:06.719920 IP6 2a00:86c0:2041::1.443 > 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260: Flags [.], ack 1, win 2050, options [nop,nop,TS val 2595614196 ecr 625696575], length 0
13:12:06.720988 IP6 2a00:86c0:2041::1.443 > 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260: Flags [P.], seq 2857:3357, ack 238, win 2050, options [nop,nop,TS val 2595614197 ecr 625696575], length 500
13:12:06.722359 IP6 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260 > 2a00:86c0:2041::1.443: Flags [.], ack 1, win 1026, options [nop,nop,TS val 625696578 ecr 2595614196,nop,nop,sack 1 {2857:3357}], length 0
13:12:07.206351 IP6 2a00:86c0:2041::1.443 > 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260: Flags [P.], seq 2857:3357, ack 238, win 2050, options [nop,nop,TS val 2595614683 ecr 625696578], length 500
13:12:07.207642 IP6 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260 > 2a00:86c0:2041::1.443: Flags [.], ack 1, win 1026, options [nop,nop,TS val 625697060 ecr 2595614196,nop,nop,sack 2 {2857:3357}{2857:3357}], length 0
13:12:07.800037 IP6 2a00:86c0:2041::1.443 > 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260: Flags [P.], seq 2857:3357, ack 238, win 2050, options [nop,nop,TS val 2595615277 ecr 625697060], length 500
13:12:07.801355 IP6 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260 > 2a00:86c0:2041::1.443: Flags [.], ack 1, win 1026, options [nop,nop,TS val 625697650 ecr 2595614196,nop,nop,sack 2 {2857:3357}{2857:3357}], length 0
13:12:08.334059 IP6 2a00:86c0:2041::1.443 > 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260: Flags [P.], seq 2857:3357, ack 238, win 2050, options [nop,nop,TS val 2595615811 ecr 625697650], length 500
13:12:08.358429 IP6 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260 > 2a00:86c0:2041::1.443: Flags [.], ack 1, win 1026, options [nop,nop,TS val 625698205 ecr 2595614196,nop,nop,sack 2 {2857:3357}{2857:3357}], length 0
13:12:08.891543 IP6 2a00:86c0:2041::1.443 > 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260: Flags [P.], seq 2857:3357, ack 238, win 2050, options [nop,nop,TS val 2595616368 ecr 625698205], length 500
13:12:08.972647 IP6 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260 > 2a00:86c0:2041::1.443: Flags [.], ack 1, win 1026, options [nop,nop,TS val 625698817 ecr 2595614196,nop,nop,sack 2 {2857:3357}{2857:3357}], length 0
13:12:09.505163 IP6 2a00:86c0:2041::1.443 > 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260: Flags [P.], seq 2857:3357, ack 238, win 2050, options [nop,nop,TS val 2595616982 ecr 625698817], length 500
13:12:09.586471 IP6 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260 > 2a00:86c0:2041::1.443: Flags [.], ack 1, win 1026, options [nop,nop,TS val 625699431 ecr 2595614196,nop,nop,sack 2 {2857:3357}{2857:3357}], length 0
13:12:10.119127 IP6 2a00:86c0:2041::1.443 > 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260: Flags [P.], seq 2857:3357, ack 238, win 2050, options [nop,nop,TS val 2595617596 ecr 625699431], length 500
13:12:10.201189 IP6 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260 > 2a00:86c0:2041::1.443: Flags [.], ack 1, win 1026, options [nop,nop,TS val 625700045 ecr 2595614196,nop,nop,sack 2 {2857:3357}{2857:3357}], length 0
13:12:10.733038 IP6 2a00:86c0:2041::1.443 > 2001:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:155f.52260: Flags [P.], seq 2857:3357, ack 238, win 2050, options [nop,nop,TS val 2595618210 ecr 625700045], length 500

So what's going on here? NetFlix is pushing us seq 2857:3357 and we're dutifully acknowledging it via a SACK (RFC 2018). NetFlix repeats the push, so obviously didn't get the SACK. We then respond with a duplicate SACK (see RFC 2883) to indicate that we've received it twice, and the remote end shouldn't back off.

Why are the SACKs getting lost or ignored? No idea. My workaround was to force NetFlix to IPv4, via simply blocking the IPv6 addresses via an npf rule, noting that anycast.ftl.netflix.com. has two AAAA records:

block in final on alc0 proto tcp to 2a00:86c0:2040::1 port 443
block in final on alc0 proto tcp to 2a00:86c0:2041::1 port 443