Ask a question
Search in U-verse Forums

U-verse Forums

Reply
Posted Dec 22, 2009
7:45:43 PM
Path MTU Discovery on the U-Verse RG

I broke this discussion out into a separate thread from where the discussion originally started since this is a separate issue.

 

The discussion originated with a discussion of using a VPN router behind the RG, in the DMZ, and doing IPSec/ESP through the RG and 1) Whether that will work properly or at all, and 2) what problems might arise in this configuration.

 

I posted the following in that thread:

 


SomeJoe7777 wrote:

I can also confirm that IPSec/ESP and IPSec/AH both work properly through the RG when the VPN router is set in the DMZ.  However, there is a caveat: The RG does not properly send ICMP packet-too-big responses back to the VPN router, which breaks PMTU discovery.  You will have to set the IP MTU manually on your IPSec tunnel to avoid fragmentation.  I used an MTU of 1350, and it was working fine.  This was on a Cisco 2811 with a tunnel to a Cisco 2851.


 

 

Another user asked the following question:

 


ka9q wrote:

Thanks for that information about path MTU discovery. But why would the RG have to generate too-large messages? Is the MTU anything less than 1500?


 

OK, well the issue is the way PMTU Discovery works.  Firs, remember that MTU = Maximum Transmission Unit, the maximum frame size that a network can handle due to its architecture.  Note that MTU is generally a layer 2 limitation.

 

In the old days, different networks had different MTU sizes that they could handle, and there was no way for the end stations to know what networks their packets might traverse over, and what the MTU was of each of these networks.  If a station sent a 1500 byte IP packet, but there was a link between the two stations that had a 576 byte limit, the IP packet would be fragmented into three layer 2 frames to be transported across that network, and would then get put back together.

 

There are several problems with fragmentation.  First, it uses a lot of CPU/memory resources on the routers that have to fragment and reassemble.  On the receiving router, fragments have to be held in memory until all the pieces have been received before they can be reassembled.  Second, if the link is dropping a small percentage of frames, it drastically increases the likelihood that a given IP packet will have to be retransmitted by TCP, because if one fragment is lost, all the fragments must be resent.

 

To avoid fragmentation, you used to have to be able to see all the network segments (or know their MTU), and manually adjust your MTU on your computer downward until the packet sizes your computer was sending out were small enough so that they wouldn't get fragmented.

 

Starting with Windows 95, a new mechanism to control IP fragmentation was introduced called path MTU discovery, or PMTUD.  What the end station does is simply send all IP packets out with the DF (Don't Fragment) bit in the IP header set.  If this packet hits a router where the network it needs to forward the packet to has an MTU smaller than the packet size (which would require fragmentation), the router is supposed to drop the packet, and send an ICMP "Packet Too Big" message back to the computer.

 

PMTUD takes advantage of that, because if it receives an ICMP Packet-Too-Big message, it lowers the IP datagram size and sends it out again.  Eventually the packet will make it through all the way to the target station without being fragmented, and the computer just remembers the correct IP datagram size for that TCP session.

 

MTU issues show up a lot in IPSec implementations because of the repeated encapsulations for all the carrier protocols.  For example, on an IPSec/ESP link coupled with GRE, a source IP packet is encapsulated in GRE, which is then encapsulated in IPSec/ESP.  Each encapsulation reduces the available MTU to the computer.

 

The issue with the U-Verse RG is that it seems to ignore the DF bit, and instead of dropping it and sending ICMP Packet-Too-Big, it just goes ahead and fragments the packet anyway.  This obviously breaks PMTUD, requiring you to manually set it on your router.

 

For a Cisco IPSec/ESP with GRE tunnel, the manually set MTU should be 1438, but I typically set it a little lower to avoid any unexpected issues, say to 1400.

 

The best document I've ever seen that explains these issues is Resolve IP Fragmentation, MTU, MSS, and PMTUD Issues with GRE and IPSEC on Cisco's site.

 

I broke this discussion out into a separate thread from where the discussion originally started since this is a separate issue.

 

The discussion originated with a discussion of using a VPN router behind the RG, in the DMZ, and doing IPSec/ESP through the RG and 1) Whether that will work properly or at all, and 2) what problems might arise in this configuration.

 

I posted the following in that thread:

 


SomeJoe7777 wrote:

I can also confirm that IPSec/ESP and IPSec/AH both work properly through the RG when the VPN router is set in the DMZ.  However, there is a caveat: The RG does not properly send ICMP packet-too-big responses back to the VPN router, which breaks PMTU discovery.  You will have to set the IP MTU manually on your IPSec tunnel to avoid fragmentation.  I used an MTU of 1350, and it was working fine.  This was on a Cisco 2811 with a tunnel to a Cisco 2851.


 

 

Another user asked the following question:

 


ka9q wrote:

Thanks for that information about path MTU discovery. But why would the RG have to generate too-large messages? Is the MTU anything less than 1500?


 

OK, well the issue is the way PMTU Discovery works.  Firs, remember that MTU = Maximum Transmission Unit, the maximum frame size that a network can handle due to its architecture.  Note that MTU is generally a layer 2 limitation.

 

In the old days, different networks had different MTU sizes that they could handle, and there was no way for the end stations to know what networks their packets might traverse over, and what the MTU was of each of these networks.  If a station sent a 1500 byte IP packet, but there was a link between the two stations that had a 576 byte limit, the IP packet would be fragmented into three layer 2 frames to be transported across that network, and would then get put back together.

 

There are several problems with fragmentation.  First, it uses a lot of CPU/memory resources on the routers that have to fragment and reassemble.  On the receiving router, fragments have to be held in memory until all the pieces have been received before they can be reassembled.  Second, if the link is dropping a small percentage of frames, it drastically increases the likelihood that a given IP packet will have to be retransmitted by TCP, because if one fragment is lost, all the fragments must be resent.

 

To avoid fragmentation, you used to have to be able to see all the network segments (or know their MTU), and manually adjust your MTU on your computer downward until the packet sizes your computer was sending out were small enough so that they wouldn't get fragmented.

 

Starting with Windows 95, a new mechanism to control IP fragmentation was introduced called path MTU discovery, or PMTUD.  What the end station does is simply send all IP packets out with the DF (Don't Fragment) bit in the IP header set.  If this packet hits a router where the network it needs to forward the packet to has an MTU smaller than the packet size (which would require fragmentation), the router is supposed to drop the packet, and send an ICMP "Packet Too Big" message back to the computer.

 

PMTUD takes advantage of that, because if it receives an ICMP Packet-Too-Big message, it lowers the IP datagram size and sends it out again.  Eventually the packet will make it through all the way to the target station without being fragmented, and the computer just remembers the correct IP datagram size for that TCP session.

 

MTU issues show up a lot in IPSec implementations because of the repeated encapsulations for all the carrier protocols.  For example, on an IPSec/ESP link coupled with GRE, a source IP packet is encapsulated in GRE, which is then encapsulated in IPSec/ESP.  Each encapsulation reduces the available MTU to the computer.

 

The issue with the U-Verse RG is that it seems to ignore the DF bit, and instead of dropping it and sending ICMP Packet-Too-Big, it just goes ahead and fragments the packet anyway.  This obviously breaks PMTUD, requiring you to manually set it on your router.

 

For a Cisco IPSec/ESP with GRE tunnel, the manually set MTU should be 1438, but I typically set it a little lower to avoid any unexpected issues, say to 1400.

 

The best document I've ever seen that explains these issues is Resolve IP Fragmentation, MTU, MSS, and PMTUD Issues with GRE and IPSEC on Cisco's site.

 

Path MTU Discovery on the U-Verse RG

3,371 views
18 replies
(0) Me too
(0) Me too
Reply
View all replies
(18)
0
(0)
  • Rate this reply
View profile
Dec 22, 2009 8:08:33 PM
0
(0)
Mentor

That's a good thumbnail sketch of PMTU. But I understand how it works, and I know that the headers added by IPSEC reduce the effective MTU as seen by the applications. So my point is this: under what circumstances should the Uverse RG ever have to return a datagram-too-large ICMP message? I presume it can handle 1500 byte packets, and since you talk to it with standard Ethernet with an MTU of 1500 bytes, how could you ever send it a packet that would be too large? Are any of your computers configured to send jumbograms?

 

The only entites that should ever have to return an ICMP datagram-too-big message over Ethernet are those with MTUs less than 1500 bytes,notably tunnels like your VPN router.  It's up to your VPN router, the thing that's doing the encapsulation into ESP or AH, to limit the size of the packets it can accomodate and to return the path MTU discovery messages to its clients to tell them that the MTU is something less than 1500 bytes.

 

That's a good thumbnail sketch of PMTU. But I understand how it works, and I know that the headers added by IPSEC reduce the effective MTU as seen by the applications. So my point is this: under what circumstances should the Uverse RG ever have to return a datagram-too-large ICMP message? I presume it can handle 1500 byte packets, and since you talk to it with standard Ethernet with an MTU of 1500 bytes, how could you ever send it a packet that would be too large? Are any of your computers configured to send jumbograms?

 

The only entites that should ever have to return an ICMP datagram-too-big message over Ethernet are those with MTUs less than 1500 bytes,notably tunnels like your VPN router.  It's up to your VPN router, the thing that's doing the encapsulation into ESP or AH, to limit the size of the packets it can accomodate and to return the path MTU discovery messages to its clients to tell them that the MTU is something less than 1500 bytes.

 

Re: Path MTU Discovery on the U-Verse RG

2 of 19 (3,371 Views)
0
(0)
  • Rate this reply
View profile
Dec 22, 2009 8:15:07 PM
0
(0)
Scholar
It could be a problem if there is a hop between you and the destination that has a smaller MTU than the 1500.
It could be a problem if there is a hop between you and the destination that has a smaller MTU than the 1500.

Re: Path MTU Discovery on the U-Verse RG

3 of 19 (3,371 Views)
0
(0)
  • Rate this reply
View profile
Dec 22, 2009 8:16:16 PM
0
(0)
Expert

That is true that the RG shouldn't have to send an ICMP PTB as long as the VDSL link can handle a 1500 byte packet.  But I'm not sure it can.

 

I believe someone has posted that the upstream packets on the broadband port have an 802.1q VLAN header, which would add 8 bytes, leaving the MTU at 1492.

 

I've also seen people over at DSL reports post that after trial-and-error, their MTU was indeed 1492.  Although some people also posted that theirs was 1500.  You probably can't really find out unless you have an ability to see what the packet fragmentation might be at the other end of the link (e.g. a router under your control).

 

That is true that the RG shouldn't have to send an ICMP PTB as long as the VDSL link can handle a 1500 byte packet.  But I'm not sure it can.

 

I believe someone has posted that the upstream packets on the broadband port have an 802.1q VLAN header, which would add 8 bytes, leaving the MTU at 1492.

 

I've also seen people over at DSL reports post that after trial-and-error, their MTU was indeed 1492.  Although some people also posted that theirs was 1500.  You probably can't really find out unless you have an ability to see what the packet fragmentation might be at the other end of the link (e.g. a router under your control).

 

Re: Path MTU Discovery on the U-Verse RG

4 of 19 (3,371 Views)
0
(0)
  • Rate this reply
View profile
Dec 22, 2009 8:42:16 PM
0
(0)
Mentor
I just ran some tests. From one of my static addresses, i.e., NOT through the NAT in the 3800, I had no trouble getting responses to 1500 byte pings to several remote sites. I.e., the ethernet frames leaving my computer were 1514 bytes long (1500 byte IP datagram plus 14-byte Ethernet header) and the Ethernet frames leaving the 3800 over the VDSL link were 1518 bytes after the addition of the 4-byte 802.1q tag.

But when I increased my packets to 1501 bytes, thus triggering IP fragmentation in my computer (since my Ethernet MTU is 1500 bytes), I stopped getting responses.

I got the same results for three separate sites: 74.53.67.2, the commercial server that hosts my personal website; Google's public DNS resolver 8.8.8.8; and the next-hop router in the Uverse network beyond my 3800, 99.71.136.1. I can still see my IP fragments leaving the 3800 over the VDSL link, complete with 802.1q tag, so the 3800 is definitely not blocking them. At the moment I have no root access to a remote ping target, nor can I see the downstream link from the VDSL modem to the 3800, so I cannot tell if my IP fragments aren't reaching their destination, or if they are and the reply (which would also consist of two IP fragments) is being blocked somewhere along the way.

So perhaps your problem is in the way your VPN box interacts with AT&T's network beyond the 3800, i.e., this could be one case where the 3800 is notat fault. If your VPN is configured in such a way as to generate IPv4 fragments instead of returning an ICMP message when a client packet plus ESP headers would exceed 1500 bytes, those packet fragments would not get through.

Since an IPSEC tunnel knows the MTU of its outbound interface, it should know how big a client packet it can tunnel without having to generate fragments. So if a client packet comes in with the DF bit set that exceeds this limit, the IPSEC tunnel should obviously return an ICMP unreachable with the proper MTU. But what if the client doesn't set the DF bit? In this case, I think the IPSEC tunnel might well go ahead and generate those fragments, and then you'll have problems over the Uverse path. Can you check that all the clients of your VPN box are setting the DF bit, i.e., they have path MTU discovery enabled?

Message Edited by ka9q on 12-22-2009 08:47 PM
I just ran some tests. From one of my static addresses, i.e., NOT through the NAT in the 3800, I had no trouble getting responses to 1500 byte pings to several remote sites. I.e., the ethernet frames leaving my computer were 1514 bytes long (1500 byte IP datagram plus 14-byte Ethernet header) and the Ethernet frames leaving the 3800 over the VDSL link were 1518 bytes after the addition of the 4-byte 802.1q tag.

But when I increased my packets to 1501 bytes, thus triggering IP fragmentation in my computer (since my Ethernet MTU is 1500 bytes), I stopped getting responses.

I got the same results for three separate sites: 74.53.67.2, the commercial server that hosts my personal website; Google's public DNS resolver 8.8.8.8; and the next-hop router in the Uverse network beyond my 3800, 99.71.136.1. I can still see my IP fragments leaving the 3800 over the VDSL link, complete with 802.1q tag, so the 3800 is definitely not blocking them. At the moment I have no root access to a remote ping target, nor can I see the downstream link from the VDSL modem to the 3800, so I cannot tell if my IP fragments aren't reaching their destination, or if they are and the reply (which would also consist of two IP fragments) is being blocked somewhere along the way.

So perhaps your problem is in the way your VPN box interacts with AT&T's network beyond the 3800, i.e., this could be one case where the 3800 is notat fault. If your VPN is configured in such a way as to generate IPv4 fragments instead of returning an ICMP message when a client packet plus ESP headers would exceed 1500 bytes, those packet fragments would not get through.

Since an IPSEC tunnel knows the MTU of its outbound interface, it should know how big a client packet it can tunnel without having to generate fragments. So if a client packet comes in with the DF bit set that exceeds this limit, the IPSEC tunnel should obviously return an ICMP unreachable with the proper MTU. But what if the client doesn't set the DF bit? In this case, I think the IPSEC tunnel might well go ahead and generate those fragments, and then you'll have problems over the Uverse path. Can you check that all the clients of your VPN box are setting the DF bit, i.e., they have path MTU discovery enabled?

Message Edited by ka9q on 12-22-2009 08:47 PM

Re: Path MTU Discovery on the U-Verse RG

5 of 19 (3,371 Views)
0
(0)
  • Rate this reply
View profile
Dec 22, 2009 9:04:05 PM
0
(0)
Expert

All Windows machines have PMTUD enabled by default, although this can be turned off with a registry entry.  However, I did not have that registry entry on the machines that I was testing from.

 

On a Cisco router with a GRE to IPSec tunnel, you use the tunnel path-mtu-discovery command to cause the router to copy the DF bit from the IP packet header to the GRE packet.  I did have that enabled on the GRE tunnel.  I beleive the IPSec implementation does this automatically when the GRE packet is encrypted by IPSec.

 

The Cisco document says that the most common cause of PMTUD failure is a firewall that improperly blocks the ICMP Packet Too Big response.  I don't think this was a problem, but I didn't specifically test it in my setup.  Next time I do this tunnel (which I do periodically as part of my job), I will specifically test it against ICMP.

 

All Windows machines have PMTUD enabled by default, although this can be turned off with a registry entry.  However, I did not have that registry entry on the machines that I was testing from.

 

On a Cisco router with a GRE to IPSec tunnel, you use the tunnel path-mtu-discovery command to cause the router to copy the DF bit from the IP packet header to the GRE packet.  I did have that enabled on the GRE tunnel.  I beleive the IPSec implementation does this automatically when the GRE packet is encrypted by IPSec.

 

The Cisco document says that the most common cause of PMTUD failure is a firewall that improperly blocks the ICMP Packet Too Big response.  I don't think this was a problem, but I didn't specifically test it in my setup.  Next time I do this tunnel (which I do periodically as part of my job), I will specifically test it against ICMP.

 

Re: Path MTU Discovery on the U-Verse RG

6 of 19 (3,371 Views)
0
(0)
  • Rate this reply
View profile
Dec 22, 2009 10:00:18 PM
0
(0)
Scholar

A couple of items to think about, first 2Wire only supports the following VPN configuration

 

  1. Verify that the VPN is not using IPSEC protocol 50 ESP in transport mode and IPSEC protocol 51 as they are incompatible with 2Wire gateways.  Only IPSEC Protocol 50 ESP in tunnel mode will work behind 2Wire gateways.
  2. You can force the upstream MTU for the RG using this page in the RG to be lower than the defautl of 1500

http://192.168.254.1/xslt?PAGE=J30&THISPAGE=J28&NEXTPAGE=J30

 Dave

 

A couple of items to think about, first 2Wire only supports the following VPN configuration

 

  1. Verify that the VPN is not using IPSEC protocol 50 ESP in transport mode and IPSEC protocol 51 as they are incompatible with 2Wire gateways.  Only IPSEC Protocol 50 ESP in tunnel mode will work behind 2Wire gateways.
  2. You can force the upstream MTU for the RG using this page in the RG to be lower than the defautl of 1500

http://192.168.254.1/xslt?PAGE=J30&THISPAGE=J28&NEXTPAGE=J30

 Dave

 

Re: Path MTU Discovery on the U-Verse RG

7 of 19 (3,371 Views)
0
(0)
  • Rate this reply
View profile
Dec 23, 2009 4:24:46 AM
0
(0)
Mentor

Can anyone explain that cryptic piece of prose from 2WIRE? What do they mean, "incompatibile with 2Wire gateways?" If it functions as a conventional NAT DMZ, and if you've designated your VPN box as the DMZ host for the primary public address assigned to the 3800, then all protocols not otherwise handled by the 3800 (TCP, UDP, ICMP) or diverted to other host applications ought to be by default sent to the DMZ host, and that should include the IPSEC protocols 50 and 51. And if you have an optional block of static IP addresses, then ALL packets to the address you've assigned to your VPN box ought to be sent to it, regardless of protocol.

 

This is what's so frustrating about the 3800, trying to figure out what the hell they actually mean by a particular phrase like that, or by "Stealth mode", or "Strict UDP session control" or "miscellaneous attack" or  ...

 

It would be interesting to do a study of all tech support forums related to home routers and their configuration and see what fraction is dominated by discussions about struggles with kludges like NAT port forwarding and DMZ. If home router makers would just implement IPv6 6to4 tunneling, we could leave all this in the past even before the ISPs implement IPv6 themselves.

 

 

Message Edited by ka9q on 12-23-2009 04:25 AM

Can anyone explain that cryptic piece of prose from 2WIRE? What do they mean, "incompatibile with 2Wire gateways?" If it functions as a conventional NAT DMZ, and if you've designated your VPN box as the DMZ host for the primary public address assigned to the 3800, then all protocols not otherwise handled by the 3800 (TCP, UDP, ICMP) or diverted to other host applications ought to be by default sent to the DMZ host, and that should include the IPSEC protocols 50 and 51. And if you have an optional block of static IP addresses, then ALL packets to the address you've assigned to your VPN box ought to be sent to it, regardless of protocol.

 

This is what's so frustrating about the 3800, trying to figure out what the hell they actually mean by a particular phrase like that, or by "Stealth mode", or "Strict UDP session control" or "miscellaneous attack" or  ...

 

It would be interesting to do a study of all tech support forums related to home routers and their configuration and see what fraction is dominated by discussions about struggles with kludges like NAT port forwarding and DMZ. If home router makers would just implement IPv6 6to4 tunneling, we could leave all this in the past even before the ISPs implement IPv6 themselves.

 

 

Message Edited by ka9q on 12-23-2009 04:25 AM

Re: Path MTU Discovery on the U-Verse RG

8 of 19 (3,371 Views)
0
(0)
  • Rate this reply
View profile
Dec 23, 2009 7:33:48 AM
0
(0)
Employee

ka9q wrote:

Can anyone explain that cryptic piece of prose from 2WIRE? What do they mean, "incompatibile with 2Wire gateways?" If it functions as a conventional NAT DMZ, and if you've designated your VPN box as the DMZ host for the primary public address assigned to the 3800, then all protocols not otherwise handled by the 3800 (TCP, UDP, ICMP) or diverted to other host applications ought to be by default sent to the DMZ host, and that should include the IPSEC protocols 50 and 51. And if you have an optional block of static IP addresses, then ALL packets to the address you've assigned to your VPN box ought to be sent to it, regardless of protocol.

 

This is what's so frustrating about the 3800, trying to figure out what the hell they actually mean by a particular phrase like that, or by "Stealth mode", or "Strict UDP session control" or "miscellaneous attack" or  ...

 

It would be interesting to do a study of all tech support forums related to home routers and their configuration and see what fraction is dominated by discussions about struggles with kludges like NAT port forwarding and DMZ. If home router makers would just implement IPv6 6to4 tunneling, we could leave all this in the past even before the ISPs implement IPv6 themselves.

 

 

Message Edited by ka9q on 12-23-2009 04:25 AM

The explaination said it was a matter of compatiility with transport mode versus tunnel mode; i.e., you cannot originate or terminate a tunnel at the RG proper, but it does support pass-through.

 

Doing 6-to-4 is stupid for anything other than necessary compatibility issues ... which do not yet exist. Do you have some IPv6-only equipment at home? Any unnecessary encaps, conversion, or translation just adds to the processing load and end-to-end propagation delay.

 

The "kludges" are usually only desired by an occasional user that want to do something well beyond the vast majority of users. If the frustration of trying to get a ten pound bag to hold twenty-five pounds of "stuff" is really bringing you down, try a commercial account (non-U-Verse) and a nice commercial router (I use a Cisco 2821) that will do all that stuff without having to kludge.To build an all-inclusive geek-satisfying unit would raise the service costs (and rates for everyone else) unnecessarily.

 

The alternative is the DMZ of the RG and an auxillary router (as long as it permits DHCP on the "outside" port), which permits virtually all activity without interference. 


ka9q wrote:

Can anyone explain that cryptic piece of prose from 2WIRE? What do they mean, "incompatibile with 2Wire gateways?" If it functions as a conventional NAT DMZ, and if you've designated your VPN box as the DMZ host for the primary public address assigned to the 3800, then all protocols not otherwise handled by the 3800 (TCP, UDP, ICMP) or diverted to other host applications ought to be by default sent to the DMZ host, and that should include the IPSEC protocols 50 and 51. And if you have an optional block of static IP addresses, then ALL packets to the address you've assigned to your VPN box ought to be sent to it, regardless of protocol.

 

This is what's so frustrating about the 3800, trying to figure out what the hell they actually mean by a particular phrase like that, or by "Stealth mode", or "Strict UDP session control" or "miscellaneous attack" or  ...

 

It would be interesting to do a study of all tech support forums related to home routers and their configuration and see what fraction is dominated by discussions about struggles with kludges like NAT port forwarding and DMZ. If home router makers would just implement IPv6 6to4 tunneling, we could leave all this in the past even before the ISPs implement IPv6 themselves.

 

 

Message Edited by ka9q on 12-23-2009 04:25 AM

The explaination said it was a matter of compatiility with transport mode versus tunnel mode; i.e., you cannot originate or terminate a tunnel at the RG proper, but it does support pass-through.

 

Doing 6-to-4 is stupid for anything other than necessary compatibility issues ... which do not yet exist. Do you have some IPv6-only equipment at home? Any unnecessary encaps, conversion, or translation just adds to the processing load and end-to-end propagation delay.

 

The "kludges" are usually only desired by an occasional user that want to do something well beyond the vast majority of users. If the frustration of trying to get a ten pound bag to hold twenty-five pounds of "stuff" is really bringing you down, try a commercial account (non-U-Verse) and a nice commercial router (I use a Cisco 2821) that will do all that stuff without having to kludge.To build an all-inclusive geek-satisfying unit would raise the service costs (and rates for everyone else) unnecessarily.

 

The alternative is the DMZ of the RG and an auxillary router (as long as it permits DHCP on the "outside" port), which permits virtually all activity without interference. 

Sent from my phone.
*I am an AT&T employee and the postings on this site are my own and don’t necessarily represent AT&T’s position, strategies or opinions.

Re: Path MTU Discovery on the U-Verse RG

9 of 19 (3,371 Views)
0
(0)
  • Rate this reply
View profile
Dec 23, 2009 9:36:20 AM
0
(0)
Mentor

Well, it certainly makes sense that the 3800 can't originate or terminate an IPSEC tunnel since it doesn't support those protocols, but I don't see why the device you've designated as the DMZ host can't do either. As long as the 3800 passes protocols 50 and 51, you should be fine.

 

I beg to differ on 6to4. It is by far the best "NAT penetration" tool currently available to those with only a single publicly routable IPv4 address. Until I got a block of extra static IPv4 addresses, the computers on my network at home (all of which have supported dual stack for years)  each had a private IPv4 address and a fully routable IPv6 address.The IPv4 addresses were subject to the usual NAT limitation to outbound-only connections. But their IPv6 addresses can be reached from the outside in a totally transparent fashion without any NAT port forwarding kludges of any kind. The only cost is the presence of both an IPv4 and an IPv6 header over the IPv4 Internet, i.e., the packets are 20 bytes larger than if I were running native IPv6. I don't even notice it.The important thing for performance, latency, etc, is that the route taken by my packets between two IPv6 hosts with 6to4 addresses is exactly the same as if I had native IPv6 addressing; there are no extra hops to anybody's tunnel.

 

Only when a 6to4 host talks to a non-6to4 IPv6 host do you need to tunnel through a third party and likely use a route that is physically longer than it otherwise could be. I don't seem to do that very often as my primary use of IPv6 is to reach back into my own network from the outside. I do think sites on the "real" IPv6 Internet should also maintain 6to4 addresses through a local 6to4 tunnel and use them when talking to a remote 6to4 site.

 

All I need is a single public IPv4 address from whoever is providing me with service and I can talk directly to any port on any of my home computers despite having only one public IPv4 address at home. And that's the only drawback to 6to4 that I've encountered so far. Most airports, coffee shops and hotels now stick you behind a NAT, and 6to4 won't work when the tunnel has a private IPv4 address. (There's Teredo, but I haven't tried it.) The solution to this is to lobby the vendors of the commodity NATs to include a 6to4 tunnel alongside the IPv4 NAT. Then traveling users with dual-stack hosts (which is virtually everybody) can have a private IPv4 address and a public IPv6 address. I've found it a very workable arrangement.

 

Well, it certainly makes sense that the 3800 can't originate or terminate an IPSEC tunnel since it doesn't support those protocols, but I don't see why the device you've designated as the DMZ host can't do either. As long as the 3800 passes protocols 50 and 51, you should be fine.

 

I beg to differ on 6to4. It is by far the best "NAT penetration" tool currently available to those with only a single publicly routable IPv4 address. Until I got a block of extra static IPv4 addresses, the computers on my network at home (all of which have supported dual stack for years)  each had a private IPv4 address and a fully routable IPv6 address.The IPv4 addresses were subject to the usual NAT limitation to outbound-only connections. But their IPv6 addresses can be reached from the outside in a totally transparent fashion without any NAT port forwarding kludges of any kind. The only cost is the presence of both an IPv4 and an IPv6 header over the IPv4 Internet, i.e., the packets are 20 bytes larger than if I were running native IPv6. I don't even notice it.The important thing for performance, latency, etc, is that the route taken by my packets between two IPv6 hosts with 6to4 addresses is exactly the same as if I had native IPv6 addressing; there are no extra hops to anybody's tunnel.

 

Only when a 6to4 host talks to a non-6to4 IPv6 host do you need to tunnel through a third party and likely use a route that is physically longer than it otherwise could be. I don't seem to do that very often as my primary use of IPv6 is to reach back into my own network from the outside. I do think sites on the "real" IPv6 Internet should also maintain 6to4 addresses through a local 6to4 tunnel and use them when talking to a remote 6to4 site.

 

All I need is a single public IPv4 address from whoever is providing me with service and I can talk directly to any port on any of my home computers despite having only one public IPv4 address at home. And that's the only drawback to 6to4 that I've encountered so far. Most airports, coffee shops and hotels now stick you behind a NAT, and 6to4 won't work when the tunnel has a private IPv4 address. (There's Teredo, but I haven't tried it.) The solution to this is to lobby the vendors of the commodity NATs to include a 6to4 tunnel alongside the IPv4 NAT. Then traveling users with dual-stack hosts (which is virtually everybody) can have a private IPv4 address and a public IPv6 address. I've found it a very workable arrangement.

 

Re: Path MTU Discovery on the U-Verse RG

10 of 19 (3,371 Views)
0
(0)
  • Rate this reply
View profile
Dec 23, 2009 12:02:15 PM
0
(0)
Employee

So you're using it as a VPN system.

 

I use a VPN server/client.  I can use it everywhere and access everything on my network as well. 

 

To each their own, I suppose.

 

So you're using it as a VPN system.

 

I use a VPN server/client.  I can use it everywhere and access everything on my network as well. 

 

To each their own, I suppose.

 

Sent from my phone.
*I am an AT&T employee and the postings on this site are my own and don’t necessarily represent AT&T’s position, strategies or opinions.

Re: Path MTU Discovery on the U-Verse RG

11 of 19 (2,609 Views)
0
(0)
  • Rate this reply
View profile
Dec 23, 2009 1:36:01 PM
0
(0)
Expert

ScottMac wrote:

So you're using it as a VPN system.


 

Not really "Private", since the IPv6 packet contents aren't encrypted or authenticated within the encapsulating 6to4 IPv4 packet.

 

More like a "VN" system. 

 

 


ScottMac wrote:

So you're using it as a VPN system.


 

Not really "Private", since the IPv6 packet contents aren't encrypted or authenticated within the encapsulating 6to4 IPv4 packet.

 

More like a "VN" system. 

 

 

Re: Path MTU Discovery on the U-Verse RG

12 of 19 (2,609 Views)
0
(0)
  • Rate this reply
View profile
Dec 23, 2009 2:44:14 PM
0
(0)
Mentor

Right, there's no encryption when you just wrap an IPv6 packet inside an IPv4 packet. The outer IPv4 header, the one the Internet sees, has the value '41' in the protocol field, meaning that what follows is an unencrypted IPv6 header.

 

The purpose of 6to4 tunneling is simply to give you IPv6 networking functionality when all you have is an IPv4 network. A single routable IPv4 address is automatically associated with a /48 IPv6 address block, meaning you have **80** bits on the right hand side to play with.  In usual practice, the rightmost 64 bits are used with stateless autoconfiguration for the host part, constructed by expanding the 48-bit Ethernet address of the interface to 64 bits by inserting the constant FFFE in the middle. That leaves 16 bits for a subnetwork ID within an organization. So you can have up to 65536 subnets, each with an essentially unlimited number of hosts, all hung off a single solitary IPv4 address, and no prior arrangements or assignments from IANA or anyone else are necessary. It might be a little extreme to build a network this big with just a single point of attachment to the outside world, but at least you won't run out of addresses if you try.

 

Of course you can always run IPSEC on top of IPv6 - technically, it's mandated to be available, if not used - and secure your traffic that way. Or you can do as I do and encrypt at the application layer, ssh for remote logins and file transfers (including rsync) and SSL/TLS for mail (IMAPS and SMTP/TLS).

 

Right, there's no encryption when you just wrap an IPv6 packet inside an IPv4 packet. The outer IPv4 header, the one the Internet sees, has the value '41' in the protocol field, meaning that what follows is an unencrypted IPv6 header.

 

The purpose of 6to4 tunneling is simply to give you IPv6 networking functionality when all you have is an IPv4 network. A single routable IPv4 address is automatically associated with a /48 IPv6 address block, meaning you have **80** bits on the right hand side to play with.  In usual practice, the rightmost 64 bits are used with stateless autoconfiguration for the host part, constructed by expanding the 48-bit Ethernet address of the interface to 64 bits by inserting the constant FFFE in the middle. That leaves 16 bits for a subnetwork ID within an organization. So you can have up to 65536 subnets, each with an essentially unlimited number of hosts, all hung off a single solitary IPv4 address, and no prior arrangements or assignments from IANA or anyone else are necessary. It might be a little extreme to build a network this big with just a single point of attachment to the outside world, but at least you won't run out of addresses if you try.

 

Of course you can always run IPSEC on top of IPv6 - technically, it's mandated to be available, if not used - and secure your traffic that way. Or you can do as I do and encrypt at the application layer, ssh for remote logins and file transfers (including rsync) and SSL/TLS for mail (IMAPS and SMTP/TLS).

 

Re: Path MTU Discovery on the U-Verse RG

13 of 19 (2,609 Views)
0
(0)
  • Rate this reply
View profile
Dec 23, 2009 2:56:27 PM
0
(0)
Mentor

You can certainly use an ordinary VPN to reach into your home network even when all you have is a single IPv4 public address. That can work just fine. You don't even have to encrypt; you could use direct tunneling, e.g. with IP protocol #4 indicating that what follows the outer IPv4 header is another IPv4 header. I did that for years with a Linux box on my company's DMZ so I could 'extrude' a block of their static address space over my cable modem to my home network.

 

Or you could use Cisco GRE and save a few bytes.

 

The only drawback to that approach happens if you want to keep multiple VPNs up to multiple private networks and they happen to use the same private addresses. How do you say that you want to reach the 192.168.0.10 through VPN tunnel A as opposed to the 192.168.0.10 through VPN tunnel B?

 

192.168.0.0/24 and 192.168.1.0/24 are used a lot more heavily than, say, 10.56.30.0/24, so these address collisions actually happen quite often.If you're in a position to renumber one or both networks to resolve the conflict, then fine; but sometimes, as in large companies, you're not. My company avoids the low 192.168 blocks internally precisely because they are so popular on home networks.

 

There are some advantages to using IPv6, even on small networks. Probably the nicest is "stateless autoconfiguration", which does away with the DHCP server. Each router (you can have more than one) issues a "router advertisement", giving the prefix (usually 64 bits) for the subnetwork that it is routing for. Hosts pick up these multicasts and fill in the lower 64 bits with a host-specific part. This can be a random number if desired for privacy purposes, but it is usually the interface Ethernet MAC address, expanded from 48 to 64 bits by inserting the constant FFFE in the middle.

 

When you combine stateless autoconfiguration with multicast DNS, you can also get rid of the need for the local DNS server, so there's one less thing to break and wreak havoc on your network. It's true that there's an autoconfiguration procedure for IPv4 as well (that's where those mysterious 169.254.xxx.xxx addresses come from) but the IPv6 mechanisms are a lot cleaner because you have so much more room.

 

You can certainly use an ordinary VPN to reach into your home network even when all you have is a single IPv4 public address. That can work just fine. You don't even have to encrypt; you could use direct tunneling, e.g. with IP protocol #4 indicating that what follows the outer IPv4 header is another IPv4 header. I did that for years with a Linux box on my company's DMZ so I could 'extrude' a block of their static address space over my cable modem to my home network.

 

Or you could use Cisco GRE and save a few bytes.

 

The only drawback to that approach happens if you want to keep multiple VPNs up to multiple private networks and they happen to use the same private addresses. How do you say that you want to reach the 192.168.0.10 through VPN tunnel A as opposed to the 192.168.0.10 through VPN tunnel B?

 

192.168.0.0/24 and 192.168.1.0/24 are used a lot more heavily than, say, 10.56.30.0/24, so these address collisions actually happen quite often.If you're in a position to renumber one or both networks to resolve the conflict, then fine; but sometimes, as in large companies, you're not. My company avoids the low 192.168 blocks internally precisely because they are so popular on home networks.

 

There are some advantages to using IPv6, even on small networks. Probably the nicest is "stateless autoconfiguration", which does away with the DHCP server. Each router (you can have more than one) issues a "router advertisement", giving the prefix (usually 64 bits) for the subnetwork that it is routing for. Hosts pick up these multicasts and fill in the lower 64 bits with a host-specific part. This can be a random number if desired for privacy purposes, but it is usually the interface Ethernet MAC address, expanded from 48 to 64 bits by inserting the constant FFFE in the middle.

 

When you combine stateless autoconfiguration with multicast DNS, you can also get rid of the need for the local DNS server, so there's one less thing to break and wreak havoc on your network. It's true that there's an autoconfiguration procedure for IPv4 as well (that's where those mysterious 169.254.xxx.xxx addresses come from) but the IPv6 mechanisms are a lot cleaner because you have so much more room.

 

Re: Path MTU Discovery on the U-Verse RG

14 of 19 (2,609 Views)
0
(0)
  • Rate this reply
View profile
Dec 23, 2009 3:18:56 PM
0
(0)
Employee

(Quote ka9q)

The only drawback to that approach happens if you want to keep multiple VPNs up to multiple private networks and they happen to use the same private addresses. How do you say that you want to reach the 192.168.0.10 through VPN tunnel A as opposed to the 192.168.0.10 through VPN tunnel B?

 

192.168.0.0/24 and 192.168.1.0/24 are used a lot more heavily than, say, 10.56.30.0/24, so these address collisions actually happen quite often.If you're in a position to renumber one or both networks to resolve the conflict, then fine; but sometimes, as in large companies, you're not. My company avoids the low 192.168 blocks internally precisely because they are so popular on home networks.

(end quote)

 

Interconnecting  multiple networks with the same address blocks can be done without too much effort with NAT. It's basically the same NAT, but operating from the inside out instead of the outside in.

 

Cisco's site has some examples of this: http://www.cisco.com/en/US/technologies/tk648/tk361/tk438/technologies_white_paper09186a0080091cb9.h...

 

 

(Quote ka9q)

The only drawback to that approach happens if you want to keep multiple VPNs up to multiple private networks and they happen to use the same private addresses. How do you say that you want to reach the 192.168.0.10 through VPN tunnel A as opposed to the 192.168.0.10 through VPN tunnel B?

 

192.168.0.0/24 and 192.168.1.0/24 are used a lot more heavily than, say, 10.56.30.0/24, so these address collisions actually happen quite often.If you're in a position to renumber one or both networks to resolve the conflict, then fine; but sometimes, as in large companies, you're not. My company avoids the low 192.168 blocks internally precisely because they are so popular on home networks.

(end quote)

 

Interconnecting  multiple networks with the same address blocks can be done without too much effort with NAT. It's basically the same NAT, but operating from the inside out instead of the outside in.

 

Cisco's site has some examples of this: http://www.cisco.com/en/US/technologies/tk648/tk361/tk438/technologies_white_paper09186a0080091cb9.html

 

 

Sent from my phone.
*I am an AT&T employee and the postings on this site are my own and don’t necessarily represent AT&T’s position, strategies or opinions.

Re: Path MTU Discovery on the U-Verse RG

15 of 19 (2,609 Views)
0
(0)
  • Rate this reply
View profile
Dec 23, 2009 5:58:54 PM
0
(0)
Mentor

Ow! My brain hurts!

 

I guess all that address translation can work (if Cisco says it can, they're probably right) but what a mess! Isn't it easier to use globally unique addresses that never have to be translated? You never have to worry about "NAT unfriendly" application protocols that exchange IP addresses, you can use new transport protocols other than UDP and TCP without any prior arrangement, and so on.

 

I'd like to think that the pain level associated with all this NAT stuff will eventually get so intense that people will simply decide that they've had enough and start using IPv6. But I've often underestimated the endurance of the average human, especially those who simply don't know that there exist better ways.

 

Ow! My brain hurts!

 

I guess all that address translation can work (if Cisco says it can, they're probably right) but what a mess! Isn't it easier to use globally unique addresses that never have to be translated? You never have to worry about "NAT unfriendly" application protocols that exchange IP addresses, you can use new transport protocols other than UDP and TCP without any prior arrangement, and so on.

 

I'd like to think that the pain level associated with all this NAT stuff will eventually get so intense that people will simply decide that they've had enough and start using IPv6. But I've often underestimated the endurance of the average human, especially those who simply don't know that there exist better ways.

 

Re: Path MTU Discovery on the U-Verse RG

16 of 19 (2,609 Views)
0
(0)
  • Rate this reply
View profile
Dec 23, 2009 6:50:38 PM
0
(0)
Expert

ka9q wrote:

 

I'd like to think that the pain level associated with all this NAT stuff will eventually get so intense that people will simply decide that they've had enough and start using IPv6. But I've often underestimated the endurance of the average human, especially those who simply don't know that there exist better ways.


 

I used to believe that as well, but I'm not so sure now.  A few things have happened in the last 10 years or so since 1998 when NAT became popular.  Remember the WebRamp? :smileyhappy:

 

  1. Nowdays we use fewer protocols that have problems with NAT.  Not much use of FTP anymore since a web application that supports upload/download is pretty easy to do.  Who uses H.323 applications anymore with all the features of it built into IM clients?
  2. NAT helper code in the routers has gotten really sophisticated -- even most of the VPN flavors work properly over NAT now.
  3. Configuring NAT on a Cisco, while not straightforward, is familiar to anyone who configures Ciscos, and extremely flexible.  You can do multiple different kinds of NAT between all the different networks connected to the router simultaneously.

 

I configure NAT on Ciscos from time to time, and it's not that much of a problem, and works just fine.  My company does a world conference every year at a resort hotel, and when we go there, I hook up a Cisco to do internet access and a GRE/IPSec VPN tunnel back to the headquarters.  I use NAT on that connection and can run the VPN, VOIP, Internet, streaming video, and things like SQL replication and Terminal Server sessions all at the same time, and I only need 1 static IP from the hotel. :smileyhappy:

 

Consumer routers can't do that, and frequently have problems in their NAT implementations.  But a seasoned product like a Cisco 2811 running IOS 12.4 is far more powerful and flexible than that.

 


ka9q wrote:

 

I'd like to think that the pain level associated with all this NAT stuff will eventually get so intense that people will simply decide that they've had enough and start using IPv6. But I've often underestimated the endurance of the average human, especially those who simply don't know that there exist better ways.


 

I used to believe that as well, but I'm not so sure now.  A few things have happened in the last 10 years or so since 1998 when NAT became popular.  Remember the WebRamp? :smileyhappy:

 

  1. Nowdays we use fewer protocols that have problems with NAT.  Not much use of FTP anymore since a web application that supports upload/download is pretty easy to do.  Who uses H.323 applications anymore with all the features of it built into IM clients?
  2. NAT helper code in the routers has gotten really sophisticated -- even most of the VPN flavors work properly over NAT now.
  3. Configuring NAT on a Cisco, while not straightforward, is familiar to anyone who configures Ciscos, and extremely flexible.  You can do multiple different kinds of NAT between all the different networks connected to the router simultaneously.

 

I configure NAT on Ciscos from time to time, and it's not that much of a problem, and works just fine.  My company does a world conference every year at a resort hotel, and when we go there, I hook up a Cisco to do internet access and a GRE/IPSec VPN tunnel back to the headquarters.  I use NAT on that connection and can run the VPN, VOIP, Internet, streaming video, and things like SQL replication and Terminal Server sessions all at the same time, and I only need 1 static IP from the hotel. :smileyhappy:

 

Consumer routers can't do that, and frequently have problems in their NAT implementations.  But a seasoned product like a Cisco 2811 running IOS 12.4 is far more powerful and flexible than that.

 

Re: Path MTU Discovery on the U-Verse RG

17 of 19 (2,609 Views)
0
(0)
  • Rate this reply
View profile
Dec 23, 2009 7:06:54 PM
0
(0)
Mentor

[quote]Nowdays we use fewer protocols that have problems with NAT.[/quote]

And why is that? I'd say it's because a lot of new and novel protocols that couldn't run over NATs have died on the vine before anyone was able to benefit from them.

 

NAT has a huge opportunity cost. It's not that we haven't managed to work around its limitations for most of the things we do every day, but isn't the Internet supposed to encourage new and innovative applications that weren't thought of by the original designers of the Internet protocols?

 

Funny you should mention VoIP. A while ago I bought a pile of VoIP phones and adapters, with the intent of learning about the protocols by doing and with a vague plan to set up a private telephone network for my family. It was easy enough to get them talking to each other without NATs in the way, but I eventually gave up trying to make them work properly through NATs. So the pile sits unused in a closet. (It doesn't really help that everybody now has cheap flat-rate long distance plans that make a private VoIP family network somewhat less nifty than it would have been.)

 

I've been working with TCP/IP since about 1984. I am one of its first implementers. If *I* can't figure out how to make this stuff work, then something is seriously wrong somewhere.

 

SIP is a lot more complicated than it ought to be, but at least it works in a fairly straightforward fashion when every phone has a public address. I just couldn't figure out how to get it to work through a Linux-based NAT; whatever application helper existed at that time simply wasn't up to it. That's why I'm so surprised with the VoIP guys; you'd think that they would be the ones clamoring the loudest for IPv6, yet few VoIP devices even support it.

 

IMHO, the easiest way to retire NAT is to encourage the commodity "router" vendors (D-Link, Netgear, Linksys) to implement 6to4 tunneling. Apple already does it in their newer Airport base stations. You just check a box and voila - assuming the base station has a public IPv4 address, and you haven't disabled IPv6 on your computers, IPv6 addresses automatically appear on their interfaces. As it was back in the good old days, it is once again possible for the average computer -- not a server in a co-lo rack, but a computer in someone's home -- to communicate directly, as client or server, using any port or set of ports, or even any transport protocol they choose, with any other similarly equipped computer, without any address translation or port forwarding hacks. It Just Works. How much easier can it get than that?

 

[quote]Nowdays we use fewer protocols that have problems with NAT.[/quote]

And why is that? I'd say it's because a lot of new and novel protocols that couldn't run over NATs have died on the vine before anyone was able to benefit from them.

 

NAT has a huge opportunity cost. It's not that we haven't managed to work around its limitations for most of the things we do every day, but isn't the Internet supposed to encourage new and innovative applications that weren't thought of by the original designers of the Internet protocols?

 

Funny you should mention VoIP. A while ago I bought a pile of VoIP phones and adapters, with the intent of learning about the protocols by doing and with a vague plan to set up a private telephone network for my family. It was easy enough to get them talking to each other without NATs in the way, but I eventually gave up trying to make them work properly through NATs. So the pile sits unused in a closet. (It doesn't really help that everybody now has cheap flat-rate long distance plans that make a private VoIP family network somewhat less nifty than it would have been.)

 

I've been working with TCP/IP since about 1984. I am one of its first implementers. If *I* can't figure out how to make this stuff work, then something is seriously wrong somewhere.

 

SIP is a lot more complicated than it ought to be, but at least it works in a fairly straightforward fashion when every phone has a public address. I just couldn't figure out how to get it to work through a Linux-based NAT; whatever application helper existed at that time simply wasn't up to it. That's why I'm so surprised with the VoIP guys; you'd think that they would be the ones clamoring the loudest for IPv6, yet few VoIP devices even support it.

 

IMHO, the easiest way to retire NAT is to encourage the commodity "router" vendors (D-Link, Netgear, Linksys) to implement 6to4 tunneling. Apple already does it in their newer Airport base stations. You just check a box and voila - assuming the base station has a public IPv4 address, and you haven't disabled IPv6 on your computers, IPv6 addresses automatically appear on their interfaces. As it was back in the good old days, it is once again possible for the average computer -- not a server in a co-lo rack, but a computer in someone's home -- to communicate directly, as client or server, using any port or set of ports, or even any transport protocol they choose, with any other similarly equipped computer, without any address translation or port forwarding hacks. It Just Works. How much easier can it get than that?

 

Re: Path MTU Discovery on the U-Verse RG

18 of 19 (2,609 Views)
0
(0)
  • Rate this reply
View profile
Jan 9, 2010 3:32:01 AM
0
(0)
Expert
I've always had the Christopher Columbus MTU (1492), never changes. :smileysad:
I've always had the Christopher Columbus MTU (1492), never changes. :smileysad:

Re: Path MTU Discovery on the U-Verse RG

19 of 19 (2,609 Views)
Share this post
Share this post