Book: An Engineering Approach to Computer Networking

A while ago the Packet Pushers had Geoff Huston on as a guest in the future of networking series. There are lots of good ideas and contrarian opinions in that podcast episode – go listen to it.

During the episode, Geoff mentioned a book that had a big influence on him called An Engineering Approach to Computer Networking. Naturally, I ordered a copy.

The first thing you’ll notice reading this book is that some aspects are dated – it was written in 1996 after all. This becomes obvious early on when ATM is mentioned as the likely replacement for Ethernet and how it will play a major role in the future – obviously that didn’t work out.

Fortunately, most of the content is much more timeless.

Almost all networking books that try to be computer science or engineering text books are much closer to being descriptions of how IP networking works vs. really teaching the science behind networking. This book is the opposite – it’s not the book to read if you want to win an IP networking quiz show.

The principles discussed throughout the book underlie all circuit and packet switched networks so digging into details that may seem out of date is still well worth the time. This helps to build a solid foundation and gives a bit of perspective on how and why so much has stayed the same.

I’m not going to bother giving this book any kind of rating. If you have an interest in computer networking you should read it. For an even more abstract and science based view of computer networking you should also read my favourite networking book – Patterns in Networking Architecture.

Internet Redundancy, Or Not

Imagine you are a business that wants to have redundant connections to the Internet. Given the importance of an active Internet connection for many businesses this is a reasonable thing for an IT shop or business owner to ask for. One could also consider the serious home gamer who can’t risk being cut off as another use case.

Let’s dig into the technical options for achieving Internet redundancy.

The first and most obvious path would be to purchase a router that has two WAN ports and ordering Internet service from two different providers. Bam, you are ready to go right? Well… not really.

The way this typically works is that the router will choose one of the two Internet connections for a given outbound connection. The policy could be always use connection A until it fails or be more dynamic and take some advantage of both connections at the same time. The problem with this approach is that because the traffic will be NATed towards each Internet provider, there is no way to fail a given connection from one Internet provider to another. So the failure of one of the Internet connections means that your voice call, SQL or game connection will die, probably after some annoyingly lengthy TCP or application level timeout expires. If the site is strictly doing short outbound connections such as the case with HTTP 1.1 traffic this isn’t such a big deal.

So the ‘get two standard Internet connections and a dual port WAN router’ approach sort of works. Let’s call it partially redundant.

How do we get to true redundancy that can survive a connection failure without dropping connections? To this we need the site’s network to be reachable through multiple paths. The standard way to do this is to obtain IP address space from one of the service providers or get provider independent IP space from one of the registries (such as ARIN). Given that IPv4 addresses are in short supply this isn’t a trivial task. The conditions that have to be met to get address space are well out of the reach of small and medium businesses. Even when the barriers can be met, it’s still archaic to have to do a bunch of paper work with a third party for something that is so obviously needed.

The real kicker is that the lack of IP space is only part of the problem. IPv6’s huge 128-bit address space doesn’t really help at all because to use both paths, the site or home’s IP prefix needs to exist in the global routing table. That is, every core router on the Internet needs an entry that tells it how to reach this newly announced chunk of address space. The specialized memory (CAM) used by these routers isn’t cheap so there is a strong incentive within the Internet operations community to keep this kind of redundancy out of the reach of everyone except other ISPs and large businesses.

So the simple option doesn’t really solve the problem and ‘true’ redundancy isn’t possible for most businesses. What about something over the top?

Consider a router that is connected to multiple standard Internet connections. It could maintain two tunnels, one over each connection, towards another router somewhere else on the Internet. To the rest of the Internet, this second router is where the business is connected to the Internet. If one of the site’s Internet connections fails, the routers can simply continue passing packets over the remaining live tunnel thereby maintaining connectivity to the end site. From an end user’s perspective, this solution mostly works but let’s think about the downsides. We’ve essentially made our site’s redundancy dependent on the tunnel termination router and its Internet connectivity whereas without this we are just at the mercy of the ISP’s network. Also, unless the end site obtains its own address space, this approach has all downsides of the first approach except the NAT related problems occur at the tunnel termination router instead of being on-site. Finally, if the site can get its own address space, why do the tunnel approach at all?

I should note, because someone will point it out in the comments, that for very large organizations it’s possible to get layer two connectivity to each site and essentially build their own internal internet. If they have enough public IP space they can achieve redundancy to the end site for connections with hosts on the public Internet. With private IP space, they can achieve redundancy for connections within their own network. Without public IP space, even these networks suffer from the NAT related failure modes.

To summarize, if you aren’t a very large business, there is no way to get true Internet connection redundancy with the current Internet protocols. That’s kinda sad.

Part 2: Redundant Connections to a Single Host

Part 3: Detecting Failure

IPv6 deployment day is here

A few minutes ago major Internet companies enabled IPv6 permanently. Happy IPv6 day!

World IPv6 Day

World IPv6 Launch

The view from my server:

[dan@alpha ~]$ ping6 -n -c 1 www.google.com; ping6 -n -c 1 www.yahoo.com; ping6 -n -c 1 www.facebook.com
PING www.google.com(2001:4860:4008:802::1012) 56 data bytes
64 bytes from 2001:4860:4008:802::1012: icmp_seq=1 ttl=59 time=5.04 ms

--- www.google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 5.048/5.048/5.048/0.000 ms
PING www.yahoo.com(2001:4998:f00d:1fe::3001) 56 data bytes
64 bytes from 2001:4998:f00d:1fe::3001: icmp_seq=1 ttl=60 time=31.6 ms

--- www.yahoo.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 31.638/31.638/31.638/0.000 ms
PING www.facebook.com(2a03:2880:10:1f02:face:b00c:0:25) 56 data bytes
64 bytes from 2a03:2880:10:1f02:face:b00c:0:25: icmp_seq=1 ttl=54 time=87.7 ms

--- www.facebook.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 87.754/87.754/87.754/0.000 ms

A little IPv6 experiment

I’ve been running IPv6 on my home network for a while now. Since my provider doesn’t provide native IPv6 all external traffic occurs via 6to4.

Last week I setup 6to4 on my server which lives inside a local ISP’s colocation facilities. This provided IPv6 connectivity between my home network and the server. The only changes required were a couple of ACL modifications and configuring sendmail to listen on an IPv6 socket. Sadly I did discover that ejabberd cannot listen to both IPv4 and IPv6 addresses on the same port.

For a little experiment I decided to add an AAAA record to www.coverfire.com and see how much IPv6 traffic arrives. I know that the IPv6 Internet is vastly smaller than the IPv4 Internet so I didn’t expect a huge amount of traffic. In order to analyze whatever traffic arrived I captured all IPv6 port 80 traffic for the duration of the experiment.

The results of this experiment were disappointing. Over about 1.5 days there were only five IPv6 hosts which visited the site. One of the five hosts wasn’t even able to establish a TCP connection. From the capture file it looks like the ACKs from my server never arrived at the remote host.  Of the five addresses, four were 6to4 addresses. I found this a little surprising. Also interesting is the fact that there was no traffic from Teredo hosts.

A more interesting question is whether or not adding an AAAA record has caused troubles for people visiting the site via IPv4. See this article for one of the reasons why AAAA records can cause IPv4 users trouble.

For anyone who is interested I have uploaded the quick Python hack I used to analyze the capture file.

ip6.py