You probably don’t think much about IP routing: how IP packets get from source to destination possibly going through intermediate nodes. Usually routing is handled for you correctly and reliably without much user effort. Much less do you think of how a reply will be routed back? But sometimes, as is the case when using load balancers such as the powerful WebMux™ Network Traffic Manager from AVANU®, it can make the difference between a functioning and non-functioning network.
Doesn’t the destination know how the packet got to it, at least the last hop whence it came, thus making routing a reply obvious? Not so fast, folks. Although there is such a thing as IP source routing where the original source specifies how a packet should journey to its destination (see section 8.5 of W. Richard Stevens’ classic TCP/IP Illustrated, Volume 1, it is rarely used. And even though it seems obvious to save the incoming path’s MAC address, and send its reply back there, such is rarely if ever done; the reply is routed without reference to this information. How can such routing thwart the load balancing traffic? The WebMux, like other load balancing products, are often configured so that when IP packets are sent to an IP address of a virtual server, this IP address is changed (undergoes NAT – network address translation) becoming the address of one of the real servers. This change implies that for load balancing to work correctly, the real servers must send packets in the reverse direction to the WebMux, so that the inverse NAT may be performed. If this inverse NAT is not done, the original client will see a reply packet from an IP address to which it never sent anything and discard it.
How may this situation arise? Suppose for the moment that the virtual server IP address is 10.0.0.1/8 and one of the real servers’ IP address is 192.168.1.200/24. Suppose further that a client is on the server’s 192.168.1.0/24 network, say 192.168.1.55. Let this client try to establish a TCP connection to 10.0.0.1.
How may the server be tipped off not to make such a mistake?
The WebMux has an IP address on the server network (192.168.1.0/24 in our example), say 192.168.1.1. Add a route on each server sending all traffic destined for 192.168.1.0/24 to 192.168.1.1.
Then all will be well. In Linux® the “route” and “ip” commands can do this job. For Windows® there as a command line command “route” also as well as a graphic interface for managing routes. The “route” command by itself will give you some usage, including:
route ADD 22.214.171.124 MASK 255.0.0.0 126.96.36.199 METRIC 3 IF 2
destination^ ^mask ^gateway metric^ ^
The route command with “print” as a parameter will give you the existing table and show you the “Interface” information. Using our example IP addresses the command line would be:
route ADD 192.168.1.0 MASK 255.255.255.0 192.168.168.1 METRIC 3
The Linux® commands might look something like this:
# route add -host 192.168.1.1 dev eth2
# route add -net 192.168.1.0/24 gw 192.168.1.1
The routes added might look like this:
# route -n | grep -F 192.168.1.
192.168.1.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
192.168.1.0 192.168.1.1 255.255.255.0 UG 0 0 0 eth2
THE BEST THINGS IN LIFE ARE FREE
Sometimes free things are worth every penny you pay for them.
Sometimes there are costs associated with getting things free.
But sometimes free is good and wonderful.
Such is the case with the packet capture and display tool Wireshark® (formerly known by the more evocative name Ethereal™). If you have trouble with connectivity on your network, for example if you made a slight slip installing a load balancer (such as your best value in load balancing, WebMux from AVANU, Wireshark can help you learn quickly what you did wrong, so you can fix it equally quickly.
Wireshark is available ready to go for Windows® and Mac® from http://www.wireshark.org/download.html . Source code is also available from this site. Binaries for the Red Hat/CentOS world are available in the package wireshark.x86_64. Get Wireshark and make it part of your toolbox. You’ll be glad you did.