Recently, I had one of my employees (let's call him Mike) installing UPS-monitoring software on all of our desktop systems and servers. The idea was to have nonessential machines shut down before the essential servers so that we could extend our UPS runtime should the generator fail to start. The UPS vendor provides a nice little SNMP adaptor for the UPS and monitoring software for our servers and desktops, so it seemed a simple task to get this done. Unfortunately, once installed, the software didn't reliably connect to the UPS from all of the machines. Nothing Mike did seemed to make any difference.
Mike dutifully contacted the manufacturer's tech support department, and after a few phone calls and email exchanges, the tech reverted to the well-worn "well, it works for me so it must be your network" ploy. Having suffered hours on hold listening to Muzak and having received numerous emails containing obtuse or totally irrelevant content, my protege was in no mood to hear this. I walked into his office just in time to hear him muttering to himself the physical damage he wanted to inflict on the hapless tech. While I appreciate the visceral pleasure that can be derived from delivering a well-deserved beating, I did point out that the distance separating him from the tech precluded indulging those fantasies, not to mention the legal hassles involved with acting upon them. I pointed out that there are pleasures to be had proving the tech wrong (flipping him the proverbial "bird") and that by using some open-source tools available to us, we would easily be able do so. This month, I'll investigate a program that you should add to your repertoire.
Black Hat Toolbox
The program that I'll be discussing this month is equally at home in the toolbox of "Black Hats" or "White Hats," depending upon the use to which it is employed. It can be used as an analytical tool for legitimate troubleshooting and for illicit "snooping" and reconnaissance. Be aware that using it where you don't have authority and a need to do so will most certainly be frowned upon should you get caught. Thus, use it only after getting the necessary permission.
Monitoring the Ether
The program, Ethereal, is a network packet sniffer (although they use the phrase "network protocol analyzer") that's used to watch the traffic that passes across a network. The concept is straightforward. Ethereal does its work by placing your Ethernet card into promiscuous mode, causing the NIC to pass all traffic it "hears" on the network back to the program. Ethereal then decodes the traffic and provides a nice listing of everything that has transpired during the monitoring period.
Figure 1 shows a screen shot of Ethereal in action. The communications depicted were captured on the wireless interface of my laptop computer. I contacted Google using the Firefox browser and stopped the monitoring as soon as I got the main Google page. As you can see, the display is quite busy. Yet if you take some time to examine it, you'll observe that the entire conversation between the browser and the server is clearly laid out in the upper pane (in green). The first packet is my laptop computer making a DNS query for www.google.com of my gateway computer, which runs a caching name server. The second packet is the response from the name server, giving me the IP address of Google. Packets three through five are my laptop making a connection request from Google's Web site. Packet number six is my laptop making an HTTP request of the Web server.
Figure 1: This is the main Ethereal screen, after capturing a simple conversation between a browser and www.google.com. (Click images to enlarge.)
Should any particular packet be of interest, you can decode it by simply clicking on it. An example of this is shown in Figure 2, in which I have clicked on the packet where the HTTP request was made (packet 6). The middle pane shows some of the information available for each packet, and the lower pane shows the packet in hexadecimal. Ethereal is capable of fully decoding packets that are encoded in ASCII or EBCDIC, as long as they are unencrypted. Encrypted packets can still be traced, but their content will be unreadable.
Figure 2: You can obtain detailed information on any specific packet by clicking on it.
As you can imagine, a busy network will have a significantly larger number of packets that you will need to wade through in order to follow one particular conversation. Fortunately, Ethereal makes this a fairly simple task by offering two particularly useful functions.
First, you can select a specific packet, click the right mouse button on that selection, and then choose "apply as filter." By repeating these steps on both sides of a conversation, you can build a filter that can then be applied to your entire monitoring session, thus enabling you to extract only the conversation that you want to examine.
Second, you can click on one side of a conversation, click on "Analyze," and then select "Follow TCP Stream." Doing so will give you a color-coded view of the conversation, (which you can see in Figure 3), making it extremely easy to follow the conversation between the client and the server. I find this handy whenever I seem to have a "failure to communicate," as it gives me an objective third-party view of the conversation, completely devoid of error messages kicked up from either of the other programs.
Figure 3: "Follow TCP Stream" is a tool within Ethereal to give you a color-coded depiction of a conversation.
There is much about Ethereal that I am omitting, simply due to space considerations. I don't feel bad about it, though, as you can easily run Ethereal yourself. I'll describe later how to do that without loading Linux onto a computer.
For Ethereal to do its job, it will need to be able to "hear" the packets you want to monitor. This isn't as easy as it used to be, since switches, which limit packet distribution to the ports involved in a conversation, have been supplanting hubs, which mindlessly blast all packets from one port to every other port. You can think of it as the difference between the private telephone lines common today and the party lines common decades ago. Clearly, it is much easier to listen to party-line conversations than it is to listen to private-line conversations.
If you have a managed switch, you may be able to configure a management port, which allows you to "tap" into all conversations happening through the switch. If this is the case, then simply plug your Ethereal-equipped PC into the management port, and you're good to go. But what if you can't configure such a port? Just insert a hub between the switch and the device you want to monitor and then plug your Ethereal box into the hub. It's not as convenient as a management port, given that you can monitor traffic only to a specific machine, but it does work. Depending upon your network configuration, you may be able to insert a hub between switches as well. You may have to get creative to gain access to the network traffic you want to monitor, but it usually can be done.
Ethereal is included with virtually all Linux distributions, so if you are running Linux, it is likely that you already have it (or have easy access to it). But what if you aren't a Linux convert and don't want to load it just for Ethereal? What if you are at a location where the computers run only that other operating system? Are you out of luck?
Should you not have a dedicated Linux machine, you can still use Ethereal by taking advantage of one of the Live CD distributions, such as Knoppix. A "Live CD" distribution is a rather clever thing to behold. You simply insert the CD into your computer's CD drive and reboot (you'll need to ensure that your computer is set to boot from CD before the hard drive). When the system comes up, you'll be running a complete Linux distribution, replete with tools such as Ethereal. No software is loaded onto your hard drive, and nothing on the hard drive will be altered. You need only remove the CD from the CD drive to restore your computer's normal operation.
From a Linux perspective, the only major difference you'll note between a Linux disk installation and a Live CD is that the Live CD typically runs more slowly, since everything must be loaded from the CD drive. Once a program gets loaded into RAM, it tends to run at the full speed.
So enamored am I of Knoppix that I keep a copy handy at all times, even if I have my Linux laptop with me. Not only does it allow me to fire up Ethereal on a different machine than my own, but the other tools included on the CD have enabled me to repair or retrieve data from disks loaded with that other operating system. It's like having a software Swiss Army knife!
As much as I'd like to think that everyone is investigating Linux and open-source tools such as Ethereal, I know that many hesitate for lack of time. It becomes a catch-22 situation. You don't have time to investigate tools such as Ethereal, yet if you had them at your disposal, you'd have more time to investigate tools such as Ethereal. The open-source community has made it much easier with Live CD distributions, thus the excuses for not adding programs such as Ethereal to your toolkit are becoming harder to come by.
I encourage you to visit the Knoppix site and download the CD ISO image, which you can use to create your own bootable Linux CD. It will give you a chance to see Linux in action, as well as give you access to Ethereal and other open-source network tools (which I'll discuss in future columns). I guarantee that it will make your life much easier.
What About Mike?
And what of Mike's situation? Using Ethereal, he was easily able to document that our servers were contacting the UPS SNMP device and that it was properly responding. That vindicated our assertion that our network was fine. Mike truly enjoyed delivering the documentation to the aforementioned technician, thus turning the tables and putting the onus back on the software provider to work on the problem. Not only did that give the tech a well-needed attitude adjustment, but it saved me a possible trip to the pokey to post bail for Mike. Open-source software to the rescue!