Microsoft's release of the XP Service Pack 2 (XP SP2) last month created a bit of a stir: The security patch was significant, relatively comprehensive and--for many users--automatically downloaded. Part of the stir was a result of the automatic nature of the download.

Some educational institutions had to shut down access to the Microsoft download site because returning students with new XP computers overwhelmed local college networks with the automatic update to their machines. IT administrators claimed that bandwidth speed was cut in half during the most critical registration period of their academic year. Clearly, in the future, the method by which Microsoft delivers updates needs some better coordination with these and other large institutions.

Hidden in the news, however, has been the effect that XP SP2 is having on pre-existing applications and services that resided on XP computers prior to the release of the update. Of significance to IT is what Microsoft has done to TCP/IP and Remote Procedure Calls (RPC).

Microsoft says that all users who connect to the Internet (and who doesn't?) should be aware of these changes to their system. But most users today treat TCP/IP like electricity, and RPC--a process by which another computer can start a program on a user's machine--is normally and legitimately used only by a few that have advanced custom-written applications. So what has Microsoft done to these important services?

TCP/IP and RPC Changes in XP SP2

TCP/IP stands for Transmission Control Protocol/Internet Protocol, and it's the base service by which millions of computer users attach their computers to the Internet. It's obviously not something that Microsoft developed, but Microsoft's implementation affects nearly 80% of all systems that attach to the global Internet.

Microsoft Windows operating systems have been vulnerable to hijacking through a number of well-documented worms and viruses. Some of these rogue programs use a combination of the services to inflict severe damage to the Internet. Unbeknownst to the user, these worms and viruses have opened TCP/IP ports and enabled the user's machine to receive commands through the RPC function; the purpose of these commands is to spread infection and coordinate Denial of Service (DoS) attacks against specific targets on the Internet.

In essence, the user's computer becomes a robot, controlled by other computers connected to the Internet. Often, the rogue code that has infected the user's computer will send out millions of copies of itself to other computers, using any email address or IP address it can find in the infected machine's hard drive. Then, the code goes into hibernation and waits for a signal from other machines (through the RPC function) to start a DoS attack.

Microsoft itself has several times been the target of these DoS attacks. The DoS attacks use something called the User Datagram Protocol (UDP) to overwhelm the targeted Internet site, making it impossible for legitimate users to access the resources of the target. So effective is the hijacking of computers via this method that a virus can infect millions of machines in just a few minutes, before it can even be detected. During the hibernation period, the user's machine appears normal, but it can be turned into a robot at the behest of its controllers, conducting massive DoS attacks against targets.

Clearly, with so many vital services connected through the Internet--including government services, emergency response systems, and others--this potential threat required Microsoft's attention. The changes to TCP/IP and RPC functions within the Windows XP SP2 represents Microsoft's current attempt to close these vulnerabilities.

Restricting Raw IP Socket Use

Microsoft had a few tactical problems in addressing the above security scenario. First of all, Microsoft does not control the technologies of TCP/IP or RPC. If it did (as IBM owned the SNA and SDLC protocols), it could change anything within these technologies, and everyone would have to comply. However, Microsoft obviously doesn't own either technology: They are standard protocols that are controlled by international standards organizations. Second, non-Windows computers on the Internet--servers and routers and non-Microsoft PCs--could not be negatively impacted by Microsoft's fiddling with the protocols. For instance, Microsoft could not use it's famous "embrace and extend" engineering tactics to break or enhance their interactions on the Internet with others.

Within the Windows operating system itself, Microsoft had to devise some strategy that would minimize the operating system's vulnerabilities, while simultaneously creating the lowest possible change to its interface with other Internet computers and services. In short, Microsoft had to make its versions of the protocols operate as "good citizens" within a much larger community of Internet computers. Microsoft's choices of actions were to eliminate the transmission of data through the raw IP sockets protocol and to restrict the level of transmission of packets that use the UDP with raw IP sockets.

A very small number of Windows applications use raw IP sockets, which provide an industry-standard way for applications to create TCP/IP packets with fewer integrity and security checks by the TCP/IP stack. The Windows implementation of TCP/IP still supports receiving traffic on raw IP sockets. However, the ability to send traffic over raw sockets has been restricted in two ways:

• TCP data can no longer be sent over raw sockets.
• UDP transmissions with invalid source addresses will not be sent over raw sockets. The IP source address for any outgoing UDP transmission must exist on a network interface; otherwise, the datagram is dropped.

How does this impact the operating system's security? This tactic, according to Microsoft, limits the ability of rogue code to create distributed DoS attacks and limits the ability of the operating system to send spoofed packets (TCP/IP packets with a forged source IP address).

Limiting TCP Connection Attempts

Another means by which XP SP2 has changed the innards of TCP/IP within Windows XP is through the monitoring of TCP/IP connection attempts. Previously, rogue computers infected with viruses often reached uninfected computers by trying to open simultaneous connections to random IP addresses. Most of those random addresses resulted in failed connection attempts, but the speed at which they occurred went unnoticed by the Windows operating system.

Now, under XP SP2, the TCP/IP stack limits the number of simultaneous incomplete outbound attempts, and after that limit has been reached, the rest are placed in a queue and resolved at a fixed rate. According to Microsoft, under normal conditions, when an application is connecting to an available host at a valid IP address, no connection governor will be activated. When the governor does kick in, an event is logged in the system's event log.

Microsoft says that the only probable result of this change to the TCP/IP stack will be that some security tools--including port scanners--will run more slowly. But they will, according to Microsoft, run.

Other WinSock Enhancements

Microsoft has also added a couple of enhancements to its WinSock protocol: the ability to remove applications that impact the Layered Service Provider (LSP) mechanism, and two new Netshare commands to reset and show the parameters of a network share configuration.

RPC Changes

Of equal significance to the security of a Windows XP are the changes that have been implemented to the RPC function of Windows XP. These changes allow the Windows server application to restrict the access to the RPC interface. Microsoft envisions a security callback process in which a registry key verifies access when the computer receives a remote program call. The server, after verifying that the calling program is from a valid IP address, calls the initiating computer back. The registry key forces RPC to perform additional security checks for all interfaces, even if the interface has no registered security callback.

This new RPC functionality could break an existing application if the application expects to receive calls from remote anonymous RPC client machines. They won't be able to connect to the server application. Likewise, the DCOM objects that use this function won't work either. Additionally, because secure RPC calls over connectionless protocols, such as UDP and IPX, use a lower level of security than calls over connection-oriented protocols, these calls will always be considered insecure and will fail by default.

To fix these applications, the new registry key--called RPC_RESTRICT_REMOTE_CLIENT_NONE--will need to be altered. Microsoft provides a number of settings to help overcome this limitation. The actual settings that need to be implemented are too specific for this article to address, and you may be required to change the RPC applications in order to make things work as you planned.

Is XP SP2 Worth the Hassle?

It's too soon to predict the impact--positive or negative--that the changes implemented to Windows XP will bring. Certainly, the changes seem, on the surface, to be a substantial reengineering of how XP deals with the Internet. But still missing are significant security control mechanisms within Windows that treat services and functions as more than just entries on a registry table, things like object security control mechanisms and better change-management security. In this regard, Windows security implementations of services still resemble the table-driven operating systems of years gone by.

And too, as Microsoft continues to resolve security issues with XP and future releases of the operating system, it should address the issue of patch distribution through other means besides automatic download distribution.

Finally, this article doesn't address all the enhancements and patches that XP SP2 provided. Many more are important, though the TCP/IP and RPC patches are probably most significant.

Nonetheless, XP SP2 is a very good beginning. Unfortunately, it is only a beginning for Microsoft. As the corporation evolves over the next five years, it will be important for IT management to monitor how Microsoft performs its role as a secure provider.

Thomas M. Stockwell is Editor in Chief of MC Press Online, LP.

Thomas M. Stockwell is an independent IT analyst and writer. He is the former Editor in Chief of MC Press Online and Midrange Computing magazine and has over 20 years of experience as a programmer, systems engineer, IT director, industry analyst, author, speaker, consultant, and editor.  

 

Tom works from his home in the Napa Valley in California. He can be reached at ITincendiary.com.

 

 

Also Read

Getting Started with Ansible on IBM i

Maximize the power of your lines of defense against cyber-attacks with IBM Storage FlashSystem and IBM Storage Defender

Building the human firewall: Navigating behavioral change in security awareness and culture