The Magic of RPC over HTTP -- Microsoft Certified Professional Magazine Online

The Magic of RPC over HTTP

If you have the latest and greatest from Microsoft—Windows Server 2003, Outlook 2003 and Exchange 2003—your users can get seamless remote access to e-mail.

By Bill Boswell
02/01/2004

Imagine a world where your traveling users can get seamless access to their Exchange e-mail from Outlook without VPNs, browsers or special configuration changes. The users simply make an Internet connection from their hotel room or living room, launch Outlook, and start reading e-mail.

This feat is possible if you have a combination of Outlook 2003 as the e-mail client, Exchange 2003 as the e-mail server, and Windows Server 2003 as the platform. Yes, yes. I know. Many of you haven’t even started to migrate off NT 4.0 domains yet, much less thought about deploying Exchange 2003 and a new version of Outlook. But even though it might take the management frogs a while to hop off the lily pad, it’s good to know what’s in store once you have the budget and the time to make the transition.

Outlook over the Internet
When an Outlook user opens a mailbox or peruses items in a public folder, the e-mail requests go to Exchange in the form of Remote Procedure Calls or RPCs. An RPC is a form of interprocess communication in which two applications share a bit of memory that they use to pass transactions back and forth to each other. RPCs aren’t a network wire protocol. If the two applications involved in an RPC transaction run on different machines, they depend on the underlying network to ferry the RPC messages back and forth.

RPCs are ideally suited for use in a LAN environment where the machines share a reliable, high-speed data communication path. When used across a WAN, though, they have some distinct disadvantages. RPCs are sensitive to network latencies; they’re fairly chatty; and unless the application developer has specifically chosen to use encryption, the RPC message content is open to inspection. And, thanks to the recent worms that exploit RPC buffer overflow vulnerabilities in Windows RPC, most companies and ISPs routinely block access to standard ports used to establish RPC communications.

Windows 2003 has a feature called RPC over HTTP that alleviates many of these deficiencies. Using RPC over HTTP, an application such as Outlook can embed standard RPC transactions into HTTP messages then encrypt the HTTP stream using SSL/TLS (Secure Socket Layer/Transport Layer Security—TLS is version 3.1 of SSL). Using HTTP as a wire protocol makes this form of RPC more resilient in the face of network latencies, and because ISPs permit traffic over SSL/TLS port 443, this technology avoids the RPC ports normally blocked.

RPC over HTTP is a feature of the operating system, so you’ll need to upgrade the OS on your e-mail servers to Windows 2003. Exchange 2000 doesn’t run on Windows 2003—and doesn’t know how to use RPC over HTTP anyway—so you’ll need to upgrade to Exchange 2003 as well. Outlook talks directly to Global Catalog (GC) servers to obtain address lists, so you’ll need to upgrade your GCs to Windows 2003, too. And you’ll need to run Outlook 2003 on a platform that understands RPC over HTTP, which means either XP SP2 (when it comes out) or XP SP1 with hotfix 331320.

I’ll grant you that this is more work than can be done on a quiet Saturday afternoon, but all those upgrades will be worthwhile in the long run, and not just to get RPC over HTTP.

RPC over HTTP
Figure 1 shows a simplified set of transactions that describe how Outlook 2003 communicates with Exchange 2003 via an RPC Proxy server. The proxy server doesn’t need to be running Exchange because RPC over HTTP is a feature of the operating system. The RPC Proxy service could actually be running on the Exchange server itself, but this isn’t a best practice because you would need to either place the Exchange server in your DMZ or punch a hole through your firewall directly to the Exchange server.

Outlook 2003 and Exchange 2003 communication

Figure 1. How Outlook 2003 communicates with Exchange 2003 over an RPC Proxy server. (Click image to view larger version.)

The transaction starts when the user launches Outlook after having configured it to use RPC over HTTP. This results in a stream of HTTP messages containing RPC instructions.

Outlook doesn’t transmit a bare HTTP stream due to the possibility of interception in transit, so it establishes an SSL/TLS session with the RPC Proxy server and uses the session key obtained as part of the TLS handshake to encrypt the stream. Think of this as a TLS “tunnel” for the RPC traffic.

The RPC Proxy server decrypts the SSL/TLS traffic, yielding a stream of HTTP messages that contain an RPC payload. These messages specify the name of the Exchange server as their final destination.

The RPC Proxy server establishes an RPC over HTTP connection with the Exchange server and uses it to forward the client’s messages. The Exchange server extracts the RPC instructions from the HTTP data stream and performs the requested operations.

As you can see from this process description, the term “RPC over HTTP” is something of a misnomer when it comes to using Outlook 2003 to connect to Exchange 2003. Outlook 2003 requires the use of SSL/TLS when communicating with an RPC Proxy server, so the true term should be “RPC through a Secure TLS Tunnel.”

RPC Elements

You don’t need an in-depth knowledge of RPC technology to deploy or troubleshoot RPC over HTTP, but it helps to know a few key elements of RPC communication.

RPC-Based Service Identifiers. If you picture the ISO networking model, an RPC-based service sits at the Application layer. The service registers itself with the OS using a Universally Unique Identifier, or UUID. For example, the Exchange Store registers an RPC interface with a UUID of a4f1db00-ca47-1067-b31f-00dd010662da. Each interface gets a different UUID, and a particular service can register several interfaces. Use the Rpcdump utility in the Windows 2003 Resource Kit to list the RPC services and their UUIDs.

RPC Client Protocols. Because RPC services reside at the Application layer, clients and servers that use RPCs rely on an underlying network transport protocol. Microsoft calls these RPC client protocols. Exchange uses five RPC client protocols, each of which has an identifier called a “protocol sequence.” The protocol sequence used for RPC over HTTP is called ncacn_http.

RPC Endpoints. When an RPC service launches, it registers its client protocol (or protocols) along with an entry point for each protocol. The combination of client protocol and entry point is called an endpoint. The nature of the endpoint depends on the client protocol. It might be a TCP or UDP port. It might be a named pipe. It might be a process interface that accepts local procedure calls. The ncacn_http client protocol uses TCP ports for endpoints. For example, the Exchange Store uses TCP port 6001.

End Point Mapper. Unlike standard Internet services such as FTP or DNS, which use specific, well-known ports for communication, an RPC service can select its ports dynamically at runtime, choosing among available ports above 1024. Because of this dynamic port registration, an RPC client application needs a way to determine the port number registered by the RPC server service. The client makes this determination by querying the End Point Mapper (EPM) service on the target server. The EPM listens on two ports: TCP port 135 and TCP port 593. A tool in the Windows 2003 Support Tools called Portqry lists the registered RPC endpoints on a server. Here’s a partial listing showing an endpoint registration:

C:\>portqry -n w2k3-ex3 -p tcp -e 135
Querying target system called:
w2k3-ex3
Attempting to resolve name to IP address...
Name resolved to 192.168.0.13
TCP port 135 (epmap service): LISTENING
Querying Endpoint Mapper Database...
Server’s response:

UUID: a4f1db00-ca47-1067-b31f-00dd010662da
Exchange 2003 Server STORE EMSMDB Interface
ncacn_http:192.168.0.6[6001]

==== End of RPC Endpoint Mapper query response ====

To support RPC over HTTP, these Exchange services register an ncacn_http client protocol with fixed TCP ports so that the RPC Proxy doesn’t need to communicate with the EPM:

Information Store: port 6001
System Attendant: port 6002
Referral (RFR) service: port 6002
NSPI (Name Server Provider Interface): port 6004 (NSPI provides access to address lists.)

Deploying RPC over HTTP
To deploy RPC over HTTP in production, requires—as you can see in Figure 2—an extensive cast of characters:

RPC over HTTP clients. These clients must be running Outlook 2003 running on either XP SP2 or XP SP1 with hotfix 331320.

Firewall. This can be any firewall capable of directing SSL/TLS traffic to a server on the private side.

RPC Proxy server. This is a server running Windows 2003 with RPC over HTTP installed and enabled.

Exchange 2003 mailbox and public folder servers. These servers must be running Windows 2003 with RPC over HTTP installed and enabled.

Windows Server 2003 Global Catalog servers. Any GC that might be referred to an Outlook client by an Exchange server must be running Windows 2003 and have RPC over HTTP installed and enabled.

Figure 2. Diagram of a production deployment of RPC over HTTP to support Outlook 2003 and Exchange 2003. (Click image to view larger version.)

In addition, the Outlook clients must have some way of resolving the name of the RPC Proxy server into an IP address. You have a couple of options in this regard. You can configure a local Hosts file on each client; but a better solution is to put host records in your public DNS zone that have the RPC Proxy server’s name and the IP address of the firewall’s public interface. You don’t need public host records for internal Exchange servers because the proxy server finds them using internal DNS.

Configure the RPC Proxy Server
Outlook requires the use of SSL/TLS, so the Web services running on the RPC Proxy server must have an X.509 certificate. You can use a Windows Certification Authority to issue the certificate or obtain a third-party certificate. The third-party certificate must be issued by a Certification Authority trusted by the client.

Figure 3 shows the Authentication configuration window for the RPC virtual folder on the RPC Proxy server. The virtual folder must reject Anonymous connections.

Figure 3. The Authentication Methods window for the RPC virtual folder, showing blocked access for anonymous users.

Figure 4 shows the Secure Communications window for the RPC virtual folder. The folder must accept only Secure Channel (SSL) connections using 128-bit encryption. This prevents users from inadvertently supplying credentials over a non-secure path.

Figure 4. The Secure Communications window for the RPC virtual folder, showing the requirement to use SSL and 128-bit encryption. (Click image to view larger version.)

Because the RPC Proxy server must route incoming messages to the correct service port on the Exchange and Global Catalog servers, the server’s Registry must be configured with the server names and ncacn_http port numbers. Here’s the Registry information:

Key: HKLM | Software | Microsoft | RPC | RPCProxy
Value: ValidPorts
Type: Reg_SZ
Data: **See Below

The format for the ValidPorts entries specify a name and port number in the format: : ; for example, W2K3-EX1:6001. An Exchange server listens for ncacn_http requests on ports 593, 6001, 6002 and 6004. Each port requires its own entry, although contiguous ports can be joined with a dash; for example, W2K3-EX1:6001-6002.

To make things even more complicated, the RPC Proxy service doesn’t automatically append DNS suffixes onto flat names, so each port must be entered in ValidPorts twice, once with the server’s flat name and once with its fully qualified DNS name. Here’s an example for a single Exchange server:

W2K3-EX1:593;W2K3-EX1.Company.com:593;W2K3-EX1:
6001-6002;W2K3-EX1.Company.com:
6001-6002;W2K3-EX1:6004;W2K3-EX1.Company.com:
6004

The entry for a GC would look like this:

W2K3-GC1:593;W2K3-GC1.Company.com:593;W2K3-GC1:
6004;W2K3-GC1.Company.com:6004

If the thought of typing all those entries for several servers (or several dozen, or several hundred) makes you want to say, “Not in this lifetime,” don’t give up hope. Microsoft has a Visual Basic script called Rpchttp_setup.vbs that finds all the Exchange and GCs in your organization and adds them to the ValidPorts value on a designated RPC Proxy server. Like so many nifty tools, this one comes from the friendly folks at Microsoft Product Support Services. The script may be published on Microsoft’s Web site by the time you read this. If not, contact PSS for a copy.

Configuring NSPI at Global Catalog Servers
Modern Outlook clients such as Outlook 2003 don’t send their address list requests to Exchange, because they know Exchange doesn’t own the directory service. Instead, they send their address list requests directly to the Name Service Provider Interface (NSPI) port on a GC.

Ordinarily, this service uses a dynamically registered port. Because the RPC Proxy requires a fixed port, you must configure a Registry entry on each GC to lock the port at a fixed value. Microsoft recommends using TCP port 6004. Here’s the information:

The Rpchttp_setup.vbs script doesn’t make Registry changes to GCs. You must do this update separately. If you don’t feel like manually adding the change to every GC, try using the Reg utility that accompanies Windows 2003. The following syntax makes the required Registry change to the specified server (the entire command should be on one line):

reg add \\<server_name>\hklm\system
\currentcontrolset\services\ntds\parameters
/v “NSPI Interface Protocol Sequence”
/t REG_MULTI_SZ /d ncacn_http:6004

By putting this command in a batch file and feeding it a list of your GCs, you can do the updates in a single step.

Configuring Outlook for RPC over HTTP
At last we’ve arrived at the Outlook client itself. Recall that it must be running on XP SP2, or SP1 with hotfix 331320.

1. From the main menu, select Tools | E-mail Accounts. This opens the -mail Accounts window.

2. Select the View or Change Existing E-mail Accounts radio button and click Next.

3. In the list of e-mail accounts, highlight Microsoft Exchange Server and click Change.

4. In the Exchange Server Settings window, click More Settings. This opens the Microsoft Exchange Server window.

5. Select the Connection tab.

6. Check the Connect To My Exchange Mailbox Using HTTP option.

7. Click Exchange Proxy Settings. This opens the Exchange Proxy Settings window.

8. In the Connection Settings field, enter the fully qualified DNS name of the RPC proxy server. This can be th ename of the front-end server or Exchange server. The format for the name you enter here must match the format of the name in the SSL certificate exactly. If it doesn’t, the connection will fail with no error messages. For example, the entry might use a Fully Qualified DNS Name (FQDN) of W2K3-FE1.Company.com. If the SSL certificate has the name W2K3-FE1, the connection will fail. There are no error messages, so this type of problem can be tough to troubleshoot.

9. In the Proxy Authentication Settings field, select NTLM Authentication from the drop-down menu. This gives a seamless experience to the user by taking the current logon credentials and using them to connect to the RPC Proxy server. For configurations involving home computers or non-domain members, use Basic authentication. Using NTLM authentication requires a Registry change.

Key: HKLM | System | Current
ControlSet | Control | LSA
Value: LMCompatibilityLevel
Data: 3 (REG_DWORD)

Test with Outlook
At this point, you’re ready to launch Outlook and make the connection. If you get prompted over and over and over again for credentials, check the server name format on the certificate from the RPC Proxy server. Here’s a quick way to make this check. Open a browser at the client and enter the secure path to the RPC virtual folder in IIS. An example of the syntax is:

https://w2k3-fe1.company.com/rpc

If the port on the firewall is configured correctly and the SSL configuration at the server is correct, you should connect and get a 403.2 error, indicating Access Denied. That’s because the browser requests Read access to a folder that only permits Execute. This is exactly what you want to see. If you get a 404 error, indicating that the client can’t find the server, then you have a DNS problem. Double-click the little padlock in the status bar to view the certificate received from the server. Verify that the server name format matches the format you used when you configured Outlook.

To make sure you’ve actually made an HTTPS connection rather than a standard TCP/IP connection, use a nifty new feature in Outlook 2003. Hold the Ctrl key down, right-click the Outlook icon in the Notification Area and select Connection Status from the flyout menu. This opens a Connection Status window, as shown in Figure 5. This window should show an HTTPS connection for each Mail, Directory and Public Folder connection.

Figure 5. The Outlook 2003 Connection Status window, showing HTTPS connections to mailbox servers, public folder servers, and Global Catalog servers. (Click image to view larger version.)

Ready to Rock and Roll
I have to admit that I was a little skeptical of RPC over HTTP when I first heard about it, because it sounded like yet another way to multiplex unmanaged services through Port 80 of a firewall. But the implementation for Outlook 2003 appears secure. By requiring SSL/TLS connections and by using an RPC Proxy server in the DMZ to avoid exposing an Exchange mailbox server, it’s possible to build an infrastructure that provides a truly useful service to users without compromising security. I highly recommend trying it out in your lab as you decide how and when you’ll deploy Exchange 2003 and Outlook 2003.

Register! Top 5 Hybrid AD Management Mistakes and How to Avoid Them