ISA 2000: CARP and NLB

Short version:
If you want to use both, for best results:


Web Proxy clients should use the array routing script/WPAD and need direct connectivity to each array member through a non-load-balanced address.
SecureNAT clients use the NLB VIP as the default gateway.
Firewall Clients connect to either the VIP or a name that maps to the VIP.


Very, very long version:
Cache Array Routing Protocol and Network Load Balancing are both scaling technologies, but have different behaviours and uses. You should assume that this information applies only to the outbound proxy scenario.


CARP


CARP was designed to be used to scale cache size and performance for web proxy servers, and is implemented using ISA Arrays.


The quick’n’dirty explanation of CARP is that if you have N proxies, 1/N of the requests will be serviced by each proxy, based on a hashing mechanism that runs both on the client and on the servers (which means, if you have either, you get some of the CARP benefit).


The standout benefit of CARP is that it multiplies your effective cache size by the number of proxies you have. Your cache size is effectively the combined sum of the size of each proxy’s cache, with negligible duplication of content, which is something not to be sniffed at*. It’s also generally faster, I’m told.


As an example pulled from my daydreams caused by the sugar consumed at lunchtime, the following might illustrate the point:


In a 3-proxy array with CARP:
http://www.microsoft.com/default.aspx – might be requested from “Proxy3”
http://www.microsoft.com/corporatelogo.jpg – might be requested from “Proxy2”
http://www.microsoft.com/includes.js – might be requested from “Proxy1”


Of course, having such an even spread is unlikely with a small number of requests, but as the volume of requests increases, it’ll be roughly equally distributed, more or less.


In the ISA Server context, CARP applies only to Web Proxy traffic, and there are a couple of layers to this:


Client CARP
Believe it or not, through the magic of the array routing script, WPAD.DAT or PAC files, the client browser actually participates in CARP. The client runs the same hash algorithm as the servers, and talks to the server that it believes will own the content for a particular URL.


This means that all other things being equal, if you run a PAC that supports CARP, you’ll get some benefit from it, as if all the clients use CARP, the servers will end up with complimentary cache content.


The client needs to know about all of the proxies, and how to contact them directly for this to work, and run the routing script/WPAD provided by the array. This point is important, so file it away for later on.


Server CARP
ISA Servers installed in an array naturally want to do CARP. This setting is controlled by the “resolve requests within array before routing” checkbox on the Outgoing Web Listener properties page. When the setting is enabled, if the “wrong” server is asked to retrieve a URL, it’ll re-run the CARP algorithm locally, pick the “right” proxy to ask for the content, and forward the request to that proxy, then return the content to the client.


CARP can be implemented purely at the server: if a client is pointed exclusively at Proxy1, Proxy1 will chat to Proxy2 and Proxy3 to retrieve the content that the client asked for. The client only talks to Proxy1, and only requires visibility of Proxy1, but each proxy will need to be able to talk to the other proxies in the array, and the level of server chatter will be increased. The logs will show the computer accounts of the other ISA Servers requesting content from each other for each set of client requests.


NLB


NLB provides box-dead failover and statistical load balancing of TCP and UDP connections across a group of clustered machines.


Box-Dead Failover: NLB has no application-layer smarts. It’s not like MSCS clustering, which can actually detect the health of applications and take nodes out of the cluster. If the service you’re trying to load balance should fail on one node, unless you actually take that box out of the cluster yourself, requests will still be routed to the dead application. Rule of thumb: If the IP stack is still responding, NLB will still be “working”. Likewise, if one node is absolutely pegging the CPU, but the other is idle, connections won’t be transferred to the idle server.


Statistical: The greater the ratio of clients to servers, the greater the probability of an even load across them.


In the ISA world, NLB is generally used to balance Firewall Client (FWC) and SecureNAT traffic – just point the Firewall Client or Default Gateway at the virtual IP (or a name that maps to it).


The default port rules for NLB are to balance TCP 0-65535 evenly across all nodes, with slight stickiness (Single Affinity – any connections from one IP go to one server). If you point a Web Proxy client at the virtual IP, it will end up on (essentially) a random server, which might not be the most efficient server to ask for objects from a CARP perspective. See the KB article 238524 on CARP and WLBS linked below for the detailed scenario.


Combination


As long as Web Proxy clients aren’t connecting to the virtual IP address of the cluster, everything works as you’d expect it to.


If Web Proxy clients are hard-coded to connect to any load balanced IP address, only server-side CARP will occur.


Likewise, if Web Proxy clients are hard-coded to any single proxy array member, only server-side CARP is possible.


If Web Proxy clients use the ISA array WPAD file, then as long as the client can connect to the individual array members directly by un-balanced IP, client-side CARP will work.


If you disable “resolve requests in array before routing”, you get only what your client script gives you (but you can create exceptions to the CARP process fairly simply – see 328248 below).


If You’re Still Interested At This Point, You’ll Love These Gems:


The Network Load Balancing white paper (Windows 2000 edition)
http://www.microsoft.com/windows2000/docs/NLBtech2.doc


Hack your Routing Script:
270524 Get.Info and Get.Routing.Script Functions Return Useful Diagnostic Information
http://support.microsoft.com/?id=270524


Tristan Finds Yet Another Useful KB Article After Writing His Blog:
238524 Using Cache Array Routing Protocol with Windows NT Load Balancing Service
http://support.microsoft.com/?id=238524


The Interesting CARP Caveat:
328428 Web Server Cannot Handle HTTP Request Made with ISA Array
http://support.microsoft.com/?id=328428

ISA 2000: Port Forwarding and Port Address Translation (PAT)

Short version:


ISA 2000 isn’t out-of-the-box able to change published ports when doing Server Publishing. (I think Application Filters can be written to do this, but I’m not totally sure). ISA 2004 can do this.


Long version:


RRAS has had the ability to translate a port mapping from a certain external port to a different internal port for a while – I’ve used it in Windows 2000 and Windows 2003 (avec Basic Firewall). This is sometimes called Port Address Translation, or PAT, and can essentially be thought of as a special case of NAT; instead of (or rather, as well as) translating the IP address, the port information in the TCP/UDP header is hacked and a mapping is maintained.


For example: you want to publish a Terminal Server (or Remote Desktop) on your external IP, port 3100, and translate this to an internal IP, port 3389. Using RRAS, this is done fairly quickly.


With ISA 2000, you’d need to actually change the internal port as well, so TS on the internal box would need to be hacked to run on port 3100 too. The capability to change the source or destination port when Server Publishing is not available. The protocol definition is the port, and the port shall be used, and only the port in the protocol definition shall be used.


With Web Publishing, you can do whatever you like with the internal ports, but it only applies to Web Proxy-handled traffic, which doesn’t include Terminal Server (RDP) or other straight-IP-based protocols (only HTTP/SSL), and will be stuck listening on whatever port you’ve configured your Incoming Web Listener for. So not helpful here.


Additionally, you typically need to create a separate protocol definition per individual port used. So, not great for publishing non-load-balanced groups of servers.


My take on this is that it’s actually a mixed blessing. If a port is open externally, you know immediately which internal port it’s going to be published on, with no mess, no fuss, no guesswork. If it looks like you’re server publishing an NNTP feed, you can be reliably assured that someone is publishing an NNTP feed.


Also useful to keep in mind – RRAS doesn’t provide any kind of inspection for any traffic, so you’re essentially punching holes to internal clients. With RDP, there’s little distinction because it’s encrypted and can’t really be inspected in realtime, but for protocols like HTTP, unless you’re also securing your back-end server, you potentially lose a layer of inspection.


ISA 2004 (Beta) has port hacking capabilities which are configurable when you publish a server, so the above scenario is do-able.

Use of Asterisks in this Blog

When I’m either being ironic or outright lying, I’ll usually include an unattributed asterisk to make it obvious, in much the same way advertisers do. Except they’re supposed to include fine print, whereas I’m not.


For example:


I like pizza. – is true.


I like being beaten at chess*. – is false.


So, wherever you see an asterisk without explanation, please feel free to add a comment that reflects your personal interpretation of the asterisk.


I’m generally not someone who uses pointers, being a no-unsafe-code type of person, so don’t expect double-dereferenced irony.

WshShell.Run and Long File Names

I hit this the other day – when using the Wsh Shell object’s Run method in VBScript, passing a long filename (LFN) tends to truncate, so:


set oShell = CreateObject(“Wscript.Shell”)
oShell.Run “C:\Documents and Settings\Tester\Desktop\TestFile.Doc”


will fail with a message similar to “Cannot find C:\Document”.


Simple solution: Double-escape the double quotes:


oShell.Run “””C:\Documents and Settings\Tester\Desktop\TestFile.Doc”””

How TSWeb / TSAC / Remote Desktop Web Connection Client Works

[Update 16 Aug 2004] I’ve posted some sample TSWeb HTM file that handles different ports too, and steps for how to get it working* with ISA 2004 (or other Port Address Translation-capable firewalls) in this post: Publishing RDP (Terminal Servers, XP Remote Desktop) with TSWeb.


There’s a common misconception that TSWeb allows you to connect to a Terminal Server over HTTP. The reality is that you just use HTTP to transfer the Remote Desktop Client ActiveX control to the client browser, which then runs and makes a regular RDP connection to the Terminal Server, just like the regular Remote Desktop client would, but presented in a browser window.

Short version: HTTP and RDP are used to connect to a TSWeb server. HTTP (TCP 80) is used to download the ActiveX control, which then connects directly using RDP (TCP 3389) to whatever server is specified by the page for the actual Terminal Server interface. Clients that can’t use port 3389 through a firewall won’t be able to connect, so clients that exclusively have Web protocol access are not able to use this method to connect. (They’ll be able to download the client and the page, but not able to do the actual Terminal Server part).

Long version:

Since the Terminal Services Advanced Client (TSAC), we have offered a web client version of the Terminal Server Client – nowadays called the Remote Desktop Client.

The TS Web client package can be installed on any Web server – it’s essentially a CAB file that contains the program itself (the ActiveX control) and some HTML or ASP to display a web page – it’s done using standard <OBJECT> tags. The web page is designed to ask your browser to download the CAB file, run the ActiveX control and pass some basic parameters to it, such as the server name and resolution.

There are several possible advantages to this method:

  • You limit the users’ need to learn server names – clicking links is all they have to do
  • You can control client settings centrally, from the web server, by editing the HTML page
  • You can upgrade the client software on the server 1

The disadvantages:

  • 1 Non-administrators generally won’t have permission to install ActiveX controls. On Win9X, no problem, everyone’s an administrator, but on more modern OSs, this can make deployment trickier – you’d have to actually roll out the client via, say, SMS or Active Directory.
  • You need to ensure client connectivity to both the Web server, and the Terminal Server
  • Clients can easily use MSTSC to connect directly to the Terminal Server – by itself, using the Web Client does not increase security because the web server’s security settings are not at all related to the Terminal Server security. You need to adjust the properties of the RDP Connection Object in the Terminal Services Configuration MMC to restrict which users or groups have permission to use Terminal Services (and any other security settings you need), just as you would as if the clients were connecting directly to it. Because they are.

And after typing most of this, here’s the KB article that also describes how it works…

270897 How Terminal Server Advanced Client Connects to a Terminal Server
http://support.microsoft.com/?id=270897

See also
294720 How to Server Publish a Terminal Server with ISA
http://support.microsoft.com/?id=294720

I love it when that happens*.

Helpful Picture:

[Updated 20 May 2004] This Helpful* Picture illustrates TSWeb Publishing when ISA Server 2000 is used – being fairly ISA-centric myself, I figured I’d pop this in. The top one is how you’d probably do it with an Integrated mode ISA Server (if you have routers in front of ISA but use ISA as the primary client connection behind them, then it works for that model too). The other one is how you’d do it if you have a separate firewall doing the NAT/Firewall stuff, and a cache-only ISA Server that publishes your websites.

For the Integrated scenario, you Web Publish the IIS instance hosting TSWeb, and then Server Publish the RDP protocol (you’ll need to create an RDP Server protocol definition for TCP 3389 Inbound, no secondary connections).

Quick Bullets:

  • The Web Server doesn’t need connectivity to the Terminal Server.
    • Connections are Client -> Web Server and Client -> Terminal Server
  • The client uses RDP to connect to the TS, not HTTP
    • so the client can’t connect if it only has Proxy connectivity (eg, just 80 and 443)
    • the server doesn’t proxy the client connection
      • you can’t connect to random servers on the internal network through the server – the client computer needs to be able to directly access the internal machine, or a port forwarded to the internal machine
      • OK, so technically you can access it using the RDP client through the server again , but a) that’s cheating and b) you’re adding more overhead, so it’s probably not the best idea
  • You need one external IP Address and port combination per internal server
    • if you have multiple TSs or WinXP Pro machines and only one IP address, use PATto forward from different external ports to 3389 on the internal machine.
      • eg, external IP:3390 -> Internal Desktop: 3389
      • using the out-of-box web client, this needs to be scripted using AdvancedSettings2: rdpclient.AdvancedSettings2.RdpPort = 3390 before connecting…
      • But as it happens, if you’d rather not fiddle with VBscript, this was the subject of Publishing RDP (Terminal Servers, XP Remote Desktop) with TSWeb , which links to a generic connection page I fiddled up that allows the user to specify their own port,

If you liked this, you’ll also love TS Licensing in 90 Words Or Less and Publishing RDP (Terminal Servers, XP Remote Desktop) with TSWeb.

About Me, About This Blog

It’s pronounced “try-stank”.

I’m Tristan Kington, a (Senior) Premier Field Engineer primarily covering Cybersecurity and PKI. I have a background spanning IIS, TMG, networking and infrastructure, and some development bits. I maintain or act as a Lead on a couple of our Assessment offerings, and I’m passionate about seeming passionate about things. Often. No, sometimes. (That was good, wasn’t it!?)

Most of the time, I’m travelling and helping customers understand, deploy, securely operate and maintain the products listed above, so there’s not a lot of time for blogging any more. Pity. But tweeting just doesn’t cut it.

 

My thinking was originally:

this blog will act as a public repository of things I’ve found that either aren’t documented elsewhere, or are but I couldn’t find them easily by a quickish web search. I tend to scribble notes in OneNote, and – just occasionally – there’s a really useful item that might not make the best KB article, but that might improve the quality of the infosphere by being out there. I like blogs where the technical content is pretty high, and the personal content relatively low, so that’s the model I’m mentally geared to, just don’t hold me to it!

I’m also freakishly erratic and a caffeine fiend, so I apologize if you catch me in my Bad Part Of The Day (before ~10pm).

This blog was originally called Extra Bits That Didn’t Fit, but I got less pretentious as the years wore on, and wanted to consider the use that moniker for something else. Yes, I’d say the use of “moniker” is also pretentious. I’m just evil.