Article - Do proxies still matter?

Author: Werner Schmidt, CISSP
Date: January 2009

First, lets look at the history of proxies. Proxies came about originally to address several concerns of the past:

The need to provide user anonymity when surfing the Internet
Slower links felt faster with caching
Proxies were used to authorize user access to the Internet
Proxies came about to add more control than traditional firewalls could

1) User anonymity came about in the 80s and 90s because at the time we were all using public addresses. The desire then was to obscure the user device’s public IP address behind one single address and prevent web sites from knowing exact IPs (and ultimately users) that were accessing various sites. In large part, this need has been addressed and replaced since then with NAT (Network Address Translation) either on a router, firewall or other network device. We no longer have a world with unlimited public addresses and one to many NAT is used to conserve IP addresses and provides that user anonymity.

2) Back in the last century, we used to have entire large campus populations supported by slower links such as 56K. While content in the preweb days wasn’t as rich back than as it is now, it also was a bit more static. Slower links, lots of users, mostly static content, this was an ideal situation for caching. If the same user hit the same site, we could use cached data for performance reasons. While caching can still help in certain situations, we now have more streaming media, live/dynamic content and various web sites using browser directives to prevent caching. Caching can also create unique problems, imagine an incorrectly designed stock ticker site where the page name remains static and there are no directives telling the browser to update as the content changes. The other reality we have now is that network and caching devices can induce more delays than fast efficient network links. Caching can introduce problems for a wide variety of newer evolving data types. That’s not to say that the various caching technologies deployed across the Internet to handle large volume global web sites are a bad thing. However, caching at a local company perimeter or edge network device is no longer as critical as it once was.

3) We also used to use proxies to authorize user access to the Internet, something I like to refer to as Internet NAC. Basically we’d use proxies to dictate who (back then this was by device IP address only) could access the Internet. This would add a control feature back. This then morphed into appropriate use discussions (e.g. porn not allowed) and eventually even included warning pages, time quotas, etc. Proxies and various content filtering morphed a bit and we had the battle of the URL filtering lists. Since then we’ve seen the URL filtering list folks drive away from proxies by trying to add rich features into their standalone products and we’ve seen businesses start to question some of the costs of these solutions, adequacy in dealing with current threats and diversion techniques and some user backlash.

4) Proxies were used to expand upon the then limited capability of firewalls. Whether this was used to add user authentication, threat scanning or even just reporting, proxies were used to deal with access control features that firewalls could not provide. Many techniques and protocols came about to allow a proxy to communicate with other supporting devices such as AV scanners. This then introduced other problems and challenges such as trickling techniques.

Proxies are a unique and interesting solution. First we need to understand that the purpose of a proxy is to not allow a direct connection from a user desktop (or server) directly to the Internet. The client must connect to the proxy (gateway becomes the proxy unless WCCP is used, but that’s just for web traffic, yet another problem), possibly due some kind of authentication and then the proxy establishes a direct connection from it to the targeted destination. This creates some interesting problems and challenges:

Proxies have to be aware of all Internet applications and not break them
It imposes latency in the network
It creates a choke point
Due to the criticality of Internet use, redundancy must be factored back in
As faster pipes are deployed, proxies must ramp accordingly

A) Proxies were notorious in the earlier days for not keeping up with new applications and making the proxy firmware current. We used to be chasing strange network web issues only to discover a new feature was being used and the proxy couldn’t render it appropriately. We would literally test sites from home without proxies and compare to corporate proxy based responses.

B) Any additional hop in a network creates latency. Keep in mind that a proxy isn’t bridging a connection or doing a sniff, it’s actively terminating a user connection and reestablishing it to a destination. Thinks such as number of sessions per user and session ramp rate are important considerations.

C) Depending upon the proxy capability, other features (example content control, IM tracking, AV scanning, authentication etc.) it may even be integrated with other network devices and the sum user experience is the collective delay of all these devices and scans, we’re not talking single payload inspection. We can be talking multiple handoffs and multiple inspections on a single payload.

D) Access to Internet sites is business critical now, a failed proxy means no outbound traffic, therefore redundant units are almost always required. This effectively doubles the costs and increases complexity.

E) It’s easy to ramp up ISP bandwidth these days and even easy to use link balancer technology to easily bundle or pseudo bind multiple ISP connections together. Bottom line, bandwidth tends to increase frequently, threats are evolving and features are being added to proxies. This implies faster and better proxies are a given requirement and expense.

So, lets step back a bit... Why use proxies today? I think it really comes down now to:

Tracking user access to the Internet
Making sure only approved applications are used and enforcing policies
Dealing with the 443 encrypted traffic problem and threats to Data Leakage (among other items)
Removing or identifying threats from the payload

a) In a world of DHCP, litigation and basically the dynamics and addresses coupled with the passage of time, it’s not about addresses (devices) any more, it’s about tracking WHO went where and when.

b) Most networking people today still confuse ports with applications. Those days are long gone friends. So many applications now are evasive and use port hopping to mask their intent and anonymous proxies to hide their destination. Today, simply stated, port DOES NOT EQUAL application. This means that except for primitive web surfing and categorization of a web site, it’s become impossible to enforce real policies on things like file transfer, IM (Instant Messaging), P2P (peer to peer) applications, etc. The only way now is to inspect the traffic, using a destination or port is not at all effective.

c) Outgoing traffic is made up of two components: destination and payload. In order to prevent data leakage, it’s mandatory to inspect the payload of all data leaving an organization. If the traffic is encrypted, how can that be done? Simple techniques such as not allowing encrypted traffic are ineffective. First, users would rebel and a lot of legitimate traffic should be encrypted. Second, don’t assume traffic is only encrypted on port 443, you can encrypt it on any allowed port. This is a much longer discussion for another article about how to effectively deal with this challenge.

d) As IT professionals, we want to inspect the payload and make sure we’re not allowing threats to enter our organization. Simple firewall UTM solutions are too primitive. Threats can come in and exit over numerous protocols these days (web, ftp, IM, Email, P2P, etc.). We already discussed how a simplistic port only view is naive. This means we have to scan all traffic and look for current and changing threats.

Looking at the list above and the problems and limitations of proxies, I don’t believe proxies are really the right approach any longer. This really came about to address deficiencies in existing firewalls. However, what about a next generation firewall? Don’t consider the clunky UTM solutions, but what about a redesign from the ground up? Think about one that does track/report users, does inspect port 80 (and others) to determine the ACTUAL application (e.g. Skype) that’s traversing the network and scans for threats. If it did all this, did it at wire speed either active or passive and could be used on internal LAN segments, then we could reconsider not needing or wanting a proxy.

We have that solution now, please check out more about the Best of Interop 2008 winner, Palo Alto Networks.

Read the rest of the articles. Please contact us for more information!