How to Properly Test a WAN Optimization Solution


There’s a right way and a wrong way to test a WAN optimization solution. First and foremost, it’s unrealistic to test an IT solution in anything but a production environment. Second, it’s critical to do an apples-to-apples comparison between vendor solutions in test scenarios.

Beyond these components of a fair test, in this article we cover additional best practices specifically for evaluating WAN optimization solutions such as Riverbed SteelHead appliances.

Create your own WAN optimization test lab

Download WAN Optimization Test Plan 8.0 for a comprehensive guide to creating your own test lab. You’ll get 72 pages of procedures, use cases, advice, and more.

Test real HTTP clients and workloads

First, to test HTTP employ real HTTP clients and avoid using a load generator. Load generators can work well when measuring the number of connections a device can handle. However, when you want to measure the effectiveness of HTTP protocol optimization, load generators rarely provide meaningful results because they download a small set of objects repeatedly. Although that process generates valid traffic for assessing switches and servers, it is too repetitive to be a useful test for WAN optimization.

Second, to judge the value of a WAN optimization device, there is no substitute for genuine application traffic. For instance, HTTP application streamlining in the SteelHead appliance includes URL-learning algorithms that allow for predictive fetching of objects referenced from webpages.

Of course, these algorithms are triggered only when optimizing genuine web applications that include common building blocks such as HTML with embedded links. If you instead use an artificial traffic generator, none of the real-world pain of these web applications is simulated, and none of the relevant features are exercised to prove their effectiveness in alleviating the pain.

For more on testing HTTP, see p. 26 of the WAN Optimization Test Plan 8.0.

Test for larger deployments

Many people mistakenly test with only two devices, which creates, in effect, a simple point-to-point network that fails to expose the issues found in larger deployments. For example, the administrative overhead in creating and managing tunnels would not be exposed in a point-to-point deployment, but it would be obvious in a larger deployment.

As another example, consider that with large deployments the way a WAN optimization device handles datastores affects storage size and efficiency. In a common WAN optimization scenario, backups done at each branch are moved to a single datacenter over the WAN. In such scenarios, knowing the difference between a per-peer datastore and a unified datastore can affect how much storage you use.

  • WAN optimization with per-peer datastores saves data for each remote site in different logical partitions on a disk. That means that the same file from multiple remote sites resides on multiple pieces of disk, which is inefficient and uses more disk space.
  • In contrast, with a unified datastore the datacenter-based WAN optimization device uses the same disk space for common data across all remote sites. The patented Riverbed Scalable Data Referencing (SDR) technology found in SteelHead solutions is a unified datastore.

Always test CIFS

Many of today’s client-server protocols were originally designed to work on LANs where bandwidth is abundant and latency is negligible. In comparison, WANs have far less bandwidth and significantly higher latency, with substantial effects for these client-server protocols. While the bandwidth problem is well understood, the latency problem is much less so.

Latency — the amount of time it takes for data to go from point A to point B — is limited by the speed of light. For example, a packet originating from the US West Coast destined for the East Coast would typically take 25–30 ms one way, or 50–60 ms round trip. Having an infinite-bandwidth pipe between the two points will not alter the latency between them, since the limiting factor is the speed of light, which remains a constant.

“Having an infinite-bandwidth pipe between the two points will not alter the latency between them, since the limiting factor is the speed of light, which remains a constant.”

Many LAN-based protocols do not take latency into consideration. After all, latency on the LAN is typically less than 1 ms, so these protocols can afford to be relatively inefficient without affecting overall performance.

Such inefficient protocols tend to request small amounts of data at a time, and do so frequently. For example, the CIFS (Common Internet File System) protocol typically reads data from the server in relatively small chunks, even though the network could support larger transfers. The protocol also waits for a reply from the server before issuing the next request. The combination of small packets and frequent round trips yields what is known as a chatty protocol.

When the same protocols are used across the WAN, performance suffers as a result of increased latency. Taking CIFS as an example, what takes less than 1 ms to complete on a LAN now takes 50–60 times longer. Furthermore, the inefficiency of the protocol in requesting small chunks of data compounds the issue.

In contrast, WAN-friendly protocols typically request the maximum amount of data in a single request and minimize the need for repeated requests. Examples include FTP, HTTP, and SMTP. Given the difference in behavior between the CIFS protocol and the other WAN-friendly protocols, you should test the CIFS protocol extensively.

It is worth noting that Riverbed Technology is the only vendor capable of providing latency optimization for all the different SMB dialects, including CIFS/SMBv1/SMBv2/SMBv3, for both signed and unsigned traffic.

Remember Exchange MAPI testing

MAPI, another chatty protocol, can have performance issues similar to CIFS when used across WANs. Email clients, such as Microsoft Outlook, use either MAPI-over-RPC-over-TCP (commonly known as MAPI-over-RPC) or MAPI-over-RPC-over-HTTPS (commonly known as RPC-over-HTTPS or Outlook Anywhere) to connect to the Microsoft Exchange Server.

Starting with Microsoft Exchange 2013, MAPI-over-RPC is no longer supported, which leaves RPC-over-HTTPS as the only method for email clients to connect to the Microsoft Exchange Server.

An informed evaluation should include MAPI if it is a significant component of network traffic. Note that Riverbed is the only vendor capable of providing latency optimization for both encrypted and unencrypted MAPI, regardless of whether it’s direct MAPI (MAPI-over-RPC) or RPC-over-HTTPS.

For more on testing MAPI and Outlook Anywhere, see p. 19 of the WAN Optimization Test Plan 8.0.

Include TCP stack tuning

When deployed on high-latency links, SteelHead appliances benefit from tuning to maximize their performance. This requirement is not unique to the SteelHead appliance, but is rather determined by how TCP works. You could, for example, increase the buffer size or change the congestion algorithm.

While the traffic will be improved even without this fine-tuning, the results will be suboptimal. An informed evaluation should include appropriate TCP tuning if performance on a high-latency link is important.

Choose a secure SSL architecture

SSL ensures that traffic is securely transported from the origin to the destination. When optimizing SSL, the WAN optimization device must therefore, at minimum, provide the same amount of protection that it did before optimization.

The Riverbed approach to optimizing SSL ensures that the private key, which is the most important element in SSL, never leaves the datacenter and never resides in the branch. That means that even if the branch WAN optimization device is stolen, the private key will not be compromised, as it does not reside on the branch device.

Additional considerations

Your evaluation should not stop there, however. Modern technologies recently added to SteelHead appliances should be part of your must-have feature list. These include:

  • Integration with network performance management solutions, such as Riverbed SteelCentral™ products
  • Both inbound and outbound QoS (quality of service) capabilities
  • Path selection for maximizing available network paths and bandwidth
  • WAN optimization options for cloud and SaaS applications

The best evaluation is the one made for you

The best advice for evaluating any IT solution is to carefully design your tests to meet your individual needs, paying particular attention to the use of real-world applications, protocols, and test data. Also analyze any solution jointly with vendors to guarantee that anomalies are resolved and tests repeated as necessary.

Furthermore, since products must ultimately work in production, start thinking beyond the lab much earlier in the selection process. And, lastly, vendors should connect you with reference customers who have deployments of comparable size that are optimizing similar applications with success.