VPN

Part 3: VPN RFP Lab Eval
RapidStream

• RapidStream Manager Alarms & Real-Time Monitoring Policies
• Logging & Diagnostics
• Keeping Hackers At Bay
• Additional Services
• Our Experience With Tech Support
• Customer Feedback
• Did RapidStream Satisfy Our RFP?

by Lisa Phifer Vice President of Core Competence, Inc. [November 21, 2001]

Earlier this year, ISP-Planet launched a VPN Appliance Review Series, evaluating IPsec hardware devices suitable for ISP deployment to broadband-enabled businesses of 10 to 200 employees. We gathered four responses that appeared, at least on paper, to satisfy our RFP. Our next step—a lab evaluation. By digging into each vendor's proposed solution, we hoped to compare and contrast these offerings.

Here, we publish our final results evaluating two RapidStream Security Appliances (RSSAs). The RSSA 2000 is a VPN concentrator or edge firewall for small-to-medium businesses. If you need a brief review of what we've accomplished so far, start with Part One or jump right into Part Two.

RSM Alarms & Real-Time Monitoring
The RSM System Information panel displays active Site-to-Site tunnels and Remote Access users, listed by policy or peer/user name. From here, one can view connect time, traffic counts, key lifetimes and negotiated security parameters—or disconnect any active IPsec SA. (To troubleshoot IKE or IPsec SA establishment, see Logging.)

The RSM Monitor panel provides on-demand, real-time monitoring (left) for Security Policies, System status and VPN activity. Simply select the objects to be monitored from a list; specify a polling interval and start monitoring, with results appearing on real-time graphs. For example, monitor incoming packets or discards, CPU utilization, port status, or even traffic tunneled between IKE Pairs.

We found monitors incredibly helpful to assess and troubleshoot current activity; historical reports on past behavior would also be nice. CPM does not provide real-time System Info and Monitoring, but similar features are now planned for CPM version 3.2.

Both RSM and CPM perform alarm monitoring. Severity, thresholds and actions—create log record, send email and/or generate SNMP trap—can be configured for each RSSA alarm (right). Built-in alarms keep an eye on system parameters (e.g., log size > N, port status = down, DDoS detected). Custom alarms can be defined by applying thresholds to counters, for specific RSSAs or globally. Some thresholds can be applied a set of Security Policies or IKE Pairs (below). For example, send customer-specific email when any critical error is detected on that customer's RSSAs.

Alarms light an LED on the RSSA's face plate and appear in RSM and CPM Alarm Logs, color-coded by severity. Either RSM or CPM can be used to acknowledge or clear pending alarms. Clear resets the LED and moves the alarm to the RSSA's Event Log. These alarm features are impressive. One nitpick—we'd break EVENT_ALARM apart so that login alarms can be downgraded or disabled without ignoring other serious events.

RSSAs can also be monitored by SNMP traps or polling. RSSAs support the IETF standard MIB-II, plus IPsec topology and monitoring MIBs and enterprise traps. This makes it possible to integrate RSSA monitoring into your existing SNMP NMS or develop your own SNMP-based performance monitoring applications. Security must always be considered when using SNMP. We created a Security Policy to tunnel SNMP traps through the RSSA's public port to our CPM Server. Note that our site-to-site tunnel was insufficient, because traps are internal, not received on the RSSA's private port. Similar policies can be used to permit secure polling, while inhibiting SNMP-based attacks from the Internet.

(Back to Top)

Logging & Diagnostics
Logs are stored on the RSSA itself or sent one SYSLOG server in WebTrends format. Separate logs are maintained for system events, firewall traffic, remote access users, and IKE and IPsec SAs. Traffic logging can be disabled, event logging can be controlled by severity, and all logs can be archived on-demand. The RSSA 2000 retains its log after reboot. When viewing logs from RSM or CPM, column filters can be applied, making it easy to find what you're looking for. These logging features are very good, but we still found room for improvement:

• Logs enumerate IKE messages and successfully negotiated security parameters. Unfortunately, one often needs failed proposals for VPN debugging. According to RapidStream, an IKE session trace is now being added.
• Due to the RapidCore architecture, only the first few packets of each session appear in the traffic log (or tcpdump output). Because documentation does not mention this, admins may incorrectly conclude that there is no traffic when there are no new records.
• Version incompatibility prevented CPM from displaying RSSA traffic records. This will be fixed in CPM 3.2—until then, traffic logs can be viewed from RSM or the SYSLOG.

RSM and CPM both read log records from the RSSA. But if records are sent to a SYSLOG, they cannot be viewed by RSM or CPM. Furthermore, if an RSSA is unreachable or down, no log info is available—just when it's needed most. If CPM cached previously read records, or were able to access the SYSLOG, it would be a more powerful central monitor.

RSM (but not CPM) includes two handy diagnostic aids: a ping command and the ability to execute any CLI script. Using CLI commands, one can view and flush active SAs or display diagnostic logs for debugging. For example, "show log diag ike." CLI debug commands include arp, netstat, ping, radius_ping, and tcpdump. In fact, the more we explored the CLI, the more we found. RapidStream seems to be adding to its arsenal of diagnostic aids faster than writers can churn out documentation.

(Back to Top)

Keeping Hackers At Bay
After several e-commerce sites were felled by denial of service (DoS) attacks last year, providers demanded improved protection. Many firewalls now discard badly-formatted packets that crash unprotected servers. Others use thresholds to stop "floods" or packets sent at high volume to consume server resources. DoS attacks cripple or kill servers, preventing legitimate requests from being processed. Distributed DoS (DDoS) attacks originate simultaneously from many sources. The tricky part is differentiating between a nefarious flood and legitimate heavy Web traffic. "Excessive" traffic varies from site to site, month to month, even day to day. To defend your Web site without painting yourself into a corner, look for configurable DoS measures.

Every RSSA incorporates "anti-hacking" features. "DDoS" thresholds limit connection requests to one server (destination) or from one client (source). "DoS" measures block invalid packets (ping of death, IP source route) and packet floods (ICMP, UDP, TCP SYN). Configurable alarms are generated whenever these attacks are detected. These measures offer potent, flexible hacker defense—with unspoken limitations.

We used VIPs to make HQ servers reachable from outside the firewall. We sent UDP datagrams from one public client to one HQ server, exceeding the configured UDP Flood threshold. To our surprise, UDP to reachable servers did not trigger alarms—packets were received at the rate sent. But UDP to unreachable servers triggered DDoS and UDP Flood alarms and subsequent datagrams were discarded by the firewall.

To understand why, consider the RSSA's architecture. The first packet in each session is handled by main CPU. Consecutive packets in that session—those with the same source IP and port—are offloaded to RapidCore. Only packets seen by the main CPU are added to the hacker prevention counters. In our test, UDP to a reachable server created a session (count = 1), while UDP to an unreachable server did not (count > threshold).

Therefore, RSSA thresholds don't throttle a single session. They defend against distributed attacks from multiple IPs, high-rate port scans from multiple ports, and TCP SYN floods appearing to be multiple connect requests. SYN floods choke servers by consuming connection resources. Flooding duplicate UDP/ICMP datagrams consumes bandwidth, but impact at the target varies. For example, most IKE implementations use a cookie hash to reduce degradation caused by UDP 500 floods. In our opinion, RapidStream defends against the most damaging DDoS attacks, but thorough documentation is needed to explain the attacks that are—and are not—covered.

(Back to Top)

Additional Services
Although our lab tests focused on managed VPN services, RapidStream appliances provide single-point policy enforcement for several additional services:

Traffic Shaping and VLAN Tagging: As previously noted, RSSAs implement 802.1Q VLAN tags, weighted fair queuing, Type of Service marking, and port shaping in Mbps. However, VLAN tags are not configurable from CPM, and bandwidth cannot be allocated to individual policies.

Server Load Balancing: The Load Balancing Actions described previously can be used to balance sessions across pools of web, remote access, or other application servers, using (weighted) round-robin, random,or least connection algorithms.

High Availability: A separate HA port is built into every RSSA 1000, 6000, and 8000. RSSA HA uses this port to share IP and MAC addresses between active/standby pairs. Configuration changes are synchronized and heartbeats are exchanged between each HA pair, with failover after three failed heartbeats. Note that the HA pair must be the same model, and that HA is not available on the RSSA 2000.

All of these features are included in standard RapidStream product; they simply become part of the network infrastructure. There are no add-on licenses for anti-virus scanning, content filtering, intrusion detection, or authentication services. When purchasing RapidStream appliances, support contracts are the only incremental cost.

(Back to Top)

Our Experience With Tech Support
Every RSSA ships with a standard RapidCare warranty that allows three months setup support by phone, one year of bug fixes and hardware repair or replacement, and unlimited eSupport that consists of online documentation and problem reporting. The RapidCare Silver program extends the hardware warranty after the first year of service and RapidCare Gold extends post-setup phone support to five days a week for 12 months, and next business day hardware replacement. Platinum service expands phone support to around the clock access 365 days a year. A software subscription service can be added to any RapidCare contract that would cover all software upgrades—minor and major revisions.

Resellers can offer RapidCare contracts at time of sale, with RapidStream providing either direct support or supplementary support at discounted pricing. RapidStream Partners are eligible for additional perks like product discounts, launch funds, sales and marketing tools and pre-sales support. In addition, Gold and Platinum Partners can participate in Seed Unit Programs or apply for market development funds, among other benefits like sales leads, free shipping, and quarterly onsite sales and technical training.

RapidStream's support Web site offers software, documentation, application notes, and frequently asked questions (FAQs). A searchable knowledge base for RapidCare Gold and Platinum accounts is under development. The so-called "appnotes" program is available today. We found the quality of the documentation very useful but light in quantity. During our evaluation, RapidCare eSupport was exceptionally responsive. Initial acknowledgment and advice was consistently returned within hours. Engineering follow-ups were usually received no later than the next day, funneled through a single point of contact. We experienced two trouble reports requiring protracted analysis, but support staff remained attentive and helpful until both cases were closed.

(Back to Top)

Customer Feedback
RapidStream is relatively new to the VPN market—the first RSSA 2000s shipped in mid-2001. RapidStream's growing customer base consists of businesses that range from integrators and enterprises to ISPs and major carriers. We contacted the reference supplied by RapidStream—Deterministic Networks, Inc. Steve Jackowski, Deterministic chief executive officer, said the company was acquired by Gilat Satellite Networks to supply VPN software for Starband, a satellite-based ISP.

Deterministic chose RapidStream after evaluating several VPN gateways last summer. From the start, Gilat's unique requirements narrowed the field. Most VPNs encrypt traffic leaving a private network, entering the Internet. Gilat wanted to encrypt traffic leaving the Internet, entering a private network—on the satellite downlink. "This is backwards from the way you usually think of VPNs," said Jackowski. Some VPN gateways could not accommodate this requirement.

Jackowski explained how Deterministic used simulators to stress-test several VPN gateways. Testers actually exceeded the vendor-specified capacity of the RSSA 6000 (64K sessions, 8000 tunnels).

"We had a hard time getting more than 3,000 or 4,000 tunnels with other products," Jackowski said. "Most of these boxes have been targeted to corporate environments and are not able to scale to the demands of large ISPs."

This highlights how Gilat's business objective differs from our RFP. Gilat is a carrier using VPN technology to protect its own network. The ISP in our RFP is seeking solutions to protect small business networks. That is, inexpensive CPE supporting hundreds of tunnels, not thousands. None of the CPE in our series—including RapidStream's 500, 1000, and 2000 appliances—would meet Gilat's data center needs. Nonetheless, there is much we can learn from Deterministic's experience with RapidStream.

"Of the vendors we worked with during trials, RapidStream was the most responsive and aggressive. They bent over backwards to work with us," Jackowski said. "We knew they were a startup and there's a certain amount of risk with that. But we found they were willing to optimize their product for the environment Gilat needed."

Ultimately, Gilat installed dual RSSA 8000s at each satellite network hub. Each satellite dish includes a preconfigured RapidStream VPN Client. "We'll use CPM to manage the RSSAs because there is a single VPN configuration for all boxes," said Jackowski. VPN Client configurations are static now, but a channel has been left for future runtime updates. "We may also allow people to set up private tunnels for corporate VPNs," said Jackowski.

As it happens, the VPN Client OEM'ed by RapidStream includes a "DNE Adapter" developed by Deterministic Networks, licensed by IRE. Deterministic and IRE collaborated to customize the client for Gilat. One proprietary example created a default tunnel that users can see, but cannot delete, to carry traffic across Gilat's network. Another customized feature accounted for one-way encryption. According to Jackowski, "On a satellite uplink, because of frequency shifting, there's no way anyone can see the data. So Gilat is doing one-way encryption on downlinks, saving overhead on the uplink."

Jackowski said that over the past year Gilat's confidence in RapidStream has grown nearly as much as Gilat aspires to grow its clientele.

"Hopefully, the support we're going to require from RapidStream will be pretty light," he said. "Starband couldn't go public at the end of last year due to changes in the market," explained Jackowski. "Today, the service is in some trials, but is not available for everyone. We expect to see full-scale deployment of Starband by December 2001."

With commercial deployment still in the early stages, like many broadband providers, Gilat has put expansion plans on hold for the time being.

Jackowski's experience paints RapidStream as a strong newcomer, flexible and eager to please. With many VPN startups being swallowed by big companies with even bigger agendas, this is refreshing. On the other hand, startups need to prove themselves in the field. Deterministic is solving a rather different problem, with different platforms, than our RFP. We were impressed after chatting with Jackowski, but would like to have also interviewed an ISP delivering CPE-based managed VPN services.

(Back to Top)

Did RapidStream Satisfy Our RFP?
The objective of this evaluation was to determine how well RapidStream's solution met our RFP's requirements. After hands-on inspection, we are comfortable that nearly all of our RFP's requirements are well satisfied. Functionally, these appliances offer more than we asked for, with just a few exceptions:

• To better satisfy multi-vendor accounts, we'd like to see ICSA VPN certification (now pending).
• RapidStream appliances are not (currently) platforms on which to deploy a la carte services. The most notable omission: content filtering, commonly available in other firewalls.
• To increase revenue opportunities at the high end of our small business market, we would like to see HA support on the RSSA 2000.
• While CPM is really quite promising, it is also new. We would hope to see derivation and logging glitches resolved in the next release.

Next, let's consider upfront cost and revenue potential for each of our RFP scenarios:

1. Entry-level Scenario: The RSSA 1000 just squeaks under our $2000 MSRP cap. We'd consider proposing the RSSA 500, a less expensive solution for a single office with 10-25 employees. In fact, the RSSA 500 is required if we want to assign dynamic public port IPs with PPPoE or DHCP. On the other hand, the RSSA 1000 is required by customers that need 10/100 Ethernet, HA, or more VPN tunnels.
2. High-Tech Office: In scenario No. 2, the original RSSA 2000 proposed by RapidStream is no longer sold. However, the original RSSA 4000, renamed the RSSA 2000, is still a good fit. Our RFP described this customer as willing to pay for value-added services. Built-in traffic shaping and server load balancing could help to land this account, but it there's no obvious way to sell these as after-market add-ons. Instead, the incremental opportunities here are selling RSSA 500s to teleworkers and delivering premium services (e.g., video conferencing) over ISP-to-CPE tunnels.
3. Distributed Mid-Size Business: In scenario No. 3, our customer can take full advantage of RapidStream features like tunnel switching, real-time monitoring, port shaping, and web server load balancing. There's just one drawback: no HA at HQ. Could this customer tunnel from his existing HQ firewall to branch office RSSA-1000s? VPNC branding gives us hope—but we'd rather fully-leverage CPM by deploying an RSSA-only VPN. An RSSA 1000 HA pair at HQ is probably not an option with only 30 tunnels.

Finally, the RSSA 500 is a key ingredient in the all three scenarios. Configuration and monitoring are basically the same for all RSSAs—only installation should be slightly different. Just the same, we would want to test-drive this teleworker appliance before inking a contract.

—End—

Update: Product upgrades released