3 Steps to Stop the Bleeding

Heartbleed is one of the worst security vulnerabilities in recent memory for organizations trusting OpenSSL to protect their data. Here are three easy steps to protect your business.

1. Get F5 BIG-IP

2. Deploy

3. Relax

#Heartbleed is likely one of the worst security vulnerabilities of all time. That isn’t just a subjective statement—Heartbleed (CVE-2014-0160) has an objective Common Vulnerability Scoring System (CVSS) score of 9.4 (out of 10). Worse, the versions of the popular OpenSSL library that make Heartbleed possible have been widely available for more than two years, meaning that the vulnerability has spread widely throughout the Internet.

Worse still, for those last two years, any attackers aware of Heartbleed’s existence could have used simple, untraceable messages to retrieve memory values from vulnerable servers.

Figure 1: A Heartbleed probe

Figure 1: A Heartbleed probe

Compromised assets could have included SSL private keys, user passwords, and other highly sensitive information. See here and here and here for more technical analysis of the vulnerability itself.

Put simply, Heartbleed is very close to a worst-case scenario.

The good news

heartbleed mitigationThe good news from F5 Networks is that if an F5 BIG-IP Application Delivery Controller (ADC) has been delivering your HTTPS application for the last two years, that application was not vulnerable to Heartbleed. There’s still work that needs to be done on behalf of F5 customers and others, but we’re beginning the transition to the post-Heartbleed world—including immediate countermeasures and next steps.

Internet Impact Analysis

After the Heartbleed vulnerability was announced, Netcraft performed an analysis based on key parameters of the threat surface. Their analysis suggests that the exposure to Heartbleed today is around 15 percent of all SSL sites. That is over a half million private keys.

Of course, different SSL keys have different levels of protection based on their values. The security of an online banking site’s private key is more significant than, say, that of a news aggregator’s private key. Root certificate authority keys are the most prized in the world.

So who is vulnerable?

The initial Heartbleed announcement indicated which sites are likely safe and which are not.

”Fortunately many large consumer sites are saved by their conservative choice of SSL/TLS termination equipment and software. Ironically smaller and more progressive services or those who have upgraded to latest and best encryption will be affected most.” —Heartbleed researchers

It’s pretty simple to figure out which sites will likely be the most vulnerable: those using recent versions of open-source software servers (such as Apache and NGINX). Those sites and most others can be grouped into the following risk categories based on the information in the Heartbleed announcement, the Netcraft analysis, and my own personal engagements with hundreds of SSL customers.

Group

Likely Heartbleed Risk

Why?

Financial

Lower

Use of FIPS 140 crypto-HSMs

Defense

Lower

Use of FIPS 140 crypto-HSMs

SLED

Medium

Lots of older, non-Linux systems

eCommerce

Medium

See F5 status below

Web 2.0

High

Likely to use affected OpenSSL libraries

Cloud

High

Most likely to use affected OpenSSL libraries

Figure 2: SSL site verticals generally at risk

 

F5 ADCs are a Heartbleed countermeasure

Let’s get to the good news. There are three ways that F5 BIG-IP devices or software can be used as a countermeasure for Heartbleed.

 

First: Normal SSL profiles already protect against Heartbleed

The first defense is via the native SSL termination ability of the BIG-IP platform’s SSL profiles. The vast majority of sites using BIG-IP Local Traffic Manager (LTM) to terminate SSL for their applications are already protected against Heartbleed. This is because the primary SSL engine within the F5 traffic management microkernel is our own highly optimized and hand-written code. Therefore, when the F5 platform is terminating SSL on behalf of a pool of servers, it is already providing the necessary protection to applications. These applications are what the Heartbleed researchers meant when they referred to “fortunate consumer sites” that were spared vulnerability—those whose businesses depend on application uptime and who have invested in SSL termination equipment (read: SSL-terminating load balancers) for capacity and provisioning.

 

Figure 3: How BIG-IP LTM protects against Heartbleed

Figure 3: How BIG-IP LTM protects against Heartbleed

 

Second: Hardware security modules provide cryptographic boundaries

The SSL keys that are likely safest are the high-value keys of the financial industry, the defense industry, and the certificate authorities. These organizations often keep their keys within a FIPS-140 Level 3 cryptographic boundary called a hardware security module (HSM) that keeps the keys safe from memory-cracking tools. Many such organizations use ADCs with an integrated HSM.

 

Figure 4: Network HSM architecture

Figure 4: Network HSM architecture

Interestingly, there’s a whole new network HSM architecture where SSL keys are stored within FIPS 140 cryptographic stores under centralized management. The ADC asks these devices to decrypt SSL session keys, but then performs the remainder of the SSL session cryptography themselves.

 

Third: F5 iRules protect the rest

An F5 ADC has something that no other ADC has: a fully programmable, data-plane scripting language. Called iRules, it is supported by the F5 DevCentral development community made up of over 100,000 active users.

For organizations that cannot use either of the two more conventional solutions for SSL termination above, the third option is to use iRules to provide mitigationwithout decrypting the SSL connections themselves.

Scenarios where a defense based on iRules might make sense include:

  • A pool of servers with a security policy restricting private keys to origin server hosts.
  • An ingress point for multi-tenant SSL sites (using SNI and different keys).
  • A virtual ADC fronting a pool of virtualized SSL sites.

For these cases, multiple groups in the F5 development community are now working with iRules to mitigate the Heartbleed vulnerability. The problem space is a fascinating mix of false-positive avoidance, performance assurance, and security, which must all be balanced within a single rule that cannot inspect the plaintext.

One of the iRules under development monitors server responses for TLS heartbeat responses and then closes the connection if those responses appear to be leaked data. (Any response larger than 128 bytes is suspect.)

Another iRule closes any connection where the client negotiates use of the TLS heartbeat extension, thus rejecting all attack clients while allowing all current browsers.

https://devcentral.f5.com/articles/ssl-heartbleed-irule-update

A third iRule watches for the unencrypted TLS heartbeat probes and drops them. For customers using virtualized ADCs, this countermeasure is far cheaper to compute than performing an entire SSL handshake.

If none of these iRules suffice, an organization can develop its own. It happens every day. That’s the power of iRules and the DevCentral community.

 

What about the cloud?

The savvy reader may have realized that cloud computing has its own Heartbleed threat profile. This is true, with the intensity of the threat varying by the specific cloud deployment situation.

A typical LAMP cloud site: A typical cloud site contains instances of a Linux distribution running Apache, MySQL, and PhP (a combination commonly known as LAMP). Given that cloud adoption is a relatively recent phenomenon that overlaps with the introduction of the Heartbleed vulnerability two years ago, these cloud LAMP sites incur the highest risk.

A cloud with F5 ADC hardware: Many F5 customers use our hardware platform to front their cloud instances. They achieve significantly higher densities with an F5 ADC merging TCP connections, caching common objects, and, of course, offloading expensive hardware cryptographic operations. Where F5 is terminating the SSL, those customers are already protected. If you’re one of them, you should simply make sure that none of the virtual servers are passing HTTPS without decrypting it first. (See below).

An F5 virtual ADC: In fully virtualized clouds, even the F5 modules are virtual instances. Without the underlying cryptographic hardware offload taking place, there may be less incentive to have the F5 module terminate the SSL for HTTPS servers. The Heartbleed risk is therefore higher than with the hardware deployment in the previous case. The F5 platform can still provide protection, however, because the software implementation of the virtual ADC’s native SSL stack is also not vulnerable to Heartbleed and therefore provides a defense.

 

Next Steps

Existing F5 customers can basically read a single solution to double-check the health of their F5 devices and sleep tight knowing that not only are they safe now, but their applications have been resistant to Heartbleed for the last two years.

What about those who aren’t F5 customers? Now that we’ve looked at three different ways the F5 platform can mitigate the Heartbleed bug, here are three steps for putting that protection in place for the rest of the world.

 

Three easy steps for those new to F5

  1. The first step is easy: Obtain BIG-IP LTM. It’s available from any integrator, directly from F5 Networks, or as a free trial download of the virtual edition with an evaluation license. The 30-day trial should provide enough coverage to help you get through the initial Heartbleed crisis. A 90-day trial is also available.

 

 

  1. Identify and map vulnerable SSL sites. Tally the sites and applications on the network. Run nmap on the network to find them all, and collate the results if necessary. Once all the 443 applications have been found, create a virtual server to either terminate the SSL for each application (even if the traffic is re-encrypted) or attach one of the HeartBleed iRules. 
  2. Sit back and relax. Maybe have some cake. Get a good night’s sleep.

 

Conclusion

American computer scientist Gerald Weinberg once said, “If builders built buildings the way programmers wrote programs, the first woodpecker to come along would destroy civilization.”

Heartbleed is one of those woodpeckers.

The good news is that the F5 BIG-IP devices and virtual instances already spread throughout the Internet can defend the HTTPS applications they serve. Administrators can verify that defense and then work with their security teams to upgrade systems at their own pace. And those without protection can get it with a simple call to F5 or an F5 trial edition.

Questions about OpenSSL Heartbleed? Ask the F5 Community

3 Steps to Stop the Bleeding

Heartbleed is one of the worst security vulnerabilities in recent memory for organizations trusting OpenSSL to protect their data. Here are three easy steps to protect your business.

1. Get F5 BIG-IP LTM

2. Deploy

3. Relax