The network and security operations team needed to know if their load-balancers, servers and applications deployed with TLS were vulnerable to the Heartbleed bug. They needed to know which systems were affected, where they were, and the origins of the attacks so they could block those attempts while they patched their systems. Of primary concern was whether any internal systems had become compromised. Although firewalls were updated and servers were patched a growing concern was that internally compromised systems could be used to launch Heartbleed attacks from the inside.
Heartbleed, a vulnerability in the OpenSSL cryptography library, affected roughly 17% of all secure web servers at the time of its disclosure in April 2014. The bug was named "Heartbleed" after the TLS Heartbeat extension that it exploited. This extension is often enabled by default, making both clients and servers vulnerable. Vendors quickly pushed out security updates, but the ubiquitous use of OpenSSL in TLS-enabled websites and services meant that 1.5% of the 800,000 most popular websites were still vulnerable a full month after Heartbleed's disclosure. Given the prevalence of SSL for web applications and the fact that TLS could be enabled on servers as well as other infrastructure like load-balancers, tracking which applications and supporting infrastructure that is affected seemed daunting.
With thousands of servers and infrastructure systems, they did not want to perform an extremely inefficient prevention and analysis by packet-filtering and customizing logs at the firewall. Filtering and logging are computationally expensive and can impact user performance and application availability as well as blocking legitimate uses of the TLS Heartbeat extension.
- Automatically discover which hosts and services across all of their applications and infrastructure were vulnerable to Heartbleed
- Determine whether malicious entities were trying to access the site and identify them by IP and geolocation
- Continuously monitor after remediation to be sure that previously decommissioned servers and infrastructure didn't accidentally come online introducing the vulnerability
Vendors quickly pushed out security updates, but the ubiquitous use of OpenSSL in TLS-enabled websites and services meant that 1.5% of the 800,000 most popular websites were still vulnerable a full month after Heartbleed's disclosure. Given the prevalence of SSL in HTTP-based deployments, tracking which applications might be affected becomes a difficult task.
Early morning the day of the Heartbleed announcement, the financial organization's Principal Security Analyst and Senior Network Architect wondered if they could use ExtraHop to rapidly detect Heartbleed attempts and discover all vulnerable systems. They had been a customer for over six months and had used the ExtraHop platform to discover, view, and audit all SSL activity across their application portfolio of over 800 applications mainly to provide audit reports for PII data.
They rapidly built a dashboard with a view of all SSL and TLS version session analysis by host and client as well as all certificates and ciphers in use which are native "out of the box metrics" provided by ExtraHop. They were able to immediately identify which hosts in the network could be vulnerable to malicious traffic. That same morning a two hours after the announcement, ExtraHop published the industry's first Heartbleed Detection bundle free to the ExtraHop Community site. A notification went out to all customers and the Security Analyst downloaded and applied it to their ExtraHop platform. The Heartbleed Detection bundle's dashboard built upon ExtraHop's native TLS / SSL analysis but included an Application Inspection (AI) Trigger to record whenever a TLS Heartbeat record was observed and to store the client IP and the Common Name (CN) from the x509 certificate so a customer would know where each attempt came from and where it was destined. The bundle also incorporated ExtraHop's Geomap feature to visualizing malicious access attempts and target systems on a world map.
ExtraHop natively collects many SSL and encryption attributes, like the TLS Heartbeat. The customer used the Heartbleed Detection bundle to see both current and past Heartbleed attempts on their systems.
They watched in real time as dozens of attempts targeted their applications. The security incident team used this information to immediately set blocking policies for those clients while they patched their vulnerable systems.
It took less than three hours from the time of the Heartbleed announcement to identification of all vulnerable systems, real-time analysis of past and current attempts by client, to remediation efforts - all at no additional product cost to the organization. The Principal Security Analyst said it best, "I didn't realize we had purchased a platform for pervasive security monitoring when we first bought ExtraHop. We originally bought you for Citrix analysis. This is the the most extensible monitoring platform I have ever seen."
The most important value measurement for this team was time. "In the security world, time is the currency that counts. The longer it takes to identify and act, the higher probability your business gets hit big and you lose your job." They also said that no other platform, that they are aware of, would be able to provide a real-time and globally observed view of all client, network, and application behavior and encryption analysis across their entire application and infrastructure portfolio. They indicated this unique perspective based on wire data analysis allowed much better situational awareness which translated to a fast and focused response.
Packet filtering at the firewall was seen as a last resort and would have cost them dearly in time and user performance. Instead, the ExtraHop platform automatically detected and classified all traffic sources and SSL server targets and continues to perform that analysis to this day.
Both the Principal Security Analyst and Network Architect were promoted a few months afterward for demonstrated leadership, agility and innovation at their company.