2024 Global Cyber Confidence Index

Arrow pointing right
ExtraHop Logo
  • Productschevron right
  • Solutionschevron right
  • Why ExtraHopchevron right
  • Blogchevron right
  • Resourceschevron right

Arrow pointing leftBlog

Detect Malware in Encrypted Traffic for Improved Security Visibility

Jesse Munos

September 28, 2021

According to the Ponemon Institute's 2021 Global Encryption Trends Study, 50% of organizations have an encryption plan consistently applied across their entire environment—up from around 40% in 2015 and 25% in 2010. While encryption is providing an increasingly critical defensive layer against tampering and eavesdropping, it is also an increasing threat as demonstrated by the 260% increase in encrypted attacks reported by Zscaler.

To restore visibility, security vendors have introduced decryption appliances and inspection techniques aimed at restoring visibility into these traffic flows and detect threats like malware in encrypted traffic. There are two common approaches to restoring traffic visibility:

  • Proxies that terminate SSL/TLS sessions, inspect the clear text traffic for a variety of indicators of compromise (IoCs), then re-encrypt the traffic and send it along to the intended destination. This approach is also known as the man-in-the-middle technique (MITM).
  • Statistical modeling algorithms that can detect encrypted command and control traffic, known malware, interactive shell sessions, and more. These encrypted traffic analysis (ETA) engines look at everything from source and destination to packet size, JA3 hashes, and entropy within the TCP session to identify attacks and alert security operations (SecOps) personnel to potential security breaches.

While both of these approaches have merit and aim to restore visibility and confidence in the concept of network threat detection, both solutions are far from restoring SecOps visibility to the Pre SSL/TLS days.

Problems with Existing SSL/TLS Decryption Solutions

Every security solution comes with its own tradeoffs, and network traffic inspection and decryption is no exception. In terms of traffic inspection and security visibility, these tradeoffs come in two primary categories:

  • Cost of implementation
  • Limited visibility

While endpoint users likely never notice the computational cost of encryption and decryption, performing these operations at scale for high volumes of traffic is computationally expensive. In fact, an NSS Labs study found that simply enabling SSL/TLS decryption degraded the next-generation firewall (NGFW) performance by as much as 80 percent and reduced transactions per second by as much as 92 percent. As such, organizations looking to enable decryption on NGFW appliances must spend significantly more for more capable appliances. NGFWs, intrusion detection systems (IDS), and proxy systems are typically designed and deployed to monitor traffic crossing gateway devices or specific subnet boundaries. Deploying these technologies to monitor intra-network traffic requires purchasing additional appliances and incurring the increase in costs of ownership.

Although traffic decryption allows a variety of security layers to inspect traffic, the architectural positioning of these solutions within networks often leaves them blind to large quantities of traffic. After all, these solutions cannot inspect traffic that does not traverse the appliance. As such, SecOps teams remain blind to the majority of east/west traffic, both clear-text and encrypted, which is key to identifying advanced attackers moving laterally within an organization.

NGFW, IDS, and proxy appliances all suffer from a mutual shortfall in the fidelity of the logging data they provide. While current generation solutions provide significantly better log data than their predecessors, the data available from these solutions is generally lacking in the fidelity necessary to easily identify and correlate attack patterns beyond single traffic flows. They're often entirely blind to the context necessary for attack-campaign correlation, even when decryption is enabled. The result is analyst time spent investigating, using multiple tools, trying to scope attacks and ensure proper response actions are taken.

The ExtraHop Solution

ExtraHop designed Reveal(x) decryption capabilities to avoid the shortcomings of past decryption approaches, using its native deployment architecture to its advantage. Reveal(x) is able to perform decryption out of band, neatly sidestepping the throughput, latency, and transaction issues that other solutions suffer from. Additionally, since Reveal(x) is typically deployed to monitor east-west traffic, advanced attackers are unable to hide using living-off-the-land techniques that normally hide their actions within natively encrypted protocols. This east/west visibility is further enhanced by the decryption and parsing of Microsoft Active Directory protocols such as Kerberos, LDAPs, MSRPC, etc., ensuring that Reveal(x) threat detection AI has all the data needed for early attack detection, coupled with the relevant context needed for rapid response.

Reveal(x) understands the context of threats due because it provides the highest fidelity network transaction record data available in the industry. Combining data fidelity with 90 days of historical retention and continuous packet capture provides the necessary foundation to fully understand the context of detections and the historical information to stop attacks in their early stages. It allows defenders to understand how they made it in and see exactly how far they got.

With Reveal(x), you can simultaneously restore security visibility to pre-encryption days and replace a variety of legacy systems such as IDS and network performance monitoring (NPM), simplifying a costly security portfolio and streamlining analyst workflows.

What About Privacy?

When done well, decryption does not compromise privacy. As SANS put it in their 2021 SOC Survey, "In our follow-on questioning, we heard that the most common reason for not using TLS intercept is corporate concern over regulations and privacy. However, no one using TLS intercept reported running into legal issues. One respondent pointed out the need for management education in this area. On the technology side, however, performance issues with inline TLS decryption were mentioned." Further support for thoughtfully-designed, out-of-band decryption.

Take Back the Advantage with Decryption

Encryption technologies are here to stay. The obvious benefits of encryption far outweigh the potential downsides of clear-text protocols. As network traffic encryption reaches ubiquity, security organizations looking to guard against the increasing risks of cyber attacks and advanced attackers must be able to identify and respond to threats leveraging encryption to hide nefarious actions. Reveal(x) provides the decryption capabilities necessary to maintain visibility, the high fidelity data and AI-based threat detection needed to identify attacks, and the integrations required for rapid response.

Explore related articles

Experience RevealX NDR for Yourself

Schedule a demo