The backbone of our technology is the real-time stream processor that transforms unstructured packets into structured wire data at up to 100 Gbps. Architected for parallel processing, the stream processor splits processing tasks across multiple computing cores — and will scale as more cores are added to new generations of server processors — so you get deeper insight at a fraction of the cost per Gbps of analysis compared to other real-time analytics platforms.
Once the real-time stream processor receives a copy of network traffic from a tap or port mirror, here's what goes on beneath the hood:
1. Line-Rate Decryption
The stream processor decrypts SSL/TLS-encrypted traffic, including cipher suites that support perfect forward secrecy, at line rate with native hardware acceleration. This bulk decryption can scale to 64,000 SSL TPS using 2048-bit keys, which no other real-time analytics can match in a single unified appliance. Check out this technical brief for specifics on our decryption methodology.
2. High-Performance TCP State Machines
Starting at the most fundamental level, the stream processor recreates the TCP state machines for every sender and receiver communicating on the network. A prerequisite for deeper application-protocol and universal payload analysis, this allows the platform to understand all TCP mechanisms and their impact. Because TCP is where the network and application meet, this approach helps you clearly identify whether problems are a network or an application issue right from the start.
3. Wire-Protocol Decoding and Full-Stream Reassembly
The real-time stream processor decodes IP-based protocols (skip to Protocols We Decode) in order to understand, define, and act on that protocol's unique application boundaries. This allows the processor to construct complete flows, sessions, and transactions for total application fluency, which in turn allows for higher-order content analysis through full-stream reassembly into wire data (derived from the wire protocol itself).
While in a perfect world this would all run pretty smoothly from start to finish, in reality traffic patterns like microbursts might result in packet loss from the tap or SPAN; in those cases the processor will automatically resynchronize and recover.
4. Full-Content Analysis
After reassembling packets into full streams, the stream processor analyzes the payload and content from layers 2-7, auto-discovering and classifying any device or client communicating on the network. The processor also continuously maps the relationships between all clients, applications, and infrastructure communicating on the network with over 4,700 metrics measured and recorded out-of-the-box.
Full-content analysis supports dozens of protocols, providing key performance indicators such as database methods used and their process time, file access by user, storage access time and errors, DNS response time and errors, web URI processing time and status codes, SSL certificates with expiration, and load-balancer and firewall latency. The stream processor also gathers sophisticated network metrics such as receive-window throttles, retransmission timeouts, and Nagle delays.
We get that not everyone is interested in knowing every detail about every layer of their environment, however, so don't worry—while the full analytics capabilities are always available to you, it's also easy to tailor your experience so you only see the precise metrics and insights you need.
5. Fully Programmable Insights
Once the stream processor has done its thing and begun supplying wire data metrics, it's time to take control of which insights you see and at what depth.
ExtraHop uses an event-driven programmable interface called Application Inspection Triggers to connect you to the stream processor and all stream transactions. Triggers allow you to programmatically extract wire data events and correlated metrics that are specific to your business, infrastructure, network, clients, and applications.
The same principle and functionality holds true for all of our natively decoded protocols. You can also use triggers to extract, measure, and visualize data from defined fields, or to decode proprietary protocols based on TCP and UDP.