BlogThis story starts off in an unusual manner. Out of the blue, a call came in from a DoD consulting firm saying they would like to learn more about ExtraHop. Someone within the firm witnessed a team leveraging the power of ExtraHop's wire data analytics and real-time dashboards in another account. This partner has historically depended on traditional NPM/APM tools, but admittedly knew that the scale of the problems they were addressing required, in their words, "a new and innovative approach with technologies to help address chronic issues within DoD's AHLTA and CHCS systems."
Obviously ExtraHop's DoD team was up for the gig!
First Step: Dependency Mapping
Example of an activity map from ExtraHop.
First, we needed to understand what AHLTA was. If you haven't heard of it, AHLTA is the electronic medical record system used by the medical providers of the Department of Defense since 2004. This was no problem for us, since ExtraHop was passively capturing packets and processing every transaction against local and remote clients and every AHLTA server component. Our partners and we stood with resolve knowing that the wire data and our handy dandy activity mapping button would stand as the definitive truth as to what transaction and device dependencies existed.
The result? Our final activity diagram enabled us to sort through misleading information from the dev team based on opinions and sketchy informationand we quickly had others within the agency asking for copies of it since its accuracy was greater than anything they've seen to date. The activity map allowed our partner, and ultimately the client, the ability to make data driven decisions in a way that was never possible for them before.
The dependency mapping was only the tip of the iceberg. Over weeks and months of monitoring, ExtraHop proved to be the MRI of the AHLTA and CHCS. They used ExtraHop to uncover a number of artifacts that could be delivered directly to network and system administrators as well as developers for immediate action. Wire data produced intelligence that pointed to esoteric and seemingly insignificant events at layer 4 and layer 7 that cascaded into larger, service affecting problems.
Valuable For Every Team
Over the months the on-site network teams have come to rely on ExtraHop as well. They have primarily relied on a NetFlow-based tool for visibility up to L4 until they realized the power of the ExtraHop platform deployed by the contractor. One day the amount of CIFS traffic showed a spike between two server devices within their main data center. Although the NetProfiler solution showed this spike tied to CIFS (UDP ports 137 and 138, TCP ports 139 and 445), it could not provide context around the flow such as which user (LDAP) was involved in the transfer, what type of CIFS event was taking place, and which files were involved.
The network analyst pivoted over to the ExtraHop dashboard, noticed the same characterization of the CIFS elephant flow at the network layer, and quickly pivoted into a search query and in under a minute had access to the user, noticed that it was an upload, and the filename was "backup.sql."
Quickly the analyst surmised that the user, an admin for the sending server, was performing a backup of his SQL database in the middle of the day at 1pm!
That's Just The Beginning
After this firm deployed ExtraHop, for the first time, all their teams had a single source of data and a holistic platform they could use to assess and solve problems, and they keep finding more ways to get value from it!
Right from the start, we were able to provide valuable insight and troubleshooting into:
- Citrix and other VDI host transactions
- HL7 (Prescriptions and order errors.)
- Oracle (Queries to both the local hospital cache database as well as the remote master Oracle database supporting these and the other 100+ sites.)
- HTTP (including AHLTA SOAP/XML transaction details that could be correlated back to the errors - they needed full stream reassembly to do this.)
- DNS
- CIFS (Imaging)
- TCP stack (Implicit problems that synthetic and bandwidth monitoring tools couldn't see because you needed to be able to replay the TCP state machines to find them.)
- And more!
Because of the value provided by both the consultant and the ExtraHop wire data platform this short term assessment has resulted in a plan to leave ExtraHop deployed within these hospitals for long-term continuous monitoring and troubleshooting. DoD healthcare now has an MRI in place for their AHLTA, CHCS, and any future EHR systems and ExtraHop is proud to be of service!
Learn more about the capabilities our DoD clients have found valuable with the following resources:
