In supply chain attacks, the kill chain starts with the supplier. If the attack on SolarWinds was detected early enough, SUNBURST operators and their malware programs would not have succeeded in accessing the SolarWinds build servers and uploading malware to SolarWinds customers through a trusted update. A supply chain kill chain would stop it there, but if not then, at the time the Orion patch update tried to download. If not there, then at testing, which should include a sandboxing component to detonate anything in a mock operational environment. Beyond that, a supply chain kill chain would block C2 activity before it establishes and starts sending out sensitive data.
See Figure 5, below for the supply chain kill chain.
Figure 5. Supply Chain Kill Chain
As of this writing, SolarWinds still doesn't know how the attack originated in their systems before it spread to the Orion update build servers. If the attackers managed to move laterally inside SolarWinds, they should have stopped them before they accessed the company's sensitive development servers. Unfortunately, that did not happen. The attackers infiltrated the SolarWinds build environment and SUNBURST Trojans were hidden in Orion updates signed by legitimate SolarWinds keys. In the supply chain attack kill chain, it's important to stop this type of attack from spreading (through a trusted patch or update to other devices in the network) and communicating to the external C2 server.
Show DNS Some Love
For a long time, DNS was hand curated, and manually edited ("named.conf" and zone files to ensure proper DNS resolution and PTR records, for example). Over the last few years, point-and-click DNS tools have made this administration easier but at the cost of proper DNS hygiene. Visibility into these misconfigurations is extremely difficult to accomplish with logs and windows events. As stated earlier, this difficulty in monitoring and the resulting lack of visibility has made DNS a preferred avenue for C2 communications as well as exfiltration. In the case of SUNBURST, programmatic decisions were made based on the returned values.
For this reason DNS needs proactive administration and protection. Shore up DNS servers by keeping them up to date, patched, and supported with security technologies like DNSSec (to prevent DNS reflection attacks) and DNS monitoring. For tighter control, restrict zone transfers, and turn off DNS recursion (enabled by default on most BIND servers on all major Linux distributions) to prevent poisoning attacks, among other best practices. Another emerging practice is to encrypt DNS traffic (often referred to as DNS over HTTPS (DoH)), but this is a two-edged sword because it blocks visibility to attackers and the host organization.
Network detection and response (NDR), unlike events or syslogs, helps identify and locate specific conditions that violate DNS hygiene. Examples of this include the aforementioned, unapproved DNS servers, DGA-like names, suspicious TLDs, and mismatched DNS answers (domain hijacking).
See What Changed
Know your typical DNS traffic and keep historic files for future lookup and comparison. With passive DNS, IT staff can flag changes to a domain's A record, for example, without alerting attackers because it's not happening live on their infected systems. DNS activity (traffic, queries, domains, and IPs) should be inspected for changes and anomalous behaviors in order to detect sophisticated C2 activity.
In the SUNBURST attack methodology, anomalies to look for include:
- A sensitive server like the Orion network management server contacting an external host for the first time.
- Connections from internal systems to external domains happening after hours.
- Unusual connections between internal systems (for example DevOps doesn't need access to the accounting system).
- Connections to external countries and locations outside routine DNS traffic and queries.
- Abnormal file reads.
- Increased SSH traffic with remote execution commands.
Decode DNS Traffic and Compare
If the traffic and queries seem suspect for any of the above reasons (or others not on the list), then it's time to pull the data aside and compare it to historic DNS data for deeper inspection. It should be decoded to metadata for keywords and suspicious domains as well as data leakage.
Once C2 channels are established and the secondary connection is made over HTTPS, it's difficult to determine what packets contain DNS requests or DNS responses and what domains and IP addresses were requested. This means a network visibility solution must be able to decrypt and see into suspect traffic.
Using passive queries to search for changes, looking into the metadata to find keywords indicative of C2 communications, and comparing the metadata in previous snapshots of time are all processes that should be automated to improve detection and response.
Detect Domain Name Trickeries
This includes looking for misspelled and inaccurate domain names. For example, the SUNBURST attackers used a common technique called typosquatting to make domains seem legitimate and route victims through to the primary C2 server. Things to look out for include:
Domain extensions outside the domain's normal geographic area, such as .au or .io that shouldn't be in the region they represent.
Domains that look legit but use one or more letters out of place or add characters, such as in the SUNBURST case using avsvmcloud[.]com rather than awscloud.com.
Appearing to sync to benign registrars or platforms but upon closer inspection are scammy and questionable.
Subdomains point to generic sync interfaces, such as appsynch.api and appear as REST-based mobile app domains.
Correlations between network traffic, historic metadata, DNS connections and behaviors, along with drill-down capabilities into suspect traffic helps investigators know what to block and helps them build to the short list, focusing further investigation. From there, they can block the C2 channels and more easily locate impacted devices and make repairs. (See how ExtraHop tracked SUNBURST through metadata and other advanced monitoring techniques in the addendum provided by ExtraHop.)