How to Recognize Malicious Network and Port Scanning

Network scanning and port scanning—processes for learning about a network's structure and behavior—aren't inherently hostile, but bad actors often use them to conduct reconnaissance before trying to breach a network.

How does malicious scanning work, and how you can detect potential reconnaissance of your network? Let's start by defining the terms at their most basic:

  • Network scanning involves detecting all active hosts on a network and mapping them to their IP addresses.
  • Port scanning refers to the process of sending packets to specific ports on a host and analyzing the responses to learn details about its running services or locate potential vulnerabilities.

Methods of Network Scanning for Host Discovery

Host discovery, the process of determining what systems on a network are up and listening, is often the first step in a hostile network scanning action. Two protocols are most commonly used for host discovery: Address Resolution Protocol (ARP) scans and several types of Internet Control Message Protocol (ICMP) scans.

Since individual ARP requests are used to map IP addresses to MAC addresses on a local subnet, ARP requests can be sent out to many IP addresses on a Local Area Network (LAN) to determine which hosts are up based on the ones that respond with an ARP reply.

For network scanning outside of a local subnet, several types of ICMP packets can be used instead, including echo, timestamp, and address mask requests. Echo (or ping) requests are used to detect if another host can be reached, while timestamp messages determine the latency between two hosts. Address mask requests are intended to discover the subnet mask in use on the network.

Host discovery for each ICMP message type depends on receiving a corresponding reply from available hosts. If no response is received, it means either that there is no host listening at that address, that the request packet was blocked by a firewall or packet filter, or that the message type isn't supported by the destination device. ICMP echo requests that originate outside an internal network are commonly blocked by firewalls, but timestamp and address mask requests are less likely to be blocked.

Methods of Port Scanning

Once available hosts on a network have been found via networking scanning, port scanning can be used to discover the services in use on specific ports. In general, port scanning attempts to classify ports into one of three designations:

  • Open: the destination responds with a packet indicating it is listening on that port, which also indicates that whatever service was used for the scan (commonly TCP or UDP) is in use as well
  • Closed: the destination received the request packet but responds with a reply indicating that there is no service listening at the port
  • Filtered: the port might be open, but the packet has been filtered out by a firewall and dropped, so no reply is received

Types of TCP Scans

As previously mentioned, TCP and UDP are frequently the protocols used in port scanning. There are several methods of performing TCP scans:

SYN scans, the most common form of TCP scanning, involve establishing a half-open connection to the target port by sending a SYN packet and evaluating the response. The host replies by sending a SYN/ACK packet if the port is open or a RST response if the port is closed. It is also possible for a closed port to reply with an ICMP port unreachable message instead of a RST packet, though this is less common. A lack of any reply indicates that the port is filtered.

A higher level method of TCP scanning is the TCP connect scan, in which the scanner tries to connect to a port via TCP using the connect system call and the full TCP handshake process. This method is utilized less often than SYN scanning, since it requires more overhead in terms of packets and time and is more easily detectable.

NULL, FIN, and Xmas scans are three scan types that involve manipulating TCP header flags. Each of them results in a RST (or ICMP port unreachable) packet from a closed port and no response from an open or filtered port, and they require that the SYN, RST, and ACK bits are not set. NULL scans send packets with no flags set in their headers, while FIN scans have only the FIN bit set. Xmas scan packets, so called because their headers are reminiscent of being "lit up like a Christmas tree," have the FIN, PSH, and URG flag bits turned on.

Download the TCP Optimization White Paper to better safeguard TCP/IP.

UDP Scanning

Like TCP scans, UDP scans send a UDP packet to various ports on a target system and evaluate the response to determine availability of the service and the host. Receiving a UDP packet in response indicates that the port is open, while an ICMP port unreachable error response signifies a closed port. If no response is received, the port could either be open or filtered by a firewall or packet filter.

How to Detect Network and Port Scans

Scan detection methods range from monitoring for simple thresholds and patterns, such as number of ports connected to from a single origin over a period of time, to probabilistic models based on expected network behavior. Network intrusion detection systems and firewalls are usually configured to detect scans, but scanners can attempt to avoid some common detection rules by altering their scanning rate, accessing ports out of order, or spoofing their source address.

ExtraHop user? Here's the free bundle that identifies instances of network and port scanning by leveraging detailed metrics about the protocols most often used and patterns typical of scanning attempts. The bundle also tracks relevant metrics over longer periods of time to account for attackers that intentionally perform slow, drawn out scans to avoid detection.

New player? See how ExtraHop Reveal(x), our award-winning security analytics product, helps SecOps proactively detect and hunt down bad actors. Launch the online demo now!

Subscribe to our Newsletter

Get the latest from ExtraHop delivered straight to your inbox.