Ruslan Rakhmetov, Security Vision
The vast majority of modern cyberattacks are carried out via data networks - the Internet, local networks, clouds, and OT networks in industry. To effectively detect and respond to cyberattacks, a large number of protection tools are used, which can be divided into network, host and cloud-based. Network-based defences monitor and analyse traffic, host-based defences secure endpoint devices, cloud-based defences protect cloud infrastructures (published applications, hosted data, integrations). Among network defence solutions, the best known are products of the following classes: Firewalls and Security Gateways (including NGFW, WAF, SWG, UTM), Network Access Control (NAC, Network Access Control), DDoS attack protection systems, Zero Trust Network Access (ZTNA, Zero Trust Network Access), Secure Edge Access Platforms (SASE, Secure Access Service Edge), as well as Network Traffic Analysis (NTA), Network Detection and Response (NDR), Network Intrusion Detection / Prevention Systems (NIDS/NIPS, Network Intrusion Detection / Prevention System). Below we describe the main functional features, as well as similarities and differences of NTA, NDR and IDS tools.
1. Network Traffic Analysis (NTA) systems.
NTA class solutions provide detection of cyber attacks in traffic by detecting suspicious and malicious network connections (e.g. exploits, VPO downloads, C&C connections, exfiltration of stolen data), potential malicious devices in the network, and deviations from normal (typical for a given infrastructure) traffic patterns. NTA class devices can analyse network traffic on the fly and record network traffic fed to them to support cyber incident investigations. Information can be transmitted to NTA either via flow messages (NetFlow v5/v9, IPFIX, sFlow, jFlow protocols), or by transmitting a copy of network traffic to NTA (SPAN / RSPAN / ERSPAN technologies, TAP connections), or using ICAP protocol. NTA solutions are used to detect hidden cyber threats and IS incidents that were not detected by other methods, detect malicious traffic, detect data leaks and abnormally high traffic consumption, as well as to diagnose and detect problems with network infrastructure and network equipment. Additionally, NTA solutions can use Deep Packet Inspection (DPI) technology to accurately identify the type of traffic and applications being used, which improves the accuracy of network threat detection.
NTA devices can act as black boxes for the infrastructure: firewalls may be overloaded or compromised at the time of a cyberattack, so it's better to get information about network connections prior to the incident from a reliable source, which is NTA. Recording traffic, especially if it is transmitted in ‘raw’ form (e.g. through a SPAN port), can be quite resource-intensive, especially in terms of information storage and processing large incoming data streams, so one of the main characteristics when choosing an NTA solution will be the cost of on-board storage, the possibility of its expansion, licence conditions on the depth of data storage, as well as the bandwidth of the device. By storing data in NTA, IS administrators are able to perform retrospective analysis of the collected information to search for indicators of compromise, suspicious connections, detect new network services (applications) and devices that have connected to the corporate network.
The main distinguishing characteristics of NTA class solutions are:
1) Principle of operation (flow messages, copy of network traffic, ICAP): work with flow and ICAP is faster, but gives less useful information, and to analyse a full copy of network traffic received via SPAN / RSPAN / ERSPAN requires high computing power, which is paid off by the detail of the results obtained.
2) Ability to analyse external and internal network communications: it is important to analyse not only network traffic from/to the corporate network (so-called north-south traffic crossing the perimeter of the company's network infrastructure), but also internal network traffic in the corporate network, including data transmission between subnets, branches, remote offices (so-called east-west traffic).
3) Ability and depth of traffic recording, on-the-fly traffic analysis: long-term storage of network traffic will help to investigate older incidents and report to regulatory authorities, and on-the-fly traffic monitoring will allow to detect threats at the moment of their realisation.
4) Diversity of supported network protocols: for monitoring enterprise and cloud infrastructures, support for the most popular L2-L7 network protocols will be sufficient, but industrial automation systems and OT infrastructures will require support for technological protocols, often proprietary. The list of supported protocols will determine the applicability of a particular NTA product to protect enterprise, cloud, industrial networks, as well as the ability to detect new hosts in the network and monitor the operation of IoT / IIoT devices, cyber-physical and industrial systems. In addition, it is important that the NTA solution supports analysis of encrypted traffic - either by inspection (decryption on the fly) or by heuristic / behavioural / statistical methods, including machine learning techniques.
5) Principles and techniques for traffic analysis and network cyber threat detection: network communications analysis techniques include heuristic, behavioural, statistical methods (e.g. to detect Domain Generation Algorithms, Fast Flux, traffic tunnelling techniques), machine learning methods, application of threat detection rules (e.g. YARA, Snort, Suricata rules), use of reputation lists (e.g. compromise indicators), application of retrospective analysis.
To effectively detect anomalies, NTA class solutions can use several of the above mentioned traffic analysis technologies, which are combined for optimal ratio of resource consumption and depth of analysis, for example, using the ‘funnel’ method, when traffic first passes through fast filters based on rules and reputation lists, then arrives at the input of the heuristics module, then is processed using machine learning, and at the end - manual retrospective analysis by analysts. To prepare the NTA system for anomaly detection based on machine learning and behavioural analytics techniques, the following steps are performed:
1) An initial baseline is formed, i.e. a baseline of normal (typical) traffic characteristic of the protected infrastructure. The duration of this learning period can be days or weeks, during which the NTA determines the properties and characteristics of routine, typical traffic, deviations from which will be considered a violation. At this stage, it is important that NTA trains on ‘clean’ traffic that does not contain malicious components (e.g., VPO strains or hidden communication channels between infected hosts) - if the system trains on such ‘dirty’ traffic, it will not be able to detect such network threats in the future.
2) Anomaly detection: a trained NTA system will be able to identify which network communications exhibit abnormalities, e.g., spikes in ICMP traffic from a particular workstation, massive API requests, a new DNS server on the network, bulk POST requests, previously unused cloud services.
3) Analysing anomalies: not all traffic deviations from the previously established norm are a sign of a cyberattack, so it is necessary to understand the reasons for the detected deviations, for example, to request a justification for using a new VCS application or installing a new server in the company's network. At the same stage it is possible to detect network equipment configuration errors that may be the cause of the detected network anomalies.
4) Formation of an actualised report based on the detected false alarms and transformation of the network traffic profile due to authorised infrastructure changes.
5) Further detection and analysis of anomalies, continuous adaptation of baseline to authorised changes in network traffic.
2. Network Detection and Response (NDR) systems.
Network Detection and Response (NDR) systems are solutions for detecting and responding to network cyber threats. The terms NDR and NTA are often confused or do not clearly distinguish between these types of protection systems, but compared to NTA, NDR solutions are still more advanced systems due to their deep Threat Intelligence, Threat Hunting, and direct response capabilities to detected network threats. Thus, NTAs are used to detect signs of cyberattacks, while NDRs enable active countermeasures against identified threats using automated cyber incident response methods - for example, through standalone orchestration of defences or integration with SOAR platforms. NDR solutions are also often integrated with sandboxes to detect and analyse VPOs in files transmitted over the network, network forensics systems (for detailed traffic analysis and preservation of legally relevant attributes of cyber incidents), SIEM systems, and support mapping of detected incidents in accordance with the MITRE ATT&CK matrix. In addition, manufacturers of NDR-class systems often integrate them with EDR (Endpoint Detection and Response) solutions, which are designed to protect endpoints with functionality that includes not only protection against VPOs, but also monitoring device status and behaviour, saving event logs, and the ability to perform active response and recovery actions on the device. NDR products complement EDR solutions and compensate for some of their shortcomings, such as the need to install EDR modules on supported endpoints, the possibility of disabling EDR components by attackers, and the inability to control unsupported OSes with EDR agents.
3. Network Intrusion Detection / Prevention Systems (NIDS/NIPS, Network Intrusion Detection / Prevention Systems).
NIDS/NIPS (Network Intrusion Detection / Prevention Systems) are solutions for analysing network traffic to detect malicious activity (exploitation of vulnerabilities, unauthorised access to protected resources, malicious or unwanted software). Network IDSs detect malicious traffic and notify the IS administrator, while network IPSs can also block malicious traffic and disconnect active network connections. Detection of malicious traffic in network IDSs/IPSs is based on rules and signatures (e.g., Snort or Suricata formats), which allows detection of already published exploits, attempts to exploit vulnerabilities, and known types of attacks; therefore, such IPSs are effective for fast initial cleanup of large volumes of incoming network traffic. In addition, advanced network IDS/IPS provide virtual patching functionality to block attempts to exploit vulnerabilities that have not yet been addressed in the infrastructure. In general, network-based intrusion detection and prevention systems are already classic IDSs and are often embedded in popular firewalls, however, compared to NTA, they do not have traffic storage functionality and advanced machine learning and network anomaly detection capabilities, and compared to NDR, they do not proactively respond to cyber incidents, nor do they support Threat Intelligence and Threat Hunting functionality.
