Anastasia Kuznetsova, Security Vision
This research was conducted as part of the agent story for a whole group of products.
We continue our series of articles about the types and methods of endpoint data collectors, or as they are commonly called - agents. In the first article we got acquainted with this entity and studied the main nuances of data collection with their help. Since we are involved in both log management and event collection as part of our product development, we would like to share a continuation of our extensive analytics in quickstart format. That's why in this issue we'll go into details about the functionality and tools used in Windows-based sources.
Let's start with the fact that the agent itself does not care for the most part what technology is used to collect data. However, when implementing agents in your own infrastructure, there are always some restrictions on the use of certain tools, ports and other things within the framework of security policy. In this regard, let's look at possible sources of data transfer for agents.
If we want to collect data from endpoints on Windows, there are three main tools. The first is the Windows auditing subsystem that stores events in EventLog format. This is a Windows event logging tool that logs operating system messages. Events are divided into several groups - logs:
Plus, some apps write their own events to a separate folder, for example:
This folder may contain logs for both CPU and software (sound drivers or Kaspersky).
There are also separate logs for:
- DNS Manager
- Failover Cluster Manager
- IIS Access
- Task Scheduler History
- Windows Component Service
System log events in EventLog format already have some initial normalisation and contain the following fields:
- Date in the format: month, day, time, and year.
- Event category as an integer
- Event Level
- Windows security ID
- Windows Source.
- Windows event log keywords.
- Windows event identifier.
- Message text
It is possible to change read, write and clear access rights for event logs. The Security log is a special case in point - the write permission is reserved exclusively for the Windows Local Security Authority (LSASS). It is also possible to add event types to the Security log and assign a source from registered applications.
For primary analytics, and even more so for incident investigation, information only from EventLog is not always enough. In this regard, another utility with advanced event auditing functions is actively used - Sysmon, which is part of the Sysinternals package. It is essentially a Windows system service that gives detailed data on incoming events, network connections and drivers, file changes, and more. I found this map of interesting data that can be collected with Sysmon (for details, I recommend checking here.
To provide an additional layer of security, Sysmon is typically installed in conjunction with other endpoint protection solutions such as traditional antivirus, event management (SIEM), or endpoint detection and response (EDR). The driver for this service allows for collection as soon as the device is switched on, i.e. before the main components of the computer are started.
In addition, this tool is quite well described and popular, which means that there are many ready-made configurations for various event auditing requirements on the Internet.
The only significant disadvantage of Sysmon is its poor compatibility with the Linux class of operating systems. For these operating systems only some part of events registered in Windows systems (Event ID 1, 3, 4, 5, 9, 11, 16, 23) is registered.
You should look for logs in the ‘Event Viewer’ interface in the Applications and Services Logs\Microsoft\Windows\Sysmon. Here is an example event:
Finally, rounding out the top three primary event collection technologies is Event Tracing for Windows (ETW), a Windows event tracing tool. The technology is provided in the OS itself by default and is implemented at the kernel level. ETW captures logs of events generated by kernel drivers and user applications. The same Sysmon generates some of its events on its basis. To understand the essence of ETW we need to use several concepts:
- Providers (providers, event suppliers) are applications (e.g. DLLs) that have event logging functions. It is the provider that monitors application (or system) states and sends events to the session.
- Sessions (sessions, session trace, logger) are entities that collect events from providers and send them to consumers. A single session can consume events from one or more providers. Sessions also sort events among themselves to pass them to the right consumers.
- Controllers are applications that manage sessions. They can create or delete sessions and change their parameters.
- Consumers are applications that can receive and somehow process events from one or more sessions or files.
Providers generate and transmit events about states occurring within processes or the kernel, and consumers use these events for their own purposes (e.g. performance analysis). The vendor provides a schematic of the data transfer process as follows:
ETW in its basic configuration is a simple inter-process communication mechanism and functions only locally, and in the vast majority of studies is used as a ‘defensive’ tool.
Events from this source can be found in the Applications and Services Logs and Windows Logs\Security tree.
If you want to go deeper into the logic of ETW operation and basic administration, I recommend you to study the following article, where the internal processes of the tracing service are described in clear language.
So, we have figured out where and what we can use to collect data from, but what should the logs be like? After all, as the commenter under the last article correctly said, it is not enough to collect ‘everything I see’ for the correct operation of the infrastructure and subsequent analysis of events. So there is a great risk of clogging the processing of devices, and at the same time making prompt response to events impossible.
So I propose some criteria for assessing the quality of the data being transmitted:
- Events must be valid, that is, we need to be sure that the recorded events actually happened and that the event attributes are correct;
- The event log must be complete, i.e. it must provide a complete picture of the actions;
- Any recorded events must have a well-defined semantics, i.e. different types of events must be distinguishable.
This article is for informational purposes only and in no way tells you which event logging tools you should use. Rather, this article will be useful for those who are just diving into auditing Windows endpoint events - friends, now you know where to dig next. And in the next article we will talk about journaling in Linux class operating systems.
