Threat hunting involves looking for threats and adversaries in an organization’s digital infrastructure that existing security tools don’t detect. It is proactively looking for threats in the environment by assuming that the adversary is in the process of compromising the environment or has compromised the environment.
Threat hunters can have different goals and mindsets while developing their hunt. For example, they can look for long-term threats in the environment that advanced threat actors can exploit. Or they can look for current trends and threats being exploited by environmental adversaries.
Threat hunting 101
How can we hunt for threats in our environment? Let’s walk through some steps along with examples.
Step 1: Research the threat actors and their tactics. We should always start our hunt by researching ongoing or past tactics and techniques used by threat actors and how they can affect our organization. We should review threat intelligence, review metrics on the security alerts and security incidents, the technology exploited by threat actors, etc.
Step 2: Develop a hypothesis. This hypothesis can be based on the adversary we are hunting, such as the tactics, techniques and procedures (TTPs) they use. Let’s consider an example where we want to search for adversaries utilizing system services as persistence mechanisms.
What can be our hypothesis?
Windows services are being created and launched by threat actors with the aim of running either an executable or a script file for persistence.
If attack group APT41 creates Windows services to establish malware persistence in our environment, we should see the activity in the endpoint detection and response (EDR) tool.
If APT41 creates Windows services to establish malware persistence in our environment, we should see Windows events showing service creations after logins from anomalous IP addresses.
Step 3: Understand the attack. Our next step will be focusing on what the attack technique is and how it can be executed based on the threat actor group we are focusing on.
Questions to consider for our example, which can be adapted to hunt for other TTPs:
- What is the name of the service being created on the system by the adversary?
- Is the service being created on the same system or a remote system?
- Is there any user account being created before the service creation?
- What does the service do?
- What tools and permissions does it require to execute?
- Is this an action performed by custom-built, licensed or open-source tools?
- Can any living-off-the-land binary perform the same action?
Example: The following are the services created and executed by the attack group APT41 described in this Group-IB blog post:
- sc \\172.26.16.81 Create SuperIe binPath= “cmd.exe /k c:\users\public\SecurityHealthSystray.exe”
- sc Create syscmd binpath=”cmd/k start”type= own type= interact sc \\192.168.111.112 create res binpath=”C:\PerfLogs\vmserver.exe” sc start LxpSrvc
The above commands create a service on the host 192.168.111.112 and 172.26.16.81 under the Public and Perflog folder.
Step 4: Understand the artifacts created by this action. This step is the key to determining if we can effectively test the hypothesis using as many methods as possible.
Usually, multiple types of artifacts are created for a type of action. Accounting for artifacts associated with different attack techniques can be helpful in creating variations in detections and hunt queries. It can also be beneficial when a threat actor tries to tamper with the evidence. If the attacker deletes one artifact or if it is not logged properly, there is always something else to fall back on.
Things to think about:
- What artifacts are created on the source system? Note that the source system is the system from where the malicious activities are originating.
- What artifacts are created on the destination system? Note that the destination system is the system that is being targeted or the system where the service is being created.
- What artifacts will be visible?
- Are there any chained events? Note that chained events mean that the threat actor is utilizing multiple techniques which can be correlated to hunt; for example, service creation after logging in from a malicious IP address.
- What platforms does your organization have that can be utilized to search for these artifacts?
- Can the artifact be deleted or modified easily?
Let’s take the example above and analyze methods to hunt for service creation.
Some of the artifacts we can look for on the destination system include:
- When a service is created, a couple of artifacts are generated on the system. For example, Windows event logs in the system or security event logs will have records for the events generated. Based on EDR the organization has, we can search for “New service created events” by the name of the service being created.
- Registry key(s) and value(s) for the service we seek.
- If the service creates additional processes on the system, there might also be artifacts related to the process execution of the malicious file in the Shimcache, Prefetch and Amcache. We can try hunting for files being created under the Prefetch folder.
Now, let’s think about other functionalities of the sc.exe executable creating the service.
The Microsoft executable sc.exe, which is used for creating a service, also has the ability to create a remote service on a specified server.
Before starting the hunt, we can think about the following questions:
- Is the attacker moving laterally and creating a service?
- Is the service being created on the same compromised system?
Destination system artifacts include:
- When a service is created on a different system, a network logon event is created on the destination system, and all the usual artifacts described above for the service creation are created.
- To hunt for these types of events, we can search for services created from private IP addresses and correlate them with network logon events.
Source system artifacts include:
- On the source system from where this service creation command was provided, there might be a logon event indicating the usage of alternate credentials. For example, event ID 4648 will be created, indicating where the threat actor has moved.
Below is one of the methods that can be utilized to create remote services:
1) Authenticate
net use \10.x.x.x\admin$ /user:nameofuser
2) Create service
sc.exe \10.x.x.x create NewServicetest binpath= c:\windows\system32\cmd.exe
3) Perform any actions
For more about sc.exe, check out this Microsoft article.
Step 5: Use labs to determine what events and artifacts are generated when a specific attack is performed (optional but highly recommended to get effective results). If our organization has a dedicated lab setup, we could do some red teaming exercises to see what kind of events and alerts are generated by the simulated threat actor activity.
Observations to make:
- How easy was it to perform the attack? For example, in another hunt focused on looking for credential dumping, the difficulty of simulating an attack would be important to know because of various methods that can be utilized to perform the attack and to determine the artifacts created by that dump. This won’t affect the artifacts created, but sometimes we don’t have custom tools built by the threat actors, so we might not get to know what’s happening and what we can see in the EDR tool. We might know the capabilities, but we are not sure how they perform the attack. For example, a password dump can be performed via dumping Chrome browsers passwords from the password store location, dumping a security account manager (SAM) registry hive, dumping lsass via procdump, or even dumping rdpclip and performing strings on it to get passwords or using valuevault to view credentials.
- What type and how many detections or alerts were generated when we performed the attack? If there were no alerts, we can start analyzing the events in our security tools. If there are alerts, we can try to find the gaps in the detections and ways we can bypass them. Also, we can think about different methods that the adversary can use to pivot to get the same results. For the example in the writeup, this refers to other ways the threat actor can create services on the target system.
Explore cyber threat hunting solutions
Step 6: Review the platform and feasibility of the hunt. Based on the research and data from the above steps, we need to see what platforms our organization has and where the hunt could be performed. We also need to determine if the hunt is even feasible to perform from the data we are recording in our environment.
Step 7: Develop the basic query. We can start developing the query now that we understand the attack and artifacts.
Examples of the queries we can run on Microsoft Advanced Threat Protection (MDATP) include (refer to the link and example above for service names and actions for which this EDR query is developed):
DeviceRegistryEvents
|where ActionType in (“RegistryValueSet”)
and RegistryKey matches regex @”HKEY_LOCAL_MACHINE\\SYSTEM\\.*\\Services\\.*”
and RegistryValueName == “Start” and RegistryValueData == “2”
//| where RegistryValueData has_any (“SecurityHealthSystray.exe”, “vmserver.exe “, “LxpSrvc”)
//|where RegistryValueName has_any (“SecurityHealthSystray.exe”, “vmserver.exe “, “LxpSrvc”)
DeviceEvents
| where ActionType == ‘ServiceInstalled’
| where InitiatingProcessCommandLine has_any (“SecurityHealthSystray.exe”, “vmserver.exe”, “LxpSrvc”)
DeviceEvents
| where ProcessCommandLine has_any (“SecurityHealthSystray.exe”, “vmserver.exe “, “LxpSrvc”)
Step 8: Fine-tune the query. If the query returns numerous results, we need to modify the query to look for unique values or sum the events and maximum or minimum entries, sort by time, display the top 10 results, exclude the baseline events observed in our organization, etc.
Step 9: Further actions. If we have the malware sample and know the functionality of the malware, we can go one step further and think about how the threat actors can modify the same malware to reuse it after the enterprise security tools block them based on the current indicators of compromise.
We can also consider whether the same malware can be tweaked with minimal efforts to bypass the detections we create or already exist on our infrastructure and EDR tool.
Step 10: Actionable items. Once the hunt is finished, the threat hunters can report any malicious systems they find to the security operations center or computer security incident response team or create a detection query for the endpoint platforms.
IBM Security can significantly improve detection rates and accelerate the time to detect threats as well as investigate and remediate threats. For more information about IBM’s threat-hunting services, visit our website.
