Alerting: Use Automated Threat Detection

Document created by RSA Information Design and Development on Jul 20, 2016Last modified by RSA Information Design and Development on Apr 26, 2017
Version 2Show Document
  • View in full screen mode
  

This topic discusses how to configure and use Automated Threat Detection.  Automated Threat Detection is a service you deploy on your ESA installation that examines your HTTP traffic to determine the probability that malicious activity is occurring in your environment.

For details on how to work with Automated Threat Detection, see the following topics:

Understanding Automated Threat Detection

This topic provides an overview of Automated Threat Detection. Automated Threat Detection is a service you deploy on your ESA. The Behavior Analytics: Suspicious Domains module examines your HTTP traffic to detect domains likely to be malware command and control servers connecting to your environment. Once Automated Threat Detection examines your HTTP traffic, it generates scores based on various aspects of your traffic behavior (such as the frequency and regularity with which a given domain is contacted). If these scores reach a set threshold, an ESA alert is generated. This ESA alert also triggers an alert in the Incident Manager. The alert in the Incident Manager is enriched with data that helps you to interpret the scores to determine what mitigation steps to take. 

This version of Automated Threat Detection provides scoring to detect Command and Control communications. Command and control communications occur when malware has compromised a system and is sending data back to a source. Often, Command and Control malware can be detected via beaconing behavior. Beaconing occurs when the malware regularly sends communications back to the Command and Control server to notify it that a machine has been compromised and the malware is awaiting further instructions. The ability to catch the malware at this stage of compromise can prevent any further harm from occurring to the compromised machine and is considered a critical stage in the "kill chain."  

This feature solves several common problems that occur when searching for malware:

  • Ability to use algorithms rather than signatures. Because many malware creators have begun using polymorphic or encrypted code segments which are very difficult to create a signature for, this approach can sometimes miss malware.  Because Automated Threat Detection uses a behavior-based algorithm, it is able to detect malware more quickly and effectively. 
  • Ability to automate hunting. Hunting through data manually is an effective but extremely time-consuming method of finding malware. Automating this process allows an analyst to use his or her time more effectively. 
  • Ability to find an attack quickly. Instead of batching and then analyzing the data, Automated Threat Detection analyzes data as it is ingested by Security Analytics, allowing for the attacks to be found in near real-time. 

Workflow

Automated Threat Detection works much like a filtering system. It checks to see if certain behavior occurs (or certain conditions exist), and if that behavior or condition occurs, it moves to the next step in the process.  This helps to make the system efficient, and frees up resources so that events which are determined to be non-threatening are not held in memory.  The following diagram provides a simplified version of the workflow. 

1.) HTTP packets are routed to the ESA. The HTTP packets are parsed by the Decoder and sent to the ESA device.

2.) Whitelist is checked. If you created a whitelist via the Context Hub, the ESA checks this list to rule out domains. If a domain in the event is whitelisted, the event is ignored.

3.) The domain profile is checked. Automated Threat Detection checks to see if the domain is newly seen (approximately three days), has few source IP connections, has many connections without a referer, or has connections with a rare user agent.  If one or several of these conditions is true, the domain is next checked for periodic beaconing. For a detailed description of these domain profile scores, see "Work with Automated Threat Detection Scores".

4.) The domain is checked for periodic beaconing. Beaconing occurs when the malware regularly sends communications back to the command and control server to notify it that a machine has been compromised and the malware is awaiting further instructions.  If the site displays beaconing behavior, then the domain registration information is checked. 

5.) Domain registration information is checked. The Whois service is used to see if the domain is recently registered or nearly expired. Domains that have a very short lifespan are often hallmarks of malware. 

6.) Command and control (C2) aggregates scores.  Each of the above factors generates a separate score which is weighted to denote various levels of importance. The weighted scores determine if an alert should be generated. If an alert is generated, the aggregated alerts appear in the Incident Manager and can then be investigated further from there. Once the alerts begin to appear in the Incident Manager, they continue to aggregate under the associated incident. This makes it easier to sort through volumes of alerts that can be generated for a command and control incident. 

If you are an analyst, you can view the alerts generated in the Alerts Summary module, or in the Incident Manager module. If you use SecOps, you can view alerts in SecOps versions 1.2 and 1.3. 

You are here
Table of Contents > Use Automated Threat Detection

Attachments

    Outcomes