Alerting: Use Automated Threat Detection

Document created by RSA Information Design and Development on Jun 26, 2017Last modified by RSA Information Design and Development on Sep 14, 2017
Version 2Show Document
  • View in full screen mode
  

This topic discusses how to configure and use Behavior Analytics Automated Threat Detection.  Behavior Analytics Automated Threat Detection is a service you deploy on your ESA installation that examines your HTTP traffic to determine the probability that malicious activity is occurring in your environment.

For details on how to work with Behavior Analytics Automated Threat Detection, see the following topics:

Understanding Behavior Analytics Automated Threat Detection

This topic provides an overview of Behavior Analytics Automated Threat Detection. Behavior Analytics Automated Threat Detection is a service you deploy on your ESA. The Behavior Analytics: Suspicious Domains module examines your HTTP traffic to detect domains likely to be malware command and control servers connecting to your environment. Once Behavior Analytics Automated Threat Detection examines your HTTP traffic, it generates scores based on various aspects of your traffic behavior (such as the frequency and regularity with which a given domain is contacted). If these scores reach a set threshold, an ESA alert is generated. This ESA alert also triggers an alert in the Incident Manager. The alert in the Incident Manager is enriched with data that helps you to interpret the scores to determine what mitigation steps to take. 

This version of Behavior Analytics Automated Threat Detection provides scoring to detect Command and Control communications. Command and control communications occur when malware has compromised a system and is sending data back to a source. Often, Command and Control malware can be detected via beaconing behavior. Beaconing occurs when the malware regularly sends communications back to the Command and Control server to notify it that a machine has been compromised and the malware is awaiting further instructions. The ability to catch the malware at this stage of compromise can prevent any further harm from occurring to the compromised machine and is considered a critical stage in the "kill chain."  

This feature solves several common problems that occur when searching for malware:

  • Ability to use algorithms rather than signatures. Because many malware creators have begun using polymorphic or encrypted code segments which are very difficult to create a signature for, this approach can sometimes miss malware.  Because Behavior Analytics Automated Threat Detection uses a behavior-based algorithm, it is able to detect malware more quickly and effectively. 
  • Ability to automate hunting. Hunting through data manually is an effective but extremely time-consuming method of finding malware. Automating this process allows an analyst to use his or her time more effectively. 
  • Ability to find an attack quickly. Instead of batching and then analyzing the data, Behavior Analytics Automated Threat Detection analyzes data as it is ingested by Security Analytics, allowing for the attacks to be found in near real-time. 

Workflow

Behavior Analytics Automated Threat Detection works much like a filtering system. It checks to see if certain behavior occurs (or certain conditions exist), and if that behavior or condition occurs, it moves to the next step in the process.  This helps to make the system efficient, and frees up resources so that events which are determined to be non-threatening are not held in memory.  The following diagram provides a simplified version of the workflow. 

1.) Packets or Logs are routed to the ESA. The HTTP packets or logs are parsed by the Decoder or Log Decoder and sent to the ESA device.

2.) Whitelist is checked. If you created a whitelist via the Context Hub, the ESA checks this list to rule out domains. If a domain in the event is whitelisted, the event is ignored.

3.) The domain profile is checked. Behavior Analytics Automated Threat Detection checks to see if the domain is newly seen (approximately three days), has few source IP connections, has many connections without a referer, or has connections with a rare user agent.  If one or several of these conditions is true, the domain is next checked for periodic beaconing. For a detailed description of these domain profile scores, see "Work with Behavior Analytics Automated Threat Detection Scores".

4.) The domain is checked for periodic beaconing. Beaconing occurs when the malware regularly sends communications back to the command and control server to notify it that a machine has been compromised and the malware is awaiting further instructions.  If the site displays beaconing behavior, then the domain registration information is checked. 

5.) Domain registration information is checked. The Whois service is used to see if the domain is recently registered or nearly expired. Domains that have a very short lifespan are often hallmarks of malware. 

6.) Command and control (C2) aggregates scores.  Each of the above factors generates a separate score which is weighted to denote various levels of importance. The weighted scores determine if an alert should be generated. If an alert is generated, the aggregated alerts appear in the Incident Manager and can then be investigated further from there. Once the alerts begin to appear in the Incident Manager, they continue to aggregate under the associated incident. This makes it easier to sort through volumes of alerts that can be generated for a command and control incident. 

If you are an analyst, you can view the alerts generated in the Alerts Summary module, or in the Incident Manager module. If you use SecOps, you can view alerts in SecOps versions 1.2 and 1.3. 

Behavior Analytics Automated Threat Detection on Packets vs. Web Proxy Logs

RSA Security Analytics provides you with the ability to perform Behavior Analytics Automated Threat Detection using either packets or Web Proxy Logs. While packet data can be streamed directly off of the wire into the Security Analytics installation and analyzed directly, if you have the ability to use a web proxy in your installation there may be benefits to doing so. Because some installations use network translation or SSL encryption, the true source IP of an outgoing connection may be masked if you are observing it at the packet level. By using a web proxy you gain the benefit of its ability to accelerate and decrypt SSL traffic as well as its ability to track the true source IP addresses of traffic it monitors.

Using Behavior Analytics Automated Threat Detection for Security Analytics 10.6.2 or later, you can use only one detection method: packets or logs, but not both for the same ESA instance. Multiple ESA instances may be used to aggregate data from both packet and web proxy log sources, however.

 

You are here
Table of Contents > Use Automated Threat Detection

Attachments

    Outcomes