Skip navigation
All Places > Products > RSA NetWitness Platform > Blog > Authors Lee Kirkpatrick

Following up from the previous blog, Web Shells and RSA NetWitness, the attacker has since moved laterally. Using one of the previously uploaded Web Shells, the attacker confirms permissions by running, whoami, and checks the running processes using, tasklist. Attackers, like most individuals, are creatures of habit:

 

The attacker also executes a quser command to see if any users are currently logged in, and notices that an RDP session is currently active:

 

The attacker executes a netstat command to see where the RDP session has been initiated from and finds the associated connection:

 

The attacker pivots into his Kali Linux machine and sets up a DNS Shell. This DNS Shell will allow the attacker to setup C&C on the new machine she has just discovered:

 

The attacker moves laterally using WMI, and executes the encoded PowerShell command to setup the DNS C&C:

 

The DNS Shell is now setup and the attacker can begin to execute commands, such as whoami, on the new machine though the DNS Shell:

 

Subsequently, as the attacker likes to do, she also runs a tasklist through the DNS Shell:

 

Finally, the attacker confirms if the host has internet access by pinging, www.google.com:

 

As the attacker has confirmed internet access, she decides to the download Mimikatz using a PowerShell command:

 

The attacker then performs a dir command to check if Mimikatz was successfully downloaded:

 

From here, the attacker can dump credentials from this machine, and continue to move laterally around the organisation, as well as pull down new tools to achieve their task(s). The attacker has also setup a failover (DNS Shell) in case the Web Shells are discovered and subsequently removed.

 

 

 

Analysis

Since the previous post, the analyst has upgraded their system to NetWitness 11.3, and deployed the new agents to their endpoints. The tracking data now appears in the NetWitness UI, and subsequently the analysis will solely take place, on the 11.3 UI.

 

Tracking Data

The analyst, upon perusing the metadata, uncovers some reconnaissance commands being executed, whoami.exe and tasklist.exe on two of their endpoints:

 

Refocusing their investigation on those two endpoints, and exposing the Behaviours of Compromise (BOC) meta key, the analysts uncovers some suspect indicators that relate to a potential compromise, creates remote process using wmi command-line tool, http daemon runs command shell, runs powershell using encoded command, just to name a few:

 

Pivoting into the sessions related to, creates remote process using wmi command-line tool, the analyst observes the Tomcat Web Server performing WMI lateral movement on a remote machine:

 

The new 11.3 version stores the entire Encoded PowerShell command and performs no truncation:

 

This allows the analyst to perform Base64 decoding directly within the UI using the new Base64 decode function (NOTE: the squares in between each character are due to double byte encoding and not a byproduct of NetWitness decoding):

  

 

Navigating back to the metadata view, the analyst opens the Indicators of Compromise (IOC) meta key, and observed the metadata, drops credential dumping library:

 

Pivoting into those sessions, the analyst see’s that Mimikatz was dropped onto the machine that was previously involved in WMI the lateral movement:

 

Packet Data

The analyst also is looking into the packet data, they are searching through DNS as they had seen an increase in the amount of traffic that they typically see. Upon opening the SLD (Second Level Domain) meta key, the culprit of the increase is shown:

 

Focusing the search on the offending SLD, and expanding the Hostname Alias Record (alias.host) meta key, the analyst observed a large number of suspicious unique FQDN’s:

 

This is indicative behaviour of a DNS tunnel. Focusing on the DNS Response Text meta key, it is also possible to see the commands that were being executed:

 

We can further substantiate that this is a DNS Tunnel by using a tool such as CyberChef, and taking the characters after cmd in the FQDN, and hex decoding them, this reveals that data is being sent hex encoded as part of the FQDN itself, and sent as chunks, and reconstructed on the attacker side, due to the constriction on how much data can be sent via DNS:

 

 

ESA Rule

DNS based C&C is noisy, this is because there is only a finite amount of information that can be sent with each DNS packet. Therefore returning information from the infected endpoint requires a large amount of DNS traffic. Subsequently, the DNS requests that are made, need to be unique, so as not to be resolved by the local DNS cache or internal DNS servers. Due to this high level of noise from the DNS C&C communication, and the variance in the FQDN, it is possible to create an ESA rule that looks for DNS C&C with a high rate of fidelity.

The ESA rule attached to this blog post calculates a ratio of how many unique alias host values there are toward a single Second Level Domain (SLD). Whereby we count the number of sessions toward the SLD, and divide that by the number of unique alias hosts for that SLD, to give us a ratio:

 

  • SLD Session Count ÷ Unique Alias Host Count = ratio

 

The lower the ratio, the more likely this is to be a DNS tunnel; due to the high connection count, and variance in the FQDN to a single SLD. The below screenshot shows the output of this rule which triggered on the SLD which was shown in the analysis section of this blog post:

 

 

NOTE: Legitimate products perform DNS tunnelling, such as McAfee, ESET, TrendMicro, etc. These domains would need to be filtered out based on what you observe in your environment. The filtering option for domains is at the top of the ESA rule.

 

The rule for import and pure EPL code in a text file are attached to this blog. 

IMPORTANT: SLD needs to be set as an array for the to rule to work.

 

 

Conclusion

This blog post was to further demonstrate the TTP’s (Tools, Techniques, and Procedures) attackers may utilise in a compromise to achieve their end goal(s). It demonstrates the necessity for proactive threat hunting, as well as the necessity for both Packet and Endpoint visibility to succeed in said hunting. It also demonstrates that certain aspects of hunting can be automated, but only after fully understanding the attack itself; this is not to say that all threat hunting can be automated, a human element is always needed to confirm whether something is definitely malicious or not, but it can be used to minimise some of the work the analyst needs to do.

This blog also focused on the new 11.3 UI. This allows analysts to easily switch between packet data and endpoint data in a single pane of glass; increasing efficiency and detection capabilities of the analysts and the platform itself.

Introduction

This blog post demonstrates a common method as to how organisations can get compromised. Initially, the viewpoint will be from the attacker’s perspective, it will then move on to show what artifacts are left over within the RSA NetWitness Packets and RSA NetWitness Endpoint solutions that analysts could use to detect this type of activity.

 

Scenario

Apache Tomcat server exposed to the internet with weak credentials to the Tomcat Manager App gets exploited by an attacker. The attacker uploads three Web Shells, confirms access to all of them and then uploads Mimikatz to dump credentials.

 

Definitions

Web Shells

A web shell is a script that can be uploaded to a web server to enable remote administration of the machine. Infected web servers can be either internet-facing or internal to the network, where the web shell is used to pivot further to internal hosts.

A web shell can be written in any language that the target web server supports. The most commonly observed web shells are written in languages that are widely supported, such as JSP, PHP, ASP, Perl, Ruby, Python, and Unix shell scripts are also used.

 

Mimikatz

Mimikatz is an open source credential dumping program that is used to obtain account login and password information, normally in the form of a hash or a clear text password from an operating system.

 

THC Hydra

When you need to brute force crack a remote authentication service, Hydra is often the tool of choice. It can perform rapid dictionary attacks against more than 50 protocols, including telnet, ftp, http, https, smb, several databases, and much more.

 

WAR File

In software engineering, a WAR file (Web Application Resource[1] or Web application Archive[2]) is a file used to distribute a collection of JAR-files, JavaServer Pages, Java Servlets, Java classes, XML files, tag libraries, static web pages (HTML and related files) and other resources that together constitute a web application.


 

The Attack

The attacker finds an exposed Apache Tomcat Server for the organisation. This can be achieved in many ways, such as a simple Google search to show default configured Apache Servers:

                                      

The attacker browses to the associated Apache Tomcat server and see’s it is running up to date software and appears to be mainly left at default configuration:

                 

 

The attacker attempts to access the Manager App, the manager app requires a username and password and therefore the attacker cannot login to make changes. Typically, these servers are setup with weak credentials:

               

 

Based off of this assumption, the attacker uses an application called THC Hydra to brute force the Tomcat Manager App using a list of passwords. After a short while, Hydra returns a successful set of credentials:

                


The attacker can now login to the Manager App using the brute forced credentials:

               

 

From here, the attacker can upload a WAR (Web application ARchive) file which contains their Web Shells:

                

 

The WAR file is nothing more than just a ZIP file with the JSP Web Shells inside. In this case, three Web Shells were uploaded:

                  

 

After the upload, it is possible to see the new application called, admin (which is based off the WAR file name, admin.war), has been created:

              


The attacker has now successfully uploaded three Web Shells on to the server and can begin to use them. One of the Web Shells named, resetpassword.jsp, requires authentication to help protect direct access by other individuals; this page could also be adapted to confuse analysts when visited:

            

 

The attacker enters the password, and can begin browsing the web servers file system and executing commands, typical commands such as, whoami, are often used by attackers:

           

 

The attacker may also choose to see what processes are running to see if there are any applications that could hinder their progression by running, tasklist: 

           

 

From the previous command, the attacker notices a lack of Anti-Virus so decides to upload Mimikatz via the WebShell: 

          

 

The ZIP file has now been uploaded. This Web Shell also has an UnPack feature to decompress the ZIP file: 

         

 

Now the ZIP file is decompressed:

         

 

The attacker can now use the Shell OnLine functionality within this Web Shell which emulates CMD in order to navigate to the Mimi directory and see their uploaded tools: 

         

 

The attacker can then execute Mimikatz to dump all passwords in memory:

        

 

The attacker now has credentials from the Web Server:

         

 

The attacker could then use these credentials to laterally move onto other machines.

 

The attacker also dropped two other Web Shells, potentially as backups in case some get flagged. Let’s access those to see what they look like. This Web Shell is the JSP file called, error2.jsp, it has similar characteristics to the resetpassword.jsp Web Shell:

       

 

We can browse the file system and execute commands:

        

 

The final Web Shell uploaded, login.jsp, exhibits odd behavior when accessed:

       

 

It appears to perform a redirect to a default Tomcat Page named, examples, this appears to be a trick to confuse anyone who potentially browses to that JSP page. Examining the code for this Web Shell, it is possible to see it performs a redirect if the correct password is not supplied:

     

           <SNIP>

     

 

Passing the password to this Web Shell as a parameter, which is defined at the top of this Web Shell’s code, we get the default response from the Web Shell:

    

 

Further analyzing the code, you can understand further parameters to pass in order to make the Web Shell perform certain actions, such as a directory listing:

   

 

This Web Shell is known as Cknife, and interacting it in this way is not efficient or easy, so Cknife comes with a Java based client in order to control the Web Shell. We can launch this using the command shown below:

  

 

The client is then displayed which would typically be used:

  

Note:  This web shell is listed in this blog post as it is something the RSA Incident Response team consistently sees in some of the more advanced attacks.

 

The Analysis

Understanding the attack is important, and hence why it comes prior to the analysis section. Understanding how an attacker may operate, and the steps they may take to compromise a Web Server, will significantly increase your ability to detect these types of threats, as well as better understand the viewpoint of the analysis while triage is performed.

 

RSA NetWitness Packets

While perusing the network traffic, a large number of 401 authentication errors towards one the Web Servers was observed; there is also a large variety of what look like randomly generated passwords:

        

 

Focusing on the 401 errors, and browsing other metadata available, we can see the authentication errors are toward the Manager App of Tomcat over port 8080, also take note of the Client Application being used, this is the default from THC Hydra and has not been altered:

       

 

Removing the 401 errors, and opening the Filename and Directory meta keys, we can see the Web Shells that were being accessed and the tools that were uploaded:

       

 

NOTE: In an actual environment, a large number of directories and filenames would exist, it is up to the analyst to search for the filenames of interest that sound out of the norm or are in suspicious directories, are newly being accessed, and not accessed as frequently as other pages on the web server. For a more in-depth explanation to hunting using NetWitness Packets, take a look at the hunting guide available here: https://community.rsa.com/docs/DOC-79618

 

The analyst could also use other meta keys to look for suspicious/odd behavior. Inbound HTTP traffic with windows cli admin commands would be worth investigating, as well as sessions with only POST’s for POST based Web Shell’s, http post no get or http post no get no referer, for a couple of examples:

     

 

Investigating the sessions with windows cli admin commands yields the following two sessions, you’ll notice one of the sessions is one of the Web Shells, resetpassword.jsp

    

 

Double clicking on the session will reconstruct the packets and display the session in Best Reconstruction view, in this case, web. Here we can see the Web Shell as the browser would have rendered it, this instantly should stand out as something suspicious:

    

 

This HTTP session also contains the error2.jsp Web Shell, from the RSA NetWitness rendering, it is possible to see the returned results that the attacker saw. Again, this should stand out as suspicious:

    

 

Coming back to the investigate view, and this time drilling into the sessions for post no get no referer, we can see one of the other Web Shells, login.jsp:

   

 

Double clicking on one of these sessions shows the results from the Cknife Web Shell, login.jsp:

    

 

As this was not a nicely formatted web based Web Shell, the output is not as attractive, but this still stands out as suspicious traffic: why would a JSP page on a Web Server return a tasklist?

 

Sometimes changing the view can also help to see additional data. Changing the reconstruction to text view shows the HTTP POST sent, this is where you can see the tasklist being executed and the associated response:

   

 

Further perusing the network traffic, it is also possible to see that Mimikatz was executed:

  

This is an example of what the traffic may look like in RSA NetWitness Packets. The analyst would only need to pick up on one of these sessions to know their organization has been compromised. Pro-active threat hunting and looking for anomalies in traffic toward your web servers will significantly reduce the attacker dwell time.

 

The activity observed was only possible due to the communication happening over HTTP. If this had been SSL, the detection via packets would be much more difficult. This is why introducing SSL Decryption/Interception/Offloading is highly recommended. SSL inspection devices are nothing more than a well-designed man-in-the-middle attack that breaks the encryption into two separate encrypted streams. Therefore, they still provide an adequate level of protection to end-users while allowing security analysts and devices to properly monitor and alert when malicious or unwanted activity takes place, such as the web shells shown here. In summary, if you are responsible for protecting your organization’s assets, you should definitely consider the pros and cons of using this technology.

 

 

RSA NetWitness Endpoint

On a daily basis, analysts should be perusing the IIOC’s within NWE, paying particular attention to the Level 1 IIOC’s. Upon logging into NWE, we can see an interesting Level 1 IIOC has hit, HTTP Daemon Runs Command Shell. This IIOC is looking for a HTTP Daemon, such as Tomcat, spawning cmd.exe:

 

      

 

If we double click on the machine name in the Machines window, we can then navigate to the tracking data for this machine to see what actually happened. Here we can see that Tomcat9.exe is spawning cmd.exe and running commands such as whoami and tasklist, this is not normal functionality and should raise concern for the analyst. We can also see the Mimikatz execution and the associated command executed for that:

 

    

 

Another IIOC that would have led us to this behavior is, HTTP Daemon Creates Executable:

 

   

 

Again, coming back into the tracking data, we can see the Tomcat9.exe web daemon writing files. This would be note-worthy and something that should be investigated further, as web daemons can perform this activity normally. In this instance, the presence of Mimikatz is enough for us to determine this would be malicious activity:

  

 

The analyst also has the capability to request files from the endpoint currently under analysis by right-clicking on the machine name, selecting Forensics and then Request File(s):

 

The analyst can specify the files they want to collect for their analysis (NOTE: wildcards can be used for the filenames but not directories). In this case, the analyst wants to look into the Tomcat Access files, and requests that five of them be returned:

 

Once the files have been downloaded, the analyst can save them locally by navigating to the Download tab, right-clicking the files of interest and selecting Save Local Copy:

 

Perusing the access files, the analyst can also see a large number of 401 authentication errors to the Tomcat Web Server which would have been from the THC Hydra brute force:

 

And also evidence of the Web Shells themselves. Some of the commands the attacker executed can be seen in the GET requests, the data in the body of the POST's however, does not show in the log file, showing why it is important to have both Packets and Endpoint visibility to understand the interaction with the Web Shell:

 

Conclusion

Understanding the potential areas of compromise within your organisation vastly increases your chances of early detection. This post was designed to show one of those potential areas of importance for attackers, and how they may go about a compromise, while also showing how that attack may look as captured by the RSA NetWitness Platform. It is also important to understand the benefits behind proactively monitoring the RSA NetWitness products for malicious activity, simply awaiting for an alert is not enough to capture attacks in their early stages.

 

It is also important to for defenders to understand how these types of attacks look within their own environment, allowing them to better understand, and subsequently protect against them.

 

This is something our RSA Incident Response practice does on a daily basis.  If your organization needs help or your interested to learn more, please contact your account manager.  

 

As always, happy hunting!  

After reading through a few SANS resources, I came across some interesting topics regarding the detection of rare processes to help pin point malicious applications running on a host; from this, I decided to create an EPL rule to baseline processes on Windows hosts and alert if any processes deviated from the norm.

 

The principle behind this rule is to profile every Windows host in the estate and keep track of the processes which run on said hosts, should they diverge from the average they are declared as rare and an alert is generated for analysts to investigate; the rule is written in a way to learn what is normal within a specific environment and baseline accordingly.

 

 

Dependencies

The following meta keys need to be indexed for the below rule to work:-

 

  • event_computer
  • process

 

Other than that, deploy the rule and you're good to go!

 

The EPL Rule

@Name('Create Window')

CREATE WINDOW winProcess.win:time(31 days) (theDay int, event_computer string, process string, counter int);

 

@Name('Insert into Window')

on Event(process IS NOT NULL AND event_computer IS NOT NULL)

merge winProcess

WHERE Event.process = winProcess.process AND Event.event_computer = winProcess.event_computer AND current_timestamp.getDayOfWeek() = winProcess.theDay

when matched

then update set counter = counter + 1

when not matched then INSERT

SELECT current_timestamp.getDayOfWeek() as theDay, event_computer, process, 1 as counter;

 

 

@Name('Alert')

@RSAAlert

SELECT * FROM winProcess as original

WHERE counter <= 0.2* (

SELECT avg(counter) FROM winProcess as recent

WHERE original.theDay = recent.theDay and original.event_computer = recent.event_computer);

Lee Kirkpatrick

DGA Detection

Posted by Lee Kirkpatrick Employee Feb 1, 2017

In one of my previous posts (Shannon. Have you seen my Entropy?) I touched on using a custom Java entropy calculator within the ESA to calculate the entropy values for domains to assist with detecting Domain Generation Algorithms (DGA's); the post was more theory than practical so I decided to implement and test it in my lab so I could share the implementation with you all.

 

The basic principle behind this form of DGA detection is to calculate an entropy value for each domain seen and store this value in an ESA window. We can then use the values in the ESA window to calculate an average entropy for the domains seen within an environment, this subsequently allows an alert to be generated if any domains exceed the average entropy by 1.3x.

 

As an example, let's take the following four domains from the Alexa top 100 (this will be what we use as an example baseline, the rule attached to this post would actually monitor your network for what is normal):-

 

  • google.com
  • youtube.com
  • facebook.com
  • baidu.com

 

Running these each through the entropy calculator we receive the following values:-

 

DomainEntropy
google.com2.6464393446710157
youtube.com3.095795255000934
facebook.com3.0220552088742
baidu.com3.169925001442312
Average2.983553702497115

 

Using this average as our baseline, we can then say that anything that is greater than 1.3x (3.87861981324625) this average, let me know about it as this is a high entropy value.

 

Taking the following values from Zeus tracker and calculating their entropy values, we can see the results:-

 

DomainEntropyStatus
circleread-view.com.mocha2003.mochahost.com
3.952216429463629Alert
cynthialemos1225.ddns.net
3.952216429463629
Alert
moviepaidinfullsexy.kz
4.061482186720775Alert
039b1ee.netsolhost.com
3.754441845713345No alert

 

Example of the alert output below:-

 

 

Rule Logic

@Name('Learning Phase Variable')
//Change the learningPhaseMinutes variable to the number of minutes for the rule to learn
CREATE VARIABLE INTEGER learningPhaseMinutes = 1440;

 

@Name('Calculate Learning Phase')
on pattern[Every(timer:at(*, *, *, *, *))] set learningPhaseMinutes = learningPhaseMinutes - 1;

 

@Name('Create Entropy Window')
CREATE WINDOW aliasHostEntropy.win:length(999999).std:unique(entropy) (entropy double);

 

@Name('Insert entropy into Window')
INSERT INTO aliasHostEntropy
SELECT calcEntropy(alias_host) as entropy FROM Event(alias_host IS NOT NULL AND learningPhaseMinutes > 1);

 

@Name('Alert')
@RSAAlert
SELECT *, (SELECT avg(entropy) FROM aliasHostEntropy as Average), (SELECT calcEntropy(alias_host) FROM Event.win:length(1) as Entropy) FROM Event(learningPhaseMinutes <= 1 AND calcEntropy(alias_host) > 1.3* (SELECT avg(entropy) FROM aliasHostEntropy));

 

 

If you are interested in implementing this DGA Detection rule, I wrote up a little guide on how to do so. Everything you need is attached to this post.

 

DISCLAIMER: The information within this blog post is here to show the capabilities of the NetWitness product and avenues of exploration to help thwart the adversary. This content is provided as-is with no RSA direct support, use it at your own risk. Additionally, you should always confirm architecture state before running content that could impact the performance of your NetWitness architecture. 

Lee Kirkpatrick

Everything is PossiEPL

Posted by Lee Kirkpatrick Employee Oct 12, 2016

Event Processing Language is utilised within the NetWitness Event Stream Analysis (ESA) component. This language is what allows us to write advanced correlation rules to detect and thwart the advanced threats we face on a constant basis; it allows us to make sense, to organise and sift through the copious amounts of metadata which is produced on a daily basis.

 

EPL can seem a little daunting upon first glance, but understanding a few basic principles will allow you to create a plethora of use cases - I have created a document to better understand those principles, to extend my knowledge, and hopefully yours as well:-

 

 

Enjoy!

Entropy is a term I am sure most of us are familiar with. In layman’s terms, it refers to randomness and uncertainty of data; it is in this randomness that we can detect potential malicious traffic.

 

A gentleman named George Zipf lead the way in the study of character frequency in the early 1930’s, his work was further expanded upon by Claude Shannon to examine the entropy of language. These two forms of analysis have become engraved in the computer security domain and often used for cryptography – but what if we used their ideas to help detect malicious traffic?

 

Some malicious actors utilise domain generation algorithms (DGA) to produce pseudo random domain names they will utilise for their C2 communications. If we apply Shannon’s Entropy to these domains, we can calculate a score of their entropy and possibly identify these maliciously formed domains from the norm:-

 

 

Using the RSA Event Stream Analysis (ESA) component and a customised Java based Shannon Calculator, we can generate these entropy scores on the fly for any given metadata, and should they exceed a score greater than X, create an alert.

 

NOTE: Java plugins can be added to the ESA component as described by Nikolay Klender in his post - Extending ESA rules with custom function.

 

Once the Java plugin is implemented, we can then create our ESA correlation rule to utilise the new plugin available and calculate the entropy. In this example, we will use the plugin to calculate entropy for DNS domains using the following EPL:-

 

@RSAAlert

SELECT * FROM Event(service = 53 AND calcEntropy(alias_host)>4);

 

The entropy value for this is set to anything greater than ‘4’ but can be edited dependent upon what results are observed.

 

I have attached the java used for calculating Shannon's Entropy should anyone be interested.

 

DISCLAIMER: This is by no means a full proof detection method for malicious traffic. The information is here to show the capabilities of the product and avenues of exploration to help thwart the adversary. This content is provided as-is with no RSA direct support, use it at your own risk. Additionally, you should always confirm architecture state before running content that could impact the performance of your SA architecture. 

Filter Blog

By date: By tag: