John Simmons

Custom TCP Shell and Mobile Messaging Apps

Blog Post created by John Simmons Employee on Feb 23, 2019

During a recent customer engagement, I found the "customtcp shell" meta with some very interesting sessions.  All of the traffic was using what appeared to be custom encryption and the destination IP was based in Korea.  Of course, I knew this couldn't be the first time someone had come across traffic like this so I looked at previous reporting for similar traffic.  Multiple analysts, like myself, had seen traffic exactly like this and their analysis led them in different directions.  Some even believed that this was NanoCore RAT traffic as it had similar attributes but I was still skeptical.  After hours of researching this traffic and trying to dissect this traffic I came across someone's master's thesis talking about end-to-end encrypted mobile messaging apps.  The link to the thesis is below. 


The traffic I was seeing matched perfectly with the handshake packet used with the propriety protocol called LOCO, which is used by the mobile messaging app KakaoTalk.  The article broke it down very well and without it I think I would still be scratching my head at this traffic.  So lets go back to what this traffic looks like and how I was able to determine it was KakaoTalk.


Here is an example of the customtcp shell meta being populated in a customer's environment.  This over a 5 day time period so it's not very common in most customer's environments.



As of today, I have seen this type of traffic in four customer environments within the RSA NetWitness Platform. Further, fellow colleagues and team members have also inquired about this type of traffic.  To recreate this traffic and avoid showing customer data, I downloaded the KakaoTalk mobile app on my personal iPhone.  Here is an example of the what the handshake packet looks like.



With the help of Stephen Brzozowski during the first engagement, we were able to dissect this packet to some extent.  The first 12 bytes of the sessions are the custom headers and always repeat during the first session.



The response and any follow-on packets would begin with the size in little-endian format.



With the help of the article mentioned above, I was able to match this traffic to the LOCO protocol's initial handshake packet.



As well as the follow-on packets that matched the LOCO encrypted packet.


Considering that every instance of the handshake packet I have seen in multiple environments always begin with the same first 12 bytes, I wrote a quick parser to find this traffic.  It's attached below.  I have deployed it on two customer environments and left it running for almost 24 hours with no false positives and approximately 20 sessions discovered in each environment.  Currently, it only detects the initial handshake, but I intend to modify it later to detect all sessions with the same high fidelity.  In addition, I have seen one other instance where the first 12 bytes didn't match because of one bit being off but I still believe it was KakaoTalk.


While deploying this parser and testing in these environments, I also discovered similar traffic that used other custom headers which I believe is another type of mobile messaging app that uses end-to-end encryption.  I believe we will continue to see additional mobile apps that populate this meta key in the future.  While documentation for these apps and custom protocols are scarce, I believe that it will present a challenge for analysts to distinguish between malicious custom TCP shells and benign traffic such as discussed in this blog.