Skip navigation
All Places > RSA Labs > Blog > Author: Naveen Sunkavally

RSA Labs

4 Posts authored by: Naveen Sunkavally Employee

Today we are announcing support for Azure IoT Edge, which is Microsoft's solution for edge computing suitable for IoT gateways. Project Iris now brings visibility and threat detection to the Azure IoT Edge platform and connected edge devices managed by it.


Azure IoT Edge Architecture

Azure IoT Edge extends Microsoft's cloud-based Azure IoT Hub architecture to the edge. 



Azure IoT Hub provides a bidirectional communication channel between devices and the cloud, enabling users to perform tasks such as configuration, data collection, and command execution from the cloud. With just Azure IoT Hub in the picture (and prior to Azure IoT Edge), IoT devices would be required to implement the Azure IoT SDK to directly communicate with Azure IoT Hub in the cloud. The supported protocols between an IoT device and Azure IoT Hub are MQTT, AMQP, and HTTPS.


Azure IoT Edge opens up the picture, allowing IoT devices not using the Azure IoT SDK to be brought into the fold. These devices make up the vast number of existing IoT devices out there, and they use an alphabet soup of IoT protocols such as modbus, BACnet, and OPC-UA. Azure IoT Edge proxies communication between these devices and the cloud. This model is especially helpful from a security perspective. 


Going into more depth, this post describes three different patterns for how Azure IoT Edge can used at the gateway:

  • Transparent gateway: Device identities are managed by the Azure IoT Hub, and devices integrate with the Azure IoT SDK. Devices use Azure IoT Edge as a proxy to Azure IoT Hub in the cloud. No protocol translation is needed since devices are already using the Azure IoT SDK.
  • Identity translation: Device identities are managed by the Azure IoT Hub, but devices don't integrate with the Azure SDK. Azure IoT Edge (or other software like EdgeX) performs protocol translation and communicates on behalf of these devices with the cloud.
  • Protocol translation: Azure IoT Hub doesn't know anything about device identities (they are managed elsewhere). Azure IoT Edge performs protocol translation and exchanges device data with the cloud. Other software on the cloud side needs to make sense of the device data to do something meaningful with it.


In addition to protocol translation, Azure IoT Edge allows for general purpose computing at the edge. For instance, running analytics at the edge can save on overall IoT solution costs, compared to shipping all the data to the cloud for processing.


Project Iris and Azure IoT Edge

To use Project Iris to monitor Azure IoT Edge, deploy the Project Iris Docker container side by side with Azure IoT Edge running on the same IoT gateway host.



Azure IoT Edge uses modules to achieve a general purpose edge computing framework. Modules are simply Docker containers. There are two special modules provided by Microsoft, edgeAgent and edgeHub. The edgeAgent module uses the Docker service to manage other modules, and the edgeHub module handles communication between other modules and the cloud. Other modules, such as the Microsoft-provided modbus module, can perform protocol translation, edge analytics, or other activities.


The Project Iris container passively monitors all Azure IoT Edge modules and their communication with other edge devices, and passes up data to the Project Iris cloud service. Based on the data gathered, the Project Iris cloud service dynamically builds out profiles of expected behavior for Azure IoT Edge modules and edge devices tailored to your deployment. Alerts are triggered when significant deviations or anomalies from expected behavior are detected.


Project Iris Runtime Arguments

The Project Iris container should be deployed with the following environment variables as container arguments:

  • AZURE_IOT_HUB_CONNECTION_STRING: This is used by the Iris container to gather metadata about Azure-managed edge devices and Azure IoT Edge deployments.
  • AZURE_EVENT_HUB_CONNECTION_STRING: (optional) Azure IoT hub can be configured to stream diagnostics to an Azure Event Hub. Diagnostics can include useful security related events such as unauthorized device access. Set this environment variable to let Project Iris capture and surface these events.


Device Inference

Device identities can be managed in Azure or elsewhere. Project Iris is intelligent about surfacing these identities, depending on the type of architectural pattern under which Azure IoT Edge is deployed at the gateway (see above).


In the "Transparent gateway" pattern, device identities are fully managed in Azure, and Project Iris gets all device related metadata from Azure. This metadata includes arbitrary tags and configuration properties that can be set in the cloud.


In the "Identity translation" pattern, device identities are managed in Azure and in another piece of software such as EdgeX. Project Iris gathers identity data from both Azure and EdgeX and merges the data together, creating a unified view of identities across both sources.


In the "Protocol translation" pattern, identities are managed outside of Azure. However Project Iris can infer device identities by inspecting Azure IoT Edge module configuration. For instance, the modbus module contains configuration describing how that module can connect to downstream modbus slaves. Project Iris manufactures device identities based off this configuration. 


In the future, as Project Iris continues to support more IoT gateway platforms, it will continue to merge device data together from disparate stores in a intelligent way to surface up a meaningful set of identities.


Threat Detection

Project Iris raises alerts when Azure IoT Edge modules running on the gateway or connected edge devices exhibit behavior that deviates significantly from an established norm. The usage of containers by the Azure IoT Edge runtime permits the development of precise behaviorlal models for describing Azure IoT Edge modules. The types of alerts covered by Project Iris include initial infection, lateral movement, command and control, data exfiltration, and denial of service. These alerts are described in more detail in this blog post.


Below are some hypothetical alerts focused on the edgeHub module. This module has perhaps the most surface area for attack as it exposes several ports for access outside the gateway host.


This alert shows the edgeHub module making an unexpected outbound network connection, for instance in the case of an initial infection to download an exploit payload or reaching out to a command and control host.



Suppose malicious code injected into the edgeHub module attempts to move laterally by probing the network. Project Iris can pick this up - in the example below the edgeHub module is shown reaching out to a modbus device. This is unusual as the edgeHub module by design doesn't directly communicate with any IoT devices.



Now suppose the edgeHub module unexpectedly crashed or was unexpectedly killed:



If configured to integrate with Azure Event Hub, Project Iris can pull in diagnostic events raised by the Azure IoT Hub. Project Iris filters these events to raise interesting security-relevant events. For instance, below is an example of unauthorized access by a device reporting to be a thermostat.



All alerts includes applicable device details gathered from Azure IoT Hub. For instance, the sample below shows configuration details and tags for the aforementioned thermostat:




Whether you're using Azure IoT Edge or other technologies at the edge, we want to hear from you! If you want to learn more about Project Iris, visit the Project Iris web site and click Notify Me. Fill out the contact form and we'll be in touch!

By design containers are meant to be disposable. They are meant to be shipped around to different environments and brought up and down at will. For instance, a container orchestration technology like Kubernetes can automatically bring up new containers in response to a spike in demand, and then tear down the same containers when the demand subsides. Or, as part of the continuous delivery life cycle, the same container image running on a developer's laptop can be spun up in a test environment for verification and then deployed in production by an operations team.


AI-based security solutions like Project Syn use training data to learn what's normal and flag abnormal activity. But the impermanence of containers poses an interesting problem: how can a machine learning system gain the right amount of insight about individual containers in order to raise meaningful alerts? An overly sensitive system that raises alerts before it has enough data to extrapolate from will generate noise and false positives. But an overly conservative system that waits too long for enough data to become available will fail to raise important alerts and result in false negatives.


Container Profiles


Project Syn addresses this problem with the concept of container profiles. In a nutshell, container profiles allow for behavior learned about one container to be shared with similar containers run later in time. This means that for many containers, the training stage can be bypassed altogether, and alerts can be generated immediately after deployment.


Let's take the example of continuous delivery, shown in the figure below. Suppose a new container image is in the process of being deployed to production. First a container from this new image is deployed in a staging environment. After a set training period, Project Syn creates a profile for this container, which captures the behavior learned about this container.



At a later point in time, after the container image has passed the requisite checks in staging, a new container (or set of containers) is deployed in production from the same image. Since Project Syn already has a profile from a previous container coming from the same image, it applies that profile to the new container. The new container bypasses training, and if it happens to be compromised shortly after deployment, Project Syn can immediately raise alerts to that effect.


Profile Matching


How does Project Syn determine when a profile can be applied to a container? It's not based simply on the container's image. Containers run from the same image can exhibit very different behavior based on how they are run. Project Syn uses containers' runtime metadata, such as command line arguments and ports, as part of profile matching.


For example, let's compare three nginx web server containers that are run from the same nginx image. Container A runs only with a private port and is only accessible on the same local virtual network as the container. Container B exposes its private port on port 80 and is accessible from outside the container host (assuming the host firewall is open). Container C exposes its private port on port 8080 and is also accessible outside the container host.



In this case, there are two unique profiles, one profile for container A, and one profile for both container B and C. The difference in public port for container B and C (80 vs 8080) doesn't represent a meaningful difference in behavior.


Profile Matching Using Labels


If you want to, you can explicitly control how profile matching works using Docker object labels. Labels are custom metadata in the form of key-value pairs that can be attached to containers.


Here's how it works: first, you tell Project Syn which label keys you want Project Syn to use for profile matching. When you run a container, you run it with those same labels, and set the label values appropriately. Containers with the same label key-value pairs are matched to the same profile.


What's Next


Container profiles today only work within the context of a single customer. It's not hard to see a future in which customers can opt-in to share profiles with and use profiles from RSA and other customers. This would enable the community to collectively improve container security for everyone. 


Stay tuned for more updates!

IoT gateways are critical pieces of enterprise infrastructure that facilitate secure communication between IoT edge devices and the cloud. As IoT gateways serve as single points of control for all edge devices, they can make an attractive target for attackers, and protecting them is paramount.


RSA Project Iris provides security monitoring and visibility at the IoT edge. This post walks through several examples of how Project Iris can monitor IoT gateways, using the open source EdgeX Foundry platform as a motivating example.


EdgeX Foundry with Project Iris

The EdgeX Foundry platform for IoT gateways consists of many microservices that are deployed as Docker containers. Almost all of these microservices expose web APIs, some for internal consumption within the gateway and others for external use. MongoDb is used for storing data, such as IoT device metadata, logs, and sensor readings from connected IoT edge devices.


Setting up Project Iris on an EdgeX Foundry gateway involves simply deploying the Project Iris Docker container on the gateway. The Project Iris container passively collects data about local EdgeX microservices and securely sends the data to the Project Iris cloud service. The Project Iris cloud service analyzes the data and uses anomaly detection techniques and threat intelligence to identify suspicious activities and raise security alerts.


Threat Detection in Action

So what can Project Iris do? Below are examples of interesting security events that Project Iris can detect.


Initial Infection and Command and Control

A compromised host or microservice container will often execute a malicious payload and initiate suspicious network connections to risky sites, from which further payloads may be downloaded or "command and control" instructions are received to execute.


Project Iris can show when these suspicious payloads are executed or suspicious network connections are made. Below are example alerts for a compromised edgex-device-bacnet service, which is responsible for managing communications to IoT devices that support the BACnet protocol. The first alert shows an anomalous Python process that runs code to connect to an external site, download a payload, and execute it. The second alert is raised for the network connection being made to a known high risk IP address based in Germany.



Lateral Movement

Malicious payloads may probe the network for other endpoints to compromise. This is especially of concern for IoT gateways, which sit on many local edge networks and have privileged access to edge devices.


Project Iris can detect when a microservice container initiates these suspicious probes. The example alerts below show the compromised edgex-support-logging container probing another IoT device, a KMC thermostat, and also trying to connect to another microservice, edgex-device-snmp, on the same host. An alert is also raised for the execution of the ping command used for probing. Project Iris understands that these activities are not typical for the edgex-support-logging microservice and flags them.



Data Exfiltration

Data exfiltration is often the end goal of a compromise. IoT gateways often contain a wealth of sensitive information about edge devices including raw device data, device metadata, and credentials and keys for secure access to edge devices. On the EdgeX Foundry platform, this information is housed within MongoDB.


In the current pre-release version of EdgeX Foundry, MongoDB is set up with remote access enabled and well known default usernames and passwords. As an example of data exfiltration, we can dump the contents of the MongoDB database remotely using the mongodump tool:



This type of activity would cause Project Iris to raise several type of alerts, as shown below. The first alert is raised for a remote network connection to MongoDB.  This connection was flagged as unusual because the database is normally only meant for local use on the gateway itself. The second alert is triggered because of an unusually large data transfer out of MongoDb.



Denial of Service

IoT gateways are especially susceptible to denial of service attacks because of the large number of edge devices they manage. Compromised edge devices could launch denial of service attacks at the gateway or through the gateway to other hosts.


As an example, we used a compromised network signal tower device to initiate a large volume of network connections to the gateway. Project Iris can detect this type of activity, as shown in the first alert below:



A denial of service attack can subsequently lead to one or more microservice containers crashing in an unexpected way. Project Iris can also detect this, as shown in the second alert above.



The goal of Project Iris is to bring security monitoring and threat detection capabilities to the IoT edge. In this post we walked through how Project Iris can be used to secure IoT gateways, which are critical enterprise assets responsible for managing edge devices. In a subsequent post, we'll talk about what Project Iris can do to bring similar visibility down to the edge devices themselves.


If you're interested in trying out Project Iris, register here and the RSA Labs team will notify you when it's available.

Web applications and web services are probably the most commonly produced type of software, and they are increasingly being developed and deployed as containers. Among the top downloaded container images on the public Docker Hub are many related to web application development, such as nginx, MySQL, PostgreSQL, the Apache HTTP server, Ruby, PHP, Tomcat, and Django.


This post walks through an example scenario of detecting a web application attack using Project Syn. The scenario is admittedly simple and contrived, but we believe it's illustrative of how Syn can help in the real world.



In our scenario we use the Damn Vulnerable Web Application (DVWA) as the web app to be exploited. The application is intentionally riddled with vulnerabilities and is often used in security pen-test training. We deploy the entire web app, based on the LAMP stack, in a single Docker container. (Typically a web application would be deployed as many containers but a single container is sufficient for our purposes.)


We also deploy the Project Syn container side-by-side with the DVWA container on the same Docker host. The Project Syn container collects security-related data about other containers on the same host (in this case the DVWA container) and forwards them to the Syn cloud service for analysis and alerting.


Here's the output of docker ps:


Exploit, Payload Delivery and Execution

Among the many vulnerabilities in the DVWA is one that permits the upload and execution of malicious code disguised as image files.



We use the OWASP Zed Attack Proxy to exploit the vulnerability to install a malicious PHP file, bad.php, and execute it. The PHP file contains a small bit of code that when executed launches a Python process that connects to an external IP hosting a Remote Admin Tool (RAT). The Python process downloads a full payload from the external IP and executes it, giving the operator of the RAT full control over the container.



The Syn service raises alerts when it detects the launching of the malicious Python process:




In addition, the Syn service raises alerts when it detects network traffic to the malicious external IP on ports 8080 and 443:

Data Exfiltration

Once the payload is installed, the operator of the RAT has full control over the container and can do any number of things. In our case, we are using the open source Pupy RAT tool. We start an interactive shell on the container,  dump the MySQL database to a file, and download it.



The Syn service detects the anomalous execution of the mysqldump process:


The Syn service also detects that data exfiltration through the producer-consumer ratio (PCR) metric.


The PCR metric tracks a normalized ratio of network bytes in and out of a component. Producers (PCR value between 0 and 1) have more data flowing out than in, while consumers (PCR value between -1 and 0) have more data flowing in than out. Components tend to have pretty stable PCR values over time.


In our scenario, the Syn service detected a significant change in the DVWA container's PCR. It changed from being a moderate producer (.613) to being a strong producer (.979) at the moment the database dump was downloaded.




The beauty of containers is that they are designed to be limited in function and behavior. As such, from a security perspective, we believe we can precisely model what the expected/normal behavior for any container should be, and raise targeted alerts when anomalies arise. We walked through a simplified scenario above of using Project Syn to detect the exploitation of a containerized web application.


If you're interested in giving Project Syn a spin, check out or the Getting Started video. Feedback is welcome!