ecat / nwe compatibility with meltdown/spectre mitigations/patches
Can you please confirm if there are any compatibility issues with ECAT 4.1-4.4 and https://support.microsoft.com/en-us/help/4072699/january-3-2018-windows-security-updates-and-antivirus-software or the registry key mitigations https://portal.msrc.microsoft.com/en-US/security-guidance/advisory/ADV180002 [our AV vendor is pushing out the registry keys and we’re deploying the patch]
Best I could find is https://community.rsa.com/docs/DOC-85418 but it doesn’t state agent compatibility with the patches.
- Community Thread
- Forum Thread
- NetWitness Endpoint
- patch compatibility
- RSA NetWitness Endpoint
- RSA NetWitness Platform
The RSA NWE team is currently working through testing to ensure there are no issues with the patch and NWE agent. The team is testing multiple flavors of Windows with the 22.214.171.124 agent. If your environment is not already on the latest version (126.96.36.199) I would strongly recommend upgrading both the server and agents to ensure there are no issues since that is what is being tested.
The team is shooting to have this completed by Monday January 15th. HTH.
the problem is the fixes can be pushed silently by people's AV vendors in signatures. e.g sophos.
the second problem - not everyone is on 188.8.131.52. and not everyone can go and upgrade to it urgently and immediately.... (generally most people have and EDR agent test and deployment internal cycle and change management contraints)
are there any plans to test the rest of the versions not officially retired.
Completely understand not all customers will be able to quickly upgrade to the latest version. However testing is normally a time consuming process and testing every agent version (4.3 for example has 6 different patches) isn't feasible for a quick turn around, which is the immediate goal. I'm not sure of any plans to test other versions of the agent at this point. Sorry 😕
Out of curiosity...what version are you on?
heh. we're on a mix of 184.108.40.206, 220.127.116.11 (yes yes i know there's awful bugs, half of which we reported) and 18.104.22.168 [roughly 1/3 each]
Personally I think RSA testing should be based on is the telemetry submitted by ecat server and the official support lifecycles. As such you should have a semi automated test harness for each version . Just testing the latest version is frankly not acceptable?
it looks like quite a few AV vendors have decided to push the patch compatible flag (here)
edit - the customers still control windows update deployment, even if the AV vendor sets the compatibility flag (and EDR is potentially incompatible), it's not the memory management settings here 4072698 that most AV engines set , but the compatibility flag from here 4072699 ]
Basically as a customer we have to delay releasing the patch until RSA finishes testing. [hopefully RSA tests both the patch and extra mitigation registry keys]
However the 15th Jan/only the latest version testing is not satisfactory
Perhaps the testing can be extrapolated for:
a) versions with older TDI driver < 4.4
b) versions with the WFP driver (4.4+)
at least to some extent.
Vladimir, those are good concerns you are raising, likely shared by others. I will break down my answer into a few separate points, and hopefully address them all:
1. Registry key update management: The Endpoint agent does not rely on registration with the Microsoft Windows Security Center in order to get updates. As such, it is not affected by the registry key. Instead of relying on the WSC, it will self-update through the Kernel Adaptation Module (KAM) mechanism, a service offered through RSA Live, as soon as the underlying OS was updated. As of today, all but two kernels have been updated, with the remaining two coming online tomorrow. (The MS KB article on this issue details how the registry key works. You will read that only agents registered with WSC respond to that key.)
2. The KAM updates support all the agents since 22.214.171.124. If anyone happens to be on an older agent, we strongly suggest an upgrade to our latest versions, both for features and stability reasons, and because support is not provided on older versions. However, older agents will still be compatible with the updates provided through KAM, as long as they are based on version 126.96.36.199 or later. Your 188.8.131.52 agents will be included.
3. Testing follows the usual procedure we've always had in place for updating the agents in sync with the underlying OS since the feature was introduced, in 4.1. We do have confidence that the patches are functionally compatible with all of our supported agents at this time. TDI should be included, as it always is.
4. We are running some supplemental server-side performance testing, (which Mike Gotham was referring to). We will share our findings, should we find any worrysome slow-downs. Note that the testing concerns the server, rather than the agents, as the MS patch will affect intense I/O operations, which the agents don't do. As of today, we have not found any issue, but testing continues.
Please let me know if this answers your concerns. I will be happy to answer any follow-up question.
is what you're saying - RSA are not going to mark the new kernels as compatible via live for KAM until you finish perf and BSOD + whatever standard testing for ecat 184.108.40.206 onwards? (I guess my only caveat is the live KAM file is ecat agent version agnostic I thought, hence needing info on ECAT version compatibility with the kernel update)
>We do have confidence that the patches are functionally compatible with all of our supported agents at this time.
specifically for the Jan updates with the AV compat flag set?
We do not depend on the flag. We simply work with whatever OS patching level you have, without triggering an update. In this, we are different from AV. When you are ready to turn on the registry key to allow the update, the agent will incorporate the new symbols to match that patch. Note that the single update needed for our agent in this is a simple update of the kernel symbols, the same we do for every other patch. There is no actual patch that is needed for our software. Because we do have access to early patches, we already know we are compatible, and will not experience BSOD -- that testing has already happened, and has included performance testing of the agents. Server side, we know we are compatible, and expect little impact on performance, if any. PSR testing on the server side takes longer, so we are still completing that due diligence, to find out if the MS patch impacts our recommendations for scale. So far, we have not seen an indication that it does.