Upgrade Guide 11.4.1: Post Upgrade Tasks

Document created by RSA Information Design and Development Employee on Apr 23, 2020Last modified by RSA Information Design and Development Employee on Apr 28, 2020
Version 2Show Document
  • View in full screen mode
 

This topic is divided into two sections. Complete the tasks in one of the following sections based on your upgrade path:

Post Upgrade Tasks for Customers Upgrading From 11.3.x.x or 11.4.0.x

Perform all the tasks in this section if you are upgrading from 11.3.x.x or 11.4.0.x to 11.4.1.0

General

Task 1. Make Sure Services Have Restarted and Are Capturing and Aggregating Data

Make sure that services have restarted and are capturing data (this depends on whether or not you have auto-start enabled).

If required, restart data capture and aggregation for the following services:

  • Decoder
  • Log Decoder
  • Broker
  • Concentrator
  • Archiver

Start Network Capture

  1. In the NetWitness Platform menu, go to Admin > Services.
    The Services view is displayed.
  2. Select each Decoder service.
  3. Under (actions), select View > System.

  4. In the toolbar, click

Start Log Capture

    1. In the NetWitness Platform menu, go to Admin > Services.
      The Services view is displayed.
    2. Select each Log Decoder service.
    3. Under (actions), select View > System.
    4. In the toolbar, click
  • Start Aggregation

    1. In the NetWitness Platform menu, go to Admin > Services.

      The Services view is displayed.

    2. For each Concentrator, Broker, and Archiver service:

      1. Select the service.
      2. Under (actions), select View > Config.
      3. In the toolbar, click

    Event Stream Analysis

    Note: These Event Stream Analysis (ESA) tasks are for upgrades from 11.3.x.x.

    Task 2. Verify the Status of the ESA Rule Deployments

    Check the status of the ESA rule deployments.

    1. Go to Configure > ESA Rules > Services tab.
      The Services view is displayed, which shows the status of your ESA services and deployments.
    2. In the options panel on the left, select an ESA service.
    3. For each service listed, look at the deployment tabs in the panel on the right. Each tab represents a separate ESA rule deployment.
    4. For each ESA rule deployment:
      1. In the Engine Stats section, look at the Events Offered and the Offered Rate. They confirm that the data is being aggregated and analyzed properly. If you see 0 for Events Offered, nothing is coming in for the deployment.
      2. In the Rule Stats section, look at the Rules Enabled and Rules Disabled. If there are any disabled rules, look in the Deployed Rule Stats section below to view the details of the disabled rules. Disabled rules show a white circle. Enabled rules show a green circle.

      ESA Rules Services view - Check the status of ESA rule deployments

    5. If you notice any disabled rules that should be enabled:
      1. Go to Configure > ESA Rules > Rules tab and redeploy the ESA rule deployments that contain disabled rules.
      2. Go back to the Services tab and check to see if the rules are still disabled. If the rules are still disabled, check the ESA Correlation service log files, which are located at /var/log/netwitness/correlation-server/correlation-server.log.

    Task 3. (Conditional) Update the Multi-Valued and Single-Valued Parameter Meta Keys for the Latest Endpoint, UEBA, and RSA Live Content Rules

    Note: If you have already completed this task during an upgrade to 11.3.0.2, 11.3.1.1, 11.3.2, or 11.4.0.x, you do not need to do it again.

    To use the latest Endpoint, UEBA, and Live content rules, you must update the multi-valued parameter field on the ESA Correlation service to include all of the meta keys in the default-multi-valued field. You must also update the single-valued parameter field to include all of the meta keys in the default-single-valued field.

    The multi-valued parameter shows the string array meta keys used for your ESA rules. This parameter is equivalent to the Event Stream Analysis service ArrayFieldNames parameter in NetWitness Platform versions 11.2 and earlier.

    Caution: Any changes that you make to the multi-valued parameter may cause an error when you deploy your existing rules. You can update the multi-valued parameter, resync your meta keys, and update the ESA rules at your convenience. You may want to add a couple meta keys at a time to reduce the number of reported errors.

    Note: If you see a warning message in the ESA Correlation server error logs that means there is a difference between the default-multi-valued parameter and multi-valued parameter meta key values, the new Endpoint, UEBA, and Live content rules will not work. Completing this procedure should fix the issue. For example warning messages, see Example ESA Correlation Server Warning Message for Missing Meta Keys.

    1. After you upgrade to 11.4.1, go to Admin > Services, and in the Services view, select an ESA Correlation service and then select  > View > Explore.
    2. In the Explore view node list for the ESA Correlation service, select correlation > stream.
    3. Compare the multi-valued parameter meta keys with the required default-multi-valued meta keys. Copy and paste the missing string array meta keys from the default-multi-valued parameter to the multi-valued parameter. (You may want to copy only a couple meta keys at one time to reduce the number of reported errors).
    4. Copy and paste the string meta keys from the default-single-valued parameter to the single-valued parameter.
    5. Apply the changes on the ESA Correlation service:
    6. Go to Configure > ESA Rules and click the Settings tab.
      • In the Meta Key References, click the Meta Re-Sync (Refresh) icon (Meta Re-Sync (refresh) icon ).
      • If you have multiple ESA Correlation services, make the same meta key changes on each ESA Correlation service.
    7. If you are using any of the default-multi-valued or default-single-valued meta keys in your ESA Advanced rules, update the rule syntax. See also Task 4. (Conditional) Adjust Custom ESA Rule Builder and ESA Advanced Rules .
    8. If you used any meta keys in the ESA rule notification templates from the default-multi-valued parameter list, update the templates with the meta key changes. See "Configure Global Notification Templates" in the System Configuration Guide.
    9. Deploy your ESA rule deployments.
    10. Check your rules for error messages in the ESA Rules section of the ESA rule deployment or check the ESA Correlation error logs for errors.
      • To access the error messages in the ESA rule deployment, go to Configure > ESA Rules > Rules tab, select a deployment in the options panel on the left, and go to the ESA Rules section.
      • To access the ESA Correlation service logs, you can use SSH to get in the system and go to: /var/log/netwitness/correlation-server/correlation-server.log.

    Task 4. (Conditional) Adjust Custom ESA Rule Builder and ESA Advanced Rules

    Note: If you have already completed this task during an upgrade to 11.3.0.2, 11.3.1.1, 11.3.2, or 11.4.0.x, you do not need to do it again.

    Update your ESA Rule Builder and ESA Advanced rules to work with the string and string array meta keys listed in the default-multi-valued and default-single valued parameter fields for the ESA Correlation service. You can add additional meta keys to the multi-valued and single-valued parameters.

    For example, if you use ec.outcome as a single-valued meta key in your ESA rule as shown below:

    @RSAAlert

    SELECT * FROM Event((ec_outcome IN ( 'Success' )))

    .win:time_length_batch(2 Minutes, 2)

    HAVING COUNT(*) >= 2;

    If you add ec.outcome to the multi-valued parameter field, you need to update your rule as shown below:

    @RSAAlert

    SELECT * FROM Event(( 'Success' = ANY( ec_outcome ) ))

    .win:time_length_batch(2 Minutes, 2)

    HAVING COUNT(*) >= 2;

    For more information, see “Configure Meta Keys as Arrays in ESA Correlation Rule Values” in the ESA Configuration Guide.

    ESA Troubleshooting information

    For more information, see ESA Troubleshooting Information.

    Investigate

    Task 5. (Conditional - For Custom Roles Only) Adjust investigate-server Permissions for Custom User Roles

    After upgrading to Version 11.4.1.0, the built-in user roles for analysts using Investigate have the following permissions enabled:

    • investigate-server.columngroup.read
    • investigate-server.metagroup.read
    • investigate-server.profile.read

    After you upgrade to 11.4.1.0, NetWitness Platform does not add these permissions to custom analyst roles so you must enable them for your custom roles as described in this procedure (see the System Security and User Management Guide for comprehensive information about user roles).

    Users who are assigned a custom user role that does not have these permissions will see issues in the Navigate view and Legacy Events view. If any of the three permissions are disabled, the Load Values button is not displayed in the Navigate view. When column groups permission is disabled, there is an additional issue in the Legacy Events view: Only the Detail view is visible and you cannot select different views and column groups.

    To enable the permissions for a user role:

    1. Go to Admin > Security and click the Roles tab.
    2. Select the custom user role that needs to be edited and click (edit icon).
    3. In the Edit Role dialog, ensure that these three permissions are enabled:
      investigate-server.columngroup.read
      investigate-server.metagroup.read
      investigate-server.profile.read
    4. Click Save to save your changes. When analysts with the custom user role log in to the NetWitness Platform, the changes will be in effect.

    Respond

    The Primary ESA server must be upgraded to 11.4.1 before you can complete these tasks.

    Note: After upgrading the primary NW Server (including the Respond Server service), the Respond Server service will not be re-enabled until after the Primary ESA host is also upgraded to 11.4.1. The Respond post-upgrade tasks only apply after the Respond Server service is upgraded and is in the enabled state.

    Task 6. (Conditional) Restore any Respond Service Custom Keys in the Aggregation Rule Schema

    Note: If you did not manually customize the incident aggregation rule schema, you can skip this task.

    If you added custom keys in the var/lib/netwitness/respond-server/data/aggregation_ rule_schema.json file for use in the groupBy clause for 11.x, modify the /var/lib/netwitness/respond-server/data/aggregation_rule_schema.json file and add the custom keys from the automatic backup file.

    The backup file is located in /var/lib/netwitness/respond-server/data and it is in the following format:
    aggregation_rule_schema.json.bak-<time of the backup>

    Task 7. (Conditional) Restore any Customized Respond Service Normalization Scripts

    Note: If you did not manually customize any alert normalization scripts, you can skip this task.

    To prevent overwriting future customizations, custom normalization script files are available in NetWitness Platform 11.4 and later. Add any custom logic to the custom_normalize_<alert type>.js files.

    1. Locate any custom logic from the backup Respond normalization scripts located in the /var/lib/netwitness/respond-server/scripts.bak-<timestamp> directory, where <timestamp> is the time that the backup completed:
      data_privacy_map.js
      normalize_alerts.js
      normalize_core_alerts.js
      normalize_ecat_alerts.js
      normalize_ma_alerts.js
      normalize_ueba_alerts.js
      normalize_wtd_alerts.js
      utils.js

    2. Edit the new 11.4 or later script files in the /var/lib/netwitness/respond-server/scripts directory to include any logic from the back up files. If you have any customizations in the normalization files, add them to the normalization files with the “custom” prefix.
      data_privacy_map.js
      custom_normalize_alerts.js
      custom_normalize_core_alerts.js
      custom_normalize_ecat_alerts.js
      custom_normalize_ma_alerts.js
      custom_normalize_ueba_alerts.js
      custom_normalize_wtd_alerts.js
      utils.js

      For Example, the custom_normalize_core_alerts.js is the normalization script for ESA to add up any custom logic. This java script file has a function ‘normalizeAlert’ with parameters headers, rawAlert, and normalizedAlert. The variable ‘normalized’ is a immutable copy object which has an embedded object of list of normalized events. So if you have any custom meta keys configured for the events then you have to iterate through the ‘normalized.events’ to populate the appropriate meta keys with values from the ‘rawAlert.events’ object. Below is the sample code.


    Task 8. (Conditional) Add Respond Notification Settings Permissions

    Note: If you already configured these permissions in 11.2 or later, you can skip this task.

    Respond Notification Setting permissions enable Respond Administrators, Data Privacy Officers, and SOC Managers to access Respond Notification Settings (Configure > RespondNotifications), which enable them to send email notifications when incidents are created or updated.

    To access these settings, you must add additional permissions to your existing built-in NetWitness Platform user roles. You must also add permissions to your custom roles.

    See the “Respond Notification Settings Permissions” topic in the NetWitness Respond Configuration Guide.

    For detailed information about user permissions, see the System Security and User Management Guide.

    Go to the Master Table of Contents to find all RSA NetWitness Platform 11.x documents.

    Post Upgrade Tasks for Customers Upgrading From 11.2.x.x

    Perform all the tasks in this section if you are upgrading from 11.2.x.x to 11.4.1.

    General

    Task 1. Make Sure Services Have Restarted and Are Capturing and Aggregating Data

    Make sure that the services have restarted and capturing data (this depends on whether or not you have auto-start enabled).

    If required, restart data capture and aggregation for the following services:

    • Decoder
    • Log Decoder
    • Broker
    • Concentrator
    • Archiver

    Start Network Capture

    1. In the NetWitness Platform menu, go to Admin > Services.
      The Services view is displayed.
    2. Select each Decoder service.
    3. Under (actions), select View > System.

    4. In the toolbar, click

    Start Log Capture

    1. In the NetWitness Platform menu, go to Admin > Services.
      The Services view is displayed.
    2. Select each Log Decoder service.
    3. Under (actions), select View > System.
    4. In the toolbar, click
  • Start Aggregation

    1. In the NetWitness Platform menu, go to Admin > Services.
      The Services view is displayed.

    2. For each Concentrator, Broker, and Archiver service:

      1. Select the service.
      2. Under (actions), select View > Config.
      3. In the toolbar, click

    Task 2. Set Up Context Menu Actions User Permissions

    Complete the following steps for Analysts, SOC Managers, Data Privacy Officers roles to set up their Context Menu Actions. You must complete these steps for the Analysts, SOC Managers, and Data Privacy Officers roles.

    1. In the NetWitness Platform menu, go to Admin > Security > Roles.
    2. Double-click on the user role (for example, Data Privacy Officers), or click to select the role and click (Edit ).

    3. In the Edit Role view, under Permissions on the Administration tab, select the Manage Logs, Manage Plugins, and Manage System Settings check boxes and click Save.

    4. Complete steps 1 through 3 for the Analysts and SOC Managers roles in addition to Data Privacy Officers.

    Task 3. Add "Manage Jobs" Permission to Roles Missing this Permission

    Add the 'Manage Jobs' Administration permission to the following roles:

    • SOC_Managers
    • Operators
    • Data_Privacy_Officers
    1. In the NetWitness Platform menu, go to Admin > Security and click Roles.
    2. Select the role you need to update (that is, SOC_Managers, Operators, or Data_Privacy_Officers) and click Edit .

    3. Click Administration, select the Manage Jobs checkbox, and click Save.

    4. Complete steps 1 through 3 inclusive for all three roles (SOC_Managers, Operators, and Data_Privacy_Officers).

    Task 4. (Conditional) Reissue Certificates for Your Hosts

    Before you upgrade, you must ensure that the internal RSA-issued certificates, such as CA Certificate and Service certificates, are renewed.

    The validity for NetWitness Platform certificates are as follows:

    • CA root certificate for 11.x deployment is valid for 10 years
    • CA root certificate for 10.6.x deployment is valid for 5 years
    • Service certificates are valid for 1000 days

    You can view the expiration details by running the ca-expire-test-sh script on the NetWitness Server. For more information, see Reissue root CA security certificates on RSA NetWitness Platform 11.x and download the script.

    To renew the CA certificates or service certificates, see the Reissue root CA security certificates on RSA NetWitness Platform 11.x.

    Note: If you have Windows Legacy Collectors (WLC) in your deployment, renew the certificates of the WLC after renewing the certificates of the NetWitness Admin Server.

    For more information, see the "Reissue Certificates" topic in the System Maintenance Guide.

    Task 5. Modify the Analyst Role investigate-server Permissions

    The default permissions for the SOC Managers, Malware Analysts, and Analysts roles in 11.3 or later have specific permissions required to view and work in the Event Analysis view. Prior to 11.3, the default permissions were different.

    In addition, the predicate.manage permission should not be assigned to the SOC Managers, Malware Analysts, and Analysts roles because it grants them access to get-predicates, edit-predicates, remove-predicates, remove-all-predicates and so on. This access could be a security risk because it allows them to circumvent settings that restrict access to certain data.

    As a result, if you are upgrading from version 11.2.x.x to 11.4 or later, you must update the default permissions to match the new default permissions, as described in the following procedure.

    1. Go to Admin > Security > Roles.
    2. Complete the following steps for SOC Managers, Malware Analysts, and Analysts roles.
      1. Select the user role checkbox (for example, Analysts) and click Edit .

      1. Under Permissions, click the Investigate-server tab.
      2. Make sure that the following permissions are not selected.
        • investigate-server.*
        • investigate-server.predicate.manage
      3. Select the following permissions.
        • investigate-server.content.export
        • investigate-server.content.reconstruct
        • investigate-server.event.read
        • investigate-server.metagroup.read
        • investigate-server.predicate.read

      4. Click Save.

    Task 6. (Conditional) Reconfigure PAM RADIUS Authentication

    If you configured PAM RADIUS authentication in 11.2.x.x using the pam_radius package, you must reconfigure it in 11.4 or later using the pam_radius_auth package.

    Run the following commands on the NW Server host.

    Note: If you configured pam_radius in 11.2.x.x, perform step 1 to uninstall the existing version. If not, you can proceed with step 2.

    1. Verify the existing page and uninstall the existing pam_radius file:
      rpm –qa |grep pam_radius
      yum erase pam_radius
    2. To install the pam_radius_auth package, run the following command:
      yum install pam_radius_auth
    3. Edit the RADIUS configuration file, /etc/raddb/server, as follows and add the configurations for the RADIUS server:
      # server[:port] shared_secret timeout (s)
      server secret 3

      For example: 111.222.33.44 secret 1
    4. Edit the NW Server host PAM configuration file /etc/pam.d/securityanalytics to add the following line. If the file does not exist, create it and add the following line:
      auth sufficient pam_radius_auth.so
    5. Provide write permission to /etc/raddb/server files by running the following command:
      chown netwitness:netwitness /etc/raddb/server
    6. Copy the pam_radius_auth library by running the following command:
      cp /usr/lib/security/pam_radius_auth.so /usr/lib64/security/

    7. After making the changes to the pam_radius_auth configurations, restart the Jetty server by running the following command:
      systemctl restart jetty

    Task 7. (Conditional) If NetWitness Platform Has No Web Access, Upload Response .bin File Again (License Server)

    If your NetWitness deployment does not have Internet access, after you upgrade to 11.4 or later, you must upload the response .bin file again to view the license information in the Admin > System > Licensing view in the NetWitness Platform User Interface. See “Upload an Offline Capability Response to NetWitness Platform” in the Licensing Management Guide for instructions. Go to the Master Table of Contents to find all RSA NetWitness Platform 11.x documents.

    Task 8. Change Minimum Password Length from Eight Characters to Nine Characters

    In versions 11.2.x.x, the NetWitness Platform minimum password length is eight characters. In 11.3.x.x and later, the minimum length is nine characters. After you upgrade from 11.2.x.x to 11.4 or later, set the minimum password length to nine characters as described under "Configure Password Complexity" in the System Security and User Management Guide.

    Event Stream Analysis

    Note: These Event Stream Analysis (ESA) tasks are for upgrades from 11.2.x.x.

    Task 9. View the String Array Type Meta Keys on the ESA Correlation Service and Next Steps

    To support Endpoint, UEBA, and RSA Live content, a data change from single-value (string) to multi-value (string array) is required for several meta keys within the ESA Correlation service for 11.3 and later. Additional string meta keys are also required.

    If the meta keys used for your ESA rules are different from the required default multi-value meta keys, your ESA rules continue to work, but you should update your ESA rules to use the required meta keys as soon as possible to ensure that your rules continue to deploy properly.

    The ESA Correlation service has the following multi-valued (string array) and single-valued (string) parameters:

    • multi-valued - Shows the string array meta keys currently used for your ESA rules. For an upgrade to NetWitness Platform 11.4 or later, it shows the existing string array meta keys before the upgrade. (This parameter is equivalent to the Event Stream Analysis service ArrayFieldNames parameter in NetWitness Platform versions 11.2 and earlier.)
    • single-valued - Shows the string meta keys currently used for your ESA rules. For an upgrade to NetWitness Platform 11.4 or later, this parameter value is empty.
    • default-multi-valued - Shows the required string array meta keys for the latest version.
    • default-single-valued - Shows the required string meta keys for the latest version.

    Note: If you have the same value in the single-valued and multi-valued parameter fields, the single-valued meta key value takes precedence over the multi-valued meta key value.

    1. View the multi-valued and single-valued meta key parameters on the ESA Correlation service:

      1. Go to Admin > Services, and in the Services view, select an ESA Correlation service and then select Actions icon> View > Explore.

      2. In the Explore view node list for an ESA Correlation service, select correlation > stream.

    2. Your ESA rules continue to work, but if you are using Live, UEBA, or Endpoint rules, follow the Task 12. (Conditional) Update the Multi-Valued and Single-Valued Parameter Meta Keys for the latest Endpoint, UEBA, and RSA Live Content Rules procedure.

    Caution: Any changes that you make to the multi-valued parameter may cause an error when you deploy your existing rules. You can update the multi-valued parameter, resync your meta keys, and update the ESA rules at your convenience. You may want to add a couple meta keys at a time to reduce the number of reported errors.

    Note: If you are using multiple ESA Correlation services, the multi-valued and single-valued parameters should be the same on each ESA Correlation service.

    Task 10. (Conditional) Update RSA Live ESA Rules with Meta Type Changes from String to Array

    The following table lists ESA rules from RSA Live that had meta key type changes from String to Array in NetWitness Platform 11.3.x and 11.4.

                                                    
    Rule #Rule NameArray Type Meta Keys in 11.3.x and 11.4
    1RIG Exploit Kitthreat_category
    2 AWS Critical VM Modifiedalert
    3Multiple Successful Logins from Multiple Diff Src to Same Dest host.src and host.dst
    4Multiple Successful Logins from Multiple Diff Src to Diff Dest host.src and host.dst
    5Multiple Failed Logins from Multiple Diff Sources to Same Dest host.src and host.dst
    6Multiple Failed Logins from Multiple Users to Same Destination host.src and host.dst
    7User Login Baselinehost.src and host.dst
    1. If you:
      • Deployed these rules before version 11.3:
        1. Note any rule parameters that you have changed so you can adjust the rules for your environment.
        2. Download the updated rules from RSA Live.
        3. Reapply any changes to the default rule parameters and deploy the rules.

          (For instructions, see “Download RSA Live ESA Rules” in the Alerting with ESA Correlation Rules User Guide.)

      • Are deploying these rules for the first time in version 11.4 or later, follow the customization directions within the ESA rule descriptions. Rules 3 to 7 in the above table require that the Context Hub lists for User_Whitelist, Host_Whitelist and IP_Whitelist to be added as enrichments to ESA. (See “Configure Context Hub List as an Enrichment Source” in the Alerting with ESA Correlation Rules User Guide.)
    2. Deploy the ESA rule deployment that contains these rules. (See “ESA Rule Deployment Steps” in the Alerting with ESA Correlation Rules User Guide.)

    Task 11. Verify the ESA Rule Deployments

    After you upgrade to 11.4 or later, verify your ESA rule deployments. For every ESA host, a new deployment is created in the format “<ESA-Hostname> – ESA Correlation”.

    1. Make sure that a new deployment was created.
    2. Make sure that the new deployment contains an ESA Correlation service, data sources, and rules for all previous deployments on that ESA host.
    3. Make sure that the ESA Correlation service has status of “Deployed”.
    4. If an ESA rule status incorrectly shows as “Disabled” or shows the Red Exclamation point icon icon in the Status column, you need to determine the issue to fix the rule. If a disabled rule has an error message, it now shows Red Exclamation point icon in the Status field. You can hover over the rule to view the error message tooltip without going to the error log. (The ESA Correlation Service log files are located at /var/log/netwitness/correlation-server/correlation-server.log)
      See ESA Troubleshooting Information.
    5. Check the status of the overall ESA rule deployment. If the ESA rule deployment is successful, the ESA Services and ESA Rules show a status of “Deployed,” the Data Sources show a green circle, and the Deploy Now button is disabled.

    For a detailed example, see the ESA Configuration Guide. For Deployment information, see “ESA Rule Deployment Steps” in the Alerting with ESA Correlation Rules User Guide. For troubleshooting information, see the Alerting with ESA Correlation Rules User Guide. Go to the Master Table of Contents to find all RSA NetWitness Platform 11.x documents.

    Task 12. (Conditional) Update the Multi-Valued and Single-Valued Parameter Meta Keys for the latest Endpoint, UEBA, and RSA Live Content Rules

    To use the latest Endpoint, UEBA, and Live content rules, you must update the multi-valued parameter field on the ESA Correlation service to include all of the meta keys in the default-multi-valued field. You must also update the single-valued parameter field to include all of the meta keys in the default-single-valued field.

    Caution: Any changes that you make to the multi-valued parameter may cause an error when you deploy your existing rules. You can update the multi-valued parameter, resync your meta keys, and update the ESA rules at your convenience. You may want to add a couple meta keys at a time to reduce the number of reported errors.

    Note: If you see a warning message in the ESA Correlation server error logs that means there is a difference between the default-multi-valued parameter and multi-valued parameter meta key values, the new Endpoint, UEBA, and Live content rules will not work. Completing this procedure should fix the issue. For example warning messages, see Example ESA Correlation Server Warning Message for Missing Meta Keys.

    1. After an upgrade to 11.4 or later, go to Admin > Services, and in the Services view, select an ESA Correlation service, and then select Actions icon> View > Explore.
    2. In the Explore view node list for the ESA Correlation service, select correlation > stream.
    3. Compare the multi-valued parameter meta keys with the required default-multi-valued meta keys. Copy and paste the missing string array meta keys from the default-multi-valued parameter to the multi-valued parameter. (You may want to copy only a couple meta keys at one time to reduce the number of reported errors).
    4. Copy and paste the string meta keys from the default-single-valued parameter to the single-valued parameter.
    5. Apply the changes on the ESA Correlation service:
    6. Go to Configure > ESA Rules and click the Settings tab.
      • In the Meta Key References, click the Meta Re-Sync (Refresh) icon (Meta Re-Sync (refresh) icon ).
      • If you have multiple ESA Correlation services, make the same meta key changes on each ESA Correlation service.
    7. If you are using any of the default-multi-valued or default-single-valued meta keys in your ESA Advanced rules, update the rule syntax. See also Task 13. (Conditional) Adjust Custom ESA Rule Builder and ESA Advanced Rules
    8. If you used any meta keys in the ESA rule notification templates from the default-multi-valued parameter list, update the templates with the meta key changes. See "Configure Global Notification Templates" in the System Configuration Guide. Go to the Master Table of Contents to find all RSA NetWitness Platform 11.x documents.
    9. Deploy your ESA rule deployments.
    10. Check your rules for error messages in the ESA Rules section of the ESA rule Deployment or check the ESA Correlation error logs for errors.
      • To access the error messages in the ESA rule deployment, go to Configure > ESA Rules > Rules tab, select a deployment in the options panel on the left, and go to the ESA Rules section.
      • To access the ESA Correlation service logs, you can use SSH to get in the system and go to: /var/log/netwitness/correlation-server/correlation-server.log.

    Task 13. (Conditional) Adjust Custom ESA Rule Builder and ESA Advanced Rules

    Update your ESA Rule Builder and ESA Advanced rules to work with the string and string array meta keys listed in the default-multi-valued and default-single valued parameter fields for the ESA Correlation service. You can add additional meta keys to the multi-valued and single-valued parameters.

    For example, if you use ec.outcome as a single-valued meta key in your ESA rule as shown below:

    @RSAAlert

    SELECT * FROM Event((ec_outcome IN ( 'Success' )))

    .win:time_length_batch(2 Minutes, 2)

    HAVING COUNT(*) >= 2;

    If you add ec.outcome to the multi-valued parameter field, you need to update your rule as shown below:

    @RSAAlert

    SELECT * FROM Event(( 'Success' = ANY( ec_outcome ) ))

    .win:time_length_batch(2 Minutes, 2)

    HAVING COUNT(*) >= 2;

    For more information, see “Configure Meta Keys as Arrays in ESA Correlation Rule Values” in the ESA Configuration Guide. Go to the Master Table of Contents to find all RSA NetWitness Platform 11.x documents.

    ESA Troubleshooting Information

    Note: To avoid unnecessary processing overhead, the Ignore Case option has been removed from the ESA Rule Builder - Build a Statement dialog for meta keys that do not contain text data values. During the upgrade to 11.4 or later, NetWitness Platform does not modify existing rules for the Ignore Case option. If an existing Rule Builder rule has the Ignore Case option selected for a meta key that no longer has the option available, an error occurs if you try to edit the statement and try to save it again without clearing the checkbox.

    To support Endpoint and UEBA content as well as changes to ESA rules from Live, a data change from single-value (string) to multi-value (string array) is required for several meta keys within the ESA Correlation service. In NetWitness Platform 11.4 or later, ESA automatically adjusts the operator in the rule statement when there is a change from string to string array, but you still may need to make manual adjustments to adjust for the string array changes.

    To change the string type meta keys to string array type meta keys manually in 11.4 or later, see “Configure Meta Keys as Arrays in ESA Correlation Rule Values” in the ESA Configuration Guide.

    To use the latest Endpoint, UEBA, and Live content rules, the following default multi-valued meta keys are required on the ESA Correlation service in NetWitness Platform version 11.4 or later:

    action , alert , alert.id , alias.host , alias.ip , alias.ipv6 , analysis.file , analysis.service , analysis.session , boc , browserprint , cert.thumbprint , checksum , checksum.all , checksum.dst , checksum.src , client.all , content , context , context.all , context.dst , context.src , dir.path , dir.path.dst , dir.path.src , directory , directory.all , directory.dst , directory.src , email , email.dst , email.src , eoc , feed.category , feed.desc , feed.name , file.cat , file.cat.dst , file.cat.src , filename.dst , filename.src , filter , function , host.all , host.dst , host.orig , host.src , host.state , inv.category , inv.context , ioc , ip.orig , ipv6.orig , netname , OS , param , param.dst , param.src , registry.key , registry.value , risk , risk.info , risk.suspicious , risk.warning , threat.category , threat.desc , threat.source , user.agent , username

    The following default single-valued meta keys are also required on the ESA Correlation service in NetWitness Platform 11.4 or later:

    accesses , context.target , file.attributes , logon.type.desc , packets

    If you used any meta keys in the ESA rule notification templates from the Required String Array or String Meta Keys list, update the templates with the meta key changes. See "Configure Global Notification Templates" in the System Configuration Guide. Go to the Master Table of Contents to find all RSA NetWitness Platform 11.x documents.

    Note: Advanced EPL rules may get disabled and are not automatically updated so they must be fixed manually.

    For additional troubleshooting information, see “Troubleshoot ESA” in the Alerting with ESA Correlation Rules User Guide for RSA NetWitness Platform. Go to the Master Table of Contents to find all RSA NetWitness Platform 11.x documents.

    Example ESA Correlation Server Warning Message for Missing Meta Keys

    If you see a warning message in the ESA Correlation server error logs that means there is a difference between the default-multi-valued parameter and multi-valued parameter meta key values, the new Endpoint, UEBA, and Live content rules will not work. Completing the Task 12. (Conditional) Update the Multi-Valued and Single-Valued Parameter Meta Keys for the latest Endpoint, UEBA, and RSA Live Content Rules procedure should fix the issue.

    Multi-Valued Warning Message Example

    2019-08-23 08:55:07,602 [ deployment-0] WARN Stream|[alert, alert_id, browserprint, cert_thumbprint, checksum, checksum_all, checksum_dst, checksum_src, client_all, content, context, context_all, context_dst, context_src, dir_path, dir_path_dst, dir_path_src, directory, directory_all, directory_dst, directory_src, email_dst, email_src, feed_category, feed_desc, feed_name, file_cat, file_cat_dst, file_cat_src, filename_dst, filename_src, filter, function, host_all, host_dst, host_orig, host_src, host_state, ip_orig, ipv6_orig, OS, param, param_dst, param_src, registry_key, registry_value, risk, risk_info, risk_suspicious, risk_warning, threat_category, threat_desc, threat_source, user_agent] are still MISSING from multi-valued

    Single Value Warning Message Example

    2019-08-23 08:55:07,602 [ deployment-0] WARN Stream|[accesses, context_target, file_attributes, logon_type_desc, packets] are still MISSING from single-valued

    Investigate

    Task 14. (Conditional - For Custom Roles Only) Adjust investigate-server Permissions for Custom User Roles

    After upgrading to Version 11.4 or later, the built-in user roles for analysts using Investigate have the following permissions enabled:

    • investigate-server.columngroup.read
    • investigate-server.metagroup.read
    • investigate-server.profile.read

    After you upgrade to 11.4 or later, NetWitness Platform does not add these permissions to custom analyst roles so you must enable them for your custom roles as described in this procedure (see the System Security and User Management Guide for comprehensive information about user roles).

    Users who are assigned a custom user role that does not have these permissions will see issues in the Navigate view and Legacy Events view. If any of the three permissions are disabled, the Load Values button is not displayed in the Navigate view. When column groups permission is disabled, there is an additional issue in the Legacy Events view: Only the Detail view is visible and you cannot select different views and column groups.

    To enable the permissions for a user role:

    1. Go to Admin > Security and click the Roles tab.
    2. Select the custom user role that needs to be edited and click (edit icon).
    3. In the Edit Role dialog, ensure that these three permissions are enabled:
      investigate-server.columngroup.read
      investigate-server.metagroup.read
      investigate-server.profile.read
    4. Click Save to save your changes. When analysts with the custom user role log in the NetWitness Platform, the changes will be in effect.

    Respond

    The Primary ESA server must be upgraded to 11.4.1 before you can complete these tasks.

    Note: After upgrading the primary NW Server (including the Respond Server service), the Respond Server service will not be re-enabled until after the Primary ESA host is also upgraded to 11.4.1. The Respond post-upgrade tasks only apply after the Respond Server service is upgraded and is in the enabled state.

    Task 15. (Conditional) Restore any Respond Service Custom Keys in the Aggregation Rule Schema

    Note: If you did not manually customize the incident aggregation rule schema, you can skip this task.

    If you added custom keys in the var/lib/netwitness/respond-server/data/aggregation_rule_schema.json file for use in the groupBy clause for 11.x, modify the /var/lib/netwitness/respond-server/data/aggregation_rule_schema.json file and add the custom keys from the automatic backup file.

    The backup file is located in /var/lib/netwitness/respond-server/data and it is in the following format:
    aggregation_rule_schema.json.bak-<time of the backup>

    Task 16. (Conditional) Restore any Customized Respond Service Normalization Scripts

    Note: If you did not manually customize any alert normalization scripts, you can skip this task.

    To prevent overwriting future customizations, custom normalization script files are available in NetWitness Platform 11.4 and later. Add any custom logic to the custom_normalize_<alert type>.js files.

    1. Locate any custom logic from the backup Respond normalization scripts located in the /var/lib/netwitness/respond-server/scripts.bak-<timestamp> directory, where <timestamp> is the time that the backup completed:
      data_privacy_map.js
      normalize_alerts.js
      normalize_core_alerts.js
      normalize_ecat_alerts.js
      normalize_ma_alerts.js
      normalize_ueba_alerts.js
      (11.3 and later versions)
      normalize_wtd_alerts.js
      utils.js

    2. Edit the new 11.4 script files in the /var/lib/netwitness/respond-server/scripts directory to include any logic from the back up files. If you have any customizations in the normalization files, add them to the normalization files with the “custom” prefix.
      data_privacy_map.js
      custom_normalize_alerts.js
      custom_normalize_core_alerts.js
      custom_normalize_ecat_alerts.js
      custom_normalize_ma_alerts.js
      custom_normalize_ueba_alerts.js
      custom_normalize_wtd_alerts.js
      utils.js

      For Example, the custom_normalize_core_alerts.js is the normalization script for ESA to add up any custom logic. This java script file has a function ‘normalizeAlert’ with parameters headers, rawAlert, and normalizedAlert. The variable ‘normalized’ is a immutable copy object which has an embedded object of list of normalized events. So if you have any custom meta keys configured for the events then you have to iterate through the ‘normalized.events’ to populate the appropriate meta keys with values from the ‘rawAlert.events’ object. Below is the sample code.


    Task 17. (Conditional) Add Respond Notification Settings Permissions

    Note: If you already configured these permissions in 11.2 or later, you can skip this task.

    Respond Notification Setting permissions enable Respond Administrators, Data Privacy Officers, and SOC Managers to access Respond Notification Settings (Configure > Respond Notifications), which enable them to send email notifications when incidents are created or updated.

    To access these settings, you must add additional permissions to your existing built-in NetWitness Platform user roles. You must also add permissions to your custom roles.

    See the “Respond Notification Settings Permissions” topic in the NetWitness Respond Configuration Guide.

    For detailed information about user permissions, see the System Security and User Management Guide.

    Go to the Master Table of Contents to find all RSA NetWitness Platform 11.x documents.

    Decoder and Log Decoder

    Task 18. Reconfigure Recurring Feed Configured from Legacy Endpoint Because Java Version Changed

    You must reconfigure the Legacy Endpoint recurring feed due to the change in Java version. Complete the following step to fix this problem.

    • Import the NetWitness Endpoint CA certificate into the NetWitness Platform Trusted store as described in "Export the NetWitness Endpoint SSL Certificate" under the "Configure Contextual Data from Endpoint via Recurring Feed" topic in the RSA NetWitness Endpoint Integration Guide to import the certificate. Go to the Master Table of Contents to find all RSA NetWitness Platform 11.x documents.

    Previous Topic:Upgrade Instructions
    You are here
    Table of Contents > Post Upgrade Tasks

    Attachments

      Outcomes