MapR: Configure MapR

Document created by RSA Information Design and Development on Sep 14, 2017Last modified by RSA Information Design and Development on Oct 15, 2017
Version 5Show Document
  • View in full screen mode
  

You can configure MapR using the following procedure:

Generate and Update the Default UUID in Appliances

You need to manually generate and update the default Universally Unique Identifier (UUID) on the Appliances in the cluster. The UUID must be unique to the Appliance in the cluster.

To generate and update the default UUID in the Appliance:

  1. Log on to the Appliance as root user.
  2. Generate the UUID and copy it in the correct files, using the following commands:
    • /opt/mapr/server/mruuidgen > /opt/mapr/hostid
    • cp /opt/mapr/hostid /opt/mapr/server/hostid.xxxxx
    Where, xxxxx refers to the 5-digit number randomly assigned to the existing file.

    Note: Review /opt/mapr/server for the full name of this file.

  3. Restart the appliance, using the following command:
    reboot

Update the Configuration Template File

You must update the configuration template file in the RSA NetWitness Warehouse Appliance. The configuration template file in the RSA NetWitness Warehouse appliance must include the following parameters:

  • nodes
  • Internalnetworks
  • clustername
  • disks

By default, a configuration template is provided with the RSA NetWitness Warehouse appliance and is located on the RSA NetWitness Warehouse appliance at /opt/rsa/saw/install.

Prerequisites

Make sure that you validated the volume in the server to identify available drive space for Warehouse to store data. The total drive space of the additional volume is considered as a single drive by the HDFS. In Warehouse, the AVRO files are stored in the drive space.

Note: The server contains additional volumes of identical size other than the operation system volume.

To check free space, enter the command fdisk -l | grep /dev/s |sort in the Warehouse node. You will get a list of disks that are not partitioned for usage. You need to list the identified disks in the configuration template file so that Warehouse utilizes this space for the Hadoop Cluster.

To update the configuration template file in the RSA NetWitness Warehouse Appliance:

  1. Log on to the appliance as the root user.
  2. Navigate to /opt/rsa/saw/install, enter the following command:

    cd /opt/rsa/saw/install

  3. Create a copy of the configuration template, enter the following command:

    cp conf.template conf.template-<name>

    where <name> is the custom name of the configuration template file.

  4. Edit the configuration template file, enter the following command:

    vi conf.template-<name>

                               
    ParameterDescription
    NodesList the IP addresses of the appliances in the cluster separated by spaces. All the appliances in the cluster must be listed in the same order in every configuration file for each RSA NetWitness Warehouse appliance.
    Internalnetworks
    List the network addresses in CIDR format separated by spaces. This Warehouse appliance cluster communication is limited to the provided network addresses.

    Note: RSA recommends that you do not leave this parameter blank.

    ClusternameName of the cluster. The cluster name is used to identify the Network File System (NFS) share.
    DisksDisplays the list of disks recognized by the operation system, and these disks will be formatted in HDFS for the Warehouse when this configuration script is executed.

    The following figure displays a sample configuration template file:

    sample configuration template file

  5. Execute the configuration template file, using the following command:

    ./configure.py conf.template-<name>

  6. Restart the appliance, using the following command:

    reboot

Upgrade the Warehouse Cluster

You must upgrade the warehouse cluster after updating the configuration template file and reboot the RSA NetWitness Warehouse appliance.

To Upgrade the Warehouse Cluster

You must manually open Hiveserver port 10000, which is not opened by default:

  1. Get the line number where the REJECT statement appears in the Iptable.
  2. Make sure that the Iptables service is running, using the following command:

    NUM=$(iptables -L INPUT -n --line-numbers |grep 'reject-with' |awk ' {print $1}')

    Note: The ACCEPT statements that follow the REJECT statement in the Iptables will not take effect. You can incorporate the line number of the REJECT statement in the command to ensure that the ACCEPT statements proceed the REJECT statement.

  3. Add the firewall exception for port 10000 to the Iptables. Enter the following command:

    iptables -I INPUT $NUM -m state --state NEW -p tcp --dport 10000 -j ACCEPT

  4. Save the Iptables. Enter the following command:

    /etc/init.d/iptables save

  5. Restart the Iptables. Enter the following command:

    /etc/init.d/iptables restart

  6. Verify if the firewall exceptions for the ports are added. Enter the following command:

    Service iptables status | grep 10000

    The following output should be displayed:

    ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:10000

Install the Warehouse License File

You need to manually install the Warehouse license file on the Warehouse appliance. If you have a cluster of Warehouse appliances, you need to install the license file on the first Warehouse appliance in the cluster.

Prerequisites

Make sure that you have:

  • Obtained the Warehouse license file.
  • Copied the license file to /root/ on the first Warehouse appliance in the cluster using a USB drive or through SCP.

To install the Warehouse license file:

  1. Log on to the appliance as a root user.
  2. Install the license file, using the following command:

    maprcli license add –is_file true –license <license_filename>

    where <license_filename> is filename of the RSA NetWitness Warehouse license file.
    The license file is installed without any output messages. If you included a network range in the internalnetworks parameter in the configuration template file, a warning message appears suggesting that the Warehouse is configured only to communicate with the network entered in the configuration template file. You can ignore this warning as this does not have any functional issue.

  3. Confirm the license file installation, using the following command:

    maprcli license list

    The output messages appears on the console screen. The last two lines of the output message should be similar to the following sample:

    hash: "b8xO1flW8EMNSqq7zztn8D2BXnQ="
    3 May 14, 2013

  4. Retrieve a list of directories, run the following command:

    hadoop fs -ls /

Generate the Virtual IP Address for Primary Appliance

Generate a virtual IP address for the primary RSA NetWitness Warehouse (Warehouse) appliance.

Prerequisites

Make sure you note down the MAC addresses of all the Warehouse appliances in the cluster. Use the following command on the appliance to view the MAC address of appliance:

ifconfig <interface> | grep HWaddr

where <interface> is the network interface.

To generate a virtual IP address for the primary Warehouse appliance:

  1. Log on to the primary appliance as root user.
  2. Create the virtual IP address. Enter the following command:
    maprcli virtualip add -virtualip <VIP_address> -netmask <netmask> -macs <mac_node1> <mac_node2> <mac_node3> ……< mac_node n>
    where:
    • <VIP_address> is the virtual IP address for the primary Warehouse appliance.
    • <netmask> is the netmask address of the primary Warehouse appliance.
    • <mac_node1> is the MAC address of the first node in the Warehouse cluster.
    • <mac_node2> is the MAC address of the second node in the Warehouse cluster.
    For example, if the MAC address for node 1 is 01:Z1:1X:00:20:Y1 and node 2 is 32:Y2:4Z:40:10:X3, and the IP address is 192.168.100.10, then enter the command as following:
    maprcli virtualip add -virtualip 192.168.100.10 -netmask <netmask> -macs 01:Z1:1X:00:20:Y1 32:Y2:4Z:40:10:X3 
  3. Verify the virtual IP address, using the following command:
    maprcli virtualip list
  4. To add or remove virtual IP addresses, you can use the command line or the MapR Control System. For more information, see "Edit and Remove Virtual IP Addresses (Command Line)" and "Add and Remove a Virtual IP Address (MapR UI)" sections in Manage MapR Cluster.

Configure other NetWitness Suite Services

Configure other NetWitness Suite services for the RSA NetWitness Warehouse (MapR). 

  1. If you are not using Vulnerability Response Management (VRM), disable the Hbase services to return the configured memory so that it is available for use elsewhere in the cluster. To stop the Hbase services, you can use the command line or the MapR Control System. For more information, see Stop the Hbase Services Using the Command Line and Stop the Hbase Services Using the MapR Control System.
  2. Add Warehouse data sources to the Reporting Engine. For the detailed procedure, see the "Add Warehouse as Data Source to Reporting Engine" topic in the Reporting Engine Configuration Guide.

Stop the Hbase Services Using the Command Line

This section provides the steps to stop the Hbase services using the command line. If you are not using Vulnerability Response Management (VRM), stop the Hbase services to return the configured memory so that it is available for use elsewhere in the cluster. 

To stop the Hbase services using the command line:

  1. Stop the Hbase RegionServer service on all of the appliances, using the following command:
     maprcli node services -hbregionserver stop -filter "[hn==*]"
  2. Stop the Hbase RegionServer service on a specific node, using the following command:
    maprcli node services -hbregionserver stop -filter "[hn==<Hostname>]"
    Where <Hostname> is the specific node hostname.
  3. Stop the Hbase Master service on all of the appliances, using the following command:
    maprcli node services -hbmaster stop -filter "[hn==*]"
  4. Stop the Hbase Master service on a specific node, using the following command:
    maprcli node services -hbmaster stop -filter "[hn==<Hostname>]"
    Where <Hostname> is the specific node hostname.

Hbase Services Stop and Start Commands Summary

The following tables summarize the commands used to stop and start the Hbase services for the HBase RegionServer and HBase Master services.

                         
HBase RegionServerCommand to run using the Command Line
Stop on All the Appliances maprcli node services -hbregionserver stop -filter "[hn==*]"
Start on All the Appliancesmaprcli node services -hbregionserver start -filter "[hn==*]"
Stop on Specific nodemaprcli node services -hbregionserver stop -filter "[hn==<Hostname>]"
Start on Specific nodemaprcli node services -hbregionserver start -filter "[hn==<Hostname>]"

 

                         
HBase MasterCommand to run using the Command Line
Stop on All the Appliancesmaprcli node services -hbmaster stop -filter "[hn==*]"
Start on All the Appliancesmaprcli node services -hbmaster start -filter "[hn==*]"
Stop on Specific nodemaprcli node services -hbmaster stop -filter "[hn==<Hostname>]"
Start on Specific nodemaprcli node services -hbmaster start -filter "[hn==<Hostname>]"

Where <Hostname> is the specific node hostname.

Stop the Hbase Services Using the MapR Control System

This section provides the steps to stop the Hbase services using the MapR Control System. If you are not using Vulnerability Response Management (VRM), stop the Hbase services to return the configured memory so that it is available for use elsewhere in the cluster. 

  1. Log on to the MapR Control System user interface. For more information see "Access MapR Control System UI for Cluster Administration" section in Manage MapR Cluster.
  2. Stop the HBase Master services, in the Services section of the dashboard, click the number in the Actv column for the HBase Master service. This is the number of active services for the HBase Master service.
    The number of active services for the Hbase Master
  3. On the Cluster Nodes tab, click Manage Services.
    Manage Services

    The Manage Node Services dialog is displayed.
    Manage Node services
  4. In the HBase Master field, select Stop.
    Stop Hbase master
  5. Click Change <number_of_nodes> Nodes.
    Where <number_of_nodes> is the number of active nodes selected.
    For example, click Change 3 Nodes.
    Change number of nodes
    The Hbase Master service on the selected nodes must be in a stopped state.
  6. Stop the Hbase RegionServer services, repeat steps 2 to 5 for the Hbase RegionServer services.
You are here
Table of Contents > Configure MapR

Attachments

    Outcomes