Virtual IP (VIP) is a technique used to load balance data access into HDFS by using a floating IP Address among the cluster nodes. This technique is mostly used by the MapR Hadoop Distribution along with the MapR-NFS Service. VIP can provide High Availability and Load Balancing by dynamically allocating the Floating IP among the nodes.
Optimal VIP Configuration
We recommend using one VIP for every three Nodes, because the replication factor for HDFS is 3 by default. This also helps in optimizing the performance of the cluster.
In the case of High Data Load ( >20K EPS ), a single NFS might overload while replicating the file into the cluster. If the NFS Server crashes before the data is replicated, you could have data loss.
Multiple NFS Servers also allow more distributed data locality which helps in High Availability and Fault Tolerance.
Calculate how many VIPs you can afford.
- We suggest One VIP per 3 Nodes.
- In case the number of nodes that you have is not a multiple of three, you can allocate multiple VIPs to more than three nodes. For example, two VIPs among five Nodes.
The steps to add the VIP are the same as adding any other VIP, but instead of choosing “all nodes” for VIP, you choose a subset of nodes to participate in the VIP.
- A node can participate in Multiple VIPs.
- For more information, see http://doc.mapr.com/display/MapR/Setting+Up+VIPs+for+NFS
Optimal Configuration with the Warehouse Connector
The best configuration is to have one VIP per Warehouse Connector. In cases where Warehouse Connector numbers are higher than VIPs, configure multiple Warehouse Connectors to write to a VIP in a way so that traffic on VIPs can be normalized.
Add a Virtual IP Address that has Multiple Nodes
- Log on to the MapR Control System.
- In the Navigation panel, select NFS-HA > VIP Assignments.
On the NFS Setup tab, click the Add VIP button.
In the Add Virtual IP dialog, do the following:
Specify the Starting and Ending VIP as the same IP address.
- Select Select the Desired Network Interfaces to choose the available Network Interfaces that need to be used for the VIP assignment. Select the NIC Cards that you want to participate in the VIP. A node can have multiple NICs, so depending on the Network Configuration you can select them.
- Click OK to add the VIP.
Example VIP Configurations
The following table shows example configurations of virtual IP addresses (VIPs) with different numbers of nodes in the cluster.
|Number of Nodes in Cluster||Number of VIPs|
|3 Nodes||1 VIP|
|5 Nodes||2 VIPs (3 Nodes each, 1 Common Node)|
|7 Nodes||2 VIPs (3 Nodes each, 1 Free Node)|
|8 Nodes||3 VIPs (3 Nodes each, 1 Common Node among 2 VIPs)|
|11 Nodes||4 VIPs (3 Nodes each, 1 Common Node among 2 VIPs)|
|11 Nodes||3 VIPs (3 Nodes each, 2 Free Nodes)|