Configuring NetBackup for a highly-available Hadoop cluster
To protect a highly-available Hadoop cluster, when you configure NetBackup for Hadoop cluster:
Specify one of the NameNodes (primary) as the client in the BigData policy.
Specify the same NameNode (primary and fail-over) as application server when you execute the tpconfig command.
Create a
hadoop.conffile, update it with the details of the NameNodes (primary and fail-over), and copy it to all the backup hosts. Thehadoop.conffile is in JSON format.Hostname and port of the NameNode must be same as you have specified with the http address parameter in the
core-site.xmlof the Hadoop cluster.User name of the primary and fail-over NameNode must be same.
Do not provide a blank value for any of the parameters, or the backup job fails.
To update the hadoop.conf file for highly-available Hadoop cluster
- Update the
hadoop.conffile with the following parameters:{ "application_servers": { "hostname_of_primary_namenode1": { "failover_namenodes": [ { "hostname": "hostname_of_failover_namenode1", "port": port_of_failover_namenode1 } ], "port":port_of_primary_namenode1 } } } - If you have multiple Hadoop clusters, use the same
hadoop.conffile to update the details. For example,{ "application_servers": { "hostname_of_primary_namenode1": { "failover_namenodes": [ { "hostname": "hostname_of_failover_namenode1", "port": port_of_failover_namenode1 } ], "port"::port_of_primary_namenode1 }, "hostname_of_primary_namenode2": { "failover_namenodes": [ { "hostname": "hostname_of_failover_namenode2", "port": port_of_failover_namenode2 } ], "port":port_of_primary_namenode2 } } } - Copy this file to the following location on all the backup hosts:
/usr/openv/netbackup/