Description:

Meant to standup a Hadoop 2.7.3 HDFS HA on multiple machines inside Docker containers. On the "active" namenode, the 1st time it starts up, it will format the namenode as well as format the ZK for failover. The "standby" namenode will bootStrapStandby.

Runtime options:

• active: Start the 1st namenode with a zkfc service.
• standby: Start the 2nd namenode and bootstrap from an already formatted/running namenode. This will also start a zkfc service.
• zkfc: Start the zkfc in as a separate service
• journalnode: Start a journalnode
• datanode: Starts a datanode
• bash: allows you to jump in and check things out

Environment Variables

CLUSTER_NAME: the HDFS URI default filesystem name

NNODE1_IP: Namenode #1 IP/hostname

NNODE2_IP: Namenode #2 IP/hostname

JNODES: semicolon separated list of journal node IPS, e.g jn01:8485;jn02:8485;jn03:8485

ZK_NODES: comma separated list of zookeeper IPS, e.g. zk01:2181,zk02:2181,zk03:2181

Volumes:

/data/hdfs/nn: inside the container, where the fsimage/namenode metadata exists

/data/hdfs/journal: inside the container, where the journal node keeps the edits

/data/jdfs/dn: inside the container, where the datanode keeps the blocks

Command Line examples

• docker run --name nn01 --net=host --env-file hdfs-envs --env NNODE_ID=nn1 hdfs active