...
BTW, yes, 8GB is woefully small for a root file system on a host running HDP, but it'll do for the exercise of installing HDP. Visit these instructions for how to change it after creation. On the Step 5 screen, just select Next: Configure Security Group. On Step 6, you want select the "Create a new security group" radio button with the Assign a security group label and add something like "ec2testSecurityGroup" to Security group name before ignoring the IP address warning and clicking on Review and Launch.
...
Do all of these file system activities to all three hosts. Once done, issue a shutdown -r now on all nodes and then SSH to them when they have restarted and make sure the df -h output shows all three of the /grid file systems being mounted.
We're just about ready to start installing HDP itself, but we need to think about the hostnames and their IP resolutions. We get new IP addresses after we shut these VMs down. In fact, we even get a new hostname as the private IP address is baked into it. Fear not, we can use the instructions at Changing the Hostname of Your Linux Instance to call our boxes ec2node1.hdp22, ec2node2.hdp22 and ec2node3.hdp22. The following, taken from the "... without a public DNS name" of that last link, are the commands I ran for the first node; ec2node1.hpd22.
| Code Block | ||||
|---|---|---|---|---|
| ||||
HW10653:Amazon lmartin$ ssh -i ec2test.pem root@52.11.30.44
Warning: Permanently added '52.11.30.44' (RSA) to the list of known hosts.
Last login: Tue Jun 30 04:46:55 2015 from 12.48.56.194
[root@ip-172-31-7-169 ~]# cp /etc/sysconfig/network ~/network.bak
[root@ip-172-31-7-169 ~]# vi /etc/sysconfig/network
[root@ip-172-31-7-169 ~]# diff ~/network.bak /etc/sysconfig/network
2c2
< HOSTNAME=localhost.localdomain
---
> HOSTNAME=ec2node1.hdp22
[root@ip-172-31-7-169 ~]# cp /etc/hosts ~/hosts.bak
[root@ip-172-31-7-169 ~]# vi /etc/hosts
[root@ip-172-31-7-169 ~]# diff ~/hosts.bak /etc/hosts
1c1
< 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
---
> 127.0.0.1 ec2node1.hdp22 ec2node1 localhost localhost.localdomain localhost4 localhost4.localdomain4
[root@ip-172-31-7-169 ~]# reboot |
Verify this FQDN is returned with, and without, the -f switch.
| Code Block | ||||
|---|---|---|---|---|
| ||||
HW10653:Amazon lmartin$ ssh -i ec2test.pem root@52.11.30.44
Last login: Wed Jul 1 02:19:08 2015 from 12.48.56.194
[root@ec2node1 ~]# hostname
ec2node1.hdp22
[root@ec2node1 ~]# hostname -f
ec2node1.hdp22 |
Do this for the other two EC2 instances. Now all boxes know who they are, but they don't know where each other are – well, not with our testing .hdp22 domain! For this, we'll just need to create /etc/hosts entries for the "other two" boxes. As shown below, we will use the public IP addresses as we'll probably want to access something like HiveServer2 or see the Resource Manager UI and this will help. For our testing case, we'll just assume that this provides not overhead compared to using the internal IPs that are available.
If they are all there, now it is time to visit http://docs.hortonworks.com/HDPDocuments/Ambari-2.0.1.0/bk_Installing_HDP_AMB/content/index.html (or the latest version) Ambari install page and start following instructions. I'm not going to use the password-less SSH connectivity, and since I'll need to do something a bit different regarding hostnames, I'll go slightly out of order. The following are the steps I executed on the box I want to run Ambari on; ec2node1.hdp22.
...