RHEL6.5上使用Ambari快速搭建Hadoop集群
ip hostname 10.7.1.1 master.coke 10.7.1.2 dn1.coke 10.7.1.3 dn2.coke 10.7.1.4 dn3.coke 10.7.1.5 dn4.coke
1,ssh
master.coke 生成pub key 作为中控机，添加到每台机器包括master.coke 的 /root/.ssh/authorized_keys
2,hostname
对照表格更改hadoop集群的每台机器的hostname 和/etc/sysconfig/network HOSTNAME=XXXX
3,ntp
启动ntp 服务保证各台机器时钟同步
4,repo
https://cwiki.apache.org/confluence/display/AMBARI/Install Ambari 1.7.0 from Public Repositories
cd /etc/yum.repos.d/
wget http://public-repo-1.hortonworks.com/ambari/CentOS6/1.x/updates/1.7.0/ambari.repo
yum install ambari-server
yum install MySQL-server mysql
提前建立库和授权
mysql>grant all on ambari.* to ‘ambari’@’localhost‘ identified by “175an.com”;
mysql> flush privileges;
[@master.coke yum.repos.d]# yum install ambari-server Loaded plugins: product-id, security, subscription-manager This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. 6ASU5-updates | 1.1 kB 00:00 HighAvailability | 3.9 kB 00:00 LoadBalancer | 3.9 kB 00:00 ResilientStorage | 3.9 kB 00:00 ScalableFileSystem | 3.9 kB 00:00 Server | 3.9 kB 00:00 Updates-ambari-1.7.0 | 2.9 kB 00:00 ambari-1.x | 1.3 kB 00:00 epel | 1.1 kB 00:00 Setting up Install Process Resolving Dependencies –> Running transaction check —> Package ambari-server.noarch 0:1.7.0-169 will be installed –> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server-1.7.0-169.noarch –> Running transaction check —> Package postgresql-server.x86_64 0:8.4.20-1.el6_5 will be installed –> Processing Dependency: postgresql(x86-64) = 8.4.20-1.el6_5 for package: postgresql-server-8.4.20-1.el6_5.x86_64 –> Processing Dependency: postgresql-libs(x86-64) = 8.4.20-1.el6_5 for package: postgresql-server-8.4.20-1.el6_5.x86_64 –> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server-8.4.20-1.el6_5.x86_64 –> Running transaction check —> Package postgresql.x86_64 0:8.4.20-1.el6_5 will be installed —> Package postgresql-libs.x86_64 0:8.4.20-1.el6_5 will be installed –> Finished Dependency ResolutionDependencies Resolved
================================================================================================================================================================================= Package Arch Version Repository Size ================================================================================================================================================================================= Installing: ambari-server noarch 1.7.0-169 Updates-ambari-1.7.0 96 M Installing for dependencies: postgresql x86_64 8.4.20-1.el6_5 6ASU5-updates 2.6 M postgresql-libs x86_64 8.4.20-1.el6_5 6ASU5-updates 201 k postgresql-server x86_64 8.4.20-1.el6_5 6ASU5-updates 3.4 M
Transaction Summary ================================================================================================================================================================================= Install 4 Package(s)
Total download size: 102 M Installed size: 152 M Is this ok [y/N]: y Downloading Packages: (1/4): ambari-server-1.7.0-169.noarch.rpm | 96 MB 00:14 (2/4): postgresql-8.4.20-1.el6_5.x86_64.rpm | 2.6 MB 00:00 (3/4): postgresql-libs-8.4.20-1.el6_5.x86_64.rpm | 201 kB 00:00 (4/4): postgresql-server-8.4.20-1.el6_5.x86_64.rpm | 3.4 MB 00:00 ——————————————————————————————————————————————————————————— Total 6.7 MB/s | 102 MB 00:15 warning: rpmts_HdrFromFdno: Header V4 RSA/SHA1 Signature, key ID 07513cad: NOKEY Retrieving key from http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins Importing GPG key 0x07513CAD: Userid: “Jenkins (HDP Builds) <jenkin@hortonworks.com>” From : http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins Is this ok [y/N]: y Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : postgresql-libs-8.4.20-1.el6_5.x86_64 1/4 Installing : postgresql-8.4.20-1.el6_5.x86_64 2/4 Installing : postgresql-server-8.4.20-1.el6_5.x86_64 3/4 Installing : ambari-server-1.7.0-169.noarch 4/4 Verifying : postgresql-server-8.4.20-1.el6_5.x86_64 1/4 Verifying : postgresql-libs-8.4.20-1.el6_5.x86_64 2/4 Verifying : postgresql-8.4.20-1.el6_5.x86_64 3/4 Verifying : ambari-server-1.7.0-169.noarch 4/4
Installed: ambari-server.noarch 0:1.7.0-169
Dependency Installed: postgresql.x86_64 0:8.4.20-1.el6_5 postgresql-libs.x86_64 0:8.4.20-1.el6_5 postgresql-server.x86_64 0:8.4.20-1.el6_5
Complete!
ambari-server setup
Using Python /usr/bin/python2.6 Setup ambari-server Checking SELinux… SELinux status is ‘disabled’ Customize user account for ambari-server daemon [y/n] (n)? y Enter user account for ambari-server daemon (root): Adjusting ambari-server permissions and ownership… Checking firewall… Checking JDK… [1] – Oracle JDK 1.7 Java Cryptography Extension (JCE) Policy Files 7 [2] – Oracle JDK 1.6 Java Cryptography Extension (JCE) Policy Files 6 [3] – Custom JDK ============================================================================== Enter choice (1): To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.htmland not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually. Do you accept the Oracle Binary Code License Agreement [y/n] (y)? y Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-7u67-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz JDK distribution size is 142376665 bytes jdk-7u67-linux-x64.tar.gz… 100% (135.8 MB of 135.8 MB) Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz Installing JDK to /usr/jdk64 Successfully installed JDK to /usr/jdk64/jdk1.7.0_67 Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/UnlimitedJCEPolicyJDK7.zip to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip Completing setup… Configuring database… Enter advanced database configuration [y/n] (n)? y ============================================================================== Choose one of the following options: [1] – PostgreSQL (Embedded) [2] – Oracle [3] – MySQL [4] – PostgreSQL ============================================================================== Enter choice (1): 3 Hostname (localhost): Port (3306): Database Name (ambari): Username (ambari): Enter Database Password (bigdata): WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java. Press <enter> to continue. ERROR: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java. ERROR: Exiting with exit code -1. REASON: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
yum install -y mysql-connector-java # resolve the error
重新来过
[@master.coke 6Server]# ambari-server setup
Using python /usr/bin/python2.6
Setup ambari-server
Checking SELinux…
SELinux status is ‘disabled’
Ambari-server daemon is configured to run under user ‘root’. Change this setting [y/n] (n)?
Adjusting ambari-server permissions and ownership…
Checking firewall…
Checking JDK…
Do you want to change Oracle JDK [y/n] (n)?
Completing setup…
Configuring database…
Enter advanced database configuration [y/n] (n)? y
==============================================================================
Choose one of the following options:
[1] – PostgreSQL (Embedded)
[2] – Oracle
[3] – MySQL
[4] – PostgreSQL
==============================================================================
Enter choice (1): 3
Hostname (localhost):
Port (3306):
Database Name (ambari):
Username (ambari): amb
Enter Database Password (bigdata):
Re-enter password:
Copying JDBC drivers to server resources…
Configuring remote database connection properties…
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)?
Copying JDBC drivers to server resources…
Extracting system views…
…
Adjusting ambari-server permissions and ownership…
Ambari Server ‘setup’ completed successfully.
导入数据库
mysql ambari -u175an -p < /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
grant all on ambari.* to “amb@localhost” identified by “amb123″;
Admin Name : admin
Cluster Name : coke
Total Hosts : 6 (6 new)
Repositories:
RedHat5 (HDP-2.2): http://public-repo-1.hortonworks.com/HDP/centos5/2.x/GA/2.2.0.0
redhat5 (HDP-UTILS-1.1.0.20): http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos5
redhat6 (HDP-2.2): http://public-repo-1.hortonworks.com/HDP/centos6/2.x/GA/2.2.0.0
redhat6 (HDP-UTILS-1.1.0.20): http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6
SUSE11 (HDP-2.2): http://public-repo-1.hortonworks.com/HDP/suse11sp3/2.x/GA/2.2.0.0
suse11 (HDP-UTILS-1.1.0.20): http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/suse11sp3
Ubuntu12 (HDP-2.2): http://public-repo-1.hortonworks.com/HDP/ubuntu12/2.x/GA/2.2.0.0
ubuntu12 (HDP-UTILS-1.1.0.20): http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu12
Services: HDFS DataNode : 6 hosts NameNode : master.coke SNameNode : dn1.coke
YARN MapReduce2 App Timeline Server : master.coke NodeManager : 6 hosts ResourceManager : master.coke
Tez Clients : 6 hosts
Nagios Server : master.coke Administrator : nagiosadmin / (qilinli@sohu-inc.com)
Ganglia Server : master.coke
Hive Metastore : master.coke HiveServer2 : master.coke WebHCat Server : master.coke Database : MySQL (Existing MySQL Database)
HBase Master : dn1.coke RegionServer : 6 hosts
Pig Clients : 6 hosts
Sqoop Clients : 6 hosts
Oozie Server : dn1.coke Database : MySQL (Existing MySQL Database)
ZooKeeper Server : 3 hosts
Falcon Server : dn3.coke
Storm DRPC Server : dn1.coke Nimbus : dn1.coke UI Server : dn1.coke Supervisor : 6 hosts
Flume Flume : 2 hosts
Slider Clients : 6 hosts
Knox Gateway : dn1.coke
Kafka Broker : 3 hosts
