背景：写这篇文章是为了后面学习监控yarn的队列，Cloudera的CDH属于即将过时的技术了，但是目前国内很多企业任然以5.x 的版本为主，如果有不会的同学可以参考搭建一下。
软件版本： center os7.4 cdh5.13.1 cm5.13.1 mysql5.7 spark2.3
物理机配置
目前市面主流的物理机配置基本都为26核384g内存机器，盘的大小一般为4T*10
1、主机操作系统选择：center OS 7.4操作系统
2、主机swap配置
Linux系统中当内存使用到一定程度后会使用swap分区，这是由/proc/sys/vm/swappiness文件中的vm.swappiness 参数进行控制的，linux默认vm.swappiness=60
但是对于hadoop集群来说，如果使用系统默认设置，会导致swap分区被频繁使用，集群会不断发出警告。所以在搭建cloudera-manager的时候，会建议调整vm.swappiness参数，我一般设置为0，vm.swappiness的值越小使用swap分区的可能性就越小。设置方法为：
代码语言：javascript复制sysctl -w vm.swappiness=0如果集群已经使用了swap分区，在设置vm.swappiness值之后建议对swap分区进行刷新,命令为：
代码语言：javascript复制swapoff -a
swapon -a也可在开机启动配置中直接增加swapoff -a
代码语言：javascript复制[hadoop@whx-9 ~]$ free -g
total used free shared buff/cache available
Mem: 125 24 8 0 92 99
Swap: 0 0 03、主机磁盘配置
对于所有已规划为datanode节点的机器的数据盘按照raid0 依次挂载在/mnt/data01-data10上，关于raid0与JBOD比较速度问题，笔者维护千节点集群暂无使用JBOD，但理论上JBOD吞吐速度好于raid方式。
主机测在做系统时有些盘是没有格式化与挂载的，可参考入戏操作进行格式化挂载。
代码语言：javascript复制for ip in `cat hostlist`; do ssh $ip “sh formatdisk.sh” ; done格式化磁盘脚本formatdisk.sh
代码语言：javascript复制#!/bin/bash
for disk_id in {b..k}
do
fdisk /dev/sd${disk_id} <<EOF
n
p
1
w
q
EOF
Done生产机器默认10块磁盘，格式化完毕，挂载写入/etc/fstab中。
代码语言：javascript复制/dev/sdb1 /mnt/sd01 xfs defaults 0 0
/dev/sdc1 /mnt/sd02 xfs defaults 0 0
/dev/sdd1 /mnt/sd03 xfs defaults 0 0
/dev/sde1 /mnt/sd04 xfs defaults 0 0
/dev/sdf1 /mnt/sd05 xfs defaults 0 0
/dev/sdg1 /mnt/sd06 xfs defaults 0 0
/dev/sdh1 /mnt/sd07 xfs defaults 0 0
/dev/sdi1 /mnt/sd08 xfs defaults 0 0
/dev/sdj1 /mnt/sd09 xfs defaults 0 0
/dev/sdk1 /mnt/sd10 xfs defaults 0 0for ip in `cat hostlist`; do scp a.txt $ip:/root/ ; done
for ip in `cat hostlist`; do ssh $ip “cat a.txt >>/etc/fstab”; done格式化磁盘vim disk.sh
代码语言：javascript复制#!bin/bash
for disk_id in {b..w}
do
mkfs.xfs /dev/sd${disk_id}1
done[hadoop@whx-8 ~]$ ll /mnt/
total 0
drwxr-xr-x 5 root root 83 May 7 2018 sd01
drwxr-xr-x 5 root root 83 May 7 2018 sd02
drwxr-xr-x 5 root root 83 May 7 2018 sd03
drwxr-xr-x 5 root root 83 May 7 2018 sd04
drwxr-xr-x 5 root root 83 May 7 2018 sd05
drwxr-xr-x 5 root root 83 May 7 2018 sd06
drwxr-xr-x 5 root root 83 May 7 2018 sd07
drwxr-xr-x 5 root root 83 May 7 2018 sd08
drwxr-xr-x 5 root root 83 May 7 2018 sd09
drwxr-xr-x 5 root root 83 May 7 2018 sd10
drwxr-xr-x 5 root root 83 May 7 2018 sd114、所有机器建议增加hadoop用户，为后续运维使用
此处需要有大量规范化项目，如果公司集群很大时建议规范建设，譬如技术部（Technical department）下边分有开发组，运维组，各组又按照项目分裂，需要考虑好。
代码语言：javascript复制[hadoop@whx-8 ~]$ cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
avahi-autoipd:x:170:170:Avahi IPv4LL Stack:/var/lib/avahi-autoipd:/sbin/nologin
systemd-bus-proxy:x:999:997:systemd Bus Proxy:/:/sbin/nologin
systemd-network:x:998:996:systemd Network Management:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
polkitd:x:997:995:User for polkitd:/:/sbin/nologin
tss:x:59:59:Account used by the trousers package to sandbox the tcsd daemon:/dev/null:/sbin/nologin
unbound:x:996:994:Unbound DNS resolver:/etc/unbound:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
chrony:x:995:993::/var/lib/chrony:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
imonitor:x:1003:1004::/home/imonitor:/bin/bash
deployer:x:1004:1005::/home/deployer:/bin/bash
maintainer:x:1005:1006::/home/maintainer:/bin/bash
ntp:x:38:38::/etc/ntp:/sbin/nologin
weihu:x:1007:1008::/home/weihu:/bin/bash
hadoop:x:1008:1009::/home/hadoop:/bin/bash
telegraf:x:1012:1012::/home/telegraf:/bin/bash5、主机名配置
建议统一配置主机名，其中主机名不建议添加大写和特殊字符，主要和Hadoop安全认证有关。
6、机架配置
机架的配置对Hadoop集群也是有一定的影响的，hadoop在分配MR到各个节点时会倾向于执行机架内的数据传输，而非跨机架数据传输，hdfs还能够更加智能的放置副本，以取得性能和弹性的平衡。
软件配置
1、修改hosts
一批新机器采购上架以后需要安装操作系统，如果用于CDH集群的建设时要注意：新机器主机名称不要出现大写字母如DSJ字母，会在cm页面出现配置告警，与安全认证有关。如果集群启用kerberos时需要注意此事项
代码语言：javascript复制[root@whx-1 ~]# cat /etc/hosts
127.0.0.1 localhost
192.168.1.1 whx-1
192.168.1.2 whx-2
192.168.1.3 whx-3
192.168.1.4 whx-4
192.168.1.5 whx-5
192.168.1.6 whx-6
192.168.1.7 whx-72、配置免密码登陆
由于集群规模超大所有机器建议免密登陆，待集群配置安装完毕后建议回收取消root免密，免密配置如下：
代码语言：javascript复制[root@whx-1 桌面]# ssh-keygen -t rsa -P ""
[root@whx-1 ~]# cd .ssh
[root@whx-1 .ssh]# ls
id_rsa id_rsa.pub
[root@whx-1 .ssh]# ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.1.13、关闭防火墙selinux
集群运行时需要关闭所有节点防火墙以及selinux，生产集群不要暴漏在外网中。
代码语言：javascript复制[root@whx-1 ~]# systemctl stop firewalld.service && systemctl disable firewalld.service
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.可批量关闭，在root家目录创建hostlist文件里面为所有机器ip。
代码语言：javascript复制for ip in `cat hostlist`; do ssh $ip "systemctl stop firewalld.service && systemctl disable firewalld.service"; done关闭selinux
及时生效
代码语言：javascript复制[root@whx-1 ~]# setenforce 0
[root@whx-1 ~]# getenforce
Permissive永久生效
代码语言：javascript复制[root@whx-1 ~]# vim /etc/selinux/config
SELINUX=enforcing改成disabled批量修改
代码语言：javascript复制for ip in `cat hostlist` ; do scp /etc/selinux/config root@$ip:/etc/; done
for ip in `cat hostlist`; do ssh $ip "setenforce 0"; done4、安装jdk
查看机器是否安装jdk
这里建议安装oracle的jdk1.8
首先卸载自带open jdk
代码语言：javascript复制[root@whx-1 ~]# rpm -aq |grep jdk
java-1.8.0-openjdk-headless-1.8.0.65-3.b17.el7.x86_64
java-1.8.0-openjdk-1.8.0.65-3.b17.el7.x86_64批量卸载
代码语言：javascript复制for ip in `cat iplist`; do ssh $ip "rpm -e java-1.8.0-openjdk-1.8.0.65-3.b17.el7.x86_64 --nodeps && rpm -e java-1.8.0-openjdk-headless-1.8.0.65-3.b17.el7.x86_64 --nodeps"; done下载oracle的jdk解压到/usr/java/下
在/etc/profile追加如下，source一下
代码语言：javascript复制PATH=$PATH:/usr/java/jdk1.8.0_161
export JAVA_HOME=/usr/java/jdk1.8.0_161
export PATH=$JAVA_HOME/bin:$MAVEN_HOME/bin:$PATH此处说明下，当集群需要多个版本java时建议不要修改全局变量，可配置用户家目录的.bashrc文件，自定义java路径在/opt/soft/java/下。当用户使用
5、安装mysql数据库
建议使用mysql-mha模式，以下为单节点mysql配置方法。
代码语言：javascript复制[root@whx-1 opt]# tar xvf mysql-5.7.18-1.el7.x86_64.rpm-bundle.tar
mysql-community-server-5.7.18-1.el7.x86_64.rpm
mysql-community-embedded-devel-5.7.18-1.el7.x86_64.rpm
mysql-community-devel-5.7.18-1.el7.x86_64.rpm
mysql-community-client-5.7.18-1.el7.x86_64.rpm
mysql-community-common-5.7.18-1.el7.x86_64.rpm
mysql-community-embedded-5.7.18-1.el7.x86_64.rpm
mysql-community-embedded-compat-5.7.18-1.el7.x86_64.rpm
mysql-community-libs-5.7.18-1.el7.x86_64.rpm
mysql-community-server-minimal-5.7.18-1.el7.x86_64.rpm（这个包可以删除）
mysql-community-test-5.7.18-1.el7.x86_64.rpm
mysql-community-minimal-debuginfo-5.7.18-1.el7.x86_64.rpm
mysql-community-libs-compat-5.7.18-1.el7.x86_64.rpm
[root@whx-1 opt]# rm mysql-5.7.18-1.el7.x86_64.rpm-bundle.tar -f
[root@whx-1 opt]# mkdir mysql
[root@whx-1 opt]# mv mysql-community-* mysql
[root@whx-1 opt]# cd mysql/
[root@whx-1 mysql]# createrepo ./
Spawning worker 0 with 12 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete配置mysql的yum源，该步骤主要是方便其他机器安装mysql。
代码语言：javascript复制[root@whx-1 mysql]# cd /etc/yum.repos.d/
[root@whx-1 yum.repos.d]# vim mysql.repo
[mysql]
name=5.7
baseurl=file:///opt/mysql/
enable=1
gpgcheck=0
[root@whx-1 yum.repos.d]# yum repolist
源标识 源名称 状态
mysql 5.7 12
repolist: 12卸载mariadb的lib
代码语言：javascript复制[root@whx-1 yum.repos.d]# rpm -qa | grep mariadb
mariadb-libs-5.5.44-2.el7.centos.x86_64
[root@whx-1 yum.repos.d]# rpm -e mariadb-libs-5.5.44-2.el7.centos.x86_64 –nodeps安装mysql数据库，此处可注释掉本地center os 7的yum源
代码语言：javascript复制[root@whx-1 yum.repos.d]# yum -y install mysql
[root@whx-1 yum.repos.d]# yum -y install mysql-server启动服务
代码语言：javascript复制[root@whx-1 log]# systemctl start mysqld.service
[root@whx-1 log]# cat /var/log/mysqld.log
2021-02-18T12:03:18.062358Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2021-02-18T12:03:18.198096Z 0 [Warning] InnoDB: New log files created, LSN=45790
2021-02-18T12:03:18.236126Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2021-02-18T12:03:18.293010Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 49b23bd5-71e1-11eb-b737-000c29cadc9d.
2021-02-18T12:03:18.294346Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
2021-02-18T12:03:18.295554Z 1 [Note] A temporary password is generated for root@localhost: Rieh!a-d,9JH
2021-02-18T12:03:20.706153Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2021-02-18T12:03:20.708998Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.18) starting as process 54291 ...
2021-02-18T12:03:20.713135Z 0 [Note] InnoDB: PUNCH HOLE support available
2021-02-18T12:03:20.713181Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2021-02-18T12:03:20.713190Z 0 [Note] InnoDB: Uses event mutexes
2021-02-18T12:03:20.713198Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2021-02-18T12:03:20.713211Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3
2021-02-18T12:03:20.713219Z 0 [Note] InnoDB: Using Linux native AIO
2021-02-18T12:03:20.713635Z 0 [Note] InnoDB: Number of pools: 1
2021-02-18T12:03:20.713822Z 0 [Note] InnoDB: Using CPU crc32 instructions
2021-02-18T12:03:20.716317Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2021-02-18T12:03:20.728818Z 0 [Note] InnoDB: Completed initialization of buffer pool
2021-02-18T12:03:20.732664Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-02-18T12:03:20.749189Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2021-02-18T12:03:20.767208Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-02-18T12:03:20.767281Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-02-18T12:03:20.784538Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-02-18T12:03:20.788882Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2021-02-18T12:03:20.788900Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2021-02-18T12:03:20.791200Z 0 [Note] InnoDB: Waiting for purge to start
2021-02-18T12:03:20.841872Z 0 [Note] InnoDB: 5.7.18 started; log sequence number 2535704
2021-02-18T12:03:20.842631Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2021-02-18T12:03:20.843650Z 0 [Note] Plugin 'FEDERATED' is disabled.
2021-02-18T12:03:20.848107Z 0 [Note] InnoDB: Buffer pool(s) load completed at 210218 20:03:20
2021-02-18T12:03:20.856534Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
2021-02-18T12:03:20.856776Z 0 [Warning] CA certificate ca.pem is self signed.
2021-02-18T12:03:20.857903Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
2021-02-18T12:03:20.857949Z 0 [Note] IPv6 is available.
2021-02-18T12:03:20.857977Z 0 [Note] - '::' resolves to '::';
2021-02-18T12:03:20.857995Z 0 [Note] Server socket created on IP: '::'.
2021-02-18T12:03:20.879682Z 0 [Note] Event Scheduler: Loaded 0 events
2021-02-18T12:03:20.879969Z 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.7.18' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Server (GPL)
2021-02-18T12:03:20.879991Z 0 [Note] Executing 'SELECT * FROM INFORMATION_SCHEMA.TABLES;' to get a list of tables using the deprecated partition engine. You may use the startup option '--disable-partition-engine-check' to skip this check.
2021-02-18T12:03:20.880009Z 0 [Note] Beginning of list of non-natively partitioned tables
2021-02-18T12:03:20.900544Z 0 [Note] End of list of non-natively partitioned tables查看mysql的root用户初始密码
代码语言：javascript复制[root@whx-1 log]# grep 'temporary password' /var/log/mysqld.log
2021-02-18T12:03:18.295554Z 1 [Note] A temporary password is generated for root@localhost: Rieh!a-d,9JH
[root@bigdata01 log]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 3
Server version: 5.7.18
Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
进入后无法进行其他操作需要先修改mysql服务的root密码。
mysql> flush PRIVILEGES;
ERROR 1820 (HY000): You must reset your password using ALTER USER statement before executing this statement.
由于5.7以后的mysql密码复杂度有设置需求所以无法设置弱密码。
mysql> alter user 'root'@'localhost' identified by '123456';
ERROR 1819 (HY000): Your password does not satisfy the current policy requirements
设置复杂度较高的密码
mysql> alter user 'root'@'localhost' identified by '1qaz@WSX';
Query OK, 0 rows affected (0.00 sec)
授权远程root登陆的密码
mysql> grant all privileges on *.* to 'root'@'%' identified by '1qaz#EDC' with grant option;
Query OK, 0 rows affected, 1 warning (0.00 sec)
查看root账号的授权的状态
mysql> show grants for root@'%' ;
-------------------------------------------------------------
| Grants for root@% |
-------------------------------------------------------------
| GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' WITH GRANT OPTION |
-------------------------------------------------------------
1 row in set (0.00 sec)建立数据库用于后边服务安装，如果不想复用数据库可不建立。
代码语言：javascript复制mysql> create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
mysql> create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
mysql> create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)6、配置ntp服务
正常生产集群环境中会有时钟源，将时钟源指定一下即可。此处大致掠过
代码语言：javascript复制[root@whx-1 yum.repos.d]# yum -y install ntp
[root@whx-1 yum.repos.d]# chkconfig ntpd on
[root@whx-1 yum.repos.d]# service ntpd start7、所有节点创建用户
代码语言：javascript复制[root@whx-1 yum.repos.d]# useradd --system --home=/opt/cm-5.13.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm集群安装
1、下载CM与CDH
代码语言：javascript复制CM地址：http://archive.cloudera.com/cm5/cm/5/
CDH地址：http://archive.cloudera.com/cdh5/parcels/5.13.1/所需文件为
代码语言：javascript复制CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel
cloudera-manager-centos7-cm5.13.1_x86_64.tar.gz
CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel.sha
manifest.json 注意sha1要改成sha
2、分发软件包
解压cm
代码语言：javascript复制[root@whx-1 opt]# tar xvf cloudera-manager-centos7-cm5.13.1_x86_64.tar.gz会生成cm-5.13.1与cloudera两个文件夹
修改mysql的jdbc驱动移动到对应的目录
代码语言：javascript复制[root@whx-1 opt]# mv mysql-connector-java-5.1.46.jar mysql-connector-java.jar
[root@whx-1 opt]# cp mysql-connector-java.jar /usr/share/java/
[root@whx-1 opt]# cp mysql-connector-java.jar /opt/cm-5.13.1/share/cmf/lib/初始化数据库
代码语言：javascript复制 /opt/cm-5.13.1/share/cmf/schema/scm_prepare_database.sh mysql -uroot -p scm scm
Enter database password: 此处是123456
Enter SCM password:此处是passwd将parcel包移动到对应目录
代码语言：javascript复制[root@whx-1 opt]# mv CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel* cloudera/parcel-repo/
[root@whx-1 opt]# mv manifest.json cloudera/parcel-repo/修改agent文件
代码语言：javascript复制[root@whx-1 opt]# vim /opt/cm-5.13.1/etc/cloudera-scm-agent/config.ini
server_host=whx-1分发其他节点
代码语言：javascript复制[root@whx-1 opt]# for i in `cat /root/iplist`;do scp /opt/cm-5.13.1/etc/cloudera-scm-agent/config.ini $i:/opt/cm-5.13.1/etc/cloudera-scm-agent/;done3、启动server和agent
代码语言：javascript复制[root@whx-1 opt]# /opt/cm-5.13.1/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server: [ 确定 ]
[root@whx-1 opt]# /opt/cm-5.13.1/etc/init.d/cloudera-scm-agent start
Starting cloudera-scm-agent: [ 确定 ]4、界面配置
程序启动后等待5分钟如无报错可通过浏览器访问xx.xxx.xxx.xxx:7180
用户名admin
密码admin
spark2.2安装
1、下载软件包
SPARK2_ON_YARN-2.2.0.cloudera1.jar
下载地址：http://archive.cloudera.com/spark2/
SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354-el7.parcel.sha1
下载地址：http://archive.cloudera.com/spark2/parcels/
SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354-el7.parcel
下载地址：http://archive.cloudera.com/spark2/parcels/
manifest.json
下载地址：http://archive.cloudera.com/spark2/parcels/
注意：版本对应，以及manifest.json对应。
2、分发软件包
将SPARK2_ON_YARN-2.2.0.cloudera1.jar放到/opt/cloudera/csd/
将SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354-el7.parcel.sha1改名字
SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354-el7.parcel.sha
与SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354-el7.parcel一起放到如下目录
/opt/cloudera/parcel-repo/
备份/opt/cloudera/parcel-repo/下的manifest.json为manifest.json.CDH
将manifest.json放到/opt/cloudera/parcel-repo/下
最终文件如下
3、安装spark
重启server与agent
[root@whx-1 opt]# /opt/cm-5.13.1/etc/init.d/cloudera-scm-server restart
Starting cloudera-scm-server: [ 确定 ]
[root@whx-1 opt]# /opt/cm-5.13.1/etc/init.d/cloudera-scm-agent restart
Starting cloudera-scm-agent: [ 确定 ]
